293 stories
·
3 followers

Starting a Journey with Clojure and ClojureScript

1 Share
JAN 3, 2017 • Written by Kamil Ogórek

Stack of Books

Clojure, as described on their official page:

Clojure is a dynamic, general-purpose programming language, combining the approachability and interactive development of a scripting language with an efficient and robust infrastructure for multithreaded programming. Clojure is a compiled language, yet remains completely dynamic – every feature supported by Clojure is supported at runtime. Clojure provides easy access to the Java frameworks, with optional type hints and type inference, to ensure that calls to Java can avoid reflection.

Clojure is a dialect of Lisp, and shares with Lisp the code-as-data philosophy and a powerful macro system. Clojure is predominantly a functional programming language, and features a rich set of immutable, persistent data structures. When mutable state is needed, Clojure offers a software transactional memory system and reactive Agent system that ensure clean, correct, multithreaded designs.

Why Functional Programming

If you’ve never tried functional programming development, I assure you that this is one of the best time investments you can make. You will not only learn a new programming language, but also a completely new way of thinking. A completely different paradigm.

Functional programming is on the rise. Clojure, Scala, F#, Erlang, Elixir, Elm, Haskell, only to name a few. Those are all functional languages, used by the biggest companies in the world to drive their major systems.

It’s good to understand how they work, but it’s even better to be able to write something using them. Always remember one thing: “A programming language is just another tool.” You should always pick a language based on your needs, not on how hyped it is or how much you love it. FP is the biggest paradigm next to OO and learning its basics will be worth it long term.

Why Clojure

“Made simple”

Rich Hickey made it very clear that Clojure is, and will always be, a simple language. Minimal syntax, very condensed and short API, no types. That simplicity, when comparing to other functional languages, makes it relatively easy to learn Clojure.

There’s no better way to understand the reasoning behind the choices than to listen to the creator himself. In just an hour, you’ll get it:

Clojure Made Simple - Rich Hickey

If you still need convincing, here is more of his reasoning:

Simplicity Matters — Rich Hickey

Hammock Driven Development — Rich Hickey

Even Robert C. Martin aka Uncle Bob likes it! In my eyes, that’s an extra five points in favor of “why I should choose Clojure.”

Java

I know, people tend to hate on Java because it’s a huge monolith, yada, yada, yada, and it’s Java. And for most of them, it should be put on the cons list rather than on the pros. But take a look at it from this perspective:

  • plenty of people already know Java
  • you can run it on almost anything
  • there are well-established tooling and libraries
  • it’s battle tested

Other than for personal beliefs, I don’t see why you wouldn’t give it a try. Just like when you were introduced to new foods as a child, there’s a huge chance that “you don’t like it” without even having tried it. Yeees, I know everyone has done that at least once in their lifetime!

ClojureScript

ClojureScript is a compiler for Clojure that targets JavaScript.

To paraphrase Sauron: “One language to rule them all, One language to find them, One language to bring them all and in the darkness bind them.”

What if you could kill two birds with one stone? Would that convince you even more?

“Developers hate him! See how he targeted all the biggest platforms using one simple trick!”

Java is huuuge, we all know that. But you know what’s the second biggest platform? Web. Browsers. Internet of Things. You can target all of those platforms with one language.

ClojureScript is exactly the same thing as Clojure, but it’s compiled and emits JavaScript code. It takes care of all the quirks and can be transpiled almost exactly 1-to-1. Almost. There’s always something, and this time we have to keep in mind that there’s Java interop for Clojure and browsers API interop for ClojureScript, but it’s a minor part of the language. All the core things work in the very same way. You can find plenty of case studies where people have taken their huge Clojure applications and translated them to JavaScript within a day or two.

The best introduction to this concept is this talk: Clojure All the Way Down: Finally a Useful LISP — Ricardo J. Méndez

How

“Alright, but how should I approach my learning process?”

There are two ways: books and videos. Some people prefer the former and some the latter. I tried both and, to be honest, the latter one is my preferred method (although you might think it’s the other way around by the photo I chose as an intro to this post). I enjoy the feeling of being “mentored” very much. I just like it more when someone is talking to me and demonstrating how things are done, rather than just reading about it.

Books

There are tons of books about Clojure. Clojure in Action, Clojure Programming, The Joy of Clojure, but I’d like to focus on two positions:

  • Clojure for the Brave and True — Daniel Higginbotham
    It’s not often the case, but this book is FREE and you can jump to reading it straight away. Of course, I highly encourage you to buy a printed copy! It’s one of the best books you can find on the topic. It explores it in a very easy to digest manner and approaches it from the point of view of people completely new to the functional programming world.

  • Living Clojure — Carin Meier
    Carin’s writing is the shortest of all the mentioned books, and it’s also the most condensed one. You’ll find everything you need to start your journey with Clojure. It’s also written somewhat as a tale about “Alice’s Adventures in Wonderland.” How cool is that?! It also contains what I found to be a great concept that could also help many of you: a “Weekly Living Clojure Training Plan!” It’s split into seven weekly assignments, and step-by-step guide to learning Clojure.

Courses

PurelyFunctionalTV — Eric Normand

This is the best resource I’ve ever found when it comes to learning a new language from scratch. It holds your hand and guides you through all the basics, advanced topics and writing a whole complex application, explaining everything along the way.

The Clojure Language — Brian Will and Clojure Fundamentals — Alan Dipert

Those two series are front-to-back introductions to Clojure. They both go through all the core concepts of the language and contain everything you need to know to start working on your projects.

Lambda Island

This one isn’t a course “per se,” but rather a series of tutorials on Clojure and ClojureScript. Very useful when we are at a given point in our learning process and would like to dive deeper into a specific topic.

Additional resources

Although I’ll openly admit that I haven’t read them yet, these are books that were often recommended. I am told they provide more understanding about Functional Programming. I’ll read them one day!

Read the whole story
futurile
79 days ago
reply
London
Share this story
Delete

Autopsy of a slow train wreck

1 Comment and 2 Shares

At DjangoCon Europe 2017, and again at DjangoCon US 2017, I gave a talk entitled "Autopsy of a slow train wreck: The life and death of a Django startup". After I gave those presentations, a number of people requested that I publish the content in blog form so they could share it with colleagues.

Transcript

I've been a frequent (almost constant) fixture at DjangoCon events over the last 10 years. And if you met me at one of those DjangoCons in the last 6 years, or seen me speak, I may have introduced myself as the CTO and co-founder of TradesCloud. TradesCloud was a software as a service company for tradespeople - plumbers, electricians, carpenters and the like.

TradesCloud was my startup. I say "was"... beacuse in January of this year, my business partner and I closed the doors on TradesCloud, and shut down the service.

Gold-plated Lamborghinis

As an industry, we're fond of promoting the glossy side of startups. That a plucky bunch of engineers can take an idea and a personal credit card, build a business empire, and drive off into the sunset in a gold-plated Lamborghini.

And yes - those unicorns - and gold plated lamborghinis - do exist. There are even a couple of them in the Django community (the unicorns, not the lamborghinis).

But it's important to remember that those stories are unicorns. They're not the normal startup experience for most people.

The reality...

In the VC-backed startup world, the general expctation is that if a VC firm invests in 20 companies, only 1 of them will actually succeed spectacularly. 4 will have some sort of exit that at least results in a breakeven financially; but 15 will fail outright, with a significant or complete financial loss.

Interestingly, this isn't something unique to tech. Tech does it at a much grander scale, especially when VCs are involved - but open any small business advice book - the sort that is targetted at the plumbers and electricians of the world - and they'll warn you that 50% of businesses fail in their first year.

And yet, despite the fact that failure happens all the time, we don't talk about it. We don't talk about why things fail. And as a result, many of the same lessons have to be learned over and over again. Those that experience failure often feel like they're doing it alone, because of the significant social stigma associated with failure.

So - this is my small attempt to restore the balance. In this talk, I'm going to talk about TradesCloud - my failed business. TradesCloud was a slow train wreck - we survived for 6 years, almost to the day. And we had plenty of optimism that success was just around the corner... but that never quite happened.

What was Tradescloud?

But what was TradesCloud? What prompted me to dedicate 6 years of my life to it?

Well, it started as a problem that thought I could solve. If you find yourself needing a plumber, how do you pick one? Well, 15 years ago, when I first had the idea for what became TradesCloud, the best option was opening the phone book and looking for the brightest, shiniest ad, maybe arranging a bunch of quotes, and pick one basically at random. If you were really lucky, you might be able to use Google - but that's still looking for the shiniest ad. If they turned out to be good... well we won't need a plumber for a while, so that knowledge is useless. And if they turn out to be awful... we can't warn anyone off, either.

"There has to be a better way". And, of course, I did nothing about it. I say nothing - I did start tinkering around with a web framework... you may have heard of it... Django. I originally got involved in Django because I wanted to add aggregation functions to the ORM so I could compute average ratings. And in 2008, I mentored a student - Nicholas Lara - to add aggregation as a Summer of Code project. So... success?

An idea is born...

In late 2010, I met up with a former boss for a drink, and he tells me about his brother. His brother owns a pest control company, he has the same problem - but from a different angle - I'm looking for tradespeople in my area that can be reccomended - he is smaller company that wants to compete with the big players with the shinier ads based on quality of service.

And so, TradesCloud was born. We had an idea. At the time, it wasn't called TradesCloud - it was called CleverPages - because it was going to be a clever Yellow Pages. In my spare time, I started hacking together a proof of concept.

Mistake 1: Validate, then build

That was our first mistake, and the first mistake most tech-oriented people make. As much as Django sells itself as a rapid development framework, any non-trivial project still takes time and effort. And I spent a couple of months of spare time hacking together a proof of concept.

German military strategist Helmuth von Moltke once noted "No battle plan survives contact with the enemy". Or, in non-military terms, Scottish poet Robert Burns said "The best-laid schemes o’ mice an’ men gang aft a-gley". And so it is with business ideas. All the time I spent working on that prototype could have been eliminated if I'd actually spoken to a plumber first.

Just because we had an idea, and I could implement the idea in software, that didn't mean we had a good business idea. It meant we had a good idea for a hobby project. And the difference is critical. A business is an idea that generates revenue. A hobby project may be fun to work on. It may even be useful for other people. But if you can't sell something, if you can't pay the bills with it - it isn't a business. And conflating the two ideas is a major problem.

What we should have done is validate the idea first, and then build it.

But, we didn't do that - and when I finally had something to show off, my business partner opened the local newspaper, picked a bunch of local plumbers, and called them in an attempt to sell the idea.

He called 10 plumbers. 5 of them suggested he place the idea in an anatomically implausible location. 4 of them had their secretary provide the same advice. One plumber did sound interested, and said he wanted to have a chat.

Mistake 2: If you can't sell it, it's not a business

This was mistake number 2. Or, at the very least, it should have been a warning flag.

At the end of the day, business is about selling something. Selling a physical product. A subscription. Selling services. But whatever you're doing, you're selling something. And in order to sell something, you have to have customers. If all your prospective customers hang up when you call... you have a problem. You don't have a sales channel. It doesn't matter if you've got a machine that turns lead into gold - if you can't get that idea in front of the people who are going to buy your product, you might as well shut up shop right now. The fact that it was very difficult to get plumbers to answer the phone should have been a warning sign that our prospective audience wasn't going to be easy to crack.

But, we persisted, and had a chat with the one plumber who would talk to us. We did our pitch, and he said "Nope. Not interested. But if you can make that pile of paper disappear, I'll give you as much money as you want."

This was a conversation that set the direction of our company for years to come. Was this a mistake or a success? Well, that's a little hard to judge. There's an extent to which we changed direction because it was the only direction that seemed open to us - which was a bad move. But it was a very lucrative direction, so... maybe it's a wash.

What we identified in that conversation was a significant business problem - a business process that was being performed manually, and took three hours a day, and identified a simple and reliable way that it could be automated. We identified a couple of other processes we could automate, and ways to report on some key performance indicators. We identified a path forward that could use mobile tech to improve communication and process management. By the time we were done, we'd worked out how to immediately free up a full time employee, with potential for more. So as long as we charged less than the cost of that employee - about $50k a year - the business owner would be ahead.

Our costs were next to nothing. Our newfound customer told us that these business processes were due to one specific contract that they had - and there were many others on the same contract. So - should have been easy to sell the same software to everyone else on the contract, and... profit! Right? So - lets keep it simple, offer them a 50% saving - and after doing some fancy footwork to reverse engineer a good explanation for why that was what we were charging, and just start making $25k a year per customer, right?

Mistake 3: Humans gonna human

Well, no. That was mistake number 3.

Mistake 3 is that we didn't take into account is the human factor. In theory, charging anything less than $50k per year - rationally - should have been a no-brainer, easy sale. But we were selling to humans. And humans don't ever behave rationally. There's an almost bottomless body of research about how bad humans are at evaluating economic decisions and consequences.

And so, when we walked in the door of a prospective customer, we did our pitch, they were almost universally blown away. And then we told them the price, and they starting describing anatomically implausible locations again. Why?

Humans aren't rational

Firstly, a sale of that size isn't easy. Asking a plumber to spend $100 a month - they know they can afford that. It's probably less than what they spend on coffee in a month. But asking them to spend $2000 a month? That's a lot harder for them to justify. That actually starts to make a dent in their bottom line. So they're going to take some convincing. They're going to want proof that it actually works, that it's actually going to deliver the benefit you promise.

Software is hard to sell to humans

Secondly - we were selling software. While we were completely honorable, and completely truthful, and we were able to deliver everything we promised, and our software made makes birds suddenly appear every time we were near - we weren't the first IT salesperson they've had to deal with. And we - we collectively - are part of an industry that has, for 40 years, systematically over promised and under delivered what software can do for a business. So that is something that needs to be overcome.

No really - Software is hard to sell to humans

Thirdly - we were selling software. Who here is currently holding a phone worth a couple of hundred, maybe even a thousand dollars? Now, how many of give more consideration to whether you should buy a 99c app from the app store than you do the decision to buy the thousand dollar phone?

That's the problem selling software. And multiply it a thousand times when you start dealing with non-tech audiences. We've been conditioned to expect that physical, tangible things are expensive - but software? That should be cheap, or better still free.

As a side note - this is one of the major problems we face with funding open source projects as well - but that's a subject for a different rant, and a rant that I've had before.

Humans are people

Lastly - we were dealing with personal relationships. If we walked into a small plumbing business to speak with the manager, there was an odds-on chance that the bookkeeper, or another significant employee in the business, was the wife of the manager. And you start talking about being able to cut an employee... well, you can guess how well that conversation goes. And even if it wasn't a family member, people don't generally want to fire people.

Mistake 4: Beware favourable patterns

Mistake number 4 happened as the result of an unfortunate coincidence. After closing our first sale, we got that customer to give us an introduction to some other possible customers. And he gave us the best possibilities first. So our first two sales were both $2k a month. Our third was a smaller business - only $500 a month - but that gave us the confidence that we had something that we could sell to medium and small businesses.

We'd closed three sales in rapid succession. We had $4500 a month in revenue, and the sales were really easy to close. We thought we had found a money printing machine.

And then we hit a wall. The next few sales calls we made just went nowhere. Never a hard no... but lots of ums and ahs about price, and "we'll have to think about it"'s...

We confused initial success with a pattern that was going to continue. After three sales in a month, we essentially didn't close a sale for another 8 months. And that's not a good sign.

Arguably, we got bitten by circumstances there - when you have lots of early success, it's easy to think that success will be ongoing. This is a time where you need to be objective. If you can't consistently close sales, if you can't reliably predict your close rate - you have a problem.

Mistake 5: Do the math

Mistake number 5, though, was completely our fault. We completely failed to do basic math.

Our value proposition - the business process that we had optimized - existed because of the processes required by one particular contract. Our pricing scheme was simple - we charged $1 per job completed. Our initial customer - they did about 2000 jobs a month, so we charged them $2000 a month. Which is great, because it also happened to hit our 50% savings target that we originally identified.

What we didn't do was add up how many jobs there actually were in the system. It turns out that if we managed to close every company on that contract, we would have only generated $12k a month in revenue. Which sounds like a lot, especially when your costs are so low... but our costs weren't low. We also had two founders who were full time, and needed to be paid. Our burn rate was closer to $22k per month.

Mistake 6: Pricing is what the customer will pay, not your burn rate

And this led to mistake number 6: We didn't have a serious pricing discussion until it was way too late. Mislead by our initial success, our pricing was really determined by taking our burn rate, and working backwards - not forwards from what the market would bear. My co-founder and I would have regular discussions about pricing - but all of those discussions happened against a background of "how are we going to make payroll this month". Which is the wrong time to be having that discussion - because two important options: drastically reduce the price, and shut down the company - are effectively off the table.

So, we had a product that was too expensive to sell, and a market that wasn't big enough. Now the good news is that the paperwork reduction niche we'd found wasn't unique to that one contract - There were many other simliar contracts with similar paperwork requriements. But we'd found the thousand pound gorilla in the market. Other contracts were smaller, and the paperwork and process requirements were subtlely different.

Mistake 7: Establish your sales channel

And that wouldn't have been a problem - if we hadn't made mistake number 7 - we never established our sales channel.

We got our first sale almost by accident. We bumped into a customer who gave us an opportunity. Subsequent sales came by word of mouth. Word of mouth is an incredible sales channel if you can get it. But as a result, we never cracked the most important problem - how do we sell to someone who hasn't heard of us? How do we get in the door? How do we establish trust? And as a result, our sales were essentially constrained by the personal networks of our existing customers. Perth is a small, geographically isolated city. When we'd exhausted personal networks... we hadn't learned the most important thing - how to sell our product to someone who didn't have a personal introduction.

Joel Spolsky once noted that there's no software product priced between $1000 and $100000. This is because a product that costs less than $1000 can be bought on a credit card. But if software costs more than can be hidden on an expense statement, you need to have salespeople, and that means you have to pay them, and their commissions, and pay for the steak dinners and drinks used to closed the sale. We had a product that was squarely in this dead zone. Too expensive to be a casual purchase, but not expensive enough to support the sales process it needed.

TradesCloud had a serious problem. Once we closed a sale, we had almost zero churn rate. The only customers we ever lost were because they closed down, or they dropped the contract where we offered an advantage.

What we didn't have - and what we never really established - was a good way to prove to new customers that we were, indeed, that good. There wasn't a good way to "trial" TradesCloud. We were managing processes that were at the core of a trades business. Those processes have to work. And they can't be duplicated or doubled up. So - there was no way to "stick your toe in the water" - you had to jump in, or stay out. And since we had a huge price tag, most people were conservative, and said no. If they got a recommendation from someone they knew, it was a little easier - but if that didn't exist, we had a problem.

In order to close a sale, people have to believe - really believe - what you're telling them. It has to be obvious, and undeniable that you will give them benefit - or the cost of trying has to be vastly less than the cost of the software itself. In our case, even if we dropped our price to zero, we didn't have a zero cost, because the cost of institutional process change involved in adopting a new piece of software at the core of business operations is huge.

Our best sales person was completely accidental. He wasn't our employee - he was an employee who changed employers every 6 months. He was in upper management, had a reputation for getting things done and turning companies around, so he kept getting poached. And he'd seen the benefits of TradesCloud with one contract, and so it was easy to get in the door every where else he went. And because he was known around the industry, his word was extremely valuable. When he said "This is good", people believed him. His word was trusted.

Mistake 8: Sales don't stop when you sell

But even when we did make a sale, the mistakes didn't stop. Mistake number 8 - we didn't pay enough attention to onboarding new customers. A sale for a product isn't closed when a contract is signed. It's closed when the person who uses the software has accepted it into their daily lives, because that is what prevents churn. If you've selling a small personal tool, the person who buys and the person who uses is probably the same - but in our case, the purchase decision was rarely made by the person who actually had to use the software. And you have to get those people on board. If anything, they're more important, because they're the ones who are going to make the bosses life hell if the software they buy isn't doing the job - or worse - is doing the job too well.

Over and over again, we saw internal sabotage. People would simply refuse to change processes, and would find any excuse. "Oh, the software didn't work, so I had to go back to doing it manually". And after a month or two, the boss would call us and say "what happened to all the benefits you promised?", and we'd say "well, you only get the benefits if you actually use the software".

What we learned - the hard way - is that you sell to the business owner - but you also have to sell to the users. If you're dealing with software that is part of a key business process, change management is key. You have to show them how your tool does what they currently do by hand. And you have to show them that their jobs aren't at risk. That first employee whose 3 hour/day task was replaced? She wasn't fired - she redeployed inside the business. She went from doing a mindless office task for most of the day, and could start expanding into other parts of the business. About 2 years after we first deployed TradesCloud, she was running accounts and payroll.

But, despite all these mistakes, we were able to stumble along, and were complete self funded for almost 2 years. Now, that meant burning a lot of personal funds, and my co-founder doing a bunch of consulting on the side. But that's just part of the startup experience, right?

Mistake 9: Establish failure criteria

Well, maybe it is. But in retrospect, mistake number 9 was an entirely personal one - I shouldn't have lost as much money on the experience as I did. I knew what I considered success criteria, but I never considered what my failure criteria would be.

After 2 years, I had reached a point where my personal financial runway was running out. TradesCloud either needed to start paying a full wage, or I wasn't going to be able to continue. And this pivoted the business. Fundraising is a full time job. Everything else goes on hold - sales, support, development - everything. We tried to get VC investment, but the VC scene in Australia is pretty bad, and even worse in Perth. Eventually, we managed to secure a $250k cash investment from a colleague of my business partner; and we got some matching funding through an Australian government program. And that gave us another 2 years of runway.

The way we were able to secure that runway was by changing our tactics. Instead of going after individual plumbers, we started going after the head contractors - the multimillion dollar facilities management companies. And we were able to sell them a really great story. These companies are all competitive, and they're looking for any advantage they can get. But they're also technologically laggards, because they're big established companies. They have inertia when it comes to adopting new technology. So we were able to walk in, and promise a mobile-enabled workforce, real-time tracking, enforced health and safety practices - all sorts of things that made them really excited, because they could use those features as differentiators against their competition.

Mistake 10: You are who you are - Don't deny it

But - and here's mistake number 10 - we forgot who we were, and who we were selling to. We were selling to multimillion dollar companies. The reason these companies are technologically laggards? They're conservative. They don't take risks. There's no incentive for individual employees to take risks. And so, they make safe decisions.

And when they adopt new technology, they don't just pick something - they put it out to tender, and get multiple bids, and then they invite bidders into the head office to interrogate them, and eventually, after 6 months, they pick someone - the safe option. We got into a tendering process with almost every major facilities managmeent company in Australia. The tendering process almost always started because we pitched them the idea of providing TradesCloud to all their subcontractors - but what they heard was "provide software to all their subcontractors". And so, at the end of the tendering process, we were told, every time - we prefer your technological solution... but we're going with your competitor, because you're too risky. A 2 person company was too much of a risk for a multimillion dollar company to trust.

So after 2 years of trying this tactic, and being turned down by every facility management company in Australia, the money was running out again, but i'd built up a bit of cash buffer again. But in between failing to sell to multinational companies, we'd found a bit of success selling to smaller facility management companies and large constructions companies. And the good news was that these companies were big enough that when they bought software, they wanted it customized - so as well as the $2k/month, they would pay $40k up front so that everything matches their requirements.

On the back of that change, were got a loan from our investor. Between making more personal sacrifices, and that cash injection, we were able to stumble along for another 18 months.

Mistake 11: Take the hint

This was mistake number 11. We didn't take the hint. Each of those points where we took investment was, potentially, a point that would have been a natural point to shut down the business. And, in retrospect, we should have. The writing was on the wall. The simple truth is if you can't close sales, you don't have a business. And yes - you can stumble along hoping that you're going to find the missing sales ingredient - but that takes resources. It takes money, and it takes emotional capital as well.

One of the reasons the failure of TradesCloud was so personally galling, is that it didn't fail for any reason that I would consider "my fault". From a purely technical perspective, we were significantly more reliable than the multi-national companies we were integrating with. We delivered new features in timeframes that our customers considered inconceivable. When I went in an did demonstrations to the multimillion dollar head contractors, they expressed doubt that we could actually do what we said we were doing... right up until I showed them the code doing it, live.

But none of that mattered. TradesCloud failed, ultimately, because we couldn't sell what we had - or, at least, we couldn't sell it in quantities that allowed us to cover costs. And when you're there giving your all, doing things that are being called magic by prospective customers... and you're still failing... that's hard to internalize. And when you layer on top of that the fact that I'm a husband and a father, and the sole income for the family - that introduces all sorts of guilt and fear into the mix. And then you take all that stress, and add in the long hours and weekend work in the desperate hope that this will be the thing that saves the company...

... and you start to understand why, 2 years ago, I had a major depressive episode.

Mistake 12: Quitting is always an option

Mistake number 12 - I lost sight of the fact that quitting was always an option - and that quitting didn't mean failure. If, after 2 years, I had taken an honest audit and said "you know what - this isn't working. I'm out". I would have had 4 years of my life back. That's four years I could have sunk into a different project. But I didn't pay attention to any of my signs. I conflated success of the company with my own personal success. I lost sight of the fact that this was a job. And it was meant to provide some income and some intellectual engagement. And if it wasn't doing that - walking away was always an option. But I didn't ever really consider it seriously.

Mistake 13: Partnerships require actual partners

Why not? Well that was mistake number 13, and it was another personal failure - I didn't stand up to my co-founder as much as I should have. And as a result, we wasted a lot of time, and effort, and in some cases, money. Now - I have to be clear - if only because there's a chance he might see this video - i'm not blaming Mark here. Mark is a great guy, and he's extremely talented, and he's got absolutely no shame at fronting up to companies thousands of times bigger than his, and telling them how they should be doing things. He opened a lot of doors that I know I wouldn't have ever even considered knocking on, let alone opening. He was a real asset to the business. The failure was a personal one, and it was mine.

We didn't go into TradesCloud as complete equals. Sure - we were 50/50 partners on paper - but when I met Mark, he was my first boss out of university. I worked for him for 4 years as a very junior subordinate. And a lot of that power dynamic remained. I let him do a lot of things because "well, he must know what he's doing". I caved on decisions because I could see his side, and he was more experienced. And, Mark is a great sales guy. He can make you believe in things. And he made me believe in TradesCloud - but that's a double edged sword. It got me through all sorts of lows - but it also meant I believed even when I probably shouldn't have. I should have putting my foot down and said "no more" a lot more often than I did - both for the benefit of the business, and for my own mental health.

The end

And so, when the money ran out for the third time, neither Mark nor myself had the energy to continue. We had a couple of last minute hail-Mary options that we thought might have saved us... but one by one, they all fell through. In the end, we were able to pay back the loan to our investor; but his equity investment was essentially lost. And in January, we closed the doors for the last time.

And that's the TradesCloud story. I will warn you, though, that the plural of anecdote is not data. This is my story. Many stories are like it, but this one is mine. I don't profess to having any particular business insight - I just know that TradesCloud didn't work. And these are the 13 reasons I can identify why.

In the aftermath, I've had a lot of people - many of them in this room - reach out and give me a virtual hug, or a spoon. And many asked me if I was sad to see TradesCloud go. But frankly, the emotion I had was relief. On January 31 2017, I slept like I hadn't slept for 6 years - because I knew I wasn't going to be woken up by a server alarm. And I knew I could sleep in, because I wasn't going to get a support call at 6AM.

The fact that I wasn't even slightly disappointed by the loss of TradesCloud from my life - that's the biggest sign for me that I waited far too long to step away.

The good news, though, it that the process of running TradesCloud hasn't burned me completely. It was an amazing learning experience. And I landed on my feet - At DjangoCon US last year, I put my name up on the jobs board saying I was looking, and it hadn't been there half a day before my good friend Andrew Pinkham approached me and said.. Uh... are you that Russell Keith-Magee looking for work?

The other silver lining is that the TradesCloud experience drew my attention to problems in the world of mobile development, which has influenced the path that my new toy, BeeWare, has taken. And I've been busy trying to work out how to turn BeeWare into something.

But this time, I'm a little older. A little grayer. Hopefully a little wiser. And I should have a better idea what to look out for. If you want to talk about that... well... get in touch.

Read the whole story
futurile
84 days ago
reply
Very well written. And worth noting how few of the key moments are to do with tech.
London
Share this story
Delete

Beautiful 30-day time lapse of a cargo ship’s voyage

4 Shares

Jeffrey Tsang is a sailor on a cargo ship. On a recent voyage from the Red Sea to Sri Lanka to Singapore to Hong Kong, he set up a camera facing the bow of the ship to record the month-long journey. From ~80,000 photos taken, he constructed a 10-minute time lapse that somehow manages to be both meditative and informative. You get to see cargo operations at a few different ports, sunrises, thunderstorms, and the clearest night skies you’ve ever seen. Highly recommended viewing. (via colossal)

Tags: time lapse   video
Read the whole story
futurile
84 days ago
reply
London
Share this story
Delete

Monologue: When It Comes to Tech Meetups, I’m a Three-a-Day Guy

1 Comment and 2 Shares

Hi there, friend!

Is this your first time at The Founder Meetup? First meetup ever? I’m a three-a-day man myself! Hearing from tech founders expands your mind, the whole human experience really. I haven’t founded my company yet, but I’m about to — just after this meetup probably! What do you found for a living? HA! That’s a tech-founder joke! I call jokes “happiness-as-a-service.”

Oh, you founded a charity? I don’t think that counts. To be a founder, you have to disrupt something with tech. That’s what TechCrunch says. Last year, I tried to get into their office to see what startups they were keeping secret. Security said I didn’t have a pass to be there and everyone thought I was being disruptive. Can you believe that?! Such an honor!

Want to join my Slack messenger team? You don’t have Slack?! Here, take a look at mine. No, it’s nothing like MSN Messenger. MSN was instant messaging with smileys that you used to cope with the jerks at school; Slack is a communication platform that’s disrupting productivity because it’s got emoji. It’s totally different. When you’re on Slack, you’re in-tune with the whole world. I’m in 63 teams! Whenever I need a fix of startup life, it’s right there in my pocket. I don’t understand a lot of the Chinese teams but when it’s 3 AM and you need more startup news, who cares, right?

Tell me about that charity you started while I have my Soylent. No, it’s not out of a movie; it’s a meal replacement drink. Sean, this totally inspiring co-founder that I follow on Medium said that all the real founders drink Soylent and if you don’t drink it, you can’t call yourself one. I started out just having it for breakfast; now I have 28 small doses a day! I also take 28 small shits a day. That’s when I check the App Store for updates. I bet you don’t get to check it that often!

Doesn’t sound healthy to you? Well, soon we’ll cure everything and live forever — that’s what Peter Thiel says. He founded PayPal to skim money off eBay buyers during BeanieBaby mania, and now he’s a billionaire. I have Google Alerts that email me each time he’s mentioned online. Once I was racing home to watch his new presentation, and I spun my car off the road, through a hedge and landed on my roof. Luckily, there was LTE in the field, so the video buffered pretty quickly. What are the chances? Actually, pretty high when you consider the current pace of technology.

You have to read his book, Zero-to-One. When I finish it, I just start reading it again. I’m currently at 358 reads. My personal best is three hours. I bought it on Kindle, Audible, and have 35 backup copies that I hide around the places I visit most often. I’ve got four with me now! You can’t have one though. I might need them. God knows what I’d do without my Thiel.

I’m itching to download something right now but this meetup’s fucking barren. Speak of the devil, that’s my Google Alert! Hope my hoverboard has enough battery. Slack me if you find any good beta invites!

Read the whole story
futurile
107 days ago
reply
"You have to read his book, Zero-to-One. When I finish it, I just start reading it again. I’m currently at 358 reads. My personal best is three hours." ... when art imitates life ...
London
Share this story
Delete

Microservices: Engineering Fashion Trend Or Fatal Blunder?

1 Comment and 2 Shares

The way we design and deploy web applications is changing, in ways that don’t seem to make sense.

A while back, I gave a talk at a local DevOps event and then a later version of that talk at “All Day DevOps”. It was essentially about questioning our foundations. While I called out agile and microservices as examples, I didn’t dig in completely.

Today I want to focus a bit on microservices, but also use this as an example to say “step back and consider who we are listening to and why”.

Microservices almost feels like one example of an existential battle between engineering principles and trendiness. I’ve only been in software a bit of time (a couple of decades if you count internships) so I imagine for others older than I there are tons of examples of apparent backsteps, of technology and approaches cast off for no reason.

We’ve probably fought battles like this many times. We could perhaps allude to Neo repeating his mission in the confusing 2nd or 3rd Matrix sequels, if those movies in fact had ever existed. But I disgress.

Let’s start with an example as we start dissecting ideas and where they came from.

Modern Containers Come From PaaS

Docker Containers as a concept were not new things, LXC existed before. Lots of technology existed before that. Jails. Zones. Older stuff. No need for a history lesson here. Google if you want it.

But with the current-wave of containers specifically, as a company, Docker evolved from “dotcloud”, a platform as a service company. As we all know, most PaaS platforms today have evolved away from PaaS and now market themselves as platforms to run containers. A few other platforms have also pivoted in similar ways. My point here is just that containers make sense in the internal Enterprise PaaS use case. They were made for that case. Understanding this case understands what they are good for — utilization in cases where groups of like-minded employees work together, and security needs are not maximal.

Most significantly, the enterprise PaaS use case is one where with a very high degree of certainty the code running in question will never evolve to run beyond the need of one VM. An example would be a timesheet application. It’s not going to need to scale much.

Web Application Scaling Invalidates The Utilization Argument

Web apps/services are usually pretty simple, or should be.

In the classic web application model, an application runs on many VMs for both fault tolerance and horizontal scaling, situated behind a load balancer, eventually speaking to a data store.

Even if composed of many services, as load increases, eventually each application will grow to need more than one VM. In a fault tolerant use case, EVERY application will need at least two VMs. This will happen if your application is at least even minimally successful.

Say the auto-scaling point of an application is when an application hits (arbitrarily) 80% average CPU. In this case, the application has already consumed the resources of the VM and there is no room for more work within the VM. So is utilization still a valid concern where we need something smaller running inside that VM? Not really.

In this web application use case, the “utilization” argument of containers falls flat.

Obfuscated Packaging

Developers like docker because it means they don’t have to write much automation to get their app out there. They describe apps in docker files.

The thing most resembling a Docker file today is a Packer file, because a Docker file is basically a direct copy of the concept. Packer is great, easy to use application that took a shell script (more or less) and produced images suitable for use in clouds or VMWare or what have you.

We must give Docker Containers some credit for making this approach to immutable infrastructure more well known, but it wasn’t invented new for containers, it was already there.

In truth, you don’t even really need Packer, because Packer can be replicated (for basic use cases) in about 250 lines of Python. Before I wrote that simple hack, there were also many other very simple ways to construct images.

So, with the utilization argument out the window, and the “build file” argument reduced to “this isn’t new — we already had that”, we are left with the idea that what is novel about containers is that it is a way to “ship” applications between a build system and a cloud.

The problem these blobs introduce are obfuscated dependencies and potentially non-reproducible environments.

One of the microservices darlings at the moment is the language Go, which typically pulls software dependencies from the tip of various GitHub branches. A build of a given project today (to fix a bug, etc) may not even be able to build tomorrow. It may require substantial re-engineering day to day to keep the software working. This is because the Go community didn’t care about dependencies at all until something like last month — I suspect they’ll come around.

Even without Go, container content should be built from source every single time, so that security configurations can be applied. You should never run opaque blobs that you do not know how to recreate, and are not guaranteed to be able to re-create. yet it is pitched and sold as a way to distribute binary blobs — the one thing that ops should never trust. Images should come out of a private build system every single time.

Security And Reliability

In addition the opaqueness, we also lose security (access between containers) and reliability (containers don’t stay up). I regularly here from folks who have to restart Docker containers with scripts and such, or reports of container processes randomly killing other processes. I don’t really need to dig into this, but compared with VMs, things are not as bulletproof yet. As I came from a place where turning on SELinux was a good idea and because I’m a strong believer in reliability and correctness, I think these speak for themselves — this tech reduces security and reliability, which in turn increases operations and security team workload.

Efficiency In Calls Between Services

In adopting an architecture of a lot of network communication that is highly “gossipy”, the efficiency of that application is going to be a lot lower than one that can make local function calls.

This will result in consuming MORE resources for a given operation, which in turn could mean increased costs (even environmental impacts through electricity consumption!) as well as a slower experience.

The reason for decoupling the code? Questionable. It’s fine to have a few tiers where large components of the stack may not be as widely used (or there is a programming language boundary), but a ton of different services in a tangled web is not a good engineering design.

It is a lack of design.

Managing the Management

Why is cloud attractive?

It’s attractive because application developers can focus on just writing the pieces of software that are unique to the particular problem they are trying to solve. Similarly, operations professionals can focus on just running the infrastructure that is unique to the problems as well.

In a scenario where we have a strong public cloud (say AWS), the need to run additional scheduling infrastructure on top of another scheduler seems kind of cross-purposes.

When that software itself requires additional software (like zookeeper, which in turn requires etcd), we introduce additional parts that the cloud itself does not run.

We complicate the voyage for ops by making MORE things to maintain, by the virtue of introducing something that allegedly was supposed to reduce the number of things we maintain.

Where before we only had to worry about the application falling over, now we have 5 or 6 underpinning applications that we also have to worry about scaling, upgrading, and how THOSE are going to fall over. Not only are the containers themselves less reliable, our whole “cloud” itself is less reliable, and the ops team needs to know how to upgrade it, and follow the development decisions of many infrastructure projects behind it.

Debugging Gets Complicated

There’s a quote I read somewhere on twitter that said (roughly) “The great thing about Microservices is that every outage can be treated like a murder mystery”.

This is because when you have 145 different applications talking to one another, you can’t really be sure where a logic error is that fed invalid data from one to another. If one crashed, who caused the crash? How were all the calls made in sequence between them.

In the get-off-my-lawn olden days, we ran a minimal tiered architecture, with maybe a background jobs layer, and a web application layer. When something failed, exceptions would be able to log a full traceback throughout the stack.

We were not reliant on something like Splunk or SumoLogic to attempt to figure out what happened in the great sea of projects.

Worst still, we reduce compile time checking across services and push more into runtime space.

Communication Solved By Not Communicating

The often misunderstood aphorism of “Conway’s Law” says that the software organizations produce mirrors the structure of those organizations. (Unlike some who get it wrong, it does not say that it SHOULD mirror the structure of those organizations).

While design-by-contract is a noble aim, the idea that the only part of a piece of software that matters is it’s contract is questionable.

Microservices are often adopted by teams that grew too large (for whatever reason) to handle code review, or decided to not collaborate on software architecture. They facilitate less communication.

Given, I understand software often feels much like an art, and being able to segment your work and be a free spirit is liberating, but it brings software closer to “art” and further than “engineering”.

In social practice, in microservices environments, you also have less people to talk to, less people to collaborate with, each team building up walls around itself that are isolating and potentially damaging to culture.

I’ve summed this up before by saying microservices are a technological solution to a people problem. Except tech problems don’t solve people problems most of the time — you have to fix the people problem. So do that, and go back to having working stacktraces, code sharing, and compile time checking.

While I don’t want to reduce employment, you can probably have less developer teams this way too — because your developers will be more productive.

Is Continuous Deployment Really Important?

First off, I’m absolutely in favor of zero-downtime and fully automated deployments. This has been a huge part of my life.

This should be no surprise given some of the things I’ve worked on. I’m completely for automated builds and automated tests.But should deployment beyond-testing and stage automatically occur?

Often the example used for pushing for “agile” and “continuous deployment” is a hosted web application company, where a marketing executive wishes to push out a new ad campaign. How many orgs does this fit?

Most organizations do not have these needs. If there is a utility service, should the team automatically get the service to hop into prod when the team desires?

I’d say usually no, there should be some engineering process signoff, as well as potentially some review with what goes into that service. Small behaviors, even that do not violate the contract, can seep in and add unneeded stability.

Even if you have a relatively monolithic app, you can still do A/B deployments, you just deploy the whole app instead of part of it. You need bulletproof tests to do microservices, so we’d hope you still had those bulletproof tests in a more classic deployment model.

Disposability And MVP Design

One of the recent articles I’ve read, unfortunately endorsed by Martin Fowler no less, argued that microservices endorsed architectures that could be thrown away in 5 months. That’s a huge loss of planning. As a developer who enjoys design for efficiency and quality, that feels like a bad idea. For someone who knows behavior that is codified can easily be broken, that is a bad idea. Software design (by developers) is incredibly important, and our application stacks are not just a sea of functions that desire to call each other.

Whether The Emporer’s Clothes Are Nice Enough

We’ve been told that these software systems — microservice architectures, continous deployment, hyper-agile disposable architectures, are things that go with them are things that we should be adopting.

These topics are sold by repeated conference speakers, popular authors, and internet personalities as being better than the other options. Yet, some of these speakers haven’t run an infrastructure for an incredibly long amount of time. They haven’t had to argue for cohesive software architectures recently in large software teams. They have seen some failures, but the solutions they advocate are not solutions they have actually had to use.

Diversity of experience and being exposed to a large number of competing viewpoints in tech is how opinions are formed. While I wouldn’t necessarily say changing jobs is good for your resume, the more you see, the more diverse places you work for, you start to acquire a very good instinct for where organizations break down. It was good for me, to build experience, even though the reasons I did it were only related to wanting to be happy.

So yeah, I question the religion we seem to be being sold. If we find it hard to communicate, splitting our application into chunks isn’t going to make it better. Running more software to juggle our applications on top of the software we already had to run isn’t going to make it better.

Tech is changing very fast, but I don’t know if it is evolving per se. It’s not a natural selection change. A lot of the changes we are seeing happen feel like the result of collective group-think and a desire for newness, they are driven by an incomplete picture, and I think we might be revisiting them soon.

Unfortunately many of the things we finally eliminate or push back on do lasting damage for decades before we figure it out. (I left three jobs because of Scrum, including at one company who later bought one of my companies — so Scrum was expensive!)

So yeah, if people see me skipping DevOps conference talks, that’s why. I don’t believe in it, I have evidence and experience to back it up, but my evidence isn’t trendy.

I’m not really into ops fashions. I’m a fan of software design well before I was a fan of ops. I worked in writing software for operationg because of an affinity for distributed applications and a minor-dislike for UI development.

I’m a fan of minimal moving parts. Of efficiency, of security, simplicity, and a really really big fan of inter-team communication. I love whiteboards and getting people together, and yep, objects. Microservices/container-infrastructure does seem to often run counter for these things, and I do not think has proved out the benefits of the pitch. It’s a hypothesis without a question.

But for me at least, it makes software much less appealing as a developer and someone who has an interest in making software simple to run in production.

And This Wasn’t About Containers?

That’s a really long-winded take-down, right? You’ll either like it or maybe think I’m really misguided. Well I didn’t really even have an agenda on saying things about microservices up front.

My point is software development — we have people causing changes right now, and many are changes in fashion that I don’t agree with. This is not because I’m contrary or want to see them not succeed — but because I believe in fewer moving parts, security, reliability, reproducibility and also keeping things simple on ops teams.

So while to me, it feels there is no real engineering behind many of these choices, they are just pushed through with a lot of excitement. When people decide they don’t like one framework or system or way, they advocate for a new one. Where are we leading it and why? We don’t know. Can we prove it is better? We can’t.

My question is, will it ever stabilize?

I find most of this churn in the things I see — web development, ops. I don’t follow the embedded world and the desktop world — or the IT or networking world — as much. Maybe it’s a little more organized there, where the application problems are harder and more rigor is required to play in the sandbox.

I would like to see software development more closely approach a formality of process. To be closer to bridge building, and less in fashion. I would like software to not be driven by developer-advocates and vendors, but by the people on the ground that build and run tools.

Now, perhaps a civil engineer will come along and document a lot of huge churn in the way we design intersections and bust me up, or an EE will say the circuit design community is fractured too. That’ll probably happen. I kind of like the SPUI. But the thing I want is … ENGINEERING.

Ok, back to regularly scheduled programming! Happy thoughts!

Read the whole story
futurile
127 days ago
reply
Interesting!
London
Share this story
Delete

A Video Guide to Everything You Need to Know About Building Climbing Anchors

1 Comment

Use this guide as a general overview. Remember, learning about rock climbing online serves as a tool, but in no way are videos or...

The post A Video Guide to Everything You Need to Know About Building Climbing Anchors appeared first on Moja Gear.

Read the whole story
futurile
145 days ago
reply
This is really extensive, each anchor is clearly explained and shown. As a European I just use slings though!
London
Share this story
Delete
Next Page of Stories