The Lean Startup, by Eric Ries, is the first book on Startup’s that I’ve read cover to cover, in turn fuelling my ambition to develop a product of my own. This book has re-evaluated my thoughts on how a founder should focus on turning their concept idea into an initial barebones product that they can continuously experiment, validate, and refine, against measurable metrics and guidelines. A definite must read for any curious entrepreneur, or any software engineer that would like to realise how they can best apply their development skills at particular stages.
I’ve created this summary for me to review from time to time when I want a refresher, as it is quite knowledge-rich, but also to remind myself to keep on track at various stages of startup product development. I figured there may be other people out there who would like a to-the-point summary of the book to read in a shorter amount of time. Mid-way through this summary I also discovered that Derek Sivers also creates summaries for the books he reads, so go check his stuff out too!
The 5 Principles of The Lean Startup
- Entrepreneurs are everywhere
- Entrepreneurship is management
- Validated learning
- Innovation accounting
Why Startups fail so badly everywhere
First Problem: The allure of a good plan, solid strategy, and thorough market research
Second Problem: Some entrepreneurs and investors adopt the “Just Do It” approach
“The passion that people bring to these new ventures are resources too precious to waste. We can—and must—do better”
1 – START
Goal: Figure out the right thing to build (what customers want and will pay for) as quickly as possible
2 – DEFINE
A startup is a human institution designed to create a new product or service under conditions of extreme uncertainty
3 – LEARN
Validated Learning is the process of demonstrating empirically that a team has discovered valuable truths about a startup’s present and future business prospects
Metcalfe’s Law: The value of a network as a whole is proportional to the square of the number of participants
Which of our efforts are value-creating and which are wasteful?
Learning is the essential unit of progress for startups
The effort that is not absolutely necessary for learning what customers want can be eliminated
Postponing getting any data until you are certain of success leads to;
- increase of the amount of wasted work
- decreasing essential feedback
- dramatically increases the risk that a startup will build something nobody wants
Learn to see every startup in any industry as a grand experiment
In the Lean Startup model, every product, every feature, every marketing campaign – everything a startup does – is understood to be an experiment designed to achieve validated learning.
4 – EXPERIMENT
If the plan is to see what happens, a team is guaranteed to succeed – at seeing what happens – but won’t necessarily gain validated learning
If you cannot fail, you cannot learn
From Alchemy to Science
- A true experiment follows the scientific method. It begins with a clear hypothesis that makes predictions about what is supposed to happen. It then tests those predictions empirically.
- Startup experimentation is guided by the startup’s vision.
- The goal of every startup is to discover how to build a sustainable business around that vision
The value hypothesis tests whether a product or service really delivers value to customers once they are using it
The growth hypothesis tests how new customers will discover a product or service
Find early adopters: the customers who feel the need for the product most acutely. These customers tend to be more forgiving of mistakes and are especially eager to give feedback.
An experiment is a product
- In the Lean Startup model, an experiment is more than just a theoretical inquiry; it is also a first product
- If this or any other experiment is successful, it allows the manager to get started with his or her campaign: enlisting early adopters, adding employees to each further experiment or iteration, and eventually start to build a product
- By the time that product is ready to be distributed widely, it will already have established customers
- It will have solved real problems and offer detailed specifications for what needs to be built
Mark Cook – Kodak Gallery – 4 questions for his team:
- Do customers recognise that they have the problem you are trying to solve?
- If there was a solution, would they buy it?
- Would they buy it from us?
- Can be build a solution for that problem?
Remember: Planning is a tool that only works in the presence of a long and stable operating history. Changing such a mind-set is hard but critical to startup success. It is imperative to make this change.
HOW VISION LEADS TO STEERING
A startup is a catalyst that transforms ideas into products. As customers interact with those products, they generate feedback and data. The feedback is both qualitative (what they like and don’t like), and quantitative (how many people use it and find it valuable)
How to build a sustainable business is the outcome of those experiments
We need to focus our energies on minimising the total time through this feedback loop: the essence of steering a startup
Leap-of-faith assumptions: The riskiest elements of a startup’s plan, the parts on which everything depends
- Two most important assumptions are the value & growth hypotheses, which give rise to tuning variables that control a startup’s engine of growth
- Each iteration of a startup is an attempt to rev this engine to see if it will turn. Once it is running, the process repeats, shifting into higher and higher gears
When clear of these leap-of-faith assumptions:
Step 1: Enter the Build phase as quickly as possible with a Minimum Viable Product.
- The MVP is the version of the product that enables a full turn of the Build-Measure-Learn loop, with a minimum amount of effort and the least amount of development time. (It may lack many features that may prove essential later on) – We must be able to measure its impact
Step 2: When we enter the Measure phase, the biggest challenge will be determining whether the product development efforts are leading to real progress. (If we’re building something that nobody wants, it doesn’t much matter if we’re doing it on time and on budget).
- Innovation accounting, a quantitative approach that allows us to see whether our engine-tuning efforts are bearing fruit. It allows us to create learning milestones (ways of assessing progress accurately and objectively)
Step 3: The Pivot. Upon completion of the Build-Measure-Learn loop, we confront the most difficult question any entrepreneur faces: whether to pivot the original strategy or persevere. If we’ve discovered that one of our hypotheses is false, it is time to make a major change to a new strategic hypothesis.
The Lean Startup method builds capital-efficient companies because it allows startups to recognise that it’s time to pivot sooner, creating less waste of time and money.
Although we write the feedback loop as Build-Measure-Learn because the activities happen in that order, our planning really works in the reverse order:
- we figure out what we need to learn
- use innovation accounting to figure out what we need to measure to know if we are gaining validated learning
- figure out what product we need to build to run that experiment and get that measurement
5 – LEAP
The role of strategy in Startups is to help figure out the right questions to ask
Every business plan begins with a set of assumptions. It lays out a strategy that takes those assumptions as a given and proceeds to show how to achieve the companies vision. The goal of a startup’s early efforts should be to test these assumptions as quickly as possible.
The first challenge for an entrepreneur is to build an organisation that can test these assumptions systematically. The second challenge is to perform that rigorous testing without losing sight of the companies overall vision
What differentiates the success stories from the failures is that the successful entrepreneurs had the foresight, the ability, and the tools to discover which parts of their plans were working brilliantly and which were misguided, and adapt their strategies accordingly
Value & Growth
- The first step in understanding a new product or service is to figure out if it is fundamentally value-creating or value-destroying. A startups earliest strategic plans are likely to be hunch or intuition-guided
- Value-destroying: A business thats grows through continuous fund-raising from investors, and lots of paid advertising but does not develop a value-creating product (success theatre – using the appearance of growth to make it seem that they are successful)
Genchi Gembutsu (japanese term for “Go and see for yourself”): While a company working on sustaining innovation knows enough about who and where their customers are to use genchi gembutsu to discover what customers want, startups’ early contact with potential customers merely revels what assumptions require the most testing
“Get out of the building” (Steve Blank)
- “Metrics are people too”. Customers behaviour is measurable and changeable
- The facts that we need to gather about customers, markets, suppliers, and channels exist only “outside the building”. Startups need extensive contact with potential customers to understand them, so get out of your chair and get to know them
- Confirm that your leap-of-faith questions are based in reality, that the customer has a significant problem worth solving
Design and the Customer Archetype
- The goal of such early contact with customers is to clarify at a basic, coarse level that we understand our potential customer and what problems they have, to craft a customer archetype. (An essential guide for product development and ensures that the daily prioritisation decisions that every product team must make are aligned with the customer to whom the company aims to appeal)
- A new breed of designers is developing brand-new techniques under the banner of Lean User Experience (Lean UX). They recognise that the customer archetype is a hypothesis and not a fact. The customer profile should be considered provisional until the strategy has shown via validated learning that we can serve this type of customer in a sustainable way.
- Two ever-present dangers when entrepreneurs conduct market research and talk to customers
- Just-do-it school of entrepreneurship are impatient to get started and dont want to spend time analysing their strategy and just build immediately after a few cursory customer conversations
- Analysis paralysis, endlessly refining their plans
- The answer to the above is a concept called the Minimum Viable Product (MVP)
6 – TEST
An MVP helps entrepreneurs start the process of learning as quickly as possible. It is the fastest way to get through the Build-Measure-Learn feedback loop with the minimum amount of effort. The goal of the MVP is to begin the process of learning, not end it. It is designed to test fundamental business hypotheses
Before products can be sold successfully to the mass market, they have to be sold to early adopters:
- They are a special breed of customer that accept-in fact prefer-an 80% solution
- They use their imagination to fill in what a product is missing, (all they care about above all is being the first to use or adopt a new product or technology)
- They are suspicious of something that is too polished
- MVP’s range in complexity from extremely simple smoke tests (little more than an advertisement) to actual early prototypes complete with problems and missing features. When in doubt, simplify.
- The lesson of the MVP is that any additional work beyond what was required to start learning is waste, no matter how important it might have seemed at the time
The Video MVP: Demonstration video targeting early adopters
The Concierge MVP: A learning activity designed to test the leap-of-faith assumptions in the companies growth model. A common outcome of a concierge MVP is to invalidate the companies proposed growth model, making it clear that a different approach is needed: even if the initial MVP is profitable.
Wizard of Oz tests are where customers believe they are interacting with the actual product, but behind the scenes human beings are doing the work. Like the concierge MVP, this approach is incredible inefficient
The Role of Quality and Design in an MVP
- Modern production processes’ rely on high quality as a way to boost efficiency
- Edwards Deming – “the customer is the most important part of the production process”
- Variation in process yields products of varying quality in the eyes of the customer that at best require rework and at worst lead to a lost customer
- For startups, a good quality principle: If we do not know who the customer is, we do not know what quality is
- Even a “low-quality” MVP can act in service of building a great high-quality product. We should use this as an opportunity to learn what attributes customers care about, which is infinitely better than mere speculation or whiteboard strategising, because it provides a solid empirical foundation on which to build future products
- The Lean Startup method is not opposed to building high-quality products, but only in service of the goal of winning over customers. We must be willing to set aside our traditional professional standards to start the process of validated learning as soon as possible. (but not in a sloppy or undisciplined way)
A simple rule to follow: Remove any feature, process, or effort that does not contribute directly to the learning you seek.
Speed Bumps in building an MVP
- Common speed bumps: legal issues, fears about competitors, branding risks, and the impact on morale
- Part of the special challenge of being a startup is the near impossibility of having your idea, company, or product be noticed by anyone, let along a competitor
- The reason to build a new team to pursue an idea is that you believe you can accelerate through the Build-Measure-Learn feedback loop faster than anyone else can. If this is true, it makes no difference what the competition knows. If this is not true, a startup has much bigger problems, and secrecy wont fix them.
- Startups have the advantage of being obscure to avoid product failure long-term damage. Rather than lamenting them, use these advantages to experiment under the radar and then do a public marketing launch once the product has proved itself with real customers.
- It helps to prepare for the fact that MVPs often result in bad news. Unlike traditional concept tests or prototypes, they are designed to speak to the full range of business questions, not just design or technical ones, and they often provide a needed dose of reality.
- If an MVP fails, teams are liable to give up hope and abandon the project altogether, but this is a solvable problem: Innovation Accounting
Successful entrepreneurs do not give up at the first sign of trouble, nor do they persevere the plain right into the group. Instead they possess a unique combination of perseverance and flexibility. The MVP is just the first step on a journey of learning.
7 – MEASURE
A startup’s job is to: Rigorously measure where it is right now, confronting the hard truths that assessment reveals, Devise experiments to learn how to move the real numbers closer to the ideal reflected in the business plan
Innovation Accounting: A kind of accounting geared specifically to disruptive innovation. It enables startups to prove objectively that they’re learning how to group a sustainable business. It begins by turning the leap-of-faith assumptions into a quantitative financial model
- Every business plan has some kind of model associated with it, even if it’s written on the back of a napkin. That model provides assumptions about what the business will look like at a successful point in the future
How Innovation Accounting Works: 3 Learning Milestones
- Use a minimum viable product to establish real data on where the company is right now (Without a clear-eyed picture of your current status – no matter how far from the goal you may be – you cannot begin to track your progress
- Attempt to tune the engine from the baseline toward the ideal. This may take many attempts. After the startup has made all the micro changes and product optimisations it can to move its baseline toward the ideal, the company reaches a decision point.
- Pivot or persevere.
Establish the Baseline
- A startup might create a complete prototype of its product and offer to sell it to real customers through its main marketing channel, which would test most of the startup’s assumptions and establish baseline metrics for each assumption simultaneously.
- Alternatively a startup might prefer to build separate MVPs that are aimed at getting feedback on one assumption at a time.
- Before building the prototype, the company might perform a smoke test with its marketing materials.
- This is an old direct marketing technique in which customers are given the opportunity to preorder a product that has not yet been built.
- A smoke test measures only one thing: whether customers are interested in trying a product. By itself, this is insufficient to validate an entire growth model. Nonetheless, it can be very useful to get feedback on this assumption before committing more money and other resource to the product.
- These MVP’s provide the first example of a learning milestone. An MVP allows a startup to fill in real baseline data in its growth model – conversion rates, sign-up and trial rates, customer lifetime value, and so on – and this is valuable as the foundation for learning about customers sand their reactions to a product even if that foundation begins with extremely bad news.
- It makes sense to test the riskiest assumptions in a business plan first, If you can’t find a way to mitigate these risks toward the ideal that is required for a sustainable business there is no point in testing the other.
- Before building the prototype, the company might perform a smoke test with its marketing materials.
Every product development, marketing, or other initiative that a startup undertakes should be targeted at improving one of the drivers of its growth model.
Pivot or Persevere
- Over time, a team that is learning its way toward a sustainable business will see the numbers in its model rise from the horrible baseline established by the MVP and converge to something like the ideal established in the business plan.
- A startup that fails to do so will see that ideal recede ever farther into the distance. When this is done right, even the most powerful reality distortion field won’t be able to cover up this simple fact: if we’re not moving the drivers of our business model, we’re not making progress. That becomes a sure sign that it’s time to pivot.
Improving a Product on Five Dollars a Day
- Some Funnel Metrics behaviours: Customer Registration, Download of application, Trial, Repeat usage, Purchase
- Budget idea: $5 per day to buy clicks on Google AdWords – good for little money (not sure if scalable/feasible now in 2017) – 100 clicks every day – for learning was priceless
- Able to to measure our products performance with a brand new set of customers. Each time product was revised, a brand new report card was available on how the product was doing the very next day
- Funnel Metrics graph example
Cohort Analysis: one of the most important tools of startup analytics. Each cohort represents an independent report card
- One looks at the performance of each group of customers that comes into contact with the product independently. Each group is called a cohort. Each conversion rate shows the percentage of customer who registered in that month who subsequently went on to take the indicated action.
- This technique is useful in many types of business, because every company depends for its survival on sequences of customer behaviour called flows.
- Customer flows govern the interaction of customers with a company’s products. They allow us to understand a business quantitatively and have much more predictive power than do traditional gross metrics.
A sign of a successful pivot: the new experiments you run are overall more productive than the experiments before.
- This is the pattern: poor quantitative results force us to declare failure and create the motivation, context, and space for more qualitative research. These investigations produce new ideas – new hypotheses – to be tested, leaning to a possible pivot.
- Each pivot unlocks new opportunities for further experimentation, and the cycle repeats. Each time we repeat this simple rhythm: establish the baseline, tune the engine, and make a decision to pivot or persevere
Optimisation versus learning
- If you are building the wrong thing, optimising the product or its marketing will not yield significant results.
- A startup has to measure progress against a high bar: evidence that a sustainable business can be built around its products or services. That’s a standard that can be assessed only if a startup has made clear, tangible predictions ahead of time.
- The innovation accounting framework makes it clear when the company is stuck and needs to change direction.
- Companies of any size that have a working engine of growth can come to rely on the wrong kind of metrics to guide their actions. This what tempts managers to resort to the usual bag of success theatre tricks: last-minute ad buys, channel stuffing, and whiz-bang demos, in a desperate attempt to make the gross numbers look better. Energy invested in success theatre is energy that could have been used to help build a sustainable business.
Vanity metrics: give the rosiest possible picture
- Innovation accounting will not work if a startup is being misled by vanity metrics: gross number of customers and so on.
- The alternative is the kind of metrics we use to judge our business an our learning milestones, actionable metrics
Split-test experiment: Where different versions of a product are offered to customers at the same time. By observing the changes in behaviour between the two groups, one can make inferences about the impact of the different variations.
- To figure out if the new X was effective, you need to keep track of the X primary goal’s total value for both groups of customers
- This technique is sometimes called A/B testing after the practice of assigning letter names to each variation
- Split testing also helps teams refine their understanding of what customers want and don’t want.
- Split testing can reveal that extra features don’t change customer behaviour, and can call a context-respective belief into question
Kanban: a lean manufacturing principle/capacity constraint
- User stories cannot be considered complete until they led to validated learning.
- Stories can be cataloged as being in one of four states of development:
- The product backlog
- Actively being built
- Done (feature complete from a technical point of view)
- In the process of being validated.
- Defined as “knowing whether the story was a good idea to have been done in the first place”
- Validation usually would come in the form of a split test showing a change in customer behaviour but also might include customer interviews or surveys
- The kanban rule permitted only so many stories in each of the four states.
- As stories flow from one state to the other, the buckets fill up.
- Once a bucket becomes full, it cannot accept more stories.
- Only when a story has been validated can it be removed from the kanban board
- If the validation fails and it turns out the story is a bad idea, the relevant feature is removed from the product
- The only way to start work on new features is to investigate some of the stories that are done but haven’t been validated. This often requires non-engineering efforts: talking to customers, looking at split-test data, and the like
- This progress occurs in fits and starts at first. Engineering may finish a big batch of work, followed by extensive testing and validation. As engineers look for ways to increase their productivity, they start to realise that if they include the validation exercise from the beginning, the whole team can be more productive (e.g. why build a new feature that is not part of a split-test experiment?)
- A solid process lays the foundation for a healthy culture, one where ideas are evaluated by merit and not by job title.
- Most important, teams working in this system begin to measure their productivity according to validated learning, not in terms of the production of new features
The Value of the Three A’s
- For a report to be considered actionable, it must demonstrate clear cause and effect. Otherwise it is a vanity metric
- By contrast, vanity metrics fail this criterion
- They wreak havoc because they prey on a weakness of the human mind. When the numbers go up, people think the improvement was caused by their actions, by whatever they were working on at the time.
- Actionable metrics are the antidote to this problem. When cause and effect is clearly understood, people are better able to learn from their actions.
- All too many reports are not understood by the employees and managers who are supposed to use them to guide their decision making.
- There is an antidote to this misuse of data. First, make the reports as simple as possible so that everyone understands them. Remember the saying “Metrics are people, too”. The easiest way to make reports comprehensible is to use tangible, concrete units.
- Each cohort analysis report deals with people and their actions, which are far more useful than piles of data points.
- First solution: We need to be able to test the data by hand, in the messy real world, by talking to customers. This is the only way to be able to check if the reports contain true facts. Managers need the ability to spot check the data with real customers.
- Second benefit: Systems that provide this level of audibility give managers and entrepreneurs the opportunity to gain insights into why customers are behaving the way the data indicates
- Second solution: Those building reports must make sure the mechanisms that generate the reports are not too complex. Whenever possible, reports should be drawn directly from the master data, rather than from an intermediate system, which reduces opportunities for error.
- Every time a team has one of its judgements or assumptions overturned as a result of a technical problem with the data, its confidence, morale, and discipline are undermined
Only 5 percent of entrepreneurship is the big idea, the business model, the whiteboard strategising, and the splitting up of the spoils. The other 95 percent is the gritty work that is measured by innovation accounting: product prioritisation decisions, deciding which customers to target or listen to, and having the courage to subject a grand vision to constant testing and feedback.
We all must face this fundamental test: deciding when to pivot and when to persevere. To understand what happens during “the photo montage” (book ref.) we have to understand how to pivot.
8 – PIVOT (OR PERSEVERE)
Everything discussed so far is a prelude to a simple question: Are we making sufficient progress to believe that our original strategic hypothesis is correct, or do we need to make a major change?
A pivot: A structured course correction designed to test a new fundamental hypothesis about the product, strategy, and engine of growth
There is no bigger destroyer of creative potential than the misguided decision to persevere
Startup productivity is about aligning our efforts with a business and product that are working to create value and drive growth
Successful pivots put us on a path toward growing a sustainable business
Innovation Accounting leads to faster Pivots
- Example of 4 big leaps of faith
- The goal of creating learning milestones is not to make the decision easy; it is to make sure that there is relevant data in the room when it comes time to decide.
- The land of the living dead: When a company has achieved a modicum of success – just enough to stay alive – but is not living up to the expectations of its founders and investors
- A pivot requires that we keep one foot rooted in what we’ve learned so far, while making a fundamental change in strategy in order to seek even greater validated learning
- A legacy product: One that was no longer suited to the goals of the company
A Startup’s runway is the number of pivots it can still make
- The true measure of runway is how many pivots a startup has left: the number of opportunities it has to make a fundamental change to its business strategy
- Measuring runway through the lens of pivots rather than that of time suggest another way to extend that runway: get to each pivot faster. In other words, the startup has to find ways to achieve the same amount of validated learning at lower cot or in a shorter time.
Entrepreneurs need to face their fears and be willing to fail, often in a public way
The Pivot or Persevere meeting
- The decision to pivot requires a clear-eyed and objective mindset
- Telltale signs of the need to pivot: the decreasing effectiveness of product experiments and the general feeling that product development should be more productive. Whenever you see these symptoms, consider a pivot
- The decision to pivot is emotionally charged for any startup and has to be addressed in a structured way.
- One way to mitigate this challenge is to schedule the meeting in advance.
- A recommendation is that every startup have a regular “pivot or persevere” meeting.
- Less than a few weeks between meetings is too often, and more than a few methods is too infrequent. Each startup needs to find its own pace.
- Each pivot or persevere meeting requires the participation of both the product development and leadership teams.
- It is not necessary to throw out everything that came before the pivot and start over. It’s about repurposing what has been built and what has been learned to find a more positive direction
- One way to mitigate this challenge is to schedule the meeting in advance.
A catalog of pivots:
- What was previously considered a single feature in a product becomes the whole product
- In reverse: A single feature is insufficient to support a whole product. What was considered the whole product becomes a single feature of a much larger product
Customer Segment Pivot
- The company realises that the product it is building solves a real problem for real customers but that they are not the type of customers it originally planned to serve. The product hypothesis is partially confirmed, solving the right problem, but for a different customer than originally anticipated
Customer Need Pivot
- This is a case where the product hypothesis is partially confirmed; the target customer has a problem worth solving, just not the one that was originally anticipated
- Refers to a change from an application to a platform or vice versa
Business Architecture Pivot
- In a business architecture pivot, a startup switches architectures. Some companies change from high margin, low volume by going mass market, others originally designed for the mass market, turned out to require long and expensive sales cycles
Value Capture Pivot
- There are many ways to capture the value a company creates. These methods are referred to commonly as monetisation or revenue models. These terms are much too limiting. Implicit in the idea of monetisation is that it is a separate “feature” of a product that can be added or removed at will. In reality, capturing value is an intrinsic part of the product hypothesis.
Engine of Growth Pivot
- There are three primary engines of growth that power startups: Viral, Sticky, Paid Growth
- In this type of pivot, a company changes its growth strategy to seek faster or more profitable growth. Commonly but not always, the engine of growth also requires a change in the way value is captured
- A channel pivot is a recognition that the same basic solution could be delivered through a different channel with greater effectiveness
- A company discovering a way to achieve the same solution by using a completely different technology
- The only question is whether the new technology can provide superior price and/or performance compared with the existing technology
A Pivot is a Strategic Hypothesis
- A pivot is better understood as a new strategic hypothesis that will require a new minimum viable product to test. Even after a company achieves initial success, it must continue to pivot.
- The critical skill for managers today is to match those theories to their present situation so that they apply the right advice at the right time. Modern managers cannot have escaped the deluge of recent books calling on them to adapt, change, reinvent, or upend their existing businesses. Many of the works in this category are long on exhortations and short on specifics.
- A pivot is not just an exhortation to change. Remember, it is a special kind of structured change designed to test a new fundamental hypothesis about the product, business model, and engine of growth. It is the heart of the Lean Startup method. It is what makes the companies that follow Lean Startup resilient in the face of mistakes: if we take a wrong turn, we have the tools we need to realise it and the agility to find another path.