Key Takeaways
- Organizations who don't measure the value of what they deliver have no idea which of their ideas are good, and which fail to meet customer needs. As a result, much of their time, effort, and investment may be wasted.
- Organizations improve their performance by framing their ideas as experiments that explicitly test the ideas for the value they have to customers. Running these experiments as quickly as possible, and with as little effort as possible, helps the organization to dramatically improve their effectiveness at meeting customer needs.
- Setting strategic goals in terms of value helps an organization be clear about what it seeks to achieve, and helps everyone in the organization align their work toward these goals.
- In addition to improving the value they deliver, most organizations also need to improve their speed and effectiveness at delivering value. Framing improvements as experiments run in fast, frequent cycles helps organizations to rapidly improve.
- For organizations to improve the value they deliver, they need relatively fast feedback loops that give them the ability to try new ideas quickly and determine which ones are worth further investment.
We all know what “agility” is about, right? It’s about delivering faster, eliminating waste, and becoming more efficient. Or at least that’s what I hear from a lot of execs. But what if that’s not really true? Or at least not the whole truth?
Let me frame the dilemma in a question: What’s better? Going 100 miles per hour in the wrong direction, or going 1 mile per hour in the right direction?
It sounds like a silly question, but many organizations I speak with don’t really know if they’re going in the right direction. They work incredibly hard to increase their delivery speed, but once they’ve shipped something they turn their attention to the next release. They have no idea whether what they just delivered improved customer outcomes. What if most of what they just delivered turned out to be waste, or lacking in any real value to their customers?
Most of what we do is waste, we just don’t know it yet
Microsoft and Amazon are regarded as two of the most successful business organizations in the world today, if not in the entirety of history. And yet, by their own measures, most of their new ideas fail to achieve the goals they set out to achieve. In research spanning more than a decade, Ronny Kohavi found, at Microsoft, the following approximate results:
- ⅓ of ideas improved their intended measures
- ⅓ of ideas had no effect on their intended measures
- ⅓ of ideas actually made their intended measures worse
Or to put it more simply, two-third of ideas had zero or negative business benefit. And it didn’t matter who conceived the ideas; executives and other highly experienced, and usually highly paid individuals had no better track record in devising great ideas than relatively inexperienced and lowly-paid employees. In short, everyone has a bad idea sometimes.
You might be tempted to think that your organization does better than that, but if you’re not measuring the impact of the work that you do, how can you be sure? Perhaps you are luckier than most, but it’s very unlikely. Organizations that measure the impact of their work report largely similar results: most of their ideas fail to improve customer outcomes. The key is to identify the ideas that don’t add value early and not develop them any further.
What is business value?
In short, organizations create business value when they improve customer experiences. This value may be realized in terms of increased brand value, improved revenue, increased employee satisfaction, or a variety of other measures, but the source of all value is customer experience.
The Evidence-Based Management framework (EBM) focuses on two kinds of business value measures: Current Value and Unrealized Value. Current Value (CV), as its name suggests, is the value that customers experience today, from the existing product or service. Understanding CV provides you with a good picture of how you’ve performed so far, but it’s not the whole picture in terms of value.
How much value is enough?
If you think of value as profit or EPS (a very limited and inward-looking view), then you might believe that “there’s never enough”, but from a customer outcome perspective, there is a point where a customer is completely satisfied, just as a diner in the best restaurant in the world will eventually find themselves unable to enjoy another bite. We describe the satisfaction gap between a customer’s current experience (CV) and their desired experience in terms of Unrealized Value (UV). (See Figure 1.)
Closing these satisfaction gaps represents opportunities to an organization, and these satisfaction gaps also set a limit on how much value an organization can create. Once a satisfaction gap is closed, an organization will have to seek out new customers with different satisfaction gaps in order to fuel their growth.
Figure 1: Unrealized Value is created by gaps between a customer’s current and desired experience
Framing goals in terms of value
Since closing satisfaction gaps and reducing Unrealized Value are the only ways that an organization can create value, setting goals in terms of UV helps an organization be clear about what it seeks to achieve. But before we say more about that, let’s talk about goals. We break them down as follows:
- At the top level are Strategic Goals, which are important things that the organization would like to achieve over a relatively long term, say, 3-5 years. We think framing these goals in terms of reducing Unrealized Value provides clarity and motivation for the entire organization, because it provides people in the organization with something specific and motivating to achieve.
- Because the time scale for Strategic Goals is relatively long, people need something nearer-term to steer toward. For this reason, organizations also need Intermediate Goals, something 3-6 months in the future. These provide more concrete focus, and provide people with an idea of whether they are heading in the right direction.
- Because most people need something even shorter-term and more concrete on which to focus. For this reason, organizations also need Immediate Tactical Goals, whose scope is anywhere from a few days up to 4 weeks in the future. These very specific targets help people steer toward longer-term Intermediate and Strategic Goals.
Progressing toward goals
We live in an uncertain world, and the path toward ambitious goals focused on customer outcomes is even more uncertain, as the data mentioned earlier underscores. Reaching ambitious goals requires organizations to use empiricism, or purposeful experiments seeking towards goals (see Figure 2).
Figure 2: Empiricism helps organizations pursue goals in uncertain times
Here’s how it works: considering their strategic and intermediate goals, teams will form Immediate Tactical Goals that they think, if they achieve them, will help them advance them toward their longer term goals. They form ideas about how they can improve toward the goals (Hypotheses). Acting on these ideas produces new information (Experiment & Measure) that they examine to see if they achieved the goal and learned interesting things (Inspect). This then informs new ideas about how to move ahead (Adapt).
And when teams adapt, they don’t just consider their progress toward the goal - they also consider whether the goals themselves need to change based on new information. A new competitor may have entered the scene, or some external event may have changed the desired outcomes of customers (as it did for many of us in the past 18 months).
What about speed and efficiency?
In the pursuit of value, speed and efficiency are important, but not for the typical reasons. Speed and efficiency are typically important when an organization is certain that what it is delivering has value, because “fast” and “efficient” translate to “more profitable”.
When value is uncertain, speed and efficiency are important for different reasons. The best way to validate an idea is to develop something minimal very quickly, deliver it, and measure the result. If the idea is valuable, you can invest more time in refining the solution, and if it’s not valuable you can move on to test other ideas. Delivery speed gives you the ability to test ideas quickly. In the EBM Framework, we refer to this Key Value Area as Time-to-Market, or T2M.
And what about efficiency?
Efficiency is focused on eliminating non-value-added activities that prevent a team from delivering quickly, or from delivering as much value as they are capable. Automation plays a role in this, but a larger role is usually played by frequent interruptions and being pulled away to work on things that don’t add value.
Examples of other non-value-added work include meetings to coordinate between teams that would not be needed if the delivery team was sufficiently skilled to simply do the work themselves. Or having to go to someone for a decision that may not really be needed if the delivery team was sufficiently empowered. In the EBM Framework, we refer to this Key Value Area as Ability to Innovate, or A2I.
Measure outcomes to understand value
When organizations measure speed and efficiency, they are looking for improvements in areas that are easy to see (or to measure), and not in the areas that are most likely to identify big opportunities for improvement. Velocity is easy to measure. Cost is easy to measure. But they’re not going to tell the organization which 60% percent of its ideas are not worth developing.
The EBM framework talks about three kinds of measures:
- Activities: These are things that people in the organization do, such as perform work, go to meetings, have discussions, write code, create reports, attend conferences, and so forth.
- Outputs: These are things that the organization produces, such as product releases (including features), reports, defect reports, product reviews, and so on.
- Outcomes. These are desirable things that a customer or user of a product experiences. They represent some new or improved capability that the customer or user was not able to achieve before.
Measuring activities and outputs is easy, and not unimportant, but only as a means to improving the organization’s ability to deliver valuable outcomes, quickly and efficiently. Measuring outcomes will tell an organization whether it is delivering valu
Different lenses put different things in focus
So where should an organization focus if it wants to improve? Should it focus on delivering value first, then become fast and efficient? Or should it become fast and efficient, then worry about value? Or should it improve across all categories.
For most organizations, the Satisfaction Gap is sizable. This may not sound like a good thing, but it is, because it means that the organization has a large opportunity to create value for customers. If the Satisfaction Gap is very small, the organization has very few opportunities to improve customer experiences and therefore few opportunities for growth
The next thing the organization needs to look at is how quickly and effectively it can deliver a new product or service capability that improves the value that customers experience. If the organization’s T2M is very long (let’s say, longer than 6 months) then they won’t be able to improve customer value very fast and they should focus there first.
If the organization can deliver quickly, but only on very small incremental improvements, then they may need to also focus on their effectiveness as well.
Once the organization can deliver a new capability very frequently (let’s say, once a month), then they can measure frequently enough to start to get a better sense of what is valuable. For digital products, however, most organizations need to get their T2M down to days, not weeks, in order to run effective experiments about value.
Once they can run frequent experiments, organizations can work on improving their value delivery while they also work on time-to-market and effectiveness.
Goals at three levels, revisited
So an organization with a Strategic Goal of eliminating the adverse health and economic effects of a particular newly discovered disease might have an intermediate goal of producing a treatment that either prevents the disease, or mitigates the effects of the disease.
Would their Intermediate Goal be to produce a vaccine? That might be one way to prevent the disease, but it may not be possible, as those treating some diseases have discovered. Besides, it is more helpful to express the goal in terms of the outcomes that you want to achieve, so that they are clear and can be tested, rather than expressing them in terms of some output (the vaccine) which may not achieve the desired result.
So both Strategic and Intermediate Goals are best expressed in terms of the outcome (expressed in CV or UV terms) that you want to achieve.
Short-term (Immediate Tactical) goals are different - for these you might need to focus on activities and outputs, like reducing the time it takes to get tested, or reducing the time it takes to run a new treatment trial, in order to develop the capability to run experiments faster. But even here, the eventual goal is to run small, fast experiments that help you determine if you are headed in the right direction.
Conclusion
Many organizations frame their goals for agility as going faster and being more efficient because they assume that what they are doing today is valuable. Once they start measuring value, they often find they have a different challenge - to improve the value they deliver. To do this effectively, they need relatively fast feedback loops that give them the ability to try new ideas quickly and determine which ones are worth further investment.
Organizations may have to improve their capability to deliver to effectively run these experiments. They may have to improve their speed and their effectiveness to become effective at quickly trying ideas, gathering and analyzing data, and adapting their goals, strategy, and tactics based on those results.
About the Author
Kurt Bittner has more than 30 years of experience delivering working software in short, feedback-driven cycles. He has helped a wide variety of organizations adopt agile software delivery practices, including large banking, insurance, manufacturing, and retail organizations, as well as large government agencies. He has worked for or with large software delivery organizations including Oracle, HP, IBM, and Microsoft, and is a former technology industry analyst with Forrester Research. His focus is on helping organizations build strong, self-organizing, high-performance teams that deliver solutions that customers love. He is the author of four books on software development-related topics, including The Nexus Framework for Scaling Scrum. He is based in Boulder, Colorado and serves as VP of Enterprise Solutions for Scrum.org.