Understanding the science and math behind lean principles and practices can enable engineering leaders to advocate for and implement them in their workplace. This way they can directly impact employee engagement and morale, as well as the bottom line, as David Van Couvering explained in his talk about applying lean principles and practices for delivering value at QCon Plus 2020.
From the book Accelerate, we learnt that the software delivery metrics that best predict improved overall company performance are lead time, deploy frequency, mean time to recovery and change failure rate. The authors of this book have also identified the specific practices that contribute to these metrics, and Van Couvering called out that two key sets of practices are Continuous Delivery and Lean.
Continuous Delivery is primarily focused on automating the process from making a change to getting that change into production, whereas Lean is focused on processes and techniques that can significantly improve overall flow and throughput of your overall development process.
It turns out that team members can also benefit from applying lean principles and practices. When you apply these principles, the improved flow, less burnout and faster feedback results in happier, more motivated teams, as Van Couvering explained:
I know that I am much happier when I can "get sh*t done" faster, and I know that what I’m building delivers real value to my customers, and leadership really sees and appreciates what I’m doing. But the research in the book Accelerate bears this out - when we adopt these practices, you can predict an improved engagement and reduced burnout of the development teams.
Van Couvering mentioned that it can be hard to convince leaders that process changes such as releasing more frequently with fewer approval gates, or investing in automation, is something more than just engineers being excited about the latest new trends in software. Knowing the math and science behind the lean principles has helped him:
I am now empowered to provide compelling evidence to often hesitant and resistant leaders that this is in their best interest and will bring them real results.
InfoQ interviewed David Van Couvering, distinguished MTS at eBay, about software delivery performance, addressing causes of delay, reducing work in progress and increasing flow, and improving our feedback and response cycle.
InfoQ: How would you define software delivery performance and what makes it important for engineering leaders?
David Van Couvering: I use the term software delivery performance as it is defined in the book Accelerate by Dr. Nicole Forsgren, Jez Humble et. al. Through four years of research, they identified a cluster of metrics that measure how effectively an organization delivers software. Their research showed that this measure of software delivery performance reliably predicts improved performance of a company as a whole (profitability, market share, and productivity). In particular, organizations that fit into the high performance cluster for software delivery were twice as likely to exceed their company performance goals compared to the low performance cluster. That’s pretty significant.
I find this incredibly valuable. For so many years software delivery practices have been adopted, or not, based primarily on instinct, emotion and intuition. But this book gives us a measure that has been shown, across a divergent set of company sizes and business verticals, to directly predict improved business outcomes and improved employee morale. It’s no longer something you have to argue about - you can just point to the data.
InfoQ: What can cause delays and how can we deal with those causes?
Van Couvering: It is important to understand that delay degrades the economic value of what we deliver - there is a cost to delays, and it can be significant. Think about the loss of opportunity or revenue if a software product is delivered late, especially in a highly competitive market segment. Delays also slow down feedback, which makes it harder to adapt to new information. You can also incur significant risk of outages or customer turnover if features are delivered late. With this in mind, just as we spend so much time optimizing and tuning the latency and throughput of our software systems, we should spend time to optimize and tune the latency and throughput of our development process.
It turns out when you look at the math and dynamics of product delivery pipelines, the biggest contributor to delay is letting queues back up. Unlike in manufacturing, these queues are invisible in software development, so it is important that we make an effort to make them visible, and then address them quickly and aggressively. Two powerful ways to reduce queues are limiting work in progress and keeping your batch sizes small.
InfoQ: What can be done to reduce work in progress and increase flow?
Van Couvering: Mathematically it has been shown that as you approach 100% utilization of your team’s resources, you exponentially increase your queue lengths. Also, once a queue has reached a high length, it is very hard to get out of that state. For these reasons, it is really important to have capacity buffering in your team to handle bursts of incoming work.
The most common approach to do this is to set some limit to the amount of work your team will take on. Note that this is not just what you’re currently working on, but what’s in your committed backlog. It’s hard to gauge exactly what that limit should be, so I recommend coming up with a best guess as some multiple of your team size (2x is a good starting place), monitor your backlog and see if it is staying short and stable, and adjust as needed. Note that you want to avoid setting it too low, because you’ll end up with a lot of idle time, and the benefit of reduced delays is eaten up by the cost of idle team members.
Once you’ve set that limit, there are various mechanisms to not accept any work over that limit. They are very similar to what you would do for a software system. You can shed load by just dropping any new work that comes in over the limit. You can use backpressure to tell the teams giving you work to stop. You can be smart about this and drop lower-value work. There are prioritizing techniques like Weighted Shortest Job First that allow you to prioritize your work not based on just business value, but taking into account contributors to cost of delay such as risk, opportunity enablement and timeliness.
Managing capacity through limiting work in progress also reduces the chance of congestion and improves flow. Keeping batch sizes small also significantly improves flow, reducing the amount of choppy variance to the rate at which you deliver work. Improved flow has significant downstream effects, such as improved feedback loops, more predictable delivery, and overall improved throughput.
InfoQ: How can we improve our feedback and response cycle?
Van Couvering: A fast feedback loop provides a ton of value. It allows you to eliminate unnecessary work in your backlogs as you learn what’s working and what is not - you are able to more efficiently head towards your goal, and you can be more confident that what you are delivering is taking the business forward.
A key aspect of this is identifying good metrics. OKRs are a common example of business metrics, but often these can’t be directly measured in what you deliver, so you need to identify good surrogate metrics. You also want to look for leading indicators that allow you to find out more quickly if you want to change course.
Reducing delays in your delivery pipeline using the techniques described above improves how quickly you get feedback. Having small batch sizes also allows you to deliver incremental value faster, which allows you to get feedback faster. This allows you to adjust, likely eliminating work from your backlog, which then improves the cycle time and gets you faster feedback.