Everyone loves a good metric. As product managers, we probably proudly proclaim that we are getting better at making decisions based on data rather than gut instinct. How everyone gathers, visualizes, and uses information — in product management and beyond — has certainly changed for the better over the years. In the past, we were overloaded with data and graphs to the point where we couldn’t see the forest through the trees anymore. But now, we have learned to become more selective. Now the few, hand-picked metrics we use should all be driving decisions and be highly actionable, right?
The truth is that measuring things in software development is hard. As Edwards Deming clearly stated “The most important things cannot be measured.” We’ve all seen how metrics can be gamed and abused (velocity, anyone?) and many have heard of the “Law of Unintended Consequences” and experienced in practice how measurement led to unintended and counterproductive behaviors. So a lot of folks have jumped on the #NoEstimates bandwagon and we’re all better off not measuring anything at all, right?
I’m not sure it’s that easy. The reality is that we’re all living in the business world and we’re being asked to help answer certain, very valid questions, such as:
What’s our customer satisfaction? How is it trending?
How are our Agile teams doing? Are we getting better?
How is our Agile / digital transformation progressing?
Are we getting the value we were hoping for out of our transformation efforts, coaching, etc.?
What ROI are we getting from the money we invest in our transformation?
For consultants and coaches: Am I moving the needle and making an impact?
Are we helping the organization deliver value faster?
Where do we need to deploy our limited coaching resources?
What bottlenecks in our value streams do we need to address?
While measurements are hard and there are certainly landmines to avoid, … Continue reading my full post here.
It comes at no surprise that predictability of Agile delivery is a big deal. I would define it as the team’s ability to predict with reasonable accuracy which backlog items they will be able to complete either by the end of a sprint or within a certain timeframe (in case of Kanban teams). And predictability scales up: multiple predictable teams may make a shared program they’re a part of predictable and multiple predictable programs combine to a predictable portfolio. Organizational leaders look for predictable delivery in order to be able to make business commitments.
Nonetheless, I’ve always been quick to point out that predictability may not always be what one wants to optimize for. A key factor affecting the importance of predictability is where a product is in its life cycle. A very young product at the beginning of its life will not require predictability. For such a product, especially if the organization follows approaches promoted by the Lean Startup, focus will be on defining an MVP and releasing it to the market as soon as possible, after which point the team will attempt to validate learnings as quickly as possible and pivot where necessary. Under these circumstances, iterating on the product quickly will be imperative. Therefore, I concluded, predictability is not important under these circumstances but something only more mature products (or larger, traditional organizations) require, right?
Recently, I realized that this it is the wrong question to ask whether a product requires its team(s) to be for what timeframe a development team needs to be predictable. If I’m iterating on my startup’s MVP, I need to be predictable, but maybe my planning horizon is only a week or two. If I’m maintaining a mature product, I may require high predictability in order to forecast 3 to 12 months out. Another factor necessitating high predictability is a market requiring long-term commitments from the companies serving it. Since the need for predictability is a function of the product needs, it’s quite possible that a well diversified company may have multiple products where the answers are different.
One caveat is that for the same product, one usually has to decide and pick one or the other: do I want to be highly predictable or highly responsive and able to turn on a dime? In most cases I can’t have my cake and eat it too.
My takeaway from this is: predictable delivery is always required, but it’s important to determine for what time horizon teams need to be able to make commitments and why.
Agile is pervasive and changes organizations, some more, some less. The same way as there are different levels of Agile planning (daily commitment, sprint, release plan, product roadmap and product vision), there are different levels of optimization an organization can achieve by applying Agile and principles based on the Lean Startup.
When a development team implements an Agile framework such as Scrum, it starts optimizing a number of its practices, which has certain benefits. However, because the focus is only the development team and not the larger organization, the level of optimization achieved is limited. The Agile team is embedded in an organization that doesn’t holistically apply Agile principles, therefore the optimization is limited to the technical team and hopefully the Product Owner, which is the only part of the team intersecting with the non-technical part of the business. While this yields some benefits, they are limited.
When the whole organization “is Agile”, it becomes possible to optimize the entire business, which includes the Development group, the Product organization and even Finance and senior management as well as other related parts of the business. At this level, the whole business is optimized to deliver on business value, the product roadmap and vision. Many organizations stop here, but on 2nd thought, the optimization is often still based on internal assumptions. The organization delivers those projects and features that it thinks will produce the highest business value and deliver the best business results. (The Product organization is pretty certain it knows exactly what features the customer base needs and works with the development organization to ensure those assumed high-value features are delivered first.) These assumptions may or not be correct.
Only once actual results after release to the customers are measured and fully understood can an organization truly optimize real business outcomes and validate the internal assumptions with external results. That’s where applying principles of the Lean Startup are helpful and allow an organization to release an MVP (minimally viable product), gather systematic feedback and then pivot on the information received if necessary. Only once those features hit production can those assumptions be given the litmus test of real customers and revenue plans be turned into actual dollars.
The bottom line is this: We obviously don’t want to practice Agile just at the team level. Many companies have understood that and strive to transform themselves to Agile organizations in a more holistic manner. While this is great, the true optimization of actual business results comes from optimizing for actual, validated customer outcomes and market adoption. Product life cycle approaches such as the Lean Startup are therefore complementary to Agile and needed to achieve the best possible business results.