Introduction
Over the past decade, agile has become the cure-all to improve software delivery. Agile adoption is now almost universal: “15th Annual State of Agile Survey” findings indicate significant growth in Agile adoption within software development teams, going from 37% in 2020 to 86% in 2021.
Even with agile adoption at an all-time high, there’s a value gap: Most organizations aren’t satisfied with the output they’re getting from agile. This is not to say that agile is failing, but most companies have difficulty communicating the value of agile to their organization.
I’ve observed this among many companies and IT organizations I’ve worked with. IT departments feel like they’re doing a great job. Agile teams are delivering every two weeks. Releases are on time. Velocity is high. Agile is a success!
But when business leaders respond with, “How does any of this help us?” IT departments are baffled. Shouldn’t agile metrics be self-evident of the methodology’s effectiveness?
The answer depends on what you’re measuring and why. This article outlines what agile metrics matter most from a business standpoint and how you can measure agile success, then communicate its value to your organization.
Commonly used Agile metrics and measurements
There was a time when agile was a huge selling point. However, many “agile” organizations see agile as a set of checklists, sticky notes, time-tracking tools, and procedures. To these organizations, agile is a toolbox rather than a mindset. This is the primary reason “agile” product development fails.
In the aforementioned “State of Agile Survey,” respondents listed common ways in which they measured agile success:
1. On-time or speed of delivery
Agile implies that your schedule and scope are fixed for the time being. “On time” doesn’t necessarily mean that what you deliver will meet stakeholders’ expectations. So, to measure on-time delivery, we need to have some context — precisely a clear picture of what is being delivered. To visualize these metrics, you can use the burndown and the burnup charts. A burndown chart demonstrates how much work is remaining, whereas a burnup chart shows the amount of work already completed.
2. Product Quality
One way to measure the quality of your product is tracking customer satisfaction, steady revenue growth, and testing success throughout the whole development cycle since continuous testing is closely intertwined with code health. While working with agile software development teams, it’s important to look at the velocity of completing working software with quality built in.
3. Project visibility
Transparency is one of the best ways to build trust across the team and the rest of the stakeholders. In the project pipeline, transparency means providing plans ahead of actions available to everyone, and consistently tracking and presenting progress and results. Visibility brings alignment among internal teams and reflects the impact each team or a team member had in the given time frame. Not to mention that sharing progress helps stakeholders make informed decisions and identify possible risks.
4. Productivity
It’s important to remember this metric measures outcome, not output. This is why evaluating a burnup chart current metrics for a product or based on value is hugely impactful. Simply following closely the count of stories or features over time is a great way to gain insights into how much the team is actually delivering.
5. Predictability
This agile metric that allows assessing predictability is the velocity trend. In a given time, you can see how much work has been completed at a sustainable pace on average. If this metric fluctuates wildly, it may be a mere reflection of team and scope changes, unpredictable events happening, or simply indicating that the team is not mature enough to define the amount of work small enough to complete in the given time.
Each of the above are great metrics for measuring agile. What they don’t measure is business value. The number of tickets you’ve moved through your project management system is irrelevant if the result is a product no one will use.
How to measure value in Agile
A more productive way to measure agile is to focus on (and measure) the value agile can bring you. This is not to say that agile development metrics like velocity and quality should not be measured—far from it. These data points are useful but shouldn’t be the end goal.
So how do we measure value? The following metrics, also listed in the survey, are closer to the mark:
Customer/User Satisfaction
The metrics to assess customer/user may include a lot of indicators, like sales numbers and growth, the number of support calls vs. the 9, 2020 of features delivered in a time period, or product usage statistics. It differs from product to product.
Business Value
Agile Manifesto recognizes the importance of delivering business value. As we know, there’s a certain amount of work to complete, and also compliance requirements and fines if we don’t finish the work on the promised time. But is this definition true for every product? Sometimes measuring value has to be viewed from a subjective, since the market drives decisions and the value is often the best guess. If we were to draw a formula, it would be a business value score applied to the features to be delivered.
Product Scope (Features, Functionalities, Requirements, etc.)
Working in an agile environment arms you with a plethora of instruments to rely on, like the aforementioned burndown and burnout charts, or even kanban boards for good measure. Setting short-term goals and tracking progress not only feels rewarding and keeps the team engaged, but also creates a real-time feedback loop with invaluable information for any team member or manager involved.
To expand on the results of the survey, an effective way to measure the value of agile is to track key performance indicators (KPIs) that align to value. These can include:
- Usage index: What features are people actually using? By analyzing the usage index of different product features, you’re able to measure what’s valuable to your users, an indispensable KPI for business value. By tracking the usage index of features in a production setting, you can prioritize the features that offer the greatest return on investment.
- Innovation rate: Innovation rate measures the capacity of a team to build value-driven features versus non-discretionary work like detect fixes and application support. The more a team is burdened with maintaining existing systems, the less bandwidth it has to innovate, grow, and create a competitive advantage. By reducing an application’s support needs, you can increase a team’s innovation rate and focus on creating new, value-driven features instead of worrying about keeping the lights on.
- On-product index: An on-product index is the measure of time a team spends working on the product. This KPI measures the efficiency of how a team is run and how much they’re contributing to a product’s growth. By reducing unnecessary meetings and other distractions, you can increase a team’s on-product index, which will “buy time” to work on valuable things.
- Installed version index: Similar to the innovation rate KPI, the installed version index measures the number of versions of an application that require support. Each supported version increases the required support needs, thus lowering a team’s ability to focus on value-driven features and innovate.
These agile KPIs allow a product owner to view and control the resources of a team and prioritize value-based features and objectives.
Agile KPIs to Measure Agile Success
A simpler approach to measuring the value of agile is to focus on defining it rather than creating it. At Forte Group, we help align the IT enterprise, business stakeholders, and product owners to define and focus on what we call value-driven development, where business value drives priorities.
For example, you could employ the best and most experienced developers to write code and build an application. However, that application itself doesn’t matter unless it offers value to customers or helps them achieve a goal.
Of course, the definition of “value” can differ from person to person and organization to organization. Therefore, it’s important for organizations to get aligned on what value is and what it means to them before they can begin measuring it.
Seeking alignment is a process that requires deep thought and conversations, as well as a leadership culture that facilitates those conversations. Leaders must understand that business plans can evolve and change, and that thought-provoking questions from project teams usually result in better outcomes for everyone involved.
What is Business Value in Agile
Business value meaning changes in different contexts, but at its core is a compilation of what all stakeholders experience or defines as value. It’s composed of both quantitative and qualitative components, as well as rational and emotional ones. To make it even more confusing, business value can vary over time. And did I mention that it also varies over time?
By analyzing principles of the Agile Manifesto (like “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software”) and the Scrum guide, we can draw a few conclusions that bring us closer to defining business value:
- It’s tied to customer satisfaction;
- It provides direction to your work;
- And nobody measures it (!).
So, How to Define Business Value in Agile for IT Enterprises and Organizations?
Here are some questions to ask that can help:
- What’s the problem or challenge we’re trying to solve?
- What’s the simplest and quickest solution to solving the problem?
- Is this useful to our customers?
- Are users using it?
- How can we validate that what we’re building is the right thing?
- What’s the shortest path to value, both to our customers and to our organization?
- What’s our minimum viable product (MVP)?
So, what exactly do we mean by MVP in this context? Value-driven development defines an MVP as the smallest change the team can make to a product that offers immediate value to end users. Not only does this process eliminate the waste of precious time on developing complex and time-consuming features, but it also helps the team achieve a greater focus on prioritizing and achieving value.
By taking the time to think through and answer these questions, IT organizations and enterprises will find themselves in a better position to plan a release roadmap that maximizes the MVP.
In addition to asking, “Are we delivering value?” you should also ask, “Are we delivering value frequently?”
The Results and Impact of Measuring Agile
Nothing actually measures agility. It’s about identifying and enabling value that’s unique to your organization, then using the right strategy to maximize that value.
If you look at the list of metrics at the beginning of this article as examples of how some IT enterprises and organizations measure agile and adopt your own strategy for measuring agile and value, you’ll begin to see better quality, higher productivity, and happier customers. Measuring agile and delivering value helps IT enterprises and organizations advance, accelerate, and evolve.
If you’re struggling to measure value, it can be helpful to have an outside resource. You can speak with one of our practice engineers about ways to measure the value of agile while evolving into a value-driven development model.
Editor’s note: The article was first published on August 9, 2020, and was updated for relevancy and accuracy on September 8, 2021.
