Jory MacKay on November 17, 2021 Understanding Agile Metrics: How to Use Burndown Charts, Velocity Charts, and More!
Metrics and data can be a touchy subject for Agile teams. In the best-case scenario, Agile metrics like sprint burndown charts, velocity, and cumulative flow diagrams are used to reduce confusion and show team progress (or bottlenecks).
However, data can also just as easily be misused to justify scope increases, mandatory weekend work, or reduced resources in the wrong hands. So it’s no wonder that so many technical teams view agile metrics with a healthy dose of skepticism.
Yet, this does a disservice to the actual value they provide.
Jump to a section:
When you understand the purpose behind your Agile metrics, pick the right ones for your team, and make reporting an integral part of your project management process, it keeps everyone informed, focused, and motivated.
In this guide, we’ll dive into the real reasons to track agile metrics, how to create some of the most common (and valuable) reports, and show how to strike a balance between being data-informed and data-driven.
Agile metrics 101: What data is most important to your team (and your goals)?
Let’s start with the basics: what are agile metrics, and which ones are important to your team?
Agile metrics are data points that help you measure your team’s productivity, progress, and effectiveness across the entire project lifecycle.
When used properly, agile metrics become part of a feedback loop that helps measure work quality, predict future resource needs, and even protect against scope creep.
At its core, Agile is built on the idea of continuous improvement. Your team builds and ships software, gathers feedback from real users, and then integrates it into future sprints to make even better software.
However, the best agile teams know this same iterative principle can apply to how their team works.
Agile metrics are the feedback component of this loop. Your team works through tasks, gathers data about their productivity, effectiveness, and output, and then integrates those insights into their processes, workflows, and sprint planning.
If you can combine the two–continuous product and self-improvement–you’ll have your own band of unstoppable Agile Argonauts.
So, where do you start when it comes to using agile metrics?
Agile metrics cover three main areas with different purposes:
- Lean metrics: Lean metrics focus on minimizing waste and continually shipping quality software to your users. Some examples include lead time and cycle time.
- Kanban metrics: Kanban metrics focus on your workflow, prioritizing tasks, and getting work done. An example would be cumulative flow.
- Scrum metrics: Scrum metrics focus on team productivity, consistency, and predictability. Some examples include burndown charts and velocity.
Don’t worry if you’re unfamiliar with most of the terms we just used. The idea is to understand the purpose of using data in project management before diving into the actual charts and reports that illustrate them.
5 powerful Agile reports you should be using, why they’re important, and how to make them
As a project manager, you might not be totally comfortable analyzing data. And that’s OK. Most project management training focuses on processes and workflows over data analysis for a good reason.
Project management is about both people and processes. You can’t be a successful project manager without connecting with the people on your team and understanding the context of why certain things happened.
On the other hand, data is almost always devoid of context. It feels wrong or misleading to remove all the different moving pieces and only focus on a single data point.
That’s why the easiest first step to using agile metrics is to connect what you do know (processes, workflows, and the people on your team) to what you don’t (data, charts, reports, and confusing terminology).
Charts and graphs are only confusing until you realize they represent what you deal with every day. Then, they become visualizations and insights of the fundamental project management skills you already have, like time management and task management, planning, risk mitigation, and quality management.
That’s how we’re going to approach agile reports–as purposeful and valuable representations of your team’s work. Here are five of them you should know, why they’re important, and how you can make them in your project management tool of choice.
Charts and graphs are only confusing until you realize they represent what you deal with every day.
Before you dive into the reports: How you interpret each report can significantly impact your team’s motivation level and how they’ll trust data in the future.
For each of the agile reports and charts below, we’ve added a few antipatterns and common mistakes to watch out for. Hopefully, this will give you a better idea of what you can get out of each and how you can use them more productively.
1. Burndown charts
What is the purpose of a burndown chart?
A burndown chart shows the amount of work that has been completed in a time-boxed sprint (or epic) and how much work remains.
The quantity of work (or effort) left is represented by the line labelled "actual", while time is displayed on the horizontal axis. The "ideal" line represents the ideally spread closing of issues within the chosen time frame and takes your non-working days into account.
Ideally, your "actual" line stays underneath or at least close to the "ideal" line. It should look like a diagonal line where you start with a number of tasks and then over the course of the sprint, work through them all.
The benefits of burndown charts
Burndown charts are simple graphical tools that give everyone a clear view of how the project is progressing. However, they have several other benefits that make them powerful tools for project managers. Most importantly, burndown charts help you to:
- Forecast if the sprint will be completed on time
- See the impact of adding feature requests or increasing the scope
- Track your actual effort against an ideal situation
- Get rid of wishful thinking around deadlines and makes releases more predictable
How to make a sprint burndown chart (and every other agile report)
Burndown charts are typically used to view how many issues are left to complete in your current sprint. However, you can also use them to view hours or even story points.
In Planio, start by clicking on the black sidebar on the right-hand side and choosing agile charts.
Then, you’ll be able to select which chart you want to view and any filters on your data (such as during a specific time frame or any other custom filter you want).
In our case, we’ll want to choose burndown chart and choose a time filter that covers our current sprint.
Finally, pick the units that you want to track: issues, story points, or hours.
This is the same process for creating and viewing all other agile reports in Planio: Burndown, burnup, cumulative flow, velocity, lead time, average lead time, and trackers cumulative flow.
All you need to do is switch the selected report, update your metrics (if applicable) and choose a filter. This makes it super easy to look through charts and find the information you need quickly.
Common mistakes and anti-patterns to watch out for
A burndown chart is a fantastic tool for quickly seeing how your project is progressing. However, it has limitations when it comes to context and knowing why you’re not moving the way you want to.
Here are a few anti-patterns to look for in a burndown chart:
- The team consistently finishes early. In this case, you might not be committing to enough work. On the other hand, if you keep missing the deadline, you might be committing to too much.
- The burndown line makes a steep drop. A burndown line should decrease gradually. If it isn’t, you might not have broken down the work into granular tasks.
2. Issue burn up charts
What is the purpose of a burnup chart?
A burnup chart is similar to a burndown chart in that it visually tracks your team’s progress during a sprint or epic.
However, a burnup chart differs in that it presents two lines that represent your total work created (or scope) alongside your progress. Your project essentially starts at zero and adds tasks or issues until your work crosses the scope threshold.
The benefits of a burnup chart
Burnup charts are useful in that they give you a quick view of your scope and how/if it’s changed. There’s no hiding additional requests or scope creep as it’s clearly displayed as a line across the top of your chart.
Beyond that, there are a few essential benefits to using a burnup chart:
- Instantly see if you’re deviating from your project plan or scope of work
- Clearly present project progress on a regular basis (in a way that’s easier to see than on a burndown chart)
- Simpler visualization of progress towards a release for reporting to stakeholders
Common mistakes and antipatterns to watch out for
Burnup charts are quite simple to understand. However, there are still a few common mistakes you can make when using them:
- Using a burnup chart to justify a project that should be adjusted or dropped. A burnup chart is an easier way to show how much work has been done. This makes it a good tool for justifying projects to clients or stakeholders. However, it doesn’t say anything about whether a project should continue. Don’t let the sunk cost fallacy of a burnup chart inform your decisions.
- Only focusing on a consistent scope. Yes, we all know that scope creep is the bane of every software team. But the ultimate goal should always be to ship valuable software, not to maintain a steady scope no matter what.
3. Cumulative flow charts
What is the purpose of a cumulative flow chart?
A cumulative flow chart (also known as a cumulative flow diagram or CFD) shows how issues and tasks ‘flow’ through different states–from new to closed.
Think of it as a chart version of your Kanban board. The vertical axis displays the number of issues, while the horizontal axis shows time. Each band of color represents a lane on your board, while the width of the band represents the number of cards in that lane.
Ideally, you’ll want to see a consistent or ‘smooth’ flow of tasks as they go through each state.
The benefits of a cumulative flow chart
Cumulative flow charts can look complex, but they help answer a few simple questions, such as:
- Are issues reaching their final state (resolved/closed?)
- How long does it take you to go from a new issue to value?
- Are you hitting bottlenecks or getting stuck in a particular task state?
- Is your work-in-progress increasing or decreasing?
Common mistakes and antipatterns to watch out for
A cumulative flow chart is especially helpful for teams working to tight deadlines who need to see where bottlenecks are forming quickly. However, a few simple mistakes can skew the data and make it unclear.
Here are a few cumulative flow chart anti-patterns and mistakes to watch out for:
- The backlog keeps growing. If you’re not regularly pruning your backlog or removing obsolete or low priority issues, the ‘new’ issue band will look more bloated than it should be.
- Large backups or ‘starvations.’ Blocking issues can cause weird behavior in your ‘flow’ from backups in certain parts to starvations in others where more work can’t be completed.
4. Velocity chart
What is the purpose of a velocity chart?
A velocity chart displays the average amount of work your team is completing during a sprint or iteration. It can be measured either in issues, story points, or hours.
Once again, the vertical axis displays how you measure effort and work (story points, issues, etc.). While the horizontal display shows time. For each iteration or sprint, you’ll see the amount of work committed to (created) mapped out against how much was completed.
The benefits of velocity charts
Velocity charts are one of your best tools for forecasting. As a project manager, you can look at past sprints and see how quickly your team worked through the backlog.
For example, let’s say your team wants to get through 25 issues to hit a milestone. If you look through past iterations and see that you average five issues per sprint, you know it should take about five sprints to hit your goal.
However, beyond forecasting, there are several other benefits to using velocity charts:
- Track new teams as they ramp up work
- Help mature teams monitor consistent output
- Identify when parts of your development process have become inefficient
Common mistakes and antipatterns to watch out for
Velocity charts feel straightforward. However, there are some small yet critical errors you can make when using them.
Here are a few velocity chart anti-patterns and mistakes to watch out for:
- Making assumptions when velocity is erratic over time. Make sure you understand the context of why velocity is changing before making any decisions. During a sprint retrospective, ask your team if there were any unforeseen development challenges, if outside pressure is pushing them to work beyond their limits, or if any other factors have impacted their ability to get through tasks.
- Comparing velocity across teams (without talking about estimation culture). Each team approaches estimation differently. As such, they’ll have a unique velocity. It’s unfair to judge Team A as ‘worse’ than Team B because they have a lower average velocity.
5. Lead time charts
What is the purpose of a lead time chart?
A lead time chart shows the time it takes for work to be completed after it’s been requested. In other words, how long does it take for an issue to go from open to closed?
A lead time chart skips the context and nuance of other agile charts and instead just focuses on value. It allows you to see how your customers perceive the work you’re doing (as the first time they’ll experience your work is after it’s been shipped).
The benefits of lead time charts
Agile teams are built around continuously shipping valuable software. So lead time charts are powerful tools to see if they’re meeting expectations–both their own and the customers.
Improving the lead time chart means you’re shipping more value to customers. However, there are a couple of other reasons why you should be using lead time charts:
- Determine the impact of any changes you make to process, sprint planning, or goal setting
- Helps you become more predictable by providing an average timeframe for completing issues
- Measure how agile you are as an organization (are you consistently shipping value? Or are certain features and requests getting stuck in backlog limbo?
Common mistakes and antipatterns to watch out for
Lead time charts are great for tracking your team’s output from a customer’s perspective. However, a high lead time can often be misread as a team issue when it really comes down to other factors.
Here are a few common lead time anti-patterns and mistakes to watch out for:
- Releases are getting held up by approvals. Getting sign-off for new features can significantly impact your lead time, especially when you’re working with multiple stakeholders.
- You’re stuck with inefficient testing processes or software deployments. Beyond approvals, other processes could be making your lead time look worse than it is. Check-in on your testing process or how you handle hand-offs and deployment.
- Your backlog isn’t being processed regularly enough. New tasks almost always end up on your backlog. But if you’re not regularly processing how those tasks progress through your software development process, they can get sidelined and end up skewing your lead time.
Putting Agile metrics to work: The delicate balance between being data-informed and data-driven
There’s a difference between being a project manager who collects data and makes reports and one who uses them to motivate and empower their team.
To paraphrase one of the most-repeated lines in comic book history, with great data comes great responsibility.
Don’t let yourself get sucked into becoming solely data-driven and forgetting the people behind each metric. Instead, use these best practices to be data-informed and make the most of your agile reports without losing context and the personal connection.
1. Know how you measure effort: story points vs. time vs. issues
Many Agile reports we looked at rely on properly estimating how much effort they should take to complete. However, as we all know, estimating can be tricky.
Agile coach Mike Cohn uses the example of two runners talking about how long a 5km race should take to run. One says 25 minutes, while the other says 45. Yet despite their wildly different paces, they can both agree that 5km is much shorter than a marathon.
Thanks to cognitive biases like the planning fallacy, time can be difficult to forecast. Instead, a different way to approach effort is to use ‘Story points.’
Story points are values you agree on as a team that represent the relative investment of a task.
For example, some teams use the Fibonacci sequence of 0, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, etc… This way, instead of arguing whether a task should have an effort score of 12 or 13, you can quickly agree it’s more of a 13 than an 8 but not a 21-pointer.
You can switch to estimating in story points in Planio by going to Administration → Agile and selecting Story Points as the estimated units.
Story points will show up on your Agile board and issues and can be automatically applied to only tasks associated with sprints (by selecting a specific “tracker” to use story points).
2. Maintain a balance (and a connection) between business and agile metrics
Agile teams need to be aware of both goals related to business intelligence and team goals.
Smashing through your task list and shipping features is pointless if you’re not making the right software for your users.
As you talk about team metrics, don’t forget to connect them to your business goals. For each initiative on your roadmap, make sure to include:
- KPIs: Include several KPIs (Key Performance Indicators) that map to your business goals.
- Success criteria: As part of your definition of done, include success criteria that measure the quality of your release such as the adoption rate by users.
3. Set rules around how and when you communicate metrics
Data should be democratic. If you’re hoarding reports or only sharing them with a select few project stakeholders, you’re doing yourself and your team a disservice.
Instead, make data-sharing a part of your regular updates. Not only does this make the data more actionable, but it also creates more trust in the metrics you’re sharing. Plus, the more regularly you show reports, the more comfortable your team will become talking about them.
Data should be democratic. Make data-sharing a part of your regular updates.
4. Use data to identify risks, not punish your team
Burndown charts can be used to spot scope creep or identify teams that are taking on too much work per sprint. But it’s up to you to interpret the data that way.
Is your team being inefficient with their time? Or are they taking on too much and at risk of burning out? Is your lead time long because of poor planning? Or because your testing process is slow and other teams are batch releasing your work?
Data only tells you what’s happened. Not why it happened. Taking charts and metrics out of context is a quick path to alienating your teammates.
Make data your differentiator (not your downfall)
Data is an incredible tool in the right hands.
Agile reports like burndown charts, cumulative flow diagrams, and others can be misleading if you don’t treat them properly. Instead, don’t forget the people and processes behind the data points you’re seeing.
When you can connect agile metrics to the real world, you’ll be in a much better position to benefit from them.
