3 Metrics for Reporting an Agile Pilot to Top Management

During the introduction of Scrum in an enterprise, e.g. via an Agile pilot, the question of how to report to the upper or top management comes up. I’d suggest to use these three metrics as a starting point:

  1. customer feedback
  2. release burnup
  3. happy index

Here are the details and the reasons for my choice:

1. Customer Feedback

You can have the highest velocity and produce a huge amount of software, but you won’t be happy if you can’t sell it. To sell it, you need customers who want to buy your software. They want to buy your software, if you provide what they need. Then they are satisfied and will give you thumbs up feedback, directly (surveys, comments, etc.) or indirectly (cash, click rate, etc.) Therefore you have to track customer feedback.

Customer feedback depends on the product type you build. Examples are:

  • conversion metrics (like e.g. the AARRR metrics from the Lean Startup movement)
  • number of customers who bought the product
  • number of customers who recommend your product
  • number of bugs
  • level of customer satisfaction (e.g. via a survey after every sprint review)

2. Release Burnup

Before using a burnup, let’s have a look at a burndown. Make the amount of work not done reportable with a release burndown. That line is plotted on a chart with the number of story or epic points on the y-axis and the sprint number on the x-axis. Where the line crosses the x-axis, that’s the sprint when you’ll run out of stories. If you want to have the line cross the x-axis at a certain date in the future, make sure to modify the content of your product backlog. That means, get rid of unnecessary stories or stories with low value. You might want to split stories to do so.

Getting rid of work leads to simplicity:

Simplicity–the art of maximizing the amount of work not done–is essential. — principle of the Agile Manifesto

Example of a Release Burnup

A product owner has to work hard to make her product simple. That’s worth to be reported to the upper management. Therefore, a better alternative to a release burndown might be a release burnup.

A release burnup shows two lines on the same axes like the burndown chart. One line plots the total content of a product backlog in story points, the other line shows the amount of work done in story points (aka velocity). Where those two lines will meet, that’s when you’ll run out of work. If you don’t like that date, you can increase the velocity (increases the gradient of the work done curve), or you can prune your product backlog (decreases the gradient of the product backlog curve).

3. Happy Index

Make the team’s mood reportable by tracking a happy index. Therefore, encourage your team to track their moods via a niko-niko-calendar. For each week, calculate the team’s average mood. Plot the average mood in a chart, where the y-achsis is the happy index scale and the x-achsis is the time in weeks. Over time you’ll get a pretty good history over the team’s mood.

Example of Happy Index

Self-similarity to the rescue regarding scaling: You can track the average mood of departments if you calculate the average of the teams’ moods. Likewise, you can track the average mood of the whole company if you calculate the average of the departments’ moods. The company’s happy index, sounds good?

You can interpret the curve of a happy index like a stock price index, where the stocks are those of your work as an Agile coach or manager. If the stock price index turns up, your work as a coach or manager is good and your latest changes improved the working environment to the better for your teams. If the stock price turns down, you’re work as a coach or manager could be improved, and your latest changes weren’t succesful in the eyes of your people.

Why those 3 Metrics?

Every one of those metrics addresses a totally different area of interest when you start an Agile Pilot.

Model of Metrics for Reporting an Agile Pilot to Top Management
  • Customer feedback reflects the economical and emotional view to the outside of the product your Agile pilot produces.
  • The release burnup reflects the economical view to the inside of the product your Agile pilot produces.
  • The happy index reflects the emotional view of the inside of the team which works in your Agile pilot.

With those three metrics, you cover the most important views of an Agile Pilot from a manager point of view. Even better, those metrics are explained rather quickly, which means they are top management compatible.

This entry was posted in General and tagged , , . Bookmark the permalink.

About Bernd Schiffer

Bernd Schiffer is consultant, trainer and coach for Agile Software Development in Melbourne, Australia. Learn more about him on his personal homepage, have a look at his company Bold Mover, or contact him on Twitter, Google+, Facebook, XING or LinkedIn.

7 Responses to 3 Metrics for Reporting an Agile Pilot to Top Management

  1. Yves Stalgies says:

    Den Happiness-Index finde ich sehr gut. Wir machen etwas ähnliches mit einer Quadranten-Messung. Wir fragen auf der x-Achse nach Spass und auf der y-Achse nach Stress. Das bringt interessante Erkenntnisse über die Stimmung hinaus. Das ist in Anlehnung an Ed Yourdons Quadrantenmessung, bei der er die Stimmung gegen die gefühlten Erfolgsaussichten aufträgt (http://books.google.de/books?id=FdAZUX9H_gAC&pg=PA52&lpg=PA52&dq=ed+yourdon+quadranten&source=bl&ots=5ETm6x9Ieg&sig=yNVA68pGuyhxChTci0LIIr6Ovds&hl=de&ei=fFt3TtPeLoTJswaw1omHCw&sa=X&oi=book_result&ct=result&resnum=6&ved=0CFAQ6AEwBQ#v=onepage&q&f=false).
    Generell gibt es sehr interessante Parameter-Kombinationen für derartige Quadranten.

    • Yves wrote: “I like the happiness index. We do something similar with quadrant measurement. We ask on the x-axis about fun, and on the y-axis about stress. That reveals interesting insights about the mood. This method is according to Ed Yourdon’s quadrant measurement, where he plots mood agains prospect of success (http://bit.ly/qsiFA9). In general there are very interesting combinations of parameters for those kind of quadrants.

      Yves, thanks for your comment. Would like to see those charts. If you don’t mind, please email me a photo.

  2. Pingback: The Three Signs of a Miserable Scrum implementation | Markus Gärtner

  3. David Toon says:

    “If you don’t like that date, you can increase the velocity (increases the gradient of the work done curve), or you can prune your product backlog (decreases the gradient of the product backlog curve).”

    Seems a bit of a strange comment – how do you increase velocity? Velocity stabilises over 2/3 sprints from my experience.

    Surely scope is the only thing to cut? Unless you are going to short cut on quality?

    • I agree that cutting scope is, in most cases in this context, a very good way to go. However, I do not agree that it’s the only one.

      Velocity will inevitably be increased by continuous improvement, which is something the whole Scrum team should strive for all the time (cf. 12th principles of the Agile Manifesto: “At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.“). After all, sprint retrospectives are part of the Scrum framework for exactly that reason: to improve the process.

  4. Pingback: Sprint Review, a Feedback Gathering Event: 17 Questions and 8 Techniques » Agile Trail

  5. Alison says:

    My struggle with the release burnup is that it means we are story pointing work more than 1-2 sprints ahead of time, which means possibly spending time talking about stories that will never hit a sprint backlog if priorities change. Additionally, I think you have to be very cautious when showing management a release burnup. If you are in the midst of a transformation and managers are not 100% on board with (or bought into) the transformation, this could become a point of contention if the team doesn’t deliver “on time” according to the release burnup.

    How do you solve for not planning too far in advance?

Leave a Reply

Your email address will not be published. Required fields are marked *