Thoughts on Outcome Based Metrics

bigstock_Budget_3049827Chris Matts recently wrote a post about Time to Value as an Outcome based Process Metric . I have some thoughts about the topic and decided to write the up here. I could have recorded a comment, but based on past history with my comments on Chris’ post (he accidentally deleted it), I decided to put my thoughts here for safe keeping. Plus, my thoughts may end up being a little lengthy, so it seemed better to record them here.

In the post, Chris suggested three outcome based metrics:

  • A quality metric – i.e. number of defects escaping to production
  • A value metric – profit, revenue, number of customers, or any other SMART Goal
  • A time to value metric – the main topic of Chris’ post.

Cycle time is a good way to capture time to value, however the tricky part is figuring out when the clock starts (i.e. when does the cycle begin). As Chris points out, in projects using iterations, the clock is typically started at the beginning of the iteration in which the team chooses to deliver a given item. This measure gives a good feel for the timing of delivery investment, but it ignores any analysis work that is done. This also ignores if there is a long queue that occurs between when an item is initially identified and when it is actually delivered. For example, someone could identify a need in January, place it in a backlog, and not see the results for several months depending on the length of the backlog and where the item falls on the priority list.

Ignoring this timeframe could mask an issue – that ideas linger for quite a while before becoming realized. This occurs frequently in internal development efforts, where the work is often organized as projects and a set of items are all related to a change needed to support a change in business process. Based on what Chris suggests, the longer the period between when an item is identified via even a small investment in analysis, the more risk the effort incurs. This is, in effect, an argument for not doing even a little bit of analysis before it’s time, which can be a tricky balancing act to figure out the appropriate amount of analysis to understand the various chunks of value. Many organizations still want to organize their work in projects, so the question remains – when do we start the clock? I think there’s validity to starting the clock shortly after an item is identified so that you see the long time the item spends in the queue in some cases. If nothing else, this may provide some ammunition for changing the way organizations approach work into smaller bits.

As with Chris, I’d be interested in hearing other’s experience and thoughts on the notes above.

Comments

  1. Chris Matts says:

    Hi Kent

    I’d thought I’d provide you with the opportunity to delete one of my comments this time.

    You are spot on of course. My suggestion for the use of duration is to measure the time between investing and getting a return. The advantage of duration is that it can be used to show an organisations improvement at developing ideas once they have decided to “exercise the option”

    As you say, it is not the only metric that should be used. You also need to consider the time from the initial idea to market release.

    Duration should be used as a metric to show how effective the organisation is at investing. This does not highlight the opportunity cost due to be under-resourced. An appropriate measure might be some form of cost of delay calculation. It also does not highlight how long it takes an organisation to take an initial idea and turn it into a viable investment worth pursuing.

    I think we need to think about process metric a lot more over the coming years.

    Thanks again Kent

    Regards

    Chris

    p.s. I saved a copy of this comment… just in case. ;-)

  2. Chris Matts says:

    Hi Kent

    I’d thought I’d provide you with the opportunity to delete one of my comments this time.

    You are spot on of course. My suggestion for the use of duration is to measure the time between investing and getting a return. The advantage of duration is that it can be used to show an organisations improvement at developing ideas once they have decided to “exercise the option”

    As you say, it is not the only metric that should be used. You also need to consider the time from the initial idea to market release.

    Duration should be used as a metric to show how effective the organisation is at investing. This does not highlight the opportunity cost due to be under-resourced. An appropriate measure might be some form of cost of delay calculation. It also does not highlight how long it takes an organisation to take an initial idea and turn it into a viable investment worth pursuing.

    I think we need to think about process metric a lot more over the coming years.

    Thanks again Kent

    Regards

    Chris

    p.s. I saved a copy of this comment… just in case. ;-)

  3. I am reminded of something I read recently about business managers wanting _their_ projects to start sooner. In truth, no one much cares when the project starts. What everyone cares about is when the project will finish!

    If you have more projects than teams, adding another project to the pipeline doesn’t help. We know, from experience and studies, that simply adding more work is a hinderance. With too much work on a team’s plate they will get less done, not more. I think we need a better understanding of the end results of potential projects. And based on expected impact, and how likely we are to actually get those results(!!), we can pick and sequence projects. Otherwise, we are spending too much time working around projects rather than delivering value.

    All of which tells me we need a lightweight means to determine if we have delivered enough value on the current project and how to compare value between projects.

Speak Your Mind

*