Think about the last time your company was faced with an investment in information technology – as an example. Perhaps you were deciding on a CRM platform, an upgrade to your replenishment or logistics software… – you can imagine such a scenario. Regardless of the exact details, you might recall reviewing a business case for this “technology investment”. Perhaps you were even responsible for the development of the business case.
If you can relate to this experience, I would propose some initial exploratory questions to frame our explanation; that is:
- Did you happen to notice if the business case spent more time analyzing costs as opposed to benefits?
- When you dug deeper into the costs, were most, if not all of the costs centered on short-term development estimates with little or no attention to long-term support estimates?
If you made these or similar observations, you are not alone. You, like many of your peers, are making decisions based on business cases that are systematically measuring almost exactly the wrong things – we’ve observed this many times.
Think about this approach from a simple, logical perspective: What do you care more about, costs or benefits? What do you spend more time analyzing? If your answers to those questions are “costs” and “benefits”, respectively, you’ve observed the phenomenon known as the measurement inversion.
Here are some observations we’ve recorded at HDR. Again, staying with our IT example (I will share a variety of further measurement inversion examples we encountered in several other industries.)
 |
||
You might be thinking, “how did we figure out column 3”?
Surprisingly, we did it in an Excel spreadsheet with a built in Monte Carlo simulation, which worked out some trivial math and ran a quick Excel macro to measure what we call the Value of Information Analysis or VIA.
The VIA is based on the Value of Information Formula that basically tells you, “this variable(s) has the biggest impact on the business case”. Moreover, it gives you an estimate of how much (in $) you would be economically justified in measuring further. It is at this point in the process that the measurement inversion comes into play. Here is what we have observed.
By the end of 2015 HDR completed its 94th major risk return analysis (this doesn’t include other projects using other tools and approaches based on the AIE Method). Over the course of these projects we’ve measured things like:
- Drought resilience in the Horn of Africa
- Box office receipts for movies
- The return on Cybersecurity controls
- The risk of a mine flooding
- The value of IT to a major financial institution
- The value of a hospice service as part of a greater health care provider
- Share of wallet for a payment service
- Attributes of an artificial organ including features, market penetration, and adoption rate
- The value of a drug for a pharmaceutical firm
In the majority of cases, what the client thought was important to the measurement turned out to have little or marginal impact on the business case. What did have a major impact? Believe it or not, 3-4 variables were typically met with the following response from the customer: “wow, I never would have guessed that.”
Simply put: We’ve observed that many of the people in charge of producing these business cases assume (wrongly) that certain variables that go into the business case have the highest impact on the return; when, in fact, the exact opposite is often the case. Perhaps these variables, like development costs, are easier to measure than say, adoption rate or max penetration.
