Originally posted on http://www.howtomeasureanything.com/forums/ on Wednesday, February 18, 2009 4:54:25 PM, by RossShott.
“Hello Mr. Hubbard,
Your book “How to Measure Anything” is remarkable!
I am currently completing my Master of Technology in Futures Studies degree by working on my thesis project. The project entails fleshing out a Foresight Maturity Model (similar to a CMM or CMMI) recently developed by Social Technologies. Meaning that I have been given the task of figuring out how to measure foresight within companies in such a way that those companies can be evaluated by a Futurist consulting firm using the FMM tool or the company could apply the FMM tool to their own organization for foresight improvement efforts. How do you measure foresight capability I wondered? I wondered even more about how do you measure the value of improved foresight? (By foresight I mean having a better ability to forecast alternative futures for robust decision-making and putting in place practices which will enable you to better prepare for changes and unexpected events in the future).
I searched the library and the internet before purchasing a handful of books on measurement and experiment design. Your book is a diamond among a sea of broken glass. “How to Measure Anything” is so well written I could barely put it down. I read it straight through (including doing the calibration exercises) in two nights after work and began reading it a second time with highlighter in hand.
I can’t thank you enough for the incredible insights and inspiration you have given me! Your book with go to the top of the stack as I begin my Ph.D. in this Fall.
Best regards,
Ross Shott
Graduate Student, University of Houston”
Ross,
Thanks for the kind endorsement and thanks for bringing this conversation over from the email conversation that we started.
My first question for you is the same one with which I begin all measurement problems: “What do you mean by foresight, exactly?” If you can think about this question and those below, I’ll get closer to solving your problem.
But while you consider that, let me propose a possible meaning and you can modify it further. I suppose you might mean the ability to predict particular events. Specifically, foresight might have more to do with forecasting trends like “What is the next big bio-tech revolution?” instead of “Will it rain tomorrow?”. But correct me if you mean something else.
For now, lets suppose you mean what I suggested and consider what its value might be. Presumably, foresight has value because of the effect it would have on your decisions and behaviors. As I ask in the book, what would you imagine you would be doing differently if you had more foresight? Imagine a list of specific events and ask what would be the value of acting on it.
Thinking about these specific events will help us focus on the right question. Tell me as much as you can about this application and let generate some specific examples of “improved foresight” and, then, some specific examples of what you would be able to do better if you had that foresight. Once we have that, let’s follow the path as shown in the book:
1) Decompose it – Think of a list of specific events you would have foresight of, a list of specific actions you might take if you had such foresight, and the specific advantages of those actions. Think of how all these might come together into a formula that computes the value of foresight.
2) State your current uncertainty – for each of the decomposed items in step #1, state what you know now about it. Estimate how often these events might occur. Estimate the possible actions and estimate the impacts of good and bad outcomes.
3) Compute the value of information for each of the variables – this requires that you think of the costs and benefits of some particular alternative you might take differently if you had more information about the value of foresight. Are you considering a specific software purchase that would allegedly improve foresight? Then that is the decision this information would help with.
4) Design the empirical method – for the variable(s) with the highest information value, think about how you would observe it. Then, based on the possible observation, design a systematic observation based, say, on random sampling or controlled experiments.
This sounds rather vague right now, but we can get more specific if you provide more specifics about the problem.
Thanks,
Doug Hubbard
Hello Mr. Hubbard,
Your definition of foresight is a good one. In addition to the concept you propose, another application is similar to a portion of your definition of measurement in your book “a set of observations that reduce uncertainty.” Obviously the future is unknowable and we don’t have “observations” yet, but there are many things about the past and present which may (when viewed in the proper context) help one reduce some of the uncertainty about the future.
In Futures Studies (Strategic Foresight) we view the future as a multiplicity. Instead of focusing on a single prediction we attempt to surmise a set of plausible and probable futures and then help set a course for an enterprise’s preferred future in light of this understanding. Futures Studies practices also tend to look long term (5-10 years) as opposed to quarterly or yearly. As you can imagine there has been some difficulty in measuring the dollar value of this foresight.
So the valuation of foresight is needed in both circumstances: forecasting a range of probable futures (i.e. maybe some aspects of the “big bio-tech revolution” you suggest) and reducing uncertainty about other futures (i.e. the likelihood of a second cold-war with Russia) with the intent of improving decision making. For a particular venture the data gathering and forecasting would be focused, of course, but still have multiple scenarios. The purpose of placing a value on this knowledge is to create a basis for the enterprise hiring a futurist consultant to perform the study or to determine if establishing (or improving) permanent foresight practices in the organization would produce a reasonable ROI.
I don’t have a specific case in mind for the project I am working on. Social Technologies developed a Foresight Maturity Model (FMM) which can be used to measure the current foresight practices within an organization (6 Disciplines with various Practices rated on a 1-5 scale). If I can figure out a way to measure the value of foresight to a particular enterprise, and then measure its current foresight practices it would be possible to estimate the dollar value of improving those practices (or justification for applying the model in general).
Can we work with the above to construct a valuation method or would you like to work with a specific case such as the future of Best Buy? If developing foresight for Best Buy we would be interested in the long-term changes in retailing in general (i.e. e-commerce, cashless society, etc.), technology for its various electronics, and copyright for its CD and DVD line.
Your reply so far has been great. Thanks!