Originally posted on http://www.howtomeasureanything.com/forums/ on Monday, February 16, 2009 11:32:17 AM.
“I am looking for some material (articles or books) on the subject of Calibration. I want to be expert in Calibration.
Thakur”
I certainly support your goal for becoming and expert in this topic. It is a well-studied topic but is still far too obscure in practical applications. Beyond my book, the most important sources are the purely academic literature…which I would definitely recommend for anyone who wants to be an expert. My next book The Failure of Risk Management, will cover this topic with a slightly different emphasis and, in some cases, in more detail. In both books, I resort to several academic studies, including the following.
A key source is Judgment Under Uncertainty: Heuristics and Biases, Cambridge University Press, 1982. It is a compilation of several research papers on the topic. You can buy this book on Amazon.
Here are several more articles:
A.H. Murphy and R. L. Winker, ‘‘Can Weather Forecasters Formulate Reliable Probability Forecasts of Precipitation and Temperature?,’’ National Weather Digest 2, 1977, 2–9.
D. Kahneman and A. Tversky, ‘‘Subjective Probability: A Judgment of Representativeness,’’ Cognitive Psychology 3, 1972, 430–454.
G.S. Tune, ‘‘Response Preferences: A Review of Some Relevant Literature,’’ Psychological Bulletin 61, 1964, 286–302.
E. Johnson, ‘‘Framing, Probability Distortions and Insurance Decisions,’’ Journal of Risk and Uncertainty 7, 1993, 35.
D. Kahneman and A. Tversky, ‘‘Subjective Probability: A Judgment of Representativeness,’’ Cognitive Psychology 4, 1972, 430–454.
D. Kahneman and A. Tversky, ‘‘On the Psychology of Prediction,’’ Psychological Review 80, 1973, 237–251.
A. Tversky and D. Kahneman, ‘‘The Belief in the ‘Law of Small Numbers,’’’ Psychological Bulletin, 1971.
A. Koriat, S. Lichtenstein, and B. Fischhoff, ‘‘Reasons for Confidence,’’ Journal of Experimental Psychology: Human Learning and Memory 6, 1980, 107–118
Dear Mr. Douglas W. Hubbard
I need something very simple but great material. Similar to what produced by you. Let me first thank you for your great efforts which came out in the form of “How to Measure Anything”. In this book I read the importance of Calibration. As such I need to practive this skill.
I always had the dream to measure the following:
1). The job I do (I am Computer Operator by profession).
2). My personal finance.
3). My personal website http://www.urood.com (Classified web site)
I bought your book from Amazon with hope that my above dream will be fulfilled.
Thanks
Hi,
I’m about half way through a very interesting book that seems to go quite deep into expert opinions, calibration, etc. Included in the book are many practice oriented examples as well as the mathematics leading to and used in deriving distributions from experts.
(In the most part I find it a lot more readable than “Judgment Under Uncertainty: Heuristics and Biases”…)
Author: Roger M. Cooke
Title: Experts in Uncertainty (Opinion and Subjective Probability in Science)
Publisher: Oxford (1991)
ISBN: 0195064658
Best regards, Les
I have collaborated with Roger Cooke and have also used his software that assists in expert calibration. It’s quite helpful in that in addition to handling experts’ interval estimates, it implements a consistent, repeatable, optimal method of combining a bunch of different experts’ interval estimates by weighting their respective estimates according to their performance on known seed questions (best calibrated). It’s called Excalibur.