Philip Tetlock – Ignore confident forecasters

But surely (and yes, it is ironic) confidence isn’t the marker to avoid: relentless adherence to inflexible dogma is. Even the foxes are confident in their predictions sometimes, but that is because they have evaluated the alternatives, not because they’ve run it through their particular pet equation and gotten “The Answer”.

[...] Philip Tetlock recently presented a Seminar About Long-Term Thinking to the effect that confident forecasters ought to be ignored. Still, it’s distressingly rare to find even a moderately systematic evaluation of political forecasts in the popular media. In this category comes an article from the online version of Radar magazine earlier this year, “The Iraq Gamble”, which scrutinises journalistic commentators’ predictions about the war in Iraq. In it, writer Jebediah Reed tracks the fortunes of predictions against those of the predictors, and is disturbed by what turns up. [M]aybe something is amiss in the world of punditry. Are the incentives well-aligned? Surely those who warned us not to invade Iraq have been recognized and rewarded, and those who pushed for this disaster face tattered credibility and waning career prospects. Could it be any other way in America? … So we selected the four pundits who were in our judgment the most influentially and disturbingly misguided in their pro-war arguments and the four who were most prescient and forceful in their opposition. … Then we did a career check … and found that something is rotten in the fourth estate. [...]

[...] enn rever i gjennomsnitt, skyldes det prisen de betaler for å ta sjanser – de kommer gjerne med mange spådommer som aldri slår til: The value of Hedgehogs is that they occasionally get right the farthest-out predictions— civil [...]

Tetlock’s work provides a foundation to what many of us have believed for some time. Were he to have left it there, his achievement would be noteworthy. His research and analysis proffers refinements to the major conclusion of fox vs hedgehog.

My concern for his work is it is ignored. Even those who taut its merits are inclined to forget the conclusion and applicability to their own behavior.

Tetlock’swork is germane to our freedom and security … economic, homeland and national. The WMD conclusion is a nightmare that consumed our youth, our riches and reputation … and much more. Had our intelligence community practiced less the expert and more the fox, maybe the conclusion would have been different and/or the courage to challenge existed.

Tetlock’s work screams out the need for asymmetric views and counter measures. But the bias in our thinking pulls our center of gravity to our hedgehog position. Illustrations are the Isreli-centric thinking in our intelligence community , dogma in financial fraud detection, and resistance to carbon emission reduction by business

The Iraq coda at the end now seems oddly out of date and off target.

[...] The Long Now Blog » Blog Archive » Philip Tetlock – Ignore confident forecasters. [...]

[...] I was just trying to summarize a podcast I listened to. A probably better summary can be found here. And if you look around just at bit you should be able to download the podcast as well. Posted [...]