The experts who were confident, and mostly wrong

Over about two decades, psychologist Philip Tetlock asked 284 people who advised on or commented about politics and economics to give probabilities for future events. By 2003 he had collected 82,361 forecasts. He published the results in 2005 in the book Expert Political Judgment.

He scored the forecasts against what really happened, and against simple statistical rules and non-experts. The average expert did only a little better than random guessing, and slightly worse than simple rules that predicted more of the same. Qualifications and access to classified information did not separate the better forecasters from the worse.

What did matter was thinking style. Borrowing an image from the philosopher Isaiah Berlin, Tetlock called some experts hedgehogs: they knew one big idea and applied it everywhere, with great confidence. Others were foxes: they drew on many sources, were comfortable with doubt and changed their minds more readily. The foxes forecast better.

From 2011, Tetlock and Barbara Mellers led the Good Judgment Project in a forecasting tournament run by IARPA, a US intelligence research agency. Their team of volunteers beat the four rival research teams. Training in probability, working in teams and grouping the top 2% into elite teams all made forecasts more accurate and better calibrated.

The study separated two things people usually mix up: sounding sure and being right. A forecaster is well calibrated when the things they call 70% likely happen about 70% of the time. Many confident experts were not.

Chatbots have the same gap, often wider. A language model writes in a fluent, assured voice whether it is right or wrong, because confident text is what it learned to produce. Its tone tells you about its writing, not about its accuracy.

Tetlock's good forecasters suggest a better way to use AI: ask for probabilities and reasons, compare several views, update when new facts arrive, and keep score of how often the tool is right on questions you can check.

The idea behind this caseLesson: Why AI makes things up →

1. 1Ask the AI how confident it is and why, and what would change its view, instead of accepting a flat answer.
2. 2Keep a simple record of how often a tool's confident answers turn out right on things you can check.
3. 3Compare answers from more than one source, or ask the same question in different ways, before acting on a prediction.
4. 4Prefer answers that admit uncertainty and give a range over ones that sound certain about an uncertain future.
5. 5Revisit important AI-based forecasts when new information arrives, as a fox would.