The quiz champion that struggled in the clinic

In February 2011, IBM's Watson system beat two of the best human champions on the US quiz show Jeopardy!. IBM then set out to apply Watson to medicine, including a cancer treatment adviser built with doctors at Memorial Sloan Kettering Cancer Center in New York.

In Texas, the MD Anderson Cancer Center ran a separate Watson-based project from 2012. A 2017 audit by the University of Texas System found it had paid more than $62 million to outside firms, mainly IBM and the consultancy PwC. The system was not being used to treat patients. The auditors noted they were not judging the science, only how the project was run.

In July 2018, the health news site STAT reported on internal IBM slides from 2017. They described examples of unsafe and incorrect treatment advice from Watson for Oncology. The slides blamed how it was trained: on a small number of made-up, or synthetic, patient cases, reflecting the views of a few specialists rather than real patient records. IBM defended the product.

In January 2022, IBM agreed to sell the data and analytics assets of Watson Health to the investment firm Francisco Partners. The price was not disclosed.

A quiz has one right answer that is written down somewhere. Cancer care does not. Real patients have messy records, several illnesses at once, and treatments that vary by hospital and country. A system that shines on tidy questions can stumble on that mess.

Training on a few invented cases meant Watson learned the habits of a handful of experts, not the variety of real patients. The impressive demo and the daily clinical work were two different tests, and only one of them had been passed.

The same gap appears with AI tools today. A vendor demo uses clean, chosen examples. Your own data is older, odder and full of exceptions. What matters is how the tool performs on your real cases, checked by people who know the work.

The idea behind this caseLesson: Review AI like a manager →

1. 1Ask what data the AI was trained and tested on, and whether it looks like your customers or patients.
2. 2Run a pilot on your own real cases, with experts checking every answer, before it touches decisions.
3. 3Measure results in plain terms, such as how often the advice was correct or unsafe, not how impressive it looked.
4. 4Keep a qualified person responsible for each final decision, and make it easy for them to overrule the tool.
5. 5Agree in the contract what happens if the tool does not reach the promised accuracy.