When the computer was believed over the people

From around 2000, the Post Office ran its branches on a computer system called Horizon, supplied by the technology company Fujitsu. Sub-postmasters, the self-employed people who run local branches, had to balance their accounts on it.

When Horizon showed money missing, the Post Office treated the gap as a real debt. Sub-postmasters had to pay it back, even when they said the figures were wrong. Many were prosecuted for theft, fraud or false accounting. The Post Office kept saying Horizon was robust and reliable.

A group of more than 500 sub-postmasters, led by Alan Bates, sued. In December 2019 Mr Justice Fraser ruled that Horizon had bugs, errors and defects, and that there was a material risk that shortfalls in branch accounts were caused by the system. He called the early version of Horizon "not remotely robust". The case had been settled just before the ruling.

In April 2021 the Court of Appeal quashed the convictions of 39 former sub-postmasters, finding that the Post Office knew of serious problems with Horizon and failed to investigate or disclose them. On 24 May 2024 a new law, the Post Office (Horizon System) Offences Act, quashed hundreds more convictions at once. A public inquiry, chaired by Sir Wyn Williams, began publishing its final report in July 2025.

The core failure was not one bug. It was a rule, in practice, that the computer was right and the person was lying. Once the Post Office assumed Horizon could not be wrong, every shortfall became evidence of theft, and the people who said otherwise had to prove a negative against a system they could not see inside.

AI is now used to flag fraud, score job applicants, check benefit claims and mark work. These systems can be wrong, and their errors can look exactly like proof. When an organisation treats an AI's output as fact, and asks the person accused to disprove it, it repeats the Horizon mistake. The output should be a lead to investigate, not a verdict.

The idea behind this caseLesson: Check what AI tells you →

1. 1Treat any AI flag about a person, such as fraud, cheating or poor performance, as a reason to look closer, never as proof.
2. 2Check the AI's finding against independent evidence before any action is taken against someone.
3. 3Give people a clear way to challenge a decision, and give them the information they need to do it.
4. 4Keep records of known errors in the system and share them with anyone affected.
5. 5When many separate people report the same problem, investigate the system, not only the people.