Case files
When AI went wrong
Real, documented incidents. Each one is a short investigation: what happened, why it happened in terms of how AI works, and what would have stopped it. Every fact links to its source.
- 01The chatbot that promised a refundAn airline's website chatbot described a refund rule that did not exist, and a tribunal made the airline pay for it.Air Canada
· Canada · 2024
· Made-up facts - 02The court cases that never existedTwo lawyers filed a court brief full of judgments invented by ChatGPT, and a federal judge fined them and their firm.Mata v. Avianca
· New York, United States · 2023
· Made-up facts - 03The video call where everyone was fakeA finance worker joined a video call with what looked like his chief financial officer and colleagues, then sent about US$25 million to fraudsters.Arup
· Hong Kong · 2024
· Deepfake fraud - 04The secret code pasted into a chatbotEngineers pasted confidential source code and meeting notes into ChatGPT, and Samsung then restricted generative AI on work devices.Samsung Electronics
· South Korea · 2023
· Data leaks - 05The hiring tool that learned to prefer menAn experimental CV-screening model learned from ten years of mostly male applicants and began marking down CVs that mentioned women.Amazon
· United States · 2018
· Bias - 06The risk score that wrecked familiesAn algorithm used to flag childcare benefit fraud treated non-Dutch nationality as a risk factor, and tens of thousands of parents were wrongly accused.Dutch Tax and Customs Administration
· Netherlands · 2021
· Bias - 07The dealer bot that "sold" a car for $1Visitors talked a car dealer's ChatGPT-powered chatbot into writing code and "agreeing" to sell a new SUV for one dollar.Chevrolet of Watsonville
· California, United States · 2023
· Security - 08The chatbot that learned abuse in a dayMicrosoft's Twitter chatbot Tay was designed to learn from conversations, and within a day users had taught it to post racist and offensive messages.Microsoft
· United States · 2016
· Security - 09The phone call in a daughter's voiceA mother heard what sounded exactly like her daughter crying for help, then a man demanded a ransom; her daughter was safe all along.A family in Arizona
· Arizona, United States · 2023
· Deepfake fraud - 10The search answer that said to add glue to pizzaGoogle's AI Overviews summarised joke posts and satire as facts, suggesting glue on pizza and eating rocks.Google
· United States · 2024
· Over-trust - 11The automatic debt letters nobody checkedAn automated system sent hundreds of thousands of Australians debt notices based on averaged income, and a Royal Commission later called it crude, cruel and illegal.Robodebt (Australian Government)
· Australia · 2023
· Over-trust - 12The friend's video call that lasted secondsScammers took over social media accounts and used short, blurry video calls showing a friend's face to make loan requests look real.Video-call scams in Vietnam
· Hanoi, Vietnam · 2023
· Deepfake fraud