Lesson 185 · 3 min
When AI products fail in public
Public AI failures usually follow a few patterns, like invented facts, promises nobody approved and bots tricked into embarrassing replies, and each has a known fix.
Over the past few years there have been many public stories: a support bot inventing a refund policy, a chatbot tricked into insulting its own company, lawyers filing documents with court cases that did not exist, an AI search answer repeating a joke as advice.
Look closely and the causes repeat. The model was allowed to answer without reliable sources. It could make promises on the company's behalf. Users found a way around its rules. Or people trusted the output without checking it.
The fixes are now well known. Ground answers in real documents and say "I don't know" when nothing matches. Never let a bot commit to money, legal or medical matters without a person. Red-team before launch. Monitor after launch.
In at least one well-known case, a tribunal held the company responsible for what its bot told a customer. For a business, the bot speaks with the company's voice. For a user, the lesson is the same as always: check anything important at an official source.
Do it now
Write a pre-mortem
- 1. Imagine a business you know launching a public chatbot tomorrow.
- 2. Write the headline of the worst story that could appear about it.
- 3. List what would have to go wrong for that to happen and one guard against each.
A [TYPE OF BUSINESS] is launching a public AI chatbot. Imagine three embarrassing news headlines that could come from it. For each, explain the likely cause and one practical safeguard.
Quick check
Keep going
Next →Building your own small AI toolWords from this lesson
- Hallucination
- When an AI states something false with confidence, such as a made-up fact, quote or source.
- Jailbreak
- A trick prompt designed to get an AI to ignore its safety rules and do something it normally refuses.
- Red teaming
- Deliberately attacking your own AI system, the way a bad actor would, to find weaknesses before release.
- Grounding
- Tying an AI's answer to real sources you supply or it looks up, so claims can be checked instead of invented.
- Guardrails
- Rules and filters around an AI system that block harmful, off-topic or unsafe requests and answers.