Lesson 182 · 2 min
Designing for an AI that is sometimes wrong
Good AI products assume the model will make mistakes and make those mistakes easy to see, fix and recover from.
Every AI feature will be wrong some of the time. The question for the designer is what happens next. Does the user notice? Can they fix it in one tap? Is any harm done before they do?
Show your working. Link to the sources an answer came from, highlight the text it extracted, or show its confidence when the model is unsure. People catch errors when they can see where an answer came from.
Keep the human in the loop for actions. Let the AI draft the email, and let the person press send. Let it suggest a category, and let the person change it easily. Make undo easy and obvious.
Be honest in the wording. Labels like "AI draft, please check" set the right expectation. Avoid designs that make AI output look final and official, because people trust what looks confident, even when it is wrong.
Do it now
Redesign one AI screen
- 1. Think of an AI feature you use, like smart replies or auto-categorised expenses.
- 2. Ask: when it is wrong, how do I notice, and how do I fix it?
- 3. Sketch one small change, like a source link or an undo button, that would make mistakes easier to catch.
This AI feature [DESCRIBE] is sometimes wrong. Suggest five small design changes that would help users notice and fix mistakes quickly, without making the feature annoying.
Quick check
Words from this lesson
- Human in the loop
- Designing a process so a person reviews or approves the AI's work before it has real consequences.
- Automation bias
- Our habit of trusting a machine's answer too readily, even when our own judgement or other evidence says otherwise.
- Explainability
- How well people can understand why an AI produced a particular answer or decision.
- Grounding
- Tying an AI's answer to real sources you supply or it looks up, so claims can be checked instead of invented.