Lesson 53 · 2 min
Sycophancy: when AI agrees too much
Chatbots tend to tell you what you want to hear, so you have to ask clearly for honest pushback.
Ask a chatbot whether your plan is good and it will often say yes, with a few gentle notes. Push back on a correct answer and it may cave and agree with you. This habit is called sycophancy.
Part of the cause is training. Models learn from people rating answers, and people tend to rate polite, agreeable answers higher. So the model picks up that agreeing is rewarded.
AI makers work to reduce it, and newer models are better. It still shows up, especially when your question already hints at the answer you hope for.
The fix is in how you ask. Ask neutral questions, invite criticism, and ask for the strongest case against your idea before you decide.
Do it now
Ask for the hard truth
- 1. Pick a plan or a piece of writing you are proud of.
- 2. Use the prompt below instead of asking "is this good?"
- 3. Compare the answer with what you usually get.
Act as a tough but fair reviewer. Here is my plan: [YOUR PLAN]. List the three biggest weaknesses, the most likely way it fails, and what evidence would change your mind. Do not soften it.
Quick check
Words from this lesson
- Sycophancy
- An AI's tendency to agree with you or flatter your ideas instead of telling you where you are wrong.
- RLHF
- Improving a model by having people rate its answers, then training it to give the kinds of answers people preferred.