Lesson 38 · 3 min
RLHF: how people shape AI answers
After a model learns from text, people rate its answers, and the model is trained to give the kind of answers people prefer.
A model fresh from pre-training is good at continuing text, but it is not yet a helpful assistant. It might ramble, ignore the question or say something harmful.
So companies add a step called reinforcement learning from human feedback, or RLHF. People compare pairs of answers and pick the better one. Those choices train a scoring model, and the chatbot is then trained to get higher scores.
This is a big part of why chatbots are polite, follow instructions and refuse some requests. It is one of the main tools for alignment, which means getting AI to behave the way people intend.
It has side effects. If raters reward answers that sound confident or agreeable, the model learns that too. Some labs now also use AI feedback or written principles alongside human ratings.
Do it now
Be the rater
- 1. Ask a chatbot for two different answers using the prompt below.
- 2. Pick the one you prefer and write down why.
- 3. Notice what you valued: length, tone, accuracy or honesty.
Give me two different answers to this question, labelled A and B, one short and direct, one detailed and friendly: How do I ask my boss for a day off?
Quick check
Keep going
Next →How fine-tuning worksWords from this lesson
- RLHF
- Improving a model by having people rate its answers, then training it to give the kinds of answers people preferred.
- Reinforcement learning
- Training by trial and error, where a system gets rewarded for good outcomes and learns to repeat what earned them.
- Alignment
- The work of making AI systems do what people actually intend, and avoid harmful behaviour, even in unexpected situations.