L031 · How AI thinks · 2 min
Attention, in plain words
Attention is how a model decides which earlier words matter most for the word it is writing now.
01 · The idea
When a chatbot reads your message, it breaks it into tokens. To understand any one token, it looks back at all the others and asks: which of these help me make sense of this one?
Take the sentence "The cat sat on the mat because it was warm." To work out what "it" means, the model gives more weight to "mat" and "warm" and less to "the" or "on". That weighting is called attention.
Modern language models are built on a design called the transformer, and attention is its core trick. The model runs many attention steps in layers, each one picking up different links: grammar, meaning, who did what to whom.
This is why chatbots can follow long, messy messages. It also explains a weakness: attention gets spread thin over very long text, so small details can get less focus.
02 · Think of it like this
It is like reading with a highlighter. For every word, you quickly mark the few earlier words that explain it, and you read the word in light of those marks.
03 · Do it now
04 · Quick check
Words in this lesson
See also
L006How AI thinks · 2 min
The context window: AI's short-term memory
A chatbot can only see a limited amount of text at once, and in a very long chat the oldest parts can slip away.
L004How AI thinks · 1 min
Tokens: how AI reads
AI does not read words; it reads tokens, small chunks of text that are often parts of words.
L003How AI thinks · 1 min
What a chatbot really does
A chatbot writes its answer one small piece at a time, each time picking a likely next word.
L048Under the hood · 2 min
Latency and streaming: why answers type themselves out
Chatbots show words as they are produced, called streaming, because a full answer takes seconds and seeing it start right away feels much faster.