The paper that gave GPT its T

By 2017 the best language software, such as translation systems, read a sentence one word at a time, carrying a running summary forward. That was slow to train, because each step had to wait for the one before. It also made it hard to link words far apart in a long text.

In June 2017 eight researchers, most of them at Google, posted a paper called "Attention Is All You Need". It proposed a design they named the Transformer. Instead of reading in order, it looks at all the words at once. For every word it works out how much attention to pay to every other word, so it can link "it" to the right noun, or tell which meaning of "bank" fits the sentence.

Because all the words are handled together, the work can be split across many chips at the same time. The paper reported better translation scores than earlier systems, after training for 3.5 days on eight graphics cards, far less than the previous best methods needed. It was presented at the NIPS conference in December 2017.

The Transformer made it practical to train much larger language models on much more text. In 2018 OpenAI released its first GPT model. GPT stands for Generative Pre-trained Transformer. Most of today's major chatbots are built on the same basic design.

For a chatbot user, this explains how a chatbot reads your message. It does not read top to bottom like a person. It looks across everything in its view at once, your question, the document you pasted and the earlier chat, and weighs which parts matter for each word it writes. That view, the context window, has a size limit. What is outside it cannot be weighed at all, and in a very long text some details can get less attention than you would like.

The idea behind this caseLesson: The context window: AI's short-term memory →

1. 1Put everything the chatbot needs into the chat: the document, the background and the question.
2. 2State clearly which part matters most, for example "focus on section 3".
3. 3For very long documents, ask about one section at a time, or ask it to quote the passage it relied on.
4. 4In a long conversation, restate key facts and decisions instead of assuming it still weighs early messages.