Lesson 167 · 3 min
The journey of one question through an AI app
Between pressing send and seeing an answer, your question passes through checks, extra context, the model, maybe some tools, and more checks.
You type a question into a company's help chat and press send. First the app may check your message: is it too long, does it contain a card number, is it trying to trick the bot? This is an input guardrail.
Next the app builds the full prompt. It adds hidden instructions about the bot's role, some recent chat history, and often a few passages pulled from the company's help pages that match your question. Your message is only one part of what the model reads.
The model then writes the answer token by token. If it needs a tool, such as looking up your order, it asks the app to run it and continues with the result. The text streams back to your screen as it is written.
Before or while it is shown, an output check may scan the answer for private data or banned topics. The app also logs the exchange so the team can spot problems later. All of this usually happens in a few seconds.
Do it now
Trace a real question
- 1. Ask any chatbot with web search a question about something recent.
- 2. Watch for each stage: a pause, a searching message, links, the answer streaming in.
- 3. Write down which stages you could see and which you think happened out of sight.
Quick check
Words from this lesson
- Guardrails
- Rules and filters around an AI system that block harmful, off-topic or unsafe requests and answers.
- System prompt
- Background instructions set by the app maker, hidden from you, that tell the AI how to behave in every conversation.
- RAG
- Having an AI look up relevant passages from your documents first, then answer using them, so replies are more accurate.
- Token
- A chunk of text, often part of a word, that AI reads and writes in. Limits and prices are counted in tokens.
- Latency
- The delay between asking an AI something and getting the start of its reply.