Lesson 184 · 3 min
The cost of each AI question
Every question sent to a paid AI model has a small cost, set mostly by how much text goes in and comes out, and those small costs add up fast.
When an app uses a model through an API, the provider charges by tokens, small chunks of text. You pay for the tokens you send, including hidden instructions, chat history and attached documents, and for the tokens the model writes back.
That is why a short question can be expensive. If the app sends a 50-page manual with every message, each tiny "thank you" costs as much as reading the manual again. Longer chat histories make each new message more expensive too.
Model choice matters. Bigger, smarter models usually cost more per token than smaller, faster ones. Reasoning models may also write long hidden thinking before answering, which counts too.
Builders keep costs down by sending only the relevant parts of documents, using smaller models for simple steps, reusing repeated prompts where the provider allows caching, capping answer length and setting a monthly spending limit. Check your provider's current price page, since prices change often.
Do it now
Estimate a feature's monthly bill
- 1. Imagine a help bot that gets 200 questions a day.
- 2. Estimate how many words go in and come out for one question, including any document it reads.
- 3. Ask an AI to help turn that into tokens and a monthly cost using a provider's published prices, and see which part costs most.
Help me estimate the cost of an AI feature. Per question it sends about [WORDS IN] words and gets back about [WORDS OUT] words. It gets [NUMBER] questions a day. Explain the token maths step by step and tell me which change would cut the cost most. I will fill in current prices from the provider's website.
Quick check
Keep going
Next →When AI products fail in publicWords from this lesson
- Token
- A chunk of text, often part of a word, that AI reads and writes in. Limits and prices are counted in tokens.
- API
- A doorway that lets one program send requests to another, for example an app asking an AI model for an answer.
- Inference
- The moment a trained model is actually used to produce an answer, as opposed to the earlier learning phase.
- Context window
- The maximum amount of text an AI can look at in one go. Anything beyond it is forgotten for that conversation.
- Reasoning model
- A model that spends extra time working through a problem internally before replying, which helps with maths, logic and planning.