Foundations
Context window
The maximum amount of text a model can consider at once — everything it knows for that request must fit inside it.
The context window is the model's working memory for a single request, measured in tokens. Modern models hold a lot — hundreds of thousands of tokens — but it is still finite, and filling it has costs.
Two misconceptions are worth correcting:
- Bigger is not automatically better. Models can lose track of detail buried in the middle of very long inputs. Supplying the right ten pages usually beats supplying all thousand.
- It resets every request. The model does not remember your last conversation. Anything it needs must be re-supplied, which is exactly the problem retrieval solves.