Learning outcomes
- Budget context accurately
- Prioritize useful context
- Identify truncation risks
Mental model
The context window is a finite working surface shared by instructions, user input, conversation state, retrieved evidence, tool results, and generated output.
Theory
A nominal maximum does not guarantee equal attention to all positions or reserve enough output capacity. Applications should treat context as a budget: allocate fixed instructions, variable user content, evidence, history, and output headroom. Relevance, ordering, isolation, and compression usually matter more than filling the window.
Alternatives and trade-offs
Summarization reduces history, retrieval selects external evidence, structured state keeps facts outside transcripts, and larger-context models increase capacity. Each changes fidelity, latency, cost, and failure modes.
Failure modes and misconceptions
Do not concatenate all available data, silently truncate, consume output headroom, or confuse maximum context with reliable recall.
Knowledge check
Why should an application reserve output tokens before assembling input context?
Decision scenario
A research assistant must analyze long reports. Partition the task, retrieve relevant sections, preserve citations, reserve output, and expose omitted material instead of silently clipping the source.
Relationships
Tokens and Tokenization
Context limits are measured and allocated in tokens.
prerequisiteMessage Roles and Instruction Priority
Messages share the finite context supplied to a model.
prerequisiteContext Engineering
Context engineering manages finite model input budgets.
Primary sources
- OpenAI API Reference - OpenAI, verified 2026-07-16