AI engineering · 23 of 42

Context Window

One fixed budget, shared by everything

Scroll

One fixed budget, shared by everything

Everything the model considers arrives in one block with a hard size limit: your instructions, the conversation so far, whatever was retrieved, the current question, and the space left to write an answer in.

They compete. A longer conversation leaves less room for retrieved evidence. More retrieved evidence leaves less room for the answer.

Bigger windows have not removed this decision, they have raised the cost of not making one — filling a large window with marginal material is slower, dearer, and often worse, because relevant things get buried. Concepts 1, 6 and 10 all exist to manage this one constraint.

Input
ONE FIXED BUDGET, SHARED BY EVERYTHING system history retrieved your question room to answer the limit grow any one of these and something else has to shrink a longer history leaves less room for retrieved evidence Everything competes: your instructions, the conversation so far, whatever was retrieved, the question, and the space left to answer in. A bigger window does not remove the decision. It raises the price of not making one.
A single budget bar divided between the system prompt, history, retrieved evidence, the question and the room left to answer.