Max Tokens

Max tokens is a request parameter that limits how many tokens a model may generate in its response. It is a hard ceiling rather than a target: the model does not aim for the number, it simply stops when it reaches it. Its purpose is control — over cost, over latency, and over the worst case where a model would otherwise keep generating far beyond anything useful. The parameter earns its place because generation length is not reliably predictable. A prompt that normally produces a short answer can, on an unusual input, produce something enormously long, and without a ceiling that becomes a slow request and a large bill. In a multi-step agent the effect compounds, since each step's output becomes the next step's input, so an unbounded response can inflate the cost of everything downstream. The characteristic failure is truncation. When the limit is reached mid-sentence, the response simply ends, and the caller receives something that looks like a complete answer until read closely. Structured output suffers most: a truncated JSON object is invalid rather than short, and code that assumes a parse will succeed fails in a way that looks like a model error rather than a configuration one. Handling this properly means checking the finish reason the provider returns, not merely the presence of a response. Setting the value well is a matter of knowing the distribution rather than picking a round number. Look at the actual length of good responses for the task, allow reasonable headroom above the longest legitimate one, and treat truncation events as a signal to investigate rather than as noise. A limit that truncates one response in a hundred is quietly damaging quality; a limit that never truncates anything may not be providing protection at all. Max tokens is a ceiling, not a way to ask for brevity. Instructing the model in the prompt to answer in a certain length, or to produce a specific structure, is what actually shortens output; the parameter merely prevents the pathological case. Using it as a length control produces answers that are cut off rather than concise, which is worse for the reader and no cheaper than a well-instructed short answer. One more interaction matters for reasoning-heavy models: tokens spent thinking may count towards the same budget as the visible answer, so a limit tuned for a short reply can leave no room for the response itself. Where a provider distinguishes the two, budget them separately; where it does not, set the ceiling against total output and verify with real cases rather than assumption.