The number on the certificate #
Every aircraft has a weight in its documentation that no pilot ever flies to.
Language models have acquired the same arrangement, and almost nobody is treating it that way. Norman Paulsen's study of what he calls the Maximum Effective Context Window, published in January, ran hundreds of thousands of measurements against the leading models to find the point where additional input stops improving the answer. The maximum context window is the number on the specification sheet, the volume of text a model will accept in one go. The effective window is where accuracy still holds.
The gap is not a rounding error. On some problem types, a few of the top models in the test group failed with as little as a hundred tokens of context, and most showed severe degradation in accuracy by a thousand. The effective ceiling also moves depending on the kind of problem posed, which means it is not a property of the model alone.
None of this produces an error. The answer still arrives, fluent and on topic. You just received a worse one than the same model would have given you with less.
All eighteen, no exceptions #
The most useful result in this field is also the one that makes writers uncomfortable.
In July 2025, Kelly Hong, Anton Troynikov and Jeff Huber at Chroma tested eighteen frontier models on retrieval and reasoning tasks at increasing input lengths. Every model degraded. Not most of them. Distractors, meaning passages that look topically relevant but do not answer the question, damaged performance more than sheer volume did, and their effect was not proportional to how many there were. On a long conversational benchmark, models did better when given only the relevant excerpts than when given the full history.
Then there is the finding that should stop anyone who assembles context for a living. The researchers took their source documents and randomly shuffled the sentences, destroying the logical flow while keeping the same material. Across all eighteen models, performance improved on the shuffled version.
I have argued for months that the value in this work sits in the context you build around the tool rather than in the tool itself. I still think that is right. The measurements add a condition I had not accounted for: a well-organised document is not automatically a well-performing input, and coherent prose may create exactly the plausible-looking distractors that make a specific fact harder to isolate.
Density altitude #
Return to the aircraft, because the discipline that governs this is worth borrowing whole.
Maximum takeoff weight is a structural limit. It is fixed, it is certified, and it appears in the manual for the life of the airframe. It is also not the weight you take off at. Before each flight you compute a performance-limited weight from the conditions of that day: air temperature, pressure altitude, runway length available, obstacles on the departure path. On a hot afternoon at a high-elevation field with a short strip, that number can sit far below the certificate, and the answer is to leave fuel or payload on the ground.
The certificate never changes. The usable figure changes every flight.
An advertised context window is the certificate. The effective window is the performance-limited weight, computed against the task in front of you: how the question relates to the material, where the decisive fact sits in the stack, how many near-misses surround it. The difference is that aviation prints the performance tables and requires the calculation, while the model's product surface offers a token counter and nothing else.
When the supervisor starts reading along #
Yesterday I argued that agent safety is a question of volume rather than character, and that the log must sit somewhere the agent cannot write. There is a term I left out of that argument, and it belongs here.
In May, Sam Martin, working through the Anthropic Fellows Program, and Fabien Roger at Anthropic published a study of language models used as monitors, the classifiers that read an agent's transcript looking for dangerous actions. Real coding-agent transcripts routinely exceed half a million tokens, while the benchmarks used to validate monitors rarely go past a hundred thousand. So they padded the test.
With 800,000 tokens of ordinary, benign activity placed before the dangerous action, Opus 4.6, GPT 5.4 and Gemini 3.1 missed it between two and thirty times more often than when the same action appeared on its own. Periodic reminders inserted through the transcript recovered part of the loss. Fine-tuning, in their attempts, did not transfer.
Read that alongside the containment argument and the picture tightens. The boundary you drew around the agent is supervised by something operating under its own performance limit, and that limit degrades in the direction that costs you most. The monitor does not raise a false alarm. It stops noticing.
The objection that holds #
Every measurement quoted here describes a generation of models that is already being replaced.
Long-context handling is an active engineering target, the numbers will move, and anyone building a permanent thesis on a specific token threshold is going to look foolish within the year. That objection is correct and I have no counter to it.
What survives the objection is structural. Whatever the effective ceiling turns out to be for the model you use next quarter, it will not be the number in the marketing material, it will vary by task, and the product will not tell you when you have gone past it. The only instrument that reveals it is a set of cases you have written yourself and can rerun, which is the same unglamorous asset that tells you whether a supplier change broke anything.
So here is the question I have not resolved for my own work. Everything you added to your prompts this year to make the answers better, the extra documents, the fuller history, the careful structure: how much of it was payload, and how much was just weight? Nothing on the aircraft weighed it for you.