Context windows got bigger. Agent accuracy didn't follow.
Chroma tested 18 models and found performance slipping as inputs grew, even on simple tasks. The fix for a flaky agent isn't a bigger window. It's someone deciding what goes in it.
Every model vendor spent the last two years selling context window size as the headline spec. A million tokens here, two million there. The implied promise was simple: a bigger window means a smarter agent. In July 2025 Chroma tested that promise against 18 models from Anthropic, OpenAI, Google and Alibaba. It doesn't hold.
The tasks were deliberately easy. Find a fact in a long document. Repeat a string of words. Answer a question from a chat history. Even here, performance changed significantly as the input got longer. One distractor, a passage that looks like the answer but isn't, was enough to hurt accuracy. On the chat history test, models did better when given only the relevant part of the conversation than when given the whole thing. And oddly, models did better when the surrounding text was shuffled than when it read like a coherent document.
Anthropic's engineering team says it plainly. Models have an "attention budget," and every token you add spends some of it. Their guidance for agents is to find "the smallest possible set of high-signal tokens" that gets the job done. In practice that means compaction, where a conversation near the limit gets summarized and restarted. Sub-agents that burn tens of thousands of tokens exploring, then hand back a 1,000 to 2,000 token summary. Clearing old tool results the agent no longer needs. All of it points the same way. Less in the window, not more.
I've worked a version of this problem at a different layer. Running QA on LLM training data pipelines, the hard part was never getting more data in front of the model. It was deciding what counted as signal and cutting the rest before it reached a training run. Context rot is the same failure, showing up live at inference time instead of once at training time.
That's why I treat context as a product decision, not an infrastructure setting. Sourcegraph's guide for agent builders draws the line well: prompt engineering "shapes one string," context engineering "owns the pipeline that produces it." Someone has to own that pipeline. The same way someone owns the eval threshold and the blast radius doc.
The practical version for anyone shipping an agent starts with an eviction policy, not a window size. Decide up front what gets summarized, what gets dropped, and at what token count that happens, before the agent starts doing it by accident in production. Test at the length your agent will actually run at, not the empty-room length of a demo, because the failure only shows up once real context piles up. And push deep exploration into a sub-agent or a scoped tool call instead of pouring every step into the main thread.
Also, retire "the agent might need this" as a reason to keep something in context. Every extra token is a tax on the ones that matter. And the tax comes due quietly, as a wrong answer with no error message attached.
That said, this isn't an argument against long context. Long context is useful, and the frontier keeps moving. It's an argument against buying window size as a substitute for curation. A bigger window gives you more room to be sloppy. It just delays when you notice.
Evals decide when the output is good enough. A blast radius decides what the agent can touch. Context engineering decides what it's looking at when it acts. Skip that decision and the model makes it for you.