NVIDIA Paper Shows Aggressive Context Compaction Matches Full-History Agents
Compacting aggressively changes the cost calculus for long-running agents: short state summaries can outperform full transcripts on both accuracy and spend, since late-token generation runs up to 7x pricier. Builders running multi-step coding or tool-use agents can test end-of-conversation compaction before paying for million-token runs. The gains were measured on Claude Opus 4.5, so teams using other frontier models should verify the effect on their own stacks.
An NVIDIA paper (arXiv 2611.05306) examines whether agents should compact their context aggressively. With one compaction action per context, Claude Opus 4.5 scores 70.2 on SWE-bench Verified versus 70.0 with full history, while median peak context falls from 32K to 8K tokens. Simple end-of-conversation compaction lifts Opus from 70.0 to 74.8, beating the full transcript. The pattern holds on Terminal-Bench, FRAMES, BFCLv3 and multilingual SWE-bench with modest losses in some settings. Later tokens can cost up to 7x more to generate than earlier ones on frontier open-weight models.
Highly recommended paper from NVIDIA. (bookmark it) There is a lot of debate about whether to use context compaction aggressively. They show that with frequent compaction, frontier language models lose little to no accuracy on a broad range of long-horizon and long-input tasks while keeping their context short. With one action per context, Claude Opus 4.5 scores 70.2 on SWE-bench Verified against 70.0 with the full history. Median peak context drops from 32K to 8K tokens. The agent carries a short summary of its state between steps instead of the whole transcript. Simple compaction at the end of a conversation often does better than the full history, and Opus goes from 70.0 to 74.8 with it. This also matters for cost. Later tokens cost more to generate than earlier ones, up to 7x on frontier open-weight models. The pattern holds on Terminal-Bench, FRAMES, BFCLv3 and multilingual SWE-bench, with modest losses in some settings. Before paying for million-token agent runs, it's worth testing aggressive compaction on your own tasks. arxiv.org/abs/2611.05306 Chat with Paper: academy.dair.ai/papers/you-don… 💬 29 🔄 5 ❤️ 43 👀 2465 📊 36 ⚡
