上下文压缩的隐性代价:智能体交互成本测量

What Does Context Compression Cost an Agent? Interaction Costs Unrevealed by Task-Completion Metrics

精选理由

这篇论文用GPT-5.5和DeepSeek实测发现,压缩上下文会让智能体反复重新查工具,任务成功率却没变。想知道为什么只看完成率会骗人,可以读读。

AI 摘要

一项研究提出了新的测量协议,在固定24轮预算和确定性规划环境中评估上下文压缩对工具使用智能体交互成本的影响。实验覆盖三个模型和两种任务,发现检索调用在所有六种模型-任务组合中都增加,且几乎贡献了全部额外交互。5倍压缩时任务完成率无显著变化,但GPT-5.5的检索调用从21.0增至63.9次(p=.002),完成率从80%变为85%(p=1.0)。DeepSeek仅在接受10倍压缩时出现显著完成率下降。在ALFWorld环境中,压缩未引发检索激增,说明该交互成本依赖于具体任务环境而非上下文缩短本身。

原文 · arXiv: DeepSeek

What Does Context Compression Cost an Agent? Interaction Costs Unrevealed by Task-Completion Metrics

Task completion is the standard metric for evaluating context compression, yet it is incomplete: compression can increase an agent's interaction cost by forcing it to reacquire dropped state while leaving completion statistically unchanged. We introduce a controlled runtime measurement protocol for reacquisition cost in a bounded-horizon tool-using agent. The agent acts in a deterministic planning environment under a fixed 24-turn horizon. We vary compression severity, compare a dropping operator with a fact-preserving operator, restore dropped state through controlled oracle interventions, and decompose tool calls into retrieval and execution. We evaluate three models across two task regimes. Retrieval calls increase in all six model-regime comparisons and account for almost all added interaction; five of six remain significant after Holm correction. At the prespecified 5x comparison point, completion changes are not significant in any cell. DeepSeek shows a significant completion drop only at 10x compression. GPT-5.5 is the clearest case: completion changes from 80% to 85% (p = 1.0) while retrieval increases from 21.0 to 63.9 calls (p = .002). Retention interventions further separate state quantity, state type, and content validity. Random selection is comparable to an offline hindsight oracle, while replacing retained D-state with semantically irrelevant content increases retrieval by 57% (p < .001) without a significant completion change. In a second environment, ALFWorld, sliding compression produces no retrieval surge, showing that the reacquisition signature is environment-dependent rather than intrinsic to shortening context. Overall, compression can impose hidden interaction costs when execution-relevant state becomes absent and must be reacquired, while completion alone may not expose those costs.