QRAE：用查询相关注意力熵为 DeepSearch 智能体压缩上下文
QRAE: Query-Relevant Attention Entropy for Context Compression in DeepSearch Agents
搜索型智能体上下文越滚越大的老毛病，这篇论文用注意力熵给检索块打分，延迟降 30%、token 省 45%，成功率还涨了 2.35 个点。
QRAE 是一种在线上下文压缩方法，针对 DeepSearch LLM 智能体反复检索导致上下文膨胀的问题。它通过稀疏证据评估头对检索到的文本块打分：用无查询的对照 prefill 抑制共享注意力响应，再用归一化熵衡量每个文本块内正向响应的广度。高分文本块进入持久证据白名单进入推理上下文，不确定的文本块保留在候选缓存中供后续重评估。QRAE 不需要额外的生成式压缩 pass，也不修改注意力算子。在多个 DeepSearch 基准和推理模型上，任务成功率平均提升 2.35 个百分点，端到端延迟和 token 用量分别平均降低 30% 和 45%，并已验证可泛化到接近 100B 参数的模型。
QRAE: Query-Relevant Attention Entropy for Context Compression in DeepSearch Agents
DeepSearch LLM agents repeatedly search and carry retrieved sources into subsequent reasoning steps. This expanding context increases token consumption and latency, although many results do not contribute to later reasoning. We propose Query-Relevant Attention Entropy (QRAE), an online context compression method that uses sparse evidence-assessment heads to decide which intact retrieved chunks should persist across reasoning steps. It scores bounded candidate batches through paired prefills: a query-free control suppresses shared attention responses, while normalized entropy measures the breadth of the positive response within each chunk. High-scoring chunks enter a persistent evidence whitelist and are included in the reasoning context. Uncertain chunks remain outside the reasoning context but are retained in a candidate cache, where they can be retrieved for later reevaluation. QRAE requires neither an additional generative compression pass nor modification of the attention operator. Across multiple DeepSearch benchmarks and reasoning models, QRAE improves task success by an average of 2.35 percentage points and reduces end-to-end latency and reported token usage by an average of 30% and 45%, respectively. These benefits generalize to a model with nearly 100B parameters.