Papers

Harvard 论文:编码智能体 94.8% 的 token 是缓存上下文重复读取,能耗被低估约 25 倍

Why we picked it

做智能体成本核算的人该看看这篇。Harvard 团队实测 10 周 Claude Code 用量,发现 EcoLogits 这类输出计价工具会把能耗低估 25 倍,因为缓存读取和子智能体调用都没算进去。文中还给了实用建议:新任务开新会话。

一项 Harvard 论文记录了 1 名工程师 10 周内真实使用 Claude Code 的情况,共约 10 亿 token、42 个会话。结果显示 94.8% 的 token 是对缓存上下文的重复读取,每生成 1 个 token 需读取约 260 个缓存 token,会话内上下文增长 5.3 倍。

Body

作者据此搭建了一个缓存感知的能耗估算器:对缓存 token 只计注意力读取成本,不计完整重新处理。按此估算 10 周能耗约 250 千瓦时,而只按输出计价的 EcoLogits 估算器给出约 10 千瓦时,相差约 25 倍。

传统 LLM 能耗测算只针对单次请求-响应调用,而智能体在每次调用时重复读取不断增长的上下文,并把工作交给子智能体,这些开销常被漏算。对需要核算智能体成本或排放的团队,论文建议把缓存读取和子智能体调用计入,并尽量为新任务开启新会话。

Original · rohanpaul_ai

New Harvard paper finds that 94.8% of a coding agent's tokens were re-reads of cached context, i.e. 94.8% of the text the AI processed was earlier conversation it was re-reading, not new work.

And that re-reading raises its estimated energy use about 25 times, from roughly 10 to 250 kilowatt-hours over 10 weeks.

Energy estimates for LLMs usually price single request-and-response calls. Agents re-read a growing context on every call and hand work to subagents, which those estimates can miss.

The authors logged 10 weeks of real Claude Code use by 1 engineer, about 1 billion tokens across 42 sessions. They built an energy estimator that charges cached tokens for attention reads but not full reprocessing.

About 260 cached tokens were read per generated token, and context grew 5.3× over a session. The cache-aware estimate came to about 250 kWh, against about 10 kWh from the output-based EcoLogits estimator.

If you track agent cost or emissions, count cached reads and subagent calls, and start new tasks in fresh sessions when possible.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include Claude Code, Harvard, agents, caching.

Source attribution: curated from rohanpaul_ai (@rohanpaul_ai).