CompressionBench：测试编码智能体能否自主压缩 LLM
CompressionBench: How can your coding agents compress LLMs?
这篇论文让 6 个编码智能体比赛压缩模型，36 次提交里 29 次跑通，看看它们怎么做到 3.4% 困惑度差距的。
论文 CompressionBench 将 LLM 压缩设定为智能体的终极任务：给定源 checkpoint、存储上限、推理环境和时间预算，智能体需自行产出压缩权重和可执行推理代码。在 6 个智能体、6 组模型与尺寸设置共 36 次提交中，29 次通过独立验证。每个任务选取困惑度最优的智能体设计后，平均困惑度比 BF16 高 3.4%，下游准确率为 77.33%，BF16 为 79.56%。实验记录显示智能体在 24 次修订中有 21 次在存储几乎不变的情况下降低困惑度，完整模型反馈推翻了 79 次局部排名中的 17 次。
CompressionBench: How can your coding agents compress LLMs?
Recursive self-improvement requires AI systems to improve the models and infrastructure used by subsequent systems. LLM compression offers a concrete efficiency objective: retain a pretrained model's behavior within a smaller model storage budget. We study whether coding agents can carry out this step autonomously. We formulate compression as a terminal task in which an agent receives a source checkpoint, storage limit, inference environment, and time budget, then constructs compressed weights and executable inference code. The agent controls its algorithm and experiments; perplexity feedback guides search, and independent verification checks the final model. Across six agents and six model--size settings, 29 of 36 submissions pass verification. Selecting the lowest-perplexity agent design per task yields an average 3.4% perplexity increase over BF16 and 77.33% downstream accuracy, compared with 79.56% for BF16. Recorded experiments reveal how agents achieve these outcomes and where improvement breaks down. Agents lower PPL in 21 of 24 revisions at nearly fixed storage; complete-model feedback reverses 17 of 79 prior local rankings; and generation and inference tests expose failures that likelihood alone misses. The results identify representation design, experimental judgment, and capability validation as central to agent-driven model compression, a measurable component of efficiency-oriented self-improvement.
