CMU 论文：用 coding agent 调优 LLM 推理栈，Gemma E4B 每美元 GPU 价值达 vLLM 4.87 倍
CMU 让 Claude Sonnet 5.5 给 Gemma E4B 手搓了一个推理系统，每美元 GPU 价值是 vLLM 的 4.87 倍，论文和基准都放出来了。
卡内基梅隆大学的 ServerBench 论文让 Claude Sonnet 5.5 驱动的 coding agent 为 Gemma E4B 从提示词到 token 服务的全栈设计专用推理系统，最终实现每美元 GPU 价值 4.87 倍于 vLLM 基线。基准评分为经延迟校验的每美元 token 价值，通过重放带依赖的真实 coding 会话测量。在 11 个 coding-agent 模型和 3 种部署环境的组合中，33 次运行有 22 次产出通过独立验证的系统，13 次超过最强基线。失败案例的主要原因是引入了未经测试的新代码路径。
Impressive paper from CMU. Soon, we will have coding agents that can tune the entire stack, from prompts to token serving. This work experiments with coding agents to tune an LLM serving stack. They get Claude Sonnet 5.5 to build a serving system for Gemma E4B that delivers 4.87x the value per GPU dollar of the vLLM reference. ServerBench asks coding agents to design a complete, specialized serving system and decide how many concurrent sessions to run. The score is latency-qualified token value per dollar of GPU cost, measured by replaying real coding sessions with their dependencies intact. An independent check verifies the system works. Across 11 coding-agent models and three deployments, 22 of 33 runs produced a verified system and 13 beat the strongest baseline. The best designs raised capacity and chose the load to match it. Failed runs often shipped optimizations whose new code paths were never tested. arxiv.org/abs/2611.06841 Chat with Paper: academy.dair.ai/papers/serverb… 💬 31 🔄 5 ❤️ 32 👀 2570 📊 32 ⚡
