SteerDistill：用教师闭环引导量化 LLM Agent，W4A16 下逼近 BF16
SteerDistill: Quantizing LLM Agents via Closed-Loop Teacher Steering
如果你在部署低比特量化的 Agent，这篇很值得看。它让 W4A16 量化模型的表现只差 BF16 一分以内，训练成本还比 QeRL 少 67% 的 GPU 时数。做端侧 Agent 或成本敏感推理的工程师可以直接参考。
论文提出 SteerDistill，一种量化感知的蒸馏方法，解决低比特量化导致 LLM Agent 任务成功率下降的问题。作者分析失败轨迹后发现，量化后的 Agent 会遗漏任务要求或环境条件，导致环境状态转移偏离全精度参考轨迹。
方法上，全精度教师引导学生修正失败响应，并利用学生执行反馈迭代调整引导，形成闭环教师引导；训练时让学生在修正回合匹配引导后学生的预测、其余回合匹配全精度教师，实现推理时无需引导。
在 5 个 agentic 基准和 3 个模型上，SteerDistill 在 W4A16 下平均得分超过最强量化基线 1.67–2.18 分，与 BF16 差距小于 1 分；在 τ²-Bench 上成功率比 QeRL 高 1.84 个百分点，训练 GPU 时数减少 67%，成功轨迹也更短。
SteerDistill: Quantizing LLM Agents via Closed-Loop Teacher Steering
Low-bit quantization reduces the inference cost of LLM agents but can compromise task success. Examining failed rollouts, we find that quantized agents overlook task requirements or environmental conditions, producing environment transitions that diverge from successful full-precision references. Correcting these behavioral divergences remains difficult: offline data may miss the student's failure contexts, while its own rollouts may provide few correct responses in those contexts. We introduce SteerDistill, a quantization-aware distillation method in which a full-precision teacher guides the student to revise failed responses and iteratively refines its guidance using feedback from student execution. This closed-loop teacher steering produces locally verified behavioral corrections. We internalize these corrections by training the student without guidance to match the steered student's predictions at corrected turns and the full-precision teacher's at other turns, enabling unassisted inference. Across five agentic benchmarks and three models, SteerDistill exceeds the strongest quantized baseline by 1.67--2.18 points in average score under W4A16 and stays within one point of BF16. On $τ^2$-Bench, it outperforms QeRL by 1.84 percentage points in success rate using 67% fewer training GPU hours and produces shorter successful trajectories than the quantized baselines.
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include SteerDistill, QeRL, τ²-Bench, 量化.
Source attribution: curated from arXiv cs.LG (@Genming Cui, Xiangyu Li, Zixu Hao, Xin Ding, Yunxin Liu, Ting Cao).