MIT 与 NVIDIA 提出 QDA:用智能体自动搜索最优量化配方
MIT 和 NVIDIA 的这篇论文解决了个量化配方要靠专家手调的痛点。用 KL 散度几分钟就能筛掉差方案,GLM-5.3-Flash 上比厂商官方 checkpoint 还高 1.4 个点。跑推理服务、做模型压缩的工程团队值得看看。
MIT 与 NVIDIA 发布论文 QDA(arXiv:2611.23388),把大模型低比特量化的配方设计交给智能体自动完成。传统方法里,每种模型和精度组合(如 W4A16、W4A8、W3A16)都需要专家手工调整流程,而 QDA 让智能体在多轮迭代中自动构建、测试候选方案,在三种精度设置下分别超过最强公开配方 0.9、1.4 和 2.2 分,每次搜索仅消耗 64 GPU 小时。
核心技术突破在评估速度:传统上给一个量化后的推理模型打分需要数十 GPU 小时,QDA 改为在智能体未见过的推理轨迹上测量量化模型与原始模型的输出 KL 散度,把单次评分压缩到几分钟。 promising 的候选方案再通过长文本生成来检测重复循环问题。系统还内置公开 PTQ 方法库,智能体可以直接组合不同方法而无需逐个移植代码库。
实际效果方面,在 320B MoE 模型 GLM-5.3-Flash 的 NVFP4 W4A4 量化上,QDA 恢复了 FP8 精度的 98.3%,高于最佳厂商 checkpoint 的 96.9%。所有产出的 checkpoint 都能运行在现有 serving kernel 上,不引入推理开销。
Great paper from MIT and NVIDIA. I think this is an important step toward personalized, accessible RSI. The best recipe changes with every model and precision, so experts usually redesign it by hand. QDA hands that loop to an agent that builds and tests candidates over many rounds. They show that QDA beats the strongest public recipes by 0.9, 1.4 and 2.2 points at W4A16, W4A8 and W3A16, using 64 GPU-hours per search. Evaluation speed is what makes this work. Scoring one quantized reasoning model normally takes tens of GPU-hours. QDA scores each candidate in minutes by measuring its output KL divergence from the original model on reasoning traces the agent never sees. Promising candidates then get long generations to catch repetition loops. A shared library of public PTQ methods lets the agent combine them without porting each codebase by hand. On the 320B MoE GLM-5.3-Flash at NVFP4 W4A4, QDA recovered 98.3% of FP8 accuracy, against 96.9% for the best vendor checkpoint. Every checkpoint runs on existing serving kernels with no inference overhead. arxiv.org/abs/2611.23388 Chat with Paper: academy.dair.ai/papers/qda-clo…📊 6 ⚡
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include QDA, NVIDIA, MIT, quantization.
Source attribution: curated from elvis (@omarsar0).