ZAI GLM-5.3-Flash 优化，Infra Agent 实现 3.2 倍吞吐提升

两周、3.2×、从首次跑通到全量生产 ～ @Zai_org GLM-5.3-Flash 优化，由 GLM-5.3 驱动 Infra Agent 完成 ！ https://t.co/dGe14GphoT...

两周、3.2×、从首次跑通到全量生产 ～ @Zai_org GLM-5.3-Flash 优化，由 GLM-5.3 驱动 Infra Agent 完成 ！ z.ai/blog/glm-built… 事实经历：做了什么，在什么约束下做 工程条件相当苛刻：国产加速器内存与互连带宽受限、1M token 上下文、多模态请求、软件栈不成熟（kernel 缺失、文档靠猜）。因此所有优化本质都是权衡而非免费收益： · ReplaySSM —— 用计算换内存 · 节点内张量并行 —— 用通信换内存 · 混合 INT8/FP8/BF16 缓存 —— 用精度换容量 · Encode–Prefill–Decode 分离 —— 用分离换调度自由度 观点内核：瓶颈不在写代码，而在“为什么变差了” · “吞吐下降 20%” 是稀疏奖励 + credit assignment 问题；它告诉你坏了，但不告诉你哪一层、哪个假设、下一步测什么； · 端到端 benchmark 一跑几小时，探索代价极高； · 资深工程师的竞争力在于脑中一套隐式的 process reward：知道何时查 timeline、何时跑微基准、该对比哪一层的输出。 Z AI 的解法是把这套隐式经验显式化为分层验证接口，即密集反馈，分三类信号，正确性（算得对不对）、系统行为（时间花在哪）、性能（哪个方案赢、在什么条件下赢）。每个信号必须满足三个标准：局部、廉价、客观可验证。这本质上是给 Agent 设计环境，而不是给 Agent 补能力。 证据层：三个案例（发现和根因） · KDA 上下文并行路径精度随序列长度漂移：TF32 舍入误差在链式状态矩阵合并中不断累积 · KV transfer 与 DeepEP dispatch 无法重叠：Agent 跨 Python/C++ 边界追调用链，发现节点内路径不释放 GIL · Decode kernel 因分块方式重复计算同一归一化四次：Agent 应用其从 SGLang/FLA/DeepGEMM 阅读 kernel 蒸馏出的 "optimization skeletons" 三个案例各代表一种能力：数值正确性归因、跨语言边界的系统级调试、跨代码库的模式迁移；选材上有代表性，不全是低垂果实。 边界与深层含义 对边界的定义：人类定义目标、搭建反馈环境、审查每一项高风险变更。由此推出工程师角色的转变：从解决问题的人，变成设计反馈的人。 更深一层的含义是：这套分层可验证的反馈环境是“一鱼两吃”，它既是 Agent 的工作环境，也是下一代模型的训练素材（每个已完成的任务都成为后继者的训练场），环境设计与数据飞轮合二为一。结语收敛到：“离递归自我改进还很远，但最小的闭环已经存在；模型优化系统，系统服务模型。” jietang @jietang Two weeks. That's how long it took to go from GLM-5.3-Flash's first run on domestic accelerators to serving all of its production traffic, with 3.2× end-to-end throughput along the way. What I keep thinking about is who did much of the work: an Infra Agent powered by GLM-5.3. A model helping optimize the system that serves it. The conditions were hard. Limited memory and interconnect bandwidth. 1M-token context. Multimodal requests. An immature software stack where kernels were missing and documentation was often guesswork. Every optimization was a trade: compute for memory (ReplaySSM), communication for memory (intra-node tensor parallelism), precision for capacity (mixed INT8/FP8/BF16 caching), and disaggregation for scheduling freedom (Encode–Prefill–Decode). But the most important lesson wasn't about any single optimization. When the agent got stuck, it was rarely because it couldn't write the code. It was because it didn't know *why* things got worse. "Throughput down 20%" tells you something broke. It doesn't tell you which layer, which hypothesis, or what to test next. In RL terms, it's a sparse reward with a credit assignment problem. And an end-to-end benchmark that takes hours makes exploration painfully slow. Senior engineers solve this with an implicit process reward in their heads. They know when to check the timeline, when to run a microbenchmark, and which layer's output to compare. So we made that explicit. We call it dense feedback: layered verification interfaces the agent can call directly. Correctness feedback: did it compute right? System behavior feedback: where did the time go? Performance feedback: which option wins, under which conditions? Each signal has to be local, cheap, and objectively verifiable. Three things the agent found: First, precision drift in KDA's context-parallel path that grew with sequence length. The cause was TF32 rounding error compounding through chained state-matrix merges. The fix is now merged upstream in Flash Linear Attention (PR #1180 ). Second, KV transfer never overlapped with DeepEP dispatch. The agent followed the call chain across the Python/C++ boundary and found that the intranode path never released the GIL. After the fix, transfer overhead fell from over 30% to under 1%. Third, a decode kernel recomputing the same normalization four times because of how it was chunked. The agent restructured it and got a 1.71× speedup. The idea came from "optimization skeletons" it had distilled by reading existing kernels across SGLang, FLA, and DeepGEMM. To be clear about the boundaries: humans still defined the goals, built the feedback environment, and reviewed every high-risk change. But the engineer's role is changing, from the person who solves the problem to the person who designs the feedback. There's a deeper implication too. A layered, verifiable feedback environment built on real infrastructure tasks is exactly what training the next generation of models needs most. Every task the agent completes can become training ground for its successor. We are still far from recursive self-improvement. But the smallest loop now exists. The model optimizes the system. The system serves the model. 🔗 View Quoted Tweet 💬 0 🔄 0 ❤️ 0 👀 398 ⚡