ModelsSelected
Together 优化推理栈，StepFun step 5 吞吐量提升超 3 倍
Why we picked it
Together 把 StepFun 的 step 5 推理吞吐拉高了 3 倍多，还公开了用智能体逐项优化 MoE 路由和投机解码的过程，做推理优化的可以看看。
Together 针对.StepFun 的 step 5 模型优化了服务端推理栈，吞吐量提升超过 3 倍，并达到了.StepFun 设定的 TTFT（首 token 延迟）和 TPS（每秒 token 数）目标。优化过程由 AI 智能体辅助搜索，覆盖 MoE 路由、调度、内核、投机解码和稀疏注意力五个方向。每一项改动都先经过基准测试验证，再进入下一项。
Original · Together AI
step 5 throughput more than tripled as we optimized the serving stack against stepfun’s ttft + tps targets
agents helped search across moe routing, scheduling, kernels, spec decode + sparse attention, with every change benchmarked before moving to the next https://t.co/TTbLH9aV7I
