Together Compiler 在 B200/B300 上推理吞吐最高提升 2.7 倍
Together 的推理编译器直接拿 InferenceX 最优配置当对手，跑 Kimi K3 吞吐翻 2.7 倍。做推理部署或买 GPU 算力的团队值得跑一遍自己的负载看看省多少钱。
Together AI 发布 Together Compiler，官方基准显示在与 InferenceX 最优配置相同的单用户速度下，吞吐最高提升 2.7 倍。测试基于 NVIDIA B200 和 B300 GPU，覆盖四个模型：Kimi K3 提升 2.7 倍，GLM-5.3 和 DeepSeek V4.1 Flash 各提升 1.8 倍，MiniMax M3.1 Flash 提升 1.5 倍。
这次对比的基准线是 InferenceX 的最优配置，而非普通默认设置，意味着提升来自编译层面的优化而非简单的超参调整。四个被测模型全部是中国团队的开源权重模型（Kimi、GLM、DeepSeek、MiniMax），Together 用美国自研推理栈直接对标这些模型的常见部署方案。
对自建推理或采购推理服务的企业来说，Kimi K3 的 2.7 倍数字最突出，说明这类编译优化对大参数模型的收益更明显；实际收益仍取决于具体模型的 KV cache 策略与批处理场景，建议以自身负载复测。
Up to 2.7× more throughput at the same speed per user.
In our benchmarks comparing Together Compiler vs. the best InferenceX configs on NVIDIA B200 and B300:
→ Kimi K3: 2.7× → GLM-5.3: 1.8× → DeepSeek V4.1 Flash: 1.8× → MiniMax M3.1 Flash: 1.5×
Read more: https://t.co/wnRX2rWDnx
This report is categorized under AITOP's "Products" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to US-China foundation model benchmarks and commercialization comparison and AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Together AI, Kimi K3, DeepSeek, 推理优化.
Source attribution: curated from Together AI (@togethercompute). Cross-source corroboration: 11 independent sources have covered this development (including NVIDIA AI, rohanpaul_ai, TestingCatalog), confirming industry-wide signal strength.
