ProductsSelected

Together Compiler 背后的技术复盘:推理优化系统如何做到 2.6 倍吞吐

Why we picked it

Together Compute 把自家推理优化工具的研发过程写成了复盘,讲清了 megakernel 哪些假设不成立、Torch Compile 为什么意外能打。做推理部署或想榨干 GPU 吞吐的工程师能直接从中借鉴调优思路,比单纯看产品公告有料得多。

Together Compute 发布 Together Compiler 两周后公开了开发复盘。该系统面向自家推理服务,通过智能体自动调整推理配置以匹配特定模型、芯片和流量模式,官方声称最高可将吞吐提升至 2.6 倍。

Body

复盘披露了几条关键经验:团队在实验中发现对 megakernel(将算子融合为单一大内核)的原有假设并不完全成立;同时 Torch Compile 在部分场景下表现出超出预期的效果,成为优化管线中值得重视的基线。文章记录了这些假设被推翻的过程,以及如何据此设计下一代推理优化系统。

对开发者而言,这篇复盘的价值在于揭示自动调优与手工内核优化的权衡:megakernel 路线强调极限融合,而 Torch Compile 这类成熟工具链仍能在许多实际负载中保持竞争力。用 Together 或自建推理服务的团队可以据此判断编译优化工具的适用边界。

Original · Together AI

Two weeks ago we announced Together Compiler: agents that tune inference to your model, chip, and traffic, for up to 2.6x more throughput.

Today, we’re sharing the journey behind it: the experiments that challenged our assumptions of megakernels, the surprising strengths of Torch Compile, and the lessons that shaped our development of a next-generation system for inference optimization.

Shout out to @dylan__lim @reyna_abhyankar @simran_s_arora and the team!

Read more: https://t.co/9Yb1toI5jq

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Products" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Together Compiler, Torch Compile, inference optimization, megakernel.

Source attribution: curated from Together AI (@togethercompute).