Together AI Launches Together Compiler for Polymorphic Inference
Automated compilation that replaces weeks of per-chip hand tuning could lower inference costs for anyone serving open models without a dedicated kernel team. Teams running self-hosted inference should evaluate whether the overnight optimization workflow fits their stack, especially against vLLM- and SGLang-based serving setups. Pricing and regional availability were not disclosed in the announcement.
Together AI announced Together Compiler, described as its first step toward what it calls Polymorphic Inference. The product performs continual optimization, and the company claims a single overnight run can match or beat manual expert tuning that previously took weeks. The announcement was made on X by Together AI, with the full vision written by its SVPs of Research, Simran Arora and Dan Fu. The pitch targets the pace of weekly model releases, daily traffic shifts, and yearly hardware turnover that make one-time static tuning impractical.
Inference is entering a new era. Models ship weekly, traffic shapes shift daily, and hardware turns over every year. The era of Static Inference, tuned once for one model with one traffic shape on one chip, is ending.
Welcome to the era of Polymorphic Inference: inference that adapts to new models, workloads, and hardware as they arrive.
Today we’re announcing Together Compiler, our first step into that era. It enables continual optimization, so a single overnight run can match or beat the manual expert tuning that used to take weeks.
Read the full vision from our SVPs of Research, @simran_s_arora and @realDanFu, on Polymorphic Inference and our research roadmap 👇
https://t.co/kqBbu0nDXZ
