D3 decision models top Jev Decision Index with vLLM recipes
Decision models that return scores and classifications instead of long generations can cut serving costs for routing, moderation and scoring workloads. Teams running their own inference stack can start with a vLLM nightly command and the merged recipes to benchmark D3 on their own hardware, including AMD MI325X GPUs. Self-hosting keeps data residency and latency budgets under direct control.
The vLLM Semantic Router team, led by Xunzhuo Liu, has merged vLLM serving recipes for all six D3 models, ranging from 0.6B to 27B parameters. D3 ranked first on both Jev Decision Index 0.3.2 boards, scoring 66.3 on text and 71.3 on vision in the 12 Oct snapshot. In the recipe benchmark, D3 27B delivered 1.8x the requests/sec of its checkpoint server on a single AMD MI325X at 64 concurrent clients. The models return choices, yes/no answers and scores with probabilities over text, JSON, images and videos, served via the /v1/systemone API alongside Clef-Flash, Clef-Omni and Liquid d1.
Great work from @XunzhuoLiu and the vLLM Semantic Router team on D3. All six models, from 0.6B to 27B, now have merged vLLM recipes!
D3 ranked #1 on both Jev Decision Index 0.3.2 boards: 66.3 text and 71.3 vision (Oct. 12 snapshot).
Run multimodal decisions on your own infrastructure through /v1/systemone—with control over data residency, latency budgets and governance.
vLLM serves decision models too: the full D3 family, Clef-Flash, Clef-Omni and Liquid d1, through the System One API.
D3 returns choices, yes/no answers and scores with probabilities over text, JSON, images and videos. Useful for routing, classification and scoring without generating a long response.
In the recipe benchmark, D3 27B delivers 1.8× the requests/sec of its checkpoint server on 1× MI325X at 64 concurrent clients, using the same mixed workload.
Start with a current vLLM nightly: vllm serve vllm-sr/d3
Recipes for all six sizes: https://t.co/qGDlWJy3bv
API and supported models: https://t.co/L071eXHP7T
