vLLM 第一时间支持 Thinking Machines 的 Inkling-2-Small 模型
Thinking Machines 发了个 500B 的多模态 MoE，vLLM 发布当天就能跑，还带了 FP4 量化和投机解码优化。想自部署大模型的人可以看看集成细节，不用等闭源 API。
vLLM 宣布 Day-0 支持 Thinking Machines Lab 的新模型 Inkling-2-Small。这是一个 500B 参数的 MoE 模型，支持文本和图像输入、文本输出，可以在同一次对话中传入图像并提出问题，由模型结合视觉上下文生成文本回答。
技术实现上有几个关键点：模型采用全新的 ViT 图像编码器，MoE 层混合使用 NVFP4 和 MXFP8 两种低精度格式，并引入 DSpark 投机解码来加速生成。集成工作由 Inferact 主导，团队与 Thinking Machines Lab 协作处理了模型代码和 checkpoint，覆盖 MoE 与 attention 内核、视觉编码器、图像预处理，以及投机解码所需的 draft-model GEMM。
Thinking Machines Lab 由前 OpenAI CTO Mira Murati 创立，此次选择 vLLM 作为开源推理栈的首发支持对象，意味着开发者可以在模型发布当天直接用开源工具链部署这个 500B 多模态 MoE，而不用等待闭源推理服务。
🎉 Day-0 support for @thinkymachines’ Inkling-2-Small in vLLM!
A 500B MoE with text + image inputs and text output. Bring the image into the conversation: ask a question with visual context, and get a text response from the same model.
Under the hood: a new ViT image encoder, mixed NVFP4/MXFP8 MoE layers, and DSpark speculative decoding.
@inferact leads the vLLM integration and optimization, working with Thinking Machines Lab on model code and checkpoints. The work spans MoE and attention kernels, the vision encoder and image preprocessing, plus draft-model GEMMs for speculative decoding.
Thanks to the TML, Inferact and vLLM teams bringing the model to an open inference stack!
This report is categorized under AITOP's "Models" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include Inkling-2-Small, vLLM, Thinking Machines Lab, MoE.
Source attribution: curated from vLLM (@vllm_project). Cross-source corroboration: 11 independent sources have covered this development (including Together AI, Thinking Machines Lab, pandaily), confirming industry-wide signal strength.
