1T参数的开源多模态模型，百万token上下文，vLLM直接跑，4×GB200飙到380 tok/s，值得试试。
Thinking Machines Lab发布了TML Inkling，一个1T参数的开源权重模型，支持文本、图像和音频多模态输入。其上下文窗口高达100万token，采用相对注意力、短卷积和MoE专家汇聚的新架构，并配备8个MTP头用于推测解码。vLLM已从首日支持该模型，在NVIDIA Blackwell和Hopper上运行，4×GB200 MTP可达380 tok/s/user。
🎉 Congrats to @thinkymachines on TML Inkling—a 1T-…
🎉 Congrats to @thinkymachines on TML Inkling—a 1T-parameter open-weight model supported in vLLM from Day 0.
Highlights: • Natively multimodal across text, image, and audio • Up to 1M-token context • New architecture with relative attention, short convolutions, and MoE expert sinks • 8 MTP heads for speculative decoding
vLLM supports both NVFP4 and BF16 checkpoints, optimized for NVIDIA Blackwell and Hopper, reaching up to 380 tok/s/user on 4× GB200 with MTP.
Huge thanks to the Thinking Machines Lab team for the close collaboration 🙏
Read about implementation below 👇 https://t.co/qsPIFoY6SK