thinkymachines开源了一个超大MoE模型Inkling，总参975B但只激活41B，还支持1M上下文和图文音频理解，已经可以在SGLang上跑了，想玩大模型的朋友可以试试。
Inkling是thinkymachines发布的首个开源MoE模型，总参数量975B，激活参数41B。它支持最高1M上下文长度，原生处理文本、图像和音频。该模型已获得SGLang和Miles的Day 0支持，采用短卷积、相对位置嵌入注意力和共享专家sink MoE等新架构。推理优化包括DFlash投机解码和MXFP8 KV缓存，并支持全参数和LoRA RL训练。
Inkling, @thinkymachines' first open model, droppe…
Inkling, @thinkymachines' first open model, dropped today: 975B total / 41B active MoE, up to 1M context, reasoning natively over text, images, and audio.
Serving and RL support are already live: you can run and shape it on an open stack, starting now.
Day 0 support on SGLang @sgl_project and Miles @radixark👇
- Inkling's new architecture (ShortConv, attention with relative positional embedding, shared expert sink MoE) is natively implemented and deeply optimized, with prefill full CUDA graph and MXFP8 KV cache
- Full parameter and LoRA RL in a customized Megatron backend, train inference consistency via customized kernels, routing replay, and cross-runtime parameter synchronization
- DFlash speculative decoding from @modal for low-latency serving
Launching now, blog and cookbook in the comments ⬇️