Moonshot AI 开放 Kimi-K3.1-Pro 权重：总参数 2.8T，每 token 激活 104B
Kimi-K3.1-Pro by Moonshot AI: interesting facts It's huge, and sparse • It has 2.8T total parameter...
Moonshot 开放了 Kimi-K3.1-Pro，2.8T 参数只激活 104B，1M 长上下文，BrowseComp 拿了 92.1，权重才 1.56TB，跟 Claude Opus 5.5 掰手腕。
Moonshot AI 发布开源权重模型 Kimi-K3.1-Pro，总参数 2.8T，每个 token 仅激活 104B，与 Kimi-K3 并列为 Hub 上最大的真实模型发布，超过 Qwen3.8-2.4T 和 DeepSeek-V4-Pro（1.6T）。模型共 896 个专家、每 token 路由到 16 个，93 层中大部分使用线性注意力（KDA），每 4 层用 DeepSeek 式稀疏注意力，支持 1M token 上下文。基准方面在 BrowseComp（92.1）、带工具 HLE（70.0）、SWE Marathon（51.7）上居模型卡首位，与 Claude Opus 5.5、GPT-6 Astra 互有胜负，纯推理和网络安全较弱。权重以 MXFP4 训练，仓库约 1.56 TB，BF16 存储则需约 5.6 TB。许可证为修改版 MIT，年收入超 2000 万美元的 API 商用需与 Moonshot 签协议。
Kimi-K3.1-Pro by Moonshot AI: interesting facts It's huge, and sparse • It has 2.8T total parameter...
Kimi-K3.1-Pro by Moonshot AI: interesting facts It's huge, and sparse • It has 2.8T total parameters, but only 104B are active per token. • Along with Kimi-K3, it's the largest real model release on the Hub. The next biggest are Qwen3.8-2.4T and DeepSeek-V4-Pro (1.6T). • It has 896 experts and sends each token to 16 of them. (DeepSeek-V4-Pro has 2 times less and Mistral-Large-4 has 7 times less) Clever attention for a 1M-token context • Most of its 93 layers use linear attention (KDA). Every 4th layer uses DeepSeek-style sparse attention, which only looks at the top 2048 tokens. • This is how it handles a 1M-token context (the same as the other frontier open models) • It uses a custom activation called SiTU-GLU and has a 401M-parameter vision encoder (MoonViT-V2), so it takes both text and images. Small on disk for its size • It was trained to run in 4-bit (MXFP4), so the whole repo is only ~1.56 TB. • Stored in regular BF16 it would be about 5.6 TB(!) Benchmarks (from the model card) • It goes back and forth with Claude Opus 5.5 and GPT-6 Astra. It's top of the card's table on BrowseComp (92.1), HLE with tools (70.0) and SWE Marathon (51.7). • It's weaker on pure reasoning without tools and on cyber security. • Moonshot ran most of their own coding evals using the Claude Code harness. License (a modified MIT) • It's free to use, modify and redistribute. • API businesses making over $20M a year need a deal with Moonshot. • Products with over 100M monthly users or over $20M monthly revenue must display "Kimi K3.1-Pro". On the Hub • It went public about 1.5 hours ago and already has 214 likes, about 9× Kimi-K3's first hour. It's #19 on the trending list. • It isn't gated, but loading it in transformers requires "trust_remote_code" Lisan al Gaib @scaling01 Kimi-K3.1-Pro is now open-weights huggingface.co/moonshotai/Kim… 🔗 View Quoted Tweet 💬 2 🔄 0 ❤️ 2 👀 314 📊 2 ⚡
