NVIDIA 发布 Nemotron-Labs-Sigma:扩散语言模型一次并行生成整块 token
NVIDIA 把扩散模型搬进了语言生成,Nemotron-Labs-Sigma 一次并行吐出一块 token,数学任务 8 个 TPF、代码任务 4 个。14 个 3B/8B 权重已开放在 Hugging Face,想试试非自回归路线或做蒸馏可控生成的开发者可以上手玩玩。
NVIDIA 发布开源模型系列 Nemotron-Labs-Sigma,提供 14 个 3B/8B 尺寸的开放权重 checkpoint。
与主流自回归模型逐 token 生成不同,该系列将视觉领域的连续扩散方法引入语言建模,以 16 维平滑 ODE 轨迹并行生成整块 token。论文(arXiv:2610.
02665)展示了扩散机制在蒸馏与可控引导上的优势,训练了面向数学的 8 个 TPF 与面向代码的 4 个 TPF。并行生成带来更快的推理速度,同时保持准确性与输出多样性,适合在数学和编程任务上并行探索多个解题路径。
所有权重已在 Hugging Face 上开放,开发者可直接下载微调。
Language generation beyond one token at a time. 💬 NVIDIA Nemotron-Labs-Sigma generates blocks of tokens in parallel, opening the door to faster generation for exploring multiple solutions to math and coding problems. Read the paper: arxiv.org/abs/2610.02665 v Try the open models: huggingface.co/collections/nv… Q Morteza Mardani @MardaniMorteza 📣 NVIDIA is releasing a collection of open model weights: Nemotron-Labs-Sigma — 14 checkpoints at 3B/8B. Language models have stayed discrete. We bring continuous diffusion to language as in vision. Smooth ODE trajectories (16-D) unlock distillation and steering: 8 TPFs for math, 4 TPFs for code, with accuracy and diversity preserved. 🧵 1/ 🔗 View Quoted Tweet📊 51 ⚡
This report is categorized under AITOP's "Models" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include Nemotron-Labs-Sigma, NVIDIA, diffusion models, open-source models.
Source attribution: curated from NVIDIA AI (@NVIDIAAI).