本地大模型用户终于可以突破内存带宽瓶颈了——MTP 让 Qwen 27B 速度翻倍还零精度损失，跑本地模型的开发者建议直接去 GitHub 试。
atomic.chat 展示了 Multi-Token Prediction（MTP）技术，让本地运行的 Qwen 27B 密集模型从 51 tokens/s 提升到 117 tokens/s，MoE 35B-A3B 模型在 2x RTX 5090 上从 218 提升到 267 tokens/s。MTP 通过一次生成并验证多个未来 token，减少 GPU 重复读取模型权重的次数，从而突破内存带宽瓶颈。测试中约 80% 的 draft token 被接受，且零精度损失，仅额外占用约 1GB VRAM。该项目完全开源，对本地大模型部署者是个重大利好。
Another good news for local-LLM from atomic[.]chat…
Another good news for local-LLM from atomic[.]chat, that runs 100% offline on your computer.
They just showed MTP (Multi-Token Prediction) pushing local Qwen models from 51 to 117 tokens/s on dense 27B.
And an MoE 35B-A3B model rose from 218 to 267 tokens/s on 2x RTX 5090.
Instead of generating and checking one token at a time, MTP (Multi-Token Prediction) drafts multiple future tokens and verifies them together, so the GPU does less repeated work for every word it prints.
And this makes local LLMs much faster when the draft tokens are accepted often enough.
For many local LLM runs, the limit is not pure compute, but memory bandwidth: how fast the GPU can keep feeding weights into computation.
A local GPU generating text often spends most of its time pulling model weights from VRAM again and again for each token, so if MTP lets the model check several drafted tokens in one forward pass, it reduces how often the same giant weight matrix has to be reread.
The most interesting claim in their test is ~80% draft acceptance with zero accuracy loss and only ~1GB extra VRAM, because speculative decoding often becomes useful only when the draft tokens are accepted often enough.
So we get this strong local AI result because it improves generation speed without changing the model’s answers, but the dense model is the real winner because memory bandwidth was its main bottleneck.
Their GitHub repo is fully open source.