inferact 开源 Kimi K3 TPU megakernel，单核跑出 709 tokens/s
Kimi K3 在 TPU 上跑到 709 tokens/s，比 GB200 还快，92 个 MoE 层塞进一个 Pallas kernel，做法挺巧，做推理的可以看看。
inferact 团队为 Kimi K3 开源了一个 TPU megakernel，推理速度达到 709 tokens/s，高于 GB200 上的 450 tokens/s。
K3 全部 92 个 MoE 层都在单个 Pallas kernel 中运行。该实现采用跨层边界的权重预取，让一层的权重传输与上一层的计算重叠进行。
代码仓库和详细写文已发布。
The @inferact team open sourced a TPU megakernel for Kimi K3 achieving 709 tokens/s, against 450 tokens/s on GB200.
All 92 of K3's MoE layers run in a single Pallas kernel, with weight prefetching that reaches across layer boundaries so transfers for one layer overlap with computation in the previous one.
Shoutout to the team! Writeup and repo link in the thread.
This report is categorized under AITOP's "Models" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include Kimi K3, TPU, Pallas, MoE.
Source attribution: curated from vLLM (@vllm_project). Cross-source corroboration: 7 independent sources have covered this development (including Thomas Wolf, eric zakariasson, IT之家), confirming industry-wide signal strength.
