Primary sourceshao__meng
View sourceEvent ·Multi-source
Cohere 开源 Megakernel 推理系统
Cohere 为 North Mini Code 模型构建了基于 decode megakernel 的推理服务系统。在单张 H100 上，batch size 1 时达到 292 tok/s，比 vLLM 快 1.58 倍。该系统利用 H100 显存带宽理论值的 62%，端到端服务在真实基准上快 1.25-1.41 倍。
Current read
Cohere 开源了比 vLLM 快 1.58 倍的 Megakernel 推理系统，解决了传统引擎在算子边界处的 GPU 空转问题。
3 sources78° AI heatLast updated 2026/9/8 23:28:19
Evidence
3 sourcesConflict check
Dedup data only shows that these sources discuss the same event, not that they agree. There is no structured supporting or refuting evidence yet, so no conflict is inferred automatically.