AITOPAI热报
Papers72°

腾讯混元提出 Stem 稀疏注意力算法，首字延迟降低 3.6 倍

IT之家
Blog/Media
Read the original
Why we picked it

长文本推理的延迟痛点终于有了低成本解法——Stem 用 25% 算力实现近无损精度，做 LLM 推理优化的团队可以直接用开源代码实测，128K 上下文下首字延迟降低 3.6 倍的效果值得关注。

腾讯混元团队提出 Stem 稀疏注意力算法，已被 ICML-26 收录。该算法通过 Token 位置衰减和输出感知度量两大创新，仅用 25% 算力即可逼近稠密注意力的精度。

Body

配套的 HPC 算子库将理论加速转化为实际性能，在 128K 上下文下首字延迟降低 3.6 倍。

该方案为长文本推理场景提供了高效、低成本的注意力加速方案，相关论文和代码已开源。

Original · IT之家

IT之家 6 月 5 日消息，腾讯混元今日宣布提出 Stem 稀疏注意力算法，已被机器学习顶会 ICML-26 收录。 官方表示，Stem 稀疏注意力算法从“因果信息流”重新审视块级稀疏，用 Token 位置衰减（TPD）和输出感知度量（OAM）两大创新， 仅用 25% 算力就逼近稠密注意力的精度 。配套的 HPC 算子库则将这份理论加速比真正转化为端到端的实测性能。 ▲ Stem 在 Hy3 preview（W8A8-FP8）上更贴近生产环境的真实落地数据 根据 Stem 算法 × HPC 算子的全栈加速方案，算法层面，Stem 通过 Token 位置衰减（TPD）和输出感知度量（OAM）实现 25% 预算下的近无损精度 ；算子层面，HPC 开源的 Stem+BSA 算子将稀疏收益转化为真实硬件加速， 128K 上下文下首字延迟降低 3.6 倍 。 ▲ 模型精度 IT之家附相关链接如下： Stem 论文链接 ： https://arxiv.org/abs/2603.06274Stem 开源地址 ： https://github.com/Tencent/AngelSlimHPC 算子开源地址 ： https://github.com/Tencent/hpc-ops

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on China's AI industry and domestic foundation model ecosystem. Key entities and technical topics under continuous tracking include 稀疏注意力, 长文本推理, 腾讯混元, ICML-26.

Source attribution: curated from IT之家.

#稀疏注意力#长文本推理#腾讯混元#ICML-26#开源/仓库

Keep reading

Recent stories with the same tag or section
ModelsBuildersSelected01:13
Zyphra's ZHA hierarchical attention speeds up long-context models

The plug-in design means no retraining of the base model, so teams running open-source models could adopt ZHA by training or downloading a small index module. It matters for anyone serving long-context workloads, where KV-cache reads dominate cost and latency. Compared with MSA, the reported 12.7-point retrieval gain is the concrete benchmark to watch when evaluating sparse attention options.

#Zyphra#ZHA#Qwen#long-context
Gorden Sun@Gorden_SunSource
PapersBuildersSelected11:52
LeanSparse Cuts GPU Memory for Sparse-Attention LLM Serving

The key insight is that prefill memory spikes, not just KV cache size, throttle concurrency — streaming layer histories through reusable buffers removes that admission bottleneck. Engineers running long-context inference on SGLang can adopt the design to serve more users per GPU without changing model weights. Compared with HiSparse and ECHO, the gains come from better indexing rather than more hardware.

#LeanSparse#DeepSeek#SGLang#sparse attention
arXiv: DeepSeek@Zaifeng Pan et al. (6)Source
PapersUS vs ChinaSelected10:50
Multi-layer Indexer Speeds DeepSeek Sparse Attention Decode

The indexer scan, not sparse attention itself, turns out to be the long-context bottleneck, and reusing selections across layers removes it without any retraining. Teams serving DeepSeek-V3.2 or GLM-5 long-context workloads on SGLang can adopt MLI for faster decode at unchanged accuracy, and it composes with the IndexShare reuse GLM-5.2 already ships.

#DeepSeek#DeepSeek Sparse Attention#GLM-5#SGLang
arXiv: DeepSeek@Serdarcan Dilbaz et al. (4)Source
PapersSelected14:29
AttentionIndex Speeds Learned Sparse Attention on GLM-5.3 and DeepSeek-V4-Flash

The method cuts indexer cost, which accounts for over half of prefill time at one million tokens, without the recall degradation that two-stage block approaches like HiSA suffer as context grows. Serving teams running long-context inference on models with learned sparse attention, such as GLM-5 or DeepSeek-V4, can use it to raise prefill throughput 1.3-1.8 times while preserving retrieval accuracy. Both target models come from Chinese labs, making it directly relevant to practitioners deploying these open-weight models.

#AttentionIndex#GLM-5#DeepSeek-V4#sparse attention
arXiv: DeepSeek@Jeongmin Brian Park, Wen-mei HwuSource