Stanford 与 UC Berkeley 推出 TokenGate：让长上下文 prefill 最快提升 11.3 倍
Researchers from Stanford and UC Berkeley Introduce TokenGate: Token-Gated Attention Makes Long-Context Prefill Up to 11.3x Faster
Stanford 和伯克利搞了个 TokenGate，往模型里一插就能把长文档 prefill 快最多 11.3 倍，内存还省 3 倍多，跑长上下文的可以看看。
Stanford 和 UC Berkeley 的研究人员发布了 TokenGate，一个可即插即用的学习式 token 剪枝模块。它能把长上下文 LLM 的 prefill 首 token 延迟降低 1.4 倍到 11.3 倍，峰值内存减少 1.5 倍到 3.4 倍，且不损失准确率。模块无需改动模型结构，可直接嵌入现有注意力机制中。
Researchers from Stanford and UC Berkeley Introduce TokenGate: Token-Gated Attention Makes Long-Context Prefill Up to 11.3x Faster
Stanford and UC Berkeley researchers released TokenGate, a drop-in learned token pruning module for LLMs. It cuts long-context prefill time-to-first-token by 1.4x to 11.3x and peak memory by 1.5x to 3.4x without accuracy loss. The post Researchers from Stanford and UC Berkeley Introduce TokenGate: Token-Gated Attention Makes Long-Context Prefill Up to 11.3x Faster appeared first on MarkTechPost .
