LSA 解决了超长上下文推理的 GPU 内存瓶颈，做长文档分析或大规模序列建模的团队可以直接参考其稀疏注意力方案，显著降低部署成本。
FlashMemory-DeepSeek-V4 提出了一种名为 Lookahead Sparse Attention (LSA) 的新型推理范式，通过神经记忆索引器预测未来上下文需求，仅保留关键 KV 块在 GPU 内存中。该架构采用解耦训练策略，将索引器作为独立双编码器训练，无需加载主模型。在 LongBench-v2、LongMemEval 等长上下文评测中，LSA 将物理 KV 缓存压缩至全上下文基线的 13.5%，同时下游准确率平均提升 0.6%。在 50 万 token 极端长度下，物理 KV 缓存开销降低超过 90%，且不损害模型核心推理能力。
FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention
Conventional LLMs keep the full KV cache loaded during decoding, causing a severe GPU memory bottleneck for ultra-long context serving. In this report, we propose Lookahead Sparse Attention (LSA), a novel inference paradigm powered by a Neural Memory Indexer built upon the DeepSeek-V4 architecture. Rather than passively attending to all historical tokens, LSA proactively predicts future context demands and preserves only the query-critical KV chunks in the GPU memory. Crucially, we instantiate this architecture via a backbone-free decoupled training strategy. By formulating the indexer as a standard dual-encoder architecture, we train it independently using standard retrieval training frameworks without ever loading the massive backbone model into GPU memory. We demonstrate that this "less is more" paradigm significantly maximizes serving efficiency while acting as an effective attention denoiser in tasks that rely on long-term global memory. Across primary long-context evaluation suites (e.g., LongBench-v2, LongMemEval, and RULER), FM-DS-V4 compresses the average physical KV cache footprint down to merely 13.5% of the full-context baseline, while consistently preserving or slightly elevating downstream accuracy (+0.6% absolute margin on average). Crucially, at extreme 500K scales, FlashMemory suppresses the physical KV cache overhead by over 90% without destabilizing the backbone's core reasoning capacities.