Apple 提出 Weighted State Souping，改进 SSM 长上下文检索
Weighted State Souping for Efficient Long-Context Retrieval
Apple 这篇 NeurIPS 2026 PALM 工作坊论文把 state souping 加了权重，专治 SSM 在长上下文里检索变差的老毛病。做 RAG、长期记忆或 agent 上下文缓存的工程师可以看看，和标准 Transformer 检索思路很不一样。
Apple 的研究论文 Weighted State Souping 被 NeurIPS 2026 的 PALM 工作坊接收，该工作坊聚焦个性化、对齐、长期记忆的 AI 系统。论文针对状态空间模型（SSM）在长、拥挤上下文下检索性能下降的问题提出改进方案。
State souping 是一种流程：SSM 独立编码文档、工具定义或记忆记录等 token 块，生成的 state 编码可缓存并在下游查询时重新组合，从而低成本构建上下文。Apple 的 weighted 变体在此基础上引入权重，缓解 vanilla SSM 和普通 state souping 在长上下文中的退化。
SSM 的价值在于推理速度快且 state 大小固定，同时保持有竞争力的语言建模表现；这项工作指向 SSM 用作长期记忆系统的检索路径，对构建带缓存上下文的 agent 应用有参考意义。
Weighted State Souping for Efficient Long-Context Retrieval
This paper was accepted at the Workshop on Personalized, Aligned, Long-term Memory for AI Systems (PALM) at NeurIPS 2026. State space models (SSMs) combine fast inference and fixed state size with competitive language-modeling performance. Recently, state souping has been proposed as a procedure whereby an SSM independently encodes chunks of tokens like documents, tool definitions, or memory records. The resulting state encodings may be cached and recombined to cheaply construct a context for downstream queries. However, both state souping and vanilla SSMs deteriorate on long, crowded…
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include SSM, State Souping, Apple, RAG.
Primary source verification: published directly by Apple ML Research with first-party authority.
