Latent Semantic Memory Compresses Agent Memory Into 16 Tokens
The result suggests frozen LLMs can accumulate and reuse knowledge over long interactions without retraining, pointing toward cheaper continually learning systems. Developers building RAG pipelines or agent memory should compare this compression approach with Mem0 and MemoRAG before defaulting to full-context retrieval, since it wins most head-to-head benchmarks at a fraction of the token cost.
A new paper (arXiv 2610.21375) introduces a memory approach that encodes documents once into reusable embeddings, with a learned module writing relevant information into 16 input tokens for a frozen LLM. On FactKG, the 16 memory tokens score 84.33%, beating full-context inference at 80.43% while using 108x fewer source-document tokens. The method outperforms Mem0, MemoRAG and other baselines on five of eight benchmarks. As agent memory in TextWorld, using the agent's own history raises success from 18.6% to 24.2% with fewer room revisits.
// Latent Semantic Memory for LLMs // This paper presents some really interesting findings about agent memory. They find that sixteen memory tokens beat full-context inference on FactKG while using 108x fewer tokens for the source documents. Approach: Documents are encoded once into reusable embeddings. For each query, a learned module gathers the relevant information from the whole collection and writes it into 16 tokens in the input space of a frozen LLM. Results: It beats Mem0, MemoRAG, and other baselines on five of eight benchmarks. On FactKG, it scores 84.33% against 80.43% with the full context. It also works as agent memory. In TextWorld, using the agent's own history as the collection raises success from 18.6% to 24.2%, and the agent revisits fewer rooms. It also seems that it could help frozen LLMs accumulate and use knowledge effectively across long-context interactions. This is applicable to building continually learning AI systems. arxiv.org/abs/2610.21375 Chat with Paper: academy.dair.ai/papers/memoril… 💬 9 🔄 3 ❤️ 14 👀 1649 📊 11 ⚡
