billion-context: agent plugin compresses context for month-long sessions
The interesting move is giving the model agency over its own memory: compression happens only when the model judges old turns are disposable, which reduces the accuracy loss typical of aggressive auto-summarisation. Developers building long-horizon agents or coding assistants that run for weeks on 100K-token models can drop it in as a proxy layer rather than rewriting prompts. The GitHub release makes it directly testable against naive summarisation baselines.
billion-context is a client-side agent plugin that sits between a client app and an AI model to extend long-running conversations far beyond the model's context window. Its author claims that with it enabled, a single long session can run stably for up to a month, processing billions of tokens without overflowing a 100K-token window, cutting total token consumption to about one-fifth of the original while keeping prefix-cache hit rates above 95%.
Mechanically, it works as an intelligent proxy layer that injects four dedicated context-management tools into the conversation: compress, decompress, search_context, and acp_status. As the conversation grows past a threshold, the model itself decides which older details can be packed away and issues a compress call, folding a stretch of old dialogue into a dense summary. When later turns need a specific earlier parameter or technical detail, the model calls decompress to expand the folded segment losslessly, or uses search_context to retrieve the relevant content.
The approach shifts context management from fixed system prompts into tool calls the model controls, letting sessions survive on small windows. Project source is on GitHub.
billion-context：节省上下文的Agent插件
billion-context充当客户端和AI模型之间的智能中继站， 开启之后，它能让单次长会话稳定运行长达一个月，处理数十亿token的数据量而不会撑爆100K的小窗口，整体消耗节省至原来的五分之一。对话中较早的历史内容会被自动折叠压缩，需要细节时又能随时无损还原，在保证AI回复准确度的同时，还能维持95%以上的前缀缓存命中率。
核心原理是在客户端与大模型之间架设一层智能代理，通过让模型自主调用压缩工具，实现动态的分层摘要与无损还原。billion-context向对话上下文中注入了四个专用的上下文管理工具：压缩（compress）、还原（decompress）、搜索（search_context）以及状态查询（acp_status）。
在实际运行中，代理服务器会根据对话增长的节奏提示模型。当上下文累积到一定阈值时，大模型会自己判断哪些过去的对话细节可以打包，主动发起压缩指令，将一大段旧对话折叠成高密度的信息摘要。
当后续对话需要确认之前的具体技术细节或参数时，模型也可以调用还原工具，精准展开被折叠的片段，或者通过搜索工具检索对应内容。
Github：https://t.co/KpGOOoRTMt
This report is categorized under AITOP's "Products" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include billion-context, agents, context compression, open-source models.
Source attribution: curated from Gorden Sun (@Gorden_Sun).
