百度新出的 Unlimted OCR 用了一种叫 R-SWA 的注意力机制，让它处理几十页文档时不会变慢，内存占用也恒定。想做长文档 OCR 的可以试试。
Unlimited OCR 模型以 DeepSeek OCR 为基线，将所有解码器注意力层替换为 Reference Sliding Window Attention (R-SWA)，使解码过程中 KV 缓存保持恒定，不再随输出长度增长。在标准最大长度 32K 下，Unlimited OCR 可一次性转录数十页文档。相比传统端到端 OCR 模型，Unlimited OCR 解决了长序列中内存和速度下降的问题。R-SWA 是一种通用解析注意力机制，还可应用于 ASR、翻译等任务。代码和权重已在 GitHub 开源。
Unlimited OCR Works
Recently, end-to-end OCR models, exemplified by DeepSeek OCR, have once again thrust OCR into the spotlight. A widely held view is that employing a large language model (LLM) as the decoder allows the model to leverage the prior distribution of language, leading to improved OCR performance. However, the downside is equally evident: as the output sequence lengthens, the accumulated KV cache drives up memory consumption and progressively slows down generation. This stands in stark contrast to humans, who exhibit no such decline in efficiency during long-horizon copying tasks. In this technical report, we propose Unlimited OCR, a model designed to emulate human parsing working memory. Taking DeepSeek OCR as the baseline, we replace all attention layers in the decoder with our proposed Reference Sliding Window Attention (R-SWA), which reduces attention computation costs while maintaining a constant KV cache throughout the entire decoding process. By combining the high compression rate of DeepSeek OCR's encoder with our constant KV cache design, Unlimited OCR can transcribe dozens of pages of documents in a single forward pass under a standard maximum length of 32K. More importantly, R-SWA is a general-purpose parsing attention mechanism - beyond OCR, it is equally applicable to tasks such as ASR, translation, etc. Codes and model weights are publicly available at http://github.com/baidu/Unlimited-OCR.