LlamaIndex 优化 OCRia-1 推理，单块 B200 token 生成提速近 10 倍
LlamaIndex 把 OCRia-1 在单块 B200 上跑快了近 10 倍，17 个小优化叠加出来的，还开源回馈 vLLM，做推理优化必看。
LlamaIndex 基础设施团队针对文档解析场景调优了 OCRia-1 的运行方式，在单块 NVIDIA B200 上让 token 生成速度提升近 10 倍。团队对每项改动同时测试速度与准确率，累计落地 17 项小优化，单项提升多在 2–5% 之间。部分优化计划回馈给开源 vLLM 项目。完整过程已发布在团队博客中。
We got OCRia-1 generating tokens nearly 10× faster on a single B200. Getting there meant making much better use of the same GPU. We tuned how OCRia-1 runs for the demands of document parsing, testing every change for both speed and accuracy. Some improvements were only 2–5%. Seventeen smaller optimizations later, “just a few percent” starts adding up. Our infrastructure team wrote up what worked, how they tested the changes, and what they learned about making document models run faster. We also plan to contribute some of these optimizations back to the open-source vLLM project. Read the full breakdown → llamaindex.ai/blog/the-infer… Watch the video below to see OCRia-1’s token generation before and after the optimizations. Your browser does not support the video tag. 🔗 View on Twitter 💬 2 🔄 0 ❤️ 1 👀 412 📊 2 ⚡
