ThinkBooster:统一框架实现LLM推理的测试时计算扩展

ThinkBooster: A Unified Framework for Seamless Test-Time Scaling of LLM Reasoning

精选理由

做LLM推理优化的开发者终于有了一个标准化工具来对比不同TTC策略的成本收益,不用再自己拼凑评估流程,建议直接试。

AI 摘要

ThinkBooster 是一个统一的测试时计算(TTC)扩展框架,旨在解决现有TTC策略和评分器碎片化、评估不一致的问题。它包含模块化Python库、联合评估性能与效率的基准测试,以及兼容OpenAI的代理服务,支持自适应推理的即插即用。在数学和编程任务上的实验揭示了性能与计算成本的权衡,并展示了实际增益。代码以MIT许可证开源。

原文 · arXiv: OpenAI

ThinkBooster: A Unified Framework for Seamless Test-Time Scaling of LLM Reasoning

Test-time compute (TTC) scaling has emerged as a powerful paradigm for improving large language model (LLM) reasoning by allocating additional compute during inference, e.g., via multi-sample generation and verifier-based reranking. Existing TTC scaling strategies and reasoning scorers remain fragmented, evaluated under inconsistent protocols, and are rarely analyzed through the lens of quality-cost trade-offs. We introduce ThinkBooster, a unified framework for seamless test-time compute scaling of LLM reasoning, which consists of (i) a modular Python library implementing state-of-the-art TTC scaling strategy and scorer families, (ii) a benchmark that jointly evaluates performance and computational efficiency, and (iii) a deployable OpenAI-compatible proxy service that enables drop-in integration of adaptive reasoning into real-world applications. We further provide a demo visual debugger for inspecting the reasoning trajectories, intermediate selection decisions, and alternative reasoning paths. Empirical results on mathematical and coding tasks reveal the performance-compute trade-offs of TTC scaling strategies and scoring methods and demonstrate that ThinkBooster provides practical gains in real-world tasks. The code is available online under an MIT license.