ACTS解决了LLM推理中token浪费和缺乏控制的问题,做推理优化或部署大模型的开发者可以直接用开源代码尝试,实现更经济的推理。
ACTS提出了一种新方法,通过智能体控制器自适应地引导冻结的推理模型,在推理过程中动态调整思考策略和预算,从而在保持生成连续性的同时大幅节省token。该方法将推理引导建模为马尔可夫决策过程,控制器根据推理轨迹和剩余预算发出策略动作。实验表明,ACTS在全思考性能下实现了显著的token节省,并支持不同推理器和任务间的可控精度-效率权衡。代码已开源。
Agentic Chain-of-Thought Steering for Efficient and Controllable LLM Reasoning
Large language models improve final-answer accuracy through extended chain-of-thought reasoning, but often spend tokens inefficiently and offer little inference-time control. Existing efficient reasoning methods control thinking length by shortening, early-stopping, or compressing traces, leaving how the model thinks implicit. In this paper, we propose Agentic Chain-of-Thought Steering (ACTS), which formulates reasoning steering as a Markov decision process where a controller agent adaptively steers a frozen reasoner during inference. At each step, the controller observes the reasoning trace and remaining thinking budget, then issues a steering action consisting of a reasoning strategy and a steering phrase that initiates the next reasoner step. This enables budget-aware strategy control for efficient reasoning while preserving the reasoner's generation continuity. We initialize the controller agent from our constructed synthetic steering trajectories with multi-budget augmentation, and further optimize it via reinforcement learning with budget-conditioned reward shaping. Experiments across multiple benchmarks show that ACTS matches full-thinking performance with substantial token savings, and enables controllable accuracy-efficiency trade-offs across different reasoners and tasks. The code is available at https://github.com/Andree-9/ACTS.