Bellman Language Models：把 LLM Agent 决策当 MDP 一次性求解
Bellman Language Models: Solving the Recurring Decisions of LLM Agents
把 Agent 反复遇到的决策当成 MDP 求解一次然后复用，Qwen3.8-27B 在 τ²-bench 上从 29% 冲到 96%，成本降 98%，思路挺新的。
论文提出 Discretize, Then Decide 方法，将 Agent 工作流日志中的对话离散化为状态，估计各动作的概率与成本，再求解马尔可夫决策过程。在 τ²-bench telecom 上，无模型的控制器可解决 79% 任务；开源模型 Qwen3.8-27B 以 BLM 方式运行后成绩从 29% 提升到 96%，模型调用减少 93%。Claude Opus 5.5 本身已解决 96% 任务，作为 BLM 运行时每任务调用与成本约降 98%，还多解决 2 个任务。在 InventoryBench 上，求解后的策略（不调用模型）超过所有已发表方法。但动作内容本身难以预先写定的场景（如解决 SymPy issue），BLM 帮助有限。
Bellman Language Models: Solving the Recurring Decisions of LLM Agents
A general observation in agentic workflows, especially bounded ones, is that the decision space of the LLM, when we separate a decision from generation, starts approaching saturation: the agent meets the same states again and again, and after enough conversations it stops meeting new ones. LLMs generally don't treat these decisions as an optimization problem and instead re-derive them from their context/prompt on every turn. We show that treating decision making within reasonable bounds as an optimization problem, solving it once, and reusing the solved decisions can often make an agent faster, cheaper and more reliable. Our approach, which we call Discretize, Then Decide, starts from data such as the workflow's own logs or, when none exists, runs of the LLM agent itself. It reads each conversation into discrete states and estimates the probability of each outcome and the cost of each move, until no new states appear and the probabilities stop changing. It then solves the resulting Markov decision process. The solution runs as a controller around the model, sending moves whose content is fixed in the state, such as a tool call or a templated reply, and any state it hasn't seen is sent back to the LLM. We call an LLM run this way a Bellman language model (BLM), after Richard Bellman's equation for the best move in each state. On $τ^2$-bench telecom, the controller with no model solves 79% of tasks. As a BLM, Qwen3.8-27B, an open model, goes from 29% to 96% with 93% fewer model calls. For Claude Opus 5.5, which solves 96% of tasks on its own, running it as a BLM cuts model calls and cost per task by about 98% and lets it solve 2 more tasks. On InventoryBench, the solved policy with no model beats every published method. Where the hard part is writing content rather than choosing moves, such as resolving SymPy issues, few decisions can be solved in advance, and a BLM helps little.
