做法律、税务等规则密集型应用的开发者,可以看看DAR如何用智能体框架解决LLM的长规则定位痛点,值得关注其设计思路。
道义推理是指根据明确规则和策略回答特定案例问题的任务,例如计算税务或移民上诉结果。LLM在此类任务中常因规则集过长且交叉引用而失败。论文提出Deontic Agentic Reasoning (DAR),一种让模型按需与法规交互的智能体推理框架。在DeonticBench的困难子集上测试发现,智能体框架能提升道义推理上限,但效果不均衡:弱模型在数值任务上表现下降且消耗更多token。
DAR: Deontic Reasoning with Agentic Harnesses
Deontic reasoning is the task of answering questions by applying explicit rules and policies to case-specific facts, for example computing tax liability under a statute or determining the outcome of an immigration appeal. A key technical challenge for LLM-based deontic reasoning is that the relevant ruleset can be long and cross-referenced, so models may still fail to locate the rules needed for a particular reasoning step. We introduce Deontic Agentic Reasoning (DAR), an agentic reasoning setup in which the model interacts with the statutes on demand. We evaluate DAR under multiple harnesses on hard subsets of DeonticBench. Across these settings, we find that agentic harnesses can push the frontier on deontic reasoning tasks, but improvements are not uniform: weaker models often degrade on numerical tasks while consuming far more tokens.