Papers

arXiv 论文:用可控测试衡量决策模型的规则迁移与信息利用

Which Differences Matter? Controlled Tests of Rule Transfer, Joint Reliability, and Information Use in Decision Models

Why we picked it

这篇论文教你怎么拆开测决策模型:用算术规则把迁移、可靠性和信息利用分开量化,方法可以直接搬到自己的评测里。

一篇 arXiv 论文提出用可执行的算术规则对决策模型做受控测试。论文通过语义规则分离和规则-状态交叉测试,把迁移能力和联合可靠性拆成可分别测量的指标。实验发现,适应增益随最近源答案一致度上升,而完全冲突的四元组只带来较小的池化增益。等价改写在任务层面产生相反效果,某决策服务在等价措辞间改变答案的幅度大于相同文本的重复测试。

Original · arXiv cs.AI

Which Differences Matter? Controlled Tests of Rule Transfer, Joint Reliability, and Information Use in Decision Models

Decision models map stated rules and observed states to discrete answers. Aggregate accuracy leaves unresolved which rule and information changes guide those answers. We connect controlled input changes to transfer, joint reliability, and directed information use through executable arithmetic rules. Semantic rule separation and crossed rule and state tests make transfer and joint success separately measurable. We compare equivalent wordings, calibrate joint correctness through marginal-preserving reassociation, and alter tables to define alternative answer targets. Adaptation gains increase with nearest-source answer agreement; fully conflicting quartets yield small pooled gains and different gain signs across source splits. Equivalent rewrites have opposing task-level effects, linking transfer gains to rule presentation. Joint-score departures depend on which rule and state associations the reference preserves. A decision service changes answers more across equivalent wordings than across identical-text repeats and usually follows the answer targets implied by altered table values or mappings.