论文精选

写AGENTS.md规则前看看Harness-IF:区分真规则与巧合

If you write rules in an AGENTS.md, this one is worth your time. When a coding agent follows your r...

精选理由

如果你在给编码智能体写AGENTS.md,这篇论文用256条规则和12个模型实测告诉你哪些规则真有用,别凭感觉写了。

AI 摘要

Harness-IF提出一种规则评估方法,通过从执行证据中逐条打分256条规则,并在九个探针构建中撤回每条规则后重跑任务,来区分规则是否真正改变了模型行为。在12个前沿模型上,原始准确率为72.1%到85.9%,剔除巧合后的Against-Prior准确率降至66.1%到78.6%,每个模型下降3.6到7.4个点。研究发现优先级不随提示词深度变化,系统提示、项目文件和用户指令的优先级都高于工具和技能描述。论文见arxiv.org/abs/2608.11727。

原文 · elvis

If you write rules in an AGENTS.md, this one is worth your time. When a coding agent follows your r...

If you write rules in an AGENTS.md, this one is worth your time. When a coding agent follows your rule, it may have been going to do that anyway. Harness-IF separates the two by scoring 256 rules one at a time from execution evidence, then re-running every task with the rule withheld across nine probe builds to find which rules actually oppose the model's defaults. Across 12 frontier models, raw accuracy runs 72.1 to 85.9%, and Against-Prior Accuracy runs 66.1 to 78.6%. Every model gets worse once coincidence is stripped out, by 3.6 to 7.4 points. One finding worth flagging. Precedence does not follow prompt depth. System prompts, project files, and user instructions all outrank tool and skill descriptions. Paper: arxiv.org/abs/2608.11727 Track more trending AI papers in our academy: academy.dair.ai 💬 2 🔄 3 ❤️ 18 👀 1899 📊 9 ⚡