Primary sourcearXiv: OpenAI
View sourceEvent ·Official first-hand
BMMS 评测套件:按具体政策条款衡量语言模型行为合规性
arXiv 论文提出 Behavioral Misalignment Metric Suite(BMMS),以任意政策文本为输入,输出该系统的合规画像,核心指标包括 Compliance Attainment Score(CAS)与 Compliance Robustness Margin(CRM)。指标需通过以归因为判据的准入测试,套件据此剔除了自身的一个指标。在来自 11 份文档的 1,003 条义务上,准入判据对厂商规范的盲评精确率达 94.4%,对分析类指南为 50.0%。实测显示,GPT-4o 在 OpenAI Model Spec 的硬性约束上失败 70.7%,而两款 Claude 模型分别失败 0.0% 和 9.1%。
Current read
这篇论文做了个叫 BMMS 的合规评测:把 OpenAI、Anthropic 自己写的规范当考卷,结果 GPT-4o 70.7% 的硬约束没过,Claude 几乎全过,挺有意思。
11 sources41° AI heatLast updated 2026/11/6 08:24:58
Evidence
11 sourcesRelated source 1kimmonismus
View sourceRelated source 2DAIR.AI
View sourceRelated source 3OpenAI
View sourceRelated source 4IT之家
View sourceRelated source 5小互
View sourceRelated source 6lmarena.ai
View sourceRelated source 7@atomic_chat_hq
View sourceRelated source 8Notion
View sourceRelated source 9Stanford AI Lab
View sourceRelated source 10Andrew Curran
View sourceConflict check
Dedup data only shows that these sources discuss the same event, not that they agree. There is no structured supporting or refuting evidence yet, so no conflict is inferred automatically.