Event ·Official first-hand

OneCodeReviewBench 发布:用类型化决策评测代码审查能力

OneCodeReviewBench 是一个代码审查基准,包含 1,968 个合成案例,覆盖 12 种编程语言和 YAML,答案形式为布尔、分类和有序三种类型。基准在 OpenReview 上比较了 10 个模型-适配器系统,OpenAI Decisions 达到 81.3% 的布尔平衡准确率,Kev-27B 达到 76.8% 的分类准确率和 0.367 的评分平均绝对误差。基准还附带 586 个原始符号搜索案例和 40 个受限 Python 导入解析案例,解析器在其支持范围内匹配了全部 40 个参考答案。论文保留了 85 个与工程师答案一致的案例,供版本化工件复现分析。仓库级迁移和延迟/成本对比评测仍属未来工作。

Current read

做代码审查工具或想测模型审查能力的话可以看看,1968 个带明确契约的案例,OpenAI Decisions 拿了 81.3% 平衡准确率。

10 sources31° AI heatLast updated 2026/10/24 01:40:29

Evidence

10 sources
Primary sourcearXiv: OpenAI
View source
Related source 1rohanpaul_ai
View source
Related source 2kimmonismus
View source
Related source 3@OpenAIDevs
View source
Related source 4TestingCatalog
View source
Related source 6VnExpress Số hóa
View source
Related source 7Nathan Lambert
View source
Related source 8Tibor Blaho
View source
Related source 9Greg Brockman
View source

Conflict check

Dedup data only shows that these sources discuss the same event, not that they agree. There is no structured supporting or refuting evidence yet, so no conflict is inferred automatically.