Rubricon：自动生成与精炼评分标准，用于 LLM 评估与训练
Rubricon: Automated Rubric Generation and Refinement for LLM Evaluation and Training
训练和评估没有标准答案的任务时，人工写 rubric 很贵，这篇论文给出全自动生成方案，对齐度比现有方法高 9%
arXiv 论文 Rubricon 提出一条无需人工偏好标签或评分标准标注的流水线，用任意 LLM 采样多个评分标准，选出区分度最高的候选，再补充遗漏要求并删掉对齐差的条目。在五个专业、消费、临床和通用 LLM 基准上，Rubricon 生成的评分标准与专家人工标准的对齐度比 RRD、QSRG、OpenRubrics 等最优合成方法高 9%。在 HealthBench 上，用 Rubricon 评分标准后训练 Qwen3.5-9B，成绩比用 RRD 标准训练高出 4.5%。该方法依靠标准通过率和 LLM 自我批评来引导精炼。
Rubricon: Automated Rubric Generation and Refinement for LLM Evaluation and Training
Rubrics enable reward assignment on tasks without verifiable answers, but are expensive to manually produce from scratch. We present Rubricon, a pipeline that generates discriminative rubrics without requiring human preference labels, rubric annotation, or generator-model training. Using any LLM, Rubricon samples multiple rubrics, selects the most discriminative candidate, adds criteria to capture overlooked requirements or response flaws, and prunes criteria that are misaligned or shallow. Alongside LLM self-critique and test-time scaling, Rubricon uses observed criterion pass rates on example model responses to guide refinement. Across five professional, consumer, clinical, and general-use LLM benchmarks, Rubricon produces rubrics whose scores of frontier LLM responses exhibit 9% greater alignment with labor-intensive expert-authored rubrics, compared to the best alternative synthetic rubric generation method among RRD, QSRG, OpenRubrics, and agentic generation via harnesses like Codex. On HealthBench, post-training a Qwen3.5-9B model with Rubricon rubrics yields 4.5% better performance than training with rubrics from the strongest alternative synthetic rubric generator, RRD.