Apple 研究：LLM 评委在个性化写作风格评估中的可靠性
Evaluating the Evaluators: Investigating LLM Judges for Personalized Writing Style Assessment
Apple 这篇论文把“用 LLM 当评委”这件事本身做了系统实验：4 个生成模型、3 种长篇写作任务、不同评委组合都测了一遍。做个性化写作或自动评估流水线的开发者，尤其是纠结选哪个模型当 judge 的，能从结论里直接抄作业。
Apple 的论文被 NeurIPS 2026 的 Can We Trust AI Evaluation? Workshop 接收，主题是 LLM 评委能否可靠地判断生成文本是否符合用户偏好的写作风格（如 upbeat）。研究用四个 LLM 在三种长篇写作任务上生成文本，再让不同 LLM 评委打分，考察写作任务类型、生成器与评委的组合、评估目标等因素对评判质量的影响。
论文还分析了评委模型的常识与推理能力如何影响评估表现，为个性化写作场景中选择哪个模型当评委提供了实验依据。
对开发者来说，这类研究直接影响个性化写作产品中自动评估环节的模型选型——评委和生成器的搭配不当可能导致评分失真。
Evaluating the Evaluators: Investigating LLM Judges for Personalized Writing Style Assessment
This paper was accepted at the Can We Trust AI Evaluation? Workshop at NeurIPS 2026. We evaluate how good large language model (LLM) judges are for personalized writing tasks. This is important when a user’s preferred writing style is known (e.g., “upbeat”) and an LLM judge is used to evaluate whether generated text adheres to this preference. We examine how the writing task, the combination of generator and judge LLM, the evaluation objective, and general commonsense and reasoning ability impact LLM judge performance. To this end, we generate text with four LLMs for three long-form writing…
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include LLM, Apple, NeurIPS, 评估方法.
Primary source verification: published directly by Apple ML Research with first-party authority.