你关注的AI研究团队发布了新看法，说以后衡量AI系统得看输出精度而不是单纯的能力，他们的方法能帮你实际优化系统，和以前的方式不一样，值得了解。
前沿语言模型常被用于比较、营销和基准测试其能力，但这些模型的最佳或平均输出所达到的目标精度（如输出集中度）才是区分不同系统的关键；当前的基准文化系统地未能测量这种精度，而是报告了输出的中心趋势而非分布范围；该研究提出一种新的测量方法，通过多次运行固定任务并计算结果的一致性来确定精度；这种方法能够区分一致失败与分散失败，帮助优化AI系统性能。
Grouping the Stochastic Machine: Precision, Not Capability, as the Frontier Metric for AI Systems
Frontier language models are compared, marketed, and benchmarked on capability -- what their best or average output can achieve. I argue this measures the wrong axis. The models have saturated accuracy: their mean output lands on the target. What now separates one system from another in practice is precision: how tightly concentrated their outputs are around that target across repeated, identical requests. Borrowing the marksman's distinction, capability is where the average shot lands; reliability is the size of the group. I make three claims. First, precision, not capability, is the frontier differentiator between systems, and benchmark culture systematically fails to measure it, reporting central tendency rather than spread. Second, precision is measurable, cheaply and without circularity, by running a fixed suite of deterministically scored tasks many times at fixed temperature and computing the per-task consistency of outcomes -- no model-in-the-loop grader required. Third, the measurement is not merely descriptive but decision-guiding: it separates consistent failures (a tight group off-centre, correctable by the operating discipline of Paper 1 -- a sight adjustment) from scattered failures (a wide group, correctable only by changing the model or its sampling -- a rifle problem). I define a grouping metric, specify a harness, and show how tracking a human-AI pair's grouping over time yields the compounding signal that Paper 1's field study requires. A first real run, since replicated, illustrates both the method and its most important limit: one measured gap was closed completely by a single rule (0/5 -> 5/5), while a suite of tasks authored from the rules themselves found no value, because a frontier model already embodies explicit good practice -- establishing that a discipline's worth is found by measurement on real work, not constructed from its own rulebook.