这篇论文拆解了被广泛引用的GPT暴露评分到底靠谱不,指出静态评分的坑,还给出了五类改进方向,搞AI政策或研究的人值得一看。
2023年Eloundou等人计算的GPTs暴露评分成为工作未来辩论的核心输入,该评分定义暴露为LLM能辅助的职业任务占比。论文指出其存在时间、地理和本体论局限,并调查了五类应对研究:动态和基准度量、集成方法、任务框架扩展、以工人为中心的指标、采纳和使用数据。研究-政策协调不足,政策分析仍引用静态评分而未采纳方法论更新。建议政策制定者拓宽证据基础,研究者采用参与式方法并构建数据基础设施。
AI Exposure Scores: what they measure, what they miss, and what comes next
A set of exposure scores calculated in 2023 has become a central empirical input to the future of work debate. Produced by Eloundou et al. (2023) and referred to here as the GPTs are GPTs scores, they define exposure as the share of occupational tasks a large language model can assist with. This work is a genuine methodological contribution, but as the scores travel from the time and place they were produced, the limitations the authors named do not always travel with them. Two gaps have widened as a result. The first is structural, between what static exposure scores measure and what policy questions actually require. Taking the diffusion of these scores as a case study, we show how their temporal, geographic, and ontological limitations compound in policy-facing analyses, and we survey five families of research responding to these limits: dynamic and benchmark-based measures, ensemble methods, task-framework extensions, worker-centered metrics, and adoption and usage data. The second gap is the one we argue needs more attention: the coordination between researchers and policymakers. The policy-relevant work which ask who is harmed, who benefits, how, and when, continues to reference the static GPTs are GPTs scores without engagement with the methodological updates that would let these questions be answered more reliably. We then ask what additional steps towards navigating uncertainty remain: ex-post frameworks and the deliberate, political work of reimagining what futures are worthy of building towards are. Closing the research-policy gap is a shared task: policymakers must widen their evidence base, engage workers as epistemic partners, and shift from prediction to preparedness; researchers must build data infrastructure, adopt participatory methods, and write with policymakers in mind. Better measurement matters, but it will not close the second gap alone.