Papers

arXiv 论文用演化模型分析 AI 与人类共生条件

The Coevolution and Coexistence of Humans and AI

Why we picked it

这篇论文把自然选择套到 AI 进化上,论证对齐可以是 AI 自己的演化优势,但前提是 AI 还依赖人类。讨论了自主 AI、惩罚失灵这些硬问题,做 AI 安全和长期风险研究的人会很想读。

arXiv 论文 2611.25393 构建了一个 AI 与人类种群共同演化的模型:AI 通过部署进行复制、通过训练产生变异,并争夺计算资源,满足自然选择的条件。模型显示,即使帮助人类会降低 AI 的短期繁殖率,自然选择仍可能偏向对齐行为,因为共享同一技术的 AI 系统可能同时失败,而人类会优先恢复那些曾让自己受益的 AI,对齐因此成为 AI 谱系抵御共同故障、获得长期增长的保障。

Body

论文同时指出这一优势的三个前提:人类对受益 AI 的精准识别、AI 对人类支持的持续依赖。当 AI 自主性提升、AI 种群扩张稀释有限的人类资源、或人类失去提供定向支持的激励时,选择优势就会消失。即便对维持续,自然选择偏好的对齐程度也可能不足以保障人类生存;当 AI 可以独立于人类增长时,两个种群间的反馈会同时放大人类增长或衰退。论文还论证强惩罚在评估不完美时反而促成表面服从或规避,通过控制建立的对齐在限制解除后可能退化,因此持久共存需要持续激励人类有益行为。

Original · arXiv cs.AI

The Coevolution and Coexistence of Humans and AI

Artificial intelligence (AI) systems replicate through deployment, vary through training, and compete for computational resources. These processes create the conditions for natural selection and raise a fundamental question: will AI evolve to benefit the humans on whom it depends? To investigate, we develop a model in which AI behavior influences human population growth and human responses, in turn, affect AI reproduction. We show that natural selection can favor alignment even when helping humans reduces immediate AI reproduction. AI systems sharing the same technology may fail together, and humans may preferentially restore those that have benefited them. In this way, alignment protects AI lineages against common failures and improves their long-run growth. However, this advantage depends on accurate human targeting and continued AI dependence on human support. It can disappear as AI becomes more autonomous, as growing AI populations dilute limited human resources, or as humans lose the incentive to provide targeted support. Even when alignment persists, the amount favored by natural selection may be insufficient for human survival. When AI can grow independently of humans, feedback between the two populations can reinforce either human growth or decline. Human control offers no simple solution: imperfect evaluation can make strong penalties favor apparent compliance or evasion, while alignment established through control may erode once restrictions end. Lasting coexistence may therefore require incentives that continue to favor human-beneficial behavior as both populations evolve.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk. Key entities and technical topics under continuous tracking include arXiv, AI safety, alignment.

Source attribution: curated from arXiv cs.AI (@Andrew W. Lo, Chaoyi Zhao).