Marcus 用简单例子戳破了 LLM 有意识的流行说法，关心 AI 伦理和意识本质的读者看完会重新审视 AI 的边界。
Gary Marcus 引用 Ted Chiang 的观点，认为声称 LLM 有意识是荒谬的。他指出，LLM 能模拟凯撒与成吉思汗的对话，不代表它包含自我意识。Marcus 强调，意识至少需要实体有生死攸关的利害关系、有驱动情绪，而 LLM 没有这些。他进一步警告，将道德决策外包给 LLM 会导致人类道德推理能力萎缩，因为 LLM 无法体验伤害、恐惧或后悔。
sorry @geoffreyhinton and @RichardDawkins
sorry @geoffreyhinton and @RichardDawkins Valerio Capraro @ValerioCapraro Ted Chiang is right: claiming that LLMs are conscious is just ridiculous. One simple example. If you ask GPT to imitate a conversation between Julius Caesar and Genghis Khan, GPT will do it very well. It will talk about wars, betrayal, and power. Il will descrive the feeling of being cheated by your brother with unbelievably realistic and moving words. Does this mean that GPT contains a self-conscious copy of Julius Caesar or Genghis Khan? Of course not. Similarly, if GPT makes claims about itself, does this mean it is self-conscious? Of course not. An LLM is just simulating language, feeling, and consciousness. True, we don’t have an accepted definition of consciousness. But, at a minimum, to be conscious, an entity must have something at stake. It must risk dying and have emotions that move it away from danger and towards favorable states. It must have a driver. This is also why I share Chiang’s worry about moral atrophy. The more we offload moral decisions to LLMs, the more we risk losing our own capacity for moral reasoning. Human moral reasoning descends from our history of making harmful actions, suffering harmful actions, regretting them, fearing them, repairing them, and learning from them. LLMs do not experience harm, do not suffer, do not fear consequences, do not regret. So they cannot do moral reasoning. We are offloading moral reasoning to systems that cannot do moral reasoning. What can go wrong? * Full piece in the first reply 🔗 View Quoted Tweet 💬 2 🔄 0 ❤️ 14 👀 1306 📊 3 ⚡