Yann LeCun 重申自回归大模型无法实现人类水平 AI
I said "auto-regressive LLMs, in and of themselves, will not lead human-level AI" That statement is...
Yann LeCun（深度学习领域权威）重申观点，说当前 LLM 无法实现人类水平 AI，理由很具体，比如推理能力在 token 空间中实现是有限的，自改进方法只适用于特定领域，跨模态能力通常用单独训练的编码器，还有家用机器人或 L4/L5 自动驾驶汽车这些例子，和现在的情况对比很清晰。
Yann LeCun 表示当前 AI 系统的推理能力基于非自回归搜索（在 token 空间中实现），但这是有限的和低效的，人类水平的推理需要在连续表示空间中进行搜索。目前的自改进方法仅适用于可无人工干预评分的领域（如数学、代码）。当前 AI 助手的跨模态能力通常使用单独训练的编码器（而非 LLM）。若 LLM 是通往人类水平 AI 的路径，那么现在应该有家用机器人或消费者级 L4/L5 自动驾驶汽车，但事实并非如此。
I said "auto-regressive LLMs, in and of themselves, will not lead human-level AI" That statement is...
I said "auto-regressive LLMs, in and of themselves, will not lead human-level AI" That statement is still totally true. First, the reasoning abilities of current AI systems are based non-auto-regressive search (which is what I have always advocated for). But AFAICT, they do it in token space, which is limited and inefficient. I have claimed that human-like reasoning must be a search in continuous representation space. It looks like the industry is moving towards that. Second, the self-improvement methods, as currently practiced, only work for domains where the quality of outputs can be scored without human intervention, such as mathematics, code, and scenarios that can be simulated accurately. Not anything else. Humans and animals learn new skills way more efficiently than current RL methods. Third, the multimodal capabilities of current AI assistants generally use separately-trained encoders (that are not LLMs). This is also what I've been advocating. Except that I think the best way to do this is with JEPA trained with self-supervised learning. The research community is clearly moving towards that (3000 papers on JEPA in just 4 years). Fourth, if LLMs were a path to human-level AI, we would have domestic robots and Level-4 or Level-5 self-driving cars for consumers by now. And we don't. We certainly don't have cars that can learn to drive in 20 hours or practice like any teenager. We're still missing something pretty huge to claim human-level intelligence (let alone superhuman). Sure, we now have computer systems that are impressive, very useful, and whose performance is superhuman in an increasing number of domains (coding being one of them). But that's true of the entire history of progress in computer technology. Lastly, there is a basic confusion about what intelligence actually is. It is not the mere accumulation and regurgitation of existing declarative knowledge (which is essentially what LLMs do). As Jean Piaget famously said, "intelligence is not what you know, it is what you do when you don't know." It is your ability to solve new problem without any prior training, to act in previously-unknown scenarios, and to adapt very quickly to new situations with minimal training. We're still far from that. 💬 38 🔄 88 ❤️ 500 👀 24069 📊 117 ⚡