现代AI系统并非基于自回归模型，其推理方式存在差异
They can't. The LLMs that underly modern AI systems are auto-regressive, but modern AI systems are n...
Yann LeCun（深度学习之父）在讨论AI推理的本质，指出当前主流LLM并非自回归模型，而是通过搜索生成序列，并强调推理应在抽象空间而非语言空间进行，观点很有深度。
现代AI系统并非基于自回归模型，而是通过生成大量标记序列并选择最佳结果进行推理。当前AI系统的错误在于在离散标记空间中进行搜索，而人类推理更多是在抽象表示空间中进行的。作者认为，应该将推理过程从离散标记空间转向抽象表示空间。
They can't. The LLMs that underly modern AI systems are auto-regressive, but modern AI systems are n...
They can't. The LLMs that underly modern AI systems are auto-regressive, but modern AI systems are not. The "reasoning" systems produce lots of token sequences and pick the best one. That's a form of non-auto-regressive search. What current AI systems do wrong is that they do this search in discrete token space. I've been advocating for inference-by-search for planning/reasoning for many years (before chatGPT). But I think it should be done in abstract representation space. Much of human reasoning/planning is completely unconnected with language. I was correct. You just did not understand my argument. 💬 11 🔄 0 ❤️ 232 👀 10620 📊 33 ⚡