François Chollet:当前 Transformer 技术栈并非最高效的学习方案
Keras 之父 François Chollet 直接放话:Transformer 加梯度下降这套组合,效率连前 50% 都排不进。做架构研究或长期押注某个技术路线的人,值得看看他对下一代的判断。
François Chollet 在 X 上发文称,学习算法的可能性空间极为广阔,覆盖长期可塑性(预训练)与即时适应(推理)两个时间尺度。他认为当前的 Transformer + 梯度下降 + token 空间思维链生成这一技术栈,效率甚至排不进前 50%。他断言未来的 AI 架构将与现在的技术栈截然不同。
这篇观点延续了 Chollet 一贯的立场——他在 ARC-AGI 基准上长期质疑 LLM 的抽象推理能力上限,主张需要 beyond-token-space 的新机制。作为 Keras 创作者和前 Google 工程师,他的判断在架构研究方向上常被视为风向参考。
I believe there is a vast, extremely diverse space of possible approaches to learning (for both time scales: long-term plasticity / pretraining & on-the-fly adaptation / reasoning), and that the current stack (Transformer + gradient descent & token-space reasoning chain generation) is not even in the top 50% most efficient.
Future AI will look nothing like the current stack.
This report is categorized under AITOP's "Industry" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include François Chollet, Transformer, ARC-AGI, reasoning models.
Source attribution: curated from François Chollet (@fchollet).