Bitter Lesson Reread: Four Common Misreadings of Sutton's Essay
https://t.co/nOVp1CTCnU
The thread's sharpest point is that scaling-with-compute is judged retrospectively, so no one can know in advance whether their design counts as bitter-lesson compliant. Engineers and researchers who cite the essay to justify current LLM orthodoxy will find Sutton's own objections to LLMs uncomfortable and useful. The CNN-to-ViT and AlphaGo-to-MuZero arcs give concrete reference points for reasoning about how much human prior to encode in a new system.
A widely shared thread by NoaX (@noahatwhere) argues that most Chinese-language takes on Rich Sutton's essay The Bitter Lesson misrepresent what it actually says, with four specific points of criticism. First, the essay already states that general methods which scale with compute, such as search and learning, are what work; treating this as a fresh insight reveals that many commentators only read the title. Second, the binary between wrong human cleverness and right cleverness is false: CNNs baked translation invariance and locality priors into their architecture, and ViT later removed those priors once data was abundant, while AlphaGo evolved into AlphaZero and MuZero by progressively deleting human knowledge. The relationship is a spectrum, not a dichotomy.
Third, scaling with compute is an after-the-fact standard: every researcher, including the expert-systems builders of earlier decades, believed their clever design was the kind that would scale, so the lesson cannot be reduced to good innovations are good and bad ones are bad. Fourth, invoking Sutton to endorse Transformers is problematic because Sutton himself is skeptical of LLMs. In his Dwarkesh interview he said LLMs imitate human data, lack goals, and do not learn from experience, and that the approach has not internalized the Bitter Lesson. His notion of clever new methods refers to continual and interactive learning. Reading him as pro-Transformer flattens a sharp critique into comfortable consensus.
https://t.co/nOVp1CTCnU
x.com/noahatwhere/st… NoaX @noahatwhere 1. “纠偏”其实是把原文重读了一遍。 《苦涩的教训》本来就写了，该做的是搜索和学习这类能随算力扩展的通用方法。把这当成“极其精准的洞见”，说明很多人只看过标题 2. “错误的聪明”和“正确的聪明”分得太干净了。 CNN 就是把平移不变性、局部性这些人类先验写死在架构里，后来数据够多，ViT 把这些先验拿掉反而更强，这本身就是一次苦涩的教训。AlphaGo 到 AlphaZero 再到 MuZero，也是一路在删人类知识。所以两者之间是光谱，不是楚河汉界 3. “能随算力变强”是个事后标准。 每个研究者都觉得自己的巧思属于正确的那种，专家系统当年也这么想。如果教训只剩“好的创新是好的，坏的创新是坏的”，那就不苦涩了，也没用了 4. 拿 Sutton 给 Transformer 站台的问题是， Sutton 本人并不买 LLM 的账。 他在 Dwarkesh 的访谈里说得很直白：LLM 靠模仿人类数据，没有目标，也不从经验中学习，他认为这条路本身就没吃透苦涩的教训。他说的“巧妙的新方法”，指的是持续学习、从交互中学习这些 LLM 还没有的东西。把他的话解读成“Transformer 这类创新很重要”，等于把一个激进的批评磨成了大家都爱听的鸡汤 🔗 View Quoted Tweet
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Bitter Lesson, Rich Sutton, LLM, reasoning models.
Source attribution: curated from 宝玉 (@dotey).
