Gary Marcus总结了提升数学推理的有效路径：SFT加RL再加推理时计算和工具，别迷信纯规模了。
Gary Marcus指出，纯LLM作为概率性下一个词预测模型存在固有局限，仅靠扩大规模无法修复硬数学问题中的组合幻觉。他提出有效的解决路径包括：在推理轨迹上进行SFT、采用o1风格的过程奖励模型进行RL、引入类似MCTS的推理时计算，以及集成可执行工具。Marcus强调现代数学AI是混合系统，而非纯Transformer。
Nice to see someone around here actually understand the technical details. (The vast majority of t...
Nice to see someone around here actually understand the technical details. (The vast majority of the attacks on me come from people who don’t.) Jen Zhu @jenzhuscott And the probabilistic next-token critics were correct abt raw LLMs. No amount of scale alone fixes compounding hallucinations on hard math. What does work: SFT on reasoning traces → RL (o1-style process reward models) → inference-time compute (MCTS-like) + executable tools. Modern math AIs are hybrid systems, not pure transformers. So yes the critics might have been underestimating the pace of progress, but they were not wrong abt pure LLMs. 🔗 View Quoted Tweet 💬 5 🔄 2 ❤️ 28 👀 3226 📊 6 ⚡