Chinese contentTaH2 自适应循环 Transformer 提升 AIME 测试时扩展效率
清华团队提出 TaH2，让循环 Transformer 只对需要的 token 多迭代，AIME 上同等算力多拿 3.4 分，代码已开源。
#recurrent Transformer
6 stories · 0 in 7 days · 0 in 30 days
Stories, products and papers tagged "recurrent Transformer", newest first.
Sep 29
Sep 14
Recurrent Looped Transformer architecture proposed
This is a significant new architecture proposal for large language models, potentially offering improved efficiency and performance.
Sep 9
OpenAI developing GPT-6 Astra model
GPT-6 Astra's Looped Transformer architecture offers improved efficiency for long-context tasks compared to current models.
Sep 6
May 25
Chinese content无需训练的循环Transformer：冻结模型推理时循环提升性能
这项研究让使用冻结模型的团队无需重新训练就能提升推理性能，做模型部署或推理优化的开发者值得关注，可以直接在现有模型上尝试。
May 13
Chinese contentAttractor Models：通过不动点求解让循环Transformer可扩展
循环Transformer训练难、部署贵的问题被Attractor Models用不动点求解优雅解决，做语言模型预训练或推理增强的团队值得关注——它用更少参数和成本实现了对更大模型的超越。