Lambda 发布开源 GPU 内核 TurboPairFormer，OpenFold3 训练提速 1.73 倍
做蛋白质结构预测或跑 OpenFold3 的团队可以直接用：同一批 Hopper 卡上单步训练快最多 1.73 倍，梯度还比原 Triton 后端更接近 FP64 参考。开源内核，H100 和 H200 都支持，600 个固定输入测试里四种梯度全部逐位一致，省下的是实打实的 GPU 时长和钱。
Lambda、Stevens Institute of Technology 与 OpenFold 团队联合发布开源 GPU 内核 TurboPairFormer，针对 NVIDIA Hopper GPU 上的三角形注意力计算。相比 OpenFold3 原有的 Triton 后端，每个优化器步骤的 GPU 计算最高提速 1.73 倍，同时在 H200 上跑 768 pixel 输入时为 14.67 秒/步，比 OpenFold3 Triton 的 22.53 秒/步快 1.54 倍。
三角形注意力随训练 crop 尺寸呈三次方增长，crop 翻倍成本约增加 8 倍，而现有后端往往在速度、精度和可复现性之间取舍。TurboPairFormer 在三个维度同时改进：与 FP64 参考相比，Q/K 和 pair-bias 梯度误差为所有对比方案中最低；在全部 600 个固定输入测试中，Q、K、value、pair-bias 四类梯度均实现逐位一致（bitwise-identical）。
该内核支持 H100 和 H200，代码开源。对从事蛋白质结构预测、依赖 OpenFold3 做大规模训练的团队来说，这意味着在不牺牲梯度精度和可复现性的前提下，可以直接压缩单步训练耗时和 Hopper GPU 租用成本。
TurboPairFormer: open-source GPU kernels that deliver up to 1.73× faster OpenFold3 GPU computation per optimizer step than the original Triton backend, with more accurate gradients and reproducible results in fixed-input tests.
Triangle attention scales cubically, so doubling the training crop makes it ~8× more expensive. Existing backends each trade away speed, precision, or reproducibility.
Researchers from Lambda, Stevens Institute of Technology, and the OpenFold team built TurboPairFormer to address all three on NVIDIA Hopper GPUs:
• Up to 1.73× faster than OpenFold3 Triton • Lowest mean Q/K and pair-bias gradient errors vs an FP64 reference • Bitwise-identical gradients across all four types (Q, K, value, pair-bias) in all 600 fixed-input cases Supports H100 and H200. On H200, 1.54× faster than OpenFold3 Triton on 768 pixel images (14.67 vs 22.53 sec/step).
Read the blog for the full method and results
https://t.co/yl3xYB9NiR
This report is categorized under AITOP's "Models" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include TurboPairFormer, OpenFold3, Lambda, H100.
Source attribution: curated from Lambda (@LambdaAPI).
