Sakana AI 这个框架解决了残差网络训练中梯度回传的瓶颈，做大规模生成模型或分布式训练的团队值得关注——它让每个模块可以独立优化，直接降低显存和通信开销。
Sakana AI 提出 DiffusionBlocks 框架，将残差网络中的每一层更新解释为逆向扩散去噪步骤，从而将网络分解为多个独立可训练的块。这种方法允许每个块单独训练，无需端到端反向传播，降低了训练复杂度和内存需求。实验表明，DiffusionBlocks 在图像生成任务上达到与标准残差网络相当的性能，同时训练效率更高。该工作为扩散模型与残差网络的结合提供了新思路，有望推动大规模模型的分布式训练。
Sakana AI Proposes DiffusionBlocks: a Block-wise Training Framework That Converts Residual Networks into Independently Trainable Denoising Modules
DiffusionBlocks converts residual networks into independently trainable blocks by interpreting layer updates as reverse diffusion denoising steps. The post Sakana AI Proposes DiffusionBlocks: a Block-wise Training Framework That Converts Residual Networks into Independently Trainable Denoising Modules appeared first on MarkTechPost .