论文精选

FFR:首个将Forward-Forward算法扩展到回归任务的方法

FFR: Forward-Forward Learning for Regression

精选理由

做回归任务的深度学习研究者终于有了FF算法的可用方案——FFR在保持低内存优势的同时逼近BP精度,资源受限场景或生物启发学习的团队值得关注。

AI 摘要

FFR(Forward-Forward for Regression)是首个将Forward-Forward(FF)算法成功扩展到回归任务的方法。FF算法通过局部逐层优化替代反向传播(BP),但原本只适用于分类任务,因为回归的连续目标空间缺乏自然“对立样本”。FFR提出三项创新:基于有序竞争的好度函数、分层阶梯架构(浅层学习粗粒度排序、深层细化回归)以及带不确定性估计的分层预测。在五个真实回归基准上,FFR平均恢复BP 98.6%的精度,峰值训练内存仅为BP的27%(深度8)和8%(深度32),每轮迭代时间约为BP的72%,显著优于所有无BP的竞品。

原文 · arXiv cs.AI

FFR: Forward-Forward Learning for Regression

The Forward-Forward (FF) algorithm offers a computationally efficient and biologically plausible alternative to backpropagation (BP) by training neural networks through purely local, layer-wise optimization. However, FF is inherently designed for classification via contrastive positive-negative sample pairs, and extending it to regression poses fundamental challenges: continuous target space lack natural "opposites" for contrastive learning, and the standard goodness function carries no information about target magnitude or ordering. We propose FFR (Forward-Forward for Regression), to our knowledge, the first framework to extend FF to real-world regression and demonstrate competitive performance across diverse real-world datasets. FFR introduces three key innovations: (1) an ordinal competitive goodness function that replaces contrastive pairs with competitive learning between partitioned neuron groups under distance-aware ordinal supervision; (2) a stratified ladder architecture where shallow layers learn coarse ordinal discrimination and deeper layers refine into fine-grained regression, with multi-scale feature aggregation for inter-layer collaboration; and (3) hierarchical prediction with uncertainty estimation, where multi-scale predictors jointly provide robust predictions and prediction confidence as a free-lunch. Extensive experimental results show FFR recovers on average 98.6% of BP's accuracy across five real-world regression benchmarks while reducing peak training memory to only 27% of BP's at depth 8 and 8% at depth 32, with per-iteration time around 72% of BP's, and substantially outperforms all BP-free competitors.