Papers

AlignedHPO:梯度对齐引导的可微损失超参数优化框架

AlignedHPO: Efficient Surrogate-Guided Loss Hyperparameter Optimization

Why we picked it

训练里调 Focal Loss、ArcFace margin 这类损失参数太费时间的话,这篇给了个基于梯度对齐的自动方案,比 implicit differentiation 快近 12 倍,长尾分类和人脸识别都有实验结果,做训练优化的工程团队可以直接看。

学术论文 AlignedHPO 提出一种针对可微损失超参数的自动调优框架,面向 Focal Loss 权重、ArcFace margin 这类通常靠人工或黑盒搜索调整的参数。方法上用 Jacobian-vector products 降低超梯度计算的时间与内存开销,并以可微指标代理(surrogate)作为验证目标,引导优化对齐到具体评估标准,同时通过归一化对齐和梯度平滑处理梯度量级歧义与随机不稳定。

Body

实验覆盖长尾分类和开集人脸识别两个任务,在 Generalized Focal Loss 与 ArcFace 上验证了框架的适配能力。结果显示其性能与 implicit differentiation 相当或更优,超梯度计算速度提升近 12 倍。对需要在大规模数据上反复调损失函数超参数的工程团队来说,12 倍的速度差异意味着原本按天计的调参流程可以压缩到小时级。

Original · arXiv cs.AI

AlignedHPO: Efficient Surrogate-Guided Loss Hyperparameter Optimization

Modern deep learning losses involve multiple hyperparameters, such as weights, scales, and margins, typically tuned manually or through expensive black-box optimization despite their differentiability. We propose AlignedHPO, an efficient framework for tuning differentiable loss hyperparameters using gradient alignment and surrogate validation objectives. Our method employs Jacobian-vector products to reduce hypergradient computation time and memory, while differentiable metric surrogates guide optimization toward task-specific evaluation criteria. We address gradient magnitude ambiguity and stochastic instability through normalized alignment and gradient smoothing. Experiments with generalized Focal Loss and ArcFace demonstrate effective adaptation across long-tailed classification and open-set face recognition. AlignedHPO achieves competitive or improved performance compared to implicit differentiation, with nearly 12x faster hypergradient computation. These results highlight the potential of efficient surrogate-guided loss hyperparameter optimization.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include hyperparameter optimization, ArcFace, Focal Loss, loss functions.

Source attribution: curated from arXiv cs.AI (@Ivan Karpukhin, Andrey Savchenko).