一种基于分位数的梯度下降方法，用于提升模型对异常值的鲁棒性
Quantile-based Loss Filtering for Outlier-Robust Stochastic Gradient Descent
这个研究提出了一种新的梯度下降优化方法，通过过滤掉不稳定的梯度来提升模型对异常值的鲁棒性，实验效果不错，对做优化相关研究的人可能有参考价值。
本文提出了一种名为 Quantile-k-Loss SGD (QkL-SGD) 的新算法，通过在每个迭代中采样 k 个损失函数并选择其中最低的 q 分位数对应的梯度进行更新，从而在优化过程中过滤掉不稳定的梯度。该方法在多项式回归、正则化逻辑回归和正则化 hinge 损失等任务上进行了实验，结果显示中间分位数的更新效果优于标准 SGD 和 min-k 损失 SGD，能够避免因反复选择已解决组件而导致的优化停滞问题。
Quantile-based Loss Filtering for Outlier-Robust Stochastic Gradient Descent
We study loss-based filtering for finite-sum optimization with a subset of corrupted component functions whose gradients may be highly unreliable. Motivated by minimum-loss-based SGD (min-$k$-loss) and quantile-based methods for corrupted linear systems, we propose and analyze a general loss-filtering framework -- Quantile-\(k\)-Loss SGD (Q\(k\)L-SGD) -- that samples \(k\) component losses at each iteration and updates using an index chosen uniformly from the lower empirical \(q\)-quantile. We prove linear convergence of this family of methods under standard convexity assumptions, requiring the sample size to scale with the number of corruptions and a subset strong-convexity threshold. For the cases when large enough sampling is impossible or undesirable, we give a complementary small-sample probabilistic analysis that covers any sample size $k$ and the convergence behavior depends on the probability of selecting an outlier and on the curvature of the selected good step. Experiments on polynomial regression, regularized logistic regression, and regularized hinge loss show that intermediate quantiles often outperform both standard SGD and min-\(k\)-loss SGD. In particular, min-\(k\) often stalls by repeatedly selecting nearly solved components, while intermediate quantiles retain robustness and produce more informative updates.
