深度学习的训练与泛化的统计特性

Statistical Properties of Training & Generalization

精选理由

这篇论文从物理学角度拆解深度学习的统计特性,解释了神经缩放定律如何打破经典统计直觉,做研究的值得看看。

AI 摘要

本文从物理学视角分析了深度学习训练与泛化的统计特性,指出其打破了经典统计学的多项直觉。重点讨论了神经缩放定律(neural scaling laws)及其与约束、归纳偏置的相互作用。文章还回顾了构建深度学习模型时的多种选择及其合理性。

原文 · arXiv cs.LG

Statistical Properties of Training & Generalization

Deep learning has managed to evade numerous intuitions from classical statistics to achieve unprecedented performance on a number of real-world tasks. In this article, we investigate the key features and surprises of deep learning from a physics-informed perspective, taking care to point out and justify where possible the many choices inherent in constructing a deep learning model. In particular, we review the phenomenon of neural scaling laws and discuss their interplay with the constraints and inductive biases which may be present when applying machine learning to problems in physics.