低维数据结构的函数计数理论

Function-Counting Theory for Low-Dimensional Data Structures

精选理由

这篇论文把Cover上世纪60年代的函数计数理论套到低维数据上,分析低维结构怎么影响分类能力,比原来的理论更贴合真实数据。

AI 摘要

该论文基于Cover(1965)提出的函数计数理论,针对低维数据结构构建了二元分类的数学框架。通过修正一般位置假设以反映低维性,导出了体现数据结构的二分计数。进一步将Cover的分离容量和泛化问题推广到低维设定,揭示了数据结构对分类能力的影响。

原文 · arXiv cs.LG

Function-Counting Theory for Low-Dimensional Data Structures

The success of deep learning models in classification and regression is widely attributed to the low-dimensional structure that real-world data tend to exhibit, despite their high-dimensional representation. This work attempts to provide a mathematical framework for binary classification on low-dimensional data, building on Cover's (1965) function-counting theory. With our framework, we aim to address the question of how the low-dimensional structure of the data affects the classification capabilities of learning models. Cover's theory relies on a general position assumption that blinds it to the underlying data structure. We refine this assumption to account for the low-dimensionality of the data and derive dichotomy counts that reflect the data structure. We further extend Cover's separation capacity and problem of generalization to the low-dimensional setting, enabling the impact of the underlying data structure on both to be analyzed.