Preventing Model Collapse with Synthetic Data
Preventing Model Collapse: A Fisher-Rao Perspective on the Dynamics of Training with Synthetic Data
For researchers working with large language models, this provides a more rigorous mathematical understanding of the stability of training on synthetic data, which is increasingly common as human data becomes scarce.
This paper presents a theoretical framework using the Fisher-Rao metric to analyze how training large language models with synthetic data can lead to model collapse. It establishes quantitative bounds on the minimum ratio of human data needed to prevent this degenerative feedback loop, addressing a gap in previous work that relied on the Euclidean metric.
Preventing Model Collapse: A Fisher-Rao Perspective on the Dynamics of Training with Synthetic Data
Large Language Models (LLMs) are now routinely trained using synthetic data, since high-quality human data has been exhausted by the ever increasing needs of larger and larger models. However, recursive training on synthetic data frequently induces model collapse, a degenerative feedback loop where models progressively forget the true underlying data distribution. Training on a mixture of synthetic and fresh human data is a logical countermeasure and can prevent model collapse. However, it is an open question as to what is the exact minimum required ratio of human-to-synthetic data to maintain training stability. In this paper, we establish rigorous theoretical guarantees on the minimum rate of human data required to prevent model collapse. Although previous work established a formal lower bound for this ratio, such bound can be vacuous for very high dimensions, as the analysis relies on the usual Euclidean metric in R^n and is not adapted to the space of categorical probability distributions. Instead, in this paper we explicitly leverage the information-geometric structure of the probability simplex by analyzing the dynamics of the process under the Fisher-Rao metric. We derive quantitative contraction and invariance bounds that are stable and do not become trivial as the dimensions increase. Thus, we show that the effective required data ratio to prevent model collapse is different than previously implied.