Stanford AI Lab tests pretraining with zero real data via self-play
The result suggests that data synthesis by self-play alone can reproduce scaling behaviour normally attributed to real corpora, which challenges assumptions about what pretraining data must contain. Researchers studying synthetic data and training dynamics will want to test whether the predictable loss curves hold at larger compute budgets and on harder tasks.
