AI and the World's Hard Problems · Theoretical computer science · arXiv · 2026-10-22
Mitigating Model Collapse through Dreaming Learning
Researchers studied the loss of diversity and performance caused by repeated training on synthetic data, using transition-operator spectra in a controlled Markov setting. Training on high-temperature samples flattened predicted distributions and mitigated collapse, with supporting spectral measurements over 5,000 generations and mitigation also observed in LSTM language models and a 124-million-parameter GPT-2.