Generative models tend to reproduce patterns they have learned rather than every rare detail in the source distribution. If one model's outputs replace the original training data for the next model, errors and omissions can compound across generations.
Research published in Nature in 2024 studied this process in language models and other generative models. It distinguished early collapse, in which rare cases disappear, from late collapse, in which the learned distribution can move far from the original. The results come from specified experimental settings.
Collapse is not an inevitable consequence of including any synthetic data. Later research showed that keeping real data available and accumulating synthetic examples alongside it can avoid the same runaway degradation under studied conditions. Curated synthetic training also helps some models.
Keep records of data origin, retain high-quality human or real-world examples and test tail coverage and held-out performance. Evaluate the actual training recipe instead of treating 'synthetic' as automatically harmful.