During training, an image or other example is progressively corrupted with noise and the model learns how to estimate or remove that noise. During generation, the system starts from noise and applies a sequence of denoising steps to produce a new sample.
Diffusion models are widely used for images and can also model audio, physical structures and scientific data. They differ from next-token language models in both their training objective and their generation process.
Researchers have also studied recursive training with diffusion models. The behavior of one such system cannot be generalized to all generative models without examining the data, sampling procedure and evaluation metrics.
Assess generated samples for fidelity, diversity and rare-case coverage. If synthetic outputs are fed back into training, keep original reference data and monitor how the output distribution changes.