The tails of a probability distribution contain observations that occur infrequently. In a text dataset they may include unusual names, minority dialects or uncommon factual combinations; in an image dataset they may include rare objects or viewpoints. These examples may matter even though they contribute little to the average count.
A generative model learns patterns from examples and tends to generate common patterns more readily than uncommon ones. If later models are trained primarily on earlier models' outputs, rare patterns may be underrepresented or disappear. Research on recursive training identifies that loss of coverage as an early sign of model collapse.
Rare does not mean wrong or unimportant. Some tails contain errors or noise, but others represent valid communities, edge cases or valuable diversity. A training filter that simply removes low-frequency examples can therefore improve one quality measure while damaging coverage of real users.
When comparing datasets or model generations, inspect performance on rare subgroups and difficult examples, not just average accuracy. Keep provenance for original examples and test whether synthetic additions preserve the distribution's tails instead of flattening them.