When a language model generates a token, many next tokens may be possible. Beam search expands several candidate sequences and retains a fixed number with strong combined scores. A beam width of five, for instance, keeps five leading candidates at each stage.
This can be useful for tasks such as translation or controlled sequence generation, where a high-likelihood output is desired. It differs from random sampling, which deliberately explores varied possibilities.
Beam search has limits. It can favor repetitive or generic language and does not guarantee a true or humanlike answer. In model-collapse experiments, researchers used five-way beam search to create synthetic text, so the generation method is relevant to interpreting the results.
Record beam width, length penalties and other decoding settings in experiments. Compare generated data for diversity as well as likelihood before using it to train another model.