Memory bandwidth is how much data can move between memory and a processor each second, usually measured in gigabytes per second (GB/s). It is set by the speed and width of the connection to memory and the type of memory used.
It matters a great deal for AI. When a language model generates text, the processor must read much of the model's data from memory for each step, so on many devices the speed of text generation is limited by how fast memory can be read rather than by how fast the chip can calculate.
A higher bandwidth is not always the deciding factor. Compute speed, the model size and the numeric precision used also matter, and in some workloads the processor is the limit instead. Peak bandwidth figures are also maxima, and shared memory used by several components can deliver less in practice.
When assessing a device for AI use, look at memory bandwidth and memory capacity together with the compute rating. Larger models need more memory, and smaller or more compressed ones need less bandwidth per step.