FP8 stores each number in 8 bits using a floating-point layout, which means a sign, an exponent and a mantissa, so that it can cover a wide range of magnitudes. Two variants are common. One gives more bits to the mantissa for finer precision, and the other gives more bits to the exponent for a wider range. Training often uses a mix, with the first for weights and activations and the second for gradients.
Compared with 16-bit formats, FP8 halves memory use and can roughly double the arithmetic throughput on hardware that supports it, so it makes training and inference cheaper and faster. Support is built into recent AI accelerators.
Lower precision brings risks of numerical problems, such as overflow or loss of accuracy. Practical use relies on scaling factors, and some operations are kept at higher precision to keep results stable.
Check that your hardware and software stack support it, compare results against a higher-precision baseline, and use it where the quality loss is acceptable.