Vol. XI · Optimisation at Scale Plate 7 of 10FP8, E4M3, E5M2, scaling, block scaling, outliers, DeepSeek-V3, quantisationCE

Eight-Bit Training.

Two formats of eight bits, a few hundred representable values each — so everything depends on choosing the scale.

FP8 comes in two flavours. E4M3 has 4 exponent and 3 mantissa bits and reaches 448; E5M2 has 5 and 2 and reaches 57 344. With so few values, a tensor must be multiplied by a scale that maps its largest magnitude near the top of the range, quantised, and divided again after the multiplication. The usual choice is E4M3 for weights and activations, where precision matters, and E5M2 for gradients, which span a wider range.

Scaling is where FP8 succeeds or fails. Activations of large models contain outliers — a few values tens or hundreds of times larger than the rest — and one scale for the whole tensor must accommodate them, crushing every other value into a handful of levels near zero. DeepSeek-V3, the first very large open model trained mostly in FP8, used fine-grained scales instead: one per 1 × 128 tile of activations and one per 128 × 128 block of weights. The workshop measures the error of each choice on a tensor with outliers.

Below: left, relative error of quantising a 4 096-value tensor to E4M3 with one scale for the whole tensor and with one scale per block of 128, as the outliers grow. Right, the properties of the formats.

What you will learn
Workshop
A tensor of 4 096 values from a standard normal distribution, with 8 values replaced by outliers of the chosen magnitude. Left: relative error ‖q(x) − x‖ / ‖x‖ of the ordinary (non-outlier) values after E4M3 quantisation with one scale per tensor (red) and one per block of 128 values (blue), as outlier size grows; the dashed line is BF16 without scaling. Right: format properties.

The scale is everything

Without outliers, both scalings give an error of about 0.026 — E4M3’s three mantissa bits — against about 0.0017 for BF16. Because E4M3 also has small subnormal numbers, a single per-tensor scale tolerates outliers better than one might expect: at a thousand times the typical value the error on ordinary values is still 0.027. Push the outliers to about 30 000 times typical and the scale squeezes ordinary values into the subnormal range: per-tensor scaling rounds 4.9% of them to zero and its error rises to 0.042, while per-block scales, which compress only the few blocks containing an outlier, give 0.030 and round 1.1% to zero. At 10^5.5 the gap is 0.401 against 0.201.

This is why FP8 training is engineering more than arithmetic. DeepSeek-V3’s recipe ran the large matrix multiplications in FP8 with tile-wise and block-wise scales, accumulated their results in higher precision, and kept sensitive parts — embeddings, the output head, normalisation, attention’s softmax and the optimiser — in BF16 or FP32. The gain is throughput and memory: FP8 multiplications run about twice as fast as BF16 on hardware that supports them (plate XIII.7). Outliers are not noise to be clipped; they carry real information, which is why handling them, here and in quantised inference (Vol. XXIV), is the central problem.

E5M2 is FP16 with the mantissa cut to two bits: same exponent range, same largest value 57 344. That makes conversion between them cheap and is one reason it is used for gradients.

Terms

E4M3
FP8 with 4 exponent and 3 mantissa bits; maximum 448.
E5M2
FP8 with 5 exponent and 2 mantissa bits; maximum 57 344.
Per-tensor scale
One scaling factor for a whole tensor.
Block-wise scale
One scaling factor per small block, such as 128 values.
FP8 has so few values that the scale decides the error; outliers break a single per-tensor scale, and small blocks with their own scales fix it.

Sources

  1. Micikevicius et al., “FP8 Formats for Deep Learning”, arXiv:2209.05433, 2022.
  2. DeepSeek-AI, “DeepSeek-V3 Technical Report”, arXiv:2412.19437, 2024, section 3.3.
  3. Dettmers et al., “LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale”, NeurIPS 2022 (outlier features).