Vol. I · What a Language Model Is Plate 3 of 10chain rule, joint probability, log-probability, surprisal, scoring textLE

The Probability of a Text.

Predict each token from the ones before it, multiply the probabilities, and the model has scored the whole document.

A model that gives the next token a probability can also give a probability to a whole text. Write the text as tokens x₁, x₂, …, x_T. The probability of all of them together is the probability of x₁, times the probability of x₂ given x₁, times the probability of x₃ given the first two, and so on: the chain rule of probability, which holds for any sequence and needs no assumption at all.

The product of many numbers smaller than one is tiny, so in practice one adds logarithms instead. Each term −log P(x_t | x_{surprisal of a token: how surprised the model was to see it. The total surprisal of a text is the score a model gives it; a fluent sentence costs fewer nats than the same words in a scrambled order.

Below: two texts scored by the course model, token by token. Each bar is the surprisal of one token given everything before it. The left text is a sentence; the right one is the same words shuffled.

What you will learn
Workshop
Surprisal in bits of each token of the two texts under micro, given the tokens before it (the first token is given). Taller bars are tokens the model did not expect.

Scoring a document

The sentence in text A costs the model 58.2 bits in total; the same words shuffled in text B cost 97.5 bits. In natural logarithms that is −40.3 against −67.6 nats, so the model considers the sentence about 10 to the power 11.8 times more probable than its scramble. Look at where the bars are tall. In the sentence the cost sits on a few content words — the model cannot guess which organ, which verb or which direction comes next — while function words such as the and and, and the second half of a split word, are cheap. In the shuffled text, nearly every word arrives where the model did not expect it.

This is the same model that plate I.1 used to predict, used the other way round. Training a language model (Vol. VIII) means adjusting it so that the total surprisal of the training texts is as small as possible; evaluating one (plate I.4) means measuring the surprisal of texts it has not seen. Scoring is also how a model ranks candidate answers, detects text that is unlike its training data, and, as the post-training volumes show, how preferences between two responses are turned into a training signal.

The chain rule makes no assumption about language. n-gram models (plate I.2) approximate each factor by forgetting all but the last n − 1 tokens; Transformers keep the whole prefix, up to the length of their context window.

Terms

Chain rule
The probability of a sequence equals the product of each element’s probability given the ones before it.
Log-probability
The logarithm of a probability; sums replace products and tiny numbers become manageable.
Surprisal
The negative log-probability of an outcome: how surprising it was, in bits or nats.
Nat
A unit of information using natural logarithms; one nat is about 1.44 bits.
The probability of a text is the product of its next-token probabilities — a sum of surprisals, in logarithms. A model that predicts well scores fluent text as more probable.