Vol. VI · Positions and Long Context Plate 1 of 10permutation invariance, word order, position information, attention, NoPEA

Attention Is Order-Blind.

Shuffle the words and attention computes exactly the same thing for each of them. Word order has to be put in by hand.

Look again at the attention formula: each output is a weighted average of value vectors, with weights from dot products between a query and the keys. If the tokens of a sentence are shuffled, every token still meets the same set of other tokens with the same keys and values, so its output is exactly the same — merely in a different position. Attention by itself treats a text as a bag of tokens: “man sees cat” and “cat sees man” are indistinguishable to it, though they mean opposite things.

Language models fix this by giving each position its own signal, added to or mixed into the token vectors before or inside attention. The next plates go through the main schemes. One subtlety is worth knowing first: the causal mask itself leaks some order, because a token near the end sees more tokens than one near the start, and decoders trained with no position signal at all (“NoPE”) can still learn something about order. The workshop shows all three cases on the first layer of micro.

Below: micro’s first attention layer applied to two sentences that are permutations of each other. Each bar is the similarity between a word’s output in the first sentence and the same word’s output in the second. Without positions and without the mask, every bar is exactly one.

What you will learn

Show that attention without positions is permutation-equivariant.
Explain why word order must be supplied separately.
See how position signals break the symmetry.
Recognise that the causal mask leaks some order.

Workshop

Cosine similarity between each word’s output from micro’s first attention layer in the first sentence and in the second, for the settings chosen. One means the layer cannot tell the two sentences apart at that word.

A bag of tokens

With no positions and no mask, each word of “man sees cat” gets exactly the same output from micro’s first attention layer as in “cat sees man”: similarity 1.0000. Add micro’s learned position vectors and the outputs differ (mean similarity 0.914), because each token now carries where it is; apply the causal mask without positions and they differ too (0.773), because the first word sees only itself while the last sees everything. The model micro actually is uses both (0.834).

A Transformer without any position signal would therefore be a strange language model: it would assign the same probability to a sentence and to its anagram whenever the mask did not happen to tell them apart. Every production model adds position information, and how it does so turns out to matter for more than word order: it decides how well a model handles texts longer than any it was trained on, which is the thread of the rest of this volume.

Amirhossein Kazemnejad and colleagues showed in 2023 that decoder-only Transformers trained with no position encoding at all (“NoPE”) learn usable position information from the causal mask, and on some length-generalisation tasks do better than models with explicit positions.

Terms

Permutation-equivariant: Shuffling the inputs shuffles the outputs the same way and changes nothing else.
Position signal: Information about where a token is, added to or mixed into its vectors.
NoPE: No position encoding: relying on the causal mask alone for order.
Bag of tokens: A text treated as an unordered collection of its tokens.

Attention alone is blind to order: shuffle the tokens and each one gets the same output. Position information must be added — and the causal mask leaks a little.