Vol. XXVIII · Interpretability Plate 7 of 9activation steering, steering vectors, representation engineering, residual stream, interventions, controlAE

Steering.

Find a direction that means something, and push the model along it.

If a concept is a direction in the residual stream, adding that direction while the model runs should make the concept more present in what it writes. Activation steering computes a steering vector — often simply the difference between mean activations on examples with and without the property (Turner et al., 2023; Rimsky et al., 2024) — and adds it, scaled by α, at one layer on every forward pass. Steering vectors from SAE features work the same way. It is a causal test of a representation and, sometimes, a practical control.

The course model micro (four layers, d = 128; plate I.9) is opened up in the page. The direction is micro’s mean residual after the chosen layer on held-out Chinese course text minus its mean on English text. The workshop adds α times that direction while micro greedily continues “The attention” for 30 tokens.

Below: the continuation at the chosen strength and layer, and the share of non-space characters that are Chinese as the strength rises.

What you will learn
Workshop
micro, greedy decoding of 30 tokens after “The attention”. Steering vector from the means over 800 characters of Chinese and 2 400 of English held-out course text, after the chosen layer, added after that layer at every step. Chinese share: CJK characters among non-space characters of the continuation.

A direction with a meaning, and a cost

Unsteered, micro continues “The attention” in English. Adding the Chinese-minus-English direction after layer 2 at α = 0.5 leaves it 0% Chinese; at α = 1 the continuation is 100% Chinese and still roughly sentence-like; at α = 2, 100% Chinese, but it has collapsed into repeating a phrase. The direction does carry “which language” — a causal confirmation of what a probe would only suggest — and pushing too hard drives the stream outside the range the later layers were trained on.

Steering is used to test representations, and experimentally to adjust behaviour — honesty, refusal, sentiment — without retraining. Its limits follow from the plate: the right strength is narrow and varies by layer and prompt, strong steering degrades fluency and capability, and a direction found from one set of contrasts may carry unintended correlates. It is an interpretability instrument first and a control method only with careful evaluation.

Steering works because the residual stream is roughly linear in the features it carries — the same property that makes the logit lens and linear probes work at all.

Terms

Steering vector
A direction added to the residual stream to change behaviour.
Mean difference
The difference of average activations on two sets of examples.
Steering strength
The scale α of the added direction.
Representation engineering
Reading and controlling behaviour through activation directions.
Adding a meaningful direction changes behaviour causally; the useful strength is narrow, and too much breaks the model.

Sources

  1. Turner et al., “Steering Language Models With Activation Engineering”, arXiv:2308.10248, 2023.
  2. Rimsky et al., “Steering Llama 2 via Contrastive Activation Addition”, ACL 2024.
  3. Zou et al., “Representation Engineering: A Top-Down Approach to AI Transparency”, arXiv:2310.01405, 2023.
  4. Templeton et al., “Scaling Monosemanticity”, Anthropic, 2024.