Vol. XX · Safety Training Plate 2 of 10model specification, constitution, principles, chain of command, helpfulness, harmlessness, conflictsAE

Writing the Rules Down.

A specification is a set of priorities for when good goals conflict.

Developers increasingly publish what their models are meant to do. Anthropic published the constitution used in Constitutional AI in 2023; OpenAI’s Model Spec (2024) sets out objectives, rules and defaults, and a chain of command for whose instructions take precedence — the platform, then the developer, then the user. A specification matters most where goals conflict: a request that is usually benign but occasionally dangerous, a user instruction that contradicts a developer’s, a truthful answer that will upset. It says which consideration wins, and gives examples.

Specifications act on models through training — they generate and filter fine-tuning data, they instruct the judges of AI feedback (plate XVIII.9), and newer methods train models to reason about the specification explicitly before answering — and through evaluation, which checks behaviour against them. The workshop uses a stylised model of one such conflict: requests with a latent level of harm, a model that perceives it imperfectly, and a specification whose relative weight on helpfulness and caution sets where the model declines.

Below: the rates of declining legitimate requests and of helping harmful ones, as the specification’s weight shifts between helpfulness and caution; the chosen balance is marked.

What you will learn
Workshop
A stylised model, not a trained network: 400 requests (75% clearly benign, 15% sensitive but legitimate, 10% harmful) with a latent harm level, perceived with noise 0.1; the model declines when the perceived level exceeds τ, set by the specification’s weights as in the formula.

Priorities, not just rules

With equal weight on helpfulness and caution, this model declines 7.8% of legitimate requests and helps 2.4% of harmful ones. Shifting the specification towards helpfulness (8 to 2) cuts the first to 1.4% and raises the second to 24.4%; shifting it towards caution (2 to 8) does the reverse: 26.7% and 0.0%. No setting of the weights removes both errors, because the model perceives harm imperfectly; the specification only chooses which error to make more often.

That is the honest content of a specification: a statement of priorities for unavoidable trade-offs, made public so it can be criticised. Real specifications go further than a single threshold — they distinguish kinds of harm and kinds of request, prefer safe partial help to blanket refusal, and settle whose instructions win — but the structure is the same. What a specification cannot do by itself is improve the model’s judgement of which requests are harmful; that is a matter of training and capability (plate XX.3).

Deliberative alignment trains a model to recall the relevant parts of the specification in its chain of thought and reason from them before answering, so the specification acts at inference as well as through training data.

Terms

Model specification
A document stating how a model should behave and resolve conflicts.
Constitution
A set of principles used to train and judge a model.
Chain of command
The order in which platform, developer and user instructions take precedence.
Default behaviour
What a model does when no instruction says otherwise.
A specification sets priorities for conflicts that cannot be avoided; it chooses which error to make, while judgement determines how often errors occur.

Sources

  1. OpenAI, “Model Spec”, May 2024 (objectives, rules, defaults; chain of command).
  2. Anthropic, “Claude’s Constitution”, May 2023.
  3. Guan et al., “Deliberative Alignment: Reasoning Enables Safer Language Models”, arXiv:2412.16339, 2024.