Drills: Safety Training.
Five questions at a time from the whole volume, until the gap between objective and intent is the first thing you look for.
This plate draws rounds of five from the questions and predictions of the nine plates before it, with the reasoning and source plate after each answer.
The volume’s thread is the gap between what is written down and what is meant: specifications narrow it, judgement and robustness close parts of it, honesty and calibration make it visible, and evaluation — with its statistics stated plainly — measures how much remains.
Below: a round of five. Answer each, read the explanation, and press “Next round” for another five.
- State the alignment problem.
- Read refusal trade-offs.
- Measure calibration and abstention.
- Interpret safety evaluations.
What to carry forward
Alignment is the gap between objective and intent (plate XX.1); specifications set priorities for unavoidable conflicts (plate XX.2); over- and under-refusal trade along a curve that judgement moves (plate XX.3); adversarial training extends judgement to reworded inputs (plate XX.4). Calibration measures honesty about uncertainty (plate XX.5), and abstention helps against hallucination as far as confidence separates right from wrong (plate XX.6).
Optimisers find the best-paying behaviour an objective allows (plate XX.7); oversight schemes stretch limited judgement (plate XX.8); and rare failures need many attempts to see (plate XX.9). Part Five turns to evaluating and serving the model.
Terms
- HHH
- Helpful, honest, harmless.
- Over-refusal
- Declining legitimate requests.
- ECE
- Expected calibration error.
- Rule of three
- Clean in n: p < 3/n.