Open Problems in Safety.
The capabilities in this course raise questions the course cannot close.
This volume describes the field as of October 2026. Claims are marked established (replicated or long-standing), reported (from a single lab or report, not independently reproduced) or contested (actively disputed). The safety volumes of this course (XX, XXV.8, XXVII, XXVIII) taught mechanisms and defences. This plate maps where research stands. Scalable oversight: how people can supervise models on tasks they cannot easily check — debate, recursive decomposition, and AI-assisted review are proposals, none established at scale. Evaluations: developers and independent evaluators test models before release for dangerous capabilities and autonomy (Vol. XXI, plate XXI.8); METR’s report on February–March 2026 put the public frontier’s 50% time horizon at about 12 hours with a 95% interval of 5 to 61 hours, and noted its task suite cannot reliably measure above 16 hours (reported).
Interpretability at scale: sparse-autoencoder features and circuit tracing on frontier models (Vol. XXVIII) are advancing, but explaining a model’s behaviour completely is not yet possible. Control: designing deployments so that even a misbehaving model cannot cause serious harm — monitoring, permissions, sandboxing (plate XXV.8) — is an active engineering field. The timeline shows the institutional milestones; select one to highlight it.
Below: a timeline of safety institutions, reports and policies, with the chosen event highlighted.
- Name the main open safety problems.
- Place key institutions and reports in time.
- Read evaluation results with their uncertainty.
- Connect the course’s mechanisms to open questions.
Measured, uncertain, open
The evaluation numbers illustrate the state of the field: METR’s best estimate for the public frontier in early 2026 was about 12 hours at 50% success, but the 95% interval ran from 5 to 61 hours and the internal frontier was likely beyond 16 — beyond what its tasks can reliably measure. At 80% success the horizon was about 1.5 hours: reliability lags capability (plate XXV.5). Measurement itself is becoming a bottleneck as models outgrow the benchmarks built to track them.
None of the four problems — oversight, evaluation, interpretability, control — is solved, and each depends on the others: interpretability could make oversight easier, evaluations decide which safeguards are needed, control limits the damage when the rest fails. Two years earlier much of this was conjecture; by 2026 it has institutions, published frameworks, government-commissioned reviews and regular reports. The direction of travel is measurable; the destination is not settled.
Terms
- Scalable oversight
- Supervising systems on tasks people cannot easily check.
- Dangerous capability evaluation
- Testing a model before release for capabilities that could cause serious harm.
- AI control
- Deployment designs that limit harm even if a model misbehaves.
- Safety framework
- A developer’s published policy tying capabilities to required safeguards.
Sources
- METR, “Frontier Risk Report (February to March 2026)”, 19 May 2026.
- Bengio et al., “International AI Safety Report 2026”, arXiv:2602.21012, 2026.
- Irving, Christiano and Amodei, “AI safety via debate”, arXiv:1805.00899, 2018.
- Greenblatt et al., “AI Control: Improving Safety Despite Intentional Subversion”, ICML 2024.
- Anthropic, “Responsible Scaling Policy”, September 2023 and later versions.