Vol. XXX · The Research Frontier Plate 10 of 11open questions, generalisation, reasoning, scaling limits, continual learning, oversight, evaluationEL

What We Do Not Know.

A course should end with the questions it could not answer, stated precisely.

This volume describes the field as of October 2026. Claims are marked established (replicated or long-standing), reported (from a single lab or report, not independently reproduced) or contested (actively disputed). Each row of the table is a question that researchers in October 2026 would recognise as central and unresolved, tagged open (little evidence either way) or contested (evidence on both sides, actively argued), with the volumes of this course that give the background to read new work on it.

Stating a question precisely is most of the work of answering it: “do models understand?” is unanswerable; “does a probe find a linear representation of the board state, and does intervening on it change the move?” is an experiment (plates XXVIII.2–3). The workshop lets you pick a question to highlight; the prose suggests how each could be sharpened into a measurement.

Below: the open questions, their status and the volumes behind them, with the chosen one highlighted.

What you will learn
Workshop
Questions selected for this course; status as judged from published evidence up to October 2026. Volumes: where this course gives the background.

Questions that can be measured

Of the 9 questions here, 4 are contested — there is evidence on both sides — and 5 are open. Every one becomes tractable when phrased as a measurement this course has shown: transfer as held-out performance on unverifiable tasks after verifiable-task RL (Vols. XIX, XXI); scaling limits as fitted curves with ceilings (plates X.1, XXX.2); continual learning as retained accuracy on old tasks after new training (plate XXX.3); understanding as probes plus interventions (Vol. XXVIII); agent reliability as pass^k and time horizons (plates XXV.5, XXI.8).

The field moves fast, and some of these may be settled by the time you read this — which is the point of having learned the methods rather than only the results. The next volume looks back at how the field reached this point, and then asks you to build each stage yourself.

“Contested” is not a polite word for “wrong”: on several of these questions careful researchers read the same evidence differently, and new measurements, not arguments, will settle them.

Terms

Open question
One with little evidence either way.
Contested question
One with evidence on both sides, actively argued.
Operationalisation
Turning a vague question into a measurement.
Held-out evaluation
Testing on data or tasks not used in training.
The field’s biggest questions are tractable when stated as measurements; the methods of this course are the tools for doing so.

Sources

  1. Bengio et al., “International AI Safety Report 2026”, arXiv:2602.21012, 2026.
  2. Mitchell and Krakauer, “The debate over understanding in AI’s large language models”, PNAS 2023.
  3. Kaplan et al., “Scaling Laws for Neural Language Models”, arXiv:2001.08361, 2020.