Vol. XXXI · History and Projects Plate 5 of 14history, RLHF, InstructGPT, ChatGPT, reasoning models, agents, METR time horizonDCE

Assistants and Reasoning.

The same pre-trained network, taught to follow instructions, then to reason, then to act.

RLHF began as a way to train agents from human comparisons (Christiano et al., 2017). InstructGPT (2022) applied it to GPT-3 after supervised instruction tuning, and labellers preferred a 1.3-billion-parameter InstructGPT to the 175-billion-parameter GPT-3: alignment made a small model more useful than a large one. ChatGPT, the same recipe in a chat interface, reached a mass audience in late 2022.

Then came reasoning models trained with RL on verifiable rewards (Vol. XIX) and agents that use tools over long tasks (Vols. XXV–XXVII). METR measures progress as a time horizon: the length of task, timed on skilled humans, that a model completes half the time. The workshop plots reported horizons and lets the reader extend the trend at a chosen doubling time.

Left: milestones from RLHF to agents. Right: reported METR 50% time horizons (log scale), with a trend from the first point at the chosen doubling time.

What you will learn
Workshop
Reported 50% time horizons: Claude 3.7 Sonnet about 50 minutes and o3 about 110 minutes (METR, 2025); public frontier about 12 hours, 95% interval 5–61 hours, in the February–March 2026 assessment window (METR, May 2026; reported). The task suite cannot reliably measure horizons above 16 hours.

A trend line, and how far to trust it

METR’s 2025 paper reported the horizon doubling about every seven months since 2019, possibly faster since 2024. Starting from Claude 3.7 Sonnet’s 50 minutes in early 2025, a seven-month doubling predicts 2.6 hours by February 2026; METR’s 2026 assessment reported about 12 hours for the public frontier — but with a 95% interval of 5 to 61 hours, and on a suite whose longest tasks barely reach 16 hours. The doubling time that fits both points, 3.0 months, is reported, not established: two points and a wide interval do not pin down a rate.

The pattern of the whole history is visible here. Each stage reused the last: assistants are pre-trained models plus post-training; reasoning models are assistants plus RL on checkable tasks; agents are reasoning models plus tools, environments and long contexts. And each step of capability was measured on benchmarks that then saturated, which is why the field keeps building new ones — and why the open questions of plate XXX.10 are mostly about measurement.

METR’s time horizon is a property of a model and a task suite together: horizons at 80% success are several times shorter than at 50% (about 1.5 hours in the 2026 report).

Terms

Instruction tuning
Supervised fine-tuning on instructions and responses.
RLHF
Reinforcement learning from a reward model trained on human comparisons.
Reasoning model
A model trained by RL to think at length before answering.
Time horizon
The human task length a model completes at a given success rate.
Assistants, reasoning models and agents were successive layers of post-training and tooling on pre-trained models; METR’s horizons put a rough, reported trend through them.

Sources

  1. Christiano et al., “Deep Reinforcement Learning from Human Preferences”, NeurIPS 2017.
  2. Wei et al., “Finetuned Language Models Are Zero-Shot Learners”, ICLR 2022.
  3. Ouyang et al., “Training language models to follow instructions with human feedback”, NeurIPS 2022.
  4. Bai et al., “Constitutional AI: Harmlessness from AI Feedback”, arXiv:2212.08073, 2022.
  5. Kwa, West et al. (METR), “Measuring AI Ability to Complete Long Tasks”, arXiv:2503.14499, 2025.
  6. METR, “Frontier Risk Report (February to March 2026)”, 19 May 2026.