Vol. XXI · Evaluation Plate 5 of 10LLM-as-a-judge, position bias, verbosity bias, self-preference, MT-Bench, evaluation, agreementEL

Models as Judges.

A model that grades answers is a measuring instrument. Calibrate it before trusting its readings.

Open-ended answers — an essay, an explanation, a plan — have no answer key, and human judging is slow. Zheng and colleagues (2023) showed that a strong model prompted to compare two answers, or to grade one, agreed with human experts about as often as experts agreed with each other on their MT-Bench questions. LLM judges now grade much of the field’s evaluation and much of its synthetic training data (plate XVII.4). They also showed the judges’ biases: position bias (preferring the answer shown first), verbosity bias (preferring longer answers) and self-enhancement (preferring answers in their own style).

The workshop simulates a judge comparing pairs of answers of known quality, with a chosen skill and chosen position and length biases. It measures how often the judge picks the better answer, how often its verdict survives swapping the two answers’ order, and how often, between answers of nearly equal quality, it picks the longer one.

Below: the judge’s agreement with true quality, its consistency when the order is swapped, and its preference for the longer of two near-equal answers, for the chosen biases.

What you will learn
Workshop
A stylised judge, not a trained model: 600 pairs of answers with qualities drawn from a standard normal and lengths long or short at random; the judge picks with the logistic rule above. Consistency: the same verdict with the order swapped. Length preference: among pairs within 0.5 in quality, the share where the longer answer wins.

Measure the instrument

With skill 1.5, position bias 0.5 and length bias 0.6, the judge picks the better answer 80% of the time and gives the same verdict when the order is swapped only 67% of the time; between answers of nearly equal quality it picks the longer one 66% of the time. Without the biases the consistency is 70% and the length preference 48%. A sharper judge (skill 4) agrees with quality 91% of the time — biases matter most when the judge is unsure.

Mitigations follow the biases. Judge both orders and count only consistent verdicts, or average them. Control for length explicitly, as length-controlled AlpacaEval does by regressing out the length difference. Avoid using a model to judge its own outputs. Provide a rubric or a reference answer, which raises agreement. And validate the judge against human labels on a sample before trusting it at scale — the same discipline as checking any measuring instrument.

A judge can be gamed like any reward: when an LLM judge is used as the reward for training (plate XVIII.9), the policy learns to exploit the judge’s biases, so a judge that is adequate for evaluation can fail as a training signal.

Terms

LLM-as-a-judge
Using a language model to compare or grade answers.
Position bias
Preferring an answer because of where it is shown.
Verbosity bias
Preferring longer answers regardless of quality.
Self-enhancement bias
Preferring answers in one’s own style.
An LLM judge is an instrument with position, length and self-preference biases: swap orders, control length, and validate against people.

Sources

  1. Zheng et al., “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena”, NeurIPS 2023 Datasets and Benchmarks.
  2. Dubois et al., “Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators”, COLM 2024.
  3. Panickssery, Bowman and Feng, “LLM Evaluators Recognize and Favor Their Own Generations”, NeurIPS 2024.