Blog

CIKM study shows LLM-as-a-Judge systems are swayed by prior scores

New studies show a single AI score hides bias and output variation. Why defensible evaluation requires repeated runs, context comparison and independent assessors.

· By

Wooden worktable with rows of near-identical hand-marked test sheets, a whiteboard of tally columns, a magnifying glass and a separate tray of coloured index cards.
A single AI score hides bias and variation; defensible evaluation requires repeated runs and independent assessors.Image: IamVera.ai — original editorial illustration

A single overall score, one benchmark or one model run hides bias and variation. Defensible evaluation combines pre-registered, representative tests with repeated identical runs, variance and error-type analyses, context and language comparisons and independent assessors whose own bias is tested separately.

The immediate occasion is a study published by the Association for Computing Machinery in November 2026 as part of CIKM ’26. In it, the authors demonstrate empirically that so-called LLM-as-a-Judge systems — language models deployed to assess the output of other models — can be swayed by previously shown scores. Seven of the eight models studied exhibited a measurable anchoring effect. In a categorical industrial task, prior score information blocked 48% of error corrections and turned 10.18% of correct judgements into wrong labels. In our analysis, this means that even an instrument organisations use to pursue more objective assessment can itself shift systematically in the wrong direction.

This concerns everyone who has AI output assessed before a decision follows. Those who rely on a single figure or a single automated assessor may be measuring mainly the bias of their own measurement setup. More background on this theme can be found in our topic hub on AI verification and controllable assessment.

What does the CIKM 2026 study on anchoring bias in LLM-as-a-Judge demonstrate?

The study demonstrates that a previously shown score systematically shifts an assessor model's later judgement. The authors of the CIKM 2026 study on anchoring bias in LLM-as-a-Judge systems measure this both for numerical assessments and for categorical yes/no decisions, and conclude that context design and validation per task and per domain are necessary.

Importantly, the effect works in two directions: it blocks corrections an unbiased assessor would make, and it turns correct judgements into wrong labels. In our assessment, this means that the order in which an assessor sees information is a design choice that must be recorded and tested, not a detail left to chance. Anyone who first shows an assessor the score of a previous model is measuring the new model together with the after-effect of the previous judgement, unless that effect has been controlled and tested.

Why is one model run insufficient to measure output variation?

Because the same question put to the same model can yield differing answers on repetition, one run can provide an incomplete picture of reliability. Researchers at the University of Edinburgh developed a Monte Carlo protocol with a hundred identical runs for data science tasks. They report substantial variability, particularly for medium and difficult tasks, showing why instability should be measured explicitly rather than inferred from a single run.

The core of their argument is that a single measurement can conceal variation and inconsistency. An average accuracy that looks good can hide a large spread. That is why consistency is, according to the authors, a separate evaluation dimension alongside the average. In practice this calls for reporting spread, error types, refusals, answer length and stylistic differences — not just a single score.

How do benchmark choice, language and context bias the outcome of an AI evaluation?

Strongly, because measured results can change with benchmark choice, language and context, even when the underlying model is unchanged. A preprint for ACM Foundations of Software Engineering examined hidden bias in Codeforces-based Elo evaluations of language models and found differences of up to 394 Elo points from the order of submissions, up to 1,122 points from the choice of contests and up to 349 points between repeated runs on the same contests. The authors recommend standardisation, multiple runs and transparent reporting of the evaluation settings.

Context and language matter independently as well. Researchers introduced the Polar benchmark for political bias in language models, comparing 38 models across US and South Korean political contexts and several presentation languages. They report that measured bias differs systematically by context, category, model group and language. A related study on gender stereotyping in AI-generated children's stories found systematic but context-dependent differences and showed that lexical metrics and human assessments do not fully measure the same thing. The practical lesson: a general bias outcome does not suffice when answers vary by language, audience and subject.

What does this mean for directors, lawyers and CISOs assessing AI output?

Our analysis: because an assessor model demonstrably leans towards previously shown scores, a CISO has the assessment order recorded and tests the assessor separately for anchoring, order and source-framing effects before that assessor may support a procurement or risk decision. Because one run hides the spread, a director demands, at acceptance, repeated identical runs with reported spread and error types, not one demonstration. Because benchmark and contest choice can shift the outcome by hundreds of points, a lawyer includes a clause in the agreement that records the test set, the assessment framework, the sampling settings and the exclusion rules in advance and that permits one's own re-testing with ring-fenced data, as we described earlier under accepting an AI performance claim only after your own test. Because bias differs by language and context, a responsible party, for multilingual or sensitive use, has the tests stratified by language, context and subject before the system goes live.

Translated into a workable protocol, this means a defensible evaluation contains at least the following layers:

  • Pre-registered, representative tests with matched prompts in which only the relevant variable changes, so that a difference found can genuinely be attributed to that variable.
  • Repeated identical runs with reporting of spread, refusals, length, style and inconsistencies alongside the average.
  • Multiple difficulty levels, languages and domains, because the outcome moves with the test selection.
  • A separate test of the assessor's behaviour on anchoring, order and position effects, preferably with deterministic or executable verifiers and independent human samples.
  • Recording of model version, prompt, sampling settings, context, test set version, assessment rubric, seeds, number of runs and exclusion rules, with uncertainty intervals and explicit abstention when the evidence falls short.

This layering aligns with broader working methods we have described before, such as verifying complex AI answers via sub-claims and setting up an LLM red-teaming workflow with defensible scoring. The common denominator is sober: a single figure is a starting point, not proof.

Sources and references

  1. Anchoring Bias in LLM-as-a-Judge Systems: Prior Scores Compromise Evaluation IndependenceAssociation for Computing Machinery (CIKM 2026) · 2026-11-06
  2. One Run Is Not Enough: A Monte Carlo Protocol for Evaluating LLM Reliability on Data Science TasksUniversity of Edinburgh via arXiv · 2026-11-09
  3. When Elo Lies: Hidden Biases in Codeforces-Based Evaluation of Large Language ModelsACM Foundations of Software Engineering preprint via arXiv · 2026-02-05
  4. Polar: A Benchmark for Evaluating Political Bias in LLMsAssociation for Computational Linguistics via arXiv · 2026-06-01
  5. Once Upon a Bias: Gender Stereotyping in LLM-Generated Children's StoriesarXiv preprint · 2026-11-19

Sources: The article draws on studies from the Association for Computing Machinery (CIKM 2026), the University of Edinburgh, an ACM FSE preprint, the Association for Computational Linguistics and an arXiv preprint on gender stereotyping.

← All articles in this topic ← All articles