The confidence score an AI model reports itself is of limited reliability. Research on arXiv and OpenReview shows that such verbalised scores are heavily quantised, often stay high even for wrong answers, and are shaped more by the wording of the prompt than by the question itself. A single figure from one model is therefore no proof of correctness.
According to MIT News, more reliable uncertainty signals come from disagreement between multiple models, and according to clinical research from task-specific calibration that differs per domain and question type. For work with confidential or high-risk information, the core question is therefore not whether a model can show a number, but which signal is reliable enough to act on, under what conditions, and with what layer of control around it.
The occasion for this piece is a series of recent publications. On 19 August 2026, arXiv published the study Improved Confidence Estimates for Black-Box Large Language Models, which builds better uncertainty estimates by combining uncertainty scores with features of comparable earlier questions. In doing so, that study builds on earlier work that exposes the weak spots of a simple confidence figure.
What exactly has the research into verbalised confidence found?
By verbalised confidence, researchers mean that you ask a model to attach a certainty percentage to its own answer. The arXiv paper On Verbalized Confidence Scores for LLMs examines how those scores come about and where they fail. Two findings are relevant for practice:
- The scores are heavily quantised: models fall back on a handful of fixed values rather than a fine-grained estimate.
- The wording of the prompt influences the score materially; the same answer can be given a different certainty with a different phrasing of the question.
The OpenReview study Always So Sure: Can LLM's Confidence be Trusted? reaches a related conclusion: a confidence figure is only weakly connected to the underlying question and often stays high, even when the answer is wrong. Both sources stress that a score should not be judged only on its presence, but on calibration, informativeness and meaning.
Our editorial assessment: a visible percentage in an interface evokes a trust that the figure itself does not always live up to. It is an interface element, not a piece of evidence.
Why is disagreement between models a better uncertainty signal?
On 19 March 2026, MIT News described a method to better identify overconfident language models. The core: disagreement between different models captures epistemic uncertainty better than repeatedly querying a single model. According to that publication, a combined uncertainty measure can better indicate when an answer is confidently wrong than a single self-report.
Why does that work? If one model is consistently wrong, it also stays consistently self-assured. When independent models diverge from one another, that is a signal that the question is difficult or ambiguous. We have previously described when disagreement between models is a usable signal and where its limits lie. Importantly: agreement between models is no guarantee of correctness, only an indication.
Why does the reliability of confidence differ per domain and question type?
That there is no universal confidence method is clear from calibration research in the Transactions of the Association for Computational Linguistics: How Can We Know When Language Models Know?. That study shows that the reliability of uncertainty estimates depends on the domain and the question type. What is well calibrated in one specialism need not be so in another.
For professions handling confidential files, this means you do not validate confidence once and then trust it blindly. Per task type — legal research, medical questions, numerical analysis — the same signal may have a different value. Anyone wanting more on this will find related pieces in our topic hub on AI verification, and in how teams build hallucination detection in layers a concrete elaboration of the same line of thinking.
How do you decide in practice which confidence signal to trust?
Based on the sources named above, we order confidence into three layers, from weakly to more strongly supported. This is our editorial ordering, not a literal classification from a single source:
- A self-reported figure from one model. Useful as an indication, but sensitive to prompt wording and often overconfident. Not to be used as the sole basis for automation.
- Disagreement between independent models. According to MIT News, more informative for detecting confidently wrong answers.
- Task-specific uncertainty calibration. Measured per domain and question type, as the calibration research and the recent black-box estimation on arXiv show.
A practical checklist that follows from this:
- Ask not only whether there is a score, but which method produced it.
- Test whether the same question with different phrasing gives a different figure.
- Let low agreement or high uncertainty trigger a human check rather than automatic handling.
- Treat confidence separately per domain; do not reuse calibration uncritically.
A verification layer fits here. The Vera verification layer for AI answers is not a chatbot and not its own language model, but a privacy-focused verification layer that can route a task through selected independent models and make verification steps, corrections, disagreement and sources visible for inspection. That can help show which confidence signal belongs to an answer and when an outcome is better sent to human control. It gives no guarantee of correctness and does not remove the need to check for hallucinations; the professional final judgement remains with the user.
The common thread from this research: a confidence figure is a starting point, not an end point. The question is not whether a model sounds self-assured, but which signal you trust enough to act on.
Sources and references
- Improved Confidence Estimates for Black-Box Large Language Models
- On Verbalized Confidence Scores for LLMs
- A better method for identifying overconfident large language models
- Always So Sure: Can LLM's Confidence be Trusted?
- How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering
Sources: The article draws on research from arXiv, OpenReview, MIT News and the Transactions of the Association for Computational Linguistics on the reliability of AI confidence.