When an AI model gives an answer, it usually sounds equally convinced — whether it is correct or not. That is precisely the problem exposed by a series of studies from 2026. Confidence scores, the certainty figures that models attach to their output, are often treated in practice as a kind of truth meter. Recent research describes this as a dangerous assumption, especially in domains where mistakes have major consequences.
The immediate occasion is the paper Demystifying Uncertainty in LLMs: Active Calibration between Model Concepts and Human Evaluations, published via ACL Anthology on 3 July 2026. The authors argue theoretically and show empirically that the calibration error of large language models in interactive applications has a hard lower bound: without targeted interaction, that error remains non-vanishing. Only through active calibration — in which queries are deliberately selected on the basis of their calibration error using an Interactive Learning Strategy — can the reliability of the certainty estimate be measured and improved. The message: confidence is not a fixed property of a model, but something that must be actively measured and adjusted.
What calibration actually means
Calibration sounds technical, but the idea is simple. The practical guide LLM Calibration and Uncertainty Quantification in Production (Zylos, April 2026) puts it this way: a confidence of 80 per cent should in practice coincide with 80 per cent empirical correctness. If that is not the case, the model is poorly calibrated — it says it is certain without that certainty meaning anything.
To make this measurable, researchers use the Expected Calibration Error (ECE) as the standard measure. The survey Uncertainty Quantification and Confidence Calibration in Large Language Models (arXiv, March 2025) defines calibration as closing the gap between reported certainty and observed correctness. That same survey distinguishes several dimensions of uncertainty — in the input, the reasoning, the parameters and the prediction — and makes clear that uncertainty is not a single number, but a multi-layered signal that must be explicitly designed into an AI system.
The Zylos guide summarises the core principles succinctly: instrument uncertainty, don't assume it. According to the authors, calibration should be part of the entire training and fine-tuning pipeline, not something you add as a slider on the interface.
Self-reported certainty is systematically skewed
A second problem is that the certainty a model expresses in words — the so-called verbalized confidence — is particularly unreliable. The study Benchmarking Uncertainty Calibration in Large Language Model Scientific Question Answering (OpenReview, February 2026) introduces a large-scale benchmark for scientific question-and-answer tasks and concludes that verbal confidence is systematically biased and correlates poorly with correctness. What does work well: the frequency with which the same answer recurs across multiple samples. That frequency yields the most reliable calibration. The authors emphasise that only well-calibrated scores within the range [0,1] are usable as a basis for risk thresholds.
That theoretical insight takes on a sharp practical face in The State of AI Reliability (Dixon, June 2026). This benchmark on verifiable facts introduces the category confident errors: answers with a source and a self-assured tone that are simply wrong in substance. In a series of ninety runs on core financial questions, roughly 3 per cent of the results turned out to be such confident errors. On questions made more difficult, the models sounded just as self-assured on poorly supported answers as on well-supported ones — with no signal in the output marking the difference.
That is the heart of the risk: not that AI makes mistakes, but that the certainty with which those mistakes are presented cannot be distinguished from justified certainty. Anyone who blindly trusts the confidence slider in a dashboard misses precisely the distinction that matters.
Uncertainty as verifiable architecture
The coherent conclusion of these sources is that confidence should be treated not as a comfort signal but as a risk signal. The arXiv survey points out explicitly that well-calibrated confidence is needed to route low-certainty predictions to human verification and to limit overconfidence on incorrect answers. That calls for a number of concrete choices: combining multiple sources of uncertainty (probabilistic, semantic, frequency), setting thresholds and handover rules, and recording calibration measurements in audit logs so that it can afterwards be verified when and why human oversight was needed.
For those who work with confidential information — lawyers, occupational physicians, journalists, compliance teams — this is more than a technical nicety. The difference between a well-supported answer and a confidently-wrong one can be decisive for an advice or a decision.
What this means for controlled AI use
Here lies the connection with a verification approach such as that of I am Vera. Vera is not a language model and not a chatbot, but a verification layer: it makes verification steps visible instead of trusting what a model says about itself. By comparing AI answers across multiple models, a multi-model verification can help make visible where models diverge from one another — precisely the places where a single confidence score could give a false sense of certainty.
The relevant contribution lies not in adding yet another certainty figure, but in orchestrating oversight: making uncertainty comparable between sources, and recording in an auditable way when an answer deserved extra attention. This happens within a working method in which the pre-processing and anonymisation take place on EU infrastructure and the Semantic Privacy Shield is designed to send only anonymised content to the selected models — if that privacy check fails, nothing is forwarded.
Vera does not eliminate errors and does not decide what is true. What it can do is give more insight into where certainty is unjustifiably suggested, so that the professional final judgement remains with the user. The 2026 studies make clear that this judgement cannot be left to a slider.