LLM Confidence vs. Accuracy: Is a Sure Answer a Right One?

Not reliably. On Humanity's Last Exam, where models give an answer and a confidence from 0% to 100%, the benchmark's own page (read on 30 September 2026) lists RMS calibration errors from 50.0% to 89.0% across ten models, and 0% would be perfectly calibrated. Explanations add to it: in experiments with 301 participants, people overestimated how accurate LLM answers were, and longer explanations raised their confidence without raising accuracy. A tally of stated confidence against answers you recompute shows where your own model sits.

Humanity's Last Exam (HLE) consists of 2,500 questions developed by subject-matter experts and was published in Nature in 2026 (arXiv preprint posted January 2025). A model is calibrated when its stated confidence matches how often it is right, so even the lowest listed error is far from calibrated. The authors also say the low accuracy is "partially by design", because questions were filtered against existing models, so read this as a stress test on hard questions, not a rate for everyday ones.

Xiong and colleagues (ICLR 2024, peer-reviewed) evaluated five LLMs on five types of datasets and found that LLMs, "when verbalizing their confidence, tend to be overconfident". Consistency among multiple responses helped reduce it. Kadavath and colleagues (arXiv preprint, 2022) found that larger models are "well-calibrated on diverse multiple choice and true/false questions when they are provided in the right format". Confidence can carry information in a constrained format; the evidence above is about what happens when a model states it in words.

Why do fluent explanations feel right?

Steyvers and colleagues (first posted 2024, published in Nature Machine Intelligence) had 301 participants judge answers from three LLMs to multiple-choice and short-answer questions. Users "tend to overestimate the accuracy of LLM responses when provided with default explanations", and "longer explanations increased user confidence, even when the extra length did not improve answer accuracy".

None of these tested attribution questions or marketing exports, and this page cites no measurement that did.

How do you measure how far to trust it?

Build a calibration tally from your own data:

1. Write the questions. Pick questions whose answers you can recompute from your export: totals by channel, order counts, the top five products.
2. Ask with a confidence. Add "give a confidence as a percentage" to each question and keep the answers.
3. Recompute each answer with a pivot or SQL and mark it right or wrong.
4. Group by stated confidence. Sort the answers into low, medium and high groups and count the share that was right in each.
5. Ask twice. Put every question again in a fresh chat. SelfCheckGPT (EMNLP 2023, peer-reviewed) rests on the idea that sampled answers to a hallucinated fact "are likely to diverge and contradict one another". Differing numbers are a warning sign; matching numbers do not show the answer is right.

Pass: the share right rises from the low group to the high group, and the high group is right about as often as it claims. Fail: the high group is wrong about as often as the low group, or the same question returns different numbers. Small groups are noisy. For illustration, with ten answers in a group, one answer more or less moves its share right by ten percentage points.

An LLM helps with this too: it can write the recomputation queries and lay out the tally. The judging stays with you, and it is worth repeating when the model version changes.

Sources, 30 September 2026: Humanity's Last Exam (Center for AI Safety, Scale AI and the HLE contributors, Nature, 2026; arXiv preprint, 2025; full text; benchmark page and leaderboard); Can LLMs Express Their Uncertainty? (Xiong et al., ICLR 2024); Language Models (Mostly) Know What They Know (Kadavath et al., arXiv preprint, 2022); What Large Language Models Know and What People Think They Know (Steyvers et al., Nature Machine Intelligence; full text); SelfCheckGPT (Manakul et al., EMNLP 2023).

Related answers

• Why LLMs invent numbers, and how to catch them
• Same question, different answer: LLM reproducibility
• Audit trails for LLM attribution answers
• Incrementality testing