Honest Book: Two Kinds of Sure

Published — Applied AI Systems

Abstract

A system’s claim that it performed an action can be checked against an independent record of what actually executed — this publication’s previous paper established that pattern and the reasoning behind it. This paper addresses a different question that sits one level up: once a system is reporting how reliably it performs a class of action, not merely whether one instance succeeded, what makes that reliability number trustworthy? The intuitive answer — report the real pass rate — is necessary but not sufficient, and this paper documents a live instance of why, caught during the course of writing it: two real, honestly-computed reliability numbers, sitting in the same interface, presented with identical visual authority, backed by categorically different statistical weight. Only an internal code comment — never the interface itself — disclosed which was which. This paper traces why that gap is not cosmetic, why the obvious fix (compute a confidence interval everywhere) fails for a precise statistical reason, and what the actual fix looks like, applied directly to the system in question.

A number can be real and still mislead, if it doesn’t say what kind of number it is.

Section 1 — The Problem, Named by the Field

The previous paper in this series established that a claim of completed action must be checked against an independent execution record before it is allowed to stand. That discipline answers whether a thing happened. It does not answer a related but distinct question that any system reporting on its own performance eventually has to face: how often does this class of thing actually happen, and how much should a single observed rate be trusted to predict the next one?

This is not a niche statistical concern. It is a measured, worsening gap between what AI systems claim about their own reliability and what independent evaluation finds. A 2026 benchmark built specifically to separate “can do it once” from “reliably does it” — 507 real executable business workflows across five domains, each run 20 times — found the field’s strongest model scoring 65.36% on a single attempt (pass@1) but only 25.25% when required to succeed on every one of 20 attempts (pass^20) (ThinkingBox, Aug 2026). That is a 66-point gap between the number a single run produces and the number that describes how the system actually behaves under repetition — and pass@1 is the number most benchmarks, and most systems narrating their own performance, report by default.

The gap is not confined to behavior; it extends to the model’s own account of its behavior. A 2026 study isolating the mechanisms behind LLM calibration failure found that models assign up to 26% higher confidence to their own generated answers than to the identical answer when it is framed as having come from someone else — a measurable “ownership bias” baked into self-assessment itself (Large Language Models Are Overconfident in Their Own Responses, ACL 2026 Findings). A system’s own report of how sure it should be is not a neutral measurement. It is generated by the same process being measured, with a documented thumb on the scale in its own favor.

Nor is under-reporting of uncertainty specific to language models. A review of 221 medical imaging segmentation papers published in a single year at MICCAI — a field with real stakes and a strong empirical culture — found that more than half assessed no performance variability at all, and exactly one paper, 0.5% of the total, reported a confidence interval for its headline result (Confidence intervals uncovered: Are we ready for real-world medical imaging AI?, MICCAI 2024). A companion analysis of outperformance claims across the same literature found a high probability of a false outperformance claim — greater than 5% — in 86% of classification papers and 53% of segmentation papers that claimed to beat a prior method (False Promises in Medical Imaging AI? Assessing Validity of Outperformance Claims, 2026). The pattern across both fields is the same: a single point estimate is treated as a fact, when the actual claim it supports depends entirely on how much uncertainty that estimate carries — uncertainty which, more often than not, goes unreported rather than merely unfavorable.

This paper does not treat this as someone else’s problem. It documents an instance of the same failure shape occurring inside the system this publication has already described as built under a constraint-driven, verify-before-claim discipline — found not in a competitor’s benchmark, but in this system’s own evaluation interface, during the writing of this paper.

Section 2 — What This Looks Like From the Inside

The system in question runs two real, independently computed self-evaluation panels side by side, in the same interface, under the same visual language.

The first reports a pass rate drawn from real production runs scored by an independent evaluation harness — not a simulated or hand-picked figure, but the actual outcome-quality scores a judge process assigns to real sandboxed agent runs. At the time of writing it read: 76% observed pass rate, 16 of 21 runs scored at or above the pass threshold, with a second line directly beneath it: projected range at 95% confidence, 55%–89%, computed as a real Wilson score interval on those 21 observations.

The second panel, directly below it in the same layout, reports a different number: 45%, 5 of 11 correct, broken down by class, with individual pass/fail rows for each test case. This number is also real — it comes from an actual subprocess call into the system’s live trained checkpoint, run against a fixed battery of hand-written test directives, not a cached or simulated response. Both numbers are large, bold, and colored by the same pass/fail convention. Nothing in the rendered interface visually distinguished one from the other.

The difference between them is not cosmetic. The first number is a statistical estimate: 21 independent observations, drawn from real usage, summarized with a confidence interval that honestly states how much that estimate could move with more data. The second number is a real but deliberately non-random check: eleven directives, hand-written specifically to be unambiguous, chosen to catch drift across retrains rather than to represent the true distribution of requests the system will actually receive. The file defining that second number says so directly, in a code comment explaining exactly why a fixed battery was chosen over a statistically held-out split at this corpus size. That explanation existed, and was correct, and a person reading only the rendered panel — the thing an actual viewer sees — had no way to encounter it. The honesty lived in the source code. The interface presented both numbers as though they meant the same kind of thing.

This was not caught by a user complaint or a benchmark run. It was caught in the course of reviewing this exact panel, for this exact paper, while verifying the Wilson-interval claim this publication intended to cite as evidence of real statistical discipline — and finding, one panel down, a real number sitting at the same visual weight with none of that discipline attached to it.

Section 3 — Why the Obvious Fix Doesn’t Work

The intuitive response is to close the gap by extension: compute a confidence interval for the second number too, the same way the first one does, and print it the same way. This would be easy to implement and visually consistent. It would also be wrong, for a specific statistical reason rather than a stylistic one.

A Wilson score interval — or any confidence interval built on a binomial proportion — is a statement about sampling error under a specific assumption: that the observations being summarized are drawn, independently and at random, from the actual distribution the estimate is meant to describe. The first panel’s 21 observations satisfy that assumption reasonably well; they are real production runs, occurring as real usage occurs, not selected by anyone for any property other than having happened. The second panel’s 11 observations do not satisfy it at all. They were not sampled from the distribution of directives the system receives — they were authored, deliberately and by hand, to be clear-cut test cases for one specific purpose: catching regression across retrains. That is a legitimate and useful thing to build. It is not a random sample of anything, and a confidence interval computed over it would not describe sampling error in the quantity anyone actually cares about — it would describe sampling error in a quantity that was never random to begin with.

Printing a Wilson interval on that number regardless would not add rigor. It would borrow the visual grammar of rigor — the same format, the same percentage-with-a-range presentation the first panel uses honestly — and apply it to a number where the underlying statistical assumption doesn’t hold. That is not a correction of the original gap. It is a more sophisticated-looking version of the same gap: a number dressed in the appearance of a guarantee it cannot actually make. This is the same failure shape this publication’s previous paper described for claims of action — a fluent, well-formed presentation standing in for a check that didn’t actually happen — now occurring one level up, in how a system describes the reliability of its own self-reporting rather than the reliability of its actions.

The conclusion this forces is the same shape as the previous paper’s, applied to a different layer: the fix is not to make every number look statistically rigorous. It is to make every number say, plainly, which kind of number it is.

Section 4 — The Pattern: Label the Kind of Number, Not Just the Number

The principle: a reliability figure is not complete on its own. It requires a second, explicit statement — visible wherever the figure itself is visible, not buried in a comment only the system’s author will ever read — of what kind of evidence stands behind it. A number drawn from real, independently sampled observations, large enough to support a meaningful interval, should say so and show the interval. A number drawn from a real but deliberately non-random check — a fixed regression battery, a hand-picked smoke test, anything chosen rather than sampled — should say that instead, as plainly as the first panel states its confidence interval. Neither disclosure requires hiding the number or apologizing for it; a drift-detection battery is a legitimate and useful thing to have, and reporting 45% on one is not dishonest. What is dishonest, or at minimum careless, is placing it next to a statistically-bounded figure with no signal distinguishing the two — letting shared visual format imply a shared epistemic status that doesn’t actually exist.

This is a narrower requirement than it might first appear, and that narrowness is what makes it enforceable rather than aspirational. It does not require every system to build statistically rigorous evaluation for everything it measures — that bar is often genuinely out of reach at realistic data volumes, exactly as this system’s own Knowledge Test comment explains for an eight-class, forty-seven-example corpus. It requires only that the system say which bar a given number has actually cleared, at the point where a person is looking at that number and about to trust it. The discipline is not “be rigorous everywhere.” It is “never let an unlabeled number borrow the authority of a labeled one sitting next to it.”

Section 5 — Evidence: Before and After, in the Actual System

Before this review: the Knowledge Test panel’s subtitle read “What NIER has actually learned — 11 real cases against her live checkpoint, never from the training corpus itself.” True, and incomplete — it disclosed where the cases came from, not what kind of statistical claim the resulting percentage could or couldn’t support, and the panel’s 45% sat one scroll below the Success Rate Projection panel’s Wilson-bounded 76% with no visual or textual signal that the two numbers rested on different foundations.

After: the same subtitle now reads, in full — “What NIER has actually learned — 11 real cases against her live checkpoint, never from the training corpus itself. Fixed battery for drift-tracking, not a statistical sample — no confidence interval.” The change is one sentence, appended where the viewer already reads the number, not relocated to documentation the viewer would have to go looking for. The underlying computation is untouched; 45% still means exactly what it meant before. What changed is that the interface now says, at the point of reading, which kind of claim that number is making — and, by omission, which kind it is explicitly not making.

This is a small change by line count and a real one by the standard this publication holds itself to: the gap it closes was not hypothetical. It was sitting in production, in a system whose first paper in this series was specifically about not letting an unverified claim leave the system unchecked, until the review for this second paper found an unlabeled one in the one place that review happened to look closely.

Section 6 — Discussion and Limitations

This paper’s fix closes a narrow, specific gap: whether a reliability number discloses the kind of evidence behind it, at the point where it’s read. It does not evaluate whether the underlying checks are well-designed in the first place — whether eleven hand-written cases are the right number, or the right cases, for a drift detector; whether twenty-one sandbox runs meaningfully represent the true distribution of future production usage; whether a 55%–89% interval is narrow enough to be useful for any actual decision riding on it. A confidence interval reported honestly can still be too wide to inform anything, and a labeled drift check can still be a bad drift check. Labeling a number correctly is a precondition for trusting it appropriately. It is not a substitute for the number being good.

The pattern also assumes something that will not always be true: that a system’s author, or the system itself, actually knows which statistical category a given check falls into. In this instance that knowledge already existed — it was sitting in a code comment, correctly reasoned, before this paper started. The gap was entirely one of surfacing, not of understanding. A system where the underlying statistical reasoning is itself wrong or absent would need that problem solved first; labeling a miscategorized number accurately as to its category does not help if the category itself was assigned incorrectly.

It is also fair to ask whether this kind of labeling scales — whether a system with dozens of self-reported metrics can keep each one honestly captioned without the captions themselves becoming the next thing nobody actually reads. This paper has evaluated exactly one instance, fixed directly, in the specific system this publication documents. Whether the discipline holds under many more metrics, enforced consistently rather than caught one panel at a time during the writing of a paper, is the open question this paper leaves for the next one to test against real data rather than assert in advance.

Section 7 — Conclusion

This publication’s first paper argued that a fact should be computed, not generated. Its second argued that a claim of action should be checked against a record, not narrated on trust. This paper makes the same reframing a third time, one level higher: a claim about reliability — how often, not just whether — should disclose what kind of evidence it rests on, not borrow the authority of whatever number happens to sit next to it. None of these three moves are really distinct ideas. They are the same discipline, found again at a different layer each time a new place was discovered where fluency could stand in for verification without anyone deciding that it should.

The incident this paper documents was not dramatic. Nothing crashed, no one was misled in a way that caused visible harm, and the underlying computation — in both panels — was genuinely real the entire time. That is precisely the point worth sitting with: a system can be built by someone actively working against unverified claims, and still ship an unlabeled one, not through carelessness but because two honest numbers sitting side by side look, by default, like they mean the same thing. Labeling them correctly cost one sentence. Noticing they needed it cost a paper.

This is the third entry in what remains an ongoing line of work, not a closed one. Section 6 already named the next real boundaries — whether the underlying checks are well-chosen, whether this discipline holds across many metrics rather than one reviewed closely. Those are not loose ends set aside for tidiness. They are, as with the two papers before this one, the actual agenda for what gets built and written about next.

Availability

This publication’s first paper pointed to a public repository as evidence a reader could inspect directly. Its second did not, because the system that incident occurred in was not open for that kind of inspection — and said so plainly rather than implying otherwise. This paper holds the same boundary the second one did, for the same system, for the same reason. What is available in full, above, is the real before-and-after text of the fix, the exact reasoning for why a confidence interval would have been the wrong fix, and the general shape of the discipline — enough to independently evaluate and build from, without requiring access to the specific system this instance was first found in.