In an LLM cascade, a cheap model answers, a verifier judges, and a router decides whether to accept or escalate. Today that router treats the verifier's verdict as ground truth. Confine instead measures how reliable each verdict mode actually is and hands the router a contract: a per-mode reliability vector, an interval around the verdict, and an explicit defer action when reliability is too low. The router consumes evaluation as a noisy signal it can reason about, rather than a fact it must obey.
Cascade routers assume the verifier is right. But verifiers disagree with each other and with human annotators, and the disagreement is not uniform: some verdict modes are reliable, others are close to coin flips. When the router cannot see that spread, two failures follow. It accepts wrong answers from low-reliability modes, and it escalates good answers it should have kept, paying for the strongest model when it did not need to.
Measured on the human-labeled subset of MAST, per-mode reliability (Gwet's AC1) spans 0.76 to 1.00 across 14 production failure modes, with four modes below 0.85. The contested modes are not the long tail: three of the four sit above 0.25 prevalence, and a single reviewer misses about 5 percent of confirmed failures overall but 17 to 20 percent on these highest-prevalence contested classes. The router is noisiest exactly where it spends its time.
Verdict-as-truth. The router has one bit where it needs a distribution. No defer path. With only accept or escalate, there is no honest "I don't know" that routes to a human or a safe fallback. Uncalibrated cost. Escalation policy is tuned to an average verifier, not to the reliability of the specific mode that produced this verdict.Confine replaces the single verdict with a small contract the router consumes at decision time. The contract carries three things: how reliable this verdict mode is, how wide the uncertainty is around the verdict, and what to do when reliability falls below a threshold.
# the router consumes a contract, not a verdict contract = confine.evaluate(answer, verifier_mode) contract.reliability # per-mode reliability vector contract.interval # [lo, hi] around the verdict contract.defer # True when reliability < threshold if contract.defer: route_to_human_or_fallback() elif contract.interval.lo >= accept_bar: accept() else: escalate()
Deployment cost is small. The reliability vector ships as a 600-byte record per calibration window from a rotating 50-to-200-cell audit set, the interval adds about 150 bytes per request (about 1 Gbps at one million requests per second), and the recommended commit threshold is 0.8. At measured contested prevalence the defer rate sits between 5 and 12 percent, about 80 to 200 ms of additional p95, with backpressure above an operator-set ceiling.
The reliability numbers are not assumed; they are measured. Confine studies annotator agreement on MAST and uses that disagreement structure to calibrate per-mode reliability. The result is a reliability vector tied to evidence about how often each verdict mode agrees with careful human judgment, which is what makes the contract trustworthy enough for the router to act on.
Noise floor. A calibrated selection loop places the routing layer's noise floor at a 3 to 5 percent true quality gap; at a 2 percent gap the worst-case workload picks wrong 26.8 percent of the time, and a reliability-aware fix recovers about two points (26.8 to 24.8), honest but modest. Prompts do not fix it. On a balanced 112-cell diagnostic, an uncertainty-aware judge rubric raises recall from 23 to 41 percent but raises false positives from 7 to 21 percent, and defers on under 1 percent of cells. Interval, not retry. Capture-recapture puts the three-annotator undercount near 1.0x: coverage is nearly complete and reproducibility is the gap, so the right primitive is an interval around the verdict, not another reviewer. Stable at pilot scale. Across all 19 leave-one-out folds the contested set is identical (Jaccard 1.00, Kendall tau 0.985), so the routing rule does not depend on which traces.Confine is under submission at HotNets 2026. Headline findings above; the full protocol, statistics, and caveats are in the paper.
Each tier of the cascade pays for the same unreliable verdict in its own unit. The paper grounds four bills:
Wrong selection. 26.8 percent worst-case wrong picks at a 2 percent true gap, exactly the regime where most agent improvements live. Cloud tier. Roughly 52 kgCO2 per million requests of wasted heavy-tier compute at a worst-case 26 percent wasted-escalation rate. Edge tier. 0.7 to 1.1 seconds of additional p95 tail under a 2 second SLO with a 4 second retry cost. Device tier. About 0.6 W of sustained draft waste at a Jetson-class power envelope with a 5 percent wrong-accept rate.Four readings of the same noisy gate, in four operator units.
Across the AgenticStack agenda, each project supplies one contract the agent control plane needs. Confine is the reliability contract: it is how a router consumes uncertainty, intervals, and a defer action instead of trusting a verdict. In the stack, Single-Call Fallacy sets the unit of accounting (the plan), Patient spends a reliability budget at the gateway, and Confine measures where to spend it: per-mode reliability on the signal the router consumes.
Every layer that consumes an AI-generated signal needs a contract on its reliability, not an assumption of it.