Last week, five identical-looking database errors hit one of our customer's Sentry within 30 seconds. Our agent analyzed all five in parallel. Four of them came back with the same lazy conclusion: "transient database issue, self-resolved, no action needed."
The fifth one actually investigated. Same model, same tools, same codebase. It found the root cause: a serverless Postgres cold start after 46 minutes of inactivity, combined with a single-connection pool that amplified the failure across all tables.
Same model. Same prompt. Same error. One agent investigated, four pattern-matched. That inconsistency kept me up at night.
The laziness problem
If you're building agents that do non-trivial reasoning (root cause analysis, code review, incident investigation) you've probably seen this. The agent sees a burst of errors that stopped on their own, and it takes a shortcut: "transient issue, likely resolved." It sounds confident. It even sounds plausible. But it didn't actually check.
I dug into the thinking traces of all five runs. The lazy ones had 200-300 characters of reasoning. The good one had 9,000+. But here's the uncomfortable part: one of the lazy runs had 8,000 characters of thinking too, and still got it wrong. It reasoned deeply in the wrong direction.
So it's not just about thinking harder. The model needs to think differently.
What I tried first (and why it failed)
My first instinct was to crank up the thinking budget. Claude's API has an effort parameter that controls how much the model thinks before responding. I ran an A/B test: same error, same everything, but one run with effort:"high" and one with effort:"max".
The results were nearly identical. 1,418 characters of thinking vs 1,674. The model doesn't use more thinking just because you allow it. It's like giving someone a bigger whiteboard. They won't fill it unless they have a reason to.
Then I tried adding rules to the prompt: "Never conclude transient without checking deploys." "Always verify with at least two data sources." This kind of works, but it's whack-a-mole. You patch one shortcut, the model finds another. And the rules are brittle, specific to patterns you've already seen fail.
I needed something more general. Something that changes how the model approaches every analysis, not just the ones I've already caught failing.
The trick: make it prove its work
The fix turned out to be embarrassingly simple. I added a mandatory block to the agent's output format:
<verification>
<hypotheses_tested>
<item>
<hypothesis>What you tested</hypothesis>
<result>What the evidence showed</result>
<verdict>confirmed|eliminated|inconclusive</verdict>
</item>
<!-- At least 2 items required -->
</hypotheses_tested>
<hypotheses_not_tested>
<item>
<hypothesis>What you couldn't test</hypothesis>
<reason>Why</reason>
</item>
</hypotheses_not_tested>
</verification>
That's it. The agent must list every hypothesis it tested, what evidence it found, and whether it was confirmed or eliminated. It must also list hypotheses it couldn't test and explain why.
The key constraint: at least two competing hypotheses, tested with actual tool calls. Not reasoning. Not "I believe." Actual grep, git log, API queries.
Why this works (the autoregressive insight)
This isn't just a formatting trick. It exploits something fundamental about how language models work.
LLMs are autoregressive. They predict the next token based on everything that came before, including what they've already committed to outputting. When the model knows its output format includes a verification block that requires evidence for multiple hypotheses, it changes how it behaves during the entire investigation, not just at the end.
Think of it this way: if you tell a student "write your answer," they might guess. If you tell them "write your answer, then show which alternatives you ruled out and how," they'll actually do the work, because they know they'll need to fill in those fields when they get there.
The model is predicting its own future output. It knows the verification block is coming. So during the tool-calling phase, it actually runs the queries it'll need to cite later. It's not conscientiousness. It's next-token prediction doing exactly what it's designed to do.
This technique has a name in the research literature: Chain-of-Verification (CoVe), from a 2023 paper by Meta. The original paper focused on reducing hallucinations in factual questions, but the core mechanism (forcing the model to output verification steps) transfers directly to agent tasks.
The before and after
I re-ran the exact same error that four agents got wrong. Same model (Claude Opus 4.6), same tools, same codebase. The only difference: the new output format with the verification block.
Before (without verification):
- 223 characters of thinking
- 3 tool calls (read the error, glance at code, done)
- Conclusion: "Transient database unavailability, self-resolved"
- No deploys checked, no traffic patterns analyzed, no alternative hypotheses
After (with verification):
- 117K characters of partial output (full investigation trace)
- 3 hypotheses tested with tool calls: code defect, database outage, missing migrations
- Each hypothesis explicitly confirmed or eliminated with evidence
- Found the actual root cause: serverless Postgres cold start + single-connection pool
- Listed what it couldn't verify: the hosting provider's internal status page
Here's the actual verification block the agent produced:
<verification>
<hypotheses_tested>
<item>
<hypothesis>Code defect: the table name is wrong in the ORM schema</hypothesis>
<result>Schema defines the correct table. Same query pattern used in 15+
other files and works normally. A wrong table name would fail permanently
in all environments, not just staging for 2 minutes.</result>
<verdict>eliminated</verdict>
</item>
<item>
<hypothesis>Transient database outage: serverless compute was temporarily
unavailable, causing all table queries to fail simultaneously</hypothesis>
<result>Logs confirm 74 errors across 5 different tables during a 2-minute
window. Zero errors before or after. The simultaneous failure of all
tables and self-resolution is consistent with a compute cold start.</result>
<verdict>confirmed</verdict>
</item>
<item>
<hypothesis>Database migration not applied: staging DB recreated without
running migrations</hypothesis>
<result>If migrations were missing, errors would be persistent.
The users table query succeeded before the auth query failed,
inconsistent with permanently missing tables.</result>
<verdict>eliminated</verdict>
</item>
</hypotheses_tested>
<hypotheses_not_tested>
<item>
<hypothesis>Internal failover or maintenance event at the time
of the errors</hypothesis>
<reason>Cannot access the database provider's internal status logs
or dashboard to verify.</reason>
</item>
</hypotheses_not_tested>
</verification>
The conclusion is actually the same: it was a transient database issue. But now it's a conclusion backed by evidence, not a guess. And crucially, the agent found something actionable: the single-connection pool configuration that amplified a brief outage into a full failure across all endpoints.
Implementation details
The implementation is three changes:
- Add the verification block to your output format. Put it at the end. The model needs to have completed its analysis before filling it in, which is exactly the point.
- Add methodology instructions that reference it. We added: "Your output includes a verification block. You MUST have the evidence to fill it BEFORE you start writing your conclusion. Minimum 2 competing hypotheses tested with actual tool calls."
- Parse it. We extract the verification data so we can track hypothesis counts and verdict distributions across runs. This gives you a quality signal you can monitor.
What we deliberately did NOT do: gate on confidence scores. Earlier, I considered adding a rule like "if confidence is below 60%, go back and investigate more." But confidence can legitimately be low when data is missing. Low confidence with honest gaps is more valuable than forced high confidence with hidden assumptions. The verification block solves the problem without introducing a brittle threshold.
The general principle
The broader lesson is this: if you want an agent to behave a certain way, don't add rules. Add structure to its output.
Rules are suggestions. "Always check deploys before concluding" is something the model might follow or might not, depending on how strongly the pattern-matching shortcut pulls. But an output field that requires deploy check results? That's a structural constraint the model can't skip, because it has to produce tokens to fill it.
This is a design pattern that generalizes beyond root cause analysis:
- Want your code review agent to consider edge cases? Add an
block to the output format. - Want your planning agent to consider risks? Add a
block with required mitigations. - Want your research agent to be thorough? Add an
block.
The pattern is always the same: make the desired behavior a required output field, and the autoregressive mechanism does the rest. The model will do the work because it needs the results to fill in the fields it knows are coming.
What this means for agent reliability
Agent reliability is the unsolved problem in production AI. Everyone has a demo that works 80% of the time. Getting to 95%+ is where the real engineering happens, and most of it isn't model selection or prompt tweaking. It's structural constraints that make the right behavior the path of least resistance.
At Sonarly, we analyze production errors and incidents for engineering teams. A wrong root cause analysis isn't just unhelpful, it's actively harmful, because it sends someone down the wrong debugging path at 3am. The verification block didn't make our agent smarter. It made it honest about its own investigation process, and that honesty translates directly into reliability.
If you're building agents that make decisions based on evidence, try this: add a verification block. Force competing hypotheses. Parse the results. You'll be surprised how much thoroughness you get just by asking for it in the right place.
Alexandre Klobb, Co-founder at Sonarly