Why a 99% accurate AI agent can still send hundreds of bad invoices
A single accuracy number hides how small mistakes stack up across a multi-step task. Two questions can help you spot the gap before it reaches your books.
What happened
Forbes published a column by Raju Malhotra, Chief Product & Technology Officer at Certinia, arguing that "99% accurate" is less reassuring than it sounds. He starts with a firm that runs 10,000 invoices a month through an AI agent (software that can take a series of steps on its own). At 99% accuracy, that still means 100 invoices a month with an error a person has to catch or explain.
Then the math gets worse. Malhotra writes that accuracy at each step multiplies across the whole task. An agent that is 99.9% accurate at every step of a 20-step invoicing process lands at about 98% on the finished invoice. That is roughly 200 flawed invoices a month, or more than 2,000 a year.
A second Forbes column, by Billy Biggs of KnowledgeLake, shows the same gap in a real agency. It notes that coverage of the Department of Veterans Affairs flagged 94% accuracy on individual issues but 83% on whole claims.
Why it matters
Most AI pitches lean on one friendly number. The question that matters is how often the whole job comes out right.
As Malhotra writes in Forbes, a person's invoice mistakes tend to be inconsistent and easy to spot once someone looks closely. An agent can repeat the same mistake a thousand times before anyone sees the pattern. He adds that the agent should check the same facts a trained employee would, such as contract terms and approved rates, before it acts.
Filip Popovic of ZoomInfo makes a related point in Forbes. Agents can act without the human sanity check that used to catch a slightly-off record, so one bad guess can ripple through later steps. He argues the bar for confidence should rise with the consequences of the action.
What it means for your business
- Accounting and bookkeeping firms: Ask any vendor for the accuracy of the finished invoice or reconciliation, not each step. Then ask what the agent looks at before it acts.
- Law firms and agencies: Billing depends on engagement terms and rate cards. If the agent can't see them, it is guessing, and the same wrong rate can repeat across every client.
- Trades and contractors: If an agent turns quotes and job notes into invoices, check that it works from the signed quote, not a rough summary.
- Clinics: Billing errors can quietly repeat for months. Decide in advance which actions need a person's approval, such as anything that changes what a patient owes.
One thing to try this week
Pull 25 recent invoices that an agent (or a vendor's demo) produced, and check each against the contract or quote behind it. Count how many are wrong anywhere, not just in the total. That whole-task error rate tells you more than any accuracy claim, and you can repeat the check monthly.