One agent's mistake is the next agent's input

An agent that is 95% accurate sounds good. Across five steps that same rate works out to 77% end to end. It is just multiplication, but intuition does not multiply - it looks at each step on its own and concludes that everything is fine.

The arithmetic is not the annoying part. When a step crashes, you find out. When it produces a wrong but well-formed output, you don't.

Everyone did their job and the answer is wrong

The classifier files the incident under the wrong category. The planner builds a flawless plan for that category. The executor follows the plan to the letter. All three did exactly what they were supposed to do.

Whoever opens the logs sees the third step, because the third step is what is visible. The mistake happened in the first one, three steps back, and is long finished.

The first time this happens you go looking for a bug in the executor. The second time you check the planner. Getting to the right place takes a while.

A summary is not data

The second problem is quieter. Information passed between steps erodes a little at every handoff.

The first agent reads a thirty-line error record and writes a three-sentence summary. The second agent gets those three sentences and never sees the original. Even when the summary is accurate, any detail the second agent turns out to need is gone for good - the source is no longer in reach.

The fix is as boring as it is effective: pass identity between steps, not summaries. The second agent receives an incident_id and pulls the fields it needs itself. Summaries are for making decisions, not for carrying data.

After that change, the degradation we used to see on longer chains dropped off noticeably.

Deciding where to cut

You cannot put a check on every step; the cost does not add up. We settled on three places.

Right after classification, because that decision is the expensive one - everything downstream leans on it. After the plan is produced but before it runs: do the steps map to known tools, is the scope within bounds. And immediately before anything irreversible, in case a mistake from the head of the chain made it that far.

The third one looks redundant on paper. In practice it earns its keep more than the other two.

Where retrying doesn't help

When a step fails, retrying is the easy reflex. It also lumps two different things together.

Timeouts, momentary access problems, malformed output - those are worth a retry. The model usually pulls itself together on the second pass.

A reasoning failure is not like that. If the model misread the incident, the same input produces the same misreading. All you have added is latency, and someone is waiting on the other end.

We tell them apart by which layer did the rejecting. Validation rejected it, retry. Business logic rejected it, cut the chain and send the incident to a person.

What we ended up measuring

We used to track per-step accuracy and it told us nothing useful. The numbers looked healthy while the system felt unreliable.

The number that actually helped turned out to be a different one: how many steps go by before an error is noticed. As that grows the system gets worse, even when the individual accuracy rates never move. A late-caught mistake does not waste one step, it wastes everything after it.

Reliability in a chain comes from where you cut it, not from how good the steps are.