Where we drew the automation line

We started with "can the agent do this". Wrong question.

The agent can do most things. On a system seeing between 21,000 and 27,000 incidents a month, what needed answering was different: what happens when it gets it wrong. And the answer varies so much by incident type that one threshold was never going to work.

We started with the confidence score and it went badly

The first setup did what everyone does. Put a cutoff on the confidence value the model produces, let anything above it run.

The trouble is the score is not calibrated. When the model says 0.9 it is not right nine times out of ten. It is a number the model produces about itself, with nothing tying it to the world. Hang your threshold on it and you have hung it on an unmeasured guess.

Did we throw the score out? No. We stopped letting it decide alone. What replaced it is duller and sturdier: can the action be undone.

Three buckets

We sorted incidents not by difficulty but by what is lost when the action is wrong.

The first bucket holds reversible, narrow work. Collecting logs, reporting disk usage, checking service state. If it runs wrongly the cost is running it again. No threshold here - the agent decides and nobody approves.

The second holds things reversible but wide. Restarting a service on one machine lives here. On the wrong host there is an impact, but it can be undone. The threshold applies, and the target gets cross-checked against inventory.

The third holds the irreversible and the very wide. Configuration changes, anything spanning multiple machines, anything near user data. No threshold because there is no automation. The agent prepares the proposal; a person approves the execution.

Once that split existed, "how good is the agent" stopped being interesting. A better model does not automate the third bucket, because what draws the line is consequence, not capability.

Blast radius beats confidence

Over time we noticed how many machines an action touches is a more reliable signal than how sure the model is.

A wrong restart on one machine affects one user. The same restart on a device group affects hundreds. The model's confidence can be identical in both. The outcome is not.

So a second rule went in beside the threshold: as scope grows, approval becomes mandatory. The agent does the same action alone on one machine, on five, but not on fifty.

Class isn't enough, context is

At the start we keyed thresholds to incident class - "disk alerts at this level, service alerts at that one". It turned out to be too coarse.

Two incidents in the same class can be very different things. A disk alert on a test server and a disk alert on a database server share a class and share nothing else. Thresholds had to key off class and target together.

That grew the threshold table and made it harder to manage. In exchange, wrong automatic actions dropped. Whether it was a good trade we still argue about - the table is no longer something one person can hold in their head.

37% was never a target

The system currently closes about 37% of incidents without human involvement. Nobody set that as a goal; it fell out of how the buckets are distributed.

There are two ways to raise it and one is a trap. The trap is loosening thresholds: the same incidents start flowing through, the rate climbs, the error count climbs with it. The other is growing the first bucket - making more incident types reversible.

Concretely: "restart the service" is irreversible as one step. A flow that captures state first and restarts afterwards is reversible. You changed the bucket without touching the threshold. Tedious work, and the most productive thing we did.

What happens when the agent stops

There was a question we ignored for too long: what the system does when the agent is not working.

When the model service is unreachable, or validation keeps failing, incidents pile up. In the first version this happened silently - the queue grew and nobody knew.

Now automation going offline raises an explicit alarm and incidents drop straight into the human queue. This is the least discussed part of agent systems: what happens when they stop tends to get less thought than what happens when they run.

Thresholds don't get set once

Everything underneath keeps moving - model version, inventory, incident mix - so the threshold cannot sit still.

We keep three numbers per incident type and every argument runs through them: recurrence rate on auto-closed incidents, approval rate on escalated ones, validation failure rate.

The second was more useful than expected. If people keep approving the agent's proposal unchanged for a type, that type is a candidate for the first bucket. It works in reverse too: a low approval rate means the problem is the step, not the threshold.

---

The automation threshold looks like a question about model quality and it isn't. Thresholds do not rise on their own as models improve.

The order that worked for us: make the action reversible, then narrow its scope, and look at the confidence score last.