Your AI Agent Is Online. Did It Actually Finish the Job?

The assistant says the support case has been updated. Its response arrived quickly, the software interface accepted the request and the technical dashboard looks healthy. Then someone opens the case and finds the old status still there.

A person now has to repair the record before the next team can act. In this hypothetical incident, the software responded successfully to several individual steps, but the work the user requested is unfinished.

For an agent that acts through tools, reliability needs to reach that final state. The customer does not experience a model call and an API response as separate achievements. They experience a task that either gets done or returns to them.

Decide what completion means for this task

An API is the interface software uses to communicate with another system. It may report that it accepted an update without establishing that the intended record has reached its final state. The assistant's statement that the work is done isn't an independent check either.

For our case, completion could mean that the correct record has the requested status, that any required notification was sent and that the result can be confirmed in the system responsible for the case. The completion test follows the result this particular task is meant to deliver. [1][2]

The distinction can be captured in ordinary language: “The next team can now act on the updated case.” That is a much more useful promise than “The tool returned without an error.”

Count the requests that fail before the final step

Suppose, in a numerical illustration, a service accepts 100 eligible updates. Ten fail before reaching the update tool; of the remaining 90, all are confirmed complete. Reporting only the tool's results gives 90 out of 90. Reporting the accepted tasks gives 90 out of 100.

The second count describes more of what users experienced. The first may still help diagnose the tool stage, but it cannot stand in for the whole service. Test requests, cancellations and other exclusions need consistent treatment so a percentage doesn't improve merely because inconvenient cases left the count.

This is the purpose of a service-level objective, or SLO: a reliability target for a defined kind of work over a defined period. The promise and counted requests give the target meaning. [3]

A person finishing the job changes the economics

In the unchanged support case, model latency measures how quickly the response arrived. It doesn't include the employee's investigation or repair.

Following a few complete tasks can reveal where that time reappears. A person might supply missing context, reconcile systems that disagree or correct a status the agent announced but never established. Counting interventions tells you that humans are involved; connecting their time to the failed step helps explain why.

I would keep the accepted request, attempted update and confirmed result visible together. That makes the next improvement specific. If the update never completed, making the model answer faster is unlikely to remove the employee's repair work.

Cost per verified completion should cover the same task, including material retries and correction effort. Otherwise one part of the system can look cheaper by making another part do more work.

Not every stopped task is the same failure

A dependency outage that prevents a legitimate update is different from a permission check blocking an unauthorized account change. Both interrupt the agent; only the first necessarily represents failure to perform an allowed task. Treating every interruption as equally bad can encourage the team to remove the control that worked.

Serious failures also need attention outside an average. A duplicate payment or sensitive-data disclosure should not become acceptable merely because the service produced many harmless successes. Ordinary reliability targets and consequences the organization will not accept have different purposes. [1]

An error budget describes the ordinary unreliability allowed under an SLO during a period; it is not permission to ignore those separate limits. When the service changes its supported tasks or authority, the definitions need to change visibly with it. [3]

The support incident then has a straightforward account: the endpoint was available, the update was not verified and an employee completed the work at additional cost. That explanation points toward a repair. A green technical response, by itself, would have left the team congratulating a service that handed its unfinished task back to the user.