Measure your AI agent by the work you can accept

In this guide

1. The useful starting point
2. Describe the change you want
3. Establish the baseline
4. Calculate cost per accepted result
5. Track the failure pattern
6. Account for work between conversations
7. Keep accountability with a person
8. Put this into practice
9. Sources and further reading

The useful starting point

Define an accepted result before running the agent. Track the share of attempts that meet that standard and the total cost of producing them, including human review and rework. Keep operational efficiency separate from evidence of customer value.

A dashboard can show an agent completing dozens of tasks while the team still struggles to use the output. The missing measure is acceptance. Was the brief accurate enough to inform a decision? Did the record update reach the correct account? Could a customer complete the promised job? Decide what counts before increasing the agent’s workload.

Describe the change you want

Separate the operational output from the customer outcome. A meeting brief is an output. A better-informed client conversation is a possible outcome. A deployed onboarding change is an output. A customer completing the first useful task is an outcome you still need to observe. The connection between them is a hypothesis to test.

For the first workflow, choose a narrow deliverable with a reviewable standard. A client brief might require current account details, links to source records, unresolved questions, and a proposed agenda. Define unacceptable defects as well: fabricated quotations, information from another client, or an unsupported claim presented as fact.

Establish the baseline

Observe a small set of comparable manual tasks. Record elapsed time, active working time, review effort, and common corrections. Use the same scope when you test the agent. Comparing a quick machine draft with a fully approved manual deliverable creates a misleading result.

Keep a note of what changed between runs. Different input quality, a new reviewer, or an easier case can explain an apparent improvement. Small pilots provide practical learning, but they rarely justify broad claims about productivity across the whole organization. Treat unusual cases as evidence to investigate.

Calculate cost per accepted result

For a defined period, add allocated subscription or model charges, tool and hosting costs, human review, and rework. Divide by the number of results that meet the acceptance standard. Count failed attempts in the cost numerator. If no result is accepted, report no accepted results and the cost incurred; do not produce a reassuring unit-cost figure.

Here is an illustrative calculation, not a benchmark: ten attempts cost $12 in model and tool usage. Review and rework consume 90 minutes valued at $40 per hour, adding $60. Eight results pass. The measured cost is $72 divided by eight, or $9 per accepted result, before any other allocated overhead. Record the labor assumption beside the number.

Track the failure pattern

A simple log needs the task type, input version, model or product, elapsed time, cost, acceptance decision, defect category, and correction. Distinguish missing sources, wrong interpretation, failed tool execution, duplicate action, and poor presentation. Each suggests a different intervention.

Improve the source pack when the evidence is missing. Narrow the instructions when the scope drifts. Add deterministic validation when a field follows a fixed rule. Improve the connector or retry behavior when execution fails. A more expensive model cannot repair every kind of process defect.

Account for work between conversations

Always-on products can spend usage on monitoring and follow-up as well as visible deliverables. Anthropic’s Claude Duet announcement explains that background checks share plan limits with other Claude use. Genspark’s Super Agent guide notes that multi-step work can consume more credits through additional execution and tool calls.

Choose a monitoring cadence that serves a real decision. A weekly review may be sufficient for a slowly changing source; checking every few minutes can create cost without useful new information. Set a maximum number of attempts and an escalation rule. Record both planned and unexpected work.

Keep accountability with a person

Assign an owner who can decide whether the result is usable and whether the workflow should continue. Ask the agent for evidence and a record of completed actions. Do not equate a confident completion message with a verified state in the destination system. Sample the actual records or deliverables.

At the end of the pilot, choose one of three actions: continue the bounded workflow, revise it to address a named failure, or stop because the benefit does not justify the cost. Scale only after you understand what produces accepted work and what that work changes for the customer.

Put this into practice

Choose one recurring deliverable and write the acceptance standard before evaluating another tool. Start with the free Agent to Outcome tutorial, then use the workflow planner to map your goal, context, tools, human checkpoints, and success criteria. Explore the Agent to Outcome workshop and kit for a guided learning path. For implementation or team training, see Waymaker Agency’s agent workflow service.

Sources and further reading

• Anthropic: Claude Duet and background usage
• Genspark: Super Agent and credit use

Sources checked October 22, 2026. Review the linked publisher information for current details.

Put one idea to work

Define the customer outcome, smallest useful deliverable, evidence, and review decision before expanding the work.

Plan an expertise pilot →Explore more from Outcome DrivenExplore the Outcome Club signup →