Evaluating and Shipping Agent Products
Evaluate outcomes, process, safety, and efficiency, then use release gates to make agent behavior controllable.
“Agent Basics for Product Managers” part eight
Previous: State Management in Agent Products · Series home · Next: From Meeting Notes to a Reviewable PRD
An agent demo usually proves that one prepared input can produce an impressive path. Production brings missing, contradictory, ambiguous, and hostile inputs, along with tool timeouts, permission changes, page refreshes, model upgrades, and provider failures.
Most of the work between demo and production happens outside the model.
Replace “looks good” with an evaluation target
Evaluate at least four layers:
- Outcome: Was the task completed correctly and usefully?
- Process: Were the tools, parameters, and steps appropriate?
- Safety: Did the agent expose data, exceed permission, or bypass approval?
- Efficiency: Are latency, cost, steps, and human intervention acceptable?
Looking only at the final answer misses systems that arrived at a correct result through a dangerous process.
A minimum evaluation matrix
| Dimension | Example metrics |
|---|---|
| Outcome | Completion, correctness, human acceptance |
| Tool process | Correct tool selection, parameter errors, wasted calls |
| Grounding | Source coverage, citation accuracy, unsupported claims |
| Safety | Permission violations, skipped approvals, data exposure |
| Efficiency | P50/P95 latency, cost, average steps |
| Reliability | Timeouts, recovery success, duplicate writes |
The task set must include normal inputs, missing information, conflicting sources, hostile instructions, tool failures, insufficient permissions, and interrupted runs.
The evaluation target is the system
Agent behavior depends on the model, instructions, context, skills, tool definitions, workflow, and business data. A change to any layer can regress a previously passing task. Important changes need versions and regression runs against the same real examples.
Eight production gates
Before release, prove that goals and refusal conditions are clear; sources are traceable; tools enforce schemas, permissions, and idempotency; long runs can pause and resume; risky actions provide a complete preview; a real regression set exists; every run can be diagnosed; and dependencies, revocation, and incident ownership are defined.
A safer rollout sequence
Begin with recommendations. Then generate drafts or proposals. Next, execute reversible actions after explicit approval. Reduce human confirmation only after production evidence shows a low-risk step is consistently reliable. High-risk actions may keep a human control plane permanently.
Product exercise
Write ten tests for one agent feature: two normal, two incomplete, two conflicting, two involving permission or safety, and two tool failures. Define what must happen and what must never happen for every test.
Productionizing an agent does not mean maximizing autonomy. It means giving every autonomous act boundaries, evidence, state, evaluation, and a recovery path.
State Management in Agent Products
Turn conversation into trustworthy product operations by separating experience, product data, orchestration, capabilities, and models.
Case Study: From Meeting Notes to a Reviewable PRD
Combine goals, runtime, skills, tools, state, approvals, and evaluation into a product blueprint from meeting notes to a review-ready PRD.