lesson depth
Mastery
not started · 0%

Human Approval Boundaries

How consequence and reversibility determine when human authorization is mandatory.

Freshness: current14 min readAgentic Systems and Protocols

Key Learning Outcomes

  • Classify actions by consequence and reversibility
  • Place approvals before consequential side effects
  • Design useful escalation context

Mental model

Approval is a gate over authority, not a generic confirmation dialog. Its strength should rise with consequence, irreversibility, uncertainty, affected scope, and the difficulty of detecting harm after execution.

Proposed action
Consequence classification
Prepared evidence
Human decision
Bound authorization token
Execute and audit
Conceptual teaching model synthesized from:Trustworthy Agents in PracticeModel Context Protocol Architecture and Tools SpecificationArtificial Intelligence Risk Management Framework

Theory

Classify actions before runtime: read-only, reversible write, consequential write, privileged administration, or prohibited. The application, not the model, maps that classification to an approval policy. Place the gate after arguments and affected resources are known but before side effects occur.

An approval request should identify the goal, exact operation, target, important parameters, expected effect, uncertainty, alternatives, and rollback path. The resulting authorization token must be short-lived and bound to the user, operation, arguments, target, and current state. Changing any material field invalidates approval.

Alternatives and trade-offs

Automatic execution is appropriate for low-consequence, reversible actions under strict limits. Batch approval reduces interruption but increases scope. Policy-based preauthorization works for repetitive operations with narrow parameters. A human checkpoint improves control only when the reviewer has time, competence, and real ability to reject.

Failure modes and misconceptions

Approval fatigue turns prompts into decoration. Asking before parameters are known is uninformed consent. Reusing a broad approval token enables confused-deputy behavior. Hiding model uncertainty or tool provenance prevents meaningful review. Human involvement cannot compensate for excessive underlying permissions.

Knowledge check

Reflect before revealing the guide

Which fields must an approval token bind so the executed operation cannot drift from what the reviewer saw?

Decision scenario

A database agent may generate a migration and run a read-only plan automatically. Applying the migration requires a reviewer to see the target database, statements, lock estimate, affected tables, backup state, and rollback command. Approval expires if the plan changes.

Learning outcomes

  • Explain Human Approval Boundaries as a system mechanism rather than a slogan.
  • Compare its alternatives, trade-offs, and production failure modes.
  • Apply the concept to a decision and identify evidence that would validate it.

Trade-offs

Using Human Approval Boundaries can improve capability or control, but it also introduces cost, latency, complexity, and failure modes that must be measured against an explicit objective.

Prerequisites & Related Concepts (2)

Private notes

0 words