Designing AI Workflows That Require Human Approval

Human approval is useful only when it changes what the system is allowed to do. A workflow is not meaningfully human-in-the-loop if a person sees a notification after an irreversible action, clicks approve without the evidence needed to judge it, or is expected to review hundreds of low-signal outputs under time pressure.

NIST's AI Risk Management Framework calls for defined human roles, documented oversight, deployment decisions, and controls matched to risk tolerance. OWASP's guidance on excessive agency reaches the same operational conclusion from a security perspective: high-impact actions should not be delegated to a model merely because the model can call a tool.

This guide provides an eight-part design method for separating preparation, recommendation, approval, and execution. It is intended for ordinary business workflows such as publishing, sending messages, changing records, purchasing, granting access, or operating connected software. It does not replace domain-specific legal, safety, security, or compliance review.

1. Map the workflow as decisions and state changes

Start with the current process, not the model. List each step from input to final outcome and mark whether it only reads information, produces a draft, recommends a decision, changes stored state, communicates externally, or creates an obligation.

A useful map records:

• the actor responsible for each step
• the data read and written
• the system of record
• the decision being made
• whether the action is reversible
• the maximum plausible harm from an error
• the evidence available to a reviewer
• the deadline for approval

Do not describe the workflow as one broad action such as “handle customer requests.” Separate classification, retrieval, drafting, approval, sending, logging, and escalation. Approval boundaries become visible only after the workflow is decomposed.

A model may safely prepare a draft while the next step remains blocked. The same model may be unsuitable for sending that draft because external communication creates reputational, contractual, privacy, or security consequences.

2. Classify actions by impact and reversibility

Approval effort should match the consequence of error. A low-impact, easily reversible change does not require the same gate as a payment, deletion, permission change, public statement, or legal commitment.

Use four practical classes.

Reversibility must be real. A “delete” action is not safely reversible because a backup might exist somewhere. Confirm the restore procedure, retention window, owner, and recovery time. A public post can be removed, but copies and screenshots may remain.

When impact is uncertain, use the stricter class until evidence supports reducing the control.

3. Give the model the minimum capability needed

The approval gate is weaker if the model already holds broader permissions than the approved action requires. Limit functionality, credentials, accessible records, network destinations, and execution scope before adding a confirmation screen.

For example:

• a summarization workflow should receive read-only access
• an email drafting tool should not also have permission to send
• a publishing workflow should write to a review queue, not directly to production
• a product recommendation agent should not be able to place an order
• a support assistant should not change account permissions while answering questions

Use narrow tools with specific parameters instead of open-ended shell, browser, database, or file-system access. Enforce authorization in the downstream system rather than asking the model whether an action is allowed.

OWASP describes excessive functionality, excessive permissions, and excessive autonomy as separate causes of excessive agency. Removing any one of them reduces risk; removing all three creates a more defensible control boundary.

4. Place approval before the first consequential action

The right approval point is the last moment at which a person can still prevent harm without reconstructing the workflow.

For a public article, approval should occur before publication, not after a deployment notification. For an email, approval should occur before send. For a database operation, approval should occur before the write transaction. For a purchase, approval should occur before the order creates a charge or contractual obligation.

Avoid approval after every internal step. Excessive prompts train people to click without thinking. Group low-risk preparation into one review package and reserve explicit confirmation for the consequential transition.

The system should stop at the gate. It should not continue automatically because the approver did not respond, because a timeout expired, or because an earlier approval existed for a different payload.

5. Present the evidence needed for a real decision

An approval request should explain exactly what will happen. Showing only the model's final sentence is insufficient.

A useful approval package includes:

• the proposed action in plain language
• the exact target and scope
• the source material used
• material assumptions and uncertainties
• changes from the previous approved state
• policy or validation results
• expected side effects
• rollback or recovery path
• the identity and authority of the approver
• an expiry time after which the request must be regenerated

For content publication, show the title, route, language pair, sources, disclosure, scheduled time, and diff. For access changes, show the subject, requested role, resources affected, duration, and current permissions. For a payment, show supplier, amount, currency, invoice, budget owner, and duplicate-check result.

The reviewer should not need to open many unrelated systems to reconstruct the decision. At the same time, the package should link to original evidence rather than replacing it with another AI summary.

6. Separate proposer, validator, approver, and executor

One model response should not propose an action, declare it valid, approve it, and execute it. Separate these responsibilities even when one person fills more than one role in a small organization.

A robust pattern is:

1. Proposer: AI or human prepares a candidate action.
2. Validator: deterministic checks confirm schema, policy, scope, and prerequisites.
3. Reviewer: a person evaluates judgment, evidence, and exceptions.
4. Approver: a named authorized person accepts the exact payload.
5. Executor: a narrow service performs only the approved action.
6. Recorder: an immutable or durable log records the result.

The validator should reject missing fields, unsafe paths, stale versions, unexpected file counts, duplicate requests, and policy violations before a person spends time reviewing. Human judgment should focus on meaning and consequence, not tasks software can check exactly.

For high-impact work, independent review may be necessary. NIST recommends involving internal experts who were not front-line developers or independent assessors in regular evaluations according to risk tolerance.

7. Bind approval to an exact, short-lived payload

An approval must not authorize a different action created later. Bind it to a stable identifier or cryptographic digest of the proposed payload, the target, the approver, and the expiry time.

After approval, reject execution when:

• the content or parameters changed
• the target changed
• the underlying record version changed
• required evidence was updated
• the approval expired
• the approver no longer has authority
• the action was already executed

Use idempotency so retries do not repeat a payment, send, publish, or destructive change. Record the expected version before modification and fail closed if the current state differs.

For Git-based publication, an expected head SHA is a practical example: the merge succeeds only if the reviewed commit is still the PR head. The approval is attached to that exact content rather than to a branch name that can move.

8. Design rejection, escalation, rollback, and monitoring

Approval is not only a yes button. The workflow needs explicit outcomes for rejection, requested changes, uncertainty, timeout, system failure, and suspected compromise.

Define:

• who receives a rejected item
• whether the model may revise it automatically
• how many retries are allowed
• when a domain expert is required
• when the workflow must stop permanently
• how emergency disablement works
• what can be rolled back and by whom
• which events trigger incident review

Monitor approval rates, rejection reasons, override frequency, time spent reviewing, errors found after approval, duplicate executions, rollback success, and attempts to bypass the gate. A 99 percent approval rate may indicate excellent preparation, but it may also indicate rubber-stamping.

Prompt injection and misleading source content must remain part of the threat model. The approval interface should distinguish trusted system instructions from untrusted retrieved text and should never treat a document's embedded instruction as authorization.

Approval placement matrix

Signs that the approval gate is ineffective

• The model can perform the action through another tool without approval.
• The reviewer sees only a summary instead of the exact payload and evidence.
• Approval remains valid after content, target, or state changes.
• Timeouts silently become approval.
• One confirmation authorizes an unlimited series of future actions.
• The reviewer is expected to approve more items than can be examined carefully.
• Rejection does not stop the executor.
• Logs cannot show who approved what and what actually happened.
• The workflow cannot be disabled quickly during an incident.

Short design checklist

• Has the workflow been decomposed into decisions and state changes?
• Is each action classified by impact and reversibility?
• Does the model have only the functions and permissions it needs?
• Is approval placed before the first consequential action?
• Does the reviewer receive exact evidence, scope, and side effects?
• Are proposal, validation, approval, execution, and recording separated?
• Is approval bound to an exact payload, version, approver, and expiry?
• Are rejection, escalation, rollback, monitoring, and emergency stop defined?

Use this design with When AI Automation Becomes Inefficient to decide whether automation should exist at all, and with AI Tool Privacy and Security Checklist Before Adoption to review the data and permission boundary before implementation.

Sources reviewed

• NIST AI RMF Core (opens in a new window)
• NIST AI RMF Appendix C: Human-AI Interaction (opens in a new window)
• NIST Generative AI Profile (opens in a new window)
• OWASP LLM06:2025 Excessive Agency (opens in a new window)
• OWASP LLM01:2025 Prompt Injection (opens in a new window)

Sources checked: 2026-07-20