Verification

AI is operating increasingly independently through workflows, operator mode, and agents. With that autonomy comes moments where the wrong decision can have negative consequences, ranging from lost data to unexpected costs to leaked personal information.

In any situation where the AI is taking action on the user's behalf, it may be appropriate to first have it receive verification from the human orchestrating it.

When to require verification

Verification isn't required for simple tasks with low risk. In these cases, the impact of a poor choice is negligible, while the negative impact of increased friction and wasted time is high. For example, running a search, subscribing to a newsletter, or drafting an email. Repeated actions after an initial verification may also fit in this category, such as an auto-fill action after the sample response is verified.

This equation changes when an errant action has real impact:

Loss of reputation from a poorly written email
Loss of money from an errant purchase
Loss of security from sharing personal or corporate data
Loss of work from overwritten records
Loss of time to clean up errors and misguided actions

In these circumstances, verifications should be required by default, though users may choose to skip them through settings or instructions.

Simple verification

A basic "go/no-go" decision pattern is common already in workflows and other repetitive actions where an erroneous prompt could inadvertently overwrite data or result in unexpectedly high compute costs. Examples include verifying an AI's action plan or a sample response before proceeding with the full action. These should be lightweight interactions, and can occur at the source or pushed to another tool. For example, many AI-powered workflow tools use connectors to send a verification to Slack, Email, SMS, or other common workspaces.

Proactive platform rules

OpenAI demonstrates how it found this balance for its operator mode, which browses the web and takes actions autonomously while the user observes through shared vision and monitoring the AI's stream of thought. The AI is explicitly instructed to halt action when payment data or other sensitive information might be shared. The company hints that they may relax this setting at a later time and grant the user permission to override this constraint.

ChatGPT’s operator mode has explicit limitations that require user intervention. When the AI reaches one of these steps, it summarizes its process thus far and prompts the user to take over. Once the user is done with the task, they can hand control back to the AI to continue.

User overrides

Over time, we will likely see user-led rules in settings panels or agent.md files that provide instructions about when to stop during unforeseen loops, similar to how workflow builders like Zapier and Relay allow these steps to be entered manually. This becomes even more complicated as agents work with each other, where a subagent may require approval from a more established or senior agent. Look to parallels in how teams of humans operate interdependently to explore what this could look like in your domain.

Workflow builders like Relay and Zapier include the option for human in the loop steps that prompt some sort of conclusive action by the user. The example from Relay on the left demonstrates the different types of actions supported. The actual confirmation can be sent via email or added as a task in another tool, as demonstrated via Zapier on the right.

Design considerations

Match friction to risk. Trigger verification based on the potential harm, scope, and reversibility of the action. High-impact or irreversible actions should require strong confirmation, while routine or easily undone tasks should use light checks or confirmation after the fact to avoid prompt fatigue.
Use opt-out settings for full control. Similar to dialogs that offer users to “never show this again” for onboarding or other prompts, requesting a verification for a first time experience while supporting a request to never verify again ensures the user is in full command of their experience.
Make clear if verification is skipped. Whether skipped through opt-out or proactively when sending a destructive prompt, provide clear affordances or help text alerting the user to the opt-out state and allow them to opt-back in at any time.
Alert the user when their action is needed. Don't stall processes for longer than they need to be because the user did not realize they needed to intervene. Use third-party tool connections and other triggers to allow users to respond rapidly if they choose.