← RESEARCH
Research22 October 2026

Autonomy is earned one action at a time

An agent that holds the keys to billing and the inbox should not get more freedom because people clicked Approve. Here is the rule we use instead: four numbers, three of them set by people, and the evidence only moves one.

An engine that works inside a company ends up holding credentials to billing, support, the CRM and Slack. It reads text written by strangers: tickets, lead forms, customer emails. Then it acts on customers and money. The question is not whether to let it act. It is how an action moves from "ask first" toward "act on its own" on evidence, while something that evidence cannot raise bounds the damage when the evidence was wrong.

Most products answer with a single dial per agent and a stream of approval prompts. Both parts of that answer have a problem.

Approval decays, and it measures the wrong thing

People stop reading what they approve. Anthropic reports that users of its coding agent accept 93 percent of permission prompts, and that the share of auto-approved actions rises from about 20 percent for new users to over 40 percent after a few hundred sessions. OpenAI's Operator fired a confirmation on 92 percent of the tasks that needed one, and still spent a reviewer's money on groceries he had not asked for. A second model as supervisor does not fix it either: in Anthropic's Project Vend, a CEO agent built on the same model authorized lenient requests about eight times as often as it declined them.

There is a deeper issue than fatigue. Approval statistics describe the agent's behavior on ordinary inputs. They say nothing about its behavior under attack, and an agent that reads third-party text is under attack whenever someone wants it to be. Prompt injection does not have a complete defense. The best published figures against an adaptive attacker are a few percent, and the researchers who report them say so plainly. Adaptive attacks have bypassed a dozen published defenses, most at over 90 percent success. We take that as a design constraint: assume the agent can be steered, and limit what a steered agent can do.

Four numbers, one of them learned

The autonomy of any single proposal is the minimum of four values. The level its action class has earned. The ceiling for its risk tier. The ceiling for its audience. And a provenance cap. People set the three ceilings in code. Evidence moves only the earned level.

An action class is something like "draft a win-back email", "retry a failed invoice", "offer a discount", "ground a drone". Each starts at ask first, and each has a tier set by a person: low, medium, high, critical. Critical never rises above ask first, however good the record: discounts, contracts, anything irreversible stay with a person. High stops at draft for approval. Medium may reach act and report if the action can be reversed within a stated window. Low may reach act. The audience adds a second ceiling: anything a customer sees stops at act and report, and only for templates with typed slots; anything public stops at draft.

The provenance cap is the one that matters under attack. If a recipient, an amount or a free-text field in the proposal derives from text a third party wrote, the proposal stays at draft for approval whatever the class has earned. This is the lethal trifecta, private data plus untrusted content plus the ability to act, applied per proposal rather than per system. The agent may still read the ticket and still draft the reply. It may not send it alone.

The gate never reads the attacker's text

Two structural choices make the cap enforceable. First, the component that reads untrusted text and the component that holds credentials are different components. A quarantined reader sees third-party text with no tools and returns typed fields only. The planner sees company-authored text, the typed fields and record handles, and emits a structured proposal. Free text from the reader can be shown to a person, labelled, and never re-enters the planner's context. Second, the policy gate is deterministic code. It computes the four values, checks budgets and a freeze flag, hashes the proposal and decides. It does not read the ticket, so there is nothing in the ticket that can address it. A separate executor holds per-class credentials the planner never sees, stages writes where the system allows it, drafts rather than sends, holds rather than refunds, and uses the proposal's id as an idempotency key, so a retry cannot act twice.

The approval message itself carries the exact action and recipient, the evidence with links to the records, the prediction and its window, the tier, whether the action can be undone and for how long, the exact payload with third-party text visibly marked, an expiry, and who may approve. Links and previews are off, because two of the better-known leaks in business tools went out through an unfurled link and an auto-fetched image.

When a class moves

A class moves up one level when it is structurally eligible and three conditions hold. At least sixty human-decided proposals with no serious incident, because zero incidents in sixty trials bounds the rate near five percent, and three hundred for anything a customer sees. A lower confidence bound on unedited approvals of 0.85 or better, because an edit is a stronger signal than a click. And enough measured outcomes, no worse than the manual baseline. A class moves down one level on any serious incident, automatically, and resets on a model change. People approve every move up in Slack; the engine proposes it with the numbers attached.

At a company of a hundred people this takes months per class, and we would rather it did. The classes that reach act first are the dull ones: rescheduling an inspection, retrying a card, opening a ticket, posting a summary. That is the point.

The control Knight Capital did not have

The last layer assumes everything above has failed. Every class carries limits per hour on actions, dollar exposure and distinct customers touched. A global freeze flag is read by the gate on every call. And a reconciler compares the effects the connected systems report, through webhooks and audit logs, with the authorizations in the ledger, and trips the freeze on any effect that has no authorization behind it. In 2012 a trading firm sent four million orders in 45 minutes because no control compared orders leaving its router with orders entered. An agent with write access to billing deserves that comparison from day one.

Every decision is written once to an append-only ledger by the gate and the executor, never by the planner: the proposal hash, the evidence, the prediction, the gate's decision and policy version, who approved, what they edited, the receipt from the system that carried out the action, and later the measured outcome. Recording the prediction before the effect is what makes "predicted against measured" something an auditor can check rather than a chart.

What this does not solve

Steering. The design bounds what a steered agent can do; it does not stop the steering, and we do not know of anyone who does. Attribution: measuring whether a win-back email worked needs a holdout, a delay and tolerance for noise, and without them "measured" is a label. Small samples: a company of a hundred people produces few events per class, so the record grows slowly, and pooling across companies raises its own questions. And the tiers themselves are human labels. The model is never allowed to lower one at runtime.

Notes

WRITING