Firms can rule each AI agent, not yet the whole fleet

The first report asked who owns an AI agent's rules. Its companion, What controls do AI agent products give firms, and how are they changing?, reads what agent products let firms control and how that has changed since 2021. It reads what vendors publish about their controls, as any buyer could, and dates when each kind of control first appeared.

From muzzles to handcuffs. The first AI controls were filters on speech: block a topic, mask a phone number, catch a jailbreak. Within about a year, new controls were mostly written for actions instead: which tools an agent may use, whose identity it carries, what it may spend, when a person must approve. Agents reached central policy over actions in about three years, faster than cloud or APIs did, likely in part by reusing the identity, gateway and policy machinery those systems had built.

But each handcuff fits one wrist. Plenty of rules are set once for the whole firm. When they are checked, nearly two times in three they still look at one action, one agent or one user. Only a small share of controls look across the whole firm, and most of those keep records or meter the vendor's bill. The analysis found just one that limits what agents do in total.

Why that matters. Picture ten thousand employees, each with well-governed agents. Every agent passes every check. Added up, small refund limits become a refund policy nobody wrote, and modest email caps become a mass mailing nobody approved. Trading learned this the hard way. In 2013 the SEC found that Knight Capital lacked controls able to stop orders beyond its capital limits "for the firm in the aggregate", and Europe now requires trading firms to hold firm-wide limits and a kill switch for all their orders. Agents have no equivalent yet.

The edge of the fleet. Every control reaches only as far as the firm owns: its devices, network and accounts. On the analysis's reading of the evidence, personal AI accounts and personal agents sit outside nearly all of them.

The report closes with twelve questions to ask vendors before scaling to thousands of agents. In short, the analysis suggests asking for controls over the fleet, not only over each agent. Both reports are on the Research page.

These are findings of an AI-assisted analysis of public sources. Its suggestions are not a consensus view of the research group or a position of the Protocol Institute.