Containment and safety are different properties

The deputy does not have to be malicious

In 1988 Norm Hardy described a compiler that ran with the authority to write its own billing file. A user passed the billing file's name as the output argument, and the compiler — behaving exactly as designed — destroyed the billing record.¹ The program was not compromised. It was plainly confused: it held authority for one purpose and was induced to spend it on another.

Broadly every property of that scenario now applies to an agent that reads a web page, a support ticket, a sensor label, or a PDF. The agent holds permissions you deliberately granted. Attacker-controlled text arrives through a channel you deliberately opened. The agent tends to spend your authority on the attacker's goal, and every individual action is authorized.

This is clearly why "the model goes rogue" is the wrong threat to design against. It is speculative, it invites an argument about machine intentions, and it is strictly weaker than the attack that already works.

The attacks are documented, not hypothetical

In 2024 researchers reported a 100% jailbreak success rate against three deployed robot systems — a self-driving LLM simulator, a Clearpath Jackal ground vehicle running GPT-4o, and a Unitree Go2 quadruped — including black-box attacks where only the public API was available.² Separately, a Bluetooth provisioning flaw yielded root access across Unitree's Go2, B2, G1 and H1 platforms.³

What has not been documented, and we will not claim, is an injury caused by a language model driving a robot. The demonstrated gap is attackability. That is probably enough to act on, and overstating it would likely be the fastest way to lose the audience that matters.

Harm inside a perfectly enforced envelope

Assume the sandbox never fails, and that the capability table is exactly right. The following are all authorized, and all plausibly harmful.

Resonance inside every limit

A setpoint stream whose instantaneous velocity never exceeds the configured limit, and whose per-tick delta never exceeds the rate limit, can still excite a drivetrain resonance. Mechanical Q typically does the rest. Per-axis kinematic limits are instantaneous predicates; they do not bound energy accumulated over time.

Cartesian composition

Every joint can sit inside its own position and velocity envelope while the tool tip traverses the space a person occupies. Speed-and-separation limits are defined in Cartesian space; joint-space limits are not a sound approximation of them. ISO/TS 15066 says so in its own normative note: a speed limit that monitors only the tool centre point "does not monitor other parts of the robot that might pose hazards to the operator."⁷

Field confusion in a signing gate

A gate that signs bytes and enforces "amount ≤ cap" is checking a field it believes is the amount. An ERC-20 approve call carries a value of zero while granting unlimited future spend. A partially-signed Bitcoin transaction can route change to an address the policy never inspected. The cap is satisfied; the account may be drained.

The lesson we took from this is architectural: the trusted side must construct the effect, not inspect bytes handed to it.

What the theory says is possible

This is the part most safety pitches skip, and it is the part that determines what can honestly be promised.

Schneider established that a policy enforceable by monitoring a single execution must be a safety property on traces, meaning informally that its violation is witnessed at a specific moment.⁸ Clarkson and Schneider later showed that noninterference, the property that secret inputs do not influence public outputs, holds of no single execution at all. It is a hyperproperty, a predicate over sets of executions.⁹

Read naively, that says a runtime monitor can never enforce information flow, and a great deal of security marketing quietly depends on readers not knowing this.

The practical consequence: strict limits, rate ceilings, envelope containment and effect confinement are enforceable at runtime and are what the product delivers. Claims about what a bounded agent cannot infer or leak require far more care, and we make none of them today.