Endstop

Rev 0.2.1 · in integration

Your AI agent doesn't have to escape the sandbox to hurt someone.

It only has to be tricked while holding permissions you correctly gave it. Endstop is a small piece of separate hardware that holds the authority instead: the agent proposes, and only the gate acts.

From 14 January 2027, EU law requires it. Machinery with self-evolving behaviour “shall not… perform actions beyond its defined task and movement space.”7

Become a design partner → What you get →

§1

What it is

A small module wired between the machine running the model and the things that matter — and the actuator lines, or the signing key, terminate at it.

The agent's computer talks to it over one serial line and nothing else. On the other side are the drives, the contactor and the secure element. There is no route from the model to an effect that does not pass through the gate, and the gate has no operating system, no allocator and no network stack to be attacked through.

Every control cycle, it does three things. It runs the agent's program in a bounded interpreter, where the only effects that exist are three entries in a fixed table — read state, propose a setpoint, request a signature — so anything else is not refused so much as unrepresentable. It checks each proposed setpoint against an envelope that was validated offline, before the model was involved. And it writes what happened, including what it refused, to a signed record.

If the host stops talking, a deadman expires and the machine goes to a safe stop. That path is hardware and does not depend on the model, the host, or our software being correct.

UNTRUSTED · OFF-BOARD Agent LLM planner any hardware any speed UART ENDSTOP · TRUSTED · ~550 LOC Verified interpreter fuel-bounded · terminates by construction Capability table read · propose · sign Envelope monitor every cycle · < 200 µs effects AUTHORITY Signing key SPI element Servo drives PWM · step/dir Contactor drive power in series position Signed audit record every command · every refusal E-STOP opens the contactor directly — the gate is not in this chain
Figure 1. The effect path. Authority terminates at the gate, not at the machine running the model; the envelope monitor checks each proposal against measured position. The emergency-stop chain bypasses the gate and opens the contactor directly.
Design targets
ParameterDesign target
Trusted computing base~550 lines
Interpreter coremachine-checked, published5
Effect vocabulary3 capabilities
Enforcementevery control cycle
Added latency< 200 µs
Guard-distance cost≈ 0.32 mm6
Terminationfuel-bounded, by construction
Audit recordEd25519-signed, replayable
Emergency stopwired around the gate

That last line matters more than the rest of the table. It means the gate can only ever permit motion, and certainly never command it. How it works →

§2

What a partner actually gets

Three properties, and they are independent — you can want one of them and not the others.

  • An envelope that holds

    Limits you validated, enforced on every command, by hardware the model cannot reach or reconfigure. Position, speed, predicted stopping position and rate ceilings, checked at multiple points on the arm rather than only at the tool.

  • A record that survives

    Every command proposed and every limit that refused it, signed and replayable, produced off the host so it outlives whatever failed. See §3.

  • A stop that does not depend on software

    Deadman expiry and a relay in series with your existing safety chain. The gate can permit motion; it can never command it, and the emergency-stop path runs around it entirely.

In hardware terms it is small and deliberately dull: an FPGA carrying a soft 32-bit processor with no cache and no speculation, a serial port in, PWM or step/direction out, a relay contact for the safety chain, and an SPI secure element holding the signing key. Integration is a wiring change on the command path, not a rewrite of your stack.

What a pilot involves, and what you have to bring →  ·  Full system definition →

§3

Signed evidence of what the model actually did

The part you can want without believing anything we say about safety.

Every command the agent issued, and every limit that refused it, is written to a tamper-evident, replayable record produced by hardware the agent cannot reach or alter.

This is what you need the day after an incident, and it is the gap nobody fills. Every other log in the stack was written by the machine that failed, or lost with it. Your controller records what it was told — not what was proposed and stopped. Nothing else records the counterfactual, and the counterfactual is the whole question in a review: did the model try something the safety layer refused?

Aviation has flight recorders. Trucking has tachographs. Payments have hardware audit logs. Autonomous machines driven by language models have nothing.

§4

Why this becomes urgent in January 2027

Machinery with “fully or partially self-evolving behaviour or logic” shall not cause the machinery or related product to perform actions beyond its defined task and movement space; shall record “data on the safety related decision-making process”; and it shall be possible at all times to correct the machinery.

Regulation (EU) 2023/1230, Annex III §1.2.1 — applicable 14 January 20277

An envelope, an evidence record and retained authority to intervene — the three things in §2, named in law with a date. In the same clause, modifications the machinery generates itself shall be prevented where they could lead to a hazardous situation. Prevention, not detection.

There is a second effect worth knowing about, and it is the commercially interesting one. Annex I Part A puts machine learning that ensures safety functions into mandatory third-party assessment, with no self-certification route. Keeping the model out of the safety function is therefore not only safer; it is a route around that obligation entirely. Our standards posture in full →

§5

Where it applies

Built for

Humanoids, quadrupeds, research VLA rigs, mobile robots, drones and custom actuator setups — the platforms where language models are being put in the loop, and where the only measures generally available today are a person on the e-stop, staying in simulation, or hoping.

Not for

Certified industrial cells. If you run FANUC with DCS or ABB with SafeMove you already have a validated envelope, and you should use it. Those systems already provide the property Endstop exists for: the motion program can change without touching the validated safety configuration.

We will tell you that rather than sell around it.

§6

Why this is not a novel safety architecture

It is not our invention, which is the strongest thing we can say for it.

The reason it is needed at all is that containment and safety are different properties. A sandbox bounds where computation can reach; nothing bounds what legitimately granted authority can do once it gets there — which is why an agent that is merely tricked, rather than escaped, is the harder problem. The problem in full →

Industrial robotics has shipped it for years under type certification. On an ABB system one component plans the motion, and the manual states in as many words that it is not a safety function; a second, safety-rated component supervises every command and stops the machine when the first one is wrong. During validation the planner is switched off altogether, because the thing being validated is the supervisor.4

Our change is a single substitution: the planner is now a language model.

Everything else follows

You do not have to trust the model, review its output, or believe anything about how it was trained. You have to bound what it can cause, and that bound has to live somewhere the model cannot reach.

§7

System state

The system is specified end to end, and its trusted core is proven and running on processors of this class today. Board integration and end-to-end measurement are the current work. In this field the distinction between what is proven, what is engineered and what is merely assumed is most of what you are buying, so we publish it as a column rather than a caveat.

  • Proven

    Memory isolation and absence of undefined behaviour in the interpreter core. Machine-checked, published, independently reviewable, and running now.

  • Engineered

    Envelope monitors, capability mediation and the signed audit chain. Verified by testing and model checking, not by proof.

  • Assumed

    The hardware model, the synthesis toolchain, and the absence of a physical attacker. Each is stated, not buried.

  • Not claimed

    No safety certification. No proof-carrying admission. No guarantee that a correctly-configured envelope is the right envelope — that remains a risk assessment, and it remains yours.

We also publish the questions we have not answered. Our open problems →

Design partners

Bring us a robot a model is driving.

We are looking for a small number of teams with real hardware and a real reason to put a model in a control loop. Platforms with no safety-rated envelope are the ones we can help most: quadrupeds, humanoids, research rigs, mobile robots, drones.

Tell us what you are running and what the model decides, which is usually enough for a useful first reply.

References

  1. Klein et al., seL4: Formal Verification of an OS Kernel, SOSP 2009; proof inventory and stated assumptions at sel4.systems/Verification/proofs.html.
  2. Hardy, The Confused Deputy, ACM SIGOPS Operating Systems Review 22(4), 1988. doi:10.1145/54289.871709
  3. Robey et al., Jailbreaking LLM-Controlled Robots (RoboPAIR), 2024. 100% attack success against NVIDIA Dolphins, Clearpath Jackal + GPT-4o, and Unitree Go2. arXiv:2410.13691
  4. ABB, Application manual — Functional safety and SafeMove2, 3HAC052610-001; and Functional safety and SafeMove (OmniCore), 3HAC066559-001. The planning component is documented as not being a safety function; the safety-rated supervisor is the backstop.
  5. The interpreter core derives from a published, independently reviewable verified bytecode virtual machine. The specific artifact and its proof development are disclosed to design partners under evaluation agreement.
  6. ISO 13855 prices reaction time at an approach speed of 1600 mm/s, giving 1.6 mm of required separation distance per millisecond of latency.
  7. Regulation (EU) 2023/1230 of the European Parliament and of the Council of 14 June 2023 on machinery, OJ L 165, 29.6.2023. Annex III §1.2.1; Annex I Part A items 5 and 6; applicable from 14 January 2027 per Article 54. eli/reg/2023/1230