Guardrails

Rules that can say no.

A written rule binds nobody. It is a preference with good posture until something is willing to refuse on its behalf, and refuse at the moment it is inconvenient.

These three are the ones that refuse. They are ordered by what each protects: who is allowed to approve, what is allowed to cost, and what is allowed to be believed. Each is stated so that a system can act on it, which is a harder test than being stated so a person can agree with it.

The tiers are the same as on the governance page: a timeless article changes only by ratification, a standing rule applies everywhere, project policy binds one body of work.

01 · Who may approve

The writer never approves its own work.

Review by the author is not review. In the current harness this is not a convention anyone has to honour, it is a tool restriction: the reviewers hold no write tools, so a reviewer that wanted to quietly fix what it found could not.

A critical change is approved as a separate, recorded act from proposing it, by a different party wherever more than one exists.

A standing rule

Executors implement exactly one assigned task each and report back. They never re-plan, never expand scope.

Project-level policy, local to one body of work

The user sets the active track. Parked items are never self-promoted above it.

A timeless article, changed only by ratification

Scope creep in agentic work is not ambition, it is drift. An agent that widens its own remit is doing work nobody reviewed against a brief nobody agreed.

Approval decides who may say yes. The next question is what a yes is allowed to cost, because an action that cannot be undone and an action that cannot be un-billed fail in the same direction.

02 · What it may cost

Spending and destroying both need a signature.

Autonomy frees execution. It never frees scope, spending, or anything that cannot be undone.

An irreversible action is not taken without explicit, recorded authorization beforehand.

A standing rule

An action that incurs real cost has a stated bound before it runs, and that cost is visible to the party who authorized it.

A standing rule

Paid operations are disarmed by default: mock providers, keys absent from the environment. Arming is deliberate and logged.

A timeless article, changed only by ratification

On a live project this meant real model spending stayed behind a hold, and a single bounded, logged arm and disarm cycle was authorised as a narrow exception without lifting the standing rule. The exception was recorded as its own decision, with its scope written down.

The first two rules constrain the people and the budget. This one constrains what the system is willing to believe, which is the surface an attacker actually reaches.

03 · What it may believe

A document is data, never an instruction.

The moment an agent reads untrusted content, everything in it is a potential instruction. Treating retrieved material as authoritative is how an agentic system gets turned against the person running it.

An artifact received from another party is data, not instruction: it is checked before it is acted on, and content inside it never acquires authority by being read.

A standing rule

Sensitive data is classified, and secrets are held by reference, never copied into the work in cleartext.

The one standing rule with a mechanical gate and no waiver

A party is granted only the access and permissions its current task requires, and no more.

A standing rule

On client work this extended to a responsible-AI deliverable mapped to NIST AI RMF, ISO/IEC 42001, the EU AI Act and Canadian requirements, with data residency and a redaction path established before any document reached a model.

04 · What must not collapse

Six pairs that must not collapse into one.

Every one of these cost something before it was insisted on. The mechanism came after the mistake, never before it - which is why the third beat of each pair names the mistake rather than the principle.

Instruction ≠ Enforcement

Collapsed
A rule written into a prompt is counted as a control. It is a request - followed until following it is inconvenient, with nothing structural in the way.
Held by
Rules behind a lock the agent cannot open, with the unlock outside the process entirely: set before the session starts, unreachable from inside it. Where it could not be locked, it went behind a pre-write hook rather than an instruction.
It cost
The first version banned the whole plugin mechanism. The rule was never "no plugins" - it was that the gate must not be modifiable by the thing it gates. One implementation had been mistaken for the principle.

"Consult the advisor subagent before your first file write or state-changing command, and again when stuck or when you believe you're done. The PreToolUse gate enforces this if installed."

A standing instruction, as written

A logged "done" ≠ A done thing

Collapsed
An agent reports the task complete and the report is filed as the outcome - a claim about reality made by the party with the strongest incentive for it to be true.
Held by
A tool logging "fixed" is not done. Done is committed, pushed, merged. No reference in version control and it stays a claim. Every material statement carries its origin: repository fact, tool report, or inference.
It cost
Learning it the expensive way. A coding agent logged completion three separate times while leaving the work uncommitted, and an automated sweep left fourteen stale branches against mixed bases that took a costly triage to untangle.

"I'm not sure if the agent was hallucinating. Make sure we don't lose anything or end up with malfunctioned code."

Said mid-project, as written

When to decide ≠ How carefully

Collapsed
Decision frameworks sum urgency and importance into one score. Cheap decisions then draw full ceremony and expensive ones get rushed, because a single number cannot carry two questions.
Held by
Timing from pressure plus cost minus learning; rigour from cost alone. Six outcomes, including settled - both cheap and certain, so resolve by convention, record it, never raise it again.
It cost
Two failed runs, documented in the source above the fix. The first sent twenty-four decisions on a static docs site to full ceremony. The sixth outcome exists because of the second failure.

"WHEN to decide and HOW CAREFULLY are independent questions."

The model

Derivation ≠ Authenticity

Collapsed
An append-only log is replayed, the state matches, and the log is called trustworthy. Replay proves the arithmetic. It says nothing about whether the entries are the original ones.
Held by
Nothing yet, and the record says so. The audit log is a plain file, append-only by contract; a hand with shell access outside a gated session can edit it like any other file. Non-repudiation is left to version control and its remote, and the trust document publishes the absence rather than implying a mechanism.
It cost
A sentence on this page. An earlier draft described a hash chain and an independent verifier that did not exist. It was written here, not sourced there, and it came out when the documentation was re-read against the tree.

"Detection instead of prevention, a record that cannot be forged."

Design note, carried ahead of any code

Who writes the memory ≠ Whether the write is gated

Collapsed
Self-improving agents get compared on what they remember. The question that decides trust is different: does anything validate the edit, and can the thing being validated do the validating?
Held by
A four-axis comparison - what is stored, who writes, when, and whether the write is gated - applied across every major approach. The gating axis splits almost perfectly along the authorship axis, and that is not a coincidence.
The finding
You cannot build a performance gate without a held-out set and an outer loop, and you cannot have an outer loop if the agent edits itself mid-conversation. The self-editing designs have no gate. Not an oversight - structural.

none · consistency · performance · human - and only one class of design reaches the third.

Gate types observed

A skill you have ≠ A skill being held up

Collapsed
Working fluently with an assistant reads as competence. Some of it is. Some would collapse the moment the assistant was removed, and nobody can tell which from outside - including the person.
Held by
A cartography placing each capability in one zone: bedrock - demonstrated repeatedly, under pressure, unassisted · load-bearing but cracked · scaffolding - held up entirely by the AI · claimed but unwitnessed · latent.
It cost
Running it on the person who built it, on session evidence rather than self-report, and keeping what it returned. It carries an anti-bias protocol aimed at the assessor and a mandatory step arguing against its own conclusions.

"A profile that cannot be wrong is astrology."

Design rule
05 · Where a guardrail failed

A guardrail that has never failed has never been tested.

Three of the four published corrections are about enforcement: one where a finding was deferred as a scheduling decision when it was not, one where a review layer overruled the person who built it, and one where a fail-closed gate did exactly what it was designed to do and stopped every project on the machine.

A review finding was set aside as a scheduling decision. We accepted the framing first, then refused it: deferring that particular check was not scheduling, it was a guardrail bypass wearing scheduling as a disguise. The check went back in before anything else moved.

Why it matters: a bypass almost never arrives as a request to bypass something. It arrives as a reasonable-sounding reason to do it later, and the only defence is noticing which findings are allowed to be deferred at all.

The habit underneath it: “all green” is not proof. A passing suite proves the tests passed, not that the fix is right - and a deferred check is a test that was never allowed to run.

During a rendering library migration we ran an equivalence sweep across pages, zoom levels, rotations and clip regions, then generalised the result over document content. An executor working a sub-task disputed the conclusion. It was correct: the sweep had varied everything except content, then drew a conclusion about content anyway.

The correction also went the other way. The executor described the deviation as at most one sample level; the measured maximum was two. Both errors are in the record.

Why it matters: the point of a review layer is that it can overrule the person who set it up. A hierarchy where challenges never land is theatre.

A syntax error in the harness left the gate module undefined. Because the gate is registered globally and fails closed, it then denied every tool call on the machine, including in projects that had never adopted it. Work stopped everywhere until it was restored.

The write-up does not blame the typo. It names the design flaw: the fail-closed path depended on the same module whose failure triggered it, so a gate that denies when it cannot verify itself has no degraded mode.

Why it matters: safety mechanisms have failure modes too, and the interesting question is never whether they will fire wrongly. It is what happens to everyone else when they do.

Next

The interesting conversation is about your gates, not ours.

If you are putting agents near anything that matters, the questions above are the ones worth an hour. A guardrail that has never been tested is a claim, so the record of the ones that were tested is on this page if you want it first.

Start a conversation Read what failed →