Guardrails — architecture that can fail a build

Three properties, and a check that is missing any of them is just a test. It names the goal it protects, so a failure explains itself instead of looking arbitrary. It can fail something that matters — a build, a deploy, a release gate — rather than emit a warning into a log nobody reads. And it has an owner, because a check that starts failing for an uninteresting reason and has no owner gets disabled within a week.

The naming is the part teams skip and the part that decides whether the guardrail survives its first inconvenient failure. A rule that says 'module boundary violation' gets deleted by whoever is blocked at 7pm. A rule that says 'this breaks QG-2, the change-failure-rate goal, per ADR-0009' gets a conversation instead.

Build-time. Dependency direction, module boundaries, banned imports, the shape of the public surface. The cheapest and strictest kind: it runs in seconds, the feedback lands while the developer still has the context, and there is no ambiguity about the result.

Test-time. Architecture tests, contract tests between services, performance budgets, security scans. Slower, but able to assert things about behaviour rather than shape — that the retry path is idempotent, that a consumer's expectations still hold, that a request budget was not blown.

Runtime. SLOs with burn-rate alerts, error budgets, saturation thresholds. The only kind that measures the property you actually care about under real load, and the only kind that cannot block a bad change before it ships. You need it, and it is not a substitute for the two above.

Review-time. A human is asked to check something. Include it when nothing else can, and be honest that it is the weakest kind: it depends on attention, attention is finite, and it is the first thing to degrade under deadline — which is exactly when architecture violations get proposed.

Setting a threshold at current behaviour ratchets nothing. It locks in whatever the system does today, including the parts that are already worse than you want, and it guarantees that the first legitimate improvement or regression both look identical to the check.

Derive the threshold from the quality goal, then compare it to the graph. If the goal says 2 seconds at p99 and the system currently does 3.4, you do not set the guardrail at 3.4 and call it protected. You either set it at 2 and accept a failing build until it is fixed, or you record that the goal is aspirational, with a date. Both are honest. Quietly encoding the status quo is not.

Fail the build. Fail the deploy. Page someone. Open an issue. Warn only. Each is legitimate for different checks, and choosing in advance is the entire point — the alternative is choosing during an incident, under pressure, with a strong incentive to pick the one that unblocks the release.

Write it in the spec as a field. The moment 'what happens on breach' is written down next to the guardrail, the team has to admit which of their quality goals they are actually willing to stop a release for. That conversation is uncomfortable and it is the most valuable hour in the whole adoption.

Structure is covered by dependency and layering rules — ArchUnit in the JVM, ts-arch and dependency-cruiser or ESLint boundary rules in TypeScript, import-linter in Python. These are the highest-yield guardrails per hour invested and the most commonly absent.

Infrastructure is covered by policy-as-code evaluated against the plan rather than the running cluster: Conftest and OPA, Checkov, or the equivalent built into your platform. Catching a public bucket in a plan diff costs nothing; catching it in production costs an incident review.

Integrations are covered by contract tests and schema registries with compatibility modes set to something stricter than 'none'. Performance is covered by a small load profile in the pull request and the full one nightly. Supply chain is covered by SBOM generation, provenance attestation and licence policy, which moved from nice-to-have to procurement requirement over the last two years.

Runtime is covered by SLOs with multi-window burn-rate alerting. The single-threshold alert is still the most common implementation and still the wrong one: it pages for a blip and stays silent through a slow burn that will exhaust the month's budget by the 20th.

Estimated infrastructure cost diffed on every pull request, with a threshold that fails or flags, is now ordinary practice rather than a finance-team fantasy. It works because it puts the number in front of the person who can still change the design cheaply.

The version that survives contact with a real team is a percentage and an escape hatch: flag when a change raises the monthly estimate beyond a set bound, and allow an explicit, labelled override that records who accepted the increase. A guardrail with no legitimate way past it gets routed around, and then you have neither the guardrail nor the record.

The pattern that scales is a shared pack of guardrails with sensible defaults, adopted by each service, with local overrides that require a reason and an expiry date. The platform team maintains the implementations; the service team owns which apply and why.

An override with no expiry is a decision. Record it as one. The failure this prevents is the exemption list that grows quietly for three years until the guardrail applies to nothing.

The metric worth tracking is not how many checks run. It is how many can fail a build today, and whether anyone can name the quality goal behind each one. In most repositories the answer is a large number and a small number respectively.

Warning-only mode is where guardrails go to die. It is a reasonable two-week ramp while the existing violations get burned down, and a permanent home for anything that was never going to be enforced. Put an expiry date on warning mode when you enable it.

Keep the pull-request set inside a wall-clock budget — ten minutes is the number past which developers start batching, branching around, and pushing at the end of the day to avoid the wait. Everything slower goes nightly, with a named owner and an alert, not into the pull-request path where it will eventually be disabled by someone with a release to ship.

An agent can produce more changed lines in an afternoon than a team used to produce in a fortnight. The reviewing capacity did not change. Something has to absorb the difference, and the only candidate that scales without adding people is the set of checks that run automatically on every change.

This inverts how a team should spend its architecture effort. Time that went into review checklists and design forums now goes further encoded as checks — not because review is worthless, but because review is now the scarce input and should be spent on the changes the checks flagged as interesting.

A check that only exists inside a CI configuration file teaches the agent nothing until after the build has failed. That is a slow, expensive loop: generate, push, fail, re-read, regenerate — and each cycle burns context and minutes.

Listed in the machine layer with an id, a rule and a threshold, the same constraint becomes something the agent reads before it writes. The check still runs and still fails; it just stops being the first time anyone learns the rule exists. This is the single highest-leverage reason for the machine layer to exist at all.

Any LLM-shaped feature has quality goals that no unit test can assert: answer quality, refusal behaviour, tone, cost per call, latency under real prompts. The guardrail is an eval suite with a versioned dataset and a threshold, run in CI, treated exactly like the load test it resembles.

Three fields make it a real guardrail rather than a dashboard: the dataset version, the threshold, and what happens on breach. And the trigger set has to include prompt and model changes, because in an LLM feature those are deploys — a model version bump can move behaviour further than a fortnight of code changes.

Given a failing check and an instruction to make the build pass, the shortest path is often to change the check. Loosen the threshold, add a path to the ignore list, mark the test skipped. The commit message will be reasonable and the diff will be small.

This needs a meta-guardrail, and it is cheap. Put arch/ and the guardrail configuration under code ownership so those files cannot be modified in a routine change. Assert that the count of enforced guardrails does not decrease. Require a superseding decision for any threshold change. None of this is hostile to agents — it is the same separation between what a contributor may change and what the team must agree, applied to a contributor that works much faster.

A check that fails intermittently was always going to lose its credibility. With agents retrying and re-running builds automatically, an unreliable guardrail becomes noise at machine speed, and the pressure to remove it arrives in days rather than quarters. Determinism is now a functional requirement of the check, not a quality-of-life improvement.