Quality goals — what this system is allowed to be good at

"The system must be scalable, secure and highly available" is three adjectives. Nobody can build against it, nobody can test it, and nobody can ever be told they have failed to meet it. It survives review precisely because it cannot be contradicted.

The fix is the scenario, and it has three parts. A stimulus: what happens. An environment: the conditions it happens under, including the bad ones. A response with a number and a unit: what the system must do about it. "During a single-AZ failure, checkout completes for 99.9% of requests within 2 seconds at p99, without operator intervention." That sentence can be met, missed, measured and argued with, which is the whole point.

The environment clause is the part most often dropped and the part that carries the cost. A latency target at average load is nearly free; the same target at three times peak, during a partial dependency outage, may be the single most expensive commitment in the document. Write which one you mean.

The limit is not tidiness. Quality attributes trade against each other — consistency against availability, latency against cost, changeability against raw performance, security against convenience — and every real design decision is a choice about which side of one of those pairs to stand on. If everything is a priority, the person making the decision at 4pm on a Thursday has no basis for it, so they will pick by taste and you will find out during an incident.

Rank them, and forbid ties. A tie is not a diplomatic compromise; it is an unmade decision that has been written down as though it were made. The uncomfortable half-hour spent deciding whether availability outranks consistency for this system is the highest-value half-hour in the whole exercise, and it is the half-hour that gets skipped when the goals are gathered by survey.

The test that the ranking is real: name a decision you would make differently if the order were reversed. If no such decision exists, the ranking is decorative and the goals are probably too vague to constrain anything.

For every goal that made the list, something was left off it. Say so explicitly: this system does not optimise for multi-region write availability; this service tolerates minutes of staleness in reporting; this admin tool is allowed to be slow.

Non-goals do more work than goals in practice. They are what stop a well-meaning engineer from spending a quarter hardening something nobody needed hardened, and they are the fastest way to answer the recurring question of whether some new capability is in scope. A spec that only lists ambitions reads as though every property matters, which is indistinguishable from having no goals at all.

Quality goals change on the timescale of the business, not the backlog. They come from what the system is for and what it would cost to be bad at it — revenue lost per minute of downtime, contractual response times, the regulatory floor, the cost of an engineer waiting on a build.

That is also the honest source of the numbers. A latency target derived from a competitor's marketing page is a guess; one derived from the measured drop-off in conversions past 2 seconds is a goal. When you cannot get the real figure, write the assumption next to the number rather than laundering it into a target that looks authoritative.

A quality goal with no guardrail is a preference with good posture. It will be met by accident for a while, then quietly stop being met, and nobody will find out until a customer does.

So the goal carries the ids of the checks that protect it, and the spec makes an unenforced goal visible as unenforced. That visibility is the useful part: a table where two of the five goals have empty guardrail lists is telling you exactly where the system's real risk sits, and it takes ten seconds to read.

It is now normal for a serious service to have SLIs, an SLO, an error budget and a multi-window burn-rate alert. It is still unusual for those numbers to be traceable back to a stated goal that names why 99.9 rather than 99.95 — which means the number is defensible operationally and indefensible commercially.

The gap shows up in the same conversation every time. Someone proposes tightening availability; nobody can price it, because the goal was never written with the cost of the alternative attached; the tightest suggestion in the room wins by sounding responsible. An extra nine bought that way is paid for out of the changeability budget, silently.

ISO 25010 remains the most useful checklist for making sure a category has not been forgotten — its 2023 revision added safety as a top-level characteristic, which matters more now than it read at the time. Use it as a prompt, not as a template: the failure mode of a standard taxonomy is a spec with one goal per characteristic, which is nine goals, which is none.

For changeability, the DORA measures are the closest thing to an industry-standard scenario: lead time, deployment frequency, change failure rate, time to restore. They are convenient because they are already instrumented in most delivery pipelines, so a changeability goal expressed in those terms comes with a guardrail almost for free.

Ten years ago cost efficiency sat in a finance conversation. Now it belongs in the goal set, with a unit — cost per thousand requests, per tenant, per completed job — because the architectural choices that move it are the same ones that move latency and availability, and if it is not ranked it loses every argument until the invoice arrives.

The specific trap of this period: an architecture whose cost scales with traffic, adopted while traffic was small, with no goal stating the ceiling. It is not discovered by monitoring. It is discovered by a finance review, at which point the fix is a redesign rather than a tuning exercise.

In an organisation with a platform team, several quality goals are not the product team's to meet — availability of the deployment path, the observability floor, patch cadence. Say which goals are inherited and from where. Both failure modes are common: a team re-solving a problem the platform already guarantees, and a team assuming a guarantee that was never made.

The clean way to write it is a goal that names its provider and its own residual obligation. The platform guarantees the tier; the service still owns the parts only it can see, which is usually its own request budget and its own degradation behaviour.

Averages are still the default in requirement documents and still nearly meaningless for user-facing paths. Write p99 or p99.9 for anything a person waits on, and state the measurement window along with it — a p99 over a month hides an outage that a p99 over five minutes catches.

Where a fan-out is involved, the arithmetic is worth doing once in front of the team: a page that makes twenty parallel calls each at a p99 of 100ms does not have a p99 of 100ms. Teams that have done that calculation write different goals, and usually fewer of them.

An LLM feature has a quality that is neither functional nor operational: how good the output is. It is not covered by availability or latency, it cannot be asserted by a unit test, and leaving it out of the goal set does not make it stop being the thing users judge the product on.

Write it as a scenario like any other, with the measurement named: on a versioned dataset of representative cases, the feature scores at or above a stated threshold on a stated grader. That makes the eval suite the guardrail, the dataset a versioned artefact, and the threshold a number someone had to choose — which is exactly the treatment every other quality goal gets.

The part that is genuinely new is that the goal must be statistical. Deterministic goals are per-request: this request completed in under 2 seconds. An answer-quality goal is per-distribution: this feature scores 0.85 across the suite. A single bad output is not a breach, and the spec has to say what a breach is instead — a drop in aggregate score, a regression on a named subset, a failure on a safety-critical slice where one is too many.

When a change takes an hour to produce and a day to verify, the property that constrains delivery is verifiability, not development speed. That is now the common case on teams generating a large share of their code, and it changes what the goals should say.

Concretely: goals that used to read as developer-experience nice-to-haves become load-bearing. Build and test-suite duration, because it gates the review loop. Blast radius of a single change, because review attention is the scarce resource. Reversibility — how fast a bad change can be taken out — because the cheapest response to unreviewable volume is to make being wrong survivable.

Once agents are doing work in the system, the unit of cost changes. A request-based ceiling says nothing about a feature where a single user action triggers a chain of model calls with a variable tail, and the tail is where the money goes.

Write the goal in the unit the money is actually spent in — cost per completed task, at a stated percentile, including retries and the failures that produced no result. The percentile matters more here than anywhere else, because the distribution has a long right tail and the mean will tell you everything is fine.

An agent asked to make something faster will make it faster, at some cost you did not ask about — cache the wrong thing, denormalise a table, drop a validation, add a dependency. It has no ranking to consult, so it optimises the attribute in the prompt.

Ranked goals in the machine layer are what turn that into a bounded instruction: improve latency without regressing the higher-ranked goal, and if the two conflict, propose a decision instead of choosing. Combined with guardrails, this is the closest thing to architectural intent an agent can actually act on — an ordered list of what matters, and the checks that prove it still does.

Most of the spine benefits from drafting help. This concern resists it, because a quality goal encodes what the organisation is willing to trade away, and nothing in the codebase contains that. An agent asked to write quality goals will produce a plausible, well-formatted, comprehensively ranked set of five that nobody in the business has agreed to — and it will look better than the real one.

Use the help where it is real: turning an agreed goal into a well-formed scenario, spotting that two goals conflict, pointing out that a goal has no guardrail, checking the percentile arithmetic on a fan-out. The ranking stays with the people who own the consequences.