Agent Architecture Patterns

Most production "agents" are not agents. They are workflows with a language model in one of the boxes — and that is usually the correct design.

The distinction is worth being precise about, because it decides everything downstream:

• A workflow has a control flow you wrote. The model fills in steps. You know the path in advance, you can test each edge, and the cost per request has an upper bound.
• An agent decides its own control flow. It chooses which tool to call and when to stop. You cannot enumerate the paths, which is exactly why it is useful and exactly why it is hard to operate.

Choose the least agentic architecture that solves the problem. Every increment of autonomy costs you predictability, latency, token spend, and the ability to reason about failure. Pay for it when the task genuinely has an unknown number of unknown steps.

The decision, in one table

1. Single call

The baseline. One model invocation with structured output.

input → [prompt + schema] → validated output

Use when the task is one transformation: classify, extract, summarise, rewrite, translate.

What people get wrong. Reaching for a framework before trying this. A surprising share of shipped agent systems reduce to one well-specified call with a JSON schema and a retry on validation failure.

Operational notes. Constrain the output with a schema and validate it. Retry on validation failure with the error text fed back — one retry recovers most of them. Set an explicit token cap.

2. Prompt chain

Fixed sequence, each step's output feeding the next, with a check between steps.

input → step 1 → [gate] → step 2 → [gate] → step 3 → output

Use when the task decomposes into stages you can name in advance: outline then draft then edit; extract then normalise then validate.

Why it beats one big prompt. Each step gets a focused instruction and a small context, which raises accuracy. Each step is independently evaluable, so a regression is attributable. And the gates let you fail early instead of paying for four more calls.

What people get wrong. No gate between steps. Without one, an error in step 1 is laundered into confident output by step 3, and the trace looks fine.

3. Routing

Classify the input, then dispatch to a specialised handler.

input → classifier → { handler A | handler B | handler C }

Use when inputs fall into categories with genuinely different handling — support triage, query type, document class, difficulty tier.

The economic case. Routing is the standard way to control cost. A small fast model classifies, and only the hard 15% reaches the expensive model. This is often a 5–10x cost reduction with no measurable quality loss, and it is the highest-return change available to most deployed systems.

What people get wrong. No fallback branch. Real inputs will not match your categories, and the default route must be a real handler rather than an error.

Evaluate it separately. Routing accuracy is its own metric. A system that is 95% accurate per handler but 80% accurate at routing is a 76% system, and the post-mortem will blame the handlers.

4. Parallelisation

Two distinct shapes, often confused.

Sectioning — split the work, run the pieces concurrently, combine:

input → { subtask 1, subtask 2, subtask 3 } → aggregator → output

Voting — run the same task several times and take consensus:

input → { attempt 1, attempt 2, attempt 3 } → vote → output

Use sectioning when subtasks are genuinely independent — reviewing ten files, checking five policies. Latency becomes the max rather than the sum.

Use voting when the failure mode is variance rather than capability, and a wrong answer is expensive. Three samples with majority agreement measurably reduces error on reasoning tasks. It also triples cost, so reserve it for the decisions that justify it.

Underrated use of sectioning: running guardrails alongside the main task rather than before it. A safety classifier and the primary generation on separate calls, concurrently, costs no additional latency and keeps the safety instruction out of a prompt where it competes for attention.

5. Evaluator–optimiser

A generator produces, a separate evaluator critiques against explicit criteria, the generator revises. Loop until the evaluator passes or the iteration cap is reached.

input → generator → evaluator ──pass──→ output
             ↑            │
             └──feedback──┘

Use when the quality criteria can be articulated and a critic can apply them more reliably than the generator can self-apply them: code that must compile and pass tests, translations with a style guide, writing with a rubric.

The hard requirement: the evaluator must have information the generator lacks, or a genuinely different vantage point. Test results, a compiler, a linter, a retrieval check, a rubric the generator never saw. An evaluator that is the same model with the same context self-approves. That is the most common way this pattern fails, and it fails quietly — the trace shows two passes and a confident conclusion.

Always cap iterations. Two to three. Improvement after three is rare, and unbounded loops are the standard way to spend a lot of money in the small hours.

6. Orchestrator–worker

An orchestrator decomposes a task into subtasks at runtime, dispatches them to workers, and synthesises the results.

                ┌→ worker → ┐
input → orchestrator → worker → synthesiser → output
                └→ worker → ┘

Different from sectioning in one specific way: the subtasks are not known in advance. The orchestrator decides what they are based on the input. If you can write the list of subtasks yourself, you want parallelisation, and you should not pay for the orchestrator.

Use when the decomposition genuinely varies: research across an unknown number of sources, a code change touching an unknown set of files.

Why it works. Each worker gets a clean, small context scoped to one subtask. Context length is the binding constraint on quality in long tasks, and this pattern is fundamentally a context-management technique.

What people get wrong. Workers that share mutable state. They should be independent, and if two workers must coordinate, the decomposition was wrong. Also: no budget. The orchestrator must have a hard cap on worker count and total tokens, because a decomposition step that runs away is unbounded by construction.

7. Tool-use loop

The actual agent. The model receives a goal and tools, and iterates — choose a tool, execute it, observe the result, decide again — until it judges the goal met or a limit stops it.

goal → [ model → tool call → environment → observation ] × N → result

Use when the number and order of steps cannot be known ahead of time and the environment gives real feedback. Coding agents, operations investigation, browsing, multi-step retrieval.

The requirement most systems miss: the feedback must be real. A tool that returns "success" without verification gives the model nothing to correct against, and it will loop confidently in the wrong direction. Return compiler output, test results, HTTP status, actual retrieved content, the diff that was applied. Ground truth is what makes the loop converge.

Non-negotiable controls:

Tool design is the leverage point. Agents fail at tool boundaries far more often than at reasoning. Fewer tools, unambiguous names, descriptions written for a reader with no other context, parameters that are hard to get wrong, and error messages that state what to do differently. A well-named tool with a good error message outperforms a better model with a poor one.

8. Multi-agent

Several agents with distinct roles, prompts, tools and contexts, coordinating through messages or a shared artefact.

Use when the subproblems require genuinely different tools, different context, or different permissions — not because "a team of specialists" is an appealing metaphor.

The cost is not incremental. A multi-agent system multiplies token spend (often 4–15x a single agent on the same task), multiplies latency, and introduces coordination failures that do not exist in a single loop: agents disagreeing, duplicating work, waiting on each other, or losing information at every handoff. Every message between agents is a lossy serialisation of context.

Before adopting it, satisfy all three:

1. A single agent with the union of the tools has been tried and measurably fails.
2. The subproblems need different context, not just different instructions.
3. You have per-agent traces and evals, because debugging a multi-agent failure without them is guesswork.

Most multi-agent systems in production would perform better as an orchestrator–worker with stateless workers. The full treatment is in multi-agent architecture: premature optimisation.

Composition

Real systems layer these. A common and effective production shape:

input
  → guardrail check (parallel)
  → router (small model)
      ├─ simple  → single call
      └─ complex → orchestrator
                     ├→ worker (tool-use loop, capped)
                     └→ worker (tool-use loop, capped)
                   → evaluator (max 2 iterations)
  → output validation → response

Note what is at the edges: a cheap classifier at the front and a deterministic validator at the back. Keep the model in the middle and determinism at the boundaries. Everything you can check with code, check with code.

The controls that apply to every pattern

Failure modes by pattern

Choosing, honestly

Start with the simplest pattern that could work. Measure. Move up only when a measurement — not an intuition — says the current pattern is the ceiling.

The order of escalation that costs least: single call → chain → routing → parallelisation → evaluator–optimiser → orchestrator–worker → tool-use loop → multi-agent. Most teams jump to the last two and spend a quarter working backwards.