Graph Engineering: When the Graph Earns Its Cost

Two Different Jobs Are Wearing the Same Name

Search graph engineering today and you get two unrelated disciplines stacked on one term.

One is the older usage: knowledge graphs. Entities, relationships, triples, an ontology you query. That graph structures what a system knows.

The other is what the agent crowd started calling graph engineering somewhere in the last eighteen months: the orchestration topology of a multi-step agentic system. Nodes are tool calls or model invocations. Edges are permitted transitions. That graph structures what the system is — its mandates, its boundaries, and what it is allowed to do next.

The label's provenance is unsettled. It appears in vendor blogs, a Claude skill repo, and analyst decks with no shared definition. I use the orchestration sense throughout, because that's where the engineering problem is. Be precise when you say it in a design review, because half the room will hear "ontology."

What is graph engineering for AI agents? Graph engineering for AI agents is the design of a system's control topology: the nodes it may execute, the edges between them, and the state each node can see. A knowledge graph structures what a system knows about the world. A graph in this sense structures what the system is allowed to do next.

The two meet at one point. If your nodes retrieve from a knowledge graph you own both problems, and a retrieval failure inside a node presents as a reasoning failure at the graph level — a confusion diagnosed in retrieval failure versus generation failure.

Read the prerequisite first

This article is about how to build the graph, not whether you should.

The whether question — and the four costs coordination has to pay off — is already argued in multi-agent architecture is premature optimization with a new name: state handoff, failure-mode multiplication, non-deterministic debugging, cost and latency compounding. If you haven't priced those four against a single-agent baseline you actually ran, start there. Everything below assumes you did and the answer came out yes. I'm not re-deriving that frame; I'm going one layer down into what the graph costs to build.

The Shift Underneath the Buzzword: Chat Loops Became State Machines

The interesting change in production agent systems is not the word "graph." It's what teams stopped doing.

The 2024-era pattern was a conversation. Agents talked to each other, a manager agent decided who spoke next, and the control flow was itself a model output. It demoed beautifully. In production it was unbounded, unreproducible, and impossible to cost-model, because the shape of execution was decided at runtime by a stochastic process.

What replaced it: an explicit graph the developer writes. Nodes are units of work — a model call, a tool call, a deterministic transform. Edges are the transitions you permit. The model still decides, but locally: which branch, which tool, whether to retry. The control structure is code.

That is a state machine. Say it plainly, because naming it a state machine inherits forty years of practice — reachability analysis, invalid-transition tests, replay, idempotent transitions, checkpointing. None of it needs reinventing with an agentic prefix.

What is genuinely new is that some transitions are non-deterministic. That's the whole delta, and it's the delta that costs you.

Workflow and Agent Are a Spectrum, Not a Binary

The question "is this a workflow or an agent" is the wrong shape. Almost every production system I've seen is a mix, and the useful axis is a single one:

How much of the control flow do you know at design time?

Read that top to bottom as a cost gradient. Every row down, you hand more discretion to the model and pay more non-determinism for it. The rule that follows: the more of the control flow you know at design time, the more of the graph should be deterministic code rather than model discretion. A routing node that always routes the same way is a hardcoded edge you paid a model call for.

Most teams get this backwards. They put the model in the control path because the framework tutorial did, then wonder why the traversal is different on every run.

What the Graph Actually Costs to Build

Here is the implementation detail the tutorials skip.

State schema design, and the globally-shared-state trap

The default framework pattern is one state object that every node reads and writes. It's convenient and it's the single worst decision in the design.

Thoughtworks moved LangGraph from Adopt to Trial on its Technology Radar in April 2026, specifically over teams treating every multi-agent system as a stateful graph with globally shared state. That is not an anti-LangGraph position, and I'm not making one either. It's a warning about a default. When every node can see and mutate everything, you have rebuilt global mutable state in a system where half the writers are non-deterministic.

Concretely, a shared state blob gives you three problems at once. Node ordering becomes load-bearing in ways nothing documents. A field written by node three and read by node six creates an invisible dependency no type checker catches. And your context cost grows with the union of everything every node ever needed, because the whole blob tends to end up in prompts.

Scope it instead:

// Wrong: one blob every node can read and write.
interface GraphState {
  task: string;
  files: string[];
  plan?: string;
  edits?: Edit[];
  testOutput?: string;
  // ...and 14 more fields nobody can attribute to a writer
}

// Right: each node declares what it reads and what it may produce.
interface NodeContract<TIn, TOut> {
  name: string;
  reads: ReadonlyArray<keyof TIn>;
  run(input: Readonly<TIn>): Promise<TOut>;
}

const planNode: NodeContract<{ task: string; files: string[] }, { plan: string }> = {
  name: "plan",
  reads: ["task", "files"],
  async run(input) {
    // planNode cannot see testOutput. It does not need to. It cannot leak it.
    return { plan: await draftPlan(input.task, input.files) };
  },
};

The contract buys you two things the blob doesn't: the compiler enforces the state handoff instead of your memory, and a node's context window contains only what its contract admits. That second point is the whole of context engineering for long-running agents applied per node instead of per session.

Checkpointing and durable execution

Agent graphs run long. Minutes, sometimes hours. A process that runs for an hour will be interrupted — deploys, OOM kills, spot instance reclaim, a rate limit that outlasts your retry budget.

Without checkpointing, an interruption means you re-run the whole graph and re-pay for every token you already spent. That is the cost-compounding problem from the prerequisite post, except now you're paying it twice for the same work.

Checkpointing means persisting state after each node transition, keyed so a resumed run picks up where it stopped:

interface Checkpoint {
  runId: string;
  nodeName: string;
  seq: number;
  state: unknown;      // serialized node output
  schemaVersion: number;
}

async function step<TIn, TOut>(
  node: NodeContract<TIn, TOut>,
  runId: string,
  seq: number,
  input: TIn,
  store: CheckpointStore,
): Promise<TOut> {
  const existing = await store.load(runId, seq);
  if (existing) return existing.state as TOut; // replay, don't re-pay
  const out = await node.run(input);
  await store.save({ runId, nodeName: node.name, seq, state: out, schemaVersion: 1 });
  return out;
}

That's twenty lines and it looks finished. It isn't. schemaVersion is there because you will deploy a changed state shape while runs are in flight, and a resumed run will hand a v1 checkpoint to a v2 node. You need a migration path or an explicit abandon policy. Nobody writes that on day one, and it is the failure that wakes you up.

Resumability after a worker crash

Checkpointing gets you the state. Resumability needs two more things.

Idempotent nodes. A crash between "node completed" and "checkpoint saved" means the node re-runs. If it wrote a file, sent a message, or created a resource, it does that twice. Every node with a side effect needs an idempotency key derived from the run and the sequence, not from a timestamp.

Ownership and leases. If a worker dies mid-node, something has to notice and reclaim the run. Without a lease, you either get an orphaned run nobody resumes, or two workers resuming the same run and racing each other's writes.

Per-node context scoping

The reason to scope context is not tidiness. It's that a node given the whole conversation makes worse decisions than a node given only its inputs. The verify node does not need the plan's reasoning. Handing it the reasoning gives it a story to agree with instead of an artifact to check.

Scoping is also the only defense against a graph that gets more expensive as it gets more nodes for no capability gain.

Fan-out multiplies more than tokens

Parallelism is the most honest reason to build a graph and the fastest way to a bill you have to explain. Seven parallel edit nodes are seven concurrent model calls, seven concurrent tool surfaces, and seven chances to hit a rate limit at the same instant. Your throughput becomes provider-bound, and your retries all fire together.

Bound the fan-out explicitly. And attribute the spend per node, or you will not be able to answer the only question leadership will ask, which is the cost accountability question: what did this node buy?

Debugging a non-deterministic traversal

The repro isn't a case. It's a distribution. Two runs of the same input can traverse different edges, and the bug lives either in a node or in the transition between two of them.

What makes this tractable is recording the traversal as a first-class artifact — the ordered node sequence, the state diff at each edge, and the reason each branch was taken. That's a trajectory, and evaluating it is a different discipline from evaluating any single node's output. It's the subject of agent evals and observability at the trajectory level.

The Same Task, Two Topologies

BENCH-1 is the task running through this pathway: a 40-file TypeScript service calls a deprecated db.queryRaw() in seven places, and every call site must move to the parameterized db.query() with the suite still green.

Note what the graph bought and what it cost. It bought a read-only survey node that cannot corrupt the repo, a verify node that never sees the plan's reasoning, and seven edits in parallel instead of serial. It cost a plan artifact that has to survive the handoff into every edit node, and a checkpoint store so an interrupted run doesn't re-survey 40 files.

The Earning Test

Here is the rule I apply per node, not per system. A node earns its coordination cost if at least one of these is true:

1. It has a distinct tool or permission surface. The survey node is read-only. The verify node can only run tests. This is a real containment property you cannot get inside one agent holding every tool. It is the same argument as the harness permission contract, enforced at the topology level.
2. It holds context that must not be shared. A node touching customer PII, or a judge that must not see the reasoning it's judging. Isolation is the product, not a side effect.
3. A human has to approve the transition. A graph with a durable checkpoint can pause at an edge for hours and resume. A single agent loop cannot pause; it can only block.
4. The work is genuinely parallel. Seven independent call sites, seven concurrent nodes, one wall-clock cost instead of seven.

If a node satisfies none of the four, it is a function call you paid a model invocation for. Collapse it.

Apply that test to the whole graph and you get the honest version of B-04: over-engineering is a confession that you didn't understand the problem. I have made this confession expensively. I built a SaaS with multi-tenancy, a plugin system, and an event bus for load that existed only in my head; it never launched and now serves two or three internal users. A planner-critic-executor-orchestrator graph with globally shared state, built for a failure nobody witnessed, is that same event bus with better branding. The full version of that argument is in the prerequisite post; the version that matters here is narrower. Every node you cannot justify against the four conditions is coordination cost with nothing on the other side of the trade.

Build or Framework: What Durable Execution Actually Costs You

Article 2 of this pathway argues you can build a working harness in around 200 lines without a framework, and you can. The graph layer is where that argument gets harder, and I want to be honest about why.

A harness is a loop with a tool dispatcher. Durable execution is a distributed systems component. Writing it yourself means writing: a checkpoint store with transactional writes; sequence numbering that survives concurrent workers; leases with expiry and reclaim; state schema versioning and migration for in-flight runs; idempotency keys on every side-effecting node; replay that reproduces a traversal exactly; and enough tracing to reconstruct why an edge was taken. That is not a weekend. That is a component with its own on-call rotation, and it is a component whose bugs surface as "the agent did something weird two hours in."

So the recommendation, plainly:

Build the graph yourself if it is small, mostly deterministic, and short-lived. Fewer than about six nodes, no human-in-the-loop pause, runs measured in seconds. A switch statement and typed node contracts will beat a framework on clarity and cost, and you'll actually understand the traversal.

Take the framework when you need durability. Human approval gates, hour-long runs, resumption after crashes, fan-out you need to bound and observe. Checkpointing and durable execution are the framework's real product — not the graph abstraction, which is trivial. Pay for the hard part.

And whichever you pick, override the shared-state default. Scope your node contracts. That is the specific trap the Radar downgrade was pointing at, and it's a design decision, not a framework limitation.

The Case Where It Was Earned

I built a custom agent-orchestrator for POC and MVP work, and it does ship MVPs faster — 3x against a cycle-time baseline we tracked. That's the system's number, not mine.

What made it worth its coordination cost maps directly onto the four conditions, and I did not know that when I built it. The nodes mirrored real process steps with genuinely different permissions — the node that clarified requirements could not touch code, and the node that scaffolded could not talk to stakeholders. Some steps needed a human to approve before the next transition, which meant durable pause was a requirement rather than a nice-to-have. And the boundaries were places where a handoff artifact already existed in the process, so the state contract wrote itself.

The cost came first. There was a real stretch where coordination returned less than it took. It became worth it only after the state contracts were explicit and the evals could tell me which node regressed. The graph didn't make it work. Paying the graph's costs down deliberately did.

BENCH-1: Single-Agent Loop vs Scoped Graph

Same task, same model, same tools, twenty runs each. Single agent loop with all four tools in one context, versus the scoped graph above with per-node tool surfaces and bounded fan-out.

I'll state the prediction before the numbers exist, because a prediction you can be wrong about is worth more than a table you interpret after the fact.

The graph should lose on tokens. Every node boundary re-pays for context. The plan artifact gets read seven times by seven edit nodes. There is no version of this where the graph is cheaper per run.

The graph should lose on median wall clock and possibly win on p95. Fan-out helps the seven edits; the survey and plan nodes are serial overhead the single agent doesn't pay separately. But the single agent's worst runs are the ones where it loses the seventh call site and burns its budget rediscovering it, and that's a p95 problem the graph's explicit survey node should not have.

So the graph has to win on pass rate, or it lost. That is the entire justification. If both configurations pass 20 out of 20, the graph is more expensive machinery producing an identical result, and the correct engineering decision is to delete it. If the single agent drops call sites — the failure I expect, since seven scattered edits is exactly where a long context loses track — then the scoped graph's survey-plan-verify structure earned every token it spent.

Watch for the outcome nobody wants to publish: the graph passes less often. A plan that flattens a call site's nuance, handed to an edit node that cannot see the original file, produces a confident wrong edit that verify catches only if the test covers that path. Coordination adds failure surface. That's cost two from the prerequisite post, arriving on schedule.

Your Monday Morning

Open your graph. For every node, write which of the four conditions it satisfies: distinct tool surface, unshared context, human approval, genuine parallelism.

Any node with a condition next to it stays. Any node with a blank gets collapsed into its neighbor this week. Then find the state object every node reads, and split it into per-node contracts — start with the node that reads the most fields and needs the fewest.

If you can't finish that exercise because you don't know what state crosses each edge, that's the finding. The graph isn't the design yet. The state contract is.

FAQ

What is graph engineering for AI agents?

Graph engineering for AI agents is the design of an agentic system's control topology: the nodes it may execute, the edges between them, and the state each node can read. You write the control structure; the model makes local decisions inside it. In practice it means designing a state machine where some transitions are non-deterministic, and pricing that non-determinism.

Is graph engineering the same as a knowledge graph?

No, and the terms collide badly in search results. A knowledge graph structures what a system knows — entities, relationships, an ontology you query. Graph engineering in the agent sense structures what a system is — its nodes, permitted transitions, and permission boundaries. Different problems, different failure modes. If your nodes query a knowledge graph, you own both.

When should you use LangGraph in production?

When you need durable execution: human approval gates, runs lasting minutes to hours, or resumption after worker crashes. Checkpointing and durability are the real product; the graph abstraction itself is easy to write. Override the globally-shared-state default — Thoughtworks moved LangGraph from Adopt to Trial in April 2026 specifically over teams treating every system as one shared stateful graph.

What is the difference between a workflow and an agent?

It's a spectrum, not a binary, and the axis is how much control flow you know at design time. Know the steps and their order: write deterministic code. Know the goal and the tool surface but not the path: that's an agent. Most production systems are a mix. The more you know upfront, the more of the graph should be code rather than model discretion.

Why is shared state a problem in agent graphs?

A state object every node reads and writes is global mutable state where half the writers are non-deterministic. Node ordering becomes silently load-bearing, cross-node dependencies escape the type checker, and context cost grows to the union of every field any node ever needed. Scope state per node with explicit read/write contracts so the compiler enforces handoffs instead of your memory.