Context Engineering for Long-Running Agents: Budget, Compaction, Handoff

The First Failure Mode I Named Was Context Loss

In mid-2025 I committed to AI-assisted development on real products, and then kept going — ask mode, agent mode, MCP, retrieval, agentic features shipped where my architecture decisions carried consequences. I ran that progression in the open, in front of senior engineers who averaged eight years and had watched several hype cycles arrive and leave.

When people asked me what actually broke, I named two things before anything else. Confident wrong answers. And context loss.

Context loss was the one nobody wanted to hear about, because the fix looks like it should be a config change. It isn't. The model does not tell you it lost the thread. There is no exception, no stop_reason, no span that goes red. An agent forty minutes into a task simply starts behaving like an agent that never read the first thirty minutes. It re-reads a file it already edited. It reintroduces a decision you overturned at turn six. It wraps up early and declares success on a third of the task.

That is the honest version of the lesson from going AI-native early: judgment did not become obsolete, it relocated. The scarce skill stopped being writing the code and became deciding what is worth putting in front of the model. Context engineering is that judgment, made mechanical.

And the head-term version of this topic — "what is context engineering" — is not the useful question. The useful question is what happens on hour three.

Context Is a Budget, Not a Container

Here is the fact that reframes everything: models degrade as occupied context grows, well before the window is full. Frontier models now ship million-token windows. The window is a hard ceiling on what fits, not a promise about what the model will reason over reliably. Recall softens. Instruction-following slackens. Earlier constraints stop binding.

The context budget is the share of a model's context window you can occupy while it still reasons reliably — a working figure well below the advertised limit. Treat it as a depleting resource with line items you allocate deliberately, not a container you fill until it errors. Every token you spend on one line item is a token unavailable to another.

The line items are these: the system prompt, tool schemas, retrieved documents, tool outputs, conversation history, and the agent's own scratchpad. Nearly every team can name them. Almost none can tell you the split on a real run. That is the gap worth closing first, because you cannot budget what you have not measured.

Two measurement traps get people. First, usage.input_tokens is the uncached remainder only — the real prompt size is input_tokens + cache_creation_input_tokens + cache_read_input_tokens. An agent that ran for two hours and reports 4K input tokens is not efficient; the rest came from cache. Second, never estimate with a tokenizer built for another vendor's models. Use messages.count_tokens, per model.

import Anthropic from '@anthropic-ai/sdk';

type BudgetLine =
  | 'system'
  | 'tool_schemas'
  | 'retrieved_docs'
  | 'tool_output'
  | 'history'
  | 'scratchpad';

interface TaggedMessage {
  line: BudgetLine;
  message: Anthropic.MessageParam;
}

const client = new Anthropic();
const MODEL = 'claude-sonnet-5';

export async function auditBudget(
  tagged: readonly TaggedMessage[],
  system: string,
  tools: readonly Anthropic.Tool[],
): Promise<Record<BudgetLine, number>> {
  const spend: Record<BudgetLine, number> = {
    system: 0, tool_schemas: 0, retrieved_docs: 0,
    tool_output: 0, history: 0, scratchpad: 0,
  };

  const probe = { role: 'user' as const, content: 'x' };
  const floor = (await client.messages.countTokens({
    model: MODEL, messages: [probe],
  })).input_tokens;

  spend.system = (await client.messages.countTokens({
    model: MODEL, system, messages: [probe],
  })).input_tokens - floor;

  spend.tool_schemas = (await client.messages.countTokens({
    model: MODEL, tools: [...tools], messages: [probe],
  })).input_tokens - floor;

  for (const { line, message } of tagged) {
    const n = (await client.messages.countTokens({
      model: MODEL, messages: [message],
    })).input_tokens - floor;
    spend[line] += Math.max(n, 0);
  }
  return spend;
}

Per-line figures will not sum exactly to the request total — request framing costs a little, and the probe subtraction is approximate. That is fine. You are looking for the line item that owns 60% of the window, and it is almost always tool output. Runnable version, with the per-run report: github-repo/agent-engineering/01-context-budget/.

The Compaction Ladder

Once you can see the split, you can reclaim room. Climb in order — each rung is more expensive and costs you more fidelity than the one below it. Do not start at the top because the top rung is the one with a config flag.

Rung (a) is where the leverage is, and it is the rung nobody instruments. Anthropic's own managed harness offloads any tool result over 100,000 characters — roughly 25,000 tokens — to a file and hands the agent a truncated preview plus the path. That is a sensible default, and it is also an admission: a single unbounded tool result can eat a quarter of your usable budget in one call. Your handlers should be doing that shaping at the source, where you know the schema, rather than letting a generic threshold decide.

Rungs (c) and (d) are different operations that get conflated. Context editing clears — stale tool results and thinking blocks are removed. Compaction summarizes — earlier context is replaced by a generated précis. Clearing is cheap and lossy in a predictable way. Summarizing is expensive and lossy in an unpredictable one.

// Rung (c): clear stale tool results — structural, predictable loss.
await client.beta.messages.create({
  model: MODEL,
  max_tokens: 16000,
  betas: ['context-management-2025-06-27'],
  context_management: { edits: [{ type: 'clear_tool_uses_20250919' }] },
  tools: [...tools],
  messages,
});

// Rung (d): server-side compaction. The trap: append response.content,
// not just the text. The compaction block IS the state — extract the
// string and you silently lose it on the next turn.
const res = await client.beta.messages.create({
  model: MODEL,
  max_tokens: 16000,
  betas: ['compact-2026-01-12'],
  context_management: { edits: [{ type: 'compact_20260112' }] },
  messages,
});
messages.push({ role: 'assistant', content: res.content });

Compaction has a second cost that does not show up in a fidelity discussion. Prompt caching is a prefix match: change the prefix and every cached token behind it is invalidated. Compaction rewrites the prefix by definition. Illustrative arithmetic on published multipliers — cache reads bill at roughly 0.1× base input, five-minute cache writes at 1.25× — a compacted 80K-token span that would have been read for the equivalent of 8K tokens is instead re-written at the equivalent of 100K. You paid tokens to save tokens. That is the context and cost compounding cost from the four-cost frame in multi-agent architecture as premature optimization, showing up inside a single agent with no graph anywhere in sight.

Compaction Preserves Continuity. It Does Not Preserve Anything.

A compaction summary lives in the message array that produced it. Kill the process and it is gone. Ask what the agent decided at turn 40 and you cannot query it — you can only ask the agent, which will answer from the summary and sound just as confident either way. Compaction is continuity without durability.

There is a second, weirder failure. Long-session agents get anxious about the window. Anthropic documents this on its own most capable model: deep into long sessions it can start worrying about running out of context and suggest a fresh session or trim its own work, most often when the harness surfaces a remaining-token countdown. The recommended mitigation is to stop showing the count. Sit with that. Your progress meter is an input to the model's behavior, and the honest one makes it quit early.

The fix for both problems is the same, and it is not a better summarizer. Stop trying to keep the record in the context. Put it on disk.

interface HandoffArtifact {
  task: string;
  goal: string;                      // restated; the one line a fresh session must honour
  status: 'in_progress' | 'blocked' | 'done';
  decisions: {
    id: string;                      // D-01, referenced by later entries
    decision: string;
    rationale: string;
    supersedes?: string;
  }[];
  completed: { step: string; evidence: string }[];   // evidence = a tool result, not a claim
  open: { step: string; blocker?: string }[];
  invariants: string[];              // constraints that must hold at every step
  artifacts: string[];               // paths, commits, branches — the durable outputs
  contentSha256: string;             // for compare-and-swap writes
}

Three properties make this beat a summary. It is structured, so a fresh session reads invariants and open without re-deriving them from prose. It is queryable, so you can diff the artifact across two runs and see exactly where a non-deterministic agent diverged. And it is durable — it outlives the session, the process, and the model version. Git history and a JSON task list are the same pattern with less ceremony.

Two things make it work in practice. Require evidence: a step is only completed if it names a tool result you can point at, which is the cheapest guard against fabricated progress. And write with a compare-and-swap on the content hash, so two agents editing the same artifact get a conflict instead of a silent overwrite. Models are measurably better at long-horizon work when they have somewhere to write learnings — even a plain Markdown file. Give them a schema and it stops being a diary.

The harness contract that owns reading and writing this artifact is agent harness design. The stop conditions that decide when to hand off rather than push on are in loop engineering.

Subagent Isolation Is a Context Strategy, Not an Org Chart

Here is the framing that makes subagents legible. The parent context is a scheduler. A subagent burns forty thousand tokens exploring, and returns a two-hundred-token answer. The saving is not parallelism and it is not specialization. The saving is that the exploration never enters the parent window.

The same mechanism shows up without any agents at all. Programmatic tool calling lets the model write a script that calls tools from inside a sandbox — results return to the running code, not to the context. Token cost scales with the script's final output rather than the intermediate data. Same principle, smaller blast radius, and usually the right thing to try first.

Whether you should reach for multiple agents at all is a separate argument with its own costs, and it is settled elsewhere: graph engineering covers the topology and what it has to earn. The point here is narrower. If you do isolate, isolate for the window, and size the returned result deliberately — a subagent that hands back its full transcript has bought you nothing.

What This Is Not

I built SpecLoom to solve a narrower problem: making context deterministic for coding agents by compiling a spec bundle, so the same task produces the same context every run. That is a different axis. SpecLoom fixes what goes in at the start; this article is about what happens over three hours to any long-running agent, coding or not. Determinism at the input does not save you from degradation over time.

Retrieval belongs here too, in one sentence, because teams treat it as a search problem when it is a context allocation problem. Every retrieved chunk is budget spent, and a chunk that is topically relevant but not decision-relevant is a pure loss — it crowds the window and dilutes attention. That failure mode, and where RAG actually breaks, is retrieval versus generation failure.

BENCH-1: Unbounded Context vs. a Budgeted Ladder

BENCH-1 runs a 40-file TypeScript service with seven db.queryRaw() call sites to migrate to the parameterized db.query(), twenty times, scored on input tokens, output tokens, wall clock, and pass rate.

Two configurations. The first appends every tool result verbatim and lets the window fill. The second caps tool output at the handler, clears superseded results, and hands off to a fresh context with the artifact above when a threshold trips. The comparison exists to test one claim: that the budgeted configuration wins on pass rate, not just on tokens. If context engineering were only a cost optimization, unbounded context would still be more correct, just dearer. The prediction is that it is less correct, because degradation and premature wrap-up cost you call sites six and seven.

FAQ

What is context engineering for AI agents?

Context engineering is deciding what occupies a model's context window at each step of a run, and what gets evicted to make room. For agents it covers four decisions: the budget split across line items, where to trim, when to compact, and what to persist outside the window so it survives the session.

Why do agents get worse in long sessions?

Because occupied context degrades quality before it exhausts capacity. Recall of early instructions softens, constraints set at turn five stop binding by turn fifty, and the agent may wrap up prematurely as the window fills. Nothing errors. A long-session agent behaves like one that never read the first hour.

What is context rot?

Context rot is the observed decline in an agent's recall and instruction-following as its occupied window grows — distinct from truncation, which is a hard limit you hit. It is an empirical property practitioners measure per workload, not a documented threshold. Treat the usable budget as well below the advertised window.

Is compaction better than starting a new session?

Compaction preserves conversational continuity; a new session preserves nothing unless you give it a handoff artifact. Compaction also invalidates your prompt cache and loses detail unpredictably. Prefer trimming and clearing first, compaction when continuity genuinely matters, and a fresh context plus artifact for anything long-horizon.

How much of the context window should you actually use?

Measure it rather than adopt a number. Run your task at rising occupancy and find where pass rate drops — that inflection is your budget, and it is workload-specific. Until you have that figure, size the ladder to keep working context modest and push everything else to disk.

Take This Further

Instrument first. Run one real agent task with the token accounting helper, print the six-line split, and find out which line item owns your window — the answer is usually tool output, and usually by more than you expect. Then fix rung (a) in the handler that produces it, before you touch a compaction flag. When you get to pricing any of this per unit of verified work, the accounting frame is in AI cost accountability.