Context Is a Budget

A year ago the running joke was that prompt engineering would be dead by Christmas. This year the AI Engineer World's Fair gave context engineering a track of its own, and the room was full.

Prompt engineering sounded like phrasing. Context engineering is a systems problem: deciding what an agent gets to see, from where, at what cost, with what freshness. The mental model that has served me best comes from somewhere unfashionable. Frontend performance budgets.

Frontend teams learned years ago that a page doesn't get slow in one commit. It gets slow three kilobytes at a time, each one individually justifiable, until someone finally sets a performance budget and makes size a reviewable number. Context windows are on the same curve right now. Context is a budget, and most teams are spending it like it's free.

A token in the context window costs three ways: money, latency, and attention. The first two show up on invoices and traces. The third is the one teams miss. A model reads everything you send, including the junk, and the junk competes with the signal. Retrieval results that were almost relevant, boilerplate headers, the full JSON of an API response when the workflow needed four fields. The model doesn't get to skim. Whatever you put in the window, you have asked it to weigh.

The attention cost has been measured. The Lost in the Middle results showed models retrieving facts reliably from the edges of a long context and missing the same facts parked in the middle, and that was on clean inputs. Filler does not sit in the window neutrally. It buys degradation.

The question worth asking is what the workflow actually needs, stated as an allowlist, with everything else excluded by default.

ResearchLog classifies development activity against the IRS four-part test, which means its prompts carry metadata about pull requests. The tempting version is to serialize what the GitHub API returns and let the model sort it out. The API returns a lot: full PR bodies, and nested base and head payloads that carry entire repository objects along for the ride.

None of that survives into the prompt. Prompt metadata is built the way the export path builds its evidence files: from an explicit allowlist, never by spreading a raw object and trimming afterward. Titles, states, the fields the classification actually reasons about. The PR body is capped. The nested payloads never enter.

The allowlist matters for a second reason that has nothing to do with cost. Whatever enters the prompt becomes model input forever, in your logs and in every eval and replay that follows. A bounded prompt is easier to reason about the same way a bounded API response is. You can read one and say what the model knew.

Prompt caching is the line item everyone claims and few verify. ResearchLog sets cache_control on its system block, exactly the way the docs suggest, and it saves nothing. Caching covers the prefix up to the breakpoint, so putting it on the system block caches the tool schema plus the system prompt: a little under 400 tokens against a minimum cacheable prefix length of 1,024. Nothing errors. The flag is set, the prefix is too short, and both cache counters stay at zero.

That is an embarrassing paragraph to write and a cheap lesson to learn. The billing surface has its own semantics, minimum cacheable sizes and prefix stability rules, and the only ground truth is usage metrics on real traffic. A comment in that file tells the next reader to check cache_read_input_tokens before claiming any savings. Nothing in the codebase logs it. The instruction to measure is not the measurement.

Cost work in AI systems is observability work. If a team claims a caching win and cannot point at cache_read_input_tokens moving, they are describing an intention. The frontend version of this lesson took years: minified is not the same as small, and a budget you don't measure is a wish.

The other place context leaks is instructions. The pattern I keep returning to, in the design-system agent work and in dibble, my open-source collection of portable agent skills, is to keep the skill thin and point it at living truth.

A skill that copies component API tables into itself spends budget twice. It pays tokens on every run to carry the copy, and it pays drift forever because the copy rots the moment the source moves. The thin version teaches the agent where the docs live, which rules are load-bearing, and which validator has the final word. Truth stays in one place. The window carries directions to the source instead of a stale snapshot of it.

The test I use for any instruction file: if this paragraph disappeared, would the agent's output change? If nobody can name the failure the paragraph prevents, it is dead weight the window pays for on every run.

A budget forces the design conversation teams were avoiding.

When a workflow's prompt wants to grow past its ceiling, that pressure is information. Either the workflow is doing too many jobs, or the retrieval is compensating for docs that should be better, or someone is shipping the whole database because choosing fields felt like work. The budget doesn't answer the question. It refuses to let the question go unasked.

Reduced to a sentence: decide what enters the window the way you decide what crosses an API boundary, and verify cost claims against usage data rather than configuration. The longer version:

• Build prompts from allowlists, never by trimming raw payloads. What you exclude by default cannot leak, bloat, or surprise you in a log.
• Cap every free-text field that enters a prompt. User-shaped and API-shaped text grows until something bounds it.
• Verify caching against usage metrics before claiming the savings. A cache_control flag with zero cache reads is a decoration.
• Wire the check into the system rather than into a comment. A note telling the next reader to verify usage metrics is not the same as logging them.
• Keep instruction files thin and pointed at live sources. A copied table is a second source of truth with a decay rate.
• Watch token spend per workflow the way you watch bundle size per route. Regressions are cheap to catch early and expensive to excavate later.
• Let the budget trigger design reviews. A prompt that keeps growing is a workflow asking to be split.

Every platform generation relearns the same economics: the resource that feels infinite in the demo becomes the constraint in production. Memory, bandwidth, bundle size, and now the context window. The teams that handle it well are the ones that made the spend visible and gave someone the job of saying no.

The work itself is unglamorous. Decide what each workflow needs, build the prompt from that list, and read the usage numbers rather than the configuration. That holds regardless of which model you are running this quarter.