The context bill

Every tool a host connects is described to the model in words and JSON Schema, and that description rides along on every single turn. Ten servers in, it adds up.

By the end of this lesson you can

  • explain why tool definitions cost context on every turn, and why prompt caching does not shrink them;
  • estimate your own setup's bill, and know how far measured servers differ;
  • describe tool search and code execution, and what each one costs you in return;
  • say why a skill is cheap until it is used.

Definitions ride along every turn

A model has no memory between calls. To let it pick a tool, the host pastes every tool's name, description and input schema into the prompt, every time it calls the model. Nothing about MCP makes them smaller: the protocol just delivers them from tools/list.

One context window, turn by turn (illustrative proportions)

Context window, 200k tokens system tool definitions 5 servers × 20 tools turn 1 turn 2 room left for the work model call 1 = system + definitions + turn 1the user has said one sentence model call 2 = system + definitions (again) + turns 1–2the definitions are re-read on every call prompt caching: repeats are billed at a discountbut they take exactly the same room in the window With tool search search + 3 turn 1 turn 2 ≈85% fewer definition tokens
  1. A 200k-token window. The host's own system prompt takes a little.
  2. Five connected servers with twenty tools each. Their definitions go in before the user has said anything.
  3. Turn 1: the user's message and the model's reply. The model call carried the system prompt, every definition, and the turn.
  4. Turn 2: the model is stateless, so the host sends everything again, definitions included. They are a fixed tax on every call.
  5. Prompt caching lets the provider bill repeated prefixes at a discount. It changes the bill, not how much of the window they occupy.
  6. With tool search, most definitions are deferred: only a search tool and the few schemas actually needed are loaded.

Common misconception

"Prompt caching makes tool definitions free." Caching changes what you are billed, not how much of the window they occupy. A model with 55,000 tokens of schemas in front of it has 55,000 fewer tokens for your code, your documents and its own reasoning, and more text competing for its attention when it picks a tool.

Work out your bill

Move the sliders, or load one of the measured setups. The bars show the share of the context window used before the conversation even starts.

All definitions loaded

tokens, on every turn

With tool search

about 85% less (vendor-reported)

As skills instead

about 100 tokens of metadata each; the body loads on use

The presets are measurements from practitioners: the GitHub server at about 55,000 tokens for 93 tools; a minimal Postgres server at about 35 tokens, a spread of roughly 1,500 times between servers. A setup of several servers commonly reaches 15,000 to 20,000 tokens per message. These are community measurements, not a benchmark: measure your own (count the tokens of your host's tool list with your provider's token counter).

Two ways to pay less

Same question, three shapes of conversation

Modelits context Servers93 tools 93 definitions in the prompt≈55,000 tokens before the question the whole result comes back10,000 rows read by the model tool search: 1 search tool loaded93 definitions marked defer_loading search "issues" → 3 schemas loadthen a normal call (one extra round trip) Sandboxruns the model'sscript servers appear as code to importthe model writes a short script 10,000 rows stay in the sandboxonly a 20-line summary returns
  1. Direct calling: all 93 definitions are in the prompt on every turn, about 55,000 tokens before the user's question.
  2. The model calls a tool, and the whole result, say 10,000 rows, comes back into its context to be read and filtered.
  3. Tool search: definitions are marked defer_loading. The model starts with one search tool.
  4. It searches, the matching three schemas load, and it calls one. Cheaper, at the price of one more round trip and the chance the search misses.
  5. Code execution: servers are presented as code the agent can import. The model writes a short script, which runs in a sandbox.
  6. The script calls the tools and filters the 10,000 rows inside the sandbox. Only its small output enters the model's context.

Tool search

Mark definitions defer_loading and give the model one search tool. It searches, loads only the matching schemas, then calls. Reported reduction: about 85% of definition tokens. Claude Code turns it on automatically when tool definitions would take more than 10% of the context window.

Code execution

Present servers as code the agent imports, and let it run a script that calls tools and filters results in a sandbox. Anthropic's example fell from 150,000 to 2,000 tokens. One independent benchmark saw input fall about 78% while output rose about 120%: the model now writes code, and code is output.

Key idea

Neither is free. Search adds a round trip and can miss the tool it needed. Code execution needs a sandbox, resource limits and monitoring, and spends output tokens to save input tokens. For a handful of small tools, direct calling is still the simplest and the most reliable.

Why a skill is cheap until used

A skill is a folder with a SKILL.md. The agent sees only its name and a one-line description, about 100 tokens, until the task needs it; then it reads the body. That is progressive disclosure, and it is the same idea tool search applies to MCP.

The comparison of servers, skills and command-line tools, and when to pick which, is the subject of the next lesson.

Sort the fixes

Which fix fits each situation best?

  • Four small tools from one internal server. A few hundred tokens of definitions: nothing worth optimising, and no extra round trip or sandbox.
  • Twelve servers and 300 tools, but each task uses two or three of them. Most definitions are dead weight on any given turn; defer them and let the model search.
  • Join two large exports and report the five rows that differ. The cost is in the results, not the definitions. Filtering in a sandbox keeps the big data out of the context.
  • Claude Code with many servers whose tools exceed 10% of the window. This is the threshold at which Claude Code turns tool search on by itself.

Quick check

Your provider caches the tool definitions. What happens to the context window?

  • Cached definitions no longer count against it
  • Nothing: they take the same room; only the bill for repeats is lower
  • The window grows by the size of the cache
Caching is a billing and latency optimisation. The model still reads every token of every definition on every call.

Why does code execution raise output tokens?

  • Because the sandbox's logs are sent to the model
  • Because the model now writes a script, and everything the model writes is output
  • Because results are returned twice
Input falls because results stay in the sandbox; output rises because the model writes code. One benchmark: about −78% input, +120% output.

Two servers are both "measured". One is 55,000 tokens, the other 35. What should you conclude about your own server?

  • It will be about the average of the two
  • MCP servers cost about 1,500 tokens each
  • Nothing until you measure it: the spread is about 1,500 times
Cost depends on how many tools and how verbose their descriptions and schemas are. Count your own.

Recap