Stateless behind a load balancer

Once your server is a URL, you will run several copies behind a load balancer. None of them remembers anything between requests. So how does a replica know who is calling, or finish a deploy another replica started?

By the end of this lesson you can

say what “stateless” means for the protocol and what it does not mean for your app;
place each kind of state (identity, data, half-done calls, subscriptions, keys) where it belongs;
finish a call on a different replica using a signed ticket, and break it three ways;
read the SDK code and the lab output that prove it.

What stateless means

The protocol keeps no session. Your application can, and usually does, keep state: it just does not live in the memory of the process that answered the last request. Routing without remembering the passengers, like levers in a signal box.

Where state goes

Who is calling arrives with every request as Authorization: Bearer …. The replica validates it and hands the principal to the handler.
What they own lives in your database or cache, keyed by that principal. Any replica can read it.
What is half-done travels with the client as a signed requestState ticket.
The retry can land on another replica. B verifies the ticket with the same key and finishes.
Who wants change notifications is the one exception: a subscriptions/listen stream is held open on one replica.
So a change handled on replica B must reach the listener on A through a shared bus. Without one, the event is lost.

The TypeScript SDK makes this concrete: createMcpHandler(factory) calls your factory once per HTTP request and throws the instance away afterwards. There is nowhere to hide a session.

Run the load balancer

Three identical replicas, no shared memory, one shared signing key. Pick a user, send calls, and watch which replica answers. Start a deploy on one replica and confirm it on another. Then try to break it. The signatures here are real HMAC-SHA256, computed in your browser.

Caller Balancing

Request

Send a call.

Response

Ticket the client holds (requestState)

none

Things to try

Call whoami a few times as alice and as bob. A different replica answers each time, and every one knows who you are, from the header alone.
Start deploy, then Confirm. Round robin sends the confirmation to a different replica, and it still completes.
Start a deploy as bob, switch the caller to alice, confirm. The ticket is bound to bob, so it is refused.
Edit the ticket: it changes staging to prod inside the payload. The payload is readable (signed, not encrypted), but the signature no longer matches.
Give replica C its own key. Tickets minted on A or B now fail there: every replica must share the secret.

Where state goes, as a table

State	Lives in	Fails how, if you get it wrong
Identity	Bearer token on each request, validated each time	Trusting a header the gateway did not check
Application data	Your database or cache	Lost on restart, different on each replica
A half-finished call	Signed `requestState`, carried by the client	Unsigned state lets a client rewrite your logic
Change subscriptions	Open stream on one replica plus a shared event bus	Events published on replica A never reach a listener on B
Signing key	Secret store, same value on every replica	Tickets fail on whichever replica has another key

Common misconception

“Stateless means my server can't remember anything.” It means the protocol has no session. Your carts, documents and preferences live in a database like any web app; the protocol just never asks a replica to hold them in memory.

The code behind it

The lab server behind this simulator, trimmed. One factory serves both transports; identity comes in through req.auth.

// http.ts: run N copies behind any balancer; they share nothing but the key
const handler = createMcpHandler(createServer, { legacy: 'stateless' });
const mcp = toNodeHandler(handler);

http.createServer(async (req, res) => {
  const token = /^Bearer (.+)$/.exec(req.headers.authorization ?? '')?.[1];
  const who = token && verify(token);          // JWT check or introspection; a stub in the lab
  if (!who) return res.writeHead(401, { 'WWW-Authenticate': 'Bearer …' }).end();
  req.auth = { token, clientId: who.sub, scopes: who.scopes };   // → ctx.http.authInfo
  await mcp(req, res);
}).listen(3000);

// createServer.ts: the same key on every replica, state bound to the caller
const codec = createRequestStateCodec({
  key: process.env.REQUEST_STATE_KEY,             // ≥ 32 bytes, shared
  ttlSeconds: 120,
  bind: ctx => `${ctx.mcpReq.method}\0${ctx.http?.authInfo?.clientId}`,
});

And the real run of the lab's two-replica script, trimmed:

$ cd mcp/02-server && ./stateless-demo.sh
1) no token -> HTTP 401
2) alice on replica A: {"principal":"alice","scopes":["deploy:read"],"replica":"A"}
3) bob   on replica B: {"principal":"bob","scopes":["deploy:read","deploy:write"],"replica":"B"}
4) bob deploy on A (round 1) -> {"resultType":"input_required",…}
5) bob retry on B (different replica, no shared memory) -> "deployed to prod"
6) alice replays bob's requestState on B -> {"code":-32602,"message":"Invalid or expired requestState"}

Quick check

Without sticky sessions, how does replica C know the caller is bob?

Replica A told it over a shared cache
The request carries bob's bearer token, which C validates
From clientInfo in _meta

Identity arrives with every request. clientInfo is self-reported by the client software and never used for security.

Tickets minted on A fail on C, but work on B. What is the most likely cause?

C is running an older protocol version
C has a different signing key
The ticket expired

Every replica must share the same key from a secret store; otherwise signatures computed on one fail on another.

Which kind of state still needs something shared between replicas beyond the database?

Identity
Half-finished calls
Change subscriptions held on subscriptions/listen streams

A listen stream lives on one replica. Events that happen elsewhere must reach it through a shared bus.

Recap

Stateless protocol, stateful app: nothing lives in one replica's memory between requests.
Identity: bearer token per request. Data: database keyed by principal. Half-done: signed requestState. Subscriptions: stream plus shared bus. Key: the same on every replica.
createMcpHandler builds a fresh server per request, so there is nowhere to hide a session.