The supporting cast: code mode, MCP, telemetry and evals

Learning goals. After this chapter you should be able to:

• explain why a model might write a script instead of calling tools one at a time, and how pi-codemode sandboxes that script;
• describe how QuickJS is interrupted, bounded and made deterministic;
• walk through an MCP session from initialize to tools/call, over both stdio and Streamable HTTP;
• explain the OAuth pieces an MCP client needs: discovery, PKCE, dynamic registration, refresh and step-up;
• design a telemetry API that cannot change program behaviour;
• read an eval report and explain pairing, counterbalancing, and why some results are withheld.

These four packages are small and independent. Each has no dependency on the rest of Pi (pi-mcp says so explicitly: "It does not depend on the official MCP SDK or other pi packages"). That makes them good case studies: each solves one well-defined problem completely.

18.1 Code mode: tools called from a program#

In a normal agent loop (Chapter 6), every tool call is a model turn. Reading 200 files costs 200 round trips, and all 200 results enter the context window. Code mode lets the model write one JavaScript program instead:

const paths = (await tools.find({ pattern: "*.test.ts" })).split("\n");
const sizes = await Promise.all(paths.map(async (path) => (await tools.read({ path })).length));
return sizes.reduce((a, b) => a + b, 0);

The README states the payoff: "Nested tool calls never enter the LLM context; only the script's output and return value do." The model uses a real programming language for loops, filtering and aggregation, which models are good at. Only the answer comes back.

The risk is obvious: the model is now running arbitrary code. pi-codemode's answer is a sandbox with exactly one capability:

Runs model-written JavaScript in a QuickJS VM (compiled to WebAssembly) where the only capability is calling injected tools.

The script can use tools.<name>(args), text(), image(), console.log, store()/load() and exit(), "Nothing else: no timers, fetch, process, require, modules, or WebAssembly." This is the object-capability model in its purest form. Authority comes only from references you were handed. Because QuickJS is a separate JavaScript engine compiled to WebAssembly, the script cannot reach the host's objects even through prototype tricks. It runs in a different heap, in a different engine, behind the WASM memory boundary. Values cross only as JSON strings:

// packages/codemode/src/runtime/protocol.ts
/**
 * Messages between the host (main thread) and the worker. Tool arguments,
 * results, and values cross as JSON strings: the worker passes them into and
 * out of the QuickJS VM as strings and never builds structured values itself.
 */

Serialising at the boundary is a deliberate copy-in/copy-out discipline. No reference can leak across, so no aliasing bug can turn into an escape.

Host and VM, separated. execution.ts keeps everything the script must not influence on the host side:

The host owns everything the script must not influence: the tool table, call records, output, the deadline, the caller's abort signal, and the decoding of what the VM reports.

The VM can run in three places with the same message protocol: a worker thread (CodemodeSandbox, Node and Bun), the calling thread (InlineCodemodeSandbox), or another isolate over RPC (RemoteCodemodeSandbox, Cloudflare). "The launchers differ in transport and termination, not in behavior." This is the strategy pattern applied to where computation happens.

18.2 Stopping a program that will not stop#

A sandbox must survive while (true) {}. The halting problem says you cannot decide in advance whether a script terminates, so you need a way to stop it from outside. pi-codemode has several layers.

Interrupt polling. QuickJS calls an interrupt handler periodically while it runs bytecode. Returning true throws an uncatchable error inside the script. The VM bridge adds one subtlety:

// packages/codemode/src/runtime/vm.ts
 * - `shouldInterrupt` is polled by QuickJS while the script runs, and here between batches of
 *   promise jobs. The job batches matter for a script such as `while (true) await
 *   null`: every iteration is a separate job that executes only a few opcodes, so QuickJS's own
 *   polling, which counts opcodes, would let it run for a long time between polls.

An infinite loop of microtasks is an infinite loop too, just spread over the job queue. The bridge therefore runs pending jobs in batches and polls between them.

Interrupt budgets where time stands still. On Cloudflare, Date.now() does not advance while code computes, so a wall-clock deadline cannot fire. The sandbox instead counts interrupt polls. DEFAULT_INTERRUPT_BUDGET is 100,000 polls, "about 2 seconds of a tight loop on an Apple M-series laptop and about 10 seconds on Cloudflare". This is fuel metering, the same idea as Ethereum gas or Wasmtime fuel. You cannot measure time, so you meter work.

Memory and stack limits. memoryLimitBytes caps the heap, and allocations beyond it fail as InternalError: out of memory inside the script. A stack guard turns deep recursion into a catchable RangeError, set below the host thread's real stack (512 KiB guard for a 4 MB worker stack) so the guard trips first.

Termination is a request, not a guarantee. The README is candid: "Aborts, timeouts, and close() request worker termination without waiting for the worker to exit." A long native built-in on Bun may not poll the interrupt handler. Hosts that need hard concurrency limits get onWorkerStarted(exited), which resolves when the worker actually exits, and are told to "Keep a concurrency slot occupied until exited resolves". This distinguishes logical completion (the result settled) from resource release (the thread is gone). Admission control must count the second.

Detecting a stuck await. "A script that waits on a promise nothing can settle (no tool call pending, and the VM has no timers or I/O) fails right away instead of hanging." Inside the sandbox, a promise can only be settled by a tool result. If the job queue is empty and no tool call is outstanding, nothing can ever happen, so waiting for the deadline would be pointless. This is deadlock detection made trivial by a closed world.

18.3 Determinism for replay#

Chapter 15 showed Durable restarting a code-mode script after a crash by re-running it and feeding it the same tool results in the same order. That only works if the script makes the same calls with the same arguments every time. vm.ts lists every source of non-determinism and removes it:

// packages/codemode/src/runtime/vm.ts
 * - Time: every WASI clock reads virtual time, `startedAt` and then the `now` of the result being processed, never going backwards.
 *   While the context is created, the realtime clock reads a value derived from the seed instead, because QuickJS
 *   seeds `Math.random()` from the first clock read, and which read comes first is the engine's business.
 * - Randomness: `random_get` draws from a PRNG over the seed.
 * - Timezone: UTC, not the host's.
 * - Garbage collection: `WeakRef` and `FinalizationRegistry` are not available; collection timing is not.
 * - Delivery order: results carry a `seq` and are applied strictly in that order, buffering early ones, because a
 *   transport (RPC replies) may deliver them in another order than the host sent them.

Each line is worth studying:

• Time becomes a logical clock. It advances only when a tool result is delivered, to the timestamp recorded with that result. Replaying the same results replays the same times.
• Randomness comes from a seeded PRNG behind WASI's random_get. QuickJS seeds Math.random() from the first clock read, so even the clock during context creation is derived from the seed.
• GC timing is the subtle one. WeakRef lets a program observe when an object was collected, and that depends on heap pressure, which differs between runs. Removing WeakRef removes the observation.
• Ordering: results are numbered and applied in sequence, with early arrivals buffered. This is the classic fix for a reordering transport, exactly what TCP does with sequence numbers.

The VM reports a fingerprint at start and an idle message with the count of calls issued after each drain. The host can then check the replay step by step. If the script asks for a different call than the journal recorded, the replay has diverged and is refused, rather than producing a subtly different history.

This is the same discipline as deterministic simulation testing (FoundationDB) and event-sourced workflow engines (Temporal). Any source of non-determinism must be either removed or recorded.

18.4 Telling the model what it can call#

The model needs to know each tool's signature. renderDeclarations() turns JSON Schemas into TypeScript declarations, because models read TypeScript fluently:

// from packages/codemode/README.md
renderDeclarations({ tools: sandbox.tools, globals: sandbox.globals });
// declare const tools: {
//   /** Read a file */
//   read(args: {
//     path: string;
//   }): Promise<string>;
// };

This is a small compiler from one type language to another. Constraints TypeScript cannot express become comment tags (// @integer @minimum 1 @default 20), and these are "dropped first when a type exceeds the size limit": a graceful loss of precision under a budget. Recursive $refs render as unknown, since an infinite type cannot be printed.

The script's optional first line is a small configuration language:

// @options: {"max_output_tokens": 2000, "timeout_ms": 30000}

The options line "is replaced by an empty line, so line numbers in stack traces still match the input". This is a detail compiler writers know well: source positions must survive preprocessing. The package also exports CODEMODE_SOURCE_GRAMMAR, a Lark grammar, so providers with grammar-constrained decoding (Chapter 5) can force the model's output to be a valid script.

The store. store/load keep JSON between executions, but the sandbox persists nothing itself. A successful result reports storeWrites: { set, delete }; "Failed executions report no writes". This makes each execution atomic with respect to the store: the caller applies all writes or none. Sizes are bounded (256 Ki characters per value, 1 Mi in total), as every resource a model controls should be.

Placement on Cloudflare. The README records a measurement worth remembering. QuickJS runs synchronously, so a computing script blocks its thread. Running it inside the session's Durable Object delayed every other request by the full ten seconds of computation. Running it in a separately deployed Worker cut the delay to about 50 ms. The lesson is general: CPU-bound work on an event loop must be moved off it, whether to a worker thread, a separate process or another machine.

18.5 MCP: the Model Context Protocol#

MCP is a standard way for an agent to discover and call tools served by another program: a filesystem server, GitHub, a database. pi-mcp is a client written from scratch. Its README draws one line through the design:

An MCP transport owns framing and I/O. It delivers individual JSON-RPC messages to McpClient; the client owns request correlation, initialization, timeouts, cancellation, server requests, and protocol-level helpers.

That is layering: a transport moves messages, a client gives them meaning. Swapping stdio for HTTP changes nothing above the transport.

JSON-RPC 2.0. MCP messages are JSON-RPC: requests (with id), notifications (without id, never answered) and responses (result or error):

// packages/mcp/src/protocol/jsonrpc.ts
export type JsonRpcMessage = JsonRpcRequest | JsonRpcNotification | JsonRpcResponse;

export const JSON_RPC_ERROR_CODES = {
	parseError: -32700,
	invalidRequest: -32600,
	methodNotFound: -32601,
	invalidParams: -32602,
	internalError: -32603,
} as const;

Both sides can send requests. The server may ask the client for roots/list or ping, so the client is also a small server. This is bidirectional RPC over one connection, the same correlation-by-id pattern as pi-env's frames (Chapter 16).

The session lifecycle.

1. initialize with the client's protocolVersion (latest 2025-11-25). The server picks a version; the client accepts only those it supports (2025-06-18, 2025-03-26, 2024-11-05), otherwise it fails.
2. notifications/initialized tells the server the handshake is done.
3. tools/list, paginated with a cursor, then tools/call.

Version negotiation in the handshake is how long-lived protocols evolve without breaking old peers.

Timeouts that progress can renew. A long tool call (say, a test suite) may legitimately take minutes. A fixed timeout would kill it; no timeout would hang forever on a dead server. pi-mcp attaches a progressToken to requests that want progress, and every notifications/progress for that token re-arms the timer:

// packages/mcp/src/client.ts
private handleProgress(params: unknown): void {
	...
	const entry = requestId === undefined ? undefined : this.pending.get(requestId);
	if (requestId === undefined || !entry) return;
	this.armTimeout(requestId, entry);

This is a liveness-based timeout: what times out is silence, not total duration. The same idea is behind TCP keepalives and pi-env's 30-second silence limit.

Cancellation sends notifications/cancelled with the request id. The spec forbids cancelling initialize, and the client honours that (cancellable: method !== "initialize").

Stdio transport. Each message is one line of JSON (${JSON.stringify(message)}\n), the newline framing from Chapter 7. Shutdown follows the spec's escalation: "close stdin, then SIGTERM, then SIGKILL", applied to the server's whole process group. Closing stdin is the polite request; signals are the impolite ones. Escalating with grace periods gives well-behaved servers a chance to clean up while guaranteeing that misbehaving ones die.

Streamable HTTP transport. Each client message is an HTTP POST with accept: application/json, text/event-stream. The server may answer with one JSON body or open a Server-Sent Events stream (Chapter 4) for a long response. The interesting parts are about failure:

• Sessions. The server assigns an Mcp-Session-Id header and the client echoes it on every request. A 404 while holding a session id means "MCP session expired", and the client must re-initialise.
• Resumption. If an SSE response stream drops and the server assigned event ids, the client reopens it with GET and Last-Event-ID, and the server replays what was missed. This is at-least-once delivery with a cursor, the same idea as resuming a Kafka consumer from an offset.
• Backoff. Reconnects wait initial * 2 ** attempt, capped at maxDelayMs (default 30 s), unless the server sent an SSE retry field, which wins. The attempt counter resets once data is received, so one hiccup after hours of uptime does not start at the maximum delay.

// packages/mcp/src/transports/streamable-http.ts
private reconnectDelay(attempt: number, serverDelayMs: number | undefined): number {
	if (serverDelayMs !== undefined) return serverDelayMs;
	const initial = this.options.reconnect?.initialDelayMs ?? DEFAULT_RECONNECT_INITIAL_DELAY_MS;
	return Math.min(initial * 2 ** attempt, this.maxDelay());
}

OAuth for remote servers. @earendil-works/pi-mcp/oauth implements the client side of MCP authorization. The pieces fit together as follows:

• Discovery. A 401 response carries a WWW-Authenticate header naming the resource_metadata URL. Otherwise the client tries /.well-known/oauth-protected-resource. That document names the authorization server, whose metadata comes from /.well-known/oauth-authorization-server or /.well-known/openid-configuration. The client hard-codes no URLs; it follows links.
• PKCE (S256). Chapter 5 explained PKCE for pi-ai's logins. Here the client refuses servers that do not support it: "Authorization server does not support PKCE S256".
• Dynamic client registration. The client registers itself with the server at runtime, because an MCP client cannot be pre-registered with every server in the world.
• Single-flight refresh. "one refresh shared by concurrent 401s". If ten requests fail with an expired token at once, one refresh runs and all ten wait for it. Without this, ten refreshes race and, with rotating refresh tokens, all but one fail. Chapter 5's CredentialStore.modify solved the same problem with a lock.
• Step-up authorization. A 403 with error="insufficient_scope" triggers a new authorization for the broader scope, instead of failing the call.

MCP results for a model. toLlmContent(result) converts MCP content blocks to pi-ai's TextContent and ImageContent. The README's adapter shows two practical constraints. Tool names are sanitised ("Providers allow at most 64 characters of [A-Za-z0-9_-]"), and schemas are forced to type: "object" with properties, because some providers reject anything else. Note also: "MCP reports tool failures in the result instead of as a protocol error." A tool that ran and failed (isError: true) is different from a protocol failure. The first should be shown to the model so it can react; the second is a bug in the connection.

18.6 Telemetry that cannot hurt you#

pi-telemetry defines how Pi packages describe what they did: spans (timed operations in a tree), attributes, events and a status. It ships no exporter and depends on no backend; applications plug in OpenTelemetry, Sentry or logs through an adapter.

The central design choice is that a span has no end():

// from packages/telemetry/README.md
return telemetryContext.startSpan({ name: 'example.parent' }, async (parentSpan) => {
  return parentSpan.startSpan({ name: 'example.child' }, async (childSpan) => {
    childSpan.addEvent('example.cache.lookup', { 'example.cache.hit': true });
    return performWork();
  });
});

There is no public end() method. startSpan() owns settlement and keeps the span open until the callback's value or promise settles.

This is the bracket pattern (Haskell's bracket, Python's with, C#'s using). The resource's lifetime is the callback's lifetime, so forgetting to end a span, or ending it twice, is impossible by construction. Parents are passed explicitly. As with Chord's Context (Chapter 17), there is no global "current span": "pi code always propagates the parent through TelemetryContext arguments".

The adapter contract is mostly about one property: observation must not change the observed.

• "make recording methods synchronous, passive, and non-throwing";
• "ignore a failed recording call atomically, suppress backend failures, and still execute the business callback exactly once";
• "preserve the callback's returned value and rejection value".

A telemetry backend that is down, slow or buggy must never make an agent fail. The README states it directly: "A span is diagnostic data, not business state."

NOOP_TELEMETRY_CONTEXT is the null object pattern. Instead of if (telemetry) ... at every call site, code always calls a context; the default does nothing. Its span is one shared frozen object, even for nested spans, so the no-op path allocates nothing:

// packages/telemetry/src/noop.ts
const noopTelemetrySpan: TelemetrySpan = {
	startSpan: startNoopSpan,
	addEvent: () => {},
	setAttributes: () => {},
	setStatus: () => {},
};
Object.freeze(noopTelemetrySpan);

The package also ships an in-memory reference adapter that records "a deterministic numeric ID ... and deterministic end sequence. It records no timestamps." Deterministic output makes tests exact. Finally, a conformance suite lets any adapter prove it follows the contract, the same technique as Durable's env and storage conformance suites (Chapters 14 and 16).

18.7 Evals: measuring an agent honestly#

How do you know a change made the agent better? Unit tests check code; evals check behaviour. packages/evals asks a concrete question: does the coding agent's documentation help it do tasks? It runs each task twice, without_docs and with_docs, and reports the lift, the difference in pass rates.

The design is a small lesson in experimental method.

Pairing. Each case runs in both arms with the same model, and results are compared within pairs. That controls for case difficulty: a hard case lowers both arms equally.

Counterbalancing. Order effects (warm caches, rate limits, time of day) could favour whichever arm runs first. The planner alternates:

// packages/evals/src/plan.ts
const variants: DocumentationVariant[] =
	runNumber % 2 === 1 ? ["without_docs", "with_docs"] : ["with_docs", "without_docs"];

Pre-registration. The runner "Plans every (case, variant, model, runNumber) arm before execution" and writes expected-runs.json and a protocol.json with a digest before any run. A missing arm then shows up as missing, and nobody can quietly drop an inconvenient run.

Isolation. Every arm runs in a fresh container with a new home directory, workspace and session directory. Both images install the same locally packed workspace tarballs, so the only difference between arms is the documentation. Documentation from internal dependency packages is removed symmetrically, "so they cannot act as alternate instructions". That is controlling confounders.

Anti-cheating. "Eval definitions, evaluator helpers, fixtures, and Vitest configuration are root-owned and unreadable after the harness permanently drops to an unprivileged UID." An agent that could read the grader could game it. Models do this under optimisation pressure, a failure known as reward hacking or specification gaming. Doc evals also allow only read, write, edit, grep, find and ls. The README is honest about the remaining gap: "Docker alone cannot prove that arbitrary code written by an agent never uses the network."

Withholding rather than guessing. "A pair contributes to pass-rate lift only when both arms produce exactly one score." If any pair is blocked (missing, duplicate, errored), "headline pass rates are withheld" and the process exits non-zero. "Missing telemetry remains unavailable rather than being treated as zero." Treating a missing value as zero is one of the most common ways data analyses go wrong.

Flags instead of false confidence. The report marks no-lift, negative-delta, control-saturated, treatment-saturated and flaky. A saturated arm (100% pass) cannot show improvement: the eval is too easy to measure anything. flaky marks cases whose outcome varied across repetitions. And: "One repetition cannot establish stability." LLM outputs are stochastic, so a single run is one sample from a distribution.

Concepts in this chapter#

• Object capabilities and sandboxing. Authority only through injected references; a separate engine behind a WASM boundary.
• Copy-in/copy-out boundaries. Serialise everything that crosses a trust boundary.
• Fuel metering. Bound computation by counting work when time cannot be measured.
• Logical vs physical completion. A result can settle before its thread exits; admission control must count the thread.
• Deterministic execution. Virtual clocks, seeded randomness, fixed timezone, hidden GC, ordered delivery.
• Divergence detection. Compare a replay to its journal step by step and refuse on mismatch.
• Schema-to-type compilation and constrained decoding.
• Atomic side-effect reporting. Report writes on success only; let the caller apply them.
• JSON-RPC. Requests, notifications, responses; id correlation in both directions.
• Version negotiation in a handshake.
• Liveness-based timeouts. Time out on silence, renewable by progress.
• Graceful-then-forceful shutdown. Close input, SIGTERM, SIGKILL, each after a grace period.
• Resumable streams. Event ids and Last-Event-ID; exponential backoff with a cap and server override.
• OAuth discovery, PKCE, dynamic registration, single-flight refresh, step-up.
• Bracket pattern and null object pattern.
• Passive observability. Recording must never change outcomes.
• Experimental design. Pairing, counterbalancing, pre-registration, confounder control, saturation, flakiness, honest handling of missing data.
• Reward hacking. Keep graders out of the agent's reach.

Exercises#

1. Escape attempts. List five things a malicious script might try inside pi-codemode (for example reading process.env, fetch, import(), prototype pollution of host objects, an infinite await loop). For each, name the mechanism that stops it.
2. Fuel. Write a tiny interpreter for arithmetic expressions with loops, and add a fuel counter that decrements per operation. Compare the behaviour of a fuel limit with a wall-clock timeout when the machine is under heavy load.
3. Determinism audit. Which of these break deterministic replay, and how would you fix each: Date.now(), Math.random(), new Date().getTimezoneOffset(), Object.keys order, WeakRef.deref(), Promise.race between two tool calls?
4. JSON-RPC by hand. Write the exact JSON messages for: initialize, the server's response, notifications/initialized, tools/list, a tools/call with a progress token, two progress notifications, and the final response. Mark which messages have ids.
5. Single flight. Implement singleFlight(fn), which returns a function that shares one in-flight promise among concurrent callers and starts a new one after it settles. Use it to wrap a token refresh, and test it with ten concurrent callers.
6. Backoff. Plot reconnect delays for attempts 0–10 with an initial delay of 1 s and a cap of 30 s. Add ±50% jitter (Chapter 4) and explain what problem jitter solves that the cap does not.
7. Bracket. Design a withSpan API for a language without closures, such as C. What does the bracket pattern cost you, and what goes wrong if users must call end() themselves?
8. Eval design. You want to know whether a new system prompt improves the agent. Design the experiment: arms, pairing, repetitions, order, what to pre-register, what to report when one arm crashes, and how to tell "no effect" from "eval saturated".