The durable harness: tasks, scheduler, generations and tools

In pi-durable all work is done by tasks: durable state machines whose every step is a commit. A scheduler reserves runnable tasks and runs their phase handlers; tasks own other tasks and conversations, forming a tree in which aborts flow down and completion waits for children; and the agent loop is rebuilt from three built-in task types — pi.generation, pi.tool and pi.compaction. This chapter explains the task abstraction, the "effect sandwich" that makes side effects recoverable, structured concurrency, the generation and tool tasks, the inbox, compaction, observation, and extensions.

Learning goals. Write a task definition with phases and an abort handler; explain why the effect sandwich makes crashes survivable and where idempotency is still required; explain structured concurrency (ownership, waiting, bottom-up abort, completing); trace one answered input through generation and tool tasks; explain tool replay policies and code mode's deterministic replay; describe how UIs observe committed state.

15.1 A task is a state machine with checkpoints#

A task definition (spec 5.1) names its phases and supplies an abort handler:

type TaskDefinition<I, S extends { phase: string }, R, H extends object> = {
  readonly name: string;
  readonly version: number;
  initial(input: I): S;
  readonly phases: {
    [P in S["phase"]]: PhaseHandler<I, Extract<S, { phase: P }>, S, R, H>;
  };
  abort(task: RunningTask<I, S, R>, runtime: TaskRuntime<I, S, R, H>, context: Context): Promise<void>;
  migrate?(input: JsonValue, checkpoint: JsonValue, fromVersion: number): { input: I; checkpoint: S };
  readonly hooks?: H;
};

The mapped type [P in S["phase"]] makes the phase map exhaustive: if the checkpoint union has a phase "charge", the compiler demands a charge handler, and that handler receives a task whose checkpoint is narrowed to the charge variant. version and migrate let a stored task created by an older version of the code be upgraded when it resumes — durable state outlives deployments, so its schema must evolve.

The task record's state is itself a state machine:

type TaskState<S, R> =
  | { readonly status: "pending"; readonly checkpoint: S }
  | { readonly status: "running"; readonly checkpoint: S }
  | { readonly status: "waiting"; readonly checkpoint: S; readonly on: readonly TaskId[]; readonly policy: JoinPolicy }
  | { readonly status: "completing"; readonly outcome: TaskOutcome<R> }
  | { readonly status: "terminal"; readonly outcome: TaskOutcome<R> };

and the outcome distinguishes completed, failed, aborted, orphaned (its definition is no longer installed) and faulted (its code threw). A handler advances the task only through runtime.commit((tx, current) => nextState): the new checkpoint is written atomically with whatever entries, documents and child tasks the callback creates. "A task changes only its own state, through its runtime."

task lifecycle · durable status transitions

A reserved task runs phase handlers, may park in waiting on other tasks, and holds in completing while work it owns is still live. After a restart, running tasks return to pending and are reserved again.

15.2 The effect sandwich#

The central recovery idea is three lines in the spec (5.2):

commit intent phase
perform external effect
commit outcome or next phase

"Reopening in an intent phase means the effect may have happened. The phase handler retries safely, polls an external handle, or records interruption." The spec's payment example shows all three parts:

// Intent, effect, outcome.
prepare: async (task, runtime, context) => {
  await runtime.commit(() => ({ status: "running", checkpoint: { phase: "charge", key: newKey() } }), context);
},
charge: async (task, runtime, context) => {
  const receipt = await payments.charge(task.state.checkpoint.key); // idempotent by key
  await runtime.commit(async (tx, current) => {
    const entry = await tx.appendEntry(current.conversationId, receiptEntry(receipt));
    return { status: "terminal", outcome: { status: "completed", result: { entryId: entry.id } } };
  }, context);
},
// The abort handler decides the outcome; returning without one faults the task.
abort: async (task, runtime, context) => {
  await payments.cancel(task.state.checkpoint);
  await runtime.commit(() => ({ status: "terminal", outcome: { status: "aborted", reason: "user" } }), context);
},

The idempotency key is generated in prepare and committed before the charge. If the process dies during charge, the task resumes in charge with the same key, and the payment provider deduplicates. Had the key been generated inside charge, a retry would create a second key and a second charge. The general rule: no durable system can make an external effect exactly-once by itself; it can only guarantee at-least-once execution plus a stable identity that the external system uses to deduplicate. This is the same conclusion as the "two generals" result in distributed systems, applied to a process and a payment API.

Tasks also have memos: "small first-writer-wins values stored in the live task envelope", where inserting a candidate and reading the winner are one commit, "so concurrent candidates return the same durable winner" — a durable compare-and-set for small facts.

15.3 The scheduler#

The scheduler (harness/scheduler.ts, about 2,000 lines) "serially reserves eligible tasks, then runs handlers off the Session line". Its drain loop:

#kick(): void {
	this.#dirty = true;
	if (this.#draining || !this.#enabled || this.#closing) return;
	this.#draining = true;
	// Never commit synchronously from a commit or registry listener.
	queueMicrotask(() => void this.#drain());
}

async #drain(): Promise<void> {
	try {
		while (this.#dirty && this.#enabled && !this.#closing) {
			this.#dirty = false;
			for (const reservation of await this.#reserve()) this.#start(reservation);
		}
	} …
}

Any event that might make work runnable (a commit, a registry change, a timer) calls #kick(), which sets a dirty flag and schedules one drain. The drain loops until no new kick arrived during its pass — the dirty-flag / coalesced wake-up pattern, which avoids both lost wake-ups and redundant passes. #reserve() then, in one commit, scans the runnable set, skips tasks that are waiting, already running or below a cancelled owner, resolves each task's definition against the current registry snapshot (a missing definition leaves it blocked, or orphans an abort-marked one), and durably moves each eligible task to running. Only after that commit does #start() launch the handler invocations, concurrently, off the line.

Reservation-before-execution is the durable-queue equivalent of a lease: a task marked running in storage belongs to this process. Because only one process owns a storage, no lease expiry is needed; after a restart, tasks found running simply become pending again and are re-reserved ("After a restart, tasks that were running show as pending until they run again").

Abort is a protocol, not a function call (spec 5.4):

commit abortRequested
signal and join active run invocation
wait until ordinary owned work is no longer live
start a fresh abort invocation
abort handler commits terminal outcome

The durable abort mark is the intent; the in-memory AbortSignal is how a running handler hears about it; and the abort handler runs as a separate invocation that decides the final outcome. "A run invocation may not commit after its durable abort mark appears" — a late write from a cancelled handler is refused. Cancelling a caller's context, by contrast, only stops that caller's wait: "Cancelling one caller's Context only cancels that call or wait; it does not durably abort shared work unless the invoked API commits an abort mark." Distinguishing "I stop waiting" from "stop the work" is one of the most common sources of bugs in concurrent systems, and the API makes the difference explicit.

15.4 Structured concurrency#

Tasks and conversations form one ownership tree (spec 5.5): every task is owned by its conversation or by another task; a conversation may be owned by a task. Two rules follow:

• Abort flows down. A live owner's cancellation intent marks its "ordinary owned work". The marks are derived in a separate reconcile commit, "and again at open, so a crash in between loses nothing".
• Owned work does not outlive its owner's finish. A task that decides its outcome while children are still live enters completing: "its outcome is decided, but it becomes terminal, and waitForTask() returns, only once that work is done."

A task can wait for children by committing { status: "waiting", on: [...], policy }. It then "runs no code while it waits"; with allSettled it resumes when all are terminal, with failFast "the first failed child also aborts the others". Waits are checked for cycles in the wait-for graph at commit time (a task cannot wait on itself or on an owner that holds it), so deadlocks are rejected rather than discovered at runtime. And "Abort order is bottom-up": a task's abort handler runs only after its owned work has finished aborting, so "each task undoes its own effects" and a parent sees its children's final outcomes. The README's checkout example: four payments, one card declined; with failFast the other three are marked, refund in their own abort handlers, and the checkout resumes to read [aborted, failed, aborted, aborted].

A task created with background: true is a boundary: work below it "survives the parent's abort and does not keep the parent busy" — the analogue of a daemon thread.

This is structured concurrency, the discipline popularized by Trio, Kotlin coroutines and Java's structured task scopes: concurrent work is nested in lexical-like scopes, children cannot outlive their parent, errors and cancellation propagate along the tree. pi-durable makes the tree durable: it survives a restart, so a crash in the middle of a fan-out does not leave orphans running or a parent forgetting what it waited for. Sub-agents fall out naturally: a subagent tool creates a conversation owned by its tool task; aborting the tool call aborts the sub-agent; the parent is idle only once the child is.

15.5 Generations: the model call as a task#

pi.generation replaces one iteration of the coding agent's loop. Its checkpoint union (spec 8.3) lists its phases: prepare, request, retry, poll (for deferred provider responses, with a durable handle and next poll time) and tools.

• prepare resolves the agent (model, tools, prompt sections), checks compaction thresholds, and in one commit appends any changed pi.system entries and moves to request, recording the model, thinking level, stream options and the newest included entry (cutoff). These "stay fixed for this request attempt; a retry prepares again."
• request streams the response through pi-ai. The partial answer is written to pi.live at most every progress.partialIntervalMs (100 ms by default). When the stream ends, one commit appends the pi.assistant entry and either finishes the run or creates one tool task per tool call and commits waiting on them (phase tools).
• retry waits until a backoff deadline; poll checks a deferred handle.
• tools resumes when every tool task is terminal, applies the run's boundary (place steering inputs, honour terminate or handoff), and creates the next generation, handing it the run.

Notice what a crash costs at each point. In prepare, nothing external happened; it reruns. In request, the request is resent — but the partial text committed so far is preserved in pi.live, and a "leftover partial" is converted before the new request. In tools, the waiting state is durable: after restart the generation simply waits again for its tool tasks.

15.6 Tools: intent, execute, replay#

Each tool call is its own pi.tool task with two phases (spec 8.4):

• call reads the call from the assistant entry, resolves the tool, validates arguments with pi-ai's validateToolArguments(), runs the beforeTool hooks (which may block or rewrite arguments), validates again, and then "One commit then records intent: it moves to execute with the final arguments and the tool's replay policy". The same handler then executes the tool, runs afterTool hooks, and commits the result as a pi.tool-result entry.
• execute is "reached only by recovery": the process died after the intent commit and before the result commit. "When both the stored and the current policy … are safe, it executes again with the stored arguments, without beforeTool, and settles like call. Otherwise it commits an interrupted error result from the slot's durable partial output, details, and diagnostics."

This is the effect sandwich applied to every tool, with a per-tool replay policy. read is safe to repeat; bash running git push is not. A tool declared replay: "safe" promises idempotence (or its own deduplication); any other tool gets an honest error result after a crash, and the model, seeing "interrupted", can inspect the world and decide. Note also that the arguments recorded at intent are the ones replayed — a beforeTool hook is not rerun, so a permission decision made before the crash is not asked twice.

Tools can call tools (api.executeTool()); each nested call is a task owned by the caller, keyed by its position among the caller's calls. "A replay-safe caller that reruns after a crash and calls in the same order, or passes the same keys, gets the nested calls it already made back, finished or still running, instead of starting them again." Tool output channels are separated: output for the model, structuredOutput for programs (validated against structuredOutputSchema), details for UIs.

Code mode survives restarts by deterministic replay. The durable codemode tool runs a model-written script in a QuickJS sandbox. How can a script resume mid-execution after a crash? The README explains: "the sandbox runs the script deterministically (a virtual clock, seeded randomness, UTC), and the tool writes each result the script gets to a journal before handing it over. After a restart, the tool runs the script again and hands it the journaled results in the same order, so it makes the same calls with the same arguments; these reattach to their nested calls by key, and then the script continues." If the script takes a different path than before, or the sandbox's fingerprint changed, the replay fails rather than guessing. This is exactly the technique of workflow engines such as Temporal: workflow code must be deterministic, every nondeterministic input (time, randomness, external results) is recorded in a history, and recovery replays the code against the history.

15.7 The inbox and boundaries#

User input while a run is active goes to the pi.inbox document as items tagged steer, followUp or write. "Steers are placed after the current tool round and join the running work. Follow-ups are placed when the run answers, and start the next run." Placement happens only at boundaries — after a tool round (postTools) or at the end of a run — never mid-request, so the transcript stays a valid alternation of calls and results. A display-only write (an entry with only kind and data, which changes nothing the model sees) is appended at once even mid-run, because "Between a tool call and its result it changes nothing". These are Chapter 6's steering and follow-up semantics, made durable.

15.8 Compaction without stopping#

Compaction is a task too (CompactionTask). The settings define two thresholds: above contextWindow - reserveTokens the next request waits for a compaction (the generation commits waiting on a compaction it owns); above contextWindow - reserveTokens - backgroundTokens a compaction starts in the background while the conversation keeps working. "The summary is placed at once when the conversation is idle, otherwise at the next turn boundary." Because several compactions may be in flight, "A summary that would cut before the start of the current context settles as stale when it is placed, so when several are in flight, the furthest cut stays in effect" — a last-writer-wins rule on the context head. On a provider overflow error, "generation compacts and retries once."

15.9 Observation: views, watches and events#

Because only committed state is observable, observation is uniform. viewState() returns a read-only Chord state holding a conversation's entries and documents, updated after each commit. watch() delivers the same view plus "the exact Chord operations of each commit", suitable for streaming to a remote client that applies them — state replication by operation shipping (Chapter 17). Slow consumers are handled explicitly: "A slow watch keeps at most 100 undelivered frames. After that, the pending frames are replaced by one frame holding the whole newest view. A client that joins late or reconnects starts from the current view; nothing is replayed." This is a bounded queue with a snapshot fallback — backpressure without blocking the producer or growing memory without bound.

watchEvents() derives coding-agent-style events (message_update, tool_execution_start, …) from commits, for consumers that prefer events to state. harness.taskGraph() shows every live task as a tree for debugging, and harness.inspect() reports blocked tasks.

15.10 Extensions and hot reload#

An extension is a named bundle of tools, prompt sections, hooks, wraps (decorators of a tool or section by name) and tasks, installed into a Registry owned by the process. Conversations store extension names in pi.agent, never code: "a stored name outlives its code: after an extension is uninstalled, conversations that select it just stop getting it until it is installed again." Installing an extension with an existing name replaces it atomically, and "Work that already started keeps the code it took: a running tool call finishes under its old implementation, and each task phase resolves hooks and the agent once, from the registry at the phase's start." Phase boundaries are the safe points at which new code takes effect — the same idea as safe points for code replacement in Erlang hot loading.

Hooks are typed per task: generation hooks beforeRequest, afterResponse, onYield (continue the run with another message) and afterTools; tool hooks beforeTool (block or rewrite) and afterTool (replace the result).

15.11 Concepts in this chapter#

• Durable state machines and checkpointing. Persist the program counter (phase) and its data at each step.
• Effect sandwich (intent → effect → outcome). Make external effects recoverable; combine with idempotency keys for effectively-once results.
• At-least-once execution and idempotency. Exactly-once effects are impossible without the cooperation of the system being affected.
• Schedulers, reservations and coalesced wake-ups. A durable run queue drained by a dirty-flag loop.
• Structured concurrency. Ownership trees; children cannot outlive parents; cancellation flows down; cleanup runs bottom-up.
• Join policies and wait-for graphs. All-settled vs fail-fast joins; cycle detection to reject deadlocks.
• Deterministic replay. Re-execute deterministic code against a recorded history of nondeterministic inputs.
• Schema evolution of durable state. Versioned task definitions with migrations.
• Bounded queues with snapshot fallback. Backpressure for observers.
• Safe points for hot code replacement. Switch code only at well-defined boundaries.

Exercises#

1. Write a SendEmail task with phases prepare and send and an abort handler. Where must the idempotency key be created, and what must the email service support for the design to be effectively-once?
2. A tool with replay: "safe" appends a line to a log file. Is it safe? How would you make it so?
3. A parent waits failFast on children A, B, C. B fails while A is mid-effect. Trace the abort marks, the order of abort handlers, and the parent's resumed outcomes.
4. Why does the scheduler reserve tasks in a commit before starting them? What breaks if it starts them first?
5. Name three sources of nondeterminism a codemode script could use if the sandbox did not control them, and what replay would do wrong with each.
6. A watch consumer stalls for 10 seconds while 500 commits happen. What does it receive when it resumes, and why is that preferable to receiving all 500 frames?
7. Draw the ownership tree for a run in which the model calls subagent twice in parallel and one sub-agent calls bash.