Durable execution: the problem and the data model

pi-durable (packages/durable) is "a durable agent harness. Conversations, model turns, tool calls, and your own state are committed to storage before anything is shown. If the process dies mid-turn, reopening the storage picks the work up where it stopped." It is a second, independent agent runtime beside pi-agent-core, designed for agents that run on servers, survive deploys and crashes, coordinate sub-agents, and serve several clients at once. This chapter states the problem, the core rule that solves it, and the data model; Chapter 14 covers transactions and storage, Chapter 15 the task machinery, Chapter 16 execution environments.

The package is marked experimental ("The API changes without notice between releases") and specified in a normative design document, packages/durable/docs/spec.md, of about 40,000 words. Quotations below come from that spec and the package README.

Learning goals. Explain what goes wrong when an in-memory agent loop crashes; state pi-durable's core rule and its invariants; describe the five kinds of record (conversations, entries, tasks, submissions, documents); explain idempotent submission with request ids; contrast the design with the coding agent's session files.

13.1 The problem#

Consider the coding agent of Part IV running on a server. Its state lives in memory: the Agent's transcript, its queues, the partial assistant message, the set of running tool calls. The session file records finished messages. Now kill the process at four different moments:

1. While the model is streaming. The partial answer is gone; the request must be resent and paid for again.
2. While a tool runs. Did git push happen? The transcript holds a tool call with no result. On restart, should the tool run again?
3. While two sub-agents work for a parent. Nothing records that the parent was waiting for them.
4. While the user's follow-up message sits in a queue. It is lost silently.

The general problem is durable execution: making a long-running, multi-step computation survive process failure without redoing completed steps or repeating side effects. Workflow engines (Temporal, Azure Durable Functions, AWS Step Functions) are built around it, and so are database-backed job queues. pi-durable applies the idea to agents.

13.2 The core rule#

The spec reduces the design to one sentence:

A Session atomically commits immutable entries, full task records, and Chord-tracked documents. Only committed state is observable.

and eight invariants, among them:

1. "One Session commit is atomic across all record and document writes."
2. "A document update is published only after its storage commit succeeds."
3. "All visible progress is durable. There is no volatile publication path."
4. "External effects do not run inside the Session mutation transaction."
5. "Entries and IDs are immutable and never reused after a committed write."

Invariant 3 is the radical one. In the coding agent, streamed text reaches the screen before anything is saved. In pi-durable, even a streaming partial answer is first committed (by default "at most every 100 ms", so "a crash loses at most that window") and the UI renders committed state. Whatever a user saw can be recovered after a crash, because there is nothing else to see. Invariant 4 separates the world into two halves — database transactions, which can be atomic, and effects (HTTP calls, processes, people), which cannot — and Chapter 15's "effect sandwich" bridges them.

13.3 Vocabulary#

The README's concept list, condensed:

One answered input, as the README draws it:

submit(input) → pi.user
  pi.generation → pi.system (only if the prompt or tools changed), pi.assistant (tool calls)
    pi.tool × n → pi.tool-result × n   (owned by the generation, which waits for them)
  pi.generation → pi.assistant (answer) → submission done

The agent loop of Chapter 6 has become a chain of durable tasks: a generation task calls the model; it creates one tool task per tool call and waits for them; then it hands the run to the next generation task.

13.4 Records and identities#

The spec defines the core records in TypeScript. Identifiers are numbers with compile-time brands:

declare const idBrand: unique symbol;
type Id<Kind extends string, Type = unknown> = number & {
  readonly [idBrand]: {
    readonly kind: Kind;
    readonly type: Type;
  };
};
type ConversationId = Id<"conversation">;
type EntryId = Id<"entry">;
type TaskId<Result = unknown> = Id<"task", Result>;

This is Chapter 3's branding trick, extended: TaskId<Result> even carries the task's result type, so waitForTask(id) can return a typed outcome. "ID brands are compile-time only. IDs and sequences remain ordinary numbers in memory, JSON, JSONL, and SQLite." All ids come from "one global numeric namespace, above every ID minted or stored before, also across reopen: scans order by ID as creation order" — so ordering by id is ordering by creation, a property the storage indexes exploit. A separate Seq brand marks commit sequence numbers, which "strictly increase between commits; gaps are permitted".

The entry record:

type EntryRecord = {
  readonly id: EntryId;
  readonly conversationId: ConversationId;
  readonly kind: string;
  readonly model?: readonly Message[];
  readonly data?: JsonValue;
  readonly head?: EntryId;
  readonly edits?: readonly ContextEdit[];
  readonly byTaskId?: TaskId;
};

An entry separates what the model sees (model, zero or more pi-ai messages) from what the application stores (data). A chat message from a teammate can be stored as { kind: "chat.mention", data: { author: "ann", … } } while the model sees ann: can you check the build?. head makes an entry a head marker: "The model sees the entries from the newest reset onward" — resets and compactions are entries that move the start of the model's context. edits omit or replace earlier entries' model content (Chapter 9's context edits, generalized). byTaskId records which task wrote it.

Conversations form two different graphs, kept apart deliberately: "parent controls inherited entries and historical documents" (a fork sees its parent's entries up to the fork point), while "owner records task attribution and connects scopes for subtree abort and idle traversal". A sub-agent's conversation is owned by the tool task that created it; aborting the task aborts the sub-agent.

13.5 Submissions and idempotency#

A submission is the durable form of "the user said something". Its record is a discriminated union whose status moves through a small state machine:

input:  queued → placed → done            (answered: has entry and answer)
                        ↘ unanswered      (with a reason: aborted, model_error, reset, stale, …)
write:  queued → done | unanswered

The TypeScript type encodes which fields exist in which state (answer only when done, reason only when unanswered), so invalid combinations do not type-check.

Submissions solve the classic at-least-once delivery problem. A client sends a message, the server crashes, the client cannot tell whether the message was stored, so it resends. Without care, the message is processed twice. pi-durable's answer is the idempotency key:

const submission = await root.submit({ type: "input", content: "Hello", requestId: "greeting-1" }, context);
// After a restart: the same request ID finds the same submission.
const again = await root.submit({ type: "input", content: "Hello", requestId: "greeting-1" }, context);
// again.id === submission.id

The SQLite schema indexes submissions by (conversation_id, request_id) for exactly this lookup. At-least-once delivery plus idempotent processing yields effectively-once behaviour, the standard technique in payment APIs and message queues.

13.6 Documents: mutable state, durably#

Transcripts are immutable, but agents also need mutable state: which tasks are running, what is queued, how much has been spent. pi-durable stores this in documents: "typed JSON state stored next to the transcript and changed in commits". Built-in documents per conversation include:

• pi.agent — the conversation's model, thinking level, selected extensions and tools, instructions, working directory;
• pi.provider — a persisted UUIDv7 forwarded to providers as sessionId "for provider prompt-cache and session affinity";
• pi.live — "running generation and tools": the streamed partial answer, retry state, tool output so far;
• pi.inbox — queued submissions;
• pi.usage — tokens and cost per model and per tool.

Applications define their own:

const Todos = defineDoc<{ items: string[] }>({
	kind: "app.todos",
	version: 1,
	scope: "conversation",
	history: "latest", // or "rewindable" to read old values with snapshotAsOf()
	fork: "initial", // what a fork starts with: "initial", "current", or "asOf"
	initial: () => ({ items: [] }),
});

await root.commit(async (tx) => {
	(await tx.doc(Todos, root.id)).items.push("write docs");
}, context);

Note the programming model: inside a commit, tx.doc() returns a draft that you mutate like an ordinary object. Under the hood the draft is tracked by Chord (Chapter 17), which records each mutation as an operation. Storage keeps a document as occasional complete bases plus a log of deltas (DocumentContent is { kind: "base", value } or { kind: "delta", ops }), and a reader materializes a value by replaying the deltas since the last base — snapshots plus a log, the same structure databases and event-sourced systems use to bound replay time. Documents are scoped to the session, a conversation or a task; task-scoped documents retire automatically when the task ends; history: "rewindable" keeps old values readable; and the fork policy says what a forked conversation starts with.

Because documents change only in commits, observing them is uniform: watchDoc(), viewState() and watch() deliver committed values and the exact Chord operations of each commit to any subscriber, local or remote (Chapter 15).

13.7 Two designs compared#

Neither is "better": the coding agent is simpler and fits a single developer at a terminal; pi-durable pays for durability with a commit per visible step and a much larger specification.

13.8 Concepts in this chapter#

• Durable execution. Persist the progress of a long computation step by step so it resumes after failure without repeating completed steps.
• Commit-before-publish. Make state observable only after it is durable; observers can never see something that will be lost.
• Immutability and append-only records. Entries never change; corrections are new entries (head markers, edits).
• Branded, monotonically allocated identifiers. Type-safe ids whose numeric order is creation order.
• State machines encoded as discriminated unions. Each status carries exactly the fields valid in it.
• Idempotency keys and effectively-once processing. Retries carry a client-chosen key; the server deduplicates.
• Snapshots plus deltas. Store occasional full bases and a log of operations; materialize by replay from the latest base.
• Separation of history and ownership graphs. Inheritance of data and lifetime/supervision are different relations.

Exercises#

1. For each of the four crash moments in 13.1, say which record or document in pi-durable preserves the relevant state.
2. Write the SubmissionRecord transitions as a state diagram. Which transitions must be atomic with an entry append, and why?
3. A client times out waiting for submit() and retries without a requestId. What can happen? With one?
4. Why are entries immutable while documents are mutable? Give an example of state that belongs in each.
5. Design a history: "rewindable" document for a todo list and describe how a fork with fork: "asOf" would initialize it.