Checkpoint Rewind and Compaction

Status: Design landing Reference epic: INK-847 ADRs: ADR-016

Checkpoint-rewind compaction is a composition pattern, not a new primitive. It reframes LangGraph checkpointing from “resume after failure” to also mean “produce then rewind.” The canonical conversation history is born compact. Expansion exists only to produce the compact form; the expansion is then discarded from history.

The problem

Every long-running, context-expensive action leaves a trace. A subagent sweep produces intermediate tool calls, partial results, and planning turns. An investigation session (see Investigation Pattern) produces FINAL output but the task() call itself, the planner decisions that surrounded it, and any streaming chunks still land in the conversation. A multi-step research loop accumulates intermediate states.

Without compaction, that accumulation raises context pressure over time. A worldbuilder working with the World Agent across many turns will find the agent’s context increasingly occupied by the scaffolding around completed work rather than by the conclusions from it.

The sequence

The pattern has four steps, applied as a unit:

1. Checkpoint. Before the expensive action starts, the LangGraph thread is checkpointed at the wrap boundary. This checkpoint is the rewind target — distinct from the per-step checkpoints the runtime already writes for resume-after-failure.

2. Expand. The action runs at full fidelity. The investigation does its full loop, the subagent does its full sweep, the research task runs end to end. Nothing is abbreviated. The turn is allowed to be expensive because it is discarded regardless.

3. Compact. After the action returns successfully, a compaction step extracts a compact form from the full trace. The compact form is a structured value object:

   • summary — a brief description of what the action did and concluded.
   • insights — a list of MemoryCandidate entries: observations, contradictions, conclusions worth surfacing to the memory system.
   • outputs — the objective outputs the action produced (pages written, structured results, retrieved conclusions).
   • derivation_trail — the entity references that the action’s work derived from; used for provenance when insights cross the submit boundary.

4. Rewind. The thread is rewound to the pre-action checkpoint. The compact form is inserted as the canonical history record in place of the full expansion. Control returns to the calling agent with the compact form as the result.

The full pre-rewind trace is written to the event log as an expanded_trace entry linked to the compact-form turn. It is auditable, recoverable, but never reloaded into context.

Two trigger modes

Bounded — wraps a single expensive action

The bounded mode wraps one specific task() dispatch. It is the compaction execution shape: task(shape="compaction", inner=…) where inner is another task() call or a callable that performs the expensive work. The compaction shape handles the checkpoint-expand-compact-rewind sequence around the inner action.

Bounded mode is synchronous. The calling agent awaits the result before continuing. The rewind must happen before control returns; there is no window where the full expansion is the canonical history.

Failure semantics: if the inner action does not complete successfully, there is no rewind. The full trace is preserved in history for diagnosis. The pattern does not silently discard a partial trace; it only rewinds when the action ran to completion and a compact form was successfully extracted.

Aging — retroactive compaction of older turns

The aging mode is a scheduled job, registered with the Rust task-runner (see Scheduling System). It runs per channel or per conversation on a policy that triggers based on configurable thresholds — turn count, token pressure, age, or explicit worldbuilder signal. The exact policy shape is an open question.

When the aging job fires, it identifies older turns in the conversation that are eligible for compaction. For each eligible turn, it reproduces the compact form from the event-log trace (the full expansion is still there) and substitutes the compact form in the canonical history via the checkpoint-rewind mechanism. The conversation history is retroactively shortened.

The aging mode runs the same isolation profile as an investigation: in a forked context, returning only the substituted history. It does not operate in the calling agent’s live context.

Compact-form value object

The compact form is a stable shape used across both trigger modes:

{
  summary: String,
  insights: Vec<MemoryCandidate>,
  outputs: Json,
  derivation_trail: Vec<EntityRef>
}

MemoryCandidate entries extracted from insights cross the submit boundary with Origin::AgentProduced, Lifecycle::Candidate, and derivation links from the derivation_trail. They are subject to the same provenance and permission model as any other agent write (see Submit Boundary).

How this differs from append-then-rewrite

Most consolidation patterns documented in agent systems follow an append-then-rewrite shape: the expensive trace lands in history first, a background pass cleans it up afterward. There is a window where the full expansion is live in context. The cleanup is asynchronous and chases the accumulation.

Checkpoint-rewind inverts this: checkpoint → expand → compact → rewind. There is never a window where the full expansion is canonical history; the expansion was always in a temporary branch. The calling agent’s history never sees the discards. Compaction is bounded around the action it summarizes, not chasing it after the fact.

The full trace is not lost — it is in the event log. The information is recoverable; it is just not in context.

Relationship to investigation FINAL semantics

The investigation pattern (see Investigation Pattern) already provides context isolation at the sandbox boundary: everything the investigation touched inside the sandbox stays inside the bubble. The FINAL answer is what crosses back.

Checkpoint-rewind compaction operates one level up. It can wrap the task() call that dispatches the investigation. The investigation itself produces a compact FINAL answer; the compaction shape further compacts the surrounding planner context — the turns the main graph spent deciding to run the investigation, waiting for it, and processing the result. Both mechanisms compose: investigation discipline handles the sandbox boundary; compaction handles the conversation-history boundary.

Relationship to existing surfaces

• LangGraph checkpointer — the rewind mechanism uses the same checkpointer that the runtime already uses for resume-after-failure. The pre-action checkpoint is an additional checkpoint entry distinguished by a checkpoint_role: "rewind_target" tag.
• task() primitive — compaction is one of the registered execution shapes. See Task Primitive.
• Event log — expanded_trace entries link the full pre-rewind trace to the compact-form turn. The event log retains everything; context does not.
• Submit boundary — extracted insights cross with full provenance. See Submit Boundary.
• Task-runner / Scheduling System — the aging-mode trigger registers with the existing task-runner infrastructure. See Scheduling System.
• Agent Memory System — the aging-mode policy (“what survives compaction?”) interacts with memory-tier consolidation, but the two concerns are distinct. Tier-to-tier consolidation is a memory system concern; compaction operates on conversation history.

Open questions

The following are open in the current design:

• Aging policy shape. Turn-count threshold? Token-pressure threshold? Age-based? Composable? Per-channel default plus per-conversation override? The aging trigger mechanism ships with a policy primitive; the specific defaults and composability model are open in the current design.
• Bounded-mode invocation. Does every task() invocation get checkpoint-rewind by default, or is it explicit opt-in? Is there a heuristic for auto-flagging expensive actions as compaction candidates?
• Nested investigations. When an investigation (see Investigation Pattern) runs nested, does the outer conversation’s checkpoint-rewind apply at the outermost boundary only, or at each nesting level? The answer has implications for budget accounting.
• Partial-completion failure. If an action partially completes — some tool calls succeeded, some failed — do we rewind, preserve the full trace, or attempt to extract a partial compact form?
• Compact-form LLM call. What model produces the compact form? Cheap analyst model? Same model as the action? Per-mode override?

What this pattern is not

It is not async bounded compaction. Bounded mode is synchronous. The rewind happens before control returns to the caller. There is no background pass.

It is not lossy. The event log linkage is mandatory. The full pre-rewind trace is always recoverable. The failure mode of compaction that discards irretrievably is mitigated by the event log, not avoided.

It is not a replacement for investigation FINAL semantics. Investigations already produce compact output at the sandbox boundary. Compaction operates on the calling agent’s thread, not inside the sandbox.

It is not tier-to-tier memory consolidation. Memory consolidation moves content between memory tiers based on durable storage policy. Checkpoint-rewind compaction operates on conversation history — the checkpointer’s state, not the memory tier storage.

It is not automatic or invisible. Bounded mode requires an explicit task(shape="compaction", …) invocation. Aging mode fires on a declared policy. Neither happens silently.

What this page does not do

• It does not describe the LangGraph checkpointer or how thread state is structured. See Agent Core System.
• It does not describe the task() primitive or the shape registry. See Task Primitive.
• It does not describe the investigation pattern or its FINAL semantics. See Investigation Pattern.
• It does not describe the event log system or expanded-trace entry semantics. See Event Log System.
• It does not describe memory tier consolidation or the decay model. See Agent Memory System.
• It does not describe how scheduled tasks are triggered by the task-runner. See Scheduling System.
• It does not describe submit-boundary provenance or derivation links. See Submit Boundary.