docs: add ref-based checkpoint backend (git-refs) architecture doc

The git-refs backend is fully implemented (registry, refs store, push queue, routing store) but had no document explaining it as a system — its behavior was only reconstructable from code comments and scattered design notes. sessions-and-checkpoints.md covered the ID/read-routing layer but its Storage and Package Structure sections remained git-branch-only.

Add docs/architecture/ref-checkpoint-backend.md covering: primary/mirror taxonomy and the gitBacked capability, the refs/entire/checkpoints/ / layout and last-two-char sharding, the per-checkpoint write path (orphan-then-parented history), the push-discovery queue and fast-forward-only push with fetch+replay recovery, on-demand ref fetch, kind-based read routing and coexistence, configuration + env override + rollout states, checkpoint-version formats, and known limitations.

Grounded in the current code (refs_naming.go, refs_store.go, pushqueue.go, routing_store.go, registry.go, checkpointpolicy/format.go, settings/checkpoints.go), not the older design notes — notably sharding is last-two-chars for both ID formats, not the earlier asymmetric plan.

Sessions

Changes

1

• docs/architecture

• Aref-checkpoint-backend.md+206

Ref-Based Checkpoint Backend (git-refs)

This document explains the git-refs checkpoint backend as a system: how it stores checkpoints, how it pushes and fetches them, how it coexists with the legacy git-branch backend, and how it is selected through configuration.

It is the companion to Sessions and Checkpoints, which covers the domain model (sessions, checkpoints, IDs) shared by both backends. Read that first for the checkpoint tree layout, checkpoint-ID linking, and the compact-transcript format; this doc focuses on what is specific to the ref-based store.

Why a second backend

The original backend stores every committed checkpoint as a subtree of a single long-lived branch, entire/checkpoints/v1 (the git-branch backend). That branch is a serialization point: every condensation rewrites its tip, every push races on one ref, and the whole history travels together.

The git-refs backend instead keeps one git ref per checkpoint:

refs/entire/checkpoints/<shard>/<id>

Each ref points at a commit whose tree root is that checkpoint's contents (metadata.json, 0/, 1/, tasks/…) — the same subtree the git-branch backend splices under <id[:2]>/<id[2:]>/ in the v1 tree. Independent refs mean checkpoints are written, pushed, and fetched independently: no shared tip to contend on, and a reader can fetch exactly the one checkpoint it needs instead of the whole branch.

Both backends are git-backed — they store the committed record in the repo's own object store — and never touch the working branch's history.

Backend taxonomy: primary and mirrors

Checkpoint storage is pluggable. The topology is a single primary plus zero or more mirrors:

• Primary — the source of truth. It serves all reads and writes, and the full checkpoint lifecycle (resume bootstrap, doctor reconcile, explain tree reads, push, cleanup, pre-push OPF) drives its record.
• Mirror — an independent backend that receives best-effort write fan-out only. Reads never come from a mirror.

Backends register in checkpoint/registry.go. Each carries a gitBacked capability:

Only a git-backed backend can be the primary, because the lifecycle paths above operate through the repo and its refs; a non-git-backed backend has no such ref to drive them. The two built-in backends — git-branch and git-refs — are both git-backed and are registered directly in the built-in registry map. The Register() entry point is for non-git-backed (mirror-only) backends and is used in practice only by test-only backends, so a production binary can never select an unregistered one.

A one-of-each-type rule lets two distinct git-backed backends run in the same topology — specifically git-refs as primary with git-branch as a mirror, which is the safe rollout configuration (see Rollout).

Ref layout and sharding

refs/entire/checkpoints/<shard>/<id>

• <id> is the full checkpoint ID (12-hex or ULID) and is always the leaf, so the ref round-trips: RefName(id) builds it and ParseRef(name) recovers the ID (checkpoint/refs_naming.go).
• <shard> is id.ShardFor() — the last two characters of the ID, for both formats.

A single positional rule (independent of ID kind) keeps ref naming impossible to compute inconsistently between callers, and the suffix distributes checkpoints evenly for either format:

• A legacy hex ID is random throughout, so its last two chars are as good as any.
• A ULID's leading chars encode a millisecond timestamp (barely varying between nearby checkpoints) while its trailing chars are random — so sharding on the suffix keeps buckets even and keeps the ID itself lexicographically time-sortable.

ParseRef validates that the shard in a ref name matches the ID's own ShardFor() and that the tail is exactly <shard>/<id> (no extra path segments), so a malformed or foreign ref is rejected rather than resolved to the wrong bucket. RefName errors on an empty or unrecognized ID rather than emitting refs/entire/checkpoints//.

Note: this is the git-refs namespace only. The git-branch backend keeps its own independent first-two-chars tree layout (<id[:2]>/<id[2:]>/) inside the v1 branch. The two sharding schemes are deliberately different and do not interact.

ID formats

Checkpoint IDs come in two shapes; the store determines which is minted:

• git-branch primary → 12-char hex IDs.
• git-refs primary → 26-char ULIDs (Crockford base32, lexicographically time-sortable).

IDs are minted by checkpoint.GenerateCheckpointID, which picks the format from the configured primary. Never call id.Generate() / id.GenerateULID() directly from a write path. The full rationale for the two formats, DisplayShort, and id.MaxIDLength lives in Sessions and Checkpoints → Checkpoint ID Linking.

Write path

The git-refs store (gitRefsStore, checkpoint/refs_store.go) shares the checkpoint-subtree machinery with the git-branch store via an embedded *treeWriter anchored at base path "". The two differ only in where the subtree is committed: a per-checkpoint ref instead of a subtree of the v1 branch.

Every persistent write (WriteSession, and the Backfill* operations for transcript / summary / attribution) follows the same shape:

1. Resolve the ref's current tip (refBase). A missing ref → (ZeroHash, nil), so the first write to a checkpoint becomes an orphan commit. A real lookup failure (IO/corruption) is surfaced, never silently treated as "new checkpoint".
2. Build the updated checkpoint subtree from the existing tree plus the new content (shared treeWriter logic).
3. Create a commit with the current tip as parent (orphan on first write, parented thereafter), so each checkpoint accretes its own per-checkpoint history.
4. Point the ref at the new commit (setRef) and enqueue it for push.

Enqueue is best-effort: a write that lands locally but fails to enqueue must not fail condensation. The ref is still local and correct; only its remote sync is deferred until the next write to the same checkpoint re-enqueues it (see the push queue below).

Push and fetch

Because reads can fetch refs on demand and there is no single branch tip to push, the git-refs backend cannot simply "push everything" at pre-push time. Deleting local refs after pushing would also hurt local workflows. Instead it tracks exactly which checkpoints changed, in a push-discovery queue, and pushes those.

Push-discovery queue

checkpoint/pushqueue.go — a flock-protected JSONL file in the git common dir (so every worktree sharing the object store enqueues into one queue):

• entire-checkpoint-push-queue.jsonl — one {"ref": …} record per queued ref.
• entire-checkpoint-push-queue.lock — the flock.

Semantics:

• Enqueue appends under the lock; enqueuing an already-present or already-pushed ref is safe (drain de-dups, the push is idempotent).
• Drain returns the de-duplicated queued refs without removing them, and compacts the file in place when it held redundant lines (so a long-lived session that keeps re-enqueuing the same ref cannot grow the file unboundedly).
• Remove deletes refs only after a confirmed push, preserving any entry appended during the push. An interrupted or failed push therefore leaves its refs queued for the next pre-push — the queue degrades toward "will retry", never toward silent loss.

Rewrites are atomic (temp file + rename under the lock) so a concurrent reader never sees a half-written queue.

Pre-push flow

ManualCommitStrategy.PrePush (strategy/manual_commit_push.go) branches on checkpoint.PrimaryIsRefs(cfg). When the primary is git-refs it:

1. Drains the queue.
2. Partitions the drained refs into those that still exist locally and stale ones (dropped from the queue).
3. Batch-pushes the existing refs in one network round-trip (batchPushRefs, strategy/push_common.go).
4. On success, removes the pushed refs from the queue and runs shadow-branch cleanup.
5. On a batch failure (typically a non-fast-forward rejection), falls back to per-ref recovery (pushCheckpointRefWithRecovery) and removes from the queue only the refs that land.

Non-force, fast-forward-only

All checkpoint-ref pushes are fast-forward-only — never a force push. There is no server-side ref protection, so a force push risks silently clobbering a checkpoint written elsewhere. Per-checkpoint refs normally advance by fast-forward (append-only per-checkpoint history), so this is the common case.

When a push is rejected as non-fast-forward — genuine divergence, e.g. the same checkpoint was written on two machines — recovery fetches the remote ref and replays the local-only commits on top (fetchAndRebaseRefCommon), then retries. After the replay the local ref is a fast-forward over the remote, so the retry is still non-force and the remote commit is preserved as an ancestor rather than overwritten. A genuine cherry-pick conflict (both sides rewrote the same file, e.g. root metadata.json) leaves the ref queued — degrading to the safe state, never forcing.

On-demand fetch for reads

A checkpoint written on another machine has no local ref. When a read misses locally and a ref fetcher is configured, resolveRefMaybeFetch fetches that one ref from the remote and retries once. It carefully distinguishes:

• genuinely absent (remote has no such checkpoint) → maps to ErrCheckpointNotFound;
• a real failure (IO, network, context cancellation) → returned as-is, never swallowed as "not found".

List at the storage level is local-refs-only — it enumerates local refs and reads each root summary. There is no remote enumeration.

Read routing and coexistence

checkpoint.Open returns a kindRoutingStore (checkpoint/routing_store.go) that resolves id-keyed reads across both git backends by the checkpoint's ID kind, so a repo running git-refs and git-branch side by side (or mid-migration) reads either format without reconfiguring:

• List unions both backends and de-dups by ID (the same checkpoint can appear in both during coexistence), keeping the most recent.
• The firstResolved helper tries stores in priority order; a non-final store that reports absent or errors falls through to the next, so a transient git-refs fetch error cannot hide a checkpoint that resolves on the branch. The final store's result (hit, absent, or error) is returned verbatim.
• The optional AuthorReader capability (explain relies on it) is preserved and routed by the same rules when both read stores provide it.
• Writes are not kind-routed. They target the configured primary (+ mirrors); the minted ID already matches the primary's format.

All general read paths — resume, explain, attribution, blame, tokens, attach — inherit this routing for free through checkpoint.Open; there is no per-command config knob.

Configuration and rollout

Backend selection lives in the checkpoints block of settings (settings/checkpoints.go):

{
  "checkpoints": {
      "primary": { "type": "git-refs" },
      "mirrors": [ { "type": "git-branch" } ]
  }
}

• primary.type is required. When the whole block is absent, the layer defaults to the git-branch backend with no mirrors — so existing repos are unchanged.
• settings.local.json's checkpoints block replaces the one in settings.json wholesale (this is a selection config, not a deep-merged document).
• Config loading is fail-soft: a missing file, a whole-file JSON syntax error, or unrelated invalid fields all resolve to "no config" → default git-branch. It errors only when a present checkpoints block is itself invalid.
• Unknown fields are rejected (DisallowUnknownFields) to surface typos. The trade-off: adding a checkpoints field is a coordinated rollout — ship the reader before any writer emits the field.

Environment override

ENTIRE_CHECKPOINTS_PRIMARY (and the optional comma-separated ENTIRE_CHECKPOINTS_MIRRORS) fully replace any settings block — env wins over file, matching other ENTIRE_* overrides. This is how e2e/CI and rollout drive a specific backend without editing settings; the CI test-canary job runs a matrix over [git-branch, git-refs] via this variable. The env override is selection-only (no per-backend config blocks).

Rollout states

Checkpoint version and policy

Checkpoint formats are named <family>-v<major> and validated in checkpointpolicy/format.go:

Both are in the CLI's read and write sets. The repo-wide checkpoint policy (refs/entire/policies/checkpoint, checkpoint_version / checkpoint_min_version) gates which formats a client may write and nudges upgrades; see Sessions and Checkpoints → Checkpoint Policy.

Migration and coexistence

The read-routing rules above are what make a hex-on-branch repo and a ULID-in-refs repo the same repo: nothing needs to move for both formats to be readable. When checkpoints are migrated from the branch into refs, they are written under RefName(hexID) — i.e. hex-named refs — which is why a hex ID under a git-refs primary is looked up in refs first and only then falls back to the branch.

Key files

Known limitations and deferred work

• Storage-level List is local-only — no remote enumeration of checkpoint refs. List at the routing layer still unions the two local stores.
• OPF (OpenAI Privacy Filter) at pre-push is git-branch-only for now. The per-ref push does not run OPF re-redaction; that is deferred until after the store lands. See strategy/manual_commit_opf_rewrite.go and security-and-privacy.md.
• v1 mirror push for full downgrade safety is a rollout concern handled by running git-branch as a mirror (the Parallel state), not an automatic behavior of the refs store itself.
• No write-time ULID⇒refs boundary guard. Nothing at write time enforces that a ULID checkpoint only condenses onto refs. A config flip or a missing ENTIRE_CHECKPOINTS_PRIMARY in an amending environment could, in principle, land a ULID on the v1 branch, which readers (routing ULIDs to refs) would then fail to find. The guard must be topology-role aware (git-branch legitimately receives ULIDs when it is a mirror of a git-refs primary), so it belongs with the routing layer, not in the branch store's write path.