compact: add FullWithBoundary for full transcript + scope marker

f1a2b7a→main·

Soph·2w ago·2 files·+183 added/-0 removed

FullWithBoundary compacts the entire transcript (ignoring StartLine for truncation) and returns the full compact bytes plus the line index at which this checkpoint's slice begins in the compact output. The boundary is derived as lineCount(full) - lineCount(delta), reusing each format's existing, independently-tested slicing behavior rather than threading source indices through all seven emitters. The only imprecision is a deterministic off-by-one when a single logical message straddles the exact boundary.

This is the foundation for storing a full, self-contained compact transcript on every checkpoint (so a session is reconstructable from any single checkpoint) while still letting readers segment a checkpoint's slice via the marker.

Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com

Sessions

9bf575d87dc5View transcript

Changes

2

• cmd/entire/cli/transcript/compact

• Mcompact.go+50

• Acompactfull_test.go+133

118 unmodified lines

// FullWithBoundary compacts the entire transcript — ignoring opts.StartLine for // the purpose of truncation — and returns the full compact bytes together with // the boundary: the line index in the compact output at which this checkpoint's // data (the content from opts.StartLine onward in the raw transcript) begins. // // Storing the full compact transcript on every checkpoint makes each checkpoint // self-contained: the session is reconstructable from any single surviving // checkpoint, and downstream readers segment this checkpoint's slice as // fullCompactLines[boundary:]. boundary is 0 when StartLine <= 0 (the first // checkpoint of a session, whose data starts at the beginning). // // boundary is derived as lineCount(full) - lineCount(delta), where delta is the // existing per-checkpoint sliced compaction (Compact with the original // StartLine). This reuses each format's existing, independently-tested slicing // behavior — line offsets for JSONL/Copilot/Droid/pi, message-index for // OpenCode/Gemini, response-item index for Codex — rather than threading source // indices through every emitter. The only imprecision is an off-by-one when a // single logical message (a streaming assistant message split across the // boundary, or a tool_result that also carries text) straddles the exact // StartLine; the result is deterministic and harmless for segmentation. func FullWithBoundary(redacted redact.RedactedBytes, opts MetadataFields) (full []byte, boundary int, err error) { fullOpts := opts fullOpts.StartLine = 0 full, err = Compact(redacted, fullOpts) if err != nil { return nil, 0, err } if opts.StartLine <= 0 { return full, 0, nil }

delta, err := Compact(redacted, opts) if err != nil { return nil, 0, err }

boundary = countCompactLines(full) - countCompactLines(delta) if boundary < 0 { boundary = 0 } return full, boundary, nil }

// countCompactLines counts newline-terminated lines in compact output. The exact // convention does not matter for CompactFull as long as it is applied uniformly // to both the full and delta outputs (the boundary is their difference). func countCompactLines(b []byte) int { return bytes.Count(b, []byte{'\n'}) }

// droppedTypes are JSONL entry types that carry no parser-relevant data. var droppedTypes = map[string]bool{ "progress": true, }

... (remaining tests and function content) ...