# Free commit graph store immediately after extracting first-parent chain

`05d5f1a`→[main](/content/gh/entireio/git-sync/commits/main/index.html)·

Soph·3mo ago·2 files·+38 added/-0 removed

The commit graph fetch for linux stores 1.4M commit objects in memory.NewStorage() (~4.6 GB). We only need the 75k first-parent chain hashes (~3 MB) for checkpoint placement. Previously the store stayed alive for the entire bootstrap run because the function scope held it.

Now: extract the chain, nil the store, force GC. The 4.6 GB becomes garbage-collectible immediately after chain extraction (~22s into the run) instead of persisting for the full ~17 minutes.

The transient spike during the fetch is unavoidable with current tools:
- git protocol v2 has no "first-parent only" fetch command
- tree:0 filter gives all 1.4M commits, not just the 75k first-parent
- go-git's pack parser needs the full store for delta resolution between commits during pack decoding

A streaming parent-map extractor could eliminate the spike entirely but requires go-git packfile internals work (custom storer or scanner integration). The GC approach is the pragmatic fix.

Also adds FirstParentChainFromMap to planner/checkpoint.go for future use when a streaming extractor is implemented.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

## Sessions

56d371ce68d4View transcript

## Changes

2

- internal
  
  - planner  
    - Mcheckpoint.go+28
  
  - strategy/bootstrap  
    - Mbootstrap.go+10

```go
41 unmodified lines

return chain, nil
}

// FirstParentChainFromMap walks a first-parent map from tip back to root.
// The map key is a commit hash, the value is its first parent hash.
// A zero-value parent marks the root. Returns the chain in root-to-tip order.
func FirstParentChainFromMap(parents map[plumbing.Hash]plumbing.Hash, tip plumbing.Hash) ([]plumbing.Hash, error) {
	chain := make([]plumbing.Hash, 0, len(parents))
	current := tip
	seen := make(map[plumbing.Hash]struct{}, len(parents))
	for {
		if _, ok := seen[current]; ok {
			return nil, fmt.Errorf("cycle detected at %s", current)
		}
		seen[current] = struct{}{}
		chain = append(chain, current)
		parent, ok := parents[current]
		if !ok {
			return nil, fmt.Errorf("commit %s not found in parent map", current)
		}
		if parent.IsZero() {
			break
		}
		current = parent
		}
	for i, j := 0, len(chain)-1; i < j; i, j = i+1, j-1 {
		chain[i], chain[j] = chain[j], chain[i]
	}
	return chain, nil
}

// SampledCheckpointCandidates generates a set of candidate indices to probe,
// sorted from largest (preferred) to smallest.
func SampledCheckpointCandidates(lo, hi int, prevSpan int) []int {
```

Minternal/planner/checkpoint.go+28

```go
11 unmodified lines

func planCheckpointsFromChain(ctx context.Context, p Params, ref planner.DesiredRef) ([]plumbing.Hash, []plumbing.Hash, error) {
	p.log("bootstrap batch fetching commit graph", "branch", ref.TargetRef.String())

// Fetch all commits (tree:0 filter) into a temporary in-memory store.
	// For linux this is ~1.4M commits at ~3.3 KiB each = ~4.6 GB transient.
	// We extract the first-parent chain (~75k hashes, ~3 MB) immediately
	// and discard the store so GC can reclaim the 4.6 GB. The transient
	// spike is unavoidable with git protocol v2 (no first-parent-only
	// fetch) and go-git's pack parser (needs full store for delta resolution).
	graphStore := memory.NewStorage()
	gpRef := gitproto.DesiredRef{SourceRef: ref.SourceRef, TargetRef: ref.TargetRef, SourceHash: ref.SourceHash}
	if err := p.SourceService.FetchCommitGraph(ctx, graphStore, p.SourceConn, gpRef); err != nil {
		return nil, nil, fmt.Errorf("fetch bootstrap planning graph for %s: %w", ref.TargetRef, err)
	}
	chain, err := planner.FirstParentChain(graphStore, ref.SourceHash)
	graphStore = nil // allow GC to reclaim the commit graph store (~4.6 GB for linux)
	runtime.GC()
	if err != nil {
		return nil, nil, fmt.Errorf("walk first-parent chain for %s: %w", ref.TargetRef, err)
	}
}
```

Minternal/strategy/bootstrap/bootstrap.go+10