# Stream-parse the pack as it uploads to expose objects-sent counter

`242ad00`→[main](/content/gh/entireio/git-sync/commits/main/index.html)·

Soph·2mo ago·4 files·+339 added/-29 removed

Replaces packReadCounter with packStreamObserver, a wrapping ReadCloser that adds two new instruments to the byte counter:

- **ObjectsSent()** — objects whose zlib stream has been fully walked by us (and therefore fully handed off to the HTTP layer)
- **TotalObjects()** — pack-header object count, populated as soon as the first 12 bytes flow through

The implementation tees the pack via io.Pipe to a goroutine running go-git's packfile.Scanner, which sequentially emits HeaderSection, one ObjectSection per object, and FooterSection. Each object increments the atomic; each header populates TotalObjects and closes HeaderReady so callers can wait without polling.

The current bootstrap loop only logs the new counters (objects_sent, total_objects_in_pack) on push failures so verbose runs can sanity-check the parse — no behaviour change yet. The next commit uses these counters to abort uploads early once we project we'll exceed the budget.

Cost note: Scanner does full zlib decompression to find object boundaries (compressed size isn't recorded in the pack format), plus a per-object SHA-1 over the inflated bytes. On modern hardware this runs at 200–500 MB/s per core, well above typical upload speeds, so the observer adds CPU but should not bottleneck the upload. If profiling later shows otherwise, the escape hatch is a custom format walker that drops Scanner's hashing — but the zlib walk itself is unavoidable.

## Sessions

a66652708dc7View transcript

## Changes

4

- internal/strategy/bootstrap
  - Mbootstrap.go+8/-23
  - Mbootstrap_test.go+12/-6
  - Apack_observer.go+153
  - Apack_observer_test.go+166

```go
400 unmodified lines

// bytesPerObject lets the caller use a per-run calibrated value instead of the static estimatedBytesPerObject default. Calibrating after each rejection (using the bytes that flowed through packReadCounter) catches blob-heavy repos where the static 750-byte average is 10–20× too low — without calibration the pre-flight would let oversized sub-packs through and the loop would only learn after another wasted ~limit-sized upload. The subdivide callback
```