# Use observed pack size to size post-rejection subdivisions

`d239127`→[main](/content/gh/entireio/git-sync/commits/main/index.html)·
Soph·2mo ago·2 files·+431 added/-21 removed

Bootstrap's batched-mode subdivide path was always halving the
remaining checkpoint count when a target rejected a pack with
HTTP 413. For a repo whose real per-object byte size is far above
the per-object heuristic (estimatedBytesPerObject = 750), this
forces a long dance: 1 → 2 → 4 → 8 → 16 → 32 sub-packs across five
rejected pushes, each preceded by a fresh source fetch. For a
~960 MiB pack against Cloudflare's ~500 MiB body cap, that's five
wasted ~500 MiB uploads.

Wrap the pack reader handed to PushPack in a packReadCounter so the
loop learns how many bytes actually went up before the cutoff and
sizes the next attempt accordingly. Three cooperating pieces:

1. observedSubdivisionFactor uses bytes_sent ÷ limit (with safety
   multiplier) to choose how many sub-packs to split into. When
   sent_bytes ≈ limit (server cut us off mid-stream — the common
   reverse-proxy 413 case), the true pack size is unknown but
   likely much larger than the cap, so the multiplier escalates to
   4× to converge in a single round. Otherwise 2× is enough.

2. subdivideToFactor halves at least once per call and keeps going
   while count < target. The unconditional first round matters
   when factor ≤ remaining: each surviving range may still
over-shoot so always making forward progress prevents a hard
failure on repeated 413s with sent_bytes ≈ limit.

3. checkPackSizeAndSubdivide now takes a calibrated bytesPerObject
   instead of using the static 750-byte default. After each 413,
   calibrateBytesPerObject derives 2 × sent_bytes ÷ pack_objects
   as a pessimistic upper bound and ratchets the running estimate
   up. Subsequent sub-packs are pre-emptively split when the
   calibrated estimate exceeds the limit — saving an entire
   ~limit-sized wasted upload on blob-heavy repos where the
default ratio is 10–20× too low.

Falls back to the legacy halving behaviour when no signal is
available (sentBytes ≤ 0 or limit ≤ 0). Surfaces sent_bytes,
limit_bytes, factor, and calibrated_bytes_per_object in the slog
output so verbose runs show the math.

## Sessions

6f5d837779f7View transcript

## Changes

2

- internal/strategy/bootstrap

- Mbootstrap.go+173/-18

- Mbootstrap_test.go+258/-3

```go
223 unmodified lines

// master and nocache-cleanup share ~99% of history).
completedRefs := planner.CopyRefHashMap(p.TargetRefs)

// calibratedBytesPerObject tracks the per-object byte estimate
// updated from observed rejected pushes. Starts at the static
// default (which under-counts blob-heavy repos) and ratchets up as
// 413s reveal that the real bytes/object are much higher than 750.
// Used in checkPackSizeAndSubdivide so subsequent sub-pack fetches
// are pre-emptively split when the calibrated estimate exceeds
// p.TargetMaxPack — saving an entire ~limit-sized wasted upload.
calibratedBytesPerObject := int64(estimatedBytesPerObject)

for _, batch := range batches {
    if batch.subsumed {
        cmds := []gitproto.PushCommand{{
            // code omitted
        }
    }
}
```