# Use projected pack size for factor & objectsSent for calibration

`21a2ee5`→[main](/content/gh/entireio/git-sync/commits/main/index.html)·

Soph·2mo ago·1 file·+34 added/-3 removed

Two refinements that fall out of having the streaming pack observer feeding live counters during the upload.

1. observedSubdivisionFactor was being called with sentBytes, which after a self-imposed early abort is just the abort-point floor (~minBytesBeforeAbort = 8 MiB) — far below the actual pack size. The factor came out as 2 every round, so the loop doubled the pack count one step at a time (1→2→4→…) instead of making informed jumps. When abortedEarly, project from observed bytes/object to the full pack size (sentBytes × totalObjects ÷ objectsSent) and feed that to the factor calculation. For a blob-front-loaded repo the first round now jumps 1→8 instead of 1→2.

2. calibrateBytesPerObject was dividing sentBytes by the full pack header object count, which understates the per-object byte size when the upload only covered the front of the pack. Use objectsSent (the count actually observed by the streaming parser) instead — sentBytes/objectsSent is the accurate per-object average for the portion we saw, and that's the pessimistic upper bound the pre-flight wants. For the user's scenario this jumped the calibrated estimate from "no update" (8M÷65k = 256 < default 750) to 29 KiB/object, which is what actually catches subsequent oversized sub-packs in the pre-flight check on the next attempt.

## Sessions

3cd79b536633View transcript

## Changes

1

- internal/strategy/bootstrap

- Mbootstrap.go+34/-3

```
450 unmodified lines

451
452
453
454
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
460
473
474
475
476
477
478
8 unmodified lines

487
488
489
475
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
3 unmodified lines

512
513
514
515
516
517
518

450 unmodified lines

// Calibrate before subdividing. The new value carries
// over to the next iteration's pre-flight check, so a
// blob-heavy repo's sub-packs get caught earlier.
if updated := calibrateBytesPerObject(sentBytes, packObjectCount, calibratedBytesPerObject); updated > 0 {
// When we have an objectsSent observation from the
// streaming parser, divide by THAT rather than the
// full pack header count. sentBytes covers exactly
// objectsSent objects (the front of the pack), so
// sentBytes/objectsSent is the accurate per-object
// average for the portion we actually observed —
// and for blob-front-loaded repos that's a pessimistic
// upper bound, which is what we want for pre-flight.
calibrationDenom := packObjectCount
if objectsSent > 0 && objectsSent < calibrationDenom {
calibrationDenom = objectsSent
}
if updated := calibrateBytesPerObject(sentBytes, calibrationDenom, calibratedBytesPerObject); updated > 0 {
p.log("bootstrap batch calibrated bytes-per-object",
"branch", batch.Plan.TargetRef.String(),
"previous_bytes_per_object", calibratedBytesPerObject,
"observed_bytes_per_object", updated,
"sent_bytes", sentBytes,
"object_count", packObjectCount)
"calibration_denom", calibrationDenom,
"object_count", packObjectCount,
"objects_sent", objectsSent)
calibratedBytesPerObject = updated
}
// Refine the self-imposed budget from observation:

8 unmodified lines

factor := observedSubdivisionFactor(sentBytes, limit)
// Pick the byte count we use for sizing the next
// subdivision. When the server cut us off, sentBytes
// is roughly the cap and using it directly is right.
// When *we* cut the upload early, sentBytes is just
// the abort point (~minBytesBeforeAbort) — much less
// than the real pack size — so the factor would
// converge slowly. Project from observed bytes/object
// to the full pack size when we have the data, so
// factor reflects the real overshoot.
sizingBytes := sentBytes
if abortedEarly && objectsSent > 0 && totalObjects > 0 {
if projected := sentBytes * totalObjects / objectsSent; projected > sizingBytes {
sizingBytes = projected
}
}
factor := observedSubdivisionFactor(sizingBytes, limit)
expanded := subdivideToFactor(batch.chain, current, batch.Checkpoints[idx:], factor)
if len(expanded) > len(batch.Checkpoints[idx:]) {
oldRemaining := len(batch.Checkpoints[idx:])

"old_remaining", oldRemaining,
"new_remaining", newCount,
"sent_bytes", sentBytes,
"sizing_bytes", sizingBytes,
"limit_bytes", limit,
"factor", factor,
"aborted_early", abortedEarly,
```

Minternal/strategy/bootstrap/bootstrap.go+34/-3