Include pack size estimates in subdivision notices · Entire
Include pack size estimates in subdivision notices
The "splitting N → M packs" notice told the user it happened but not how big the resulting packs would be. Surface the numbers we already have at each subdivision point:
Pre-push (header estimate): now reads "estimated pack
3.75 GB exceeds target limit 512 MB — splitting 1 → 8 packs (480 MB each)". The estimate flows out of checkPackSizeAndSubdivide via its subdivide callback signature.Post-rejection: parses the body limit out of the server error and includes it: "target rejected pack (target limit 256 MB) — splitting 2 → 4 packs". We don't know the actual rejected pack size from the error, but the limit is the more actionable number anyway.
One-shot → batched: already showed the new limit; left as-is.
Sessions
Changes
2
- internal/strategy/bootstrap
- Mbootstrap.go+23/-10
- Mbootstrap_test.go+8/-5
360 unmodified lines
var packObjectCount int64
if p.TargetMaxPack > 0 && len(batch.chain) > 0 {
subdivided := false
packReader, packObjectCount, err = checkPackSizeAndSubdivide(packReader, p.TargetMaxPack, calibratedBytesPerObject, func() bool {
packReader, packObjectCount, err = checkPackSizeAndSubdivide(packReader, p.TargetMaxPack, calibratedBytesPerObject, func(estimated int64) bool {
expanded := subdivideCheckpoints(batch.chain, current, batch.Checkpoints[idx:])
if len(expanded) > len(batch.Checkpoints[idx:]) {
oldRemaining := len(batch.Checkpoints[idx:])
newCount := len(expanded)
p.log("bootstrap batch subdividing before push (pack header estimate)",
"branch", batch.Plan.TargetRef.String(),
"old_remaining", oldRemaining,
"new_remaining", len(expanded),
"new_remaining", newCount,
"estimated_bytes", estimated,
"calibrated_bytes_per_object", calibratedBytesPerObject)
p.notice(fmt.Sprintf("pack would exceed target limit — splitting %d → %d packs",
oldRemaining, len(expanded)))
p.notice(fmt.Sprintf(
"estimated pack ~%s exceeds target limit %s — splitting %d → %d packs (~%s each)",
humanBytes(estimated), humanBytes(p.TargetMaxPack),
oldRemaining, newCount, humanBytes(perPack),
))
batch.Checkpoints = append(batch.Checkpoints[:idx], expanded...)
subdivided = true
return true
}
})
}
}
// packReadCounter) catches blob-heavy repos where the static 750-byte
// average is 10–20× too low — without calibration the pre-flight
// would let oversized sub-packs through and the loop would only learn
// after another wasted ~limit-sized upload.
// after another wasted ~limit-sized upload. The subdivide callback
// receives the estimated total bytes so user-facing messages can
// quote the projected size that triggered the split.
func checkPackSizeAndSubdivide(
r io.ReadCloser,
batchLimit int64,
bytesPerObject int64,
subdivide func() bool,
subdivide func(estimatedBytes int64) bool,
) (io.ReadCloser, int64, error) { //nolint:unparam // error return kept for future use
if bytesPerObject <= 0 {
bytesPerObject = estimatedBytesPerObject
}
}
This content reflects the necessary changes and technical specifications related to pack size estimates.