Default ref-update batch to 5000; make it env-tunable

88a1ebf→main·

Soph·4w ago·2 files·+43 added/-6 removed

GitHub returns 500 Internal Server Error when a single push updates ~10k refs at once but accepts 5k — well below entire-server's 25k cap. Lower the default batch to 5_000 so mirroring a many-ref repo works against GitHub out of the box, and add GITSYNC_MAX_REF_UPDATES_PER_PUSH to raise it for targets known to tolerate larger pushes (entire-server, up to 25k) and cut round trips.

Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com

Sessions

7f01a5ec2f19View transcript

[?
We have a repo with 50k refs and we go this error: 2026/06/18 13:17:39 WARN Permanent sync error, terminating error="sync: replicate relay failed: replicate execute: push target refs: target receive-pClaude Code·Opus 4.8[1m]·5 steps](/content/gh/entireio/git-sync/session/76731dd6-65de-4d92-be87-716b3e380099#timeline-7f01a5ec2f19/index.html)

Changes

2

• internal/gitproto

• Mpush.go+29/-6

  • Mpush_test.go+14

8 unmodified lines
9
10
11
12
13
14
15
35 unmodified lines

51
52
53
53
54
55
56
57
54
55
56
57
58
59
60
61
62
63
64
65
66
64
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90

8 unmodified lines

"io"
    "os"
    "slices"
    "strconv"
    "strings"
    "sync/atomic"
    "time"
35 unmodified lines

return &Pusher{Conn: conn, Adv: adv, Verbose: verbose}
}

// maxRefUpdatesPerPush bounds how many ref-update commands ride in a single
// receive-pack request. entire-server rejects a push carrying more than 25_000
// commands (server/githttp.maxRefUpdateCommands), and other servers may impose
// their own caps; staying well under that lets a sync of a many-ref repo split
// across several pushes instead of failing outright.
// defaultMaxRefUpdatesPerPush bounds how many ref-update commands ride in a
// single receive-pack request. The default is deliberately conservative:
// GitHub returns 500 Internal Server Error when a single push updates ~10k refs
// at once but accepts 5k, so 5_000 mirrors a many-ref repo there without
// tripping its (undocumented) ceiling. entire-server tolerates far more — its
// hard cap is 25_000 (server/githttp.maxRefUpdateCommands) — so trusted callers
// pushing to entire-server raise this via MaxRefUpdatesEnv to cut round trips.

const maxRefUpdatesPerPush = 20_000
const defaultMaxRefUpdatesPerPush = 5_000

// MaxRefUpdatesEnv overrides defaultMaxRefUpdatesPerPush with a positive
// integer. Raise it for targets known to accept large ref-update pushes (e.g.
// entire-server, up to its 25_000 cap) to reduce round trips; lower it for a
// provider that rejects even the default. Invalid or non-positive values fall
// back to the default.
const MaxRefUpdatesEnv = "GITSYNC_MAX_REF_UPDATES_PER_PUSH"

// maxRefUpdatesPerPush is resolved once from the environment so the limit can be
// tuned per target without rebuilding (see MaxRefUpdatesEnv).
var maxRefUpdatesPerPush = resolveMaxRefUpdatesPerPush()

func resolveMaxRefUpdatesPerPush() int {
    if v := os.Getenv(MaxRefUpdatesEnv); v != "" {
        if n, err := strconv.Atoi(v); err == nil && n > 0 {
            return n
        }
    }
    return defaultMaxRefUpdatesPerPush
}

// chunkRefUpdates splits commands into batches no larger than
// maxRefUpdatesPerPush. Input that already fits is returned as a single batch

Minternal/gitproto/push.go+29/-6

789 unmodified lines

790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809

789 unmodified lines

return cmds
}

func TestResolveMaxRefUpdatesPerPush(t *testing.T) {
    t.Setenv(MaxRefUpdatesEnv, "")
    require.Equal(t, defaultMaxRefUpdatesPerPush, resolveMaxRefUpdatesPerPush())

t.Setenv(MaxRefUpdatesEnv, "20000")
    require.Equal(t, 20000, resolveMaxRefUpdatesPerPush())

// Invalid or non-positive values fall back to the default.
    for _, bad := range []string{"0", "-5", "lots"} {
        t.Setenv(MaxRefUpdatesEnv, bad)
        require.Equal(t, defaultMaxRefUpdatesPerPush, resolveMaxRefUpdatesPerPush(), "value %q", bad)
    }
}

func TestChunkRefUpdates(t *testing.T) {
    require.Len(t, chunkRefUpdates(nil), 1)
    require.Len(t, chunkRefUpdates(make([]PushCommand, maxRefUpdatesPerPush)), 1)
}

Minternal/gitproto/push_test.go+14