sha256convert: exclude foreign pull/merge-request refs from --all-refs · Entire
sha256convert: exclude foreign pull/merge-request refs from --all-refs
49ecbac→main·
nodo·1mo ago·4 files·+238 added/-15 removed
refs/pull/* (GitHub/Gitea), refs/pull-requests/* (Bitbucket), and refs/merge-requests/* (GitLab) hold code proposed from forks and other branches — foreign to the repository until merged. --all-refs pulled them in, and since the converted repo is typically mirrored onward with git push --mirror, a destination forge that doesn't treat those namespaces as read-only PR refs could republish unreviewed code as repo content.
Exclude those namespaces from --all-refs by default; --include-pull-refs opts back in (e.g. for a faithful archival mirror). The run reports how many such refs it dropped (status line + Result.SkippedPullRefs) so an --all-refs conversion never silently omits them.
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
Sessions
40a74bd2030dView transcript
[?
can you look at the comments on https://github.com/entireio/git-sync/pull/66 and address them?Claude Code·Opus 4.8[1m]·1 step](/content/gh/entireio/git-sync/session/ffbceeb3-b71f-4055-b943-2bb07e2d6b03#timeline-40a74bd2030d/index.html)
Changes
4
cmd/git-sync
Mconvert_sha256.go+12/-5
internal/sha256convert
Msha256convert.go+75/-5
- Msha256convert_test.go+128
docs
Mconvert-sha256.md+23/-5
24 unmodified lines
25
26
27
28
29
30
31
32
28
29
30
31
32
33
34
35
36
37
38
39
40
46 unmodified lines
87
88
89
90
91
92
93
94
24 unmodified lines
All branches and tags on the source are always converted — partial scope risks stranding cross-branch references in commit messages. Pass --all-refs to also include refs/notes/*, refs/pull/*, and other custom namespaces; pass --exclude-ref-prefix to subtract specific namespaces from --all-refs. Exclude prefixes that would drop any branch or tag (e.g. refs/heads/feature/, refs/tags/, refs/) are rejected at run time to preserve the always-convert invariant. --all-refs to also include refs/notes/* and other custom namespaces; pass --exclude-ref-prefix to subtract specific namespaces from --all-refs. Exclude prefixes that would drop any branch or tag (e.g. refs/heads/feature/, refs/tags/, refs/) are rejected at run time to preserve the always-convert invariant.
Server-internal pull/merge-request refs (refs/pull/*, refs/pull-requests/*, refs/merge-requests/*) are NOT converted even under --all-refs: they hold unmerged code foreign to the repository, and mirroring the result onward would republish it. Pass --include-pull-refs to convert them anyway.
The conversion is destructive in two ways the caller should be aware of:
GPG signatures on commits and tags are dropped (they sign over the
46 unmodified lines
allRefsFlag(cmd, allRefsUsageScopeOnly, &req.AllRefs)
excludeRefPrefixFlag(cmd, &req.ExcludeRefPrefixes)
cmd.Flags().BoolVar(&req.IncludePullRefs, "include-pull-refs", false,
"with --all-refs, also convert server-internal pull/merge-request refs (refs/pull/*, refs/pull-requests/*, refs/merge-requests/*); off by default because they hold unmerged foreign code")
addProtocolFlag(cmd, &protocolVal)
cmd.Flags().BoolVarP(&req.Verbose, "verbose", "v", false, "verbose logging")
cmd.Flags().BoolVar(&req.Progress, "progress", false,
Mcmd/git-sync/convert_sha256.go+12/-5
52 unmodified lines
53
54
55
56
57
56
57
58
59
60
61
62
3 unmodified lines
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
47 unmodified lines
130
131
132
133
134
135
136
53 unmodified lines
190
191
192
193
194
195
196
197
198
103 unmodified lines
302
303
304
288
289
305
306
307
308
309
310
293
311
312
313
314
11 unmodified lines
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
404 unmodified lines
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
52 unmodified lines
// Scope is intentionally fixed: every branch and every annotated/lightweight
// tag on the source is always converted. Partial scope risks stranding
// cross-branch references in commit messages, which defeats the point of a
// one-off cutover. AllRefs additionally pulls in refs/notes, refs/pull, and
// other custom namespaces; ExcludeRefPrefixes subtracts from that.
// one-off cutover. AllRefs additionally pulls in refs/notes and other custom
// namespaces; ExcludeRefPrefixes subtracts from that. Server-internal
// pull/merge-request namespaces are excluded from AllRefs by default (see
// IncludePullRefs) because they carry unmerged foreign code.
type Request struct {
SourceURL string
SourceAuth gitsync.EndpointAuth
3 unmodified lines
AllRefs bool
ExcludeRefPrefixes []string
// IncludePullRefs opts back into converting the server-internal
// pull/merge-request namespaces (refs/pull/*, refs/pull-requests/*,
// refs/merge-requests/*) that AllRefs would otherwise pull in. They are
// excluded by default: those refs hold code proposed from forks
// and other branches — foreign to the repository until merged — and
// the converted repo is typically mirrored onward with
// `git push --mirror`, where a destination forge may surface them as
// ordinary refs and republish unreviewed code as repo content. No
// effect without AllRefs (the namespaces are out of scope anyway).
IncludePullRefs bool
ProtocolMode gitsync.ProtocolMode
Verbose bool
Progress bool
47 unmodified lines
SignaturesStripped int `json:"signaturesStripped"`
MessageRewrites int `json:"messageRewrites"`
AmbiguousMessageRefs []string `json:"ambiguousMessageRefs,omitempty"`
SkippedPullRefs int `json:"skippedPullRefs,omitempty"`
OriginNotesRef string `json:"originNotesRef,omitempty"`
MappingFile string `json:"mappingFile,omitempty"`
SignedTags []string `json:"signedTags,omitempty"`
53 unmodified lines
}
lines = append(lines, line)
}
if r.SkippedPullRefs > 0 {
lines = append(lines, fmt.Sprintf("excluded %d foreign pull/merge-request ref(s) (refs/pull/*, refs/pull-requests/*, refs/merge-requests/*) from --all-refs; pass --include-pull-refs to convert them", r.SkippedPullRefs))
}
if r.OriginNotesRef != "" {
lines = append(lines, fmt.Sprintf("origin notes ref: %s (use `git notes --ref=%s show <sha256>` to recover old SHA1)",
r.OriginNotesRef, strings.TrimPrefix(r.OriginNotesRef, "refs/notes/")))
}
103 unmodified lines
// Source connection + ref discovery -----------------------------------
// Scope is fixed: always include every branch and every tag. AllRefs
// extends to refs/notes/*, refs/pull/*, and other namespaces;
// ExcludeRefPrefixes can subtract from that under AllRefs.
// extends to refs/notes/* and other namespaces; ExcludeRefPrefixes
// can subtract from that under AllRefs. Pull/merge-request namespaces
// are excluded by default (foreign code) unless --include-pull-refs.
planCfg := planner.PlanConfig{
IncludeTags: true,
AllRefs: req.AllRefs,
ExcludeRefPrefixes: append([]string(nil), req.ExcludeRefPrefixes...),
ExcludeRefPrefixes: effectiveExcludePrefixes(req.ExcludeRefPrefixes, req.AllRefs, req.IncludePullRefs),
}
conn, refService, sourceRefList, err := openSource(ctx, req, planCfg)
if err != nil {
11 unmodified lines
return res, errors.New("no source refs matched the requested scope")
}
// Surface how many pull/merge-request refs the default exclusion
// dropped, so an --all-refs run doesn't silently omit them. Only
// meaningful when we actually excluded them.
if req.AllRefs && !req.IncludePullRefs {
if skipped := countForeignPullRefs(sourceRefs); skipped > 0 {
res.SkippedPullRefs = skipped
fmt.Fprintf(out, "excluding %d foreign pull/merge-request ref(s) from --all-refs (pass --include-pull-refs to convert them) ...\n", skipped)
}
}
// Refuse before any further I/O if the source carries refs that
// would collide with our side outputs. writeRefs runs before
// writeOriginNotes / signBranchTips, so without this check the
404 unmodified lines
SignModeTips = "tips"
// foreignPullRefPrefixes are the server-internal pull/merge-request
// namespaces that hold code proposed from forks and other branches —
// content foreign to the repository's own history until merged. They are
// excluded from --all-refs by default: the converted repo is usually
// mirrored onward with `git push --mirror`, and a destination forge may
// surface these refs as ordinary refs, republishing unreviewed code as if
// it were part of the repo. --include-pull-refs opts back in.
var foreignPullRefPrefixes = []string{
"refs/pull/", // GitHub, Gitea, Forgejo
"refs/pull-requests/", // Bitbucket Server / Data Center
"refs/merge-requests/", // GitLab
}
// effectiveExcludePrefixes combines the user's --exclude-ref-prefix values
// with the default pull/merge-request exclusions. The latter only apply
// under --all-refs (the namespaces are out of scope otherwise) and only
// when the user did not pass --include-pull-refs.
func effectiveExcludePrefixes(userPrefixes []string, allRefs, includePullRefs bool) []string {
out := append([]string(nil), userPrefixes...)
if allRefs && !includePullRefs {
out = append(out, foreignPullRefPrefixes...)
}
return out
}
// countForeignPullRefs reports how many source refs fall under a
// pull/merge-request namespace, so the run can tell the user exactly how
// many refs the default exclusion dropped rather than silently omitting
// them from an --all-refs conversion.
func countForeignPullRefs(refs map[plumbing.ReferenceName]plumbing.Hash) int {
n := 0
for name := range refs {
for _, p := range foreignPullRefPrefixes {
if strings.HasPrefix(string(name), p) {
n++
break
}
}
}
return n
}
// protectedExcludePrefixes returns the subset of prefixes that, under
// planner.IsRefExcluded's string-prefix semantics, would knock out at
// least one branch or tag. A prefix matches a branch if either side
Mcmd/git-sync/internal/sha256convert/sha256convert.go+75/-5
954 unmodified lines
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
109 unmodified lines
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
954 unmodified lines
}
}
// TestRun_GitHTTPBackend_PullRefs verifies that --all-refs excludes
// pull/merge-request namespaces by default — including the foreign commit
// that only a pull ref makes reachable — and that --include-pull-refs opts
// back in.
func TestRun_GitHTTPBackend_PullRefs(t *testing.T) {
if os.Getenv(gitHTTPBackendEnv) == "" {
t.Skipf("set %s=1 to run the convert-sha256 git-http-backend integration test", gitHTTPBackendEnv)
}
gitBin, err := exec.LookPath("git")
if err != nil {
t.Skipf("git binary not available: %v", err)
}
root := t.TempDir()
srcBare := filepath.Join(root, "source.git")
worktree := filepath.Join(root, "work")
mustGit(t, root, "init", "--bare", srcBare)
mustGit(t, root, "init", "-b", "main", worktree)
mustGit(t, worktree, "config", "user.name", "convert-sha256 test")
mustGit(t, worktree, "config", "user.email", "test@example.com")
mustWrite(t, filepath.Join(worktree, "README"), "hello\n")
mustGit(t, worktree, "add", "README")
mustGit(t, worktree, "commit", "-m", "initial")
mustGit(t, worktree, "remote", "add", "origin", srcBare)
mustGit(t, worktree, "push", "origin", "HEAD:refs/heads/main")
// A server-internal PR ref whose tip is foreign: the commit is on no
// branch, so only refs/pull/1/head makes it reachable.
mustWrite(t, filepath.Join(worktree, "evil.txt"), "foreign\n")
mustGit(t, worktree, "add", "evil.txt")
mustGit(t, worktree, "commit", "-m", "foreign PR commit")
mustGit(t, worktree, "push", "origin", "HEAD:refs/pull/1/head");
srv := newCGIBackend(t, gitBin, root)
defer srv.Close()
// Default: --all-refs must drop refs/pull/* and report the count.
defaultDir := filepath.Join(root, "default.git")
res, err := Run(context.Background(), Request{
SourceURL: srv.URL + "/source.git",
TargetDir: defaultDir,
AllRefs: true,
Out: io.Discard,
})
if err != nil {
t.Fatalf("convert (default): %v", err)
}
if res.SkippedPullRefs != 1 {
t.Errorf("SkippedPullRefs (default): got %d, want 1", res.SkippedPullRefs)
}
if refs := mustGitOutput(t, defaultDir, "for-each-ref", "--format=%(refname)"); strings.Contains(refs, "refs/pull/") {
t.Errorf("default --all-refs conversion leaked a pull ref:\n%s", refs)
}
// Opt-in: --include-pull-refs converts refs/pull/* and the foreign tip.
inclDir := filepath.Join(root, "incl.git")
res2, err := Run(context.Background(), Request{
SourceURL: srv.URL + "/source.git",
TargetDir: inclDir,
AllRefs: true,
IncludePullRefs: true,
Out: io.Discard,
})
if err != nil {
t.Fatalf("convert (include): %v", err)
}
if res2.SkippedPullRefs != 0 {
t.Errorf("SkippedPullRefs (--include-pull-refs): got %d, want 0", res2.SkippedPullRefs)
}
if refs := mustGitOutput(t, inclDir, "for-each-ref", "--format=%(refname)"); !strings.Contains(refs, "refs/pull/1/head") {
t.Errorf("--include-pull-refs did not convert the pull ref:\n%s", refs)
}
}
func mustGit(t *testing.T, dir string, args ...string) {
t.Helper()
cmd := exec.CommandContext(t.Context(), "git", args...)
109 unmodified lines
}
}
func TestEffectiveExcludePrefixes(t *testing.T) {
user := []string{"refs/changes/"}
tests := []struct {
name string
allRefs bool
includePullRefs bool
wantPullExcluded bool
}{
{"without --all-refs the pull namespaces are not added", false, false, false},
{"--all-refs excludes pull namespaces by default", true, false, true},
{"--all-refs --include-pull-refs keeps them in scope", true, true, false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := effectiveExcludePrefixes(user, tt.allRefs, tt.includePullRefs)
// The user's own prefixes are always preserved.
if len(got) == 0 || got[0] != "refs/changes/" {
t.Fatalf("user prefixes not preserved: %v", got)
}
hasPull := false
for _, p := range got {
if p == "refs/pull/" {
hasPull = true
}
}
if hasPull != tt.wantPullExcluded {
t.Errorf("refs/pull/ excluded = %v, want %v (got %v)", hasPull, tt.wantPullExcluded, got)
}
})
}
}
func TestCountForeignPullRefs(t *testing.T) {
refs := map[plumbing.ReferenceName]plumbing.Hash{
"refs/heads/main": plumbing.NewHash("aaaa000000000000000000000000000000000000001"),
"refs/pull/1/head": plumbing.NewHash("aaaa000000000000000000000000000000000000002"),
"refs/pull/2/head": plumbing.NewHash("aaaa000000000000000000000000000000000000003"),
"refs/merge-requests/9/head": plumbing.NewHash("aaaa000000000000000000000000000000000000004"),
"refs/pull-requests/5/from": plumbing.NewHash("aaaa000000000000000000000000000000000000005"),
"refs/tags/v1": plumbing.NewHash("aaaa000000000000000000000000000000000000006"),
}
if got, want := countForeignPullRefs(refs), 4; got != want {
t.Errorf("countForeignPullRefs = %d, want %d", got, want)
}
}
// The default pull-ref exclusions must never trip the branch/tag
// protection guard — they live outside refs/heads/ and refs/tags/.
func TestForeignPullRefPrefixes_NotProtected(t *testing.T) {
if bad := protectedExcludePrefixes(foreignPullRefPrefixes); len(bad) != 0 {
t.Errorf("foreign pull-ref prefixes rejected by protectedExcludePrefixes: %v", bad)
}
}
func TestRun_RejectsExcludePrefixesThatDropBranchesOrTags(t *testing.T) {
// We never reach the network here — the validation fires before
// any I/O — so a non-empty target dir is the only thing the early
Mcmd/git-sync/internal/sha256convert/sha256convert_test.go+128
20 unmodified lines
21
22
23
24
25
26
27
24
25
26
27
28
29
30
31
32
145 unmodified lines
178
179
180
179
181
182
183
184
185
186
187
188
189
44 unmodified lines
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
20 unmodified lines
`refs/notes/sha1-origin` ref recording each commit's pre-conversion SHA1.
Scope is fixed: every branch and every tag on the source is always converted. Pass `--all-refs` to also include `refs/notes/*`, `refs/pull/*`, and other custom namespaces; pair with `--exclude-ref-prefix` to subtract specific namespaces (e.g. `--exclude-ref-prefix refs/pull/` on GitHub mirrors).
converted. Pass `--all-refs` to also include `refs/notes/*` and other custom namespaces; pair with `--exclude-ref-prefix` to subtract specific namespaces. Server-internal pull/merge-request refs (`refs/pull/*`, `refs/pull-requests/*`, `refs/merge-requests/*`) are excluded even under `--all-refs` — see [Sharp Edges](#sharp-edges). Pass `--include-pull-refs` to convert them anyway.
## Operational Notes
**One-off, not incremental.** Each run produces a fresh SHA256 repo