gitproto: use go-git's WithObjectSelector to stream materialized push

38c6b8d→main·Soph·1mo ago·5 files·+125 added/-153 removed

Replaces the temp-file spool in PushObjects with the streaming approach unlocked by go-git PR #2142 (merged 2026-05-21):

• Run packfile.DeltaSelector.ObjectsToPack synchronously up front, showing "selecting deltas, elapsed X" progress (no byte counter here — the selector is opaque, elapsed time is the only signal).
• Construct a packfile.Encoder with WithObjectSelector pointing at a local precomputedSelector that returns the pre-selected []*ObjectToPack. Encoder.Encode then skips its own selection step and runs the write phase only.
• Pipe encoder output through io.Pipe into sendReceivePack, which streams it to the receive-pack POST as chunked transfer encoding.

The mid-stream stall that originally caused "use of closed network connection" against Cloudflare's git frontend is gone because selection no longer happens between the request body opening and pack bytes flowing — it has already completed.

Removed:

• SpooledBody, NewSpooledBody, (*SpooledBody).applyTo
• postReceivePack helper (used to bypass sendReceivePack's body construction; no longer needed now that we hand sendReceivePack a streaming pipe again)
• The SpooledBody type-assertion in PostRPCStreamBody

What we lose vs. the spool:

• Content-Length on the request (chunked again)
• req.GetBody-based retry
• No more $TMPDIR requirement proportional to pack size

What we gain:

• Pack bytes flow without ever landing on local disk
• Materialized push is streaming end-to-end (within the constraint that the source object closure already has to be local)
• Smaller surface in gitproto

Bumped go-git to a pseudo-version pinning the merge commit (v6.0.0-alpha.4.0.20260521151600-590487407c38). Will switch to a tagged release once one is cut.

TestPushObjectsBuffersBody → TestPushObjectsStreamsBody now asserts chunked encoding and Content-Length=-1 (unknown), the inverse of what we previously asserted.

Sessions

680da8552908View transcript

Changes

5

• Mgo.mod+1/-1

• Mgo.sum+2/-2

• internal/gitproto

  • Mpush.go+98/-63
  • Mpush_test.go+23/-15
  • Msmarthttp.go+1/-72

require (
    github.com/go-git/go-billy/v6 v6.0.0-alpha.1
    github.com/go-git/go-git/v6 v6.0.0-alpha.4
    github.com/go-git/go-git/v6 v6.0.0-alpha.4.0.20260521151600-590487407c38
    github.com/spf13/cobra v1.10.2
    github.com/stretchr/testify v1.11.1
    github.com/zalando/go-keyring v0.2.8
)

Mgo.mod+1/-1

github.com/go-git/go-billy/v6 v6.0.0-alpha.1/go.mod h1:eaCUpHbedW7//EwcYmUDfJe2N6sJC9O12AT0OTqJR1E=
github.com/go-git/go-git-fixtures/v6 v6.0.0-alpha.1 h1:gmqi2jvsreu0s8JMLylYDFq4sbjHwwlhktMw0DUg3mA=
github.com/go-git/go-git-fixtures/v6 v6.0.0-alpha.1/go.mod h1:ECf1MqJlBdYpKggBrOXjo/0EnvRZx6D++I86UYjPgAQ=
github.com/go-git/go-git/v6 v6.0.0-alpha.4 h1:aDTc2UGanmaE7FkGLSlBEB9nohMnQ+RKXcfq/D+esDQ=
github.com/go-git/go-git/v6 v6.0.0-alpha.4/go.mod h1:4ODa/G7hPWrh4Y+7lmt59Ij3zW38IEfvRoAZxLYYBhc=
github.com/go-git/go-git/v6 v6.0.0-alpha.4.0.20260521151600-590487407c38 h1:uA2L2RZQTkmvHjzBqMNMFR+UWdjicJBc0UqhCrgodZs=
github.com/go-git/go-git/v6 v6.0.0-alpha.4.0.20260521151600-590487407c38/go.mod h1:4ODa/G7hPWrh4Y+7lmt59Ij3zW38IEfvRoAZxLYYBhc=
github.com/godbus/dbus/v5 v5.2.2 h1:TUR3TgtSVDmjiXOgAAyaZbYmIeP3DPkld3jgKGV8mXQ=
github.com/godbus/dbus/v5 v5.2.2/go.mod h1:3AAv2+hPq5rdnr5txxxRwiGjPXamgoIHgz9FPBfOp3c=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=

Mgo.sum+2/-2

// some unmodified lines omitted for brevity
if packData != nil {
    body = io.MultiReader(body, packData)
}
return postReceivePack(ctx, conn, req, body, verbose, onRejection)
}
// postReceivePack POSTs an already-built receive-pack request body and decodes the response. Split from sendReceivePack so the materialized push path can construct a spooled body (header + pack in one temp file) and reuse the response handling.

...

// PushObjects pushes locally-materialized objects to the target.
func PushObjects(
    ctx context.Context,
    conn Conn,
    ...) error {
    // implementation logic...
}

Test Methods

type observation struct {
    transferEncoding []string
    contentLength    int64
}

func TestPushObjectsBuffersBody(t *testing.T) {
    // test logic...
}
...

func TestPushObjectsStreamsBody(t *testing.T) {
    // assert streaming
}

The SpooledBody Struct

type SpooledBody struct {
    io.ReadCloser
    path string
    size int64
}

func NewSpooledBody(write func(io.Writer) error) (*SpooledBody, func(), error) {
    // Implementation...
}