Multi-cell Fan-out and Cell Routing Infrastructure · Entire
Log in
can you take a look at this https://github.com/entireio/cli/pull/1616 and the cell/jurisdiction logic, then the logic for the expert command (already merged) and what was started here https://github.com/entireio/cli/pull/1592
You can also look at ../entiredb (which has the core logic) and ../entire-api (which has the cell logic)
I wonder if we should build the foundation correctly now before merging this one
1w ago·9m·6 calls
All three research agents are running — one mapping the CLI's existing cell/jurisdiction plumbing (experts + #1592), one dissecting #1616's fan-out implementation, and one reading the backend cell model in entiredb/entire-api plus the BFF fan-out pattern. I'll synthesize a recommendation on whether to build the shared foundation first once they report back.
Important meta-note first
The Copilot review overview is stale/misleading: it says the PR touches "2 out of 2 changed files" (only the codesearch package) and hits POST /search/api/search. The actual diff is 4 files (1,880 lines) and the client uses GET, not POST. The bulk of the PR — the multi-region orchestration in 500 lines) and its tests (~940 lines) — is not described by Copilot's summary at all. The two Copilot inline comments also appear to describe an earlier revision (details in section 5).search_cmd.go (
Diff file line numbers below refer to the unified diff from gh pr diff 1616.
1. Orchestration in search_cmd.go
Flag/query pre-processing (RunE, diff lines 319-391).
--case-sensitivewithout--codeis rejected (323-325).- Under
--code: rejects--author/--date/--branch/--page(327-340). extractInlineRepoFilters(query)(diff 445-462) extracts ONLYrepo:tokens, deliberately leavingauthor:/date:/branch:as literal search text (unlikesearch.ParseSearchInput).- Repo scoping logic (349-381): combines
--repo+ inlinerepo:values;repo:*or--all-repos→ no filter (search everything); otherwise if empty, falls back tocurrentRepoSlug(ctx)(reused fromrecap.go:236), erroring if it can't be determined. - Calls
runCodeSearch(ctx, cmd, codeSearchOpts{...}).
runCodeSearch (diff 467-498). Gate via codeSearchEnabled() (ENTIRE_CODE_SEARCH == "1", diff 417-420) → "code search is not yet available"; empty-query check; always delegates to searchAllCells; renders JSON (writeCodeSearchJSON) when --json or non-terminal, else writeCodeSearchText.
searchAllCells(ctx, opts) (*codesearch.SearchResponse, error) (diff 517-606) — the core fan-out, 5 steps:
- Build control-plane client via
newCodeSearchCoreClient()(→coreapi.New()); mapsauth.ErrNotLoggedInto a login hint.ListReposunder a 10s timeout; warns ifrepoIndex.Truncated. - If filters present,
resolveRepoFilters→ resolved ULIDs + narrowed index subset; errors "no matching repositories found" (with truncation hint) if nothing matches. groupReposByCell(indexRepos)→[]cellGroup; empty → empty response.- Step 3b:
ListClustersunder a 10s timeout (best-effort; on error logs a warning and falls back to jurisdiction routing). BuildsslugToClusterkeyed bystrings.ToLower(cl.Slug)and sets eachcells[i].baseURL = TrimRight(TrimSpace(cl.ApiUrl.Or("")), "/"). - Fan-out:
doSearch := searchCell(overridable viaopts.searchCellFntest seam). Single cell → serial; multiple → one goroutine per cell +sync.WaitGroup, writing into a pre-sizedresults[]codeSearchCellResultby index (no shared-mutation race). ThenmergeSearchResults.
groupReposByCell([]coreapi.RepoIndexEntry) []cellGroup (diff 610-628). Groups by lowercased/trimmed r.Cell (the grouping key, matching the BFF), carrying jurisdiction (lowercased r.Jurisdiction) and accumulating repoIDs (r.ID). One entry per distinct cell.
resolveRepoFilters(filters, repos) (repoIDs, matched) (diff 643-673). Builds byName/byID maps; for each filter strips only a gh/ prefix, then matches in BFF order id===filter || full_name===slug || full_name===filter; dedups by ID. Explicitly mirrors BFF code-search.ts lines 315-319.
searchCell(ctx, opts, cg) (*codesearch.SearchResponse, error) (diff 679-722). Per-cell 30s timeout (codeSearchCellTimeout, diff 465). Builds *auth.CellTarget: baseURL != "" → {BaseURL, Jurisdiction}; else jurisdiction != "" → {Jurisdiction}; else nil (home fallback). Calls auth.NewEntireAPICellClient(cellCtx, opts.insecureHTTP, target). Passes per-cell cg.repoIDs only when a filter is active. Calls codesearch.Search.
mergeSearchResults(ctx, limit, cells, results) (*codesearch.SearchResponse, error) (diff 734-853).
- Sums
Stats.TotalMatches/TotalFiles/ReposSearched;DurationMs = max("wall-clock = slowest cell"). - If
successCount == 0 && lastErr != nil→ error"code search failed: %w". - Tracks
failedJurisdictions(cell name, else jurisdiction, else "home"); logs partial-failure warning. sort.SliceStableby Score desc, tiebreak Repo→Path→Line (deterministic JSON).- Dedups results by key
repo\x00path\x00line:col; dedupsRepoStatsby repo summing counts. - Caps
Resultstolimit; setsmerged.FailedJurisdictions.
Output:writeCodeSearchJSON (diff 856-881) emits {query, results, total, stats, repo_stats, failed_jurisdictions} with [] for nil results; writeCodeSearchText (diff 889-913) is grep-style repo:path:line: contextline, truncating context lines >200 chars (maxContextLineLen, diff 886) with an ellipsis, plus a summary line and partial-failure warning.
2. The codesearch package (diff 1-125)
- Endpoint:
GET /api/v1/search/api/search?q=...&max_results=...&repo=...&case_sensitive=true— peregrine reached through the cell's entire-api gateway. Usesclient.Get(diff 91), not POST. Search(ctx, client *api.Client, req SearchRequest) (*SearchResponse, error)(diff 74).- Types:
SearchRequest{Query, Repos []string, MaxResults int, CaseSensitive bool};Stats{TotalMatches, TotalFiles, DurationMs, ReposSearched};RepoStats{Repo, MatchCount, FileCount};Result{Repo, Path, Line, Column, ContextBefore/Line/After, Score};SearchResponse{Query, Stats, RepoStats, Results, FailedJurisdictions}— the last field is documented as CLI-merge-layer-only, not from peregrine (diff 66-68). case_sensitiveis sent "optimistically" with aponytail:note (diff 81-84) since peregrine's proto doesn't define it yet.- Auth expectation: "The client must already be authenticated against the cell" (diff 73) — i.e. the caller supplies a cell-identity-token
api.Client(built byNewEntireAPICellClient). - Robustness: 8 MiB response cap (
maxResponseBytes, diff 21) viaio.LimitReader(body, max+1)+ overflow error; non-2xx wrapped in*api.HTTPErrorusingapi.ErrorResponse.Message(), falling back tostrings.TrimSpace(body)(diff 105-116).
3. Generic/reusable vs. search-specific
Generic (any future multi-cell command would need this — currently inlined in search_cmd.go):
cellGrouptype andgroupReposByCell(repo-index → per-cell groups).resolveRepoFilters(slug/ULID/gh/-prefix → ULIDs + cell narrowing, BFF-parity matching).- The
searchAllCellsfan-out skeleton: list repos → group → resolve clusterApiUrlviaListClusters/slugToCluster→ parallel per-cell dispatch withWaitGroupand index-safe result collection → single-vs-multi handling. - Per-cell client construction from
cellGroup→auth.CellTarget→NewEntireAPICellClient(searchCell's target-building switch). codeSearchCoreClientinterface +newCodeSearchCoreClientswap seam; per-cell/per-op timeouts; partial-failure tracking (FailedJurisdictions, "all cells failed" escalation) and the warn-and-continue policy.
Genuinely search-specific:
codesearch.Searchcall,SearchRequest/SearchResponse.- Score-based sort + result dedup key (
repo/path/line/col) and Stats/RepoStats merge semantics. - Output rendering (
writeCodeSearchText/writeCodeSearchJSON,maxContextLineLen). extractInlineRepoFilters,--code/--case-sensitiveflag gating,codeSearchEnabledenv gate.
All the "generic" logic lives inline in search_cmd.go with codeSearch-prefixed names, so a second multi-cell command would have to copy or refactor it.
4. Duplication with existing CLI code
There is substantial conceptual overlap with cmd/entire/cli/experts_cell_target.go, which already does cell resolution — but the PR does not reuse it:
- Parallel interface + seam pattern:
codeSearchCoreClient+newCodeSearchCoreClient(diff 437-443) mirrorexpertsCoreClient+newExpertsCoreClient(experts_cell_target.go:23-31) almost exactly. - Cluster→cell resolution is duplicated but keyed differently: experts uses
matchClusterByHost(matches oncluster.PublicUrlhost, from a repo'sClusterHost); the PR buildsslugToClusterkeyed oncluster.Slug. Both then do the sameTrimRight(TrimSpace(ApiUrl.Or("")),"/")+ToLower(TrimSpace(Jurisdiction))normalization (compare diff 577 vsexperts_cell_target.go:78-82). Both produce anauth.CellTarget{BaseURL, Jurisdiction}. - Single-repo vs multi-repo: experts resolves ONE repo's cell (
resolveExpertsCellTarget, used viaapi_client.go:63); the PR resolves ALL repos' cells from theListReposindex. Different scale, same underlying concern — no shared helper. - Mirror/placement filtering is NOT reused (and arguably a gap): experts carefully filters placements via
isActiveMirror/distinctActiveClusterHosts(excludes archived/failed/suspended, detects multi-region ambiguity). The PR ignoresRepoIndexEntry.Placementsentirely and groups solely on the singleRepoIndexEntry.Cellfield — so a repo mirrored across multiple cells is only searched in its indexCell, with no active/archived filtering. CellTargetconstruction differs: experts always supplies bothBaseURL+Jurisdiction(or nil); the PR adds an intermediate "jurisdiction-only target" tier (diff 694-695) when the cluster catalog lookup failed.- Timeouts re-declared:
codeSearchCellTimeout(30s), plus two 10s control-plane timeouts, vs experts'expertsCellResolveTimeout(5s) — no shared constant. - Properly reused:
currentRepoSlug(fromrecap.go),auth.NewEntireAPICellClient,auth.CellTarget,api.HTTPError/api.ErrorResponse,search.AllReposFilter— these are not duplicated.
5. Review comments
Only the Copilot bot reviewed (state COMMENTED); no human review comments, no issue comments. Two inline comments, both of which appear to already be satisfied by the current diff (likely written against an earlier revision):
codesearch.goline ~109: "When falling back to using the raw response body as the HTTPError message, the body isn't trimmed... Consider trimming and only setting Message when the trimmed body is non-empty." — The current code at diff 113-114 already doesif apiErr.Message == "" && len(body) > 0 { apiErr.Message = strings.TrimSpace(string(body)) }. So this reads as already-addressed/outdated.codesearch_test.goline ~129: the non-JSON error test "only asserts that an error occurred"; suggests asserting the body text surfaces and that it wraps*api.HTTPErrorwith status 502. — The currentTestSearch_NonJSONError(diff 237-260) already asserts bothstrings.Contains(err.Error(), "Bad Gateway")anderrors.As(err, &httpErr)withStatusCode == 502. Also already-addressed/outdated.
Net: no outstanding actionable review feedback; the substantive orchestration in search_cmd.go received no review coverage from Copilot at all.
1w ago·12.6s
The #1616 analysis is in — key finding: its multi-cell fan-out logic (group-by-cell, cluster catalog resolution, per-cell client construction, partial-failure handling) is all inlined in search_cmd.go and partially duplicates what the experts command already built, while skipping the placement/active-mirror filtering experts does. Still waiting on the other two agents (CLI cell-infra map and the entiredb/entire-api backend model) before I give you a recommendation.
Cell / Jurisdiction Routing Infrastructure Map (github.com/entireio/cli)
Repo state note: working tree is clean on main (HEAD 4398a44c1). PR 1592 is NOT merged here. Current main already contains the experts cell-routing stack (experts_cell_target.go, api_client.go, the isActiveMirror helper). PR 1592's net-new is the activity/recap wiring plus a few cell_data_api.go hardening fixes (details in §3). The gh pr diff for experts_cell_target.go is a stale-base artifact — that refactor already landed on main.
Layer overview (bottom to top)
| Layer | File | Responsibility |
|---|---|---|
| Control plane (coreapi) | internal/coreapi/client.go, cross_juris_transport.go |
Repo/mirror/cluster catalog. Cross-region reads via 421-follow + RFC 8693 (transparent). |
| Cell auth / token exchange | cmd/entire/cli/auth/cell_data_api.go |
Mint jurisdictional identity token, resolve which cell host to dial. |
| Data-API auth (BFF/apex) | cmd/entire/cli/auth/data_api.go |
Legacy path: bearer for the data API origin via /.well-known discovery. |
| CLI cell-target resolver | cmd/entire/cli/experts_cell_target.go |
Map a repo → its owning cluster host → cell apiUrl + jurisdiction. |
| CLI client factories | cmd/entire/cli/api_client.go |
NewAuthenticatedAPIClient (data) / NewAuthenticatedEntireAPICellClient (cell). |
| Commands | experts_cmd.go, api_cmd.go, activity_cmd.go, recap.go |
Decide repo identity, call a factory. |
1. cmd/entire/cli/auth/cell_data_api.go — the token-exchange + cell-resolution core
This is the single source of truth for "which cell host does a data-plane request dial, and what token does it carry."
Entry point
1
func NewEntireAPICellClient(ctx context.Context, insecureHTTP bool, target *CellTarget) (*api.Client, error) // :99
Steps (:99–:134):
resolveStoredCellSubject(:205) — deliberately ignoresENTIRE_TOKEN; resolves the active stored login context, discovers trusted login servers via/.well-known/entire-api.json, and refreshes the login JWT. Returns acellSubject{loginJWT, discoveredCore, dataOrigin, httpClient}(:183).targetJurisdiction(:300) →resolveJurisdiction(:314): theCellTarget.Jurisdictionoverride, else thehome_jurisdictionJWT claim. Validated againstjurisdictionLabelPattern = ^[a-z0-9]([a-z0-9-]{0,38}[a-z0-9])?$(:43) so the claim can only name a sibling DNS label, never inject host/scheme.jurisdictionCoreURL(:432) — the entire-core origin to run the exchange at. Precedence: loopback discovered core verbatim →ENTIRE_CORE_BASE_URL_TEMPLATE→https://{jurisdiction}.auth.<family>→ discovered core.<family>(entire.io/partial.to) comes fromenvironmentFamily(:397).resolveTargetCellBaseURL(:334) — the cell-host decision (precedence doc at:91):
target.BaseURLset → use it verbatim (the repo-scoped path).dataOriginis not a BFF (host contains.api.or is loopback,isBFFOrigin``:351) → keep verbatim.- else BFF/apex →
resolveCellAPIBaseURLhome-jurisdiction fallback.
jurisdictionAudience(:411) — theaud(https://{jurisdiction}.<family>), mirrors the BFF'sbuildAudience.exchangeJurisdictionToken(:562) — RFC 8693 exchange:httputil.TokenExchangeForm(loginJWT, audience, scope=openid)POSTed viahttputil.PostOAuthToken.JurisdictionIdentityScope = "openid"(:30) — identity semantics only, authorization is per-request server-side (COR-666).- Returns
api.NewClientWithBaseURL(token, cellBaseURL).
Jurisdiction → cell host mapping (aws-us-east-2.api.entire.io)
Two independent resolvers produce the cell apiUrl:
- Repo-scoped (preferred): CLI resolves it upfront in
experts_cell_target.go(§2) and passesCellTarget.BaseURL. - Home-jurisdiction fallback:
resolveCellAPIBaseURL(:508) hand-parsesGET /api/v1/clusters(clustersAPIPath``:33), filters rows byjurisdiction, prefersisDefault, returnsapiUrl. It hand-rolls the HTTP call rather than reusingcoreapi.ListClustersto avoid an import cycle (coreapi imports auth) — noted at:502–:507.
home_jurisdiction claim reader
1
func HomeJurisdictionFromLoginJWT(loginJWT string) (string, error) // :478
Decodes JWT payload without signature verification (routing only; server re-verifies). Shared with git-remote-entire. Returns "" (no error) when absent.
CellTarget
1
2
3
4
type CellTarget struct { // :51
BaseURL string // e.g. https://aws-eu-west-1.api.entire.io
Jurisdiction string // drives token audience + exchange core
}
ErrNoCellForJurisdiction
Does not exist on current main. Today resolveCellAPIBaseURL returns plain fmt.Errorf strings (:548, :550). PR 1592 introduces the sentinel (§3).
Other entry point
1
func JurisdictionToken(ctx, insecureHTTP bool, jurisdiction string) (string, error) // :154
Raw-token variant used by entire auth token --jurisdiction; does honour ENTIRE_TOKEN (via resolveCellSubject``:195), unlike NewEntireAPICellClient.
2. Experts command — cell targeting
Call chain: experts_cmd.go runExperts → newExpertsAPIClient (:32) → NewAuthenticatedEntireAPICellClient (api_client.go:62) → resolveExpertsCellTarget (experts_cell_target.go:53) → auth.NewEntireAPICellClient.
1
2
func NewAuthenticatedEntireAPICellClient(ctx context.Context, insecureHTTP bool, fullName, ulid string) (*api.Client, error) // api_client.go:62
func resolveExpertsCellTarget(ctx context.Context, fullName, ulid string) *auth.CellTarget // experts_cell_target.go:53
resolveExpertsCellTarget (5s deadline, expertsCellResolveTimeout``:18):
resolveRepoClusterHost(:93): ULID →coreapi.GetRepo().ClusterHost; owner/repo →listMirrorsForRepo→distinctActiveClusterHosts.c.ListClusters→matchClusterByHost(:168, matches onPublicUrlhost) →cluster.ApiUrl+cluster.Jurisdiction→CellTarget.
Single target only — no fan-out.resolveRepoClusterHost:113 bails to nil (home-jurisdiction fallback) when len(hosts) != 1, i.e. a repo mirrored in multiple regions is explicitly treated as ambiguous and not fanned out (:117). Every failure path returns nil → NewEntireAPICellClient falls back to home-jurisdiction routing (best-effort contract, :39–:44).
runExperts (experts_cmd.go:276) selects one repo identity (ULID or owner/repo) and gets one client (:282). Cross-region misroute is handled reactively: a 404 with "region" in the body prints a region hint (:316–:319).
1
2
func distinctActiveClusterHosts(mirrors []coreapi.Mirror) []string // :143
func isActiveMirror(m coreapi.Mirror) bool // :129 (archived / failed / suspended excluded)
3. PR 1592 (entireio/cli) — what it adds
Theme: route activity/recap through the same cell client as experts, with a data-API fallback; harden cell_data_api.go.
New file cmd/entire/cli/entireapi_client.go (shared client seam):
runAuthenticatedActivityAPI(ctx, errW, insecureHTTP, fn)— triesauth.NewEntireAPICellClient(ctx, insecureHTTP, nil)(home cell); any error →logCellClientFallback+runAuthenticatedDataAPI. Both backends serve/me/*, sofnis backend-agnostic.logCellClientFallback— silent forErrNoCellForJurisdiction/ErrNotLoggedIn(normal during rollout), else debug-logs.forgeToMirrorProvider(forge)—gh/github→github, else not-ok (GitHub-only today).currentRepoID(ctx)— resolves origin remote → mirror → repo ULID (for/me/recap?repo=);""on any failure.firstActiveRepoID(mirrors)— firstisActiveMirrorplacement'sMirrorId.
activity_cmd.go:runActivity switches runAuthenticatedDataAPI → runAuthenticatedActivityAPI.
recap.go:newRecapClient now returns (*api.Client, repoSlug string, error) — prefers the cell client (currentRepoID = ULID) and falls back to data API (currentRepoSlug = owner/repo slug), tolerating not-logged-in.
cell_data_api.go hardening:
- Adds
var ErrNoCellForJurisdictionsentinel;resolveCellAPIBaseURLwraps it with%w(soerrors.Isworks for the fallback). - Fallback lists the cluster catalog against
selected.CoreURL(the login core that signedloginJWT), not the templated jurisdiction core — renames paramcoreURL→listCoreURL; the exchange still usescoreURL. targetJurisdictioncase-folds the JWT claim (strings.ToLower(TrimSpace)) before the strict label check, so an uppercasehome_jurisdictionroutes instead of hard-failing.
Note: the PR diff also shows adding isActiveMirror / refactoring distinctActiveClusterHosts in experts_cell_target.go, but those already exist on main — a stale-base overlap, not net-new.
4. Multi-cell fan-out audit
There is no data-plane fan-out anywhere in the CLI today. Every path resolves exactly one target cell:
distinctActiveClusterHosts(experts_cell_target.go:143) is the only place that computes the set of clusters a repo lives on — and its sole consumer (resolveRepoClusterHost:112) rejectslen != 1.selectCloneTarget(repo_clone.go:211) enumerates a repo's placements but picks one (via--clusteror interactive prompt) — clone, not query.- The mirror-create wizard (
repo_mirror_create_wizard.go) iterates the cluster catalog only to let the user choose a placement. - No
groupReposByCell, no per-cluster query iteration, no "run request X against every cell" loop exists.coreapi.ListRepos(internal/coreapi/oas_client_gen.go:4900) is a single control-plane call.
The only transparent cross-region mechanism is at the control-plane layer: coreapi.New() (internal/coreapi/client.go:35) wires newCrossJurisHTTPClient (cross_juris_transport.go), whose transport follows 421 cross-jurisdiction redirects and does the RFC 8693 exchange a foreign core requires. So control-plane reads (repo/mirror/cluster) work across regions from a home login — but that is redirect-follow, not client-side fan-out, and it applies only to coreapi, not to entire-api cell data.
5. Repo → cell/cluster association (CLI's view)
- Cluster catalog (
coreapi.Cluster,oas_schemas_gen.go:~655):ApiUrl OptString(cell data host),Jurisdiction string,PublicUrl string(git/public host).ListClustersis the authoritative jurisdiction→apiUrlsource. - Repo placement:
coreapi.Repo.ClusterHost OptString(:6493) for the ULID path;coreapi.Mirror.ClusterHost(per placement) for the owner/repo path. listMirrorsForRepo(repo_clone.go:180): lists all placements of a repo across clusters (server filters by provider+owner; repo matched client-side). Used by experts targeting,api_cmd.go``{repo_id}expansion, PR 1592'scurrentRepoID, and clone.- Join:
matchClusterByHost(experts_cell_target.go:168) links a placement'sClusterHostto a catalog cluster viaPublicUrlhost, yieldingApiUrl+Jurisdiction. - Default when unplaced:
defaultClusterHost = "aws-us-east-2.entire.io"(repo_mirror.go:55).
Single source of truth & gaps
"Which cell does request X go to?"
- Repo-scoped data-plane (experts, and recap-with-repo under PR 1592):
resolveExpertsCellTarget(CLI layer) →CellTarget→NewEntireAPICellClient. Authoritative catalog =coreapi.ListClusters. - Personal/home data-plane (activity, recap personal side):
home_jurisdictionJWT claim →resolveCellAPIBaseURLfallback. - Control-plane: always the home core; cross-region handled by 421-follow transport.
Duplication / gaps observed:
- Two cluster-catalog parsers.
resolveCellAPIBaseURL(auth,cell_data_api.go:508) hand-parsesGET /api/v1/clusters, duplicatingcoreapi.ListClusters— forced by the auth↔coreapi import cycle. The repo-scoped path sidesteps it by resolving in the CLI layer; the home-fallback cannot. This is the main structural wart. - Repo-ID resolution duplicated 3×:
resolveCurrentRepoID(api_cmd.go:194),resolveExpertsRepoID(experts_cmd.go:387), and PR 1592'scurrentRepoID/firstActiveRepoID— all "list mirrors → first active → MirrorId." PR 1592 partially consolidates via the sharedisActiveMirror, but the three call sites remain. - Multi-region repos are a hard gap, not a feature. Any repo mirrored across >1 cluster silently degrades to home-jurisdiction routing and relies on a reactive 404-region hint. No fan-out, no "ask each cell." If cross-region querying is the goal, this is the layer to build on —
distinctActiveClusterHostsalready computes the needed host set but its callers throw it away. ENTIRE_TOKENasymmetry:NewEntireAPICellClientignores it;JurisdictionTokenhonours it (cell_data_api.go:100,:154). Intentional but a subtle footgun for anyone adding cell commands.
Key signatures (entry points)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
// auth layer
func NewEntireAPICellClient(ctx context.Context, insecureHTTP bool, target *CellTarget) (*api.Client, error)
func JurisdictionToken(ctx context.Context, insecureHTTP bool, jurisdiction string) (string, error)
func HomeJurisdictionFromLoginJWT(loginJWT string) (string, error)
type CellTarget struct { BaseURL string; Jurisdiction string }
// cli layer
func NewAuthenticatedEntireAPICellClient(ctx context.Context, insecureHTTP bool, fullName, ulid string) (*api.Client, error)
func NewAuthenticatedAPIClient(ctx context.Context, insecureHTTP bool) (*api.Client, error)
func resolveExpertsCellTarget(ctx context.Context, fullName, ulid string) *auth.CellTarget
func listMirrorsForRepo(ctx context.Context, c mirrorLister, provider, owner, repo string) ([]coreapi.Mirror, error)
func distinctActiveClusterHosts(mirrors []coreapi.Mirror) []string
// PR 1592 (proposed)
func runAuthenticatedActivityAPI(ctx context.Context, errW io.Writer, insecureHTTP bool, fn func(context.Context, *api.Client) error) error
func newRecapClient(ctx context.Context, insecureHTTP bool) (*api.Client, string, error)
var ErrNoCellForJurisdiction = errors.New("no entire-api cell configured for jurisdiction")
```</result>
<usage><subagent_tokens>61269</subagent_tokens><tool_uses>26</tool_uses><duration_ms>474752</duration_ms></usage>
</task-notification>
1w ago·15.4s
Second report is in. The CLI map confirms the picture: today there is exactly one cell per request everywhere — experts resolves a single repo → single cell, #1592 adds home-cell routing with data-API fallback, and no data-plane fan-out exists anywhere. It also surfaced a structural wart (the auth package hand-parses /api/v1/clusters because of an auth↔coreapi import cycle) and three duplicated repo-ID resolvers. One agent left: the entiredb/entire-api backend model, which should tell us what invariants a shared fan-out layer can rely on (repo↔cell placement, token audience scope, and how the BFF does its fan-out). I'll write up the recommendation when it lands.
Server-side cell/jurisdiction model — how a CLI should route/fan out
Two-tier system. entiredb (control plane, "core") owns the global catalog: identities, the cluster/cell registry, repo placements, authz (SpiceDB), and all token minting. entire-api (data plane, "cell") is deployed once per jurisdiction and serves residency-safe activity/product reads for repos that live in that cell. There is no server-side cross-cell aggregation in the Go services — fan-out is the client's job, and today the entire.io Cloudflare-Workers BFF is the reference client that does it.
Note: entire.io is a TypeScript/Hono monorepo (api/, frontend/, website/), not Rails — the BFF fan-out lives in api/src/.
1. entiredb (core) — the catalog and token minting
Cluster / cell / jurisdiction model
Clusterstruct:core/cluster/cluster.go. Fields:Slug(globally unique, embedded in JWT audiencesservice:entiredb:<slug>),Jurisdiction(e.g.us/eu/au),PublicURL(git data-plane host),InternalURL(mTLS, never exposed),Cell(physical cell, e.g.aws-us-east-2),IsDefault,Hidden. Stored in theclusterstable; reconciled from an embedded fixture on deploy (core/cluster/store.go,reconcile.go). Lookups by public host / slug / slug-global / default-per-jurisdiction.- Invariant:
clusters.slugis globally unique (migration 009), so a slug names exactly one cluster regardless of jurisdiction (GetBySlugGlobal). This lets one core render clone hosts for repos in any jurisdiction.
Data model (core/model/types.go)
Account.HomeJurisdiction— the account's residency (accounts.home_jurisdiction, global identity store). "Our Account is the architecture's Person."PrivateProfile/VerifiedEmail— regional PII, resident in home jurisdiction, "never leaves the region."Repo(jurisdictional record) carriesClusterSlugbut no jurisdiction field — "every row lives in exactly the jurisdiction whose store it was read from."RepoRegistration(global registry row):ID,Jurisdiction,ClusterSlug— "what exists and where it lives," used for id→jurisdiction routing.- Invariant: a repo/placement lives in exactly one cluster (one cell, one jurisdiction). A project is hosted in a single region and every repo it owns lives in that same region.
Catalog API endpoints (huma, /api/v1, core/coreapi/)
GET /api/v1/clusters(coreapi/clusters.go, wireapi/corev1/clusters.go): lists all clusters across jurisdictions. Each entry:slug,jurisdiction,publicUrl(git host),isDefault, andapiUrl= the jurisdiction's cell-bound entire-api host (fromJURISDICTION_API_HOSTS, keyed by jurisdiction). Comment explicitly names this "the BFF's /me proxy" consumer. Requires auth, no per-cluster authz (topology is shared infra).GET /api/v1/repos— the consolidated repo index / placement catalog (coreapi/repos.go``ListRepos, wireapi/corev1/repos.go``RepoIndexEntry/RepoPlacement). One entry per logical repo (mirrors collapsed); scalar fields describe the canonical placement (caller's home-jurisdiction copy, else lowest-ULID);placements[]lists every readable cell-local copy with{id, jurisdiction, cell, mirror}.Cell(e.g.aws-eu-west-1) is the addressable routing key ("the BFF formshttps://<cell>.api.partial.to");clusterSlugis a logical name. Not paginated — capped + alphabetical +truncatedflag. This is the index a CLI should use to know which cells host which repos.GET /api/v1/mirrors(coreapi/mirrors.go, wireapi/corev1/mirrors.go): mirror placements the caller can pull (SpiceDBrepo#pullreverse lookup, across all jurisdictions). EachMirrorcarriesclusterHost,jurisdiction,cell, status. "The optional cluster filter is just a filter, not a router: a single core returns mirrors on every cluster the caller can pull, regardless of jurisdiction." Create/Delete/collaborators address a placement by upstream coords +clusterHost.GET /api/v1/repos/lookup?slug=(LookupBySlugOutput): unauthenticated slug→{repoId, cell, jurisdiction, owner, repo}routing resolver.- Home jurisdiction surfaced on identity endpoints (
api/corev1/identity.go``HomeJurisdiction,/accounts/{id}internal mTLS).
home_jurisdiction claim minting & RFC 8693 token exchange
Token exchange endpoint is the canonical OIDC
/oauth/tokenwithgrant_type=urn:ietf:params:oauth:grant-type:token-exchange(core/authn/oidcop/token_exchange.go; URNs inhttputil/oauth.go). It dispatches on the requested audience (ValidateTokenExchangeRequest):repo audience
https://<cluster>/git/repo/<id>→ scopepull|push/admin,/debug, deploy,service:entiredb- core/AS audience →
entire:api-access(narrowing) or SA-session - jurisdiction audience (path-less origin, e.g.
https://us.partial.to) → identity token:validateIdentityExchangemints an identity-only access token withscope=openid, subject's identity claims, andhome_jurisdictionstamped viastampHomeJurisdiction(reads globalaccountsrow). This is the jurisdictional identity token entire-api consumes. Input must carryentire:session(non-re-entry: the openid-only output can't bootstrap another exchange).
home_jurisdictionis stamped on every identity-bearing token — the login JWT (core audience branch) and the identity token (jurisdiction audience branch) — so any reader can route home-jurisdiction work off it.Scopes:
core/authn/scopes.go.ScopeOpenID,ScopeEntireSession, exchange-capability scopesentire:repo-access|ops-access|api-access|sa-session.Relying-party verifier:
core/authn/jurisdiction_token.go(JurisdictionTokenVerifier) — pins exact jurisdiction aud +scope=openid, delegates to the dependency-lightmodules/jwtverify.Cross-jurisdiction (
validateForeignSessionExchange, ADR 20260520): a sibling region's login JWT can exchange only for this core's login audience or its jurisdiction identity audience; to get repo/ops credentials the caller must "foreign-session first, then re-exchange at the new home." Chained cross-juris hops are barred (foreign_iss). Repo-audience exchange additionally pins the audience host to the repo's actual hosting cluster (assertAudienceHostOwnsRepo) — a repo token can't be replayed at another cluster.Invariant: identity tokens are per-jurisdiction, not per-cell — the aud is the jurisdiction host, and every cell in a jurisdiction reports the same
apiUrl, so any cluster row in the jurisdiction is an acceptable route target.
2. entire-api (cell) — what a cell serves and how it authenticates
Auth: jurisdictional identity token only
internal/auth/auth.go: bearer-only, no cookies. TheAuthenticatorverifies against Core JWKS, pins the token aud to this cell's jurisdiction host and requiresscope=openid(ADR 20260612). It readshome_jurisdiction,handle,providerstraight off the token — no callback to Core. Read-time authz is a live query to the regional read-only SpiceDB (internal/authz), not a Core call. This is not the control-plane bearer — it's specifically the jurisdictional identity token minted by core's identity exchange. The gateway'srequireBearer(below) forwards the caller bearer; the workload mТLS cert is only the pipe, never the authorization.
Endpoints (internal/httpapi/, huma under /api/v1, router.go)
/me/*— two surfaces. The drop-in owns bare/me/*; the forward-looking residency-safe pointer contract is under/experimental/me*(experimental_me.go):GET /me(identity + jurisdictions the caller has data in),/me/commits,/me/checkpoints,/me/activity,/me/recap. All read the local region'smy_activitystore and filter by the caller's currently-pullable repos (fail-closed via SpiceDB). Aggregation math inme_aggregate.go./me/recap— per-agent recap; team column signalsteam_stats_foreign_regionwhen the repo isn't local (client should federate to the repo's region).- Sessions/preferences:
me_sessions.go,preferences.go,repo_preferences.go. - Experts (
experts.go):POST /api/v1/repos/{repo_id}/experts— repo-scoped (gated on repo pull), returns residency-safe agent-provenance pointers. Natural-languagequerymode returns 503 because a cell's activity-api doesn't run the code-search index (see search below). - Search / peregrine: not served by activity-api directly — it's a separate cell-local upstream aggregated by the gateway. In
internal/gateway/config/topology.go,peregrineis a cell service (PEREGRINE_SPEC_URL,PathPrefix: /search), so its surface is exposed at/api/v1/search/...(the BFF callssearch/api/searchandsearch/api/symbols). Optional per cell.
Cell scoping — strictly local, with two residency mechanisms
- Repo data is strictly scoped to repos placed in that cell.
docs/architecture.md: "A repo's full history lives only in the repo's own jurisdiction." The repo store holds only repos that live in this region; project aggregates are complete because a project + all its repos co-locate. Git content isn't replicated — it's read through to the regional entiredb at request time (bearer forwarded, re-authorized against SpiceDB), cached by branch tip. - User activity is consolidated into the user's HOME cell (not per-repo). The
forwarder(internal/forwarder/subjects.go) publishesuser_activity_v1.{home_jurisdiction}residency-safe metadata into the recipient's home-region NATS island. There is explicitly no NATS supercluster — cross-region delivery is a direct publish per region. So/meon a user's home cell already sees their cross-cell activity as forwarded pointers. - Last-resort foreign repo naming (
me_foreign_repo.go): a/merow for a repo with no local placement is named via a read-time Core batch call (shown-not-stored, best-effort) — reinforces that the cell holds no metadata for foreign repos.
3. Server-side fan-out / aggregation across cells — NONE in Go; the BFF does it
Neither core nor entire-api aggregates across cells. Core returns catalogs spanning jurisdictions (repos/mirrors/clusters) but never proxies to cells. Each cell is strictly local. Fan-out is the client's responsibility. The reference implementation is the entire.io BFF (entire.io/api/src/).
The documented BFF pattern (multi-region fan-out)
Reference: entire.io/api/src/routes/code-search.ts — POST /code-search/stream, SSE cross-jurisdiction code search. Its own comment: "Same fan-out pattern as /repos/stream: index → group by cell → per-cell search → stream. Each cell settles independently." Steps:
- Index:
listReposIndexcalls coreGET /api/v1/repos(api/src/lib/repos-federation/list-repos-index.ts) — authority for the row set + placement (jurisdiction,cell), canonical placement +placements[], plustruncated. - Group by cell:
byCell = Map<cell, {jurisdiction, repoIds}>— one request per cell, each cell searches only its own repos. - Per-jurisdiction token mint (single-flighted per jurisdiction, cached): login JWT → RFC 8693 exchange at the regional core (
ENTIRE_CORE_BASE_URL_TEMPLATE, COR-666) for a jurisdiction identity token whose aud = that cell'sENTIRE_API_AUDIENCE_TEMPLATE.api/src/lib/entire-core/jurisdiction-token.ts. - Fetch per cell: base URL
buildBaseUrl(template, cell)(https://<cell>.api...), pathsearch/api/search, bearer = the jurisdiction token.max_resultsdivided across cells. - Stream + settle independently:
fanOutStream(api/src/lib/sse-fan-out.ts) emitsready/fill/error/doneframes; a failing/slow cell (30s budget) emits an error frame while healthy cells stream through.
Two routing helpers make the repo-vs-user distinction explicit:
- Repo-scoped (
resolve-cell.ts/proxy.ts): resolve a repo to its{cell, jurisdiction, repoId}via core's mirror/repo listing, mint a jurisdiction token, call that one cell. Multi-cell → fan out and merge. - User-scoped
/me(resolve-home.ts/me-proxy.ts): "Personal activity is consolidated in the user's home jurisdiction … a single point-to-point call to that one cell — no repo, no fan-out, no merge." Home jurisdiction read from thehome_jurisdictionlogin-JWT claim; api base from the cluster listing'sapiUrl. (CLI/bearer callers fall through — CLI home routing is a separate CLI-side change: "point its data base at the home entire-api directly.")
4. Future direction (docs/ADRs)
- entiredb
docs/adrs/:20260612-regional-authz-service-list-queries.md(the identity-token + regional-SpiceDB model),20260520-cross-juris-token-exchange.md,20260424-regional-repo-metadata.md,20260403-regional-login-flow.md,20260620-identity-first-auth.md,20260529-entire-core-crdb-read-locality.md,20260527-three-tier-session-tokens-and-fid-liveness.md.docs/architecture.md,docs/auth.md,docs/replication-flows.md. - entire-api
docs/:architecture.md(the residency model — "Data has a residence"; "The API never serves content"),adr/0001-store-boundary-via-roles.md,0002-commit-author-account-resolution.md,0003-session-sharing-encrypted-snapshot.md,mirror-origin-rollout.md. - The
/experimental/meprefix inexperimental_me.gois explicitly "a forward-looking contract, not the BFF's current one" — the residency-safe pointer + cross-region federation model the CLI should target. - The forwarder doc reference
docs/multi-region/multi-region-fanout-design.md(mirror-pipeline'scmd/fanout) is cited as the model for per-region NATS islands.
Key invariants for CLI routing
- A repo/placement lives in exactly one cell (one cluster, one jurisdiction); a "logical repo" may have several placements (native + mirrors) across cells — core's
GET /api/v1/reposcollapses them with aplacements[]list. cellis the addressable routing key (aws-eu-west-1→https://<cell>.api...);clusterSlugis a logical name;publicUrlis the git data-plane host;apiUrl(per jurisdiction) is the entire-api host.- Tokens are per-jurisdiction, not per-cell. One identity token (aud = jurisdiction host,
scope=openid) is accepted by every cell in that jurisdiction. Mint once per jurisdiction, reuse across its cells. - The
home_jurisdictionclaim rides on both the login JWT and the identity token — it's the routing key for user-scoped/mereads. - User (
/me) reads = single call to the home cell (activity is forwarded home). Repo/search reads = fan out to each cell that hosts the caller's repos, then merge. No server-side aggregator exists — the CLI must fan out itself. - Cells are strictly local + fail-closed. A cell only holds/serves data for repos placed in it; authz is live regional SpiceDB; git content is read-through to the regional entiredb (forward the caller's bearer).
- Cross-jurisdiction credential rule: you cannot mint a repo/ops token in region B from region A's session directly — foreign-session to B first, then re-exchange. For a CLI this means: get an identity/login token valid at the target jurisdiction's core before hitting its cells.
1w ago·2m·2 calls
All three investigations are done, and I verified the one suspicious join myself. Here's the synthesis.
TLDR
Yes — I'd build the foundation first, but the "foundation" is a modest extraction, not a framework. #1592 is already part of that foundation and should land first. #1616 should then be reworked on top of a small shared cell-routing package before merge: it currently inlines ~300 lines of generic multi-cell orchestration under codeSearch-prefixed names, mints tokens wastefully (once per cell instead of once per jurisdiction), and joins the repo index to the cluster catalog on a fragile key. Since it's gated behind ENTIRE_CODE_SEARCH=1 and has had no human review yet, now is the cheapest moment to fix the shape.
Why the timing question is real: three routing shapes are converging
The backend model (entiredb + entire-api) defines exactly three request shapes, and the BFF (entire.io/api/src) already has a named helper for each:
- Repo-scoped → one cell (BFF
resolve-cell.ts): a placement lives in exactly one cell; resolve it, mint a jurisdiction token, call that cell. The CLI's version is the experts path (resolveExpertsCellTargetinexperts_cell_target.go). - User-scoped
/me→ home cell, never fan-out (BFFresolve-home.ts): activity is forwarded into the user's home region, so/meis a single point-to-point call routed by thehome_jurisdictionJWT claim. That's what #1592 adds (entireapi_client.go,ErrNoCellForJurisdiction, the case-folded claim). - Repo-set queries → fan out to every hosting cell, merge client-side (BFF
code-search.ts): no server-side aggregator exists anywhere — core returns catalogs, cells are strictly local. #1616 is the CLI's first instance of this shape.
The CLI is growing these one command at a time, each with its own seams. #1616 would make it three commands, three sets of plumbing.
Concrete problems with merging #1616 as-is
1. The generic layer is inlined and search-named.groupReposByCell, resolveRepoFilters, the ListRepos→ListClusters→CellTarget resolution, the WaitGroup fan-out with per-cell timeouts, and the partial-failure tracking (FailedJurisdictions) are all in search_cmd.go — none of it is search-specific. The next fan-out command (experts natural-language query is an obvious candidate: cells return 503 for it today precisely because it needs the search index) copies or refactors it. It also duplicates experts' patterns with different choices: a second coreClient interface+seam, different timeout constants, and no placement/active-mirror filtering.
2. Token minting ignores the key backend invariant. Identity tokens are per-jurisdiction, not per-cell — every cell in a jurisdiction accepts the same token. #1616 calls auth.NewEntireAPICellClient once per cell, and each call independently re-runs discovery, login-JWT refresh, and the RFC 8693 exchange. The BFF single-flights and caches one token per jurisdiction. With N cells that's N× redundant auth round trips per search, growing with every future fan-out command.
3. The catalog join is fragile — verified. #1616 groups repos by RepoIndexEntry.Cell (physical cell, e.g. aws-us-east-2) but looks that value up in a map keyed by Cluster.Slug (slugToCluster[cells[i].cell], diff line ~576). The cluster catalog response doesn't expose a cell field at all — RepoIndexEntry carries ClusterSlug for exactly this join, and #1616 doesn't use it. Today slug and cell often coincide, so a mismatch degrades to the jurisdiction fallback (right region, but it triggers a hand-rolled per-cell cluster-catalog re-fetch inside auth) rather than misrouting — but it's a silent behavioral cliff. This is now the third distinct join key against the same catalog: experts joins by PublicUrl host, #1616 by slug-as-cell, the auth fallback by jurisdiction.
4. Existing warts get a third copy instead of a fix.resolveCellAPIBaseURL in auth/cell_data_api.go:508 already hand-parses GET /api/v1/clusters because of the auth↔coreapi import cycle, and repo-ID resolution exists three times (api_cmd.go, experts_cmd.go, #1592's currentRepoID).
What the foundation should be
A small package (say cmd/entire/cli/cellrouting) with three entry points mirroring the BFF's helpers, built from what already exists:
RepoCell(ctx, fullName, ulid) *auth.CellTarget— liftresolveExpertsCellTargetout of experts naming.HomeCell(ctx)— #1592'sentireapi_client.gologic, essentially as-is.CellsFor(ctx, repoFilter) []CellGroup+FanOut(ctx, groups, timeout, fn)— extracted from #1616'ssearchAllCells/searchCell, with the join fixed toClusterSlug↔Slug, and onecoreClientseam shared with experts. Merge semantics stay per-command.- Per-jurisdiction token cache inside or in front of
NewEntireAPICellClient: resolve the stored subject once, exchange once per jurisdiction, reuse across cells in a fan-out.
Deferrable: breaking the auth↔coreapi cycle so there's one cluster-catalog parser, and consolidating the three repo-ID resolvers.
Suggested sequencing
- Merge #1592 now. It's small, it establishes the home-cell shape and
ErrNoCellForJurisdiction, and the foundation builds on it rather than replacing it. - Extract the routing package (mostly moving code from #1616's diff plus experts), with the per-jurisdiction token cache and the
ClusterSlugjoin fix. - Rebase #1616 onto it. The search-specific parts — the
codesearchclient package, merge/sort/dedup semantics, output rendering, flag handling — are genuinely good and carry over unchanged; its ~940 lines of tests mostly test logic that moves rather than dies.
The counterargument — merge behind the env gate, refactor later — is defensible, but weak here: the PR has had no human review of the orchestration (only two stale Copilot nits), the extraction is cheap while the code is fresh, and gated code has a way of calcifying into the pattern the next command copies.
the issue is that 1592 is currently red and has open comments, so merging is not straight forward, so let's fix that and build the routing package extraction on top?
1w ago·6m·33 calls
Now tidy the redundant fold in resolveJurisdiction and its stale doc comment:
continue, sorry
1w ago·48m·94 calls
Lint clean. Adding a brief architecture note to CLAUDE.md so the new layer is discoverable, then committing:
e9115e5test: define the writeJSON helper the corecmd stub handlers call\ \ PR #1626 landed corecmd_list_test.go / corecmd_mutation_test.go calling a\ writeJSON helper that was never defined anywhere in the package, so main's\ Lint (typecheck) and Tests jobs are red and every open PR inherits the\ failure from the merge preview.\ \ Co-Authored-By: Claude Fable 5 noreply@anthropic.com\ Entire-Checkpoint: 876de296c7bd+8
32e8a17cli: address Bugbot/Copilot review on cell routing fallback\ - Bound currentRepoID's control-plane lookup with a 5s timeout\ (currentRepoIDTimeout, mirroring expertsCellResolveTimeout): the lookup\ is best-effort decoration, so a stalled core must not hang recap.\ - Case-fold the cluster catalog row's jurisdiction in\ resolveCellAPIBaseURL: the claim side is already folded, so a\ differently-cased row misreported ErrNoCellForJurisdiction.\ - Rewrite newRecapClient's fallback contract comment to match the\ implementation: ANY cell-client failure falls back to the data API\ (expected cases silent, others debug-logged), not just the two named\ errors.\ - Rename runRecap's repoSlug local to repoScope: once cell-routed it\ holds a repo_id ULID, only the data-API fallback passes a slug.\ \ Co-Authored-By: Claude Fable 5 noreply@anthropic.com\ Entire-Checkpoint: 76ad7ee5e8b2+24/-9
070412fcli/auth: add CellClientFactory — one subject, one token per jurisdiction\ \ NewEntireAPICellClient resolved the stored login subject (discovery +\ login refresh) and ran the RFC 8693 exchange on every call. For a\ single-cell command that's fine, but a multi-cell fan-out (one request\ per cell hosting the caller's repos, the BFF's code-search pattern)\ would pay all of it once per cell — even though identity tokens are\ per-jurisdiction, not per-cell: every cell in a jurisdiction accepts\ the same token.\ \ CellClientFactory resolves the subject once at construction and caches\ minted identity tokens by jurisdiction; ClientFor(target) reuses them\ across cells. NewEntireAPICellClient stays as the single-cell wrapper\ (factory of one), so existing callers are unchanged.\ \ Co-Authored-By: Claude Fable 5 noreply@anthropic.com\ Entire-Checkpoint: 935f91398e25+123/-10
00f1f52test: define the writeJSON helper the corecmd stub handlers call\ \ PR #1626 landed corecmd_list_test.go / corecmd_mutation_test.go calling a\ writeJSON helper that was never defined anywhere in the package, so main's\ Lint (typecheck) and Tests jobs are red and every open PR inherits the\ failure from the merge preview.\ \ Co-Authored-By: Claude Fable 5 noreply@anthropic.com\ Entire-Checkpoint: 876de296c7bd+8
6a82a9bcli: generalize the experts cell-target resolver to cell_target.go\ \ resolveExpertsCellTarget was experts-named but nothing in it is\ experts-specific: it maps any repo (ULID or owner/repo) to the entire-api\ cell hosting it, exactly what every future repo-scoped or fan-out command\ needs (code search is next, PR #1616). Rename the file and identifiers —\ resolveRepoCellTarget, cellCoreClient, newCellCoreClient,\ cellResolveTimeout — and keep experts as the documented example consumer.\ Mechanical rename; no behavior change.\ \ Co-Authored-By: Claude Fable 5 noreply@anthropic.com\ Entire-Checkpoint: 89b4d13bd2b0+391/-391
73b1b2ecli: add multi-cell fan-out helpers (groupReposByCell, fanOutCells)\ \ The data plane has no server-side cross-cell aggregator: a query over all\ of the caller's repos must be fanned out to each cell hosting them and\ merged client-side (the entire.io BFF's code-search pattern). PR #1616\ inlines that orchestration into search_cmd.go; this extracts the generic\ layer so code search — and any later repo-set command — shares one\ implementation:\ - groupReposByCell: repo index → per-cell groups, deterministic order.\ - resolveCellBaseURLs: cluster-catalog join on ClusterSlug↔Cluster.Slug.\ The catalog exposes no cell field, so joining the cell name against\ Slug (as #1616 currently does) only works when the two coincide.\ - fanOutCells: parallel per-cell calls under a per-cell timeout, one\ shared auth.CellClientFactory (subject resolved once, one identity\ token per jurisdiction), partial failures isolated per result slot.\ Merge semantics stay with the command.\ \ Also documents the three cell-routing shapes in CLAUDE.md.\ \ Co-Authored-By: Claude Fable 5 noreply@anthropic.com\ Entire-Checkpoint: 842e421e74a8+457
1febc7etest: route the corecmd stub handlers through printJSON\ \ PR #1626 landed corecmd_list_test.go / corecmd_mutation_test.go calling a\ writeJSON helper, green against a base that predated #1606 — which\ deleted writeJSON to standardize package cli on printJSON. #1626 merged\ two days later without a re-run against the moved base, so main's Lint\ (typecheck) and Tests are red and every open PR inherits the failure\ via the merge preview.\ \ Point the three call sites at printJSON, the same conversion #1606\ applied to the other test fixtures (repo_mirror_test.go,\ resolveref_test.go, ...), rather than reintroducing the deleted\ duplicate.\ \ Co-Authored-By: Claude Fable 5 noreply@anthropic.com\ Entire-Checkpoint: d541b17e10f8+3/-3
be925e4cli: generalize the experts cell-target resolver to cell_target.go\ \ resolveExpertsCellTarget was experts-named but nothing in it is\ experts-specific: it maps any repo (ULID or owner/repo) to the entire-api\ cell hosting it, exactly what every future repo-scoped or fan-out command\ needs (code search is next, PR #1616). Rename the file and identifiers —\ resolveRepoCellTarget, cellCoreClient, newCellCoreClient,\ cellResolveTimeout — and keep experts as the documented example consumer.\ Mechanical rename; no behavior change.\ \ Co-Authored-By: Claude Fable 5 noreply@anthropic.com\ Entire-Checkpoint: 89b4d13bd2b0+391/-391
990a660cli: add multi-cell fan-out helpers (groupReposByCell, fanOutCells)\ \ The data plane has no server-side cross-cell aggregator: a query over all\ of the caller's repos must be fanned out to each cell hosting them and\ merged client-side (the entire.io BFF's code-search pattern). PR #1616\ inlines that orchestration into search_cmd.go; this extracts the generic\ layer so code search — and any later repo-set command — shares one\ implementation:\ - groupReposByCell: repo index → per-cell groups, deterministic order.\ - resolveCellBaseURLs: cluster-catalog join on ClusterSlug↔Cluster.Slug.\ The catalog exposes no cell field, so joining the cell name against\ Slug (as #1616 currently does) only works when the two coincide.\ - fanOutCells: parallel per-cell calls under a per-cell timeout, one\ shared auth.CellClientFactory (subject resolved once, one identity\ token per jurisdiction), partial failures isolated per result slot.\ Merge semantics stay with the command.\ \ Also documents the three cell-routing shapes in CLAUDE.md.\ \ Co-Authored-By: Claude Fable 5 noreply@anthropic.com\ Entire-Checkpoint: 842e421e74a8+457
c20a5dctest: route the corecmd stub handlers through printJSON\ \ PR #1626 landed corecmd_list_test.go / corecmd_mutation_test.go calling a\ writeJSON helper, green against a base that predated #1606 — which\ deleted writeJSON to standardize package cli on printJSON. #1626 merged\ two days later without a re-run against the moved base, so main's Lint\ (typecheck) and Tests are red and every open PR inherits the failure\ via the merge preview.\ \ Point the three call sites at printJSON, the same conversion #1606\ applied to the other test fixtures (repo_mirror_test.go,\ resolveref_test.go, ...), rather than reintroducing the deleted\ duplicate.\ \ Co-Authored-By: Claude Fable 5 noreply@anthropic.com\ Entire-Checkpoint: d541b17e10f8+3/-3
86a21a8cli: address Bugbot/Copilot review on cell routing fallback\ - Bound currentRepoID's control-plane lookup with a 5s timeout\ (currentRepoIDTimeout, mirroring expertsCellResolveTimeout): the lookup\ is best-effort decoration, so a stalled core must not hang recap.\ - Case-fold the cluster catalog row's jurisdiction in\ resolveCellAPIBaseURL: the claim side is already folded, so a\ differently-cased row misreported ErrNoCellForJurisdiction.\ - Rewrite newRecapClient's fallback contract comment to match the\ implementation: ANY cell-client failure falls back to the data API\ (expected cases silent, others debug-logged), not just the two named\ errors.\ - Rename runRecap's repoSlug local to repoScope: once cell-routed it\ holds a repo_id ULID, only the data-API fallback passes a slug.\ \ Co-Authored-By: Claude Fable 5 noreply@anthropic.com\ Entire-Checkpoint: 76ad7ee5e8b2+24/-9
64230afcli/auth: add CellClientFactory — one subject, one token per jurisdiction\ \ NewEntireAPICellClient resolved the stored login subject (discovery +\ login refresh) and ran the RFC 8693 exchange on every call. For a\ single-cell command that's fine, but a multi-cell fan-out (one request\ per cell hosting the caller's repos, the BFF's code-search pattern)\ would pay all of it once per cell — even though identity tokens are\ per-jurisdiction, not per-cell: every cell in a jurisdiction accepts\ the same token.\ \ CellClientFactory resolves the subject once at construction and caches\ minted identity tokens by jurisdiction; ClientFor(target) reuses them\ across cells. NewEntireAPICellClient stays as the single-cell wrapper\ (factory of one), so existing callers are unchanged.\ \ Co-Authored-By: Claude Fable 5 noreply@anthropic.com\ Entire-Checkpoint: 935f91398e25+123/-10
2c36071cli: generalize the experts cell-target resolver to cell_target.go\ \ resolveExpertsCellTarget was experts-named but nothing in it is\ experts-specific: it maps any repo (ULID or owner/repo) to the entire-api\ cell hosting it, exactly what every future repo-scoped or fan-out command\ needs (code search is next, PR #1616). Rename the file and identifiers —\ resolveRepoCellTarget, cellCoreClient, newCellCoreClient,\ cellResolveTimeout — and keep experts as the documented example consumer.\ Mechanical rename; no behavior change.\ \ Co-Authored-By: Claude Fable 5 noreply@anthropic.com\ Entire-Checkpoint: 89b4d13bd2b0+391/-391
48d5ac5cli: add multi-cell fan-out helpers (groupReposByCell, fanOutCells)\ \ The data plane has no server-side cross-cell aggregator: a query over all\ of the caller's repos must be fanned out to each cell hosting them and\ merged client-side (the entire.io BFF's code-search pattern). PR #1616\ inlines that orchestration into search_cmd.go; this extracts the generic\ layer so code search — and any later repo-set command — shares one\ implementation:\ - groupReposByCell: repo index → per-cell groups, deterministic order.\ - resolveCellBaseURLs: cluster-catalog join on ClusterSlug↔Cluster.Slug.\ The catalog exposes no cell field, so joining the cell name against\ Slug (as #1616 currently does) only works when the two coincide.\ - fanOutCells: parallel per-cell calls under a per-cell timeout, one\ shared auth.CellClientFactory (subject resolved once, one identity\ token per jurisdiction), partial failures isolated per result slot.\ Merge semantics stay with the command.\ \ Also documents the three cell-routing shapes in CLAUDE.md.\ \ Co-Authored-By: Claude Fable 5 noreply@anthropic.com\ Entire-Checkpoint: 842e421e74a8+457
8f05352cli/auth: add CellClientFactory — one subject, one token per jurisdiction\ \ NewEntireAPICellClient resolved the stored login subject (discovery +\ login refresh) and ran the RFC 8693 exchange on every call. For a\ single-cell command that's fine, but a multi-cell fan-out (one request\ per cell hosting the caller's repos, the BFF's code-search pattern)\ would pay all of it once per cell — even though identity tokens are\ per-jurisdiction, not per-cell: every cell in a jurisdiction accepts\ the same token.\ \ CellClientFactory resolves the subject once at construction and caches\ minted identity tokens by jurisdiction; ClientFor(target) reuses them\ across cells. NewEntireAPICellClient stays as the single-cell wrapper\ (factory of one), so existing callers are unchanged.\ \ Co-Authored-By: Claude Fable 5 noreply@anthropic.com\ Entire-Checkpoint: 935f91398e25+123/-10
574c5e4cli: generalize the experts cell-target resolver to cell_target.go\ \ resolveExpertsCellTarget was experts-named but nothing in it is\ experts-specific: it maps any repo (ULID or owner/repo) to the entire-api\ cell hosting it, exactly what every future repo-scoped or fan-out command\ needs (code search is next, PR #1616). Rename the file and identifiers —\ resolveRepoCellTarget, cellCoreClient, newCellCoreClient,\ cellResolveTimeout — and keep experts as the documented example consumer.\ Mechanical rename; no behavior change.\ \ Co-Authored-By: Claude Fable 5 noreply@anthropic.com\ Entire-Checkpoint: 89b4d13bd2b0+391/-392
6055b38cli: add multi-cell fan-out helpers (groupReposByCell, fanOutCells)\ \ The data plane has no server-side cross-cell aggregator: a query over all\ of the caller's repos must be fanned out to each cell hosting them and\ merged client-side (the entire.io BFF's code-search pattern). PR #1616\ inlines that orchestration into search_cmd.go; this extracts the generic\ layer so code search — and any later repo-set command — shares one\ implementation:\ - groupReposByCell: repo index → per-cell groups, deterministic order.\ - resolveCellBaseURLs: cluster-catalog join on ClusterSlug↔Cluster.Slug.\ The catalog exposes no cell field, so joining the cell name against\ Slug (as #1616 currently does) only works when the two coincide.\ - fanOutCells: parallel per-cell calls under a per-cell timeout, one\ shared auth.CellClientFactory (subject resolved once, one identity\ token per jurisdiction), partial failures isolated per result slot.\ Merge semantics stay with the command.\ \ Also documents the three cell-routing shapes in CLAUDE.md.\ \ Co-Authored-By: Claude Fable 5 noreply@anthropic.com\ Entire-Checkpoint: 842e421e74a8+457
Continue from where you left off.
1w ago
No response requested.
continue
1w ago·3m·13 calls
All caught up — the whole stack is now either merged or waiting only on human review:
Done since we started:
- #1640 (red-main fix) — merged. Main is green again.
- #1592 (activity/recap → home-cell client) — merged, with all five bot findings fixed.
- #1641 (the cell-routing foundation, now based directly on main) — CI fully green, and all five of its bot findings are fixed and replied to:
tokenFornow uses per-jurisdiction mint slots so no lock is held across the network exchange, the fan-out catalog lookup got thecellResolveTimeoutbound,resolveCellBaseURLsrefuses to set abaseURLwithout a jurisdiction (which would have dialed a foreign cell with a home token), and empty-cell index entries group per-jurisdiction instead of collapsing into one mixed group. - #1616 coordination — I posted a comment on evis's PR pointing at #1641 and sketching the rebase: the
codesearchclient, merge/dedup semantics, and rendering carry over unchanged; the inline orchestration collapses ontogroupReposByCell/resolveCellBaseURLs/fanOutCells.
One thing worth flagging from #1616's history: it actually had a commit that correctly switched the catalog join to ClusterSlug, which a later commit reverted citing BFF parity — but the BFF templates cell URLs from the cell name rather than doing a catalog lookup, so the parity argument doesn't apply to a Slug-keyed lookup. The comment spells that out so the join doesn't flip back again; #1641 pins it with a slug≠cell test.
The only remaining step is human: #1641 needs a code-owner review from entireio/cli-maintainers (you can't self-approve, and the ruleset blocks admin bypass). Once it lands, #1616 can rebase — I'm happy to do that rebase or review it when the time comes.
1641 had a review comment with the last approval, that's resolved?
1w ago·5m·8 calls
Memory updated too. Once evisdren re-approves, #1641 merges and #1616 can rebase onto it — I've already left the rebase guidance on that PR.
ffc362dcli/auth: make the two-tier mint locking contract explicit in comments\ \ Review nit on #1641: CellClientFactory.mu's comment said "never held\ across I/O" while mintSlot.mu is deliberately held across the exchange\ (that's the single-flight). Say both halves explicitly so neither\ comment reads as contradicting the other. Comment-only change.\ \ Co-Authored-By: Claude Fable 5 noreply@anthropic.com\ Entire-Checkpoint: c8ee1a75c9a4+8/-6