Add inspect + judge-panel handoff doc · Entire
Add inspect + judge-panel handoff doc
Sessions
Changes
1
docs/handoff
Ainspect-judges-handoff.md+168
Handoff — entire inspect + Judge Panel
Last updated: 2026-06-14 · branch review-profiles @ ead8c12dc
TL;DR
entire inspect is the evolved multi-agent code-inspection command (formerly the hidden review). A profile runs a crew of inspectors (parallel review agents) and then a panel of judges that render the verdict; with ≥2 judges a chair merges them. The command, profiles, guided setup, scripted config, --list, and the judge panel are implemented, building, and green. The Pi-specific work lives in a separate stacked PR (#1313) that now needs another rebase.
- Build:
go build ./...✓ - Tests:
go test ./...→ 72 ok, 0 fail ✓ - Lint:
golangci-lint→ 0 issues on touched packages ✓ review-profilesis pushed; local ==origin/review-profiles==ead8c12dc.
Terminology (final)
- inspectors — the worker agents that review the change in parallel (
agentsin settings). - judges — the panel that evaluates inspector reports (
judgesin settings). - chair — the judge that merges a ≥2-judge panel into the final verdict (
chairin settings; defaults to the first judge). - Command is
entire inspect;entire reviewis a kept alias (it shipped onmain).entire scoutwas removed (never shipped). - Internal identifiers, settings keys, and env vars are still
review_*/ENTIRE_REVIEW_*— only user-facing surfaces use inspector/judge/chair.
Command surface
entire inspect # interactive: profile chooser. non-interactive: list + error (never silent default)
tire inspect <profile> # run a named profile
tire inspect --list # list profiles (inspectors + judges, default marked)
tire inspect --configure # interactive wizard; non-interactive discovery view
tire inspect --configure --profile P \
--set-agents claude-code,codex \ # inspectors (simple)
--set-slot claude-code=opus --set-slot codex \ # inspector slots (dupes ok)
--set-judge claude-code=opus --set-judge codex=gpt-5 \ # judges (repeatable; >1 = panel)
--set-chair claude-code=opus \ # chair for a panel
--set-model codex=gpt-5-codex --set-task "..."
tire inspect --edit # advanced skill picker
tire inspect --agent N # run one inspector
tire inspect --agent N --model M
tire inspect --agents # list inspectors (valid --agent values)
tire inspect --models [--agent N]
tire inspect --prompt "..." # one-off instructions
tire inspect --findings # browse local findings
tire attach --review <id> # post-hoc tag a session (the old `review attach` was removed)
Settings schema (review_profiles)
{
"review_default_profile": "general",
"review_profiles": {
"general": {
"task": "Review this change for correctness, regressions, tests, and maintainability.",
"agents": { "claude-code": {"skills": ["/review"]}, "codex": {"skills": ["/review"]} },
"judges": [{"agent": "claude-code", "model": "opus"}]
},
"security": {
"task": "...",
"agents": { "claude-sonnet": {"agent": "claude-code", "model": "sonnet"}, "codex": {"model": "gpt-5-codex"} },
"judges": [{"agent": "claude-code", "model": "opus"}, {"agent": "codex", "model": "gpt-5"}],
"chair": "claude-code:opus"
}
}
}
Back-compat: legacy master (an inspector id) and master_agent / master_model are still honored as a single judge when judges is empty. New configs write judges/chair.
How the judge panel works
profileJudges(profile)resolves the panel[]judgeSpec+ chair index: explicitjudges→ legacymaster_agent→ legacy workermaster.PanelSynthesisProvider(synthesis_panel.go) implementsSynthesisProvider, soSynthesisSinkconsumes it unchanged:- fans out to each judge in parallel over the same synthesis prompt,
- one surviving verdict → passthrough,
- ≥2 → chair merges via
composeChairPrompt, individual verdicts appended as a## Panelsection, - failed/empty judges dropped; all-fail surfaces "final report unavailable".
- The chair runs twice by design: once as a panel judge (its own independent verdict) and once to merge the panel. This is intentional and commented in the code.
runMultiAgentPathbuilds anAgentSynthesisProviderfor a single judge or aPanelSynthesisProviderfor a panel. Validation requires only that ≥1 judge resolves; text-gen failures degrade gracefully at synthesis time.
Done
- Command renamed
review/scout→inspect;reviewalias kept;scoutremoved. - Bare
inspectrequires explicit selection (interactive chooser / non-interactive error+list). -
--listprofiles (inspectors + judges, chair marked). - Custom focus/task option in guided setup; guided setup edits the existing profile.
- Slot-based crew (
--set-slot, duplicates allowed). - Judge panel: schema,
profileJudges,PanelSynthesisProvider, chair merge, tests. - Scripted
--set-judge/--set-chair(replaced--set-master). - Guided picker:
pickSlotListfor inspectors + judges, chair pick for panels. - Inspector/judge/chair terminology across help,
--list, catalog, errors. - Dropped legacy
[master]marker in--agents;judges=in catalog. - Removed fabricated codex/gemini model lists (only claude-code advertises models).
- Codex JSON error envelopes surfaced instead of bare
exit status 1. - Pi-specific reviewer/model files kept out of
review-profiles(live on #1313). - Merged latest
origin/main(incl. attribution/trail work); clean. - Docs refreshed:
docs/architecture/review-command.md+CLAUDE.mdsummary. - Build/tests/lint/gofmt all green; branch pushed.
Pending / next steps
- Rebase PR #1313 (
review-pi-reviewer) ontoorigin/review-profiles(ead8c12dc). It is behind again after the judge-panel + merge + doc commits.- PR: https://github.com/entireio/cli/pull/1313 (base
review-profiles, headreview-pi-reviewer) - Contains: Pi review-runner adapter, Pi live model list (
pi --list-models), Pi generate/text-gen. The Pi adapter is the obvious first panel-capable text-gen judge to validate the panel end-to-end with a real second judge.
- PR: https://github.com/entireio/cli/pull/1313 (base
- Consider validating scripted
--set-judgeagents at config time (currently only validated at runtime, where failures are dropped). Intentional for now; revisit if users hit silent typos. - Optional: include the profile task / scope context in
composeChairPrompt(today the chair reconciles verdicts only). - Optional: bound judge-panel concurrency if panels ever grow large (currently unbounded; fine for 2–3 judges).
Key files
cmd/entire/cli/review/cmd.go—NewCommand, dispatch,--list, catalog,composeMultiAgentSinks, judge-panel wiring (runMultiAgentPath).cmd/entire/cli/review/picker.go— guided setup, focus picker,pickSlotList(inspectors + judges), chair picker, profile chooser.cmd/entire/cli/review/profile.go— profile resolution,profileJudges, default tasks.cmd/entire/cli/review/synthesis_panel.go(+_test.go) —PanelSynthesisProvider,composeChairPrompt.cmd/entire/cli/review/synthesis_sink.go/synthesis_prompt.go— verdict sink.cmd/entire/cli/review/marker_fallback.go— manual fallback for non-adapter agents.cmd/entire/cli/review/env.go—ENTIRE_REVIEW_*constants + skills codec.cmd/entire/cli/agent/model_lister.go—ModelListercapability.cmd/entire/cli/agent/claudecode/models.go— only realModelListeron this branch.cmd/entire/cli/settings/settings.go—ReviewProfileConfig(Agents,Judges,Chair, legacyMaster/MasterAgent/MasterModel).cmd/entire/cli/attach.go—entire attach --review(consumes pending marker).docs/architecture/review-command.md— full architecture reference (current).
Verify
go build ./...
go test ./... # expect 72 ok, 0 fail
golangci-lint run ./cmd/entire/cli/review/... ./cmd/entire/cli/ ./cmd/entire/cli/settings/...
go run ./cmd/entire inspect --list
go run ./cmd/entire inspect --help
Gotchas
- Entire's pre-push hook also pushes
entire/checkpoints/v1to a checkpoint remote; if that errors (signal: killed/ unreachable), it does not mean the code branch failed to push — verify withgit ls-remote origin review-profiles. [entire-dev] project isn't compiling; falling back to the entire binary on PATHduring git ops is expected noise from the dev hook, not an error.