inspect: stop default judge prompts from padding reports · Entire

inspect: stop default judge prompts from padding reports

1689b25→main·

dipree·1mo ago·2 files·+17 added/-20 removed

The default profile ships a single judge, and the synthesis prompt mandated a fixed six-section report with no length cap and no omit-empty rule, so even small/clean changes produced long documents padded with 'none' filler under every header.

Rework the judge and chair instructions to lead with a one-line verdict, include a section only when it has real content, and stay proportional to the change without inventing findings to fill a template. The section keywords (Common/Unique findings, Disagreements, Priority order) are retained where relevant; the per-judge panel appendix (multi-judge only) is unchanged.

Sessions

3572e91bd21fView transcript

Changes

2

96 unmodified lines

97
98
99
100
101
102
103
100
101
102
103
104
105
106
107

96 unmodified lines

var b strings.Builder
    b.WriteString("You are the presiding judge on a panel reviewing a code change. " +
        "Several judges independently evaluated the inspectors' reports and produced the verdicts below. " +
        "Write a single final verdict that reconciles them: state the decision, " +
        "explicitly call out where the judges AGREED and where they DISAGREED, and " +
        "resolve disagreements on the merits (note which judge is right and why). " +
        "Do not simply concatenate the verdicts.\n\n")
        "Write a single, concise final verdict that reconciles them:\n" +
        "  - Open with the decision and a one-sentence rationale.\n" +
        "  - Note only the substantive points of agreement and disagreement; resolve each disagreement on the merits (which judge is right and why).\n" +
        "  - Merge duplicates and drop unsupported claims. Do not concatenate the verdicts or repeat findings verbatim.\n" +
        "  - Keep it proportional to the change; do not pad.\n\n")
    for _, i := range ok {
        label := fmt.Sprintf("judge %d", i+1)
        if i < len(labels) && strings.TrimSpace(labels[i]) != "" {

Mcmd/entire/cli/review/synthesis_panel.go+5/-4

25 unmodified lines

26
27
28
29
30
31
32
33
34
35
29
30
31
32
33
34
35
36
37
38
39
40
41
34 unmodified lines

76
77
78
79
79
81
82
83
84
85
86
87
80
81
82
83
84
89
85
86
87
88

25 unmodified lines

//
// 	...
//
//
//
//
//
//
// Synthesize a unified verdict with these sections:
//   - Common findings (issues all agents flagged)
//   - Unique findings (issues only one agent caught)
//   - Disagreements (areas where agents reached different conclusions)
//   - Priority order (top 5 issues to address first)
//
//   Be concise; aim for ~300 words.
//  <instructions to write a tight verdict: lead with a one-line decision,
//  then only the sections that have real content (Common/Unique findings,
//  Disagreements, Priority order), proportional to the size of the change>
//
//  <perRunPrompt, if any 					— appended as user's per-run instructions>
//
// The instructions deliberately do not mandate a fixed multi-section template:
// forcing every header produced padded "none" filler on small/clean changes,
// so sections are opt-in and the judge is told to stay proportional.
//
// Agents with no usable narrative (empty AssistantText) are filtered out
// upstream by usableAgentRuns, so the header count and the body are both
// scoped to agents that produced narrative output. SynthesisSink already
34 unmodified lines

- Discard unsupported or speculative claims unless they are clearly labeled as needing verification.
  - Identify contradictions between workers and decide which claim is better supported.
  - Merge duplicate findings.
  - Call out important uncertainty instead of pretending certainty.

Produce one canonical final report with these sections:
  - Executive verdict
  - Common findings / high-confidence findings, prioritized
  - Unique findings worth keeping
  - Needs verification / uncertain findings
  - Disagreements or rejected false positives
  - Priority order / recommended next actions
Write a tight final report:
  - Open with a one-line verdict (approve / approve with nits / request changes) and a one-sentence rationale.
  - Then list only the findings that matter, highest priority first, each as a single bullet with an evidence pointer.
  - Include a section only when it has real content; omit empty sections instead of writing "none". Use these as needed, in this order: Common findings, Unique findings, Disagreements (or rejected false positives), Priority order / next actions.

Be concise but specific; include evidence pointers where available.`
Be brief and proportional to the change: a small or clean change should get a verdict and a few bullets, nothing more. Do not pad, restate the diff, or invent findings to fill a template.`

if perRunPrompt != "" {
        b.WriteString("\n\nPer-run user instructions:\n")