e2e: keep roger-roger off rerun-fails (PR #1577 review)

e20f01d→main·

toothbrush·2w ago·1 file·+9 added/-1 removed

The default test:e2e task now enables gotestsum --rerun-fails=1, but the workflow invoked the deterministic roger-roger agent through that same task, so a real roger-roger regression could be retried and masked. Route roger-roger through its dedicated test:e2e:roger-roger task, which has no reruns.

Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com

Sessions

7b7e2efbfbbfView transcript

[?
Make E2E Tests Reliable on MainClaude Code·Opus 4.8[1m]·4 steps](/content/gh/entireio/cli/session/cb256a5c-45a5-4513-be01-ea2f88739aec#timeline-7b7e2efbfbbf/index.html)

Changes

1

• .github/workflows

• Me2e.yml+9/-1

108 unmodified lines

109
110
111
112
112
113
114
115
116
117
118
119
120
121
122
123

108 unmodified lines

FACTORY_API_KEY: ${{ secrets.FACTORY_API_KEY }}
          COPILOT_GITHUB_TOKEN: ${{ github.token }}
          E2E_CONCURRENT_TEST_LIMIT: ${{ matrix.agent == 'gemini-cli' && '6' || matrix.agent == 'factoryai-droid' && '1' || matrix.agent == 'cursor-cli' && '2' || '' }}
        run: mise run test:e2e --agent ${{ matrix.agent }} ${{ matrix.agent == 'roger-roger' && 'TestExternalAgent' || '' }}
        # roger-roger is deterministic, so it runs through its dedicated task,
        # which does NOT enable --rerun-fails. Routing it through the default
        # task (`test:e2e`) would retry a real regression and mask it.
        run: |
          if [ "${{ matrix.agent }}" = "roger-roger" ]; then
            mise run test:e2e:roger-roger TestExternalAgent
          else
            mise run test:e2e --agent ${{ matrix.agent }}
          fi

- name: Upload artifacts
        if: always()

M.github/workflows/e2e.yml+9/-1