correx/docs/qa/QA-static-first-reviewer.md

# QA Plan: static-first reviewer gate — 365eb8a

Role-reliability §5 (static-first ordering, infra half). Drafted from the diff (commit `365eb8a`)
per the BACKLOG QA rule. The reviewer prompt half shipped earlier in `3467826`.

**Status:** DRAFT
**Run date / operator:**
**BACKLOG item:** §B "§5 reviewer static-first INFRA".

---

## Preconditions

- [ ] **server build/branch:** master @ `365eb8a` (or later) — _rebuild only with the server stopped_.
- [ ] **llama-server + model:** a real implementer-capable model (the static gate runs on the
      *implementer* stage's output; exercising the retry/feedback path needs a model that actually
      writes files and can react to fed-back tool output). Stub providers won't drive the loop.
- [ ] **external deps:** none (no network) — but the **configured commands must be runnable** in the
      workspace (e.g. a Kotlin/Gradle repo for `./gradlew compileKotlin`). Use a workspace where the
      command both exists and can be made to pass *and* fail on demand.
- [ ] **config synced:** copy the updated `examples/workflows/role_pipeline.toml` into
      `~/.config/correx/workflows/` and **uncomment / set** `static_analysis = [...]` on the
      `implementer` stage with commands valid for the test workspace. (Commented by default —
      a fresh copy runs with the gate OFF.)
- [ ] **fixtures/seed:** a workspace (`StartSession` workingDir) where the implementer will write code,
      and where you can arrange a deliberately-failing static check (e.g. a detekt violation / compile error).

## Acceptance gate (one sentence)

> The gate is correct **iff** a stage that declares `static_analysis` runs every configured command
> against its produced output, records a `StaticAnalysisCompletedEvent` with each command's exit code,
> lets the run proceed to the reviewer only when all commands are clean, and on any non-clean command
> fails the stage *retryably* (feeding the output back) so the reviewer never runs on broken code —
> while a stage that declares no commands behaves exactly as before.

## Checks

| # | Action | Expected observable evidence | Result |
|---|--------|------------------------------|--------|
| 1 | **Serialization registration trap.** `grep StaticAnalysisCompleted core/events/.../serialization/Serialization.kt` | `subclass(StaticAnalysisCompletedEvent::class)` present — without it the event deserializes silently wrong despite passing unit tests | |
| 2 | Run `role_pipeline` (gate ON) where the implementer's output passes all configured commands | `correx events <id>` shows a `StaticAnalysisCompleted` event after the implementer stage with every finding `exitCode = 0`; flow transitions implementer → reviewer; a `review_report` is produced | |
| 3 | Inspect that `StaticAnalysisCompleted` event | `findings[]` lists each configured command with its exit code and a non-empty `summary` (output tail); the command strings match the TOML `static_analysis` list | |
| 4 | Arrange the implementer's output to FAIL one command (e.g. introduce a detekt/compile error in the written code) | `StaticAnalysisCompleted` with that command `exitCode != 0`; the implementer stage **fails retryably** (a `StageFailed`/retry, NOT a transition to reviewer); the reviewer has **not** produced a `review_report` for this attempt | |
| 5 | Let the retry run with the fed-back output | The next implementer attempt sees the failing command's output verbatim in its context; once it fixes the issue the next `StaticAnalysisCompleted` is all-clean and flow proceeds to the reviewer; retries bounded by `implementer.max_retries`, then surfaces to the operator | |
| 6 | **Absence check — gate OFF.** Run any workflow / a `role_pipeline` copy with `static_analysis` left commented | **No** `StaticAnalysisCompleted` event appears; the implementer→reviewer transition behaves exactly as before | |
| 7 | **No-runner / no-workspace no-op.** (If reproducible) run a stage declaring `static_analysis` but with no workspace root | Server log WARN "stage declares static_analysis but no runner/workspace root is wired — skipping"; the run proceeds (no block), and no `StaticAnalysisCompleted` event | |
| 8 | `correx replay <id>` on a session that recorded a mismatch (check 4) | The recorded `StaticAnalysisCompleted` findings are read back as-is; replay does **not** re-run the commands (no new process output, deterministic double-read digest) | |

Evidence sources: `correx events <id>` (the `StaticAnalysisCompleted` chain + retry/stage events),
server logs (MDC sessionId, the skip WARN), the workspace files for the introduced/fixed defect, and
`correx replay <id>` for determinism.

## Out of scope (explicitly NOT covered this pass)

- Per-command timeout tuning (`ProcessStaticAnalysisRunner` default 5 min) — adjust only if a real
  build legitimately exceeds it.
- Quoted/space-containing command arguments — commands are whitespace-split into argv (no shell);
  keep configured commands simple. Revisit only if a real command needs shell semantics.
- Wiring the gate onto `freestyle` execution plans / any stage other than the role_pipeline implementer.
- Structured parsing of tool output into per-line findings, and the shared `CritiqueFinding` schema
  (§6) — deliberately not built; the gate works on exit codes + raw output tails.

## Disposition

- **PASS** → MOVE the §B "§5 reviewer static-first INFRA" entry into `RETRO.md` (cite `365eb8a` +
  `3467826` for the prompt half, run date, evidence). Set Status: PASSED.
- **FAIL** → file each failure as a numbered finding back into `BACKLOG.md` (action + exact repro +
  the wrong/missing signal), fix, re-run only the failed checks. Set Status: FAILED until green.