correx/docs/qa/QA-static-first-reviewer.md

QA Plan: static-first reviewer gate — 365eb8a

Role-reliability §5 (static-first ordering, infra half). Drafted from the diff (commit 365eb8a) per the BACKLOG QA rule. The reviewer prompt half shipped earlier in 3467826.

Status: DRAFT Run date / operator: BACKLOG item: §B "§5 reviewer static-first INFRA".

---

Preconditions

• server build/branch: master @ 365eb8a (or later) — rebuild only with the server stopped.
• llama-server + model: a real implementer-capable model (the static gate runs on the implementer stage's output; exercising the retry/feedback path needs a model that actually writes files and can react to fed-back tool output). Stub providers won't drive the loop.
• external deps: none (no network) — but the configured commands must be runnable in the workspace (e.g. a Kotlin/Gradle repo for ./gradlew compileKotlin). Use a workspace where the command both exists and can be made to pass and fail on demand.
• config synced: copy the updated examples/workflows/role_pipeline.toml into ~/.config/correx/workflows/ and uncomment / set static_analysis = [...] on the implementer stage with commands valid for the test workspace. (Commented by default — a fresh copy runs with the gate OFF.)
• fixtures/seed: a workspace (StartSession workingDir) where the implementer will write code, and where you can arrange a deliberately-failing static check (e.g. a detekt violation / compile error).

Acceptance gate (one sentence)

The gate is correct iff a stage that declares static_analysis runs every configured command against its produced output, records a StaticAnalysisCompletedEvent with each command's exit code, lets the run proceed to the reviewer only when all commands are clean, and on any non-clean command fails the stage retryably (feeding the output back) so the reviewer never runs on broken code — while a stage that declares no commands behaves exactly as before.

Checks

#	Action	Expected observable evidence	Result
1	Serialization registration trap. grep StaticAnalysisCompleted core/events/.../serialization/Serialization.kt	subclass(StaticAnalysisCompletedEvent::class) present — without it the event deserializes silently wrong despite passing unit tests
2	Run role_pipeline (gate ON) where the implementer's output passes all configured commands	correx events <id> shows a StaticAnalysisCompleted event after the implementer stage with every finding exitCode = 0; flow transitions implementer → reviewer; a review_report is produced
3	Inspect that StaticAnalysisCompleted event	findings[] lists each configured command with its exit code and a non-empty summary (output tail); the command strings match the TOML static_analysis list
4	Arrange the implementer's output to FAIL one command (e.g. introduce a detekt/compile error in the written code)	StaticAnalysisCompleted with that command exitCode != 0; the implementer stage fails retryably (a StageFailed/retry, NOT a transition to reviewer); the reviewer has not produced a review_report for this attempt
5	Let the retry run with the fed-back output	The next implementer attempt sees the failing command's output verbatim in its context; once it fixes the issue the next StaticAnalysisCompleted is all-clean and flow proceeds to the reviewer; retries bounded by implementer.max_retries, then surfaces to the operator
6	Absence check — gate OFF. Run any workflow / a role_pipeline copy with static_analysis left commented	No StaticAnalysisCompleted event appears; the implementer→reviewer transition behaves exactly as before
7	No-runner / no-workspace no-op. (If reproducible) run a stage declaring static_analysis but with no workspace root	Server log WARN "stage declares static_analysis but no runner/workspace root is wired — skipping"; the run proceeds (no block), and no StaticAnalysisCompleted event
8	correx replay <id> on a session that recorded a mismatch (check 4)	The recorded StaticAnalysisCompleted findings are read back as-is; replay does not re-run the commands (no new process output, deterministic double-read digest)

Evidence sources: correx events <id> (the StaticAnalysisCompleted chain + retry/stage events), server logs (MDC sessionId, the skip WARN), the workspace files for the introduced/fixed defect, and correx replay <id> for determinism.

Out of scope (explicitly NOT covered this pass)

• Per-command timeout tuning (ProcessStaticAnalysisRunner default 5 min) — adjust only if a real build legitimately exceeds it.
• Quoted/space-containing command arguments — commands are whitespace-split into argv (no shell); keep configured commands simple. Revisit only if a real command needs shell semantics.
• Wiring the gate onto freestyle execution plans / any stage other than the role_pipeline implementer.
• Structured parsing of tool output into per-line findings, and the shared CritiqueFinding schema (§6) — deliberately not built; the gate works on exit codes + raw output tails.

Disposition

• PASS → MOVE the §B "§5 reviewer static-first INFRA" entry into RETRO.md (cite 365eb8a + 3467826 for the prompt half, run date, evidence). Set Status: PASSED.
• FAIL → file each failure as a numbered finding back into BACKLOG.md (action + exact repro + the wrong/missing signal), fix, re-run only the failed checks. Set Status: FAILED until green.