kami
33baaed903
Per the hygiene rule: the §B-§5 static_check stage seam + emitter and the §B-§6/A3 critique-outcome producer are built and unit-verified, so move them out of the live "missing" list. Both narrowed to their remaining live-QA-gated *activations* (a sandboxed production CommandRunner for §B-§5; model-emitted CritiqueFindingsRecordedEvents for §6), with a dated RETRO entry + commit map. Updated QA-reviewer-static-first (the producing stage now exists — check 4 reframed from "blocked on unbuilt seam" to an activation step) and its README index line. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
45 lines
3.8 KiB
Markdown
45 lines
3.8 KiB
Markdown
# QA Plan: reviewer static-first (findings exclusion) — `447fc7a`
|
||
|
||
**Status:** DRAFT (PARTIAL — see gap)
|
||
**Run date / operator:**
|
||
**BACKLOG item:** §B-§5 "reviewer static-first" — runner + event + context-exclusion `447fc7a`; deterministic `static_check` stage seam + emitter `e46777e`.
|
||
|
||
---
|
||
|
||
## Gap up front (read first)
|
||
|
||
The exclusion **filter** is live-wired into reviewer context assembly; `StaticAnalysisRunner` + `StaticFindingsRecordedEvent` are built; and the deterministic **`static_check` stage seam + emitter now exist** (`e46777e`): the orchestrator runs any `stage_type = "static_check"` stage through `StaticCheckStageExecutor`, and `role_pipeline.toml` carries one between implementer and reviewer. The remaining gap is **activation**: that stage is a **no-op until** a sandboxed, workspace-scoped `CommandRunner` is wired into the orchestrator AND the stage sets `static_argv`. So checks 1–3 verify the **filter by injecting the event** (unchanged); check 4 is now an **activation** check (configure `static_argv` + wire a runner) rather than "blocked on an unbuilt seam".
|
||
|
||
## Preconditions
|
||
|
||
- [ ] **server build/branch:** `feat/backlog-burndown`.
|
||
- [ ] **llama-server + model:** a model that runs `review_loop` / `role_pipeline` reviewer stage.
|
||
- [ ] **external deps:** none.
|
||
- [ ] **config synced:** `review_loop.toml` (or role_pipeline) present.
|
||
- [ ] **fixtures/seed:** a way to append a `StaticFindingsRecordedEvent` for the session (manual event-store seed), carrying a finding that matches a compile/lint issue the code actually has.
|
||
|
||
## Acceptance gate (one sentence)
|
||
|
||
> When a `StaticFindingsRecordedEvent` has already recorded a static finding this session, the reviewer's assembled context **mechanically omits** any feedback entry that merely restates that finding.
|
||
|
||
## Checks
|
||
|
||
| # | Action | Expected observable evidence | Result |
|
||
|---|--------|------------------------------|--------|
|
||
| 1 | Seed a `StaticFindingsRecordedEvent` for the session with a known finding (e.g. an unused-import lint at `Foo.kt:12`). | The event is in `correx events {id}`. | |
|
||
| 2 | Drive the implementer→reviewer step so reviewer context is assembled. | The reviewer's input context (server log of the assembled prompt / context pack) **does not** contain a feedback entry restating the seeded static finding — `excludeStaticFindingsFromReview` stripped it. | |
|
||
| 3 | Seed a finding that does **not** match any retry-feedback entry. | Non-matching entries are untouched; only restatements of recorded static findings are dropped. | |
|
||
| 4 | **Activate** the `static_check` stage: wire a sandboxed `CommandRunner` into the orchestrator and set `static_argv` (e.g. `./gradlew detekt --console=plain`) on the role_pipeline stage, then run a workflow that produces a patch with a real lint/compile issue. | A `StaticFindingsRecordedEvent` with the tool's parsed findings appears in `correx events {id}`, and the reviewer never re-raises those issues. _(Activation is the open §B-§5 follow-up — the seam + emitter are built.)_ | |
|
||
|
||
Evidence sources: server logs of the assembled reviewer context / context pack, `correx events {id}`.
|
||
|
||
## Out of scope (explicitly NOT covered this pass)
|
||
|
||
- **Production runner activation** (open BACKLOG §B-§5 follow-up): wiring a sandboxed, workspace-scoped `CommandRunner` is itself live-QA-gated. The seam + emitter are built (`e46777e`) and unit-verified (`StaticCheckStageExecutorTest`, `StaticCheckStageTest`); check 4 exercises the activation once a runner is wired.
|
||
- The reviewer prompt half (shipped earlier, `3467826`).
|
||
|
||
## Disposition
|
||
|
||
- **PASS (partial)** → record in `RETRO.md` that the **filter** is live-verified (checks 1–3) and keep the §B-§5 *production-runner activation* follow-up open in BACKLOG. Do not claim the full static-first loop until check 4 passes.
|
||
- **FAIL** → numbered findings. Status: FAILED until green.
|