Files

T

kami 12ff2e97b0 docs(qa): live-QA plans + env bring-up for the burndown features

Six QA plans (docs/qa/QA-*.md) pinning observable evidence for the shipped tracks:
research egress + batch source-approval (D), architect contradiction (B§4), llama health
probe (A§4), idea promotion (E), reviewer static-first filter (B§5, partial), and the
brief-echo gate ARM-IT plan (C-A1) — the latter is the go/no-go for arming the gate in
production (it breaks the planner if the model can't emit a parseable brief_echo).

Plus the env: docs/qa/ENV.md runbook + docs/qa/README.md index, and scripts/qa/
(sync-config.sh, searxng-up/down.sh with the JSON-format gotcha, README). Scripts are
set -euo pipefail, syntax-checked, executable. Authored from the repo build/config
(server :apps:server:run, CLI :apps:cli:run, tui-go GOTOOLCHAIN=auto, sample-config.toml).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-06-21 09:14:36 +00:00

3.1 KiB

Raw Blame History

QA Plan: reviewer static-first (findings exclusion) — `447fc7a`

Status: DRAFT (PARTIAL — see gap) Run date / operator: BACKLOG item: §B-§5 "reviewer static-first INFRA" — runner + event + context-exclusion 447fc7a.

Gap up front (read first)

The exclusion filter is live-wired into reviewer context assembly, and StaticAnalysisRunner + StaticFindingsRecordedEvent are built and unit-tested — but no stock workflow stage emits StaticFindingsRecordedEvent yet (the deterministic static-check stage-type seam is an open BACKLOG follow-up). So this pass verifies the filter behavior by injecting the event; the end-to-end "a stage runs detekt and the reviewer skips those findings" flow is blocked on that seam.

Preconditions

server build/branch: feat/backlog-burndown.
llama-server + model: a model that runs review_loop / role_pipeline reviewer stage.
external deps: none.
config synced: review_loop.toml (or role_pipeline) present.
fixtures/seed: a way to append a StaticFindingsRecordedEvent for the session (manual event-store seed), carrying a finding that matches a compile/lint issue the code actually has.

Acceptance gate (one sentence)

When a StaticFindingsRecordedEvent has already recorded a static finding this session, the reviewer's assembled context mechanically omits any feedback entry that merely restates that finding.

Checks

#	Action	Expected observable evidence
1	Seed a `StaticFindingsRecordedEvent` for the session with a known finding (e.g. an unused-import lint at `Foo.kt:12`).	The event is in `correx events {id}`.
2	Drive the implementer→reviewer step so reviewer context is assembled.	The reviewer's input context (server log of the assembled prompt / context pack) does not contain a feedback entry restating the seeded static finding — `excludeStaticFindingsFromReview` stripped it.
3	Seed a finding that does not match any retry-feedback entry.	Non-matching entries are untouched; only restatements of recorded static findings are dropped.
4	(When the static-check stage seam lands) run a workflow whose static stage emits the event for real.	The reviewer never re-raises the compiler/detekt issues the deterministic tool already caught. (Deferred — seam not built.)

Evidence sources: server logs of the assembled reviewer context / context pack, correx events {id}.

Out of scope (explicitly NOT covered this pass)

The producing static-check stage (open BACKLOG §B-§5 follow-up: "a stage that emits StaticFindingsRecordedEvent"). Check 4 stays deferred until that seam exists.
The reviewer prompt half (shipped earlier, 3467826).

Disposition

PASS (partial) → record in RETRO.md that the filter is live-verified (checks 1–3) and keep the §B-§5 producing-stage follow-up open in BACKLOG. Do not claim the full static-first loop until check 4 passes.
FAIL → numbered findings. Status: FAILED until green.

3.1 KiB Raw Blame History Unescape Escape

QA Plan: reviewer static-first (findings exclusion) — 447fc7a