docs(backlog): mark plan-lint Slice 1 shipped, add QA plan

2026-06-15 00:27:31 +04:00
parent 68136e77d7
commit 63863a90ae
2 changed files with 75 additions and 6 deletions
@@ -0,0 +1,62 @@
+# QA Plan: deterministic plan lint — 68136e7
+
+Plan-generation pipeline epic, Slice 1 (plan-pipeline-spec §5). Drafted from commit `68136e7`.
+
+**Status:** DRAFT
+**Run date / operator:**
+**BACKLOG item:** §C★ "Slice 1 — §5 deterministic lint".
+
+---
+
+Note: the lint logic is **deterministic and unit-covered** (`PlanLinterTest` exercises every
+hard/soft check incl. the review-loop-vs-trap nuance; `FreestyleDriverTest` covers reject-on-hard-fail
+and record-on-clean). This live pass is **confirmatory** — it verifies the wiring fires in a real
+freestyle run and the event surfaces in the log.
+
+## Preconditions
+
+- [ ] **server build/branch:** master @ `68136e7` (or later) — _rebuild only with the server stopped_.
+- [ ] **llama-server + model:** a model capable of driving the freestyle planning phase to emit an
+      `execution_plan` (needed to reach `FreestyleDriver.lockAndRun`). The lint itself needs no model.
+- [ ] **how to force a bad plan:** to exercise the reject path you need a plan that *compiles* but fails
+      lint — i.e. a stage whose `needs` names an artifact no stage produces, or an inescapable loop. Easiest
+      is a crafted `execution_plan` (seed/stub the planning output) rather than hoping the model emits one.
+- [ ] **fixtures/seed:** a freestyle session (workingDir) that reaches the plan-lock gate.
+
+## Acceptance gate (one sentence)
+
+> The lint is correct **iff** a compiled freestyle plan with an unproduced `needs` or an inescapable
+> (exit-less) loop is rejected with `ExecutionPlanRejectedEvent(source="lint")` — recording a
+> `PlanLintCompletedEvent` with the matching hard failure, never locking and never executing — while a
+> clean plan (including one with a legitimate verdict-gated review loop) records a passing
+> `PlanLintCompletedEvent` and proceeds, and soft findings never block.
+
+## Checks
+
+| # | Action | Expected observable evidence | Result |
+|---|--------|------------------------------|--------|
+| 1 | `grep PlanLintCompleted core/events/.../serialization/Serialization.kt` | `subclass(PlanLintCompletedEvent::class)` present (unregistered = silent runtime deser failure) | |
+| 2 | Run a freestyle session to a **clean** plan | `correx events <id>` shows `PlanLintCompleted` with empty `hardFailures`, then `ExecutionPlanLocked`, then execution | |
+| 3 | Drive freestyle with a plan whose stage `needs` an artifact no stage produces | `PlanLintCompleted` with a `unproduced_need` hard failure naming the stage+artifact; `ExecutionPlanRejected` `source="lint"`; **no** `ExecutionPlanLocked`; no stage executes | |
+| 4 | Drive a plan with an inescapable loop (cycle with no edge to `done`) | `PlanLintCompleted` with `trap_state` hard failure(s); rejected `source="lint"`; no lock | |
+| 5 | Drive a plan with a legit verdict-gated review loop (implementer⇄reviewer, reviewer→done) | **No** `trap_state`; `PlanLintCompleted` passes; plan locks and runs (the nuance: cycles-with-exit are fine) | |
+| 6 | Drive a clean-but-noisy plan (>4 fan-out, or two identical stage briefs) | `PlanLintCompleted` lists `fan_out`/`duplicate_brief` in `softFindings` with `score>0`, **but the plan still locks** (soft = non-blocking) | |
+| 7 | `correx replay <id>` on a session from check 3/4 | The recorded `PlanLintCompleted` is read back identically; lint is not recomputed against a changed repo (pure function of the recorded plan); double-read digest stable | |
+
+Evidence sources: `correx events <id>` (the `PlanLintCompleted` + `ExecutionPlanRejected`/`Locked` chain),
+server logs (the "plan failed lint" WARN with the summary), and `correx replay <id>` for determinism.
+
+## Out of scope (explicitly NOT covered this pass — later slices)
+
+- File-path grounding + symbol resolution (Slice 1b — symbols need an index, BACKLOG §B §3).
+- Multi-candidate generation / templates (Slice 2), critique/aggregate (Slice 3), tournament/selection +
+  the §E comparison UI (Slice 4), preference logging (Slice 5).
+- Token-budget and ADR-0000 harness-bypass soft/hard checks.
+- `role_pipeline` `impl_plan` lint (freestyle `execution_plan` only this slice).
+
+## Disposition
+
+- **PASS** → mark §C★ Slice 1 done; cite `68136e7` + this plan's run date/evidence. (Keep the epic open;
+  the slice is one step of it — no RETRO move until the epic completes or you choose to retire the slice.)
+- **FAIL** → file each failure as a numbered finding back into `BACKLOG.md` (action + repro + wrong/missing
+  signal), fix, re-run only the failed checks. Set Status: FAILED until green.