From 63863a90ae3913f890a53d0a49a2afc6f9bd418c Mon Sep 17 00:00:00 2001 From: kami Date: Mon, 15 Jun 2026 00:27:31 +0400 Subject: [PATCH] docs(backlog): mark plan-lint Slice 1 shipped, add QA plan --- BACKLOG.md | 19 +++++++++---- docs/qa/QA-plan-lint.md | 62 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+), 6 deletions(-) create mode 100644 docs/qa/QA-plan-lint.md diff --git a/BACKLOG.md b/BACKLOG.md index a52a306a..a338b00f 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -120,12 +120,15 @@ built analog is `ExecutionPlanLockedEvent` (extend it with a candidate ref; don' Slices (spec §, in build order): -- [ ] **Slice 1 — §5 deterministic lint** *(in progress — see plan)*. Pure Kotlin, zero inference: - `PlanLinter` over the compiled `WorkflowGraph` → hard failures (missing file paths, unresolved - symbols, unproduced `needs`, produces-kind not in registry, ADR-0000 violations) discard a - candidate; soft findings (stage count, fan-out, budget, dup briefs) are scored. Records - `PlanLintCompletedEvent` (env observations — repo-map/symbol existence — per invariant #9). Wired - into `FreestyleDriver` before lock so it pays off **now** for single-plan freestyle. Standalone. +- [~] **Slice 1 — §5 deterministic lint** — **SHIPPED (pending live-QA)** `68136e7`. Pure-Kotlin + `PlanLinter` over the compiled `WorkflowGraph`, run in `FreestyleDriver` before lock; records + `PlanLintCompletedEvent` (pure function of the recorded plan → replay-safe, no env observation), + and a hard failure → `ExecutionPlanRejectedEvent(source="lint")` before the operator is asked or + the plan locks. **Hard:** `unproduced_need` (real gap — the freestyle compiler never checked needs), + `trap_state` (no path to terminal; uses terminal-reachability so verdict-gated review loops pass). + **Soft (non-blocking):** `stage_count`, `fan_out`, `empty_brief`, `duplicate_brief`. QA gate in §F → + `docs/qa/QA-plan-lint.md`. **Deferred to Slice 1b:** file-path grounding + symbol resolution (env; + symbols need an index, §B §3), token-budget/ADR-bypass checks, a dedicated `:core:planning` module. - [ ] **Slice 2 — §4 multi-candidate round-1 + templates.** `PlanTemplate` library, template-select + slot-fill, GBNF-constrained, N₁=3 high-temp. `PlanCandidateGeneratedEvent`. - [ ] **Slice 3 — §6 critique + §7 aggregate.** One-shot GBNF critic per survivor → @@ -213,6 +216,10 @@ These are SHIPPED in code but prompt-/server-/network-dependent and not yet live gate uncommented in the copied `role_pipeline.toml`). Plan: `docs/qa/QA-static-first-reviewer.md` (clean→reviewer runs, non-clean→`StaticAnalysisCompleted`+retryable fail w/ output fed back, gate-OFF absence check, no-runner no-op WARN, replay reads recorded findings without re-running). +- [ ] **Plan-lint (§C★ Slice 1) live-QA** — `68136e7`; logic is unit-covered, so this is confirmatory: + needs a freestyle run reaching the plan-lock gate (+ a crafted bad plan to force the reject path). + Plan: `docs/qa/QA-plan-lint.md` (unproduced_need/trap_state → `PlanLintCompleted`+`ExecutionPlan- + Rejected(source=lint)`+no lock; review-loop NOT trapped; soft findings non-blocking; replay stable). - [ ] **TUI kernel-steering + AMD gauge fresh live-QA** — approve+note actually revising the *same* stage's output; AMD VRAM/RAM gauge on the real box. - [ ] **Unit-tested-only fixes, not live-verified:** `1a7eb05` (verdict-edge rejection), diff --git a/docs/qa/QA-plan-lint.md b/docs/qa/QA-plan-lint.md new file mode 100644 index 00000000..61185232 --- /dev/null +++ b/docs/qa/QA-plan-lint.md @@ -0,0 +1,62 @@ +# QA Plan: deterministic plan lint — 68136e7 + +Plan-generation pipeline epic, Slice 1 (plan-pipeline-spec §5). Drafted from commit `68136e7`. + +**Status:** DRAFT +**Run date / operator:** +**BACKLOG item:** §C★ "Slice 1 — §5 deterministic lint". + +--- + +Note: the lint logic is **deterministic and unit-covered** (`PlanLinterTest` exercises every +hard/soft check incl. the review-loop-vs-trap nuance; `FreestyleDriverTest` covers reject-on-hard-fail +and record-on-clean). This live pass is **confirmatory** — it verifies the wiring fires in a real +freestyle run and the event surfaces in the log. + +## Preconditions + +- [ ] **server build/branch:** master @ `68136e7` (or later) — _rebuild only with the server stopped_. +- [ ] **llama-server + model:** a model capable of driving the freestyle planning phase to emit an + `execution_plan` (needed to reach `FreestyleDriver.lockAndRun`). The lint itself needs no model. +- [ ] **how to force a bad plan:** to exercise the reject path you need a plan that *compiles* but fails + lint — i.e. a stage whose `needs` names an artifact no stage produces, or an inescapable loop. Easiest + is a crafted `execution_plan` (seed/stub the planning output) rather than hoping the model emits one. +- [ ] **fixtures/seed:** a freestyle session (workingDir) that reaches the plan-lock gate. + +## Acceptance gate (one sentence) + +> The lint is correct **iff** a compiled freestyle plan with an unproduced `needs` or an inescapable +> (exit-less) loop is rejected with `ExecutionPlanRejectedEvent(source="lint")` — recording a +> `PlanLintCompletedEvent` with the matching hard failure, never locking and never executing — while a +> clean plan (including one with a legitimate verdict-gated review loop) records a passing +> `PlanLintCompletedEvent` and proceeds, and soft findings never block. + +## Checks + +| # | Action | Expected observable evidence | Result | +|---|--------|------------------------------|--------| +| 1 | `grep PlanLintCompleted core/events/.../serialization/Serialization.kt` | `subclass(PlanLintCompletedEvent::class)` present (unregistered = silent runtime deser failure) | | +| 2 | Run a freestyle session to a **clean** plan | `correx events ` shows `PlanLintCompleted` with empty `hardFailures`, then `ExecutionPlanLocked`, then execution | | +| 3 | Drive freestyle with a plan whose stage `needs` an artifact no stage produces | `PlanLintCompleted` with a `unproduced_need` hard failure naming the stage+artifact; `ExecutionPlanRejected` `source="lint"`; **no** `ExecutionPlanLocked`; no stage executes | | +| 4 | Drive a plan with an inescapable loop (cycle with no edge to `done`) | `PlanLintCompleted` with `trap_state` hard failure(s); rejected `source="lint"`; no lock | | +| 5 | Drive a plan with a legit verdict-gated review loop (implementer⇄reviewer, reviewer→done) | **No** `trap_state`; `PlanLintCompleted` passes; plan locks and runs (the nuance: cycles-with-exit are fine) | | +| 6 | Drive a clean-but-noisy plan (>4 fan-out, or two identical stage briefs) | `PlanLintCompleted` lists `fan_out`/`duplicate_brief` in `softFindings` with `score>0`, **but the plan still locks** (soft = non-blocking) | | +| 7 | `correx replay ` on a session from check 3/4 | The recorded `PlanLintCompleted` is read back identically; lint is not recomputed against a changed repo (pure function of the recorded plan); double-read digest stable | | + +Evidence sources: `correx events ` (the `PlanLintCompleted` + `ExecutionPlanRejected`/`Locked` chain), +server logs (the "plan failed lint" WARN with the summary), and `correx replay ` for determinism. + +## Out of scope (explicitly NOT covered this pass — later slices) + +- File-path grounding + symbol resolution (Slice 1b — symbols need an index, BACKLOG §B §3). +- Multi-candidate generation / templates (Slice 2), critique/aggregate (Slice 3), tournament/selection + + the §E comparison UI (Slice 4), preference logging (Slice 5). +- Token-budget and ADR-0000 harness-bypass soft/hard checks. +- `role_pipeline` `impl_plan` lint (freestyle `execution_plan` only this slice). + +## Disposition + +- **PASS** → mark §C★ Slice 1 done; cite `68136e7` + this plan's run date/evidence. (Keep the epic open; + the slice is one step of it — no RETRO move until the epic completes or you choose to retire the slice.) +- **FAIL** → file each failure as a numbered finding back into `BACKLOG.md` (action + repro + wrong/missing + signal), fix, re-run only the failed checks. Set Status: FAILED until green.