docs(backlog): mark plan-lint Slice 1 shipped, add QA plan

This commit is contained in:
2026-06-15 00:27:31 +04:00
parent 68136e77d7
commit 63863a90ae
2 changed files with 75 additions and 6 deletions
+13 -6
View File
@@ -120,12 +120,15 @@ built analog is `ExecutionPlanLockedEvent` (extend it with a candidate ref; don'
Slices (spec §, in build order): Slices (spec §, in build order):
- [ ] **Slice 1 — §5 deterministic lint** *(in progress — see plan)*. Pure Kotlin, zero inference: - [~] **Slice 1 — §5 deterministic lint** **SHIPPED (pending live-QA)** `68136e7`. Pure-Kotlin
`PlanLinter` over the compiled `WorkflowGraph` → hard failures (missing file paths, unresolved `PlanLinter` over the compiled `WorkflowGraph`, run in `FreestyleDriver` before lock; records
symbols, unproduced `needs`, produces-kind not in registry, ADR-0000 violations) discard a `PlanLintCompletedEvent` (pure function of the recorded plan → replay-safe, no env observation),
candidate; soft findings (stage count, fan-out, budget, dup briefs) are scored. Records and a hard failure → `ExecutionPlanRejectedEvent(source="lint")` before the operator is asked or
`PlanLintCompletedEvent` (env observations — repo-map/symbol existence — per invariant #9). Wired the plan locks. **Hard:** `unproduced_need` (real gap — the freestyle compiler never checked needs),
into `FreestyleDriver` before lock so it pays off **now** for single-plan freestyle. Standalone. `trap_state` (no path to terminal; uses terminal-reachability so verdict-gated review loops pass).
**Soft (non-blocking):** `stage_count`, `fan_out`, `empty_brief`, `duplicate_brief`. QA gate in §F →
`docs/qa/QA-plan-lint.md`. **Deferred to Slice 1b:** file-path grounding + symbol resolution (env;
symbols need an index, §B §3), token-budget/ADR-bypass checks, a dedicated `:core:planning` module.
- [ ] **Slice 2 — §4 multi-candidate round-1 + templates.** `PlanTemplate` library, template-select + - [ ] **Slice 2 — §4 multi-candidate round-1 + templates.** `PlanTemplate` library, template-select +
slot-fill, GBNF-constrained, N₁=3 high-temp. `PlanCandidateGeneratedEvent`. slot-fill, GBNF-constrained, N₁=3 high-temp. `PlanCandidateGeneratedEvent`.
- [ ] **Slice 3 — §6 critique + §7 aggregate.** One-shot GBNF critic per survivor → - [ ] **Slice 3 — §6 critique + §7 aggregate.** One-shot GBNF critic per survivor →
@@ -213,6 +216,10 @@ These are SHIPPED in code but prompt-/server-/network-dependent and not yet live
gate uncommented in the copied `role_pipeline.toml`). Plan: `docs/qa/QA-static-first-reviewer.md` gate uncommented in the copied `role_pipeline.toml`). Plan: `docs/qa/QA-static-first-reviewer.md`
(clean→reviewer runs, non-clean→`StaticAnalysisCompleted`+retryable fail w/ output fed back, (clean→reviewer runs, non-clean→`StaticAnalysisCompleted`+retryable fail w/ output fed back,
gate-OFF absence check, no-runner no-op WARN, replay reads recorded findings without re-running). gate-OFF absence check, no-runner no-op WARN, replay reads recorded findings without re-running).
- [ ] **Plan-lint (§C★ Slice 1) live-QA**`68136e7`; logic is unit-covered, so this is confirmatory:
needs a freestyle run reaching the plan-lock gate (+ a crafted bad plan to force the reject path).
Plan: `docs/qa/QA-plan-lint.md` (unproduced_need/trap_state → `PlanLintCompleted`+`ExecutionPlan-
Rejected(source=lint)`+no lock; review-loop NOT trapped; soft findings non-blocking; replay stable).
- [ ] **TUI kernel-steering + AMD gauge fresh live-QA** — approve+note actually revising the - [ ] **TUI kernel-steering + AMD gauge fresh live-QA** — approve+note actually revising the
*same* stage's output; AMD VRAM/RAM gauge on the real box. *same* stage's output; AMD VRAM/RAM gauge on the real box.
- [ ] **Unit-tested-only fixes, not live-verified:** `1a7eb05` (verdict-edge rejection), - [ ] **Unit-tested-only fixes, not live-verified:** `1a7eb05` (verdict-edge rejection),
+62
View File
@@ -0,0 +1,62 @@
# QA Plan: deterministic plan lint — 68136e7
Plan-generation pipeline epic, Slice 1 (plan-pipeline-spec §5). Drafted from commit `68136e7`.
**Status:** DRAFT
**Run date / operator:**
**BACKLOG item:** §C★ "Slice 1 — §5 deterministic lint".
---
Note: the lint logic is **deterministic and unit-covered** (`PlanLinterTest` exercises every
hard/soft check incl. the review-loop-vs-trap nuance; `FreestyleDriverTest` covers reject-on-hard-fail
and record-on-clean). This live pass is **confirmatory** — it verifies the wiring fires in a real
freestyle run and the event surfaces in the log.
## Preconditions
- [ ] **server build/branch:** master @ `68136e7` (or later) — _rebuild only with the server stopped_.
- [ ] **llama-server + model:** a model capable of driving the freestyle planning phase to emit an
`execution_plan` (needed to reach `FreestyleDriver.lockAndRun`). The lint itself needs no model.
- [ ] **how to force a bad plan:** to exercise the reject path you need a plan that *compiles* but fails
lint — i.e. a stage whose `needs` names an artifact no stage produces, or an inescapable loop. Easiest
is a crafted `execution_plan` (seed/stub the planning output) rather than hoping the model emits one.
- [ ] **fixtures/seed:** a freestyle session (workingDir) that reaches the plan-lock gate.
## Acceptance gate (one sentence)
> The lint is correct **iff** a compiled freestyle plan with an unproduced `needs` or an inescapable
> (exit-less) loop is rejected with `ExecutionPlanRejectedEvent(source="lint")` — recording a
> `PlanLintCompletedEvent` with the matching hard failure, never locking and never executing — while a
> clean plan (including one with a legitimate verdict-gated review loop) records a passing
> `PlanLintCompletedEvent` and proceeds, and soft findings never block.
## Checks
| # | Action | Expected observable evidence | Result |
|---|--------|------------------------------|--------|
| 1 | `grep PlanLintCompleted core/events/.../serialization/Serialization.kt` | `subclass(PlanLintCompletedEvent::class)` present (unregistered = silent runtime deser failure) | |
| 2 | Run a freestyle session to a **clean** plan | `correx events <id>` shows `PlanLintCompleted` with empty `hardFailures`, then `ExecutionPlanLocked`, then execution | |
| 3 | Drive freestyle with a plan whose stage `needs` an artifact no stage produces | `PlanLintCompleted` with a `unproduced_need` hard failure naming the stage+artifact; `ExecutionPlanRejected` `source="lint"`; **no** `ExecutionPlanLocked`; no stage executes | |
| 4 | Drive a plan with an inescapable loop (cycle with no edge to `done`) | `PlanLintCompleted` with `trap_state` hard failure(s); rejected `source="lint"`; no lock | |
| 5 | Drive a plan with a legit verdict-gated review loop (implementer⇄reviewer, reviewer→done) | **No** `trap_state`; `PlanLintCompleted` passes; plan locks and runs (the nuance: cycles-with-exit are fine) | |
| 6 | Drive a clean-but-noisy plan (>4 fan-out, or two identical stage briefs) | `PlanLintCompleted` lists `fan_out`/`duplicate_brief` in `softFindings` with `score>0`, **but the plan still locks** (soft = non-blocking) | |
| 7 | `correx replay <id>` on a session from check 3/4 | The recorded `PlanLintCompleted` is read back identically; lint is not recomputed against a changed repo (pure function of the recorded plan); double-read digest stable | |
Evidence sources: `correx events <id>` (the `PlanLintCompleted` + `ExecutionPlanRejected`/`Locked` chain),
server logs (the "plan failed lint" WARN with the summary), and `correx replay <id>` for determinism.
## Out of scope (explicitly NOT covered this pass — later slices)
- File-path grounding + symbol resolution (Slice 1b — symbols need an index, BACKLOG §B §3).
- Multi-candidate generation / templates (Slice 2), critique/aggregate (Slice 3), tournament/selection +
the §E comparison UI (Slice 4), preference logging (Slice 5).
- Token-budget and ADR-0000 harness-bypass soft/hard checks.
- `role_pipeline` `impl_plan` lint (freestyle `execution_plan` only this slice).
## Disposition
- **PASS** → mark §C★ Slice 1 done; cite `68136e7` + this plan's run date/evidence. (Keep the epic open;
the slice is one step of it — no RETRO move until the epic completes or you choose to retire the slice.)
- **FAIL** → file each failure as a numbered finding back into `BACKLOG.md` (action + repro + wrong/missing
signal), fix, re-run only the failed checks. Set Status: FAILED until green.