From 618f6e91c0ba995707da3cf0d347e1daebd27e2e Mon Sep 17 00:00:00 2001 From: kami Date: Mon, 15 Jun 2026 00:16:29 +0400 Subject: [PATCH] =?UTF-8?q?docs(backlog):=20add=20plan-generation=20pipeli?= =?UTF-8?q?ne=20epic=20(=C2=A7C=E2=98=85),=20re-home=20orphans?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Base spec docs/plans/plan-pipeline-spec.md is now in the repo. Add a §C★ epic section (5 slices: lint → templates/round-1 → critique+aggregate → round-2/tournament/selection → preference logging) with build-state (freestyle = degenerate N=1; substrate exists). Re-point the orphans as components: §E plan-compare = Slice 4 UI; §B §6 CritiqueFinding = Slice 3 (plan critic); A3 calibration = layer above it. Flag PlanConfirmedEvent vs ExecutionPlanLockedEvent naming + the cycle-vs-review-loop lint nuance. --- BACKLOG.md | 73 +++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 59 insertions(+), 14 deletions(-) diff --git a/BACKLOG.md b/BACKLOG.md index 83273f73..a52a306a 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -56,15 +56,16 @@ and §5 reviewer prompt/needs wiring (3467826) are shipped — RETRO candidates. (resolved upstream) — no output-parsing, no shared-findings schema needed. `ProcessStaticAnalysis- Runner` is the injected adapter (nullable seam; null → logged no-op). Prompt half was `3467826`. QA gate in §F → `docs/qa/QA-static-first-reviewer.md`. -- ⏸ **§6 `CritiqueFinding` schema + `CriticCalibrationProjection` (= addenda A3)** — **DEFERRED until - calibration is actually built.** Substrate with no live consumer today (assessed 2026-06-14): zero - `CritiqueFinding` references anywhere (the old "test expectations exist" note was stale — none do), - no `CriticCalibrationProjection`/`CritiqueOutcomeCorrelatedEvent`, and **no plan critic** — so the - spec's "single definition, two consumers" rationale has zero consumers. The one reviewer that exists - emits `review_report` = `verdict` + free-text `notes`; code reads only `verdict`, and the implementer - reads `notes` as prose fine. Structured findings only earn their place when the calibration - projection is built — which needs runtime critique/outcome history first (spec ordering #5) and a - second consumer (a plan critic) to justify sharing. Build the schema *with* calibration, not before. +- ⏸ **§6 `CritiqueFinding` schema** — **build with §C★ Slice 3 (the plan critic), not standalone.** No + live consumer today (assessed 2026-06-14): zero `CritiqueFinding` references anywhere (the old "test + expectations exist" note was stale — none do), and **no plan critic** exists yet. The one reviewer + that exists emits `review_report` = `verdict` + free-text `notes`; code reads only `verdict`, and the + implementer reads `notes` as prose fine. The schema's first real consumer is the plan critic + (plan-pipeline-spec §6) — it lands there. The role-reliability reviewer is the second reuse. +- ⏸ **§6 `CriticCalibrationProjection` (= addenda A3)** — a further layer on top of `CritiqueFinding`: + correlates critique findings with runtime stage outcomes (`CritiqueOutcomeCorrelatedEvent`). Needs + runtime critique/outcome history to exist first (spec ordering #5) — i.e. after Slice 3 ships and + runs. Don't build before there's data for it to fold. - [ ] **§4 architect contradiction-check** — L3 retrieval over decision journal + ADRs → `PossibleContradictionFlag` (display-only in v1). Unbuilt. - [ ] **§3 symbol grounding** — deferred; needs a real symbol index (file-path grounding done). @@ -77,8 +78,9 @@ and §5 reviewer prompt/needs wiring (3467826) are shipped — RETRO candidates. ## C. Plan pipeline addenda — `docs/plans/2026-06-13-plan-pipeline-addenda.md` -A1 is shipped (bcc59d2) — RETRO candidate once live-QA'd. A2 is **superseded** (see below); -A3 is the same item as §B §6 calibration (tracked there). +These are amendments to the base **plan-generation pipeline** spec (§C★ below). A1 is shipped +(bcc59d2) — RETRO candidate once live-QA'd. A2 is **superseded** (see below); A3 is the critique +calibration layer (tracked under §B §6). - [~] **A1 brief echo-back gate** — **SHIPPED (pending live-QA)** `bcc59d2`. New `brief_echo` stage before the planner in `role_pipeline`: the model restates the analyst brief as a structured @@ -101,6 +103,49 @@ A3 is the same item as §B §6 calibration (tracked there). consumers). Tracked under §B §6; needs runtime critique/outcome history first (spec ordering #5). Not duplicated here. +## C★. Plan-generation pipeline epic — `docs/plans/plan-pipeline-spec.md` (base spec) + +The foundation the §C addenda amend. Turns a freestyle prompt into a confirmed `ExecutionPlan` via +**bounded multi-candidate generation + deterministic lint + structured critique + tournament + operator +selection** — built for the project's founding constraint (local models are weak planners): narrow every +model call, keep every joint deterministic Kotlin, make the *operator* the terminal verifier. Replaces the +single-plan freestyle planning phase. + +**Build state:** freestyle already ships a **degenerate N=1 version** (analyst brief → one +`execution_plan` → `ExecutionPlanCompiler` → operator `requestPlanApproval` → `ExecutionPlanLockedEvent`). +Substrate in place: `GbnfGrammarConverter`, repo-map, L3 retrieval, CAS, approval gate, and a **partial +lint** inside `ExecutionPlanCompiler` (reachability, unknown-kind, edge field-condition). The pipeline +generalizes this to N candidates. ⚠️ Spec says `PlanConfirmedEvent` is "existing" — it is **not**; the +built analog is `ExecutionPlanLockedEvent` (extend it with a candidate ref; don't add a parallel event). + +Slices (spec §, in build order): + +- [ ] **Slice 1 — §5 deterministic lint** *(in progress — see plan)*. Pure Kotlin, zero inference: + `PlanLinter` over the compiled `WorkflowGraph` → hard failures (missing file paths, unresolved + symbols, unproduced `needs`, produces-kind not in registry, ADR-0000 violations) discard a + candidate; soft findings (stage count, fan-out, budget, dup briefs) are scored. Records + `PlanLintCompletedEvent` (env observations — repo-map/symbol existence — per invariant #9). Wired + into `FreestyleDriver` before lock so it pays off **now** for single-plan freestyle. Standalone. +- [ ] **Slice 2 — §4 multi-candidate round-1 + templates.** `PlanTemplate` library, template-select + + slot-fill, GBNF-constrained, N₁=3 high-temp. `PlanCandidateGeneratedEvent`. +- [ ] **Slice 3 — §6 critique + §7 aggregate.** One-shot GBNF critic per survivor → + `CritiqueFinding {stageId, severity, finding, suggestedChange}` (= §B §6 schema, first consumer); + deterministic dedupe/contradiction-drop/severity-sort → `DefectList`. `PlanCritiqueCompletedEvent`. +- [ ] **Slice 4 — §8 round-2 + §9 tournament + operator selection.** N₂=2 low-temp regen from best + survivor + DefectList; deterministic ranking; **top-2 → operator** at the approval gate + (**= §E §3/§4 plan comparison view**, the terminal UI). Reject-both → `PlanGenerationFailedEvent`. +- [ ] **Slice 5 — §10 preference-pair logging.** `PreferencePairRecordedEvent` (winner/loser/context) + falls out of selection; Bradley–Terry data for the future learned ranker (§12, deferred). + +Open questions to resolve while building (spec §13): module home (recommend new `:core:planning`); +symbol resolution `ctx.py` subprocess vs in-process tree-sitter (subprocess = env observation → record, +never re-run on replay); defect-string dedup algorithm (pin it, version it). ⚠️ Spec lists "graph contains +cycles" as a hard failure, but freestyle plans legitimately contain verdict-gated review loops +(implementer⇄reviewer) — Slice 1 must only fail cycles **without** an exit edge, not all cycles. + +🚫 Non-responsibilities (spec §11): the pipeline never executes a stage, never auto-confirms, never uses an +LLM to aggregate/rank, never mutates a candidate in place, never exceeds 2 gen rounds / 1 critique round. + ## D. Research workflow spec — `docs/plans/2026-06-13-research-workflow-spec.md` Core workflow is **SHIPPED end-to-end, ON by default** (fb1d970..378ee39): extractor + @@ -129,9 +174,9 @@ card (65df3f4) are shipped — RETRO candidates. Remaining: - [ ] **§2 full Go↔Kotlin per-event-type render matrix** — golden-fixture tests asserting a defined render for every event type (last §2 bullet; partially covered today). -- [ ] **§3/§4 plan comparison view** — top-2 candidates side-by-side, compact stage graphs, - diff-highlighted, lint/critique findings inline. Lands with the planning pipeline; - load-bearing for the future preference ranker. +- [ ] **§3/§4 plan comparison view** — **= §C★ Slice 4** (the operator-selection terminal UI of the + plan-generation pipeline, spec §9): top-2 candidates side-by-side with lint/critique findings + rendered inline. Blocked until the pipeline produces candidates; tracked under §C★. - [ ] **§3 approval ergonomics audit** — single-key y/n/e for T0–T2, mandatory confirm T3+, queued-approval navigable list (never modal-stacked). Verify against current band impl. - [ ] **Render markdown in chat turns** (QA feature note b) — TUI currently ignores md.