docs(backlog): add plan-generation pipeline epic (§C★), re-home orphans
Base spec docs/plans/plan-pipeline-spec.md is now in the repo. Add a §C★ epic section (5 slices: lint → templates/round-1 → critique+aggregate → round-2/tournament/selection → preference logging) with build-state (freestyle = degenerate N=1; substrate exists). Re-point the orphans as components: §E plan-compare = Slice 4 UI; §B §6 CritiqueFinding = Slice 3 (plan critic); A3 calibration = layer above it. Flag PlanConfirmedEvent vs ExecutionPlanLockedEvent naming + the cycle-vs-review-loop lint nuance.
This commit is contained in:
+59
-14
@@ -56,15 +56,16 @@ and §5 reviewer prompt/needs wiring (3467826) are shipped — RETRO candidates.
|
||||
(resolved upstream) — no output-parsing, no shared-findings schema needed. `ProcessStaticAnalysis-
|
||||
Runner` is the injected adapter (nullable seam; null → logged no-op). Prompt half was `3467826`.
|
||||
QA gate in §F → `docs/qa/QA-static-first-reviewer.md`.
|
||||
- ⏸ **§6 `CritiqueFinding` schema + `CriticCalibrationProjection` (= addenda A3)** — **DEFERRED until
|
||||
calibration is actually built.** Substrate with no live consumer today (assessed 2026-06-14): zero
|
||||
`CritiqueFinding` references anywhere (the old "test expectations exist" note was stale — none do),
|
||||
no `CriticCalibrationProjection`/`CritiqueOutcomeCorrelatedEvent`, and **no plan critic** — so the
|
||||
spec's "single definition, two consumers" rationale has zero consumers. The one reviewer that exists
|
||||
emits `review_report` = `verdict` + free-text `notes`; code reads only `verdict`, and the implementer
|
||||
reads `notes` as prose fine. Structured findings only earn their place when the calibration
|
||||
projection is built — which needs runtime critique/outcome history first (spec ordering #5) and a
|
||||
second consumer (a plan critic) to justify sharing. Build the schema *with* calibration, not before.
|
||||
- ⏸ **§6 `CritiqueFinding` schema** — **build with §C★ Slice 3 (the plan critic), not standalone.** No
|
||||
live consumer today (assessed 2026-06-14): zero `CritiqueFinding` references anywhere (the old "test
|
||||
expectations exist" note was stale — none do), and **no plan critic** exists yet. The one reviewer
|
||||
that exists emits `review_report` = `verdict` + free-text `notes`; code reads only `verdict`, and the
|
||||
implementer reads `notes` as prose fine. The schema's first real consumer is the plan critic
|
||||
(plan-pipeline-spec §6) — it lands there. The role-reliability reviewer is the second reuse.
|
||||
- ⏸ **§6 `CriticCalibrationProjection` (= addenda A3)** — a further layer on top of `CritiqueFinding`:
|
||||
correlates critique findings with runtime stage outcomes (`CritiqueOutcomeCorrelatedEvent`). Needs
|
||||
runtime critique/outcome history to exist first (spec ordering #5) — i.e. after Slice 3 ships and
|
||||
runs. Don't build before there's data for it to fold.
|
||||
- [ ] **§4 architect contradiction-check** — L3 retrieval over decision journal + ADRs →
|
||||
`PossibleContradictionFlag` (display-only in v1). Unbuilt.
|
||||
- [ ] **§3 symbol grounding** — deferred; needs a real symbol index (file-path grounding done).
|
||||
@@ -77,8 +78,9 @@ and §5 reviewer prompt/needs wiring (3467826) are shipped — RETRO candidates.
|
||||
|
||||
## C. Plan pipeline addenda — `docs/plans/2026-06-13-plan-pipeline-addenda.md`
|
||||
|
||||
A1 is shipped (bcc59d2) — RETRO candidate once live-QA'd. A2 is **superseded** (see below);
|
||||
A3 is the same item as §B §6 calibration (tracked there).
|
||||
These are amendments to the base **plan-generation pipeline** spec (§C★ below). A1 is shipped
|
||||
(bcc59d2) — RETRO candidate once live-QA'd. A2 is **superseded** (see below); A3 is the critique
|
||||
calibration layer (tracked under §B §6).
|
||||
|
||||
- [~] **A1 brief echo-back gate** — **SHIPPED (pending live-QA)** `bcc59d2`. New `brief_echo` stage
|
||||
before the planner in `role_pipeline`: the model restates the analyst brief as a structured
|
||||
@@ -101,6 +103,49 @@ A3 is the same item as §B §6 calibration (tracked there).
|
||||
consumers). Tracked under §B §6; needs runtime critique/outcome history first (spec ordering #5).
|
||||
Not duplicated here.
|
||||
|
||||
## C★. Plan-generation pipeline epic — `docs/plans/plan-pipeline-spec.md` (base spec)
|
||||
|
||||
The foundation the §C addenda amend. Turns a freestyle prompt into a confirmed `ExecutionPlan` via
|
||||
**bounded multi-candidate generation + deterministic lint + structured critique + tournament + operator
|
||||
selection** — built for the project's founding constraint (local models are weak planners): narrow every
|
||||
model call, keep every joint deterministic Kotlin, make the *operator* the terminal verifier. Replaces the
|
||||
single-plan freestyle planning phase.
|
||||
|
||||
**Build state:** freestyle already ships a **degenerate N=1 version** (analyst brief → one
|
||||
`execution_plan` → `ExecutionPlanCompiler` → operator `requestPlanApproval` → `ExecutionPlanLockedEvent`).
|
||||
Substrate in place: `GbnfGrammarConverter`, repo-map, L3 retrieval, CAS, approval gate, and a **partial
|
||||
lint** inside `ExecutionPlanCompiler` (reachability, unknown-kind, edge field-condition). The pipeline
|
||||
generalizes this to N candidates. ⚠️ Spec says `PlanConfirmedEvent` is "existing" — it is **not**; the
|
||||
built analog is `ExecutionPlanLockedEvent` (extend it with a candidate ref; don't add a parallel event).
|
||||
|
||||
Slices (spec §, in build order):
|
||||
|
||||
- [ ] **Slice 1 — §5 deterministic lint** *(in progress — see plan)*. Pure Kotlin, zero inference:
|
||||
`PlanLinter` over the compiled `WorkflowGraph` → hard failures (missing file paths, unresolved
|
||||
symbols, unproduced `needs`, produces-kind not in registry, ADR-0000 violations) discard a
|
||||
candidate; soft findings (stage count, fan-out, budget, dup briefs) are scored. Records
|
||||
`PlanLintCompletedEvent` (env observations — repo-map/symbol existence — per invariant #9). Wired
|
||||
into `FreestyleDriver` before lock so it pays off **now** for single-plan freestyle. Standalone.
|
||||
- [ ] **Slice 2 — §4 multi-candidate round-1 + templates.** `PlanTemplate` library, template-select +
|
||||
slot-fill, GBNF-constrained, N₁=3 high-temp. `PlanCandidateGeneratedEvent`.
|
||||
- [ ] **Slice 3 — §6 critique + §7 aggregate.** One-shot GBNF critic per survivor →
|
||||
`CritiqueFinding {stageId, severity, finding, suggestedChange}` (= §B §6 schema, first consumer);
|
||||
deterministic dedupe/contradiction-drop/severity-sort → `DefectList`. `PlanCritiqueCompletedEvent`.
|
||||
- [ ] **Slice 4 — §8 round-2 + §9 tournament + operator selection.** N₂=2 low-temp regen from best
|
||||
survivor + DefectList; deterministic ranking; **top-2 → operator** at the approval gate
|
||||
(**= §E §3/§4 plan comparison view**, the terminal UI). Reject-both → `PlanGenerationFailedEvent`.
|
||||
- [ ] **Slice 5 — §10 preference-pair logging.** `PreferencePairRecordedEvent` (winner/loser/context)
|
||||
falls out of selection; Bradley–Terry data for the future learned ranker (§12, deferred).
|
||||
|
||||
Open questions to resolve while building (spec §13): module home (recommend new `:core:planning`);
|
||||
symbol resolution `ctx.py` subprocess vs in-process tree-sitter (subprocess = env observation → record,
|
||||
never re-run on replay); defect-string dedup algorithm (pin it, version it). ⚠️ Spec lists "graph contains
|
||||
cycles" as a hard failure, but freestyle plans legitimately contain verdict-gated review loops
|
||||
(implementer⇄reviewer) — Slice 1 must only fail cycles **without** an exit edge, not all cycles.
|
||||
|
||||
🚫 Non-responsibilities (spec §11): the pipeline never executes a stage, never auto-confirms, never uses an
|
||||
LLM to aggregate/rank, never mutates a candidate in place, never exceeds 2 gen rounds / 1 critique round.
|
||||
|
||||
## D. Research workflow spec — `docs/plans/2026-06-13-research-workflow-spec.md`
|
||||
|
||||
Core workflow is **SHIPPED end-to-end, ON by default** (fb1d970..378ee39): extractor +
|
||||
@@ -129,9 +174,9 @@ card (65df3f4) are shipped — RETRO candidates. Remaining:
|
||||
|
||||
- [ ] **§2 full Go↔Kotlin per-event-type render matrix** — golden-fixture tests asserting a
|
||||
defined render for every event type (last §2 bullet; partially covered today).
|
||||
- [ ] **§3/§4 plan comparison view** — top-2 candidates side-by-side, compact stage graphs,
|
||||
diff-highlighted, lint/critique findings inline. Lands with the planning pipeline;
|
||||
load-bearing for the future preference ranker.
|
||||
- [ ] **§3/§4 plan comparison view** — **= §C★ Slice 4** (the operator-selection terminal UI of the
|
||||
plan-generation pipeline, spec §9): top-2 candidates side-by-side with lint/critique findings
|
||||
rendered inline. Blocked until the pipeline produces candidates; tracked under §C★.
|
||||
- [ ] **§3 approval ergonomics audit** — single-key y/n/e for T0–T2, mandatory confirm T3+,
|
||||
queued-approval navigable list (never modal-stacked). Verify against current band impl.
|
||||
- [ ] **Render markdown in chat turns** (QA feature note b) — TUI currently ignores md.
|
||||
|
||||
Reference in New Issue
Block a user