docs(backlog): add BACKLOG + QA plan template
Synthesize the five 2026-06-13 design specs and session memory into a single outstanding-work list. Header rules: resolved items move to RETRO.md; every finished feature gets a live-QA plan before it counts as resolved. Add docs/qa/TEMPLATE.md as the observable-evidence QA form.
This commit is contained in:
+169
@@ -0,0 +1,169 @@
|
||||
# CORREX — BACKLOG
|
||||
|
||||
> **⚠️ MAINTENANCE RULE — READ FIRST**
|
||||
> **When an item here is resolved, MOVE it out of this file and into `RETRO.md`.**
|
||||
> Do not just tick a box and leave it. Cut the entry, paste it into `RETRO.md`
|
||||
> with the resolving commit hash(es) and the date it landed/was verified. This file
|
||||
> is the *live* outstanding-work list; `RETRO.md` is the archive of what got done.
|
||||
> "Resolved" = built **and** verified (live-QA passed where a QA gate is noted).
|
||||
> Items marked **SHIPPED (pending live-QA)** stay here until their QA gate passes,
|
||||
> then they move.
|
||||
>
|
||||
> **QA RULE — after a feature is done, plan its QA before calling it resolved.**
|
||||
> Run `/qa-plan <commits|spec>` (or copy `docs/qa/TEMPLATE.md` by hand) to produce a
|
||||
> `docs/qa/QA-<feature>.md` with observable-evidence checks. A feature is not "resolved"
|
||||
> until that plan passes live. On PASS → move to `RETRO.md`; on FAIL → refile the misses
|
||||
> here as numbered findings.
|
||||
|
||||
Synthesized 2026-06-14 from the five `docs/plans/2026-06-13-*.md` design specs and
|
||||
session memory. Pointers are commit hashes / spec sections, not gospel — verify
|
||||
against current code before acting (memory is point-in-time).
|
||||
|
||||
---
|
||||
|
||||
## A. Observability spec — `docs/plans/2026-06-13-observability-spec.md`
|
||||
|
||||
Metrics + CLI + TUI pane and the health backbone are shipped (fe94140, 4f6bfa8,
|
||||
f8fd260) — those are RETRO candidates once live-QA'd. Remaining:
|
||||
|
||||
- [ ] **§4 LLAMA_SERVER health probe** — liveness + tokens/sec degradation trend
|
||||
(session-scoped, needs HTTP). Same `HealthProbe` framework as the shipped disk probe.
|
||||
- [ ] **§4 EVENT_STORE health probe** — append + fsync latency heartbeat.
|
||||
- [ ] **§4/§3 live health TUI pane** — deferred to follow the metrics cadence
|
||||
(projection + CLI first, then pane). `correx health` CLI already exists.
|
||||
- [ ] **§3 tier-2 stats** — full session-completion summary pane polish (basic pane shipped).
|
||||
- 🚫 **§3 tier-3 Prometheus/Grafana export** — spec says "deferred indefinitely". **Do not build.**
|
||||
|
||||
## B. Role reliability spec — `docs/plans/2026-06-13-role-reliability-spec.md`
|
||||
|
||||
§3 analyst keystone (894969d), entity grounding (b050dc4), §2 write-manifest (4e5e4e5),
|
||||
and §5 reviewer prompt/needs wiring (3467826) are shipped — RETRO candidates. Remaining:
|
||||
|
||||
- [ ] **§5 reviewer static-first INFRA** — run compiler/detekt/formatters as a pipeline
|
||||
step and *mechanically exclude their findings* from reviewer context. Needs a
|
||||
static-findings event representation. (Prompt half is done; infra half is unbuilt.)
|
||||
- [ ] **§6 `CritiqueFinding` schema in code** — only test expectations exist today; no
|
||||
structured type. Shared by plan critic + code reviewer.
|
||||
- [ ] **§6 `CriticCalibrationProjection`** (keyed by modelHash, role) — needs runtime
|
||||
history to exist first (spec ordering #5). Same projection as pipeline-addenda A3.
|
||||
- [ ] **§4 architect contradiction-check** — L3 retrieval over decision journal + ADRs →
|
||||
`PossibleContradictionFlag` (display-only in v1). Unbuilt.
|
||||
- [ ] **§3 symbol grounding** — deferred; needs a real symbol index (file-path grounding done).
|
||||
- [ ] **§2 dynamic plan-derived diff manifest** — allowed files from the `impl_plan`
|
||||
artifact. Future refinement; static per-stage `writes = [globs]` form is shipped.
|
||||
- ⚠️ **§4 architect ADR shape** (`alternativesConsidered` required non-empty) —
|
||||
**DELIBERATELY NOT TAKEN.** Conflicts with the standing "don't raise alternatives
|
||||
unless real cost/breakage" preference already encoded in `architect.md`. Revisit only
|
||||
with an explicit rationale; do not add by default.
|
||||
|
||||
## C. Plan pipeline addenda — `docs/plans/2026-06-13-plan-pipeline-addenda.md`
|
||||
|
||||
Entire spec is **unbuilt**.
|
||||
|
||||
- [ ] **A1 brief echo-back gate** — planner restates brief as `BriefEcho`; harness diffs
|
||||
referencedFiles/symbols/acceptanceCriteria vs original; divergence → `BriefEchoMismatchEvent`,
|
||||
halt before any plan generation. (Cheapest failure point.)
|
||||
- [ ] **A2 stage-level plan checkpointing** — reconciliation compares produced artifacts vs
|
||||
the confirmed plan's produces-slots per stage; `StageCheckpointPassedEvent` /
|
||||
`StageCheckpointFailedEvent` (halt + surface).
|
||||
- [ ] **A3 critique calibration tracking** — `CritiqueOutcomeCorrelatedEvent` +
|
||||
`CriticCalibrationProjection` (= role-reliability §6; build once, two consumers).
|
||||
|
||||
## D. Research workflow spec — `docs/plans/2026-06-13-research-workflow-spec.md`
|
||||
|
||||
Core workflow is **SHIPPED end-to-end, ON by default** (fb1d970..378ee39): extractor +
|
||||
web_search (T1) + web_fetch (T2) in `:infrastructure:tools`, `research.toml`
|
||||
(decompose→gather→report), schemas, prompts. Remaining:
|
||||
|
||||
- [ ] **Live-QA the full run** — could not validate in-session (no SearXNG/network in sandbox).
|
||||
Needs SearXNG running; exercise triage→fetch-approval→report→artifact viewer.
|
||||
- [ ] **§6 web approval client** — the one larger unbuilt piece. Third client on the existing
|
||||
Ktor REST+WS bus: approval cards + event strip + session list. Behind WireGuard/Tailscale
|
||||
only, never public. Independent track (spec ordering #5).
|
||||
- [ ] **§3 batch fetch-approval at source-list level** — currently per-fetch T2; operator
|
||||
should approve the source list once, not each URL.
|
||||
- [ ] **Dedicated `SourceFetched` / `LowQualityExtraction` events** — quality +
|
||||
content_sha256 currently ride inside `ToolExecutionCompletedEvent` metadata only.
|
||||
- [ ] **Dynamic per-session egress allowlist** — currently static `networkAllowedHosts` + T2 gate.
|
||||
- [ ] **Fresh-machine runtime install** — copy `research.toml` + `prompts/` + `schemas/` and
|
||||
the `[[artifacts]]` / `[tools.research]` config entries into `~/.config/correx`
|
||||
(already done on this box; repo source is `examples/workflows/` + `docs/schemas/`).
|
||||
- 🚫 No headless browser, no LLM-based extraction, no public approval endpoint (non-responsibilities).
|
||||
|
||||
## E. TUI requirements spec — `docs/plans/2026-06-13-tui-requirements-spec.md`
|
||||
|
||||
§2 last-event clock + unknown-event fallback (8b896b5) and §5.1 deterministic command-parse
|
||||
card (65df3f4) are shipped — RETRO candidates. Remaining:
|
||||
|
||||
- [ ] **§2 full Go↔Kotlin per-event-type render matrix** — golden-fixture tests asserting a
|
||||
defined render for every event type (last §2 bullet; partially covered today).
|
||||
- [ ] **§3/§4 plan comparison view** — top-2 candidates side-by-side, compact stage graphs,
|
||||
diff-highlighted, lint/critique findings inline. Lands with the planning pipeline;
|
||||
load-bearing for the future preference ranker.
|
||||
- [ ] **§3 approval ergonomics audit** — single-key y/n/e for T0–T2, mandatory confirm T3+,
|
||||
queued-approval navigable list (never modal-stacked). Verify against current band impl.
|
||||
- [ ] **Render markdown in chat turns** (QA feature note b) — TUI currently ignores md.
|
||||
- [ ] **Session list / resume view** (QA feature note c) — after server restart + TUI reopen
|
||||
there's no way to see a prior session or its state; `GET /sessions` data exists server-side.
|
||||
- [ ] **Idea board → profile promotion** — deferred hook; auto-captured ideas promote into the
|
||||
curated `.correx/project.toml` profile. (Board itself shipped 3ac0a14..02e963d.)
|
||||
- ⏸ **§5.2 command-card LLM annotation layer** — deferred by spec ("ship layer 1, live with it,
|
||||
then decide whether layer 2 earns its place"). Build only after using layer 1.
|
||||
|
||||
## F. In-flight QA gates (move to RETRO when each passes)
|
||||
|
||||
These are SHIPPED in code but prompt-/server-/network-dependent and not yet live-verified:
|
||||
|
||||
- [ ] **Epic 15 live-QA gate** — server up → `correx events` / `correx replay` on a real
|
||||
session, confirm MDC sessionIds in logs. (Code complete a7d5211..75703ec.)
|
||||
- [ ] **Idea board live-QA** — needs a real router model to emit the `{"ideas":[…]}` block.
|
||||
- [ ] **Clarification loop + workflow-propose panel live-QA** — both prompt-dependent (need a
|
||||
real router/analyst model to emit the structured json blocks).
|
||||
- [ ] **Research workflow live-QA** — see §D.
|
||||
- [ ] **TUI kernel-steering + AMD gauge fresh live-QA** — approve+note actually revising the
|
||||
*same* stage's output; AMD VRAM/RAM gauge on the real box.
|
||||
- [ ] **Unit-tested-only fixes, not live-verified:** `1a7eb05` (verdict-edge rejection),
|
||||
`e195c79` (stale-parking `OrchestrationPaused ABANDONED_STALE`), `df3ee0b` (bare-id
|
||||
narration model match).
|
||||
|
||||
## G. Low-severity hardening
|
||||
|
||||
- [ ] **turnId prefix collision** — `startsWith("repomap:$repoRoot")` can collide where one
|
||||
root path prefixes another (`/repo` vs `/repo2`). Fix = trailing `:` delimiter in 3
|
||||
places (ProjectMemoryService write+check, L3RepoKnowledgeRetriever filter).
|
||||
- [ ] **Narration lane lag** — stale pause-trigger narrations blend with current workflow
|
||||
status; drop queued pause narrations once their approval resolves.
|
||||
- [ ] **WorkspaceStateProbe fingerprint** — hashes `path|mtime|size` without newline separators
|
||||
though the docstring says "lines". Internally consistent/deterministic; cosmetic.
|
||||
- ⚠️ **`PreemptRedirectEvent` has no producer** (QA bug #6) — B3 hard re-route is unreachable
|
||||
from the TUI. Intended flow: steering → router proposes → operator confirms. Skipped by
|
||||
design; build only if the hard-reroute path is actually wanted.
|
||||
|
||||
## H. Freestyle mode — deferred follow-ups (epic COMPLETE, B4 PASSED)
|
||||
|
||||
- [ ] **Graph re-routing by preemptive steering** — only priority-directive surfacing is built
|
||||
today; mid-execution input cannot re-route the compiled graph.
|
||||
- [ ] **Child-session `SubagentRunner` impl** — seam exists (`InSessionSubagentRunner`);
|
||||
child-session is the intended drop-in swap. Sequential only (no-parallel-agents invariant).
|
||||
- [ ] **Post-lock `ApprovalDecisionResolvedEvent.userSteering` → L0 promotion** — currently
|
||||
stays L2; out of freestyle's exercised path (gate is pre-lock).
|
||||
|
||||
## I. Gamedev / Godot readiness (operator intent)
|
||||
|
||||
- [ ] **`.cs` in `RepoMapIndexer` SYMBOL_PATTERNS** — add if the gamedev repo is Godot Mono
|
||||
(`.gd` and `.md` headings already shipped: 81a21bf, f521ac8).
|
||||
- [ ] **Structural `.tscn` / `.tres` validator** — parse scene format, node-tree-aware diff
|
||||
preview. Prerequisite validation layer before any scene editing.
|
||||
- [ ] **(far future) Scene editing behind approval+diff** + derive a fine-tuning dataset from
|
||||
the event log. ⚠️ Fine-tuning is on CLAUDE.md's intentionally-NOT-implemented list — needs
|
||||
an explicit epic + operator de-listing, not an incidental add.
|
||||
|
||||
---
|
||||
|
||||
## Ops notes (not backlog items, but bite repeatedly)
|
||||
|
||||
- Config copies of `examples/*` in `~/.config/correx` go **stale silently** — e.g. a missing
|
||||
`inject_artifact_kinds` in a copied workflow TOML caused invented artifact kinds. Re-sync
|
||||
after editing the repo examples.
|
||||
- **Do not rebuild jars while the server JVM is running** — lazy classloading breaks
|
||||
(`NoClassDefFoundError`). Stop the server first.
|
||||
@@ -0,0 +1,53 @@
|
||||
# QA Plan: <feature> — <commits>
|
||||
|
||||
<!--
|
||||
Copy this file to docs/qa/QA-<feature>.md and fill it in.
|
||||
Generated by /qa-plan, or written by hand.
|
||||
This plan is the artifact that authorizes the BACKLOG -> RETRO move.
|
||||
-->
|
||||
|
||||
**Status:** DRAFT | RUNNING | PASSED | FAILED
|
||||
**Run date / operator:**
|
||||
**BACKLOG item:** <link/quote the §F entry this closes>
|
||||
|
||||
---
|
||||
|
||||
## Preconditions
|
||||
|
||||
- [ ] **server build/branch:** <jar/branch> — _rebuild only with the server stopped (lazy classloading breaks otherwise)_
|
||||
- [ ] **llama-server + model:** <model id> — _note model-dependent behavior (strict templates, prompt-block emission)_
|
||||
- [ ] **external deps:** <SearXNG / network / real router model / none>
|
||||
- [ ] **config synced:** `~/.config/correx` matches `examples/*` + `docs/schemas/*` — _stale copies cause silent bugs_
|
||||
- [ ] **fixtures/seed:** <workspace, .correx/project.toml, prior session to resume, etc.>
|
||||
|
||||
## Acceptance gate (one sentence)
|
||||
|
||||
> The feature is correct **iff** <single observable outcome>.
|
||||
|
||||
## Checks
|
||||
|
||||
Every row must name a **recorded, observable signal** — an event in the log, a CLI/log
|
||||
line, a TUI render, a file on disk, or the *absence* of an event. "It worked" is not
|
||||
evidence.
|
||||
|
||||
| # | Action | Expected observable evidence | Result |
|
||||
|---|--------|------------------------------|--------|
|
||||
| 1 | | | |
|
||||
| 2 | | | |
|
||||
| 3 | | | |
|
||||
|
||||
Evidence sources: `correx events <id>` / `GET /sessions/{id}/events` (the log is the
|
||||
truth), server logs (MDC sessionId), the TUI render, files under the workspace, and
|
||||
`correx replay <id>` for determinism.
|
||||
|
||||
## Out of scope (explicitly NOT covered this pass)
|
||||
|
||||
-
|
||||
|
||||
## Disposition
|
||||
|
||||
- **PASS** → MOVE the BACKLOG entry into `RETRO.md` with this plan's run date + the
|
||||
evidence cited above. Set Status: PASSED.
|
||||
- **FAIL** → file each failure as a numbered finding back into `BACKLOG.md` (action +
|
||||
exact repro + the signal that was wrong/missing), fix, then re-run only the failed
|
||||
checks. Set Status: FAILED until green.
|
||||
Reference in New Issue
Block a user