From b1c0f88dee1a4cdf381d4574b80342f7912ed292 Mon Sep 17 00:00:00 2001 From: kami Date: Sun, 14 Jun 2026 14:51:22 +0400 Subject: [PATCH] docs(backlog): add BACKLOG + QA plan template Synthesize the five 2026-06-13 design specs and session memory into a single outstanding-work list. Header rules: resolved items move to RETRO.md; every finished feature gets a live-QA plan before it counts as resolved. Add docs/qa/TEMPLATE.md as the observable-evidence QA form. --- BACKLOG.md | 169 ++++++++++++++++++++++++++++++++++++++++++++ docs/qa/TEMPLATE.md | 53 ++++++++++++++ 2 files changed, 222 insertions(+) create mode 100644 BACKLOG.md create mode 100644 docs/qa/TEMPLATE.md diff --git a/BACKLOG.md b/BACKLOG.md new file mode 100644 index 00000000..df1c3511 --- /dev/null +++ b/BACKLOG.md @@ -0,0 +1,169 @@ +# CORREX — BACKLOG + +> **⚠️ MAINTENANCE RULE — READ FIRST** +> **When an item here is resolved, MOVE it out of this file and into `RETRO.md`.** +> Do not just tick a box and leave it. Cut the entry, paste it into `RETRO.md` +> with the resolving commit hash(es) and the date it landed/was verified. This file +> is the *live* outstanding-work list; `RETRO.md` is the archive of what got done. +> "Resolved" = built **and** verified (live-QA passed where a QA gate is noted). +> Items marked **SHIPPED (pending live-QA)** stay here until their QA gate passes, +> then they move. +> +> **QA RULE — after a feature is done, plan its QA before calling it resolved.** +> Run `/qa-plan ` (or copy `docs/qa/TEMPLATE.md` by hand) to produce a +> `docs/qa/QA-.md` with observable-evidence checks. A feature is not "resolved" +> until that plan passes live. On PASS → move to `RETRO.md`; on FAIL → refile the misses +> here as numbered findings. + +Synthesized 2026-06-14 from the five `docs/plans/2026-06-13-*.md` design specs and +session memory. Pointers are commit hashes / spec sections, not gospel — verify +against current code before acting (memory is point-in-time). + +--- + +## A. Observability spec — `docs/plans/2026-06-13-observability-spec.md` + +Metrics + CLI + TUI pane and the health backbone are shipped (fe94140, 4f6bfa8, +f8fd260) — those are RETRO candidates once live-QA'd. Remaining: + +- [ ] **§4 LLAMA_SERVER health probe** — liveness + tokens/sec degradation trend + (session-scoped, needs HTTP). Same `HealthProbe` framework as the shipped disk probe. +- [ ] **§4 EVENT_STORE health probe** — append + fsync latency heartbeat. +- [ ] **§4/§3 live health TUI pane** — deferred to follow the metrics cadence + (projection + CLI first, then pane). `correx health` CLI already exists. +- [ ] **§3 tier-2 stats** — full session-completion summary pane polish (basic pane shipped). +- 🚫 **§3 tier-3 Prometheus/Grafana export** — spec says "deferred indefinitely". **Do not build.** + +## B. Role reliability spec — `docs/plans/2026-06-13-role-reliability-spec.md` + +§3 analyst keystone (894969d), entity grounding (b050dc4), §2 write-manifest (4e5e4e5), +and §5 reviewer prompt/needs wiring (3467826) are shipped — RETRO candidates. Remaining: + +- [ ] **§5 reviewer static-first INFRA** — run compiler/detekt/formatters as a pipeline + step and *mechanically exclude their findings* from reviewer context. Needs a + static-findings event representation. (Prompt half is done; infra half is unbuilt.) +- [ ] **§6 `CritiqueFinding` schema in code** — only test expectations exist today; no + structured type. Shared by plan critic + code reviewer. +- [ ] **§6 `CriticCalibrationProjection`** (keyed by modelHash, role) — needs runtime + history to exist first (spec ordering #5). Same projection as pipeline-addenda A3. +- [ ] **§4 architect contradiction-check** — L3 retrieval over decision journal + ADRs → + `PossibleContradictionFlag` (display-only in v1). Unbuilt. +- [ ] **§3 symbol grounding** — deferred; needs a real symbol index (file-path grounding done). +- [ ] **§2 dynamic plan-derived diff manifest** — allowed files from the `impl_plan` + artifact. Future refinement; static per-stage `writes = [globs]` form is shipped. +- ⚠️ **§4 architect ADR shape** (`alternativesConsidered` required non-empty) — + **DELIBERATELY NOT TAKEN.** Conflicts with the standing "don't raise alternatives + unless real cost/breakage" preference already encoded in `architect.md`. Revisit only + with an explicit rationale; do not add by default. + +## C. Plan pipeline addenda — `docs/plans/2026-06-13-plan-pipeline-addenda.md` + +Entire spec is **unbuilt**. + +- [ ] **A1 brief echo-back gate** — planner restates brief as `BriefEcho`; harness diffs + referencedFiles/symbols/acceptanceCriteria vs original; divergence → `BriefEchoMismatchEvent`, + halt before any plan generation. (Cheapest failure point.) +- [ ] **A2 stage-level plan checkpointing** — reconciliation compares produced artifacts vs + the confirmed plan's produces-slots per stage; `StageCheckpointPassedEvent` / + `StageCheckpointFailedEvent` (halt + surface). +- [ ] **A3 critique calibration tracking** — `CritiqueOutcomeCorrelatedEvent` + + `CriticCalibrationProjection` (= role-reliability §6; build once, two consumers). + +## D. Research workflow spec — `docs/plans/2026-06-13-research-workflow-spec.md` + +Core workflow is **SHIPPED end-to-end, ON by default** (fb1d970..378ee39): extractor + +web_search (T1) + web_fetch (T2) in `:infrastructure:tools`, `research.toml` +(decompose→gather→report), schemas, prompts. Remaining: + +- [ ] **Live-QA the full run** — could not validate in-session (no SearXNG/network in sandbox). + Needs SearXNG running; exercise triage→fetch-approval→report→artifact viewer. +- [ ] **§6 web approval client** — the one larger unbuilt piece. Third client on the existing + Ktor REST+WS bus: approval cards + event strip + session list. Behind WireGuard/Tailscale + only, never public. Independent track (spec ordering #5). +- [ ] **§3 batch fetch-approval at source-list level** — currently per-fetch T2; operator + should approve the source list once, not each URL. +- [ ] **Dedicated `SourceFetched` / `LowQualityExtraction` events** — quality + + content_sha256 currently ride inside `ToolExecutionCompletedEvent` metadata only. +- [ ] **Dynamic per-session egress allowlist** — currently static `networkAllowedHosts` + T2 gate. +- [ ] **Fresh-machine runtime install** — copy `research.toml` + `prompts/` + `schemas/` and + the `[[artifacts]]` / `[tools.research]` config entries into `~/.config/correx` + (already done on this box; repo source is `examples/workflows/` + `docs/schemas/`). +- 🚫 No headless browser, no LLM-based extraction, no public approval endpoint (non-responsibilities). + +## E. TUI requirements spec — `docs/plans/2026-06-13-tui-requirements-spec.md` + +§2 last-event clock + unknown-event fallback (8b896b5) and §5.1 deterministic command-parse +card (65df3f4) are shipped — RETRO candidates. Remaining: + +- [ ] **§2 full Go↔Kotlin per-event-type render matrix** — golden-fixture tests asserting a + defined render for every event type (last §2 bullet; partially covered today). +- [ ] **§3/§4 plan comparison view** — top-2 candidates side-by-side, compact stage graphs, + diff-highlighted, lint/critique findings inline. Lands with the planning pipeline; + load-bearing for the future preference ranker. +- [ ] **§3 approval ergonomics audit** — single-key y/n/e for T0–T2, mandatory confirm T3+, + queued-approval navigable list (never modal-stacked). Verify against current band impl. +- [ ] **Render markdown in chat turns** (QA feature note b) — TUI currently ignores md. +- [ ] **Session list / resume view** (QA feature note c) — after server restart + TUI reopen + there's no way to see a prior session or its state; `GET /sessions` data exists server-side. +- [ ] **Idea board → profile promotion** — deferred hook; auto-captured ideas promote into the + curated `.correx/project.toml` profile. (Board itself shipped 3ac0a14..02e963d.) +- ⏸ **§5.2 command-card LLM annotation layer** — deferred by spec ("ship layer 1, live with it, + then decide whether layer 2 earns its place"). Build only after using layer 1. + +## F. In-flight QA gates (move to RETRO when each passes) + +These are SHIPPED in code but prompt-/server-/network-dependent and not yet live-verified: + +- [ ] **Epic 15 live-QA gate** — server up → `correx events` / `correx replay` on a real + session, confirm MDC sessionIds in logs. (Code complete a7d5211..75703ec.) +- [ ] **Idea board live-QA** — needs a real router model to emit the `{"ideas":[…]}` block. +- [ ] **Clarification loop + workflow-propose panel live-QA** — both prompt-dependent (need a + real router/analyst model to emit the structured json blocks). +- [ ] **Research workflow live-QA** — see §D. +- [ ] **TUI kernel-steering + AMD gauge fresh live-QA** — approve+note actually revising the + *same* stage's output; AMD VRAM/RAM gauge on the real box. +- [ ] **Unit-tested-only fixes, not live-verified:** `1a7eb05` (verdict-edge rejection), + `e195c79` (stale-parking `OrchestrationPaused ABANDONED_STALE`), `df3ee0b` (bare-id + narration model match). + +## G. Low-severity hardening + +- [ ] **turnId prefix collision** — `startsWith("repomap:$repoRoot")` can collide where one + root path prefixes another (`/repo` vs `/repo2`). Fix = trailing `:` delimiter in 3 + places (ProjectMemoryService write+check, L3RepoKnowledgeRetriever filter). +- [ ] **Narration lane lag** — stale pause-trigger narrations blend with current workflow + status; drop queued pause narrations once their approval resolves. +- [ ] **WorkspaceStateProbe fingerprint** — hashes `path|mtime|size` without newline separators + though the docstring says "lines". Internally consistent/deterministic; cosmetic. +- ⚠️ **`PreemptRedirectEvent` has no producer** (QA bug #6) — B3 hard re-route is unreachable + from the TUI. Intended flow: steering → router proposes → operator confirms. Skipped by + design; build only if the hard-reroute path is actually wanted. + +## H. Freestyle mode — deferred follow-ups (epic COMPLETE, B4 PASSED) + +- [ ] **Graph re-routing by preemptive steering** — only priority-directive surfacing is built + today; mid-execution input cannot re-route the compiled graph. +- [ ] **Child-session `SubagentRunner` impl** — seam exists (`InSessionSubagentRunner`); + child-session is the intended drop-in swap. Sequential only (no-parallel-agents invariant). +- [ ] **Post-lock `ApprovalDecisionResolvedEvent.userSteering` → L0 promotion** — currently + stays L2; out of freestyle's exercised path (gate is pre-lock). + +## I. Gamedev / Godot readiness (operator intent) + +- [ ] **`.cs` in `RepoMapIndexer` SYMBOL_PATTERNS** — add if the gamedev repo is Godot Mono + (`.gd` and `.md` headings already shipped: 81a21bf, f521ac8). +- [ ] **Structural `.tscn` / `.tres` validator** — parse scene format, node-tree-aware diff + preview. Prerequisite validation layer before any scene editing. +- [ ] **(far future) Scene editing behind approval+diff** + derive a fine-tuning dataset from + the event log. ⚠️ Fine-tuning is on CLAUDE.md's intentionally-NOT-implemented list — needs + an explicit epic + operator de-listing, not an incidental add. + +--- + +## Ops notes (not backlog items, but bite repeatedly) + +- Config copies of `examples/*` in `~/.config/correx` go **stale silently** — e.g. a missing + `inject_artifact_kinds` in a copied workflow TOML caused invented artifact kinds. Re-sync + after editing the repo examples. +- **Do not rebuild jars while the server JVM is running** — lazy classloading breaks + (`NoClassDefFoundError`). Stop the server first. diff --git a/docs/qa/TEMPLATE.md b/docs/qa/TEMPLATE.md new file mode 100644 index 00000000..f8bf0826 --- /dev/null +++ b/docs/qa/TEMPLATE.md @@ -0,0 +1,53 @@ +# QA Plan: — + + + +**Status:** DRAFT | RUNNING | PASSED | FAILED +**Run date / operator:** +**BACKLOG item:** + +--- + +## Preconditions + +- [ ] **server build/branch:** — _rebuild only with the server stopped (lazy classloading breaks otherwise)_ +- [ ] **llama-server + model:** — _note model-dependent behavior (strict templates, prompt-block emission)_ +- [ ] **external deps:** +- [ ] **config synced:** `~/.config/correx` matches `examples/*` + `docs/schemas/*` — _stale copies cause silent bugs_ +- [ ] **fixtures/seed:** + +## Acceptance gate (one sentence) + +> The feature is correct **iff** . + +## Checks + +Every row must name a **recorded, observable signal** — an event in the log, a CLI/log +line, a TUI render, a file on disk, or the *absence* of an event. "It worked" is not +evidence. + +| # | Action | Expected observable evidence | Result | +|---|--------|------------------------------|--------| +| 1 | | | | +| 2 | | | | +| 3 | | | | + +Evidence sources: `correx events ` / `GET /sessions/{id}/events` (the log is the +truth), server logs (MDC sessionId), the TUI render, files under the workspace, and +`correx replay ` for determinism. + +## Out of scope (explicitly NOT covered this pass) + +- + +## Disposition + +- **PASS** → MOVE the BACKLOG entry into `RETRO.md` with this plan's run date + the + evidence cited above. Set Status: PASSED. +- **FAIL** → file each failure as a numbered finding back into `BACKLOG.md` (action + + exact repro + the signal that was wrong/missing), fix, then re-run only the failed + checks. Set Status: FAILED until green.