fix: anchor LSP diagnostics to server readiness

docs/sprints/2026-07-23.md

@@ -0,0 +1,98 @@
+# Sprint: "Close the loop" — 2 weeks
+
+**Dates:** 2026-07-23 → 2026-08-06
+**Source:** Vikunja Correx project (id 4), 30 open tasks reviewed
+**Theme:** Real per-stage validation, warm-discovery freestyle runs, robust orchestration under failure
+
+---
+
+## Goal 1 — Real per-stage validation: land the LSP gate + fix build-gate wiring
+
+**Why.** Multiple freestyle QA runs hit the same wall: the build-gate only fires at the terminal stage, so 8-10 hopeful writes stack before the truth-check, and when it fails there's no budget left (#167 epic calls this the "open-loop" disease). #80 is the filed fix and its deferral condition was observed on 2026-07-18 run 4a41417b — promote now.
+
+**Tasks.**
+- **#80** [EPIC] LSP validation gate — real per-stage gate + terminal typecheck replacement. Design finalized in the task body; break into the 5 sub-tasks listed there. Container for the work below.
+  - **#310** ✅ LSP diagnostics runner: anchor read on server readiness, not the 750ms timer (kills phantom unresolved-import).
+  - **#311** LSP gate: lint-class diagnostics (unused/deprecated) must not fail the run — classify by `DiagnosticTag`, not severity.
+- **#263** Auto build-gate never fires on real freestyle scaffold — promotion attaches to no reachable stage. Blocks the terminal safety net #80 doesn't replace.
+- **#267** Verify build-gate actually fires + re-scope #40 (LSP obsoletes the typecheck alias) — one live run settles both. Acceptance gate for the whole goal; do last.
+
+**Exit.** One live freestyle run where every write stage gets LSP diagnostics scoped to its write blast-radius, terminal build-gate fires, and a stranded-scaffold case goes to recovery instead of `WorkflowFailed`.
+
+**Sequencing.**
+- Mon: #80 sub-task 1 (cheap floor — populate the existing per-stage `static_analysis` seam with file-local one-shots in the freestyle compiler). Same-day ship; de-risks everything below.
+- Week 1: parallel track on #263 (build-gate wiring).
+- Week 2: #80 sub-tasks 2/3/4 — LSP4J wiring, pull-diagnostics, server pool, per-stage integration.
+- Week 2 tail: #310, #311 (readiness anchor + DiagnosticTag classification), then #267 as the acceptance live run.
+
+---
+
+## Goal 2 — Make freestyle runs start warm and converge: discovery prompts, impl decomposition, ACR
+
+**Why.** The kernel executes what it's told; near-term leverage is what it's told. #260/#261 sharpen upstream (discovery + per-feature stages), #305/#306 make run N+1 actually carry forward from the event log — the two halves of the ACR thesis still unimplemented. #297 kills a budget-burning failure mode mid-discovery.
+
+**Tasks.**
+- **#260** Freestyle SDLC: prompt-only upgrades — exhaustive discovery prompt + analyst DoD artifact (the contract the rest of the run is judged against).
+- **#261** Freestyle SDLC: decompose impl stages into features/sub-tasks (structural, deferred — but it's what makes per-stage LSP from Goal 1 actually bound blast-radius).
+- **#305** ACR: accrete task knowledge externally so discovery starts warm (model-agnostic) — warm-start half of #168's remaining follow-on.
+- **#306** Sticky ACR steer-away hint: fires every turn + coarse signature collapse — makes delivered concepts usable; without this ACR delivery is noise.
+- **#297** Analyst CoT indecision loop burns full reasoning budget, emits nothing — recovery hygiene in the stage the goal is sharpening.
+
+**Exit.** A second freestyle run on a fresh repo where DoD is recorded, impl stages are feature-bounded (so Goal 1's LSP scope is real), and the discovery stage carries accreted task knowledge from a prior run on an adjacent repo.
+
+**Sequencing.**
+- Day 1 (parallel): #260 — files-only, doesn't block on anything.
+- Week 1 mid: #297 (analyst CoT fix — same stage family #260 touches).
+- Week 2: #261 (impl decomposition — depends on #260's DoD existing to decompose against), #305/#306 (ACR — pairs with the warmed discovery stage).
+
+---
+
+## Goal 3 — Orchestration & recovery robustness: stop the unrecoverable kills and runaway recoveries
+
+**Why.** Two failure modes currently end runs that shouldn't end: a single provider going down mid-run (#299), and the recovery stage burning its budget against a stale failure-cap it can't clear (#304). Both waste full event logs. #307 is the cheap observability floor that makes the rest debuggable.
+
+**Tasks.**
+- **#299** Single provider death → unrecoverable session kill (`NoEligibleProvider` on retry) — re-route, don't die.
+- **#300** HealthMonitor detects provider loss ~18s too late (reactive, not gating) — gates #299's recovery path on a fast signal.
+- **#304** Recovery stage runs expensively then run dies on stale failure-cap — wasted work; the cap must reset on the recovery's own progress.
+- **#309** Recovery stage: apply same-fingerprint loop-breaker + repair-ledger — the runaway root cause; closes the loop Goal 3 started.
+- **#307** Observability: no event records the assembled stage-context manifest — cheap event, makes every above failure diagnosable post-run.
+
+**Stretch (if #307 lands fast):** **#308** Background-process execution + monitor tool for long-running shell commands — unblocks real test gates but not load-bearing for the goals above.
+
+**Exit.** A live run where provider downtime is logged + recovered around, and a recovery stage that either converges or breaks the loop with a recorded repair ledger rather than dying on a stale cap.
+
+**Sequencing.**
+- Week 1: #307 (manifest event — cheap, unblocks debugging of everything below).
+- Week 2: #299 + #300 together (provider-death path), #304 + #309 together (recovery runaway path).
+
+---
+
+## Cross-goal sequencing
+
+| Week | Track A (validation) | Track B (freestyle content) | Track C (robustness) |
+|---|---|---|---|
+| 1 M | #80 sub1 — static_analysis seam populated | #260 — discovery prompts | #307 — manifest event |
+| 1 W-F | #263 — build-gate wiring | #297 — analyst CoT loop | — |
+| 2 M | #80 sub2/3/4 — LSP4J + server pool + per-stage | #261 — impl decomposition | #299 + #300 — provider death |
+| 2 W | #80 sub4 — blast-radius filter | #305 + #306 — ACR external + sticky hint | #304 + #309 — recovery runaway |
+| 2 F | #310, #311, #267 — readiness anchor + tag class + acceptance run | — | — |
+
+---
+
+## Intentionally deferred (seen, not dropped)
+
+| ID | Title | Why out |
+|---|---|---|
+| **#167 / #168** | Closed-loop + ACR design epics | Bodies marked IMPLEMENTED for landed slices; remaining work folded into Goal 1 (#80) and Goal 2 (#305/#306). Keep open as epic containers. |
+| **#193** | Frontier-parity design-review round | Design/review work, not a 2-week deliverable. Next cycle after Goals land. |
+| **#31** | Interactive workflow creation TUI/web-ui | Visible polish; not load-bearing for run reliability. Schedule its own sprint. |
+| **#265** | TUI clarification modal not dismissed on external resolve | TUI cluster; pair with the next TUI sprint. |
+| **#295** | TUI token usage display for router/talkie | TUI cluster. |
+| **#296** | TUI execution plan viewer | TUI cluster. |
+| **#298** | TUI output: show CoT/reasoning on artifact + tool-call turns | TUI cluster. |
+| **#301** | Escalate repeated scope/manifest write-block to user approval | Falls under Goal 2 once DoD lands; premature now. |
+| **#302** | Mid-run hard steering (drop inference, inject operator message, restart) | Bigger surface; pairs with the steering-channel design, post-Goal 3 reliability. |
+| **#303** | Auto-repair collapsed-argv shell calls | Nice-to-have shell hygiene. |
+| **#312** | Audit USER- vs SYSTEM/tool-role context placement | Context-salience sweep — better run after #307's manifest event exists so the audit has ground truth. |
+| **#25** | Backlog (deferred/spec-level from memory) | Meta-task; verify-against-code before any sub-item is promoted. |

infrastructure/workflow/src/main/kotlin/com/correx/infrastructure/workflow/Lsp4jDiagnosticsRunner.kt

@@ -146,23 +146,52 @@ class Lsp4jDiagnosticsRunner(
 
     /**
      * Block until every opened URI has received at least one push (or the timeout elapses), then
-     * a short settle window so servers that push an empty report first, real diagnostics second
-     * (tsserver does this after project load) land their final result before we read it.
+     * wait for a quiet period anchored to the last server publication. Servers such as tsserver
+     * can publish an empty report while loading the project and real diagnostics afterward; a
+     * timer started immediately after didOpen reads the workspace before the server is ready.
      */
     private fun awaitDiagnostics(client: CollectingLanguageClient, uris: Set<String>) {
         val deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(timeoutSeconds)
-        while (System.nanoTime() < deadline && !client.hasAll(uris)) {
-            Thread.sleep(POLL_MS)
-        }
-        Thread.sleep(SETTLE_MS)
+        client.awaitAll(uris, deadline)
+        client.awaitQuiet(deadline, TimeUnit.MILLISECONDS.toNanos(SETTLE_MILLIS))
     }
 
     private class CollectingLanguageClient : LanguageClient {
         private val byUri = ConcurrentHashMap<String, List<Diagnostic>>()
+        private val updates = Object()
+        @Volatile private var lastUpdateNanos = 0L
         fun latestFor(uri: String): List<Diagnostic> = byUri[uri].orEmpty()
         fun hasAll(uris: Set<String>): Boolean = byUri.keys.containsAll(uris)
+
+        fun awaitAll(uris: Set<String>, deadline: Long) {
+            synchronized(updates) {
+                while (!hasAll(uris)) {
+                    val remaining = deadline - System.nanoTime()
+                    if (remaining <= 0) return
+                    updates.wait(TimeUnit.NANOSECONDS.toMillis(remaining).coerceAtLeast(1))
+                }
+            }
+        }
+
+        fun awaitQuiet(deadline: Long, quietNanos: Long) {
+            synchronized(updates) {
+                while (true) {
+                    val remaining = deadline - System.nanoTime()
+                    if (remaining <= 0) return
+                    val sinceUpdate = System.nanoTime() - lastUpdateNanos
+                    if (lastUpdateNanos != 0L && sinceUpdate >= quietNanos) return
+                    val waitNanos = minOf(remaining, quietNanos - sinceUpdate)
+                    updates.wait(TimeUnit.NANOSECONDS.toMillis(waitNanos).coerceAtLeast(1))
+                }
+            }
+        }
+
         override fun publishDiagnostics(diagnostics: PublishDiagnosticsParams) {
             byUri[diagnostics.uri] = diagnostics.diagnostics.orEmpty()
+            synchronized(updates) {
+                lastUpdateNanos = System.nanoTime()
+                updates.notifyAll()
+            }
         }
         override fun telemetryEvent(`object`: Any?) = Unit
         override fun showMessage(messageParams: MessageParams) = Unit
@@ -172,7 +201,6 @@ class Lsp4jDiagnosticsRunner(
     }
 
     private companion object {
-        const val POLL_MS = 100L
-        const val SETTLE_MS = 750L
+        const val SETTLE_MILLIS = 750L
     }
 }