diff --git a/docs/evals/2026-08-06-routing-trajectory.md b/docs/evals/2026-08-06-routing-trajectory.md new file mode 100644 index 0000000..5fdca3c --- /dev/null +++ b/docs/evals/2026-08-06-routing-trajectory.md @@ -0,0 +1,71 @@ +# The routing trajectory, and the number that is missing + +**06-08-2026. V-464.** Not a new measurement. This collates the figures already recorded +in `docs/evals/` and CLAUDE.md, and names one measurement that has not been taken. Dated +because the conclusion expires the moment the missing number is measured. + +## The question + +126 of the 1023 commits between 03-07-2026 and 06-08-2026 touch `internal/router`. Is the +routing between the core functions and his speech getting better? + +## The trajectory + +RU routing fixture, classifier plus the ONNX embedder, no LLM arm in any of these runs. + +| date | change | fixture | source | +|---|---|---|---| +| 02-08-2026 | classifier re-measured | 68.8% of 77 | CLAUDE.md | +| 04-08-2026 | V-498, rest-of-day and narrative rules | 58/82, 70.7% | CLAUDE.md | +| 06-08-2026 | V-626 baseline | 64/91, 70.3% | `2026-08-06-seeds-to-prompt-boundary.md` | +| 06-08-2026 | V-626, seeds onto the prompt boundary | 66/91, 72.5% | same | +| 06-08-2026 | V-627, alarm verbs reach stage 0 | 69/91, 75.8% | `2026-08-06-alarm-verbs-reach-stage-0.md` | +| 06-08-2026 | V-633, Russian acts reach tools | 69/91, unchanged | `2026-08-06-russian-acts-reach-tools.md` | + +The fixture grew from 77 to 82 to 91 cases across this window. So the percentages are +comparable and the counts are not. + +## Accuracy moved late + +It sat near 70% for a month. V-626 and V-627 landed the same day and took the +deterministic path from 64/91 to 69/91. That is the first real accuracy movement since the +stage-0 rules went in. + +## Most of the work was reach, not accuracy + +Praxis went 0/12 to 11/12 and lifecycle 0/5 to 5/5 (V-516, +`2026-08-05-praxis-reach.md`). No Russian utterance could reach a tool before V-633. That +one landed at 69/91 unchanged, because the fixture holds no case for it. Alarm verbs, +ordinal selection, spoken corrections and the claimant ladder share the shape. + +So the fixture undercounts the month. Things that were structurally unreachable now reach, +and a fixture that never asked about them cannot show it. Judge reach against +`make eval-reach` and the ecosystem fixture, not against the routing one. + +## The missing number + +On 05-08-2026 the cascade with the resident model scored 69/91, 75.8% full, 80.2% +intent-only, at p50 1.19s (`2026-08-05-routing-resident-model.md`). + +On 06-08-2026 the classifier and stage 0 alone reached 69/91, 75.8% full, at p50 22.9ms. + +Those are the same full-accuracy score. The cascade has not been re-measured since V-626 +and V-627 landed. Both are stage-0 changes, and stage 0 runs inside the cascade, so the +cascade should have gained from them too. + +One of two things is true, and nothing on the box says which: + +- The cascade gained as well, the model still separates from the floor on intent-only, and + it earns its place. +- The deterministic floor has caught up on this fixture, and the resident model is costing + 1.17 seconds a turn for nothing measurable. + +Take that measurement before planning more routing work. It needs a second llama-server on +a fixed host port, because the resident one binds `--port 0` inside the container. + +## What this does not settle + +Intent-only is the more honest comparison for the model arm. The model routes `reminder` +and leaves the time to the daemon, which is what the contract asks. The 05-08 run puts it +at 80.2% through the cascade and 61.5% for the model alone. There is no 06-08 intent-only +figure for the deterministic path to set beside those. diff --git a/docs/plans/24-no-deadline-on-the-turn-path.md b/docs/plans/24-no-deadline-on-the-turn-path.md new file mode 100644 index 0000000..331223f --- /dev/null +++ b/docs/plans/24-no-deadline-on-the-turn-path.md @@ -0,0 +1,91 @@ +# No deadline on the turn path + +Last verified: 06-08-2026 @ 06c1cf2 + +V-638. Sibling of V-607, which is the same class of bug in `internal/worker`. +Reads with `docs/offload.md` and `docs/protocol.md`. + +## What is missing + +A chat turn starts in a mavweb HTTP handler and ends at llama-server. Nothing between those +two points can be cancelled, and one hop has a timeout. + +Four places, all on the same path. + +`voice.Replier.Reply` takes no context (`internal/voice/replier.go:41`). So `llmReplier` +calls `PhraseReply(context.Background(), d)` at `cmd/mavend/replier_llm.go:42`. The turn +cannot deadline its own reply. The only bound is `phraser.timeout`, 60s in deploy. + +`ipc.Client.roundtrip` sets no connection deadline (`internal/ipc/client.go:202`). A daemon +that stops answering parks the caller for as long as the socket stays open. + +`ipc.Client.call` checks the context once, before sending (`client.go:149`), then blocks in +`roundtrip`. Cancelling mid-call does nothing. + +`ipc.Server.serveConn` dispatches under `context.Background()` (`internal/ipc/server.go:253`). +A client that hangs up does not cancel the turn, and neither does `Server.Close`. + +## And every call queues behind the slowest one + +`ipc.Client` serialises on one connection and one mutex. mavweb routes `/api/chat` and +`/api/ptt` through the shared client, so one turn blocks all 28 handlers while it runs. +Worst case is a 60s page load. + +This is understood for exactly one route already. `cmd/mavweb/main.go:57` opens a second +connection for `/models`, and the comment there says why. A model swap is a multi-minute +call, and sharing the connection would freeze every other page. + +## The pattern is already in the repo + +`internal/voice/client.go:101` derives a connection deadline from the caller's context, +falls back to 120s, and clears it with a defer. `internal/ipc/client.go` never learned it. +Copy that rather than inventing a second convention. + +## The work + +One commit each. + +**Context on the reply seam.** `phraser.Replier.PhraseReply` already takes a context and the +interface has two implementations, so this is small. Change `Reply` to take a context, have +`StubReplier` ignore it, and pass it through `llmReplier` to `PhraseReply`. Both call sites +already hold one: `cmd/mavend/voice.go:461` and `cmd/mavend/clarify.go:574`. + +**Deadlines and cancellation on the client.** Pass the context into `roundtrip` and set +`SetDeadline` from it. For cancellation mid-call, a watchdog goroutine that calls `c.drop()` +on `ctx.Done()` is enough. `drop` exists, and the retry split already separates a lost write +from a lost read. So a cancelled call lands in `errReadLost` and is never retried for a +mutation. Check that against `internal/ipc/maperr_test.go`. + +**A request context on the server.** `serveConn` should derive from a server-scoped context +so `Close` cancels a dispatch in flight. `Server` already carries `done` and a conn registry +for this class of problem. The registry comment records what the last version of it cost: +eleven days of stale ciphertext. + +**Stop serialising mavweb.** Give `/api/chat` and `/api/ptt` their own connection, the way +`/models` has one. Roughly ten lines, and it changes no shared code. + +A connection pool inside `ipc.Client` is the general form and is deliberately not the first +step. Each connection is already its own request and response stream. So a pool preserves +frame pairing by construction. It still has to keep re-dial on drop, the +`errWriteLost` and `errReadLost` split, and `Close`. Do the narrow fix, measure, and reach +for the pool only if a second module turns out to queue. + +## How it is judged + +`make test` stays green. It is green at `06c1cf2`. + +Nothing here changes routing or recall, so `make eval-router` and `make eval-recall` are +unchanged rather than re-measured. + +By hand: load `/dash` while a chat turn is in flight. Before the change it waits for the +length of the turn. + +There is no test today that a cancelled context aborts an in-flight `ipc.Client` call. That +absence is why two of these four went unnoticed, so the test is part of the work. + +## What is not done here + +The store is still `SetMaxOpenConns(1)` (`internal/store/store.go:99`) under WAL. WAL is +built for concurrent readers against one writer, and the cap makes every read queue. +`Store.DB(ctx)` hands the digestion worker a read transaction on that same connection. This +plan does not touch it. It is measurable first and should be measured before it is changed. diff --git a/docs/plans/25-the-two-boot-paths.md b/docs/plans/25-the-two-boot-paths.md new file mode 100644 index 0000000..72d2b45 --- /dev/null +++ b/docs/plans/25-the-two-boot-paths.md @@ -0,0 +1,82 @@ +# The two boot paths have drifted + +Last verified: 06-08-2026 @ 06c1cf2 + +V-639. Reads with `docs/operations.md`. + +## What is wrong + +`run()` in `cmd/mavend/main.go` brings the daemon up two ways. A box with a key in the +environment starts unlocked and wires everything at lines 280 to 621. A box without one +starts locked. It wires the same things again inside the unlock closure, at lines 500 to +579, after a passkey assertion. + +The two lists have drifted apart. Three ways. + +**Seven workers start untracked.** The unlocked path puts every one through +`goWorker(&wg, ...)`, so `waitWorkers` at line 637 can wait for them. The unlock path +starts `tl.run`, `factWorker`, `evalWorker`, `feedWkr`, `crawlWkr`, `mcp.run` and +`home.run` as bare `go func()`. Nothing waits for any of them. + +That is the shutdown bug the code already documents at lines 631 to 636, reintroduced on +the other path. The comment there records what it cost the first time. `run()` never +returned, so `defer st.Close()` never sealed the database. The deployed ciphertext was +eleven days stale before anyone noticed. + +**A shadowed WaitGroup hides it.** Line 529 declares `var wg sync.WaitGroup` inside the +`if voiceW != nil` block, shadowing the one from line 359. It is `Add`ed and `Done`d and +never waited. Reading the block, the voice server looks tracked. It is not. + +**Two `daemonAPI` fields are never set.** The unlocked path fills `nexus` at line 295 and +`getMCPServers` at line 305. The unlock path fills neither. So after a passkey unlock, +`ResolveEntity` answers `ErrNotImplemented` with a `nexus` block configured, and +`MCPServers` answers empty with an `mcp` block configured. + +The second is the worse one. Empty is not a degraded answer, it is a wrong answer, and +`/tools` renders it as "not configured". + +## Why it drifted + +`wireTelegramIntake` was added to both paths on 06-08-2026 (V-637) and it does use the +outer `wg`, at line 519. So the newest line on that path is correct and the older ones +around it are not. The path gets touched one line at a time and is never read whole. + +The shape of `cmd/mavend` is what allows that. It is 155 files and 9,551 lines of code. +Six things live in it with no seam between them: + +- the handler +- the action dispatch +- the 19 query sources +- the wiring functions +- the six background workers +- these two boot paths + +Nothing in the package makes the divergence visible. + +## The fix + +Make the two paths call one function instead of listing the same wiring twice. + +One `startBackground(ctx, &wg, deps)` that takes what it needs and starts every worker +through `goWorker`. One `newDaemonAPI(deps)` that fills every field, including `nexus` and +`getMCPServers`, so a field added later cannot reach one path and miss the other. Both +call sites then read as one call each, and a future addition has one place to go. + +Delete the shadowed `wg` at line 529 as part of it. + +## How it is judged + +`make test` stays green. + +The regression that matters is a test asserting the two paths wire the same set. Compare +the constructed `daemonAPI` field by field, and assert the worker count started under the +outer `wg` matches. Without that, this drifts again the next time a wiring line is added. + +Then confirm on a locked box: unlock by passkey, ask something that needs Nexus, and check +`/tools` lists the MCP servers. Both answer wrongly today. + +## Priority + +Latent, not live. `deploy/mavend.json` sets `db_key_env`, so homesrv boots unlocked and +takes the correct path. This bites the locked deployment that `docs/operations.md` +describes, and it bites silently.