eval: score and time the resident model as router (V-320)

Item 2 was blocked because the resident llama-server binds --port 0 inside the
container, so no host process can reach it. Cleared by taking the first of the
three ways out the task listed: a second llama-server on the same gguf, on a
fixed host port.

Cascade + resident model scores 75.8% full and 80.2% intent-only at p50 1.19s
and p95 1.65s, on the fixture as it now stands at 91 cases. That is a new
baseline rather than a movement: 14 cases were added since the 77-case number
in CLAUDE.md.

The model alone scores 37.4% full against 61.5% intent-only. The gap is slots,
not routing. Every reminder case leaves the time to the daemon, which is what
the contract asks of it, and the cascade fills them.

Item 3: the ~6s figure recorded in the task was one sample through the whole
of POST /api/chat, not the router, and is not comparable.

Item 4 is still not run. Killing the resident llama-server needs a permission
this session does not have, and it now has a second half anyway, since with the
workstation up only killing both proves the classifier answers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SoL7EBdYC5Mhz3DJd49GJy

CLAUDE.md

@@ -165,6 +165,15 @@ that stood here until 2026-08-02 was contention, not the model.** See `docs/eval
 p50 825ms / p95 1.2s / max 3.0s and the full cascade at p50 0.80-1.04s. Do not plan latency
 work off the bakeoff table.
 
+**Re-measured 2026-08-05 on the fixture as it now stands, 91 cases** (V-320 item 2,
+`docs/evals/2026-08-05-routing-resident-model.md`): cascade + resident model scores
+**75.8% full / 80.2% intent-only at p50 1.19s / p95 1.65s**. That is a new baseline and not
+a movement, because 14 cases were added since the 77-case number above. The model alone
+scores 37.4% full against 61.5% intent-only, and the gap is slots rather than routing: it
+routes `reminder` and leaves the time to the daemon, which is what the contract asks. To
+re-run it, start a **second** llama-server on a fixed host port — the resident one binds
+`--port 0` inside the container and no host process can reach it.
+
 **The numbers above are the homesrv floor, not the ceiling.** With the workstation up, routing
 completes through `llm.Pair` against gemma-4-12b and scores **84.4% full / 93.5% intent-only at
 p50 329ms** — better than the resident model and about 2.5× faster (`docs/evals/2026-08-02-workstation-gemma4-12b.md`,

docs/evals/2026-08-05-routing-resident-model.md

@@ -0,0 +1,69 @@
+# Routing with the resident model, re-measured
+
+Date: 2026-08-05. Vikunja #320 items 2 and 3.
+Fixture: `internal/router/eval/ru_routing_v1.json`, now **91 cases** (76 ru, 15 en).
+Model: Qwen3-1.7B-UD-Q4_K_XL, llama-server on the host at 127.0.0.1:8899.
+Harness: `TestLLMRouterBaseline`, `make eval-models`.
+
+## How the block was cleared
+
+Item 2 was blocked because the resident llama-server binds `--host 127.0.0.1
+--port 0` inside `maven-mavend-1`. The port is kernel-assigned, scraped from
+stderr and never published, so no `go test` on the host can reach it. The task
+listed three ways out. This run took the first: a **second** llama-server on
+the same gguf, on a fixed host port. The Vega takes the second copy of a 1.7B
+without complaint.
+
+## The numbers
+
+| configuration | full | intent-only | p50 | p95 |
+|---|---|---|---|---|
+| llm-only | 34/91 (37.4%) | 61.5% | 1.24s | 1.65s |
+| cascade + llm + hash fallback | 69/91 (75.8%) | 80.2% | 1.19s | 1.65s |
+
+By language, through the cascade: ru 57/76, en 12/15.
+Clarify: 3 false, 1 missed. No errors. Six slots deferred to the daemon.
+
+For comparison, the figures that stood in CLAUDE.md were 72.7% full and 77.9%
+intent-only, measured on 77 cases. The fixture has grown by 14 cases since, so
+this is a new baseline rather than a movement.
+
+## llm-only is low for a reason that is not routing
+
+37.4% full against 61.5% intent-only is the gap, and it is almost entirely
+slots. Every reminder case fails with "no time slot, want one". The model
+routes `reminder` correctly and leaves the time to the daemon, which is what
+the contract asks of it. The cascade fills those slots. That is why the same
+model scores 38 points higher inside it.
+
+Three cases errored in the llm-only arm and none in the cascade, which is the
+fallback working as designed.
+
+## Item 3: latency
+
+Router p50 1.19s, p95 1.65s, max 1.79s through the cascade. The one earlier
+data point in the task, roughly 6s wall clock for `привет` through
+`POST /api/chat`, was the whole path and not the router. It is not comparable
+and should not be quoted as a routing number.
+
+These numbers are the homesrv floor. With the workstation up, routing completes
+against gemma-4-12b at p50 329ms, measured separately in
+`docs/evals/2026-08-02-workstation-gemma4-12b.md`.
+
+## What still misses
+
+The confusion is concentrated in one direction: `query→fact ×4`,
+`query→note ×3`, `query→system ×3`. A question about his own rows that carries
+no interrogative reads as a statement to the model. Ten of the twenty-two
+failures are that shape, including "я сегодня вообще пил воду" and "чем я
+занимался в среду". This is the case V-546's three-head classifier is aimed at.
+
+The two `разбуди меня` cases clarify at 0.300 instead of routing `reminder`.
+
+## Item 4 is still not run
+
+Killing the resident llama-server to confirm the classifier floor needs a
+permission this session does not have. The test is otherwise ready. It now has
+a second half. With the workstation up, killing the resident server should
+still complete a turn through `modelSeam`. Only killing both proves the
+classifier answers.