eval: the resident model never reaches Praxis either (V-517)

V-405 measured reach with the classifier only, and the LLM router is the
deployed default, so 16/30 was the floor rather than the shipped behaviour.
TestReachWithLLMRouter scores the same 30 cases with the model, gated on
MAVEN_LLM_URL like TestLLMRouterBaseline.

The open question was whether the model writes a literal Praxis capability
into the fn slot and reaches a service the classifier structurally cannot. It
does not. Praxis is 0/12 with the model alone, exactly what the classifier
alone scores, and all twelve fail the same way: local, empty fn. Nothing in the
router prompt names a Praxis capability, so there is no string for it to write.

So V-516's stage-0 grammars are the only path to Praxis, not a determinism
argument. Through the cascade the model scores 28/30 with praxis 11/12, one
point above the classifier baseline. Hexis is 10/10 either way.

Overreach is 1 in both configurations, under the 4 the harness asserts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SoL7EBdYC5Mhz3DJd49GJy

docs/evals/2026-08-05-reach-llm-router.md

@@ -0,0 +1,70 @@
+# Ecosystem reach with the resident model as router
+
+Date: 2026-08-05. Vikunja #517, split out of #405.
+Fixture: `internal/router/eval/ru_ecosystem_v1.json`, 30 held-out Russian cases.
+Model: Qwen3-1.7B-UD-Q4_K_XL, llama-server on the host at 127.0.0.1:8899.
+Harness: `TestReachWithLLMRouter` in `internal/router/eval/llmrouter_test.go`.
+
+## The numbers
+
+| configuration | reached the right place | praxis | hexis | none |
+|---|---|---|---|---|
+| classifier + hash (V-405 floor) | 16/30 | 0/12 | — | — |
+| classifier + ONNX, after the V-516 grammars | 27/30 | 11/12 | — | — |
+| **llm-only** (resident model alone) | **17/30 (56.7%)** | **0/12** | 10/10 | 7/8 |
+| **cascade + llm** + hash fallback | **28/30 (93.3%)** | **11/12** | 10/10 | 7/8 |
+
+Latency: llm-only p50 1.29s, p95 1.65s. Cascade p50 1.11s, p95 1.64s.
+No case errored in either configuration.
+
+## The open question is answered: the model never reaches Praxis
+
+The route grammar lets the model write any string into the `fn` slot. So it
+could in principle emit a literal Praxis capability name, and reach a service
+the classifier structurally cannot. It does not. **Praxis is 0/12 with the
+model alone.** That is exactly what the classifier alone scores. Every one of
+the twelve fails the same way: the utterance stays local with an empty `fn`.
+
+So the stage-0 Praxis grammars from V-516 are not a determinism argument. They
+are the only path to Praxis that exists. Deleting them takes reach from 11/12
+back to 0/12 whichever engine is answering.
+
+The failure is not that the model routes these badly in its own terms. It
+spreads them across `query`, `fact`, `system` and `chat`. Those are reasonable
+readings of "что требует внимания" and "готово, закрывай" for a model that has
+never been told Praxis exists. Nothing in the prompt names a Praxis capability,
+so there is no string for it to write.
+
+## What the model does buy
+
+Hexis is 10/10 with the model alone, and the mutating tag is 10/15 llm-only
+against 15/15 through the cascade. The model reaches everything Hexis owns
+without help, which is the half the act allowlist already names in the prompt.
+
+Cascade + llm scores one point above the classifier baseline: 28/30 against
+27/30, the difference being one attention case. That is the same shape as the
+routing fixture, where the router buys about 4 points rather than a doubling.
+
+## The two that still miss
+
+- `eco-ru-021 "что там с нексусом"`. Routes `query`, stays local, wants Praxis.
+  Asking after a named service reads as a question about a thing, and no
+  grammar claims a service name.
+- `eco-ru-029 "сделай это"`. Routes `act` and reaches Hexis. The fixture wants
+  nothing reached, because "это" names no target. This is the overreach case
+  and it is the one direction worth failing on. The confirmation binding
+  downstream still resolves a canonical entity id before anything executes.
+  The fixture is right that the turn should have asked.
+
+Overreach is 1 in both configurations, under the 4 the harness asserts.
+
+## How to re-run
+
+```sh
+MAVEN_LLM_URL=http://127.0.0.1:8899 \
+  deps/go/go/bin/go test -v -count=1 -timeout 40m \
+  -run TestReachWithLLMRouter ./internal/router/eval/
+```
+
+The host `http_proxy` answers 503 for 127.0.0.1. `noProxyLoopback` in the test
+excludes it. A run that scores every case as a route error measured the proxy.