claude
661b5c1099
A repo-wide sweep on 06-08-2026 at 06c1cf2. Three docs, three tasks.
docs/plans/24-no-deadline-on-the-turn-path.md (V-638). Nothing between a
mavweb handler and llama-server can be cancelled, and one hop has a timeout.
Replier takes no context, the ipc client sets no conn deadline and checks ctx
once, and the ipc server dispatches under Background. Four commits, and the
pattern to copy is already in internal/voice/client.go:101.
docs/plans/25-the-two-boot-paths.md (V-639). The passkey-unlock path starts
seven workers outside the WaitGroup that shutdown waits on, shadows that
WaitGroup at main.go:529, and builds a daemonAPI with no nexus and no
getMCPServers. Latent, because db_key_env means the box boots unlocked.
docs/evals/2026-08-06-routing-trajectory.md (V-464). The deterministic path
and the cascade now score the same 69/91, and the cascade has not been
re-measured since V-626 and V-627. Either the model still earns its place or
it is costing 1.17s a turn for nothing. Dated, so it is not edited later.
Committed with --no-verify, on the owner's instruction of 06-08-2026. The
pre-commit hook refuses master and the alternative was three PRs for three
markdown files. Markdown is already exempt from the size cap for the same
reason: docs land as one batch.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
72 lines
3.4 KiB
Markdown
72 lines
3.4 KiB
Markdown
# The routing trajectory, and the number that is missing
|
|
|
|
**06-08-2026. V-464.** Not a new measurement. This collates the figures already recorded
|
|
in `docs/evals/` and CLAUDE.md, and names one measurement that has not been taken. Dated
|
|
because the conclusion expires the moment the missing number is measured.
|
|
|
|
## The question
|
|
|
|
126 of the 1023 commits between 03-07-2026 and 06-08-2026 touch `internal/router`. Is the
|
|
routing between the core functions and his speech getting better?
|
|
|
|
## The trajectory
|
|
|
|
RU routing fixture, classifier plus the ONNX embedder, no LLM arm in any of these runs.
|
|
|
|
| date | change | fixture | source |
|
|
|---|---|---|---|
|
|
| 02-08-2026 | classifier re-measured | 68.8% of 77 | CLAUDE.md |
|
|
| 04-08-2026 | V-498, rest-of-day and narrative rules | 58/82, 70.7% | CLAUDE.md |
|
|
| 06-08-2026 | V-626 baseline | 64/91, 70.3% | `2026-08-06-seeds-to-prompt-boundary.md` |
|
|
| 06-08-2026 | V-626, seeds onto the prompt boundary | 66/91, 72.5% | same |
|
|
| 06-08-2026 | V-627, alarm verbs reach stage 0 | 69/91, 75.8% | `2026-08-06-alarm-verbs-reach-stage-0.md` |
|
|
| 06-08-2026 | V-633, Russian acts reach tools | 69/91, unchanged | `2026-08-06-russian-acts-reach-tools.md` |
|
|
|
|
The fixture grew from 77 to 82 to 91 cases across this window. So the percentages are
|
|
comparable and the counts are not.
|
|
|
|
## Accuracy moved late
|
|
|
|
It sat near 70% for a month. V-626 and V-627 landed the same day and took the
|
|
deterministic path from 64/91 to 69/91. That is the first real accuracy movement since the
|
|
stage-0 rules went in.
|
|
|
|
## Most of the work was reach, not accuracy
|
|
|
|
Praxis went 0/12 to 11/12 and lifecycle 0/5 to 5/5 (V-516,
|
|
`2026-08-05-praxis-reach.md`). No Russian utterance could reach a tool before V-633. That
|
|
one landed at 69/91 unchanged, because the fixture holds no case for it. Alarm verbs,
|
|
ordinal selection, spoken corrections and the claimant ladder share the shape.
|
|
|
|
So the fixture undercounts the month. Things that were structurally unreachable now reach,
|
|
and a fixture that never asked about them cannot show it. Judge reach against
|
|
`make eval-reach` and the ecosystem fixture, not against the routing one.
|
|
|
|
## The missing number
|
|
|
|
On 05-08-2026 the cascade with the resident model scored 69/91, 75.8% full, 80.2%
|
|
intent-only, at p50 1.19s (`2026-08-05-routing-resident-model.md`).
|
|
|
|
On 06-08-2026 the classifier and stage 0 alone reached 69/91, 75.8% full, at p50 22.9ms.
|
|
|
|
Those are the same full-accuracy score. The cascade has not been re-measured since V-626
|
|
and V-627 landed. Both are stage-0 changes, and stage 0 runs inside the cascade, so the
|
|
cascade should have gained from them too.
|
|
|
|
One of two things is true, and nothing on the box says which:
|
|
|
|
- The cascade gained as well, the model still separates from the floor on intent-only, and
|
|
it earns its place.
|
|
- The deterministic floor has caught up on this fixture, and the resident model is costing
|
|
1.17 seconds a turn for nothing measurable.
|
|
|
|
Take that measurement before planning more routing work. It needs a second llama-server on
|
|
a fixed host port, because the resident one binds `--port 0` inside the container.
|
|
|
|
## What this does not settle
|
|
|
|
Intent-only is the more honest comparison for the model arm. The model routes `reminder`
|
|
and leaves the time to the daemon, which is what the contract asks. The 05-08 run puts it
|
|
at 80.2% through the cascade and 61.5% for the model alone. There is no 06-08 intent-only
|
|
figure for the deterministic path to set beside those.
|