docs: record the workstation routing numbers where the router is documented (V-485)
CLAUDE.md carried only the homesrv figures, which now read as the whole story. Also points offload.md's order at #490 for the naming half. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NJYcaBiuny9UGSpFweQVQ1
This commit is contained in:
@@ -143,7 +143,15 @@ seed additions, both of which now score inside the classifier baseline. Qwen3-1.
|
|||||||
not a doubling, and the trade is worth re-arguing rather than assuming. **The ≈2.7s figure
|
not a doubling, and the trade is worth re-arguing rather than assuming. **The ≈2.7s figure
|
||||||
that stood here until 2026-08-02 was contention, not the model.** See `docs/evals/2026-07-31-routing.md` line 61, which measures the LLM router at
|
that stood here until 2026-08-02 was contention, not the model.** See `docs/evals/2026-07-31-routing.md` line 61, which measures the LLM router at
|
||||||
p50 825ms / p95 1.2s / max 3.0s and the full cascade at p50 0.80-1.04s. Do not plan latency
|
p50 825ms / p95 1.2s / max 3.0s and the full cascade at p50 0.80-1.04s. Do not plan latency
|
||||||
work off the bakeoff table. `Confidence: 1.0` used to be hardcoded in `llmrouter.go`, so the LLM
|
work off the bakeoff table.
|
||||||
|
|
||||||
|
**The numbers above are the homesrv floor, not the ceiling.** With the workstation up, routing
|
||||||
|
completes through `llm.Pair` against gemma-4-12b and scores **84.4% full / 93.5% intent-only at
|
||||||
|
p50 329ms** — better than the resident model and about 2.5× faster (`docs/evals/2026-08-02-workstation-gemma4-12b.md`,
|
||||||
|
Vikunja #485). The workstation is never assumed up, so both sets of numbers are live. Judge a
|
||||||
|
routing change against the classifier and the resident model, since those are what always answer.
|
||||||
|
|
||||||
|
`Confidence: 1.0` used to be hardcoded in `llmrouter.go`, so the LLM
|
||||||
path could never ask for clarification (6/6 refusal cases missed on the fixture) — Vikunja
|
path could never ask for clarification (6/6 refusal cases missed on the fixture) — Vikunja
|
||||||
#359. Fixed 31-07-2026 with structural signal (single-token utterance, keyless fact, act with
|
#359. Fixed 31-07-2026 with structural signal (single-token utterance, keyless fact, act with
|
||||||
no allowlisted fn) feeding the same stage-3 gate the classifier path already had — see
|
no allowlisted fn) feeding the same stage-3 gate the classifier path already had — see
|
||||||
|
|||||||
+1
-1
@@ -113,7 +113,7 @@ Then the embedder above, **whisper.cpp** in `mavsttd`, and **piper** in `mavttsd
|
|||||||
builds an `llm.Pair` in `modelSeam` (`cmd/mavend/voicewire.go`), and routing
|
builds an `llm.Pair` in `modelSeam` (`cmd/mavend/voicewire.go`), and routing
|
||||||
and replies complete through it. Both are the silent half of the rule. The
|
and replies complete through it. Both are the silent half of the rule. The
|
||||||
naming half is not wired. A world question still goes to the resident model
|
naming half is not wired. A world question still goes to the resident model
|
||||||
through `PhraseQuery`.
|
through `PhraseQuery`. That, and the four callers 485 did not reach, are #490.
|
||||||
Measured, `docs/evals/2026-08-02-workstation-gemma4-12b.md`: gemma-4-12b
|
Measured, `docs/evals/2026-08-02-workstation-gemma4-12b.md`: gemma-4-12b
|
||||||
through the cascade scores 84.4% full accuracy at p50 329ms. The resident
|
through the cascade scores 84.4% full accuracy at p50 329ms. The resident
|
||||||
model scores 72.7% at p50 0.80-1.04s. On the talk fixture it is 25/27
|
model scores 72.7% at p50 0.80-1.04s. On the talk fixture it is 25/27
|
||||||
|
|||||||
Reference in New Issue
Block a user