docs: measure gemma-4-12b on the workstation against the resident model (V-485)

Both fixtures, run from homesrv across the LAN with the proxy env stripped.
Routing: 84.4% full / 93.5% intent-only at p50 329ms through the cascade, against
72.7% / 77.9% at p50 0.80-1.04s for Qwen3-1.7B. Talk: 25/27 against 20/27, with
knowledge 9/9. Nudges 15/15. Settles #485's first assumption by measurement.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NJYcaBiuny9UGSpFweQVQ1
This commit is contained in:
2026-08-02 22:51:47 +04:00
parent 4f33fc7952
commit ee7631158b
2 changed files with 87 additions and 2 deletions
+5 -2
View File
@@ -113,8 +113,11 @@ Then the embedder above, **whisper.cpp** in `mavsttd`, and **piper** in `mavttsd
builds an `llm.Pair` in `modelSeam` (`cmd/mavend/voicewire.go`), and routing
and replies complete through it. Both are the silent half of the rule. The
naming half is not wired. A world question still goes to the resident model
through `PhraseQuery`, and the fixture measurement has not been run.
Biggest quality delta. A 16GB card runs a 7-14B,
through `PhraseQuery`.
Measured, `docs/evals/2026-08-02-workstation-gemma4-12b.md`: gemma-4-12b
through the cascade scores 84.4% full accuracy at p50 329ms. The resident
model scores 72.7% at p50 0.80-1.04s. On the talk fixture it is 25/27
against 20/27. Biggest quality delta. A 16GB card runs a 7-14B,
which fixes what the 1.7B gets wrong: world knowledge, and the persona the CPT
targets. The degradation path is already written and measured, since the
classifier scores 68.8% full accuracy at p50 16.6µs on its own.