kami/Maven
1
0
Fork 0
Code Issues 1 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Swap the workstation model to gemma-4-E4B (V-486)

Browse Source 
Owner's call. E4B is 4.2GB against 6.7GB plus a 0.86GB draft, so with CW2
resident the card holds 5.8GB of 16GB instead of 9.2GB.

Measured against a same-session 12B control on the 96-case fixture: 83.3% full
against 84.4%, 89.6% intent-only against 91.7%, destination 19/33 against
23/33, p50 294ms against 344ms. Destination is the column that moved. E4B names
nothing where the 12B names recall or calendar, which walks the whole chain
rather than answering wrong.

MTP is gone with the 12B and cannot come back. It is a separate gguf of
architecture gemma4-assistant with nextn_predict_layers=4, and the only one on
disk is trained against the 12B's hidden states. Neither target gguf carries
nextn tensors, so neither self-speculates.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
This commit is contained in:
claude
2026-08-09 01:15:00 +04:00
parent 22a4978459
commit a1a2fa3704
3 changed files with 72 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+13 -3
View File
@@ -255,11 +255,21 @@ re-run it, start a **second** llama-server on a fixed host port — the resident
`--port 0` inside the container and no host process can reach it.
**The numbers above are the homesrv floor, not the ceiling.** With the workstation up, routing
completes through `llm.Pair` against gemma-4-12b and scores **84.4% full / 93.5% intent-only at
p50 329ms** — better than the resident model and about 2.5× faster (`docs/evals/2026-08-02-workstation-gemma4-12b.md`,
Vikunja #485). The workstation is never assumed up, so both sets of numbers are live. Judge a
completes through `llm.Pair` against the model mavgpud holds, which is better than the resident
model and about 2.5× faster. gemma-4-12b scored **84.4% full / 93.5% intent-only at p50 329ms**
(`docs/evals/2026-08-02-workstation-gemma4-12b.md`, Vikunja #485). The workstation is never
assumed up, so both sets of numbers are live. Judge a
routing change against the classifier and the resident model, since those are what always answer.
**The workstation runs gemma-4-E4B since 2026-08-09** (owner's call), and it is a
step down measured the same day (`docs/evals/2026-08-09-e4b-vs-12b-routing.md`).
Against a same-session 12B control it scores **83.3% full / 89.6% intent-only,
destination 19/33 against 23/33, at p50 294ms against 344ms**. So it costs four
destination cases and buys 50ms. Read destination as the finding: it names nothing
where the 12B names `recall` or `calendar`, which is safe but walks the whole chain.
It also has no MTP and cannot be given any here. The only `gemma4-assistant`
draft on disk is trained against the 12B's hidden states.
**The intended third engine is not a generative model** (owner's call, 05-08-2026, V-546,
`docs/plans/18-routing-heads-on-e5-small.md`). Routing has a bounded output space, so it is
classification, and the 118M multilingual-e5-small is already resident. Three heads on one