claude
a1a2fa3704
Owner's call. E4B is 4.2GB against 6.7GB plus a 0.86GB draft, so with CW2 resident the card holds 5.8GB of 16GB instead of 9.2GB. Measured against a same-session 12B control on the 96-case fixture: 83.3% full against 84.4%, 89.6% intent-only against 91.7%, destination 19/33 against 23/33, p50 294ms against 344ms. Destination is the column that moved. E4B names nothing where the 12B names recall or calendar, which walks the whole chain rather than answering wrong. MTP is gone with the 12B and cannot come back. It is a separate gguf of architecture gemma4-assistant with nextn_predict_layers=4, and the only one on disk is trained against the 12B's hidden states. Neither target gguf carries nextn tensors, so neither self-speculates. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
2.4 KiB
gemma-4-E4B against gemma-4-12B on the routing fixture
Measured 2026-08-09 on workpc. The owner asked for the swap. This is what it costs.
Both arms ran the same 96-case fixture through TestLLMRouterBaseline, minutes
apart, against the same llama-server build and the same mavgpud. The 12B arm is a
control run and not the 2026-08-02 number. That one predates five fixture cases,
the destination labels and a llama.cpp upgrade.
| full | intent-only | destination | p50 | p95 | |
|---|---|---|---|---|---|
| gemma-4-12B-it-qat-UD-Q4_K_XL, MTP draft | 81/96 (84.4%) | 91.7% | 23/33 (69.7%) | 344ms | 471ms |
| gemma-4-E4B-it-qat-UD-Q4_K_XL | 80/96 (83.3%) | 89.6% | 19/33 (57.6%) | 294ms | 562ms |
E4B costs one case of full accuracy, two of intent and four of destination, and buys 50ms at p50. Read the destination column as the finding. One case is three points on 33. So 23 against 19 is outside the noise a single case makes, and the other two columns are not.
Both arms produce three false clarifies and one missed clarify, and neither errored on any case.
What E4B loses
Four of the five destination regressions are the same shape: it names nothing
where the 12B names recall or calendar. ru-query-015 ("сколько я прошёл
шагов") goes further and names self. Naming nothing is the safe direction,
because SourceUnknown walks the whole chain, so these turns are still answered.
They cost latency and they are what a fourth head is meant to fix (V-546).
Two Russian intent cases regress, both with the interrogative off the front.
ru-chat-003 ("расскажи анекдот про программистов") goes to query.
ru-fact-003 ("поужинал") goes to chat.
MTP
E4B has none, and there is no way to give it any on this box. MTP on workpc is
a separate gguf of architecture gemma4-assistant carrying
nextn_predict_layers=4, and mtp-gemma-4-12B-it-BF16.gguf is the only one on
disk. Its head is trained against the 12B's hidden states, so it cannot drive an
E4B target. Scanning both target ggufs finds no nextn tensors in either, so
neither model self-speculates.
So the 12B arm above ran with speculative decoding and E4B ran without, and E4B was still faster.
Cost on the card
E4B is 4.2GB against 6.7GB plus a 0.86GB draft. With CW2 resident at 1.6GB that is 5.8GB of 16GB against 9.2GB. Nothing in Maven needs the difference, so this is headroom for the owner's own jobs rather than a capability.