Measure E4B on phrasing, the half nobody had scored (V-668)

The 2026-08-09 model swap was measured on routing the same day and E4B lost
four destination cases. Phrasing was not measured, and phrasing is the half the
owner hears.

E4B scores nudges 15/15 and the talk fixture 29/36 at p50 516ms, against the
resident model's 25/36 at p50 2.97s on the same 36 cases. lang, feminine and
address are all 36/36, where the resident model loses three on address. Every
failure is ontopic and none is a parse error.

29/36 is one case off the ceiling. The temperature sweep of 2026-08-05 found
two reply cases that fail at every temperature and named a defect in the reply
path, capping the fixture at 30/36. Both are in E4B's failure list. So the swap
costs nothing on phrasing.

One defect no check catches: in chat E4B writes "Я записала несколько идей!"
when nothing was stored. A claim to have saved something is a claim about state.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
This commit is contained in:
2026-08-09 12:07:13 +04:00
parent 0db9ca084c
commit 229890abd7
6 changed files with 127 additions and 0 deletions
+10
View File
@@ -269,6 +269,16 @@ destination cases and buys 50ms. Read destination as the finding: it names nothi
where the 12B names `recall` or `calendar`, which is safe but walks the whole chain.
It also has no MTP and cannot be given any here. The only `gemma4-assistant`
draft on disk is trained against the 12B's hidden states.
**Phrasing was the unmeasured half and it is measured now**
(`docs/evals/2026-08-09-e4b-phrasing.md`). E4B scores nudges 15/15 and the
36-case talk fixture **29/36 at p50 516ms**, against the resident model's 25/36
at p50 2.97s. Persona is clean: `lang`, `feminine` and `address` are all 36/36,
where the resident model loses three on `address`. Every failure is `ontopic`
and none is a parse error. The 2026-08-05 temperature sweep put this fixture's
ceiling at 30/36, because two reply cases fail at every temperature (V-537), and
both are in E4B's failure list. So the swap costs nothing here. One defect no
check catches: in chat E4B claims "Я записала несколько идей!" when nothing was
stored, which is a wrong claim about state.
**The intended third engine is not a generative model** (owner's call, 05-08-2026, V-546,
`docs/plans/18-routing-heads-on-e5-small.md`). Routing has a bounded output space, so it is