# Talk fixture against the resident model, 2026-08-05

Vikunja #44 step 1. `MAVEN_LLM_URL=http://127.0.0.1:8899 make eval-phrasing`,
Qwen3-1.7B-UD-Q4_K_XL on the host, no workstation in the run. The fixture holds
36 cases now, against 27 when the bakeoff measured it. So the old score is not
a column in this table.

## Result

| | before the escape fix | after |
|---|---|---|
| talk, passes every check | 2/36 (5.6%) | 25/36 (69.4%) |
| failed generations | 31 | 0 |
| by path: chat | 0/9 | 4/9 |
| by path: knowledge | 1/9 | 6/9 |
| by path: query | 0/9 | 9/9 |
| by path: reply | 1/9 | 6/9 |
| feminine | 5/36 | 36/36 |
| address | 5/36 | 33/36 |
| ontopic | 2/36 | 28/36 |
| p50 latency | 3.05s | 2.97s |
| nudges (15 cases) | 15/15 | 15/15 |

## What the 31 errors were

Not the model. `escapeRawControls` in `internal/phraser/llmphraser.go`, added
for #537 to repair a raw newline written inside a string, escaped the whole
object. Qwen3-1.7B pretty-prints: it opens `{` and writes three newlines before
the first key. Those newlines became a literal backslash-n, which is legal
nowhere outside a string, so the object stopped parsing and `parseResponseMood`
reported `errBrokenJSON`.

The comment said escaping unconditionally could not turn valid JSON into
anything else, because JSON permits no control character outside a string. It
permits three. Newline, tab and return are whitespace between tokens, and that
is what pretty-printing is made of.

Every chat reply and every knowledge answer the resident model wrote was being
discarded for a stub line. The nudge path never showed it, because the nudge
prompt gets compact JSON back.

## The 11 that still fail

Eight are `ontopic`, three are `address`.

The address failures are all plural imperatives written to a formal listener:
`держите`, `уточните`, `попробуйте`. Feminine self-reference held in all 36,
which is the half #122 is training for. So the persona gap the CPT is aimed at
is now the address half, not the gender half.

The ontopic failures are the resident model answering next to the question
rather than in it. `chat-joke` describes crying dolls instead of telling one,
`know-hiccups` calls hiccups an icon, `know-boil-egg` answers about an omelette.
`chat-about-me` answers "Я - записка", which is the same confabulation the
bakeoff recorded.

## Not measured here

The workstation. Every number above is the homesrv floor. `make eval-phrasing`
points at one URL, so a gemma-4-12b column needs its own run.
