# The CPT+SFT Qwen3-1.7B routes better and cannot hold a sentence

*Measured 2026-08-19 on homesrv, CPU. Unfiled: Vikunja has returned 503 since
2026-08-13. Build `5cae33a` on master. The deployment was not changed.*

The candidate is `/mnt/hdd1/llms/maven/maven-model-Q4_K_XL.gguf`, built on workpc
from a continued-pretraining stage plus a supervised fine-tune, quantised by
replaying the Unsloth imatrix recipe with 197 per-tensor overrides read out of
the reference gguf. The incumbent is
`/mnt/hdd1/llms/qwen3/Qwen3-1.7B-UD-Q4_K_XL.gguf`, the deployed resident model.

Both arms ran on the same homesrv `llama-server`, same flags, same port, minutes
apart, one model resident at a time. CPU inference throughout, so **every latency
figure here is incomparable to any GPU run**, including the 116.7 ms the build
side reported. Accuracy is the only number that carries.

The build side's own measurement claimed 88/96 for the candidate. This run
reproduces it at 87/96 on the same fixture, so the routing claim stands.

## Routing, `TestLLMRouterBaseline`, 96 cases

| | full | intent-only | destination | ru | en | p50 |
|---|---|---|---|---|---|---|
| maven-model-Q4_K_XL | **87/96 (90.6%)** | 94.8% | **24/33 (72.7%)** | 72/81 | 15/15 | 1.29s |
| Qwen3-1.7B-UD-Q4_K_XL | 81/96 (84.4%) | 87.5% | 16/33 (48.5%) | 67/81 | 14/15 | 1.22s |

Both ran 0 errors and 1 missed clarify on the cascade. Destination is where the
candidate earns it: eight more correct query sources, 72.7% against 48.5%. That
equals what `docs/evals/2026-08-09-e4b-vs-12b-routing.md` records for
gemma-4-12B at seven times the parameter count.

The `llm-only` section of the same run reads 59/96 against 45/96. It bypasses
stage 0 and the daemon slot fillers on purpose. **Do not cite it as the routing
number**; `cascade+llm` is the deployed path.

## Phrasing, `internal/phraser/eval`

| | nudges | conversational |
|---|---|---|
| maven-model-Q4_K_XL | 15/15 (100%) | **8/36 (22.2%)** |
| maven-model-Q4_K_XL, eos corrected | 15/15 (100%) | 11/36 (30.6%) |
| Qwen3-1.7B-UD-Q4_K_XL | 15/15 (100%) | 22/36 (61.1%) |

Nudges are tied at ceiling. Conversational phrasing falls by half.

## Two defects, and they are independent

**The gguf carries the wrong eos token.** `tokenizer.ggml.eos_token_id` is
151643 (`<|endoftext|>`) against the incumbent's 151645 (`<|im_end|>`), and
`bos` is 151643 too. Both files carry the identical Qwen3 chat template, and
that template closes every turn with `<|im_end|>`. The candidate is told to stop
on a token the template never emits, so generation runs through the turn
boundary. The conversion read the base config, not the instruct one.

Re-running with `--override-kv tokenizer.ggml.eos_token_id=int:151645` isolates
its cost:

| check | as shipped | eos corrected |
|---|---|---|
| address | 27/36 | 32/36 |
| knowledge path | 3/9 | 7/9 |
| ontopic | 14/36 | 15/36 |
| reply path | 0/9 | 0/9 |

Routing is untouched by this, because GBNF-constrained decoding terminates on
the grammar and never reaches the eos token. That is why the defect is invisible
in the routing score and fatal in free generation.

**The remaining half is not packaging.** Corrected, the candidate reaches 11/36
against the incumbent's 22/36. `ontopic` moves one case and the reply path stays
at 0 of 9. The failures are broken Russian, not persona drift:

- `Ты не одиноден?!` — not a word; `одинок`.
- `Привёз с собой смайлики-котята и вручила им подарок` — masculine and feminine
  verbs in one clause, wrong case on the noun.
- `Вчера у вас не будет никакого лёг.` — yesterday in the future tense, cut off
  mid-word.
- `Заберите диск от системы или установочный ковш.` — "installation bucket".
- `Вам сказал Игоря Валерьевич` — a first name declined as accusative.
- `know-hiccups` answered with weight-loss advice.

## What this rules out

- **Not a harness mismatch.** Both arms ran the identical target, server and
  fixture, and the candidate reproduced the build side's routing claim to within
  one case.
- **Not the missing `ROLEPLAY_ACTION` and `MASC_SELF` fixes alone.** Those were
  the build side's stated reason to expect persona failures. `feminine` scores
  35/36 corrected. The failures are grammatical, not persona.
- **Not weights damage alone either.** The eos defect is real and costs five
  address cases and four knowledge cases on its own.

`inference:` the SFT stage worked and the CPT stage cost general Russian
coherence. Routing is constrained decoding over 7 intents and a destination, and
it improved. Free generation is where the loss shows.

## The swap was not made

`deploy/mavend.json:19` still reads the incumbent. One `phraser.model_path`
serves both the router and the phraser and `voice.llm_router` is true, so
swapping buys 6 routing cases and pays 11 conversational ones. Both ggufs stay
at `/mnt/hdd1/llms/maven/` for the re-run.

The two-week replay (`scripts/usage-run.py`,
`docs/evals/2026-08-08-two-weeks.md`) was not run. It drives the deployed stack
through `POST /api/chat`, so it measures whatever `mavend` has loaded, and the
candidate was never loaded.