diff --git a/MODEL-BAKEOFF-31-07-2026.md b/MODEL-BAKEOFF-31-07-2026.md index c6532eb..0a14993 100644 --- a/MODEL-BAKEOFF-31-07-2026.md +++ b/MODEL-BAKEOFF-31-07-2026.md @@ -1,8 +1,18 @@ # Resident model bake-off — 31-07-2026 -**Recommendation: keep Qwen3.5-0.8B.** LFM2.5-1.2B is worse at routing (52.6% vs 60.5% -intent accuracy), and the loss is almost entirely Russian (18/61 vs 22/61 RU, while EN is a -wash). It is also 2.4× slower. The Thinking variant is far worse again. +**Outcome: the resident model is stock Qwen3-1.7B** (`UD-Q4_K_XL`). Two sweeps ran this +evening and the second one changed the answer — read to the end before acting on any table +here. [Second sweep](#second-sweep-same-evening--five-models-and-a-resident-model-change) +is the one that holds. + +## First sweep — LFM2.5-1.2B vs Qwen3.5-0.8B + +**Verdict, scoped to this pair: keep Qwen3.5-0.8B over LFM2.5-1.2B.** LFM2.5-1.2B is worse +at routing (52.6% vs 60.5% intent accuracy), and the loss is almost entirely Russian +(18/61 vs 22/61 RU, while EN is a wash). It is also 2.4× slower. The Thinking variant is +far worse again. This verdict still stands as written — it rejects LFM2.5-1.2B. It is +**not** a recommendation to keep 0.8B as the resident model; the second sweep replaced it +with Qwen3-1.7B. Settles Vikunja **#278 / #250**.