From 533f0acda89e2fce6e49022d958af4c035745fb0 Mon Sep 17 00:00:00 2001 From: kami Date: Fri, 31 Jul 2026 21:17:07 +0400 Subject: [PATCH] Lead the bake-off with the answer, not the superseded one MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The file ran two sweeps and the second one changed the resident model, but the lede still opened with "Recommendation: keep Qwen3.5-0.8B". Anyone landing on the file read the wrong conclusion and had to scroll 100 lines to find that it had been replaced — and it contradicted CLAUDE.md, which already says the resident model is Qwen3-1.7B. Both sweeps are accurate, so nothing is rewritten. The lede now states the outcome and the first sweep's verdict is scoped to what it actually tested: it rejects LFM2.5-1.2B, which still holds. It never was a case for keeping 0.8B as the resident model. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01CGeSZxh1DCtRxmFVSYVGvJ --- MODEL-BAKEOFF-31-07-2026.md | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/MODEL-BAKEOFF-31-07-2026.md b/MODEL-BAKEOFF-31-07-2026.md index c6532eb..0a14993 100644 --- a/MODEL-BAKEOFF-31-07-2026.md +++ b/MODEL-BAKEOFF-31-07-2026.md @@ -1,8 +1,18 @@ # Resident model bake-off — 31-07-2026 -**Recommendation: keep Qwen3.5-0.8B.** LFM2.5-1.2B is worse at routing (52.6% vs 60.5% -intent accuracy), and the loss is almost entirely Russian (18/61 vs 22/61 RU, while EN is a -wash). It is also 2.4× slower. The Thinking variant is far worse again. +**Outcome: the resident model is stock Qwen3-1.7B** (`UD-Q4_K_XL`). Two sweeps ran this +evening and the second one changed the answer — read to the end before acting on any table +here. [Second sweep](#second-sweep-same-evening--five-models-and-a-resident-model-change) +is the one that holds. + +## First sweep — LFM2.5-1.2B vs Qwen3.5-0.8B + +**Verdict, scoped to this pair: keep Qwen3.5-0.8B over LFM2.5-1.2B.** LFM2.5-1.2B is worse +at routing (52.6% vs 60.5% intent accuracy), and the loss is almost entirely Russian +(18/61 vs 22/61 RU, while EN is a wash). It is also 2.4× slower. The Thinking variant is +far worse again. This verdict still stands as written — it rejects LFM2.5-1.2B. It is +**not** a recommendation to keep 0.8B as the resident model; the second sweep replaced it +with Qwen3-1.7B. Settles Vikunja **#278 / #250**.