Tonight's work as one branch: all 46 reviewed PRs, verified green #47
@@ -1,8 +1,18 @@
|
||||
# Resident model bake-off — 31-07-2026
|
||||
|
||||
**Recommendation: keep Qwen3.5-0.8B.** LFM2.5-1.2B is worse at routing (52.6% vs 60.5%
|
||||
intent accuracy), and the loss is almost entirely Russian (18/61 vs 22/61 RU, while EN is a
|
||||
wash). It is also 2.4× slower. The Thinking variant is far worse again.
|
||||
**Outcome: the resident model is stock Qwen3-1.7B** (`UD-Q4_K_XL`). Two sweeps ran this
|
||||
evening and the second one changed the answer — read to the end before acting on any table
|
||||
here. [Second sweep](#second-sweep-same-evening--five-models-and-a-resident-model-change)
|
||||
is the one that holds.
|
||||
|
||||
## First sweep — LFM2.5-1.2B vs Qwen3.5-0.8B
|
||||
|
||||
**Verdict, scoped to this pair: keep Qwen3.5-0.8B over LFM2.5-1.2B.** LFM2.5-1.2B is worse
|
||||
at routing (52.6% vs 60.5% intent accuracy), and the loss is almost entirely Russian
|
||||
(18/61 vs 22/61 RU, while EN is a wash). It is also 2.4× slower. The Thinking variant is
|
||||
far worse again. This verdict still stands as written — it rejects LFM2.5-1.2B. It is
|
||||
**not** a recommendation to keep 0.8B as the resident model; the second sweep replaced it
|
||||
with Qwen3-1.7B.
|
||||
|
||||
Settles Vikunja **#278 / #250**.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user