diff --git a/CLAUDE.md b/CLAUDE.md index 9c17503..f297282 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -7,9 +7,20 @@ talking over unix sockets; one resident small model for routing + phrasing; whis Deploy target is a Ryzen laptop (homesrv) with Vulkan offload to the Vega iGPU (`n_gpu_layers: 99`, compose passes `/dev/dri` + the render gid) — the resident model stays ≤1.7B either way. -**Resident model:** currently **Qwen3.5-0.8B** (`Q4_K_M`), the smallest checkpoint in the gguf -library, picked for CPU/iGPU latency. The **target** is the locally CPT'd **Qwen3-1.7B**; that -training is still in flight (Vikunja #122), so no such gguf exists yet. Model files live in +**Resident model:** currently **Qwen3-1.7B** (`UD-Q4_K_XL`), stock — not yet the CPT'd one. +It replaced Qwen3.5-0.8B on 2026-07-31 because it measured better on both fixtures we have: +67.5% vs 59.7% intent-only on the 77-case RU routing fixture, and 20/27 vs 11-17/27 on the +talk fixture. See `MODEL-BAKEOFF-31-07-2026.md`. It is a Thinking variant, so `n_ctx` is 4096 +— reasoning tokens need the room, and 4096 is what the scores above were measured at. + +The **target** is still the locally CPT'd **Qwen3-1.7B** (Vikunja #122, training in flight). +Stock already speaks good Russian; what it gets wrong is the persona — it writes `я рад`, +masculine, where Maven needs `рада`. That is what the CPT is for. + +**Do not bother with sub-500M models.** LFM2.5-230M and 350M were measured on 2026-07-31 and +both are unusable in Russian: the 350M routes at 5.2% (worse than guessing) and answers +"столица Франции?" with the invented non-word "Сторзит"; the 230M replies to Russian in +Spanish. Their strong published IFEval/BFCL numbers are English-only. Model files live in `/mnt/hdd1/llms`, bind-mounted to `/opt/maven/models/llm` — which **shadows** the repo's `models/llm/`, so the LFM2.5 gguf sitting there is not loaded by anything. Swapping the resident model is a one-line change to `phraser.model_path` in `deploy/mavend.json`. diff --git a/deploy/mavend.json b/deploy/mavend.json index 2c684b4..962e47d 100644 --- a/deploy/mavend.json +++ b/deploy/mavend.json @@ -6,10 +6,10 @@ "state_dir": "/var/lib/maven", "phraser": { - "model_path": "/opt/maven/models/llm/qwen3.5/Qwen3.5-0.8B.Q4_K_M.gguf", + "model_path": "/opt/maven/models/llm/qwen3/Qwen3-1.7B-UD-Q4_K_XL.gguf", "bin_path": "llama-server", "n_gpu_layers": 99, - "n_ctx": 2048, + "n_ctx": 4096, "timeout": "60s", "llm_nudges": false },