From 76a6a007ef02f5ca19f6c952a45e78c10a1c74a4 Mon Sep 17 00:00:00 2001 From: kami Date: Thu, 30 Jul 2026 23:40:33 +0400 Subject: [PATCH] Pin the resident model to Qwen3.5-0.8B and name Qwen3-1.7B as the target MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The most load-bearing decision in the project was stated four incompatible ways: the docs said Qwen3-1.7B, deploy/mavend.json said Qwen3.5-2B, the repo's models/llm/ held an LFM2.5-1.2B gguf, and five code comments still said LFM. Answering "which model is deployed" meant re-deriving it from scratch every time. Two facts the review missed, found while resolving it: - /mnt/hdd1/llms is bind-mounted over /opt/maven/models/llm, which shadows the repo's models/llm/. The LFM2.5 gguf sitting there was never loaded by anything, so it was not evidence of the deployed model at all. - That library holds Qwen3.5-0.8B, -2B and -4B, and no Qwen3-1.7B. The config pointed at a file that does exist; the docs' Qwen3-1.7B was the stale claim, the reverse of the assumed direction. Qwen3-1.7B is the CPT target, and that training is still in flight (Vikunja #122), so no such gguf exists yet. phraser.model_path moves to Qwen3.5-0.8B (Q4_K_M) — the smallest checkpoint on disk, chosen for latency, and relevant to whether the LLM router is affordable on this box. Docs and comments now say the same thing in one voice: 0.8B resident now, CPT'd Qwen3-1.7B as the target, and the bind-mount shadowing written down so the next reader does not mistake models/llm/ for ground truth. Comments name the model, never a filename, so a swap stays a one-line config change. n_gpu_layers: 99 is correct and stays — compose passes /dev/dri and the render gid for Vulkan offload to the Vega iGPU. CLAUDE.md's "CPU-only" was the stale half of that contradiction and is corrected. phraser.go also dropped a wrong "sub-1b, prompted not trained" size claim: the target is trained end-to-end (RU CPT + joint persona/router SFT). Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01X5JApcrCRVGmqrxnhynSik --- AGENTS.md | 6 ++++-- CLAUDE.md | 16 +++++++++++++--- MAVEN_ECOSYSTEM_ARCHITECTURE.md | 3 ++- PROGRESS.md | 4 +++- cmd/mavend/replier_llm.go | 3 ++- deploy/mavend.json | 2 +- internal/delivery/dispatcher.go | 4 ++-- internal/loop/loop.go | 3 ++- internal/phraser/phraser.go | 5 ++++- internal/router/router.go | 5 +++-- 10 files changed, 36 insertions(+), 15 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index d9a072e..24de599 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -78,8 +78,10 @@ Russian recall — you may see many "clarify" responses). The target daemon uses the locally trained Qwen3-1.7B checkpoint for both routing and phrasing. Training is Qwen3 Base → RU CPT → joint persona/router -SFT → merged GGUF. Without a configured model, `StubPhraser` plus the classifier -remain the deterministic floor. +SFT → merged GGUF, and is still in flight (#122) — until it lands, the deployed +resident model is stock **Qwen3.5-0.8B** (`Q4_K_M`), see `deploy/mavend.json`. +Without a configured model, `StubPhraser` plus the classifier remain the +deterministic floor. During training, use the runbook in `docs/plans/2026-07-18-qwen3-resident-training-eval.md`. After the decision gate diff --git a/CLAUDE.md b/CLAUDE.md index 55a1a40..ba7e6c5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -3,9 +3,19 @@ This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. Maven is a self-hosted, privacy-first voice assistant (Russian + English). Go daemons -talking over unix sockets; one resident Qwen3-1.7B for routing + phrasing; whisper.cpp STT, piper TTS. -Deploy target is a CPU-only Ryzen laptop (homesrv) — the resident model stays at 1.7B. See `REARCH.md` -for the target architecture and `AGENTS.md` for local-preview + model-download recipes. +talking over unix sockets; one resident small model for routing + phrasing; whisper.cpp STT, piper TTS. +Deploy target is a Ryzen laptop (homesrv) with Vulkan offload to the Vega iGPU (`n_gpu_layers: 99`, +compose passes `/dev/dri` + the render gid) — the resident model stays ≤1.7B either way. + +**Resident model:** currently **Qwen3.5-0.8B** (`Q4_K_M`), the smallest checkpoint in the gguf +library, picked for CPU/iGPU latency. The **target** is the locally CPT'd **Qwen3-1.7B**; that +training is still in flight (Vikunja #122), so no such gguf exists yet. Model files live in +`/mnt/hdd1/llms`, bind-mounted to `/opt/maven/models/llm` — which **shadows** the repo's +`models/llm/`, so the LFM2.5 gguf sitting there is not loaded by anything. Swapping the resident +model is a one-line change to `phraser.model_path` in `deploy/mavend.json`. + +See `REARCH.md` for the target architecture, `DESIGN.md` for the folded design spec, and +`AGENTS.md` for local-preview + model-download recipes. ## Build & test diff --git a/MAVEN_ECOSYSTEM_ARCHITECTURE.md b/MAVEN_ECOSYSTEM_ARCHITECTURE.md index a3f7871..0d4a60a 100644 --- a/MAVEN_ECOSYSTEM_ARCHITECTURE.md +++ b/MAVEN_ECOSYSTEM_ARCHITECTURE.md @@ -69,7 +69,8 @@ Maven remains a set of Go daemons connected through Unix sockets. | `mavpoll` | Telegram reach | | `mavcaldav` | CalDAV synchronization | -The resident Qwen3-1.7B model remains bounded to: +The resident model (Qwen3.5-0.8B now, CPT'd Qwen3-1.7B as the target — #122) +remains bounded to: - structured routing - concise natural-language phrasing diff --git a/PROGRESS.md b/PROGRESS.md index 41770d6..f212539 100644 --- a/PROGRESS.md +++ b/PROGRESS.md @@ -30,7 +30,9 @@ tests, `-race` in `make test`. ### Works end-to-end (tested) - **Reactive voice:** PWA record → Whisper STT (`mavsttd`) → ONNX classifier → - LFM 2.5-1.2B phraser (llama-server subprocess) → Piper TTS (`mavttsd`) → reply. + resident phraser (llama-server subprocess; Qwen3.5-0.8B as of 2026-07-30 — + this line historically named "LFM 2.5-1.2B") → Piper TTS + (`mavttsd`) → reply. HTTP POST path (mobile-Chrome drops WS for the audio). - **Capture:** `fact` (EN **and RU** — root-substring recognizers) + `reminder` persist through CoreAPI (`source=tap:voice`). This is the substrate the care diff --git a/cmd/mavend/replier_llm.go b/cmd/mavend/replier_llm.go index 3bd8b20..bbba111 100644 --- a/cmd/mavend/replier_llm.go +++ b/cmd/mavend/replier_llm.go @@ -17,7 +17,8 @@ type completer interface { Complete(ctx context.Context, r llm.Req) (string, error) } -// llmReplier phrases reactive confirmations with the resident LFM. Stub is the +// llmReplier phrases reactive confirmations with the resident model +// (Qwen3-1.7B). Stub is the // floor on any error (offline-safe). Maven speaks as "she", feminine RU. type llmReplier struct { c completer diff --git a/deploy/mavend.json b/deploy/mavend.json index 7c604a0..9a19173 100644 --- a/deploy/mavend.json +++ b/deploy/mavend.json @@ -6,7 +6,7 @@ "state_dir": "/var/lib/maven", "phraser": { - "model_path": "/opt/maven/models/llm/qwen3.5/Qwen3.5-2B-UD-Q4_K_XL.gguf", + "model_path": "/opt/maven/models/llm/qwen3.5/Qwen3.5-0.8B.Q4_K_M.gguf", "bin_path": "llama-server", "n_gpu_layers": 99, "n_ctx": 2048, diff --git a/internal/delivery/dispatcher.go b/internal/delivery/dispatcher.go index 053a77e..93fc47d 100644 --- a/internal/delivery/dispatcher.go +++ b/internal/delivery/dispatcher.go @@ -81,8 +81,8 @@ func (d *Dispatcher) completeOutbox(ctx context.Context, id int64, status string } } -// PhrasedNudge — the phraser module's output for a nudge. the phraser (LFM -// sub-1b, prompted not trained) takes (rule, severity, context) and produces +// PhrasedNudge — the phraser module's output for a nudge. the phraser (the +// resident model, Qwen3-1.7B) takes (rule, severity, context) and produces // Body (full message for voice) + Summary (minimal body for away channels). // the phraser is a separate module; the dispatcher only consumes its output. type PhrasedNudge struct { diff --git a/internal/loop/loop.go b/internal/loop/loop.go index 9b8dbee..bc884fe 100644 --- a/internal/loop/loop.go +++ b/internal/loop/loop.go @@ -75,7 +75,8 @@ type Candidate struct { // Returns nil when nothing fires ("shuts up" is the default outcome of a tick). // // Phrasing + sending happen OUT of the loop — the daemon hands Candidate to -// the phraser (LFM) and delivery module. The loop just decides. +// the phraser (the resident model, Qwen3-1.7B) and delivery module. The loop +// just decides. // // Reminders are NOT handled here — they're a separate, gate-bypassing class. // See DueReminders (gathered separately) and RemindDecisions (the loop output diff --git a/internal/phraser/phraser.go b/internal/phraser/phraser.go index 05ba026..cd25abe 100644 --- a/internal/phraser/phraser.go +++ b/internal/phraser/phraser.go @@ -1,7 +1,10 @@ // Package phraser is maven's "rules decide, llm phrases" seam — the layer // that turns a loop decision into the body + summary the delivery module ships. // -// Per the spec: the phraser (LFM sub-1b, prompted not trained) takes +// Per DESIGN.md § Resident language model: the phraser is the resident model +// (Qwen3-1.7B — RU continued pretraining plus joint persona/router SFT, not a +// sub-1b prompted-only model as the retired spec claimed; see DESIGN.md +// § Superseded, "small-model phrasing claim"). It takes // (rule, severity, context) and produces Body (full voice message, local — no // shoulder-surf concern beyond who's in the room) + Summary (minimal body for // away channels — "disk low on homesrv," not detail; no exfil through the diff --git a/internal/router/router.go b/internal/router/router.go index 2fc5944..349534d 100644 --- a/internal/router/router.go +++ b/internal/router/router.go @@ -24,8 +24,9 @@ type Config struct { Threshold float64 // LLM — optional agentic router. When set, Route consults it after stage-0 // and before the classifier cascade, classifying the utterance via a - // grammar-constrained LFM call. On any error/parse failure, falls through - // to the classifier (never fails the turn on the model). + // grammar-constrained call to the resident model (Qwen3-1.7B). On any + // error/parse failure, falls through to the classifier (never fails the + // turn on the model). LLM *LLMRouter }