diff --git a/AGENTS.md b/AGENTS.md index d9a072e..24de599 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -78,8 +78,10 @@ Russian recall — you may see many "clarify" responses). The target daemon uses the locally trained Qwen3-1.7B checkpoint for both routing and phrasing. Training is Qwen3 Base → RU CPT → joint persona/router -SFT → merged GGUF. Without a configured model, `StubPhraser` plus the classifier -remain the deterministic floor. +SFT → merged GGUF, and is still in flight (#122) — until it lands, the deployed +resident model is stock **Qwen3.5-0.8B** (`Q4_K_M`), see `deploy/mavend.json`. +Without a configured model, `StubPhraser` plus the classifier remain the +deterministic floor. During training, use the runbook in `docs/plans/2026-07-18-qwen3-resident-training-eval.md`. After the decision gate diff --git a/CLAUDE.md b/CLAUDE.md index 55a1a40..ba7e6c5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -3,9 +3,19 @@ This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. Maven is a self-hosted, privacy-first voice assistant (Russian + English). Go daemons -talking over unix sockets; one resident Qwen3-1.7B for routing + phrasing; whisper.cpp STT, piper TTS. -Deploy target is a CPU-only Ryzen laptop (homesrv) — the resident model stays at 1.7B. See `REARCH.md` -for the target architecture and `AGENTS.md` for local-preview + model-download recipes. +talking over unix sockets; one resident small model for routing + phrasing; whisper.cpp STT, piper TTS. +Deploy target is a Ryzen laptop (homesrv) with Vulkan offload to the Vega iGPU (`n_gpu_layers: 99`, +compose passes `/dev/dri` + the render gid) — the resident model stays ≤1.7B either way. + +**Resident model:** currently **Qwen3.5-0.8B** (`Q4_K_M`), the smallest checkpoint in the gguf +library, picked for CPU/iGPU latency. The **target** is the locally CPT'd **Qwen3-1.7B**; that +training is still in flight (Vikunja #122), so no such gguf exists yet. Model files live in +`/mnt/hdd1/llms`, bind-mounted to `/opt/maven/models/llm` — which **shadows** the repo's +`models/llm/`, so the LFM2.5 gguf sitting there is not loaded by anything. Swapping the resident +model is a one-line change to `phraser.model_path` in `deploy/mavend.json`. + +See `REARCH.md` for the target architecture, `DESIGN.md` for the folded design spec, and +`AGENTS.md` for local-preview + model-download recipes. ## Build & test diff --git a/MAVEN_ECOSYSTEM_ARCHITECTURE.md b/MAVEN_ECOSYSTEM_ARCHITECTURE.md index a3f7871..0d4a60a 100644 --- a/MAVEN_ECOSYSTEM_ARCHITECTURE.md +++ b/MAVEN_ECOSYSTEM_ARCHITECTURE.md @@ -69,7 +69,8 @@ Maven remains a set of Go daemons connected through Unix sockets. | `mavpoll` | Telegram reach | | `mavcaldav` | CalDAV synchronization | -The resident Qwen3-1.7B model remains bounded to: +The resident model (Qwen3.5-0.8B now, CPT'd Qwen3-1.7B as the target — #122) +remains bounded to: - structured routing - concise natural-language phrasing diff --git a/PROGRESS.md b/PROGRESS.md index 41770d6..f212539 100644 --- a/PROGRESS.md +++ b/PROGRESS.md @@ -30,7 +30,9 @@ tests, `-race` in `make test`. ### Works end-to-end (tested) - **Reactive voice:** PWA record → Whisper STT (`mavsttd`) → ONNX classifier → - LFM 2.5-1.2B phraser (llama-server subprocess) → Piper TTS (`mavttsd`) → reply. + resident phraser (llama-server subprocess; Qwen3.5-0.8B as of 2026-07-30 — + this line historically named "LFM 2.5-1.2B") → Piper TTS + (`mavttsd`) → reply. HTTP POST path (mobile-Chrome drops WS for the audio). - **Capture:** `fact` (EN **and RU** — root-substring recognizers) + `reminder` persist through CoreAPI (`source=tap:voice`). This is the substrate the care diff --git a/cmd/mavend/replier_llm.go b/cmd/mavend/replier_llm.go index 3bd8b20..bbba111 100644 --- a/cmd/mavend/replier_llm.go +++ b/cmd/mavend/replier_llm.go @@ -17,7 +17,8 @@ type completer interface { Complete(ctx context.Context, r llm.Req) (string, error) } -// llmReplier phrases reactive confirmations with the resident LFM. Stub is the +// llmReplier phrases reactive confirmations with the resident model +// (Qwen3-1.7B). Stub is the // floor on any error (offline-safe). Maven speaks as "she", feminine RU. type llmReplier struct { c completer diff --git a/deploy/mavend.json b/deploy/mavend.json index 7c604a0..9a19173 100644 --- a/deploy/mavend.json +++ b/deploy/mavend.json @@ -6,7 +6,7 @@ "state_dir": "/var/lib/maven", "phraser": { - "model_path": "/opt/maven/models/llm/qwen3.5/Qwen3.5-2B-UD-Q4_K_XL.gguf", + "model_path": "/opt/maven/models/llm/qwen3.5/Qwen3.5-0.8B.Q4_K_M.gguf", "bin_path": "llama-server", "n_gpu_layers": 99, "n_ctx": 2048, diff --git a/internal/delivery/dispatcher.go b/internal/delivery/dispatcher.go index 053a77e..93fc47d 100644 --- a/internal/delivery/dispatcher.go +++ b/internal/delivery/dispatcher.go @@ -81,8 +81,8 @@ func (d *Dispatcher) completeOutbox(ctx context.Context, id int64, status string } } -// PhrasedNudge — the phraser module's output for a nudge. the phraser (LFM -// sub-1b, prompted not trained) takes (rule, severity, context) and produces +// PhrasedNudge — the phraser module's output for a nudge. the phraser (the +// resident model, Qwen3-1.7B) takes (rule, severity, context) and produces // Body (full message for voice) + Summary (minimal body for away channels). // the phraser is a separate module; the dispatcher only consumes its output. type PhrasedNudge struct { diff --git a/internal/loop/loop.go b/internal/loop/loop.go index 9b8dbee..bc884fe 100644 --- a/internal/loop/loop.go +++ b/internal/loop/loop.go @@ -75,7 +75,8 @@ type Candidate struct { // Returns nil when nothing fires ("shuts up" is the default outcome of a tick). // // Phrasing + sending happen OUT of the loop — the daemon hands Candidate to -// the phraser (LFM) and delivery module. The loop just decides. +// the phraser (the resident model, Qwen3-1.7B) and delivery module. The loop +// just decides. // // Reminders are NOT handled here — they're a separate, gate-bypassing class. // See DueReminders (gathered separately) and RemindDecisions (the loop output diff --git a/internal/phraser/phraser.go b/internal/phraser/phraser.go index 05ba026..cd25abe 100644 --- a/internal/phraser/phraser.go +++ b/internal/phraser/phraser.go @@ -1,7 +1,10 @@ // Package phraser is maven's "rules decide, llm phrases" seam — the layer // that turns a loop decision into the body + summary the delivery module ships. // -// Per the spec: the phraser (LFM sub-1b, prompted not trained) takes +// Per DESIGN.md § Resident language model: the phraser is the resident model +// (Qwen3-1.7B — RU continued pretraining plus joint persona/router SFT, not a +// sub-1b prompted-only model as the retired spec claimed; see DESIGN.md +// § Superseded, "small-model phrasing claim"). It takes // (rule, severity, context) and produces Body (full voice message, local — no // shoulder-surf concern beyond who's in the room) + Summary (minimal body for // away channels — "disk low on homesrv," not detail; no exfil through the diff --git a/internal/router/router.go b/internal/router/router.go index 2fc5944..349534d 100644 --- a/internal/router/router.go +++ b/internal/router/router.go @@ -24,8 +24,9 @@ type Config struct { Threshold float64 // LLM — optional agentic router. When set, Route consults it after stage-0 // and before the classifier cascade, classifying the utterance via a - // grammar-constrained LFM call. On any error/parse failure, falls through - // to the classifier (never fails the turn on the model). + // grammar-constrained call to the resident model (Qwen3-1.7B). On any + // error/parse failure, falls through to the classifier (never fails the + // turn on the model). LLM *LLMRouter }