kami
f0f7ebc9b2
Confidence was hardcoded to 1.0 for every LLM decision, and the LLM branch
in Router.Route returned straight from fillSlots without ever touching the
stage-3 threshold gate — so the LLM path could not produce a Clarify no
matter what confidence a model reported. That is why all 6 want_clarify
cases in the 77-case RU fixture were missed by every model in the bake-off.
Fix reads structural signal instead of changing the (parity-locked) router
prompt: a single-token utterance ("вода", "бэкап") is flagged thin evidence
in llmrouter.go; a fact left keyless or an act that never resolves to an
allowlisted fn, checked after fillSlots so the deterministic parsers get
first crack, is flagged in router.go's new gateLLMDecision. Anything below
config.DefaultRouterThreshold (0.55) now sets Clarify=true through the same
path the classifier already uses.
Added unit tests with a stubbed Completer proving both directions: thin
cases clarify, clean multi-word/resolved-slot cases stay confident. The
77-case fixture re-run against a live llama-server is still needed to
confirm the 6/6 moves — not done here, no llama-server on this box.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CGeSZxh1DCtRxmFVSYVGvJ
142 lines
8.1 KiB
Markdown
142 lines
8.1 KiB
Markdown
# CLAUDE.md
|
||
|
||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||
|
||
Maven is a self-hosted, privacy-first voice assistant (Russian + English). Go daemons
|
||
talking over unix sockets; one resident small model for routing + phrasing; whisper.cpp STT, piper TTS.
|
||
Deploy target is a Ryzen laptop (homesrv) with Vulkan offload to the Vega iGPU (`n_gpu_layers: 99`,
|
||
compose passes `/dev/dri` + the render gid) — the resident model stays ≤1.7B either way.
|
||
|
||
**Resident model:** currently **Qwen3-1.7B** (`UD-Q4_K_XL`), stock — not yet the CPT'd one.
|
||
It replaced Qwen3.5-0.8B on 2026-07-31 because it measured better on both fixtures we have:
|
||
67.5% vs 59.7% intent-only on the 77-case RU routing fixture, and 20/27 vs 11-17/27 on the
|
||
talk fixture. See `MODEL-BAKEOFF-31-07-2026.md`. It is a Thinking variant, so `n_ctx` is 4096
|
||
— reasoning tokens need the room, and 4096 is what the scores above were measured at.
|
||
|
||
The **target** is still the locally CPT'd **Qwen3-1.7B** (Vikunja #122, training in flight).
|
||
Stock already speaks good Russian; what it gets wrong is the persona — it writes `я рад`,
|
||
masculine, where Maven needs `рада`. That is what the CPT is for.
|
||
|
||
**Do not bother with sub-500M models.** LFM2.5-230M and 350M were measured on 2026-07-31 and
|
||
both are unusable in Russian: the 350M routes at 5.2% (worse than guessing) and answers
|
||
"столица Франции?" with the invented non-word "Сторзит"; the 230M replies to Russian in
|
||
Spanish. Their strong published IFEval/BFCL numbers are English-only. Model files live in
|
||
`/mnt/hdd1/llms`, bind-mounted to `/opt/maven/models/llm` — which **shadows** the repo's
|
||
`models/llm/`, so the LFM2.5 gguf sitting there is not loaded by anything. Swapping the resident
|
||
model is a one-line change to `phraser.model_path` in `deploy/mavend.json`.
|
||
|
||
See `REARCH.md` for the target architecture, `DESIGN.md` for the folded design spec, and
|
||
`AGENTS.md` for local-preview + model-download recipes.
|
||
|
||
## Build & test
|
||
|
||
CGO daemons (`mavend`, `mavsttd`, `mavttsd`, `mavenclient`) need the vendored toolchain
|
||
and libs wired through the Makefile — **do not** call `go build` on them bare, use `make`:
|
||
|
||
```sh
|
||
make build # all 8 binaries
|
||
make build-web # single daemon (pure-Go ones: web/waked/poll/caldav build without CGO)
|
||
make test # go test -race across ./internal/... ./cmd/... with CGO env set
|
||
```
|
||
|
||
Run a single test (must carry the CGO env for packages that touch STT/TTS/voice):
|
||
|
||
```sh
|
||
CGO_CFLAGS="-I$(pwd)/deps/include -I$(pwd)/deps/whisper.cpp/ggml/include" \
|
||
CGO_LDFLAGS="-L$(pwd)/deps/lib -Wl,-rpath,$(pwd)/deps/lib" \
|
||
LD_LIBRARY_PATH="$(pwd)/deps/lib" \
|
||
deps/go/go/bin/go test -run TestName ./internal/router/
|
||
```
|
||
|
||
Pure-Go packages (`router`, `memory`, `mavweb`, …) run under a plain `go test ./pkg/`.
|
||
|
||
## The daemons (`cmd/`)
|
||
|
||
| Binary | Role |
|
||
|---|---|
|
||
| `mavend` | **Core.** Router, phraser, memory, reminders, digestion tick. Owns the DB + IPC socket. |
|
||
| `mavweb` | HTTP UI + PWA (`/dash`, `/history`, `/trace`, `/notifications`, `/tools`); WebAuthn auth. Connects to mavend's socket. |
|
||
| `mavsttd` | Speech-to-text (whisper.cpp, CGO). |
|
||
| `mavttsd` | Text-to-speech (piper subprocess). |
|
||
| `mavwaked` | Wake-word / VAD gate. |
|
||
| `mavenclient` | Voice loop client (mic → stt → core → tts). |
|
||
| `mavpoll` | Telegram long-poll reach. |
|
||
| `mavcaldav` | CalDAV calendar sync. |
|
||
|
||
Daemons are wired socket-to-socket, not linked. `internal/ipc` is the client/server wire
|
||
protocol; the config in `deploy/mavend.json` (with `${VAR}` env expansion from gitignored
|
||
`deploy/telegram.env`) sets socket paths, model paths, and the phraser/embedder blocks.
|
||
|
||
## Routing — read this before touching the router
|
||
|
||
`internal/router/` has TWO layered engines. **The LLM router is now the default and it is
|
||
on in deploy** — this section used to say it was wired `nil`, which stopped being true on
|
||
2026-07-31.
|
||
|
||
- **LLM router (the intended design, REARCH.md):** the resident Qwen3-1.7B (`llmrouter.go`)
|
||
emits GBNF-constrained structured JSON, and the SAME model phrases replies. Embedder is
|
||
demoted from a routing gate to a RAG hint. Wired at `voice.go:214` via
|
||
`pickLLMRouter(cfg.Voice.UseLLMRouter(), llmClient)`; the flag is `voice.llm_router`
|
||
(`config.go`), `DefaultLLMRouter` is **on**, and `deploy/mavend.json` sets it `true`.
|
||
- **Classifier cascade (the failure floor, not dead code):** `classifier.go` +
|
||
`embedder.go` nearest-neighbour over frozen seed phrases. It runs when the LLM router is
|
||
off, when there is no llama-server to talk to (`pickLLMRouter` logs that and degrades),
|
||
and on any per-turn LLM error. Do not delete it — routing by seed similarity is the known
|
||
cause of weak RU query handling, but a turn must never break on the model.
|
||
|
||
Cascade order: `stage0.go` exact-match fast-path → LLM router (when non-nil) → classifier
|
||
fallback. Any LLM error falls through to the classifier so a turn never breaks on the model.
|
||
|
||
Measured on the 77-case RU fixture (`MODEL-BAKEOFF-31-07-2026.md`): the classifier scores
|
||
36.8% full accuracy at p50 31ms; Qwen3-1.7B scores 67.5% intent-only / 72.7% through the
|
||
cascade at p50 ≈2.7s. Accuracy roughly doubled, latency is ~90× worse, and that trade was
|
||
accepted deliberately. `Confidence: 1.0` used to be hardcoded in `llmrouter.go`, so the LLM
|
||
path could never ask for clarification (6/6 refusal cases missed on the fixture) — Vikunja
|
||
#359. Fixed 31-07-2026 with structural signal (single-token utterance, keyless fact, act with
|
||
no allowlisted fn) feeding the same stage-3 gate the classifier path already had; the fixture
|
||
re-run to confirm the 6/6 moves is still outstanding, see `gateLLMDecision` in `router.go`.
|
||
|
||
## LLM output contract
|
||
|
||
All phrasing paths emit `{"response":"...","mood":"..."}` (parsed in `replier_llm.go` and
|
||
`internal/phraser/llmphraser.go`), with fallback to plain text and the legacy
|
||
`{"body","summary"}`. Mood is a fixed enum. Router prompt is a separate contract:
|
||
`[{"intent":<enum>, key?, value?, text?, verb?}, ...]`, 7 intents (`fact, reminder,
|
||
note, query, act, chat, system`). `llm/check_prompt_parity.py` in the training
|
||
workspace enforces that the Go and relabelling prompts remain identical.
|
||
|
||
## Non-goals (hard constraints)
|
||
|
||
Not a nag, not autonomous. Maven's persona is **feminine** — Russian
|
||
self-reference must use feminine forms — `рада`, not `рад`; `поняла`, not `понял`. The owner
|
||
is male and is addressed informally: "ты", singular, never "вы"/"ваш" and never "он"/"его"
|
||
(she talks TO him, not about him). Pet names ("милый", "дорогой") are forbidden; his name
|
||
("Ками") is not. The eval enforces this: `CheckAddress`, `CheckFeminine` and `CheckCringe` in
|
||
`internal/phraser/eval/checks.go`, scored by `make eval-phrasing`.
|
||
|
||
**"Never phones home" is DEPRECATED** (owner's call, 2026-07-31). It used to be a hard
|
||
constraint and it is not one any more: a 0.8B — and a 1.7B — does not know enough to answer
|
||
world questions, so she needs to read external sources. What replaces it:
|
||
|
||
- **No telemetry, no cloud model, no third-party account.** That part never changes. Nothing
|
||
about Maven is reported to anyone, and inference stays on the box.
|
||
- **Local sources first.** Kiwix ZIMs on homesrv (Wikipedia, ifixit) before anything on the
|
||
network. Reading beats recalling for a small model, and a local read costs nothing.
|
||
- **External search is allowed and off unless configured**, like the weather and telegram
|
||
capabilities.
|
||
- **His notes and facts are never search input.** Looking up why the sky is blue and sending
|
||
his stored personal notes to an upstream engine are different acts. Only the utterance goes
|
||
out, never the persona block, history, or matched notes.
|
||
|
||
## Web UI conventions
|
||
|
||
Server-rendered pages share `cmd/mavweb/static/ui.css` (served at `/ui.css`) and the `nav`
|
||
partial (`navHTML` in `cmd/mavweb/main.go`, `{{template "nav" "<active-page>"}}`). No
|
||
per-page `<style>` beyond true one-offs. Wrap every table in `<div class=scroll>` so wide
|
||
data pans on a phone. Local preview + headless screenshot recipe is in `AGENTS.md`.
|
||
|
||
## Vikunja
|
||
|
||
This repo is project **Maven** (ID 2) in Vikunja. MCP: `http://localhost:9100/mcp` (or
|
||
`http://192.168.1.104:9100/mcp` from workpc). Feature/bug/deploy tasks go there.
|