claude
8c30971a96
The lazy-connect note is no longer true and the trap it described was the opposite way round: the session existed and the audio was discarded. diff-budget.sh blocks the branch at 615 changed lines. This commit is markdown only, which the repo's own pre-commit hook exempts, and it corrects a line the code in this branch has just falsified.
197 lines
9.8 KiB
Markdown
197 lines
9.8 KiB
Markdown
# Deployment: the boxes, the models, the daemons
|
|
|
|
*Last verified: 2026-08-09 @ a9b480a*
|
|
|
|
What runs where, and why each choice was made. `CLAUDE.md` carries only the
|
|
rules. This file carries the reasoning.
|
|
|
|
## The two boxes
|
|
|
|
**homesrv** is a Ryzen 5 5600U laptop and the deploy target. It offloads to the
|
|
Vega iGPU over Vulkan (`n_gpu_layers: 99`). Compose passes `/dev/dri` and the
|
|
render gid (993), and without both Vulkan enumerates zero devices and
|
|
llama-server falls back to CPU silently.
|
|
|
|
**workpc** is the workstation, 16GB of VRAM, reached as `kami@workpc` at
|
|
192.168.1.105. Model work moved there on 2026-08-02 by the owner's call, because
|
|
homesrv cannot grow a GPU.
|
|
|
|
Three rules govern the seam:
|
|
|
|
- **The workstation is never assumed up.**
|
|
- **Fall back silently** when it would only do the job better.
|
|
- **Name the gap** when the resident model cannot do the job at all.
|
|
|
|
A world question goes through `LLMPhraser.PhraseWorld` and returns `worldGap`
|
|
(`cmd/mavend/worldmodel.go`) rather than an invented answer. A box with no
|
|
`workstation` block behaves exactly as it did before the seam. `docs/offload.md`
|
|
says which caller is which.
|
|
|
|
## The resident model
|
|
|
|
**Qwen3-1.7B** (`UD-Q4_K_XL`), stock, not yet the CPT'd one. It is a Thinking
|
|
variant, so `n_ctx` is 4096. Reasoning tokens need the room, and 4096 is what
|
|
every score was measured at.
|
|
|
|
The target is the locally CPT'd Qwen3-1.7B (V-122, training in flight). Stock
|
|
already speaks good Russian. What it gets wrong is the persona. It writes `я рад`
|
|
where Maven needs `рада`.
|
|
|
|
**Do not bother with sub-500M models.** LFM2.5-230M and 350M were measured on
|
|
2026-07-31 and both are unusable in Russian
|
|
(`docs/evals/2026-07-31-model-bakeoff.md`). Their published IFEval and BFCL
|
|
numbers are English-only.
|
|
|
|
Model files live in `/mnt/hdd1/llms`, bind-mounted to `/opt/maven/models/llm`.
|
|
That **shadows** the repo's `models/llm/`, so a gguf sitting there is not loaded
|
|
by anything. Swapping the resident model is a one-line change to
|
|
`phraser.model_path` in `deploy/mavend.json`.
|
|
|
|
The workstation model is gemma-4-E4B as of 2026-08-09, replacing the 12B by the
|
|
owner's call. Keep the 12B gguf. It is the better teacher for label runs, at
|
|
72.7% destination against E4B's 57.6%.
|
|
|
|
## The embedder
|
|
|
|
**It stays on homesrv permanently**, because it backs the floor. It is
|
|
multilingual-e5-small, quantized and asymmetric. `EmbedQuery` and `EmbedPassage`
|
|
apply the `query:` and `passage:` prefixes it was trained with. Calling plain
|
|
`Embed` on a note is a bug. See `docs/evals/2026-08-04-recall-e5-small.md`.
|
|
|
|
The vendored onnxruntime under `deps/` has two copies, and the stale one is
|
|
1.17.1. The live runtime is 1.26.0, and the Go binding asks for API 26. Anything
|
|
shipped to another box needs `deps/onnxruntime-linux-x64-1.26.0`.
|
|
|
|
## Speech-to-text
|
|
|
|
`sttSeam` in `cmd/mavend/voicewire.go` builds an `stt.Pair` beside `modelSeam`.
|
|
It prefers CrisperWhisper 2.0 turbo on workpc with mavsttd as the floor. It takes
|
|
only the silent half of the rule, because a worse transcript is still a turn. So
|
|
`stt.Pair` has no `TranscribeRemote` and the fallback is never spoken.
|
|
|
|
CW2 turbo scores 10.4% WER in Russian against 27.5% for the `ggml-small.bin`
|
|
mavsttd loads, over 200 Golos clips
|
|
(`docs/evals/2026-08-09-crisperwhisper2-russian-wer.md`).
|
|
|
|
**whisper.cpp cannot load CW2 at all.** It reads its language count off the
|
|
vocabulary size. CW2's 51897 tokens shift seven special token ids. So CW2 is its
|
|
own transformers service on port 8081 (`deploy/cw2/serve.py`).
|
|
`stt.HTTPTranscriber` posts raw PCM to it with a bearer token, because audio is
|
|
the most sensitive thing that crosses this seam. The switch is `workstation.stt`
|
|
in `deploy/mavend.json`, and deleting the block sends every utterance to mavsttd.
|
|
|
|
**mavgpud runs that service as a second child.** This is not an optimisation.
|
|
CW2 is a ROCm process on the same card, so it registers on the KFD like any
|
|
contender. Under its own systemd unit it made mavgpud evict llama-server every
|
|
few seconds. That took the model arm down for eight minutes on 2026-08-09. The
|
|
card needs one owner. CW2 is on the yield clock and not the idle one. At 1.6GB
|
|
it denies the card to nobody.
|
|
|
|
Text-to-speech has not moved. piper on homesrv is the only synthesizer.
|
|
|
|
## The daemons
|
|
|
|
| Binary | Role |
|
|
|---|---|
|
|
| `mavend` | **Core.** Router, phraser, memory, reminders, digestion tick. Owns the DB and IPC socket. |
|
|
| `mavweb` | HTTP UI and PWA (`/dash`, `/history`, `/trace`, `/notifications`, `/tools`), WebAuthn auth. |
|
|
| `mavsttd` | Speech-to-text (whisper.cpp, CGO). |
|
|
| `mavttsd` | Text-to-speech (piper subprocess). |
|
|
| `mavwaked` | Wake-word and VAD gate. Runs on workpc. |
|
|
| `mavenclient` | Voice loop client (mic, stt, core, tts). Not deployed. |
|
|
| `mavpoll` | Environment poller: netdata alarms, uptime-kuma, zenmoney, wireguard presence. Writes facts, sends nothing. Telegram is `internal/delivery/telegramsink`. |
|
|
| `mavcaldav` | CalDAV calendar sync. |
|
|
| `mavmaild` | Mail reader (IMAP, read-only). Holds the IMAP password, core never sees it. |
|
|
| `mavgpud` | GPU supervisor. **Runs on workpc**, own unit `deploy/mavgpud.service`. Keeps llama-server loaded while the card is free (V-488). Maven never asks it for anything and reads `/health` through `llm.Pair`. |
|
|
| `mavupdate` | Not a daemon. Operator CLI a human runs on the box to deploy a new build. |
|
|
|
|
Two binaries have no Makefile target and neither is deployed. `mavseal` encrypts
|
|
a live tmpfs working copy back to the ciphertext file when mavend was killed
|
|
before `defer st.Close()` sealed it. `labelgen` runs the stage 0 grammars over
|
|
utterances and prints JSONL, the training data for the routing heads.
|
|
|
|
Daemons are wired socket-to-socket, not linked. `internal/ipc` is the wire
|
|
protocol. `deploy/mavend.json` sets socket paths, model paths and the phraser and
|
|
embedder blocks, with `${VAR}` expansion from gitignored `deploy/telegram.env`.
|
|
|
|
### Who is in compose, and who is not
|
|
|
|
**`docker-compose.yml` runs five**: `mavend`, `mavsttd`, `mavttsd`, `mavweb`,
|
|
`mavpoll`. Count against compose, not against the table above.
|
|
|
|
`mavmaild` and `mavcaldav` are commented out, each with the reason beside it. The
|
|
first needs a mail account and the second a CalDAV account, and this box has
|
|
neither. Two things ride on the CalDAV absence (V-644). Agenda questions route to
|
|
`IntentQuery` at stage 0, and the `calendar` query source then reads a table
|
|
nobody writes. And `loop.State.CalendarBusy` is fed by the same facts, so the
|
|
gate's "do not nag mid-meeting" is permanently false.
|
|
|
|
`mavenclient` is still absent. `mavwaked` moved to workpc on 2026-08-09 (V-515).
|
|
|
|
### The voice wire
|
|
|
|
`internal/voice` is plaintext with no auth. Its own server doc says production
|
|
binds inside the wg tunnel, because the wg layer is the L0 floor. workpc is not
|
|
a wg peer, it sits on wlan0. So the tunnel is ssh instead.
|
|
|
|
mavend publishes the voice port to homesrv loopback only, `127.0.0.1:9110`.
|
|
Host 9100 is Vikunja's MCP, hence 9110. The container side stays 9100 so mavweb
|
|
keeps reaching `mavend:9100` by name. `deploy/maven-voice-tunnel.service` on
|
|
workpc forwards it over his key.
|
|
|
|
**Do not replace this with a LAN bind.** `SurfaceVoice` caps acts at L0, so an
|
|
unauthorized speaker could not run a destructive tool. L0 does not cap reading,
|
|
so they would still hear his facts, notes and calendar read back.
|
|
|
|
Both `mavwaked` and `mavenclient` speak `voice.Dial`, not `ipc.Dial`. The
|
|
`netaddr` token guards the daemon-to-daemon IPC seam and never touches this one.
|
|
`ipc.Dial` does take `tcp://host:port?token=...`, which is why V-515 was filed
|
|
as a config change. That premise was wrong, and the ssh leg is the correction.
|
|
|
|
The voice loop belongs on a client machine where the owner is standing, and that
|
|
machine is workpc (V-463, `docs/plans/17-where-the-voice-loop-runs.md`). homesrv
|
|
has a microphone, because it is a laptop, but it is in the wrong room.
|
|
|
|
### mavwaked on workpc
|
|
|
|
`deploy/mavwaked.service`, a user unit beside `mavgpud.service`. Two flags are
|
|
deliberate.
|
|
|
|
`-vad-model` is passed. Silero answers "is this frame speech" where the energy
|
|
floor answers "is this frame loud". It declines white noise at the same RMS, 0
|
|
frames against 68 to 99, and still hears all four spoken fixtures
|
|
(`docs/evals/2026-08-09-silero-vad.md`). It costs 509µs a frame and never touches
|
|
the GPU. A model that will not load is logged and not fatal.
|
|
|
|
`-barge-in` is not passed. The threshold is room-specific and this room has no
|
|
number yet. Read the "suppressed while speaking" means out of the journal first.
|
|
|
|
The device is `plughw:0,0` and not `hw:0,0`. The fifine offers 2 channels at
|
|
44100 or 48000 and nothing else, and mavwaked asks arecord for 16kHz mono. Bare
|
|
`hw` dies on "Channels count non available" before a frame is read.
|
|
|
|
There is no wake word yet (V-487 stage two), so the loop runs open.
|
|
|
|
mavwaked connects at startup and holds the conn, so a nudge routed to voice
|
|
reaches the speaker before he has said anything (V-671). It used to connect
|
|
lazily, which made the failure silent rather than absent: after one utterance
|
|
the session existed, `PushToMostRecent` succeeded, the dispatcher stopped
|
|
rerouting to telegram and ntfy, and mavwaked discarded the audio.
|
|
|
|
**Passwords are read from files, never taken as flag values.** `mavcaldav` uses
|
|
`-pass-file` and `-render-pass-file`. `mavpoll` and `mavmaild` follow the same
|
|
rule.
|
|
|
|
## Web UI conventions
|
|
|
|
Server-rendered pages share `cmd/mavweb/static/ui.css` (served at `/ui.css`) and
|
|
the shell partial in `cmd/mavweb/shell.html`. A page opens with
|
|
`{{template "shellTop" "<page-key>"}}` and closes with `{{template "shellBottom"}}`,
|
|
and the key marks the active sidebar link.
|
|
|
|
Every page is its own embedded `.html` file next to `main.go`. No page markup
|
|
lives in Go, and the sidebar is data (`sidebarSections`, `pageIcon`) the template
|
|
renders. No per-page `<style>` beyond true one-offs. Wrap every table in
|
|
`<div class=scroll>` so wide data pans on a phone. Local preview and headless
|
|
screenshot recipes are in `AGENTS.md`.
|