Compare commits
23 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 99e73ea653 | |||
| ab1784f5e1 | |||
| 2c73493bf8 | |||
| ff202c0c35 | |||
| 02d96e611d | |||
| 62eef01c18 | |||
| 1a8aed35b8 | |||
| ce6a6821a9 | |||
| 479b0c4475 | |||
| 877b1fd4f8 | |||
| 21a42cb3e6 | |||
| b8279f6a22 | |||
| 8c30971a96 | |||
| d0ea927ac3 | |||
| 9c7bafd5b1 | |||
| 1f1e002789 | |||
| ce91d20ac8 | |||
| 50130cdffb | |||
| a9b480a78f | |||
| c5264deb46 | |||
| 1cb269886e | |||
| 7a9b9cc669 | |||
| 31b5093403 |
@@ -1,805 +1,200 @@
|
|||||||
# CLAUDE.md
|
# CLAUDE.md
|
||||||
|
|
||||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
Guidance for Claude Code (claude.ai/code) working in this repository.
|
||||||
|
|
||||||
Maven is a self-hosted, privacy-first voice assistant (Russian + English). Go daemons
|
**This is a rules file.** It loads into every session, so it carries only what
|
||||||
talking over unix sockets; one resident small model for routing + phrasing; whisper.cpp STT, piper TTS.
|
changes what an agent does. A measurement belongs in `docs/evals/`, dated and
|
||||||
Deploy target is a Ryzen laptop (homesrv) with Vulkan offload to the Vega iGPU (`n_gpu_layers: 99`,
|
never edited after the day. A subsystem's reasoning belongs in its living doc
|
||||||
compose passes `/dev/dri` + the render gid) — the resident model stays ≤1.7B either way.
|
under `docs/`. Read that doc before changing the subsystem.
|
||||||
|
|
||||||
**Resident model:** currently **Qwen3-1.7B** (`UD-Q4_K_XL`), stock — not yet the CPT'd one.
|
| Read this | Before |
|
||||||
It replaced Qwen3.5-0.8B on 2026-07-31 because it measured better on both fixtures we have:
|
|---|---|
|
||||||
67.5% vs 59.7% intent-only on the 77-case RU routing fixture, and 20/27 vs 11-17/27 on the
|
| `docs/routing.md` | touching `internal/router/` or `queryWalk` |
|
||||||
talk fixture. See `docs/evals/2026-07-31-model-bakeoff.md`. It is a Thinking variant, so `n_ctx` is 4096
|
| `docs/deployment.md` | touching a daemon, compose, a systemd unit or the web UI |
|
||||||
— reasoning tokens need the room, and 4096 is what the scores above were measured at.
|
| `docs/offload.md` | touching a daemon seam or adding a model caller |
|
||||||
|
| `docs/world.md` | touching search, Kiwix or the world chain |
|
||||||
|
| `docs/language.md` | changing a prompt contract or a Russian word list |
|
||||||
|
| `docs/ecosystem.md` | touching Nexus, Praxis or Hexis |
|
||||||
|
| `docs/rearchitecture.md`, `docs/design.md` | changing the shape of anything |
|
||||||
|
| `docs/workflow.md` | the five stores, the doc tiers, the guards |
|
||||||
|
| `AGENTS.md` | local preview, screenshots, model downloads |
|
||||||
|
|
||||||
The **target** is still the locally CPT'd **Qwen3-1.7B** (Vikunja #122, training in flight).
|
## What Maven is
|
||||||
Stock already speaks good Russian; what it gets wrong is the persona — it writes `я рад`,
|
|
||||||
masculine, where Maven needs `рада`. That is what the CPT is for.
|
|
||||||
|
|
||||||
**Do not bother with sub-500M models.** LFM2.5-230M and 350M were measured on 2026-07-31 and
|
A self-hosted, privacy-first voice assistant in Russian and English. Go daemons
|
||||||
both are unusable in Russian: the 350M routes at 5.2% (worse than guessing) and answers
|
talk over unix sockets. One resident small model routes and phrases. whisper.cpp
|
||||||
"столица Франции?" with the invented non-word "Сторзит"; the 230M replies to Russian in
|
does speech-to-text and piper does text-to-speech.
|
||||||
Spanish. Their strong published IFEval/BFCL numbers are English-only. Model files live in
|
|
||||||
`/mnt/hdd1/llms`, bind-mounted to `/opt/maven/models/llm` — which **shadows** the repo's
|
|
||||||
`models/llm/`, so the LFM2.5 gguf sitting there is not loaded by anything. Swapping the resident
|
|
||||||
model is a one-line change to `phraser.model_path` in `deploy/mavend.json`.
|
|
||||||
|
|
||||||
See `docs/rearchitecture.md` for the target architecture, `docs/design.md` for the folded design spec, and
|
The resident model is **Qwen3-1.7B** (`UD-Q4_K_XL`) on homesrv, a Thinking
|
||||||
`AGENTS.md` for local-preview + model-download recipes.
|
variant at `n_ctx` 4096. Keep it at 1.7B or under. Sub-500M models are unusable
|
||||||
|
in Russian (`docs/evals/2026-07-31-model-bakeoff.md`). Model files live in
|
||||||
|
`/mnt/hdd1/llms`, bind-mounted over the repo's `models/llm/`, so a gguf sitting
|
||||||
|
in the repo is loaded by nothing.
|
||||||
|
|
||||||
**Model work is moving to the workstation** (owner's call, 2026-08-02). homesrv cannot grow a
|
The workstation is workpc and it holds the remote model and speech-to-text.
|
||||||
GPU and the workstation has 16GB of VRAM. So the resident model, STT and TTS become preferred
|
**It is never assumed up.** **Fall back silently** when it would only do the job
|
||||||
remotes with a floor on homesrv. The workstation is never assumed up. Fall back silently when
|
better. **Name the gap** when the resident model cannot do the job at all.
|
||||||
it would only do the job better. Name the gap when the 1.7B cannot do it at all. The embedder
|
|
||||||
stays on homesrv permanently, because it backs that floor. It is multilingual-e5-small,
|
|
||||||
quantized and asymmetric — `EmbedQuery` and `EmbedPassage` apply the `query:`/`passage:`
|
|
||||||
prefixes it was trained with, and calling plain `Embed` on a note is a bug. It replaced
|
|
||||||
MiniLM and bought ten points of recall@1 and 2.5× the speed; see
|
|
||||||
`docs/evals/2026-08-04-recall-e5-small.md`. Read `docs/offload.md` before
|
|
||||||
touching a daemon seam or adding a model caller. Vikunja #483 is the umbrella, #484 to #487
|
|
||||||
are the work.
|
|
||||||
|
|
||||||
Both halves are wired as of 2026-08-03. Routing and replies prefer the workstation silently
|
**The embedder stays on homesrv permanently**, because it backs that floor.
|
||||||
through `modelSeam`; nudge and reminder phrasing prefer it silently inside the phraser. A
|
`EmbedQuery` and `EmbedPassage` apply the `query:` and `passage:` prefixes
|
||||||
world question goes through `LLMPhraser.PhraseWorld` and names the gap when the card is not
|
multilingual-e5-small was trained with. Calling plain `Embed` on a note is a bug.
|
||||||
free — `worldGap` in `cmd/mavend/worldmodel.go`, which the owner hears instead of an invented
|
|
||||||
answer. A box with no `workstation` block behaves exactly as it did before the seam: naming
|
|
||||||
a gap requires a gap. The offload table in `docs/offload.md` says which caller is which.
|
|
||||||
|
|
||||||
**Speech-to-text moved on 2026-08-09** (V-486). `sttSeam` in `cmd/mavend/voicewire.go`
|
## Build and test
|
||||||
builds an `stt.Pair` beside `modelSeam`, preferring CrisperWhisper 2.0 turbo on workpc
|
|
||||||
with mavsttd as the floor. It takes only the silent half of the rule. A worse
|
|
||||||
transcript is still a turn, so `stt.Pair` has no `TranscribeRemote`. The fallback is
|
|
||||||
never spoken. CW2 turbo scores **10.4% WER in Russian against 27.5%** for the `ggml-small.bin`
|
|
||||||
mavsttd loads, over 200 Golos clips
|
|
||||||
(`docs/evals/2026-08-09-crisperwhisper2-russian-wer.md`). It runs in Intended mode, not
|
|
||||||
Verbatim, though that corpus cannot separate the two.
|
|
||||||
**whisper.cpp cannot load CW2 at all.** It reads its language count off the vocabulary
|
|
||||||
size, and CW2's 51897 tokens shift seven special token ids. So it is not a second
|
|
||||||
endpoint on mavgpud. It is its own transformers service on port 8081
|
|
||||||
(`deploy/cw2/serve.py`), which Maven reaches directly. `stt.HTTPTranscriber`
|
|
||||||
posts raw PCM to it with a bearer token, because audio is the most sensitive thing that
|
|
||||||
crosses this seam. The switch is `workstation.stt` in
|
|
||||||
`deploy/mavend.json`, and deleting the block sends every utterance to mavsttd.
|
|
||||||
**mavgpud runs that service as a second child.** That is not an optimisation. CW2 is a
|
|
||||||
ROCm process on the same card, so it registers on the KFD like any contender. Under its own
|
|
||||||
systemd unit it made mavgpud evict llama-server every few seconds. That took the
|
|
||||||
gemma-4-12b arm down for eight minutes on 2026-08-09 before anyone noticed. The card needs
|
|
||||||
one owner. Any GPU service added beside this daemon has the same defect, so add it to
|
|
||||||
`cmd/mavgpud` and not to systemd. CW2 is on the yield clock and not the idle one. At 1.6GB
|
|
||||||
it denies the card to nobody, and unloading it would only send the next voice turn to the
|
|
||||||
homesrv floor.
|
|
||||||
Text-to-speech has not moved and piper on homesrv is still the only synthesizer.
|
|
||||||
|
|
||||||
## Build & test
|
CGO daemons (`mavend`, `mavsttd`, `mavttsd`, `mavenclient`) need the vendored
|
||||||
|
toolchain wired through the Makefile. **Do not call `go build` on them bare**,
|
||||||
CGO daemons (`mavend`, `mavsttd`, `mavttsd`, `mavenclient`) need the vendored toolchain
|
and **do not hand-write the CGO preamble**. This box runs zsh, so an unquoted
|
||||||
and libs wired through the Makefile — **do not** call `go build` on them bare, use `make`:
|
`-run Test*` dies on "no matches found" before `go` is reached. `make t` also
|
||||||
|
carries `-count=1` and sets `MAVEN_ONNX_LIB`. Without that variable the four
|
||||||
|
`TestONNX*` measurements self-skip and the run still prints `ok`.
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
make build # all 11 binaries
|
make build # all 11 binaries. make build-web for one (web/waked/poll/caldav skip CGO)
|
||||||
make build-web # single daemon (pure-Go ones: web/waked/poll/caldav build without CGO)
|
|
||||||
make test # go test -race across ./internal/... ./cmd/... with CGO env set
|
make test # go test -race across ./internal/... ./cmd/... with CGO env set
|
||||||
```
|
|
||||||
|
|
||||||
Run one package or one test with `make t`. **Do not hand-write the CGO preamble.**
|
|
||||||
Past sessions pasted it about 390 times. That is where the shell-quoting failures
|
|
||||||
came from. This box runs zsh, so an unquoted `-run Test*` or `--include=*.go`
|
|
||||||
dies on "no matches found" before `go` is ever reached.
|
|
||||||
|
|
||||||
```sh
|
|
||||||
make t PKG=./internal/router/
|
|
||||||
make t PKG=./cmd/mavend/ RUN=TestSimulator
|
|
||||||
make t PKG=./internal/router/eval/ RUN='TestONNX' V=1 # V=1 for -v, RACE=0 to drop -race
|
make t PKG=./internal/router/eval/ RUN='TestONNX' V=1 # V=1 for -v, RACE=0 to drop -race
|
||||||
```
|
```
|
||||||
|
|
||||||
`t` carries `-race`, so a green `make t` cannot turn red under `make test`. It carries
|
## The daemons
|
||||||
`-count=1`, so a cached PASS from before your edit is never mistaken for a result.
|
|
||||||
|
|
||||||
It also sets `MAVEN_ONNX_LIB`, which the hand-written recipe did not. The four
|
Eleven binaries under `cmd/`, wired socket-to-socket over `internal/ipc`, not
|
||||||
`TestONNX*` measurements self-skip when that variable is unset. The run still prints
|
linked. `mavend` is the core and owns the DB and the IPC socket.
|
||||||
`ok`. So every targeted eval done the old way reported the hash ratchet while reading
|
`deploy/mavend.json` sets sockets, model paths and the phraser and embedder
|
||||||
as a real embedder score.
|
blocks, with `${VAR}` expansion from gitignored `deploy/telegram.env`.
|
||||||
|
**`docker-compose.yml` runs five**: `mavend`, `mavsttd`, `mavttsd`, `mavweb`,
|
||||||
|
`mavpoll`. Count against compose, not against `make build`. `mavwaked` runs on
|
||||||
|
workpc under systemd. `docs/deployment.md` says who else is absent and why.
|
||||||
|
|
||||||
Pure-Go packages (`router`, `memory`, `mavweb`, …) also run under a plain `go test ./pkg/`,
|
- **Passwords are read from files, never taken as flag values.**
|
||||||
but `make t` works everywhere and is one thing to remember.
|
- **The voice wire is plaintext with no auth.** mavend's voice port stays on
|
||||||
|
homesrv loopback and reaches workpc over ssh. Do not LAN-bind it.
|
||||||
## The daemons (`cmd/`)
|
`SurfaceVoice` caps acts at L0, and L0 does not cap reading.
|
||||||
|
- **A GPU service added beside mavgpud goes in `cmd/mavgpud`, never in systemd.**
|
||||||
| Binary | Role |
|
The card needs one owner. A second unit made mavgpud evict llama-server every
|
||||||
|---|---|
|
few seconds and took the model arm down for eight minutes.
|
||||||
| `mavend` | **Core.** Router, phraser, memory, reminders, digestion tick. Owns the DB + IPC socket. |
|
|
||||||
| `mavweb` | HTTP UI + PWA (`/dash`, `/history`, `/trace`, `/notifications`, `/tools`); WebAuthn auth. Connects to mavend's socket. |
|
|
||||||
| `mavsttd` | Speech-to-text (whisper.cpp, CGO). |
|
|
||||||
| `mavttsd` | Text-to-speech (piper subprocess). |
|
|
||||||
| `mavwaked` | Wake-word / VAD gate. **Not on homesrv** — see below. |
|
|
||||||
| `mavenclient` | Voice loop client (mic → stt → core → tts). **Not on homesrv** — see below. |
|
|
||||||
| `mavpoll` | Environment poller: netdata alarms, uptime-kuma, zenmoney, wireguard presence. Writes facts, sends nothing. Telegram is `internal/delivery/telegramsink`, not this. |
|
|
||||||
| `mavcaldav` | CalDAV calendar sync. |
|
|
||||||
| `mavmaild` | Mail reader (IMAP, read-only). Holds the IMAP password; core never sees it. |
|
|
||||||
| `mavgpud` | GPU supervisor. **Runs on workpc, not homesrv** — own unit, `deploy/mavgpud.service`. Keeps llama-server loaded while the card is free (V-488). Maven never asks it for anything, it reads `/health` through `llm.Pair`. |
|
|
||||||
| `mavupdate` | Not a daemon. Operator CLI a human runs on the box to deploy a new build. |
|
|
||||||
|
|
||||||
Two more binaries have no Makefile target and are built with `go run` or `go build` when
|
|
||||||
they are needed. Neither is deployed.
|
|
||||||
|
|
||||||
| Binary | Role |
|
|
||||||
|---|---|
|
|
||||||
| `mavseal` | Recovery tool. Encrypts a live tmpfs working copy back to the ciphertext file when mavend was killed before `defer st.Close()` sealed it. |
|
|
||||||
| `labelgen` | Runs the stage 0 grammars over utterances and prints JSONL, the training data for the routing heads (V-546). |
|
|
||||||
|
|
||||||
Daemons are wired socket-to-socket, not linked. `internal/ipc` is the client/server wire
|
|
||||||
protocol; the config in `deploy/mavend.json` (with `${VAR}` env expansion from gitignored
|
|
||||||
`deploy/telegram.env`) sets socket paths, model paths, and the phraser/embedder blocks.
|
|
||||||
|
|
||||||
**`docker-compose.yml` runs five: `mavend`, `mavsttd`, `mavttsd`, `mavweb`, `mavpoll`.**
|
|
||||||
Count against compose, not against the table. Four of the nine daemons are absent, and each
|
|
||||||
absence has a different reason.
|
|
||||||
|
|
||||||
`mavmaild` and `mavcaldav` are commented out in compose, each with the reason written
|
|
||||||
beside it: the first needs a mail account, the second a CalDAV account, and this box has
|
|
||||||
neither. `mavcaldav` used to appear nowhere at all, which was an oversight; it became a
|
|
||||||
recorded decision on 07-08-2026 (V-644). Two things ride on that absence and the block
|
|
||||||
names them. Agenda questions route to `IntentQuery` at stage 0 (V-498) and the `calendar`
|
|
||||||
query source then reads a table nobody writes. And `loop.State.CalendarBusy` is fed by the
|
|
||||||
same facts, so the gate's "do not nag mid-meeting" is permanently false. Its password is
|
|
||||||
read from a file (`-pass-file`, and `-render-pass-file` for the render collection), never
|
|
||||||
taken as a flag value, which is the rule `mavpoll` and `mavmaild` follow too.
|
|
||||||
|
|
||||||
**`mavwaked` and `mavenclient` are absent by decision, not oversight** (Vikunja #463,
|
|
||||||
`docs/plans/17-where-the-voice-loop-runs.md`).
|
|
||||||
homesrv has a microphone — it is a laptop — but it is in the wrong room, so a wake-word
|
|
||||||
daemon there listens to nobody. They belong on a client machine where the owner is standing.
|
|
||||||
|
|
||||||
**That machine is workpc** (owner's correction, 2026-08-05). This section used to say no
|
|
||||||
such machine existed, which was written when the workstation was only a model host. It is
|
|
||||||
where he sits most of the day and it has the microphone. `ipc.Dial` already takes
|
|
||||||
`tcp://host:port?token=...` through the netaddr seam, so the two daemons need deploying,
|
|
||||||
not building. V-515 is that deployment.
|
|
||||||
|
|
||||||
Until they are deployed, **the wake word and the VAD gate are covered by unit tests and by
|
|
||||||
nothing else**, and push-to-talk through `/dash` is what QA actually covers. Note that
|
|
||||||
deploying them does not by itself prove a wake word: `mavwaked` gates on energy and has no
|
|
||||||
keyword model (V-487), so the loop runs open until that lands.
|
|
||||||
|
|
||||||
## The ecosystem: Nexus, Praxis, Hexis
|
## The ecosystem: Nexus, Praxis, Hexis
|
||||||
|
|
||||||
Maven is one of four services. It owns conversation and personal memory. It does not
|
Nexus identifies, Praxis observes, Hexis acts, Maven understands. Maven does not
|
||||||
own identity, operational state, or execution. Full contract in
|
own identity, operational state, or execution. Full contract in
|
||||||
`docs/ecosystem.md`.
|
`docs/ecosystem.md`. All three are `nil` unless configured and each degrades
|
||||||
|
alone. An outage means a named gap, never a broken turn or a guess.
|
||||||
|
|
||||||
```text
|
- **No component reads another component's database.** Praxis attention comes
|
||||||
Nexus identifies. Praxis observes. Hexis acts. Maven understands and coordinates.
|
over HTTP, never from its SQLite file.
|
||||||
```
|
- **Identity lives in Nexus.** Do not invent a local fact key for something
|
||||||
|
Nexus resolves. `cmd/mavend/factenrichment.go` resolves `actionFact.Subject`.
|
||||||
| Service | Owns | Maven's client | Configured at |
|
- **Free text never reaches a mutating Hexis call.** Resolve to a canonical
|
||||||
|---|---|---|---|
|
entity id first. Ambiguous resolution asks the owner, it does not pick.
|
||||||
| **Nexus** | Canonical entity ids, names, aliases, relationships. Projects, services, devices, people, pets, places. | `nexusClient` in `cmd/mavend/ecosystem.go`, `POST /api/v1/resolve` | `nexus.url` (`http://nexus:9740`) |
|
|
||||||
| **Praxis** | Operational attention and item lifecycle. What needs looking at, what changed, what is still unresolved. | `praxisClient`, the HTTP tools API under `/api/v1/tools/` | `praxis.url` (`http://praxis:8989`) |
|
|
||||||
| **Hexis** | The capability registry and the only path to executing anything. | vendored `github.com/kami/hexis/pkg/client` | `hexis.url` (`http://hexis:9741`) |
|
|
||||||
|
|
||||||
All three are `nil` unless configured, and every one of them degrades on its own.
|
|
||||||
An outage means a named gap in the answer, never a broken turn and never a guess.
|
|
||||||
|
|
||||||
Rules that are not negotiable:
|
|
||||||
|
|
||||||
- **No component reads another component's database.** Praxis attention comes over
|
|
||||||
HTTP, never from its SQLite file.
|
|
||||||
- **Identity lives in Nexus.** Do not invent a local fact key for something Nexus
|
|
||||||
resolves. `actionFact` already sets `Subject`, and `cmd/mavend/factenrichment.go`
|
|
||||||
resolves it in the background against Nexus.
|
|
||||||
- **Free text never reaches a mutating Hexis call.** Resolve to a canonical entity id
|
|
||||||
first. Ambiguous resolution asks the owner, it does not pick.
|
|
||||||
- **LLM output is not authorization.** Confirmation binds capability id, target
|
- **LLM output is not authorization.** Confirmation binds capability id, target
|
||||||
entity, arguments, requester and expiry. See `cmd/mavend/confirm.go`.
|
entity, arguments, requester and expiry (`cmd/mavend/confirm.go`).
|
||||||
- **Praxis lifecycle words mean different things.** Surfaced is not acknowledged,
|
- **Praxis lifecycle words differ.** Surfaced is not acknowledged, acknowledged
|
||||||
acknowledged is not resolved, execution success is not recovery. Reading an item
|
is not resolved, execution success is not recovery. Reading an item aloud
|
||||||
aloud calls `Surface`, never `Acknowledge`.
|
calls `Surface`, never `Acknowledge`.
|
||||||
- **No automatic attention-to-action path.** Digestion may summarise Praxis. It may
|
- **No automatic attention-to-action path.** Digestion may summarise Praxis and
|
||||||
not call Hexis.
|
may not call Hexis.
|
||||||
|
- Every cross-service call carries a correlation id minted once per action
|
||||||
Every cross-service call carries a correlation id minted once per action
|
|
||||||
(`withCorrelationID`), a contract version header, and `X-Requested-By: maven`.
|
(`withCorrelationID`), a contract version header, and `X-Requested-By: maven`.
|
||||||
|
|
||||||
## Routing — read this before touching the router
|
## Routing
|
||||||
|
|
||||||
`internal/router/` has TWO layered engines. **The LLM router is now the default and it is
|
**Read `docs/routing.md` before touching `internal/router/` or `queryWalk`.** It
|
||||||
on in deploy** — this section used to say it was wired `nil`, which stopped being true on
|
carries the reasoning, the measurements and every rule's why. A route produces
|
||||||
2026-07-31.
|
two decisions. **Intent** is one of seven values. **Source** is where the answer
|
||||||
|
lives and is read on `IntentQuery` alone. Score them separately. The cascade is
|
||||||
|
stage 0 grammars, then the routing heads, then the resident model, then the
|
||||||
|
classifier. Every stage may decline and the next one answers.
|
||||||
|
|
||||||
- **LLM router (the intended design, docs/rearchitecture.md):** the resident Qwen3-1.7B (`llmrouter.go`)
|
- **The classifier is the floor, not dead code.** It answers when the resident
|
||||||
emits GBNF-constrained structured JSON, and the SAME model phrases replies. Embedder is
|
model is off, absent, or erroring. **Any model error falls through.**
|
||||||
demoted from a routing gate to a RAG hint. Wired at `voice.go:214` via
|
- **`baselineGrammars` in `eval_test.go` mirrors `buildRouter`.** A grammar
|
||||||
`pickLLMRouter(cfg.Voice.UseLLMRouter(), llmClient)`; the flag is `voice.llm_router`
|
added to one belongs in both, or the fixture scores a set nobody runs.
|
||||||
(`config.go`), `DefaultLLMRouter` is **on**, and `deploy/mavend.json` sets it `true`.
|
- **Go's `\b` is ASCII-only** and never fires after a Cyrillic letter. A Russian
|
||||||
- **Classifier cascade (the failure floor, not dead code):** `classifier.go` +
|
pattern needs an explicit `(\s|[?!.]|$)`.
|
||||||
`embedder.go` nearest-neighbour over frozen seed phrases. It runs when the LLM router is
|
- **`PraxisGrammars()` is the only path to Praxis**, not a faster one.
|
||||||
off, when there is no llama-server to talk to (`pickLLMRouter` logs that and degrades),
|
- **`voice.embedder.heads_path` must never point at `model_path`.** Recall
|
||||||
and on any per-turn LLM error. Do not delete it — routing by seed similarity is the known
|
depends on the resident e5-small scoring what it scored. Fine-tune a copy.
|
||||||
cause of weak RU query handling, but a turn must never break on the model.
|
- **Routing traces are retained 14 days**, enforced on write and again on start.
|
||||||
|
- **Bump `tokenizerRev` on any change to what `encodeWord` emits**, so a
|
||||||
|
tokenizer fix triggers `ReembedAll` the way swapping the model file does.
|
||||||
|
- **A new rung in the `runTurn` ladder needs its name in `preRouteLadder`**
|
||||||
|
(`cmd/mavend/decisiontrace.go`), or it is missing from the decision record.
|
||||||
|
|
||||||
Cascade order: `stage0.go` exact-match fast-path → LLM router (when non-nil) → classifier
|
**`queryWalk` takes query sources out and moves none** (`actions_query.go`). That
|
||||||
fallback. Any LLM error falls through to the classifier so a turn never breaks on the model.
|
is the safety argument and it is not negotiable. The table's order is
|
||||||
|
load-bearing and carries "the owner's data first, then the world".
|
||||||
|
`SourceUnknown` is the floor and walks the whole chain. A named destination
|
||||||
|
removes only the sources marked `guesses: true`, so a source that looks rather
|
||||||
|
than guesses is always asked. **The personal boundary is the one exception and
|
||||||
|
it is deliberate.** It guesses, so naming `SourceWorld` drops it. **Only a stage
|
||||||
|
0 grammar may drop it** (owner's call, V-666). `queryWalk` reads
|
||||||
|
`Decision.SourceAnchored` for the source marked `boundary: true` and no other.
|
||||||
|
|
||||||
**A stage-0 decision is slot-extracted too, since 06-08-2026** (V-572). `fillMatchedSlots`
|
Judge a routing change against the classifier (76.0% intent, 36.4% destination)
|
||||||
in `router.go` runs the stage-2 extractor over whatever a grammar built and fills only the
|
and the resident model (80.2% intent), since those always answer. The fixture
|
||||||
slots it left empty — a matched value always wins, because the rule read a literal pattern
|
has grown from 77 cases to 96, so a number compares only to another number on
|
||||||
and the extractor guesses. It did not run before, so `ReminderGrammar` handed the daemon
|
the same fixture.
|
||||||
`HasTime: false` for "напомни в 11:00 позвонить маме" and `missingFor` read the silence as
|
|
||||||
absence and asked "Когда?". It applies to every grammar and is inert for all but the
|
|
||||||
reminder: `Extract` fills Time, Fn and Key and nothing else, and the query, clock, agenda,
|
|
||||||
feed, list, task and narrative rules all emit intents with no such slot. Benchmarked at
|
|
||||||
20000x, a stage-0 query costs 3.7µs against 3.9µs before. **`Slots.Text` is deliberately not
|
|
||||||
filled** — a grammar that left it empty meant it, and `agendaQueryBuild` hands the query
|
|
||||||
chain the utterance itself. Fixture unchanged at 64/91, with "slots deferred to daemon"
|
|
||||||
6 → 0.
|
|
||||||
|
|
||||||
Measured on the 77-case RU fixture. **Re-measured 2026-08-02: the classifier scores 68.8%
|
## Language: model output and Russian
|
||||||
full accuracy at p50 16.6µs**, not the 36.8% at p50 31ms that stood here from
|
|
||||||
`docs/evals/2026-07-31-model-bakeoff.md`. That older figure predates the stage 0 rules and the
|
|
||||||
seed additions, both of which now score inside the classifier baseline. Qwen3-1.7B scores
|
|
||||||
77.9% intent-only / 72.7% through the cascade. So the router buys about 4 points of accuracy,
|
|
||||||
not a doubling, and the trade is worth re-arguing rather than assuming. **The ≈2.7s figure
|
|
||||||
that stood here until 2026-08-02 was contention, not the model.** See `docs/evals/2026-07-31-routing.md` line 61, which measures the LLM router at
|
|
||||||
p50 825ms / p95 1.2s / max 3.0s and the full cascade at p50 0.80-1.04s. Do not plan latency
|
|
||||||
work off the bakeoff table.
|
|
||||||
|
|
||||||
**Re-measured 2026-08-05 on the fixture as it now stands, 91 cases** (V-320 item 2,
|
Both contracts are in `docs/language.md`. What must not be broken:
|
||||||
`docs/evals/2026-08-05-routing-resident-model.md`): cascade + resident model scores
|
|
||||||
**75.8% full / 80.2% intent-only at p50 1.19s / p95 1.65s**. That is a new baseline and not
|
|
||||||
a movement, because 14 cases were added since the 77-case number above. The model alone
|
|
||||||
scores 37.4% full against 61.5% intent-only, and the gap is slots rather than routing: it
|
|
||||||
routes `reminder` and leaves the time to the daemon, which is what the contract asks. To
|
|
||||||
re-run it, start a **second** llama-server on a fixed host port — the resident one binds
|
|
||||||
`--port 0` inside the container and no host process can reach it.
|
|
||||||
|
|
||||||
**The numbers above are the homesrv floor, not the ceiling.** With the workstation up, routing
|
- **One parser for model text, `parseResponseMood`** in
|
||||||
completes through `llm.Pair` against the model mavgpud holds, which is better than the resident
|
`internal/phraser/parse.go`. Every phrasing path reaches it. Mood is an enum.
|
||||||
model and about 2.5× faster. gemma-4-12b scored **84.4% full / 93.5% intent-only at p50 329ms**
|
- **The router prompt is a separate contract** over 7 intents, and
|
||||||
(`docs/evals/2026-08-02-workstation-gemma4-12b.md`, Vikunja #485). The workstation is never
|
`llm/check_prompt_parity.py` keeps the Go and relabelling copies identical.
|
||||||
assumed up, so both sets of numbers are live. Judge a
|
- **Russian words are matched by three mechanisms and no fourth**:
|
||||||
routing change against the classifier and the resident model, since those are what always answer.
|
`internal/lexicon` for closed classes, `internal/morph` for grammar, and
|
||||||
|
`cmd/mavend/topics.go` with the embedder for open sets. A regex whose output
|
||||||
|
is a fact or a route is the defect. A regex over structured input is not.
|
||||||
|
- **Seeds are scoring data.** Editing one moves a recogniser and must be
|
||||||
|
re-measured against the `TestONNX*` tests, not eyeballed.
|
||||||
|
|
||||||
**The workstation runs gemma-4-E4B since 2026-08-09** (owner's call), and it is a
|
## Non-goals and hard constraints
|
||||||
step down measured the same day (`docs/evals/2026-08-09-e4b-vs-12b-routing.md`).
|
|
||||||
Against a same-session 12B control it scores **83.3% full / 89.6% intent-only,
|
|
||||||
destination 19/33 against 23/33, at p50 294ms against 344ms**. So it costs four
|
|
||||||
destination cases and buys 50ms. Read destination as the finding: it names nothing
|
|
||||||
where the 12B names `recall` or `calendar`, which is safe but walks the whole chain.
|
|
||||||
It also has no MTP and cannot be given any here. The only `gemma4-assistant`
|
|
||||||
draft on disk is trained against the 12B's hidden states.
|
|
||||||
**Phrasing was the unmeasured half and it is measured now**
|
|
||||||
(`docs/evals/2026-08-09-e4b-phrasing.md`). E4B scores nudges 15/15 and the
|
|
||||||
36-case talk fixture **29/36 at p50 516ms**, against the resident model's 25/36
|
|
||||||
at p50 2.97s. Persona is clean: `lang`, `feminine` and `address` are all 36/36,
|
|
||||||
where the resident model loses three on `address`. Every failure is `ontopic`
|
|
||||||
and none is a parse error. The 2026-08-05 temperature sweep put this fixture's
|
|
||||||
ceiling at 30/36, because two reply cases fail at every temperature (V-537), and
|
|
||||||
both are in E4B's failure list. So the swap costs nothing here. One defect no
|
|
||||||
check catches: in chat E4B claims "Я записала несколько идей!" when nothing was
|
|
||||||
stored, which is a wrong claim about state.
|
|
||||||
|
|
||||||
**The intended third engine is not a generative model** (owner's call, 05-08-2026, V-546,
|
Not a nag, not autonomous.
|
||||||
`docs/plans/18-routing-heads-on-e5-small.md`). Routing has a bounded output space, so it is
|
**The persona is feminine.** Russian self-reference takes feminine forms: `рада`
|
||||||
classification, and the 118M multilingual-e5-small is already resident. Three heads on one
|
not `рад`, `поняла` not `понял`. The owner is male and she speaks to him
|
||||||
forward pass: intent, mood, and BIO slot tags. Roughly 5e15 FLOPs to train, so 10 to 30
|
informally. Use "ты", singular, never "вы" or "ваш", and never "он" or "его".
|
||||||
minutes on the workstation. A 100M decoder from scratch is 10 to 20 GPU hours. Two things
|
She talks TO the owner, not about him. Pet names such as "милый" are forbidden.
|
||||||
it buys that a decoder cannot. No grammar is needed, because a softmax cannot emit a value
|
The name "Ками" is not. `CheckAddress`, `CheckFeminine` and `CheckCringe` in
|
||||||
that does not exist. And max softmax is a calibratable confidence, where `Confidence: 1.0`
|
`internal/phraser/eval/checks.go` enforce this, scored by `make eval-phrasing`.
|
||||||
was a hardcode. **Fine-tune a copy of the weights.** The resident embedder backs memory
|
|
||||||
recall. Training it in place couples routing accuracy to recall@1, with nothing in the
|
|
||||||
suite to name the trade.
|
|
||||||
|
|
||||||
**Two of those heads are trained as of 08-08-2026, and they are not the three
|
**"Never phones home" is deprecated** (owner's call, 2026-07-31). She reads
|
||||||
above** (V-661, `docs/evals/2026-08-08-routing-heads-two-head.md`). Intent and
|
external sources, and `docs/world.md` carries that chain. What holds regardless:
|
||||||
destination share one masked mean pool. Destination scores a mean **80.8%** over
|
|
||||||
three seeds, best **29/33 (87.9%)**. The classifier cascade scores 12/33 and the
|
|
||||||
cascade with gemma-4-12b scores 24/33, so a 118M encoder beats the 12B teacher it
|
|
||||||
was distilled from. Read the best run as one seed and not a headline, because one
|
|
||||||
case is 3 points on a fixture this small.
|
|
||||||
Recall is 15/15 and world is 5/5. Intent is 93.6% mean over three seeds. That is
|
|
||||||
**not** comparable to the 76.0% and 84.4% those two arms scored: a softmax has no
|
|
||||||
clarify class, so the head's fixture is the 88 cases carrying an intent.
|
|
||||||
|
|
||||||
**A fourth head asks instead of guessing, same day** (V-661,
|
- **No telemetry, no cloud model, no third-party account.** Inference stays on
|
||||||
`docs/evals/2026-08-08-clarify-head-four-head.md`). Clarify is not a value
|
the box and nothing about Maven is reported to anyone.
|
||||||
of intent, so a softmax cannot emit it. It is a second question over the
|
- **The owner's data first, then the world.** Every source reading his facts,
|
||||||
same pooled vector: can Maven act on this at all. That is why the head's
|
notes, calendar, tasks or house runs first, and the personal boundary sits
|
||||||
fixture was 88 cases and not 96. Over three seeds it catches **7.0 of the 8
|
between them and anything outside.
|
||||||
`want_clarify` cases and produces 2.3 false clarifies of 88**. The cascade
|
- **His notes and facts are never search input.** Only the utterance goes out,
|
||||||
today misses 1 and produces 2, so this is parity with no rules in front of
|
never the persona block, the history, or matched notes.
|
||||||
it. Accuracy is the wrong number here and a head that never asks scores
|
- **External search is allowed and off unless configured.** Deleting the
|
||||||
91.7%. Confidence is the other half. Max softmax over the intent head reads
|
`search` block in `deploy/mavend.json` turns it off.
|
||||||
**0.851 where it is right against 0.604 where it is wrong**, ranking right
|
- **`Response.Empty()` is the whole gate** on a world answer. There is no
|
||||||
above wrong in 83.4% of pairs. `Confidence: 1.0` was a hardcode, and this
|
quality threshold in front of it and four candidate signals all failed.
|
||||||
replaces it with a signal. The two are not the same signal: one says which
|
|
||||||
intent is unclear, the other says the utterance carries too little to act
|
|
||||||
on. **The fourth head is not free the way the third was.** Intent,
|
|
||||||
destination and slot F1 each move down one to four points, inside the seed
|
|
||||||
spread. `поужинал` is a false clarify on every seed, which is the same
|
|
||||||
defect `thinSingleToken` was narrowed for on 2026-08-01.
|
|
||||||
|
|
||||||
The corpus for it is generated, because every existing row is answerable by
|
|
||||||
construction. **The router-prompt agreement filter cannot work here**, since
|
|
||||||
`routeGrammar` has no clarify value and a generated line always agrees with
|
|
||||||
itself. A gemma judge replaces it. The first judge called 24 of 40
|
|
||||||
answerable rows underspecified, because it judged against a generic
|
|
||||||
assistant rather than against Maven's contract.
|
|
||||||
|
|
||||||
**Mood is cut, not deferred.** The enum describes her own reply state, not the
|
|
||||||
speaker's emotion, and no dataset maps onto it.
|
|
||||||
|
|
||||||
**A third head landed the same day** (`docs/evals/2026-08-08-slot-head-three-head.md`).
|
|
||||||
BIO slot tags had no Maven-domain corpus, which was true of found corpora and
|
|
||||||
false of made ones. `label_slots.py` distils spans out of gemma-4-12b under a
|
|
||||||
GBNF closed over Maven's own five slots. A span survives only when it is a
|
|
||||||
literal substring of the utterance, so the agreement filter costs no second
|
|
||||||
call. 2178 spans over 1702 rows, 37 dropped, nothing unparsed. Three heads score
|
|
||||||
intent **92.8%**, destination **82.8%** and slot span F1 **72.4%** over three
|
|
||||||
seeds. The slot head is free: both other numbers move less than their own seed
|
|
||||||
spread. Epoch selection reads the intent dev slice alone. Slot F1 is still
|
|
||||||
climbing when it stops, which costs about 4 points.
|
|
||||||
|
|
||||||
The MASSIVE warm-start of step 2 is worth nothing here. Stock e5-small ties it on
|
|
||||||
intent and leads by a third of a case on destination. Nothing argues for keeping
|
|
||||||
that step.
|
|
||||||
|
|
||||||
The floor was a corpus defect and it is fixed. The first 120 floor rows carried
|
|
||||||
one sentence shape, so the head named a destination where the fixture says walk
|
|
||||||
the chain. Rotating six shapes took the floor 3/7 to 6/7 and destination 75.8% to
|
|
||||||
80.8%. What is left is calendar at 3/6 on every seed, which training cannot move:
|
|
||||||
the possessive agenda rules claim those cases at stage 0 and name nothing, so no
|
|
||||||
label reaches the head. That is the same trade V-660 flagged and it wants the
|
|
||||||
owner's call.
|
|
||||||
|
|
||||||
**The heads run in Go and route every turn, since 08-08-2026** (V-664,
|
|
||||||
`docs/evals/2026-08-08-routing-heads-in-go.md`). This section used to say
|
|
||||||
nothing of it ran. `RouterHeads` in `internal/router/heads.go` loads
|
|
||||||
`router_heads.onnx` and reads intent, destination and clarify off one forward
|
|
||||||
pass. It is stage 0b: after the grammars, **before** the resident model, and the
|
|
||||||
classifier is still behind both. Through the cascade it scores intent **96.9%**
|
|
||||||
and destination **75.8%** at p50 27.9ms. That beats the gemma-4-12b cascade,
|
|
||||||
84.4% and 72.7%, at a twelfth of its 329ms. The workstation stays the better
|
|
||||||
phraser and is no longer the better router.
|
|
||||||
|
|
||||||
Three rules around it. The **clarify head decides first**, before the intent
|
|
||||||
threshold. It answers a different question. A thin utterance scores low
|
|
||||||
intent by construction, so gating it cost 6 of 8 ambiguous cases. The
|
|
||||||
**destination head is read on `IntentQuery` only**, since no other intent
|
|
||||||
reaches `queryWalk`. And `headsThreshold` is 0.6, the measured knee: every value
|
|
||||||
to 0.85 drops right answers and keeps the same two wrong ones.
|
|
||||||
|
|
||||||
`voice.embedder.heads_path` is the whole switch. Empty, missing or unloadable
|
|
||||||
means the heads are nil and the cascade is byte-for-byte what shipped before
|
|
||||||
them. **It must never be pointed at `model_path`.** The resident e5-small must
|
|
||||||
not be replaced by the fine-tuned copy. Recall depends on that file scoring
|
|
||||||
what it scored.
|
|
||||||
|
|
||||||
**The hand-written tokenizer read every long word backwards** until this task
|
|
||||||
(`encodeWord`, `onnxembedder.go`). It cost recall@1 7.4 points and recall@3 11.1.
|
|
||||||
Nothing caught it because seeds and queries were mangled the same way, so cosine
|
|
||||||
survived. The heads found it. They are trained through transformers and read
|
|
||||||
through this. The embedder id now carries a tokenizer revision
|
|
||||||
(`@384/tok2`), so fixing the tokenizer triggers `ReembedAll` the way swapping the
|
|
||||||
model file does. Bump `tokenizerRev` on any change to what it emits.
|
|
||||||
|
|
||||||
`Confidence: 1.0` used to be hardcoded in `llmrouter.go`, so the LLM
|
|
||||||
path could never ask for clarification (6/6 refusal cases missed on the fixture) — Vikunja
|
|
||||||
#359. Fixed 31-07-2026 with structural signal (single-token utterance, keyless fact, act with
|
|
||||||
no allowlisted fn) feeding the same stage-3 gate the classifier path already had — see
|
|
||||||
`gateLLMDecision` in `router.go`. Note the second half of that bug: the LLM branch never
|
|
||||||
consulted `r.threshold` at all, so a correct low confidence would have been discarded anyway.
|
|
||||||
|
|
||||||
Re-measured on the fixture after the fix: **missed clarify 6/6 → 1**, at the cost of 3 false
|
|
||||||
clarifies and 2.6pt of full accuracy (72.7% → 70.1%, intent-only 67.5% → 74.0%). Two of the
|
|
||||||
three false clarifies are acts the model mis-routed and the gate caught — asking beats wrongly
|
|
||||||
executing, so the fixture and the daemon disagree about what is correct there. The third,
|
|
||||||
`"поужинал"`, was a real defect: the single-token rule was an English intuition and does not
|
|
||||||
transfer to Russian, where one word is routinely a whole sentence.
|
|
||||||
|
|
||||||
Narrowed 01-08-2026. `thinSingleToken` (`internal/router/singletoken.go`) still thins a bare
|
|
||||||
one-word nominal — "вода", "бэкап" — but spares two classes: a closed lexicon of social and
|
|
||||||
control singles ("привет", "спасибо", "стоп", "yes"), and any token carrying a Russian verb
|
|
||||||
ending (past tense, 2nd person, reflexive), because a verb already contains its subject. Both
|
|
||||||
tests are offline and cost nothing. Re-measured: **false clarifies 3 → 2, intent-only 74.0% →
|
|
||||||
75.3%, full accuracy unchanged at 70.1%, missed clarify still 1.** The two remaining false
|
|
||||||
clarifies are the act-with-no-allowlisted-fn arm of the gate, not this rule.
|
|
||||||
|
|
||||||
Agenda questions taken off the model, 01-08-2026. `AgendaQueryGrammars` (`stage0.go`, wired
|
|
||||||
after the clock rules in `buildRouter`) routes "что у меня сегодня", "во сколько у меня
|
|
||||||
встреча" and anything naming a calendar to `IntentQuery` at stage 0. They were going to
|
|
||||||
`IntentSystem`, where `replySystem` has no agenda arm and answered "пока не умею" — the
|
|
||||||
fixture had said `query` since ru-query-019 was written. Measured: **full accuracy 70.1% →
|
|
||||||
72.7%, intent-only 75.3% → 77.9%, calendar 0/2 → 2/2**, clarify counts unchanged. Note that
|
|
||||||
Go's `\b` is ASCII-only and never fires after a Cyrillic letter; the pattern needs an
|
|
||||||
explicit `(\s|[?!.]|$)`.
|
|
||||||
|
|
||||||
Two more shapes taken off the model, 04-08-2026 (V-498). `rest-of-day-query` inside
|
|
||||||
`AgendaQueryGrammars` claims "что дальше?" / "what's next", and `NarrativeQueryGrammar`
|
|
||||||
(`stage0.go`, wired **last** in `buildRouter`, after the capture marker) claims "расскажи про
|
|
||||||
X", "объясни X", "опиши X". Neither carries a question mark or an interrogative, so the model
|
|
||||||
called both `IntentFact`; the write was caught downstream by `IsQuestionShaped`, so this was a
|
|
||||||
latency and fixture defect, not a correctness one. The narrative rule reads the same
|
|
||||||
`narrativeRequests` lexicon `IsQuestionShaped` reads, and declines `chatNarrativeTopics` — a
|
|
||||||
joke, a bedtime story, herself — because the query chain has no source that answers those.
|
|
||||||
New fixture cases ru-query-024 and ru-query-025. Classifier + ONNX baseline **56/80 (70.0%) →
|
|
||||||
58/82 (70.7%)**, no case regressed, no new false clarify. The LLM arm was not measured (no
|
|
||||||
llama-server in that run), so judge it again before quoting a cascade number.
|
|
||||||
|
|
||||||
Praxis taken off the model, 05-08-2026 (V-516). `PraxisGrammars()`
|
|
||||||
(`internal/router/praxis.go`, wired in `buildRouter` before the capture marker because
|
|
||||||
"отметь" is a capture verb) fills `Slots.Fn` with a Praxis capability name.
|
|
||||||
**These grammars are the only path to Praxis, not a faster one.** Measured
|
|
||||||
2026-08-05 with the resident model as router (V-517,
|
|
||||||
`docs/evals/2026-08-05-reach-llm-router.md`): the model alone reaches Praxis
|
|
||||||
**0/12**, the same as the classifier alone, because nothing in the router
|
|
||||||
prompt names a Praxis capability and there is no string for it to write.
|
|
||||||
Through the cascade it is 11/12. Deleting these rules costs every point. Praxis reach
|
|
||||||
was **0/12 and structurally so**: `handlePraxisAct` compares `Slots.Fn` to a capability
|
|
||||||
alias, and that slot is filled from the deployment's enabled tool names, which no Praxis
|
|
||||||
alias is on. Measured **16/30 → 27/30 overall, praxis 0/12 → 11/12, lifecycle 0/5 → 5/5**
|
|
||||||
(`docs/evals/2026-08-05-praxis-reach.md`). Two rules to know before editing: a **stative**
|
|
||||||
lifecycle word ("готово", "принято") needs an item named beside it, while a bare
|
|
||||||
**imperative** ("закрывай") may ask which one. The bare arm additionally requires that
|
|
||||||
the sentence name no object of its own, or "закрой шторы в комнате" goes to Praxis instead
|
|
||||||
of the house. A demonstrative ("отметь это как сделанное") resolves against
|
|
||||||
`h.surfacedItems` only when exactly one item was spoken. Otherwise the turn goes back to
|
|
||||||
the cascade rather than transitioning the wrong item.
|
|
||||||
|
|
||||||
**Who claimed a turn is now recorded, and so is who did not** (V-564, umbrella
|
|
||||||
V-558). Arbitration between the claimants on the utterance stream is order,
|
|
||||||
hardcoded in the pre-route resolver ladder, in `buildRouter` and in
|
|
||||||
`querySources`. `internal/decision` records one `Record` per turn: every
|
|
||||||
claimant, what it would have made the turn, the score it reported, and whether
|
|
||||||
it won, declined, lost on score, was thinned by a gate or was **never asked**.
|
|
||||||
The record rides the context, the same seam `querysource.go` uses, so a claim
|
|
||||||
site cannot change a route and a context with no record costs nothing. It is
|
|
||||||
installed in `runTurn`, so the mic, telegram and the web all leave the same
|
|
||||||
trail. Storage is a 25-turn in-memory ring on the handler (`decision.Ring`),
|
|
||||||
read over `ipc.TurnDecisions` and rendered as the second table on `/trace`.
|
|
||||||
**It also persists, since 06-08-2026, and that reverses what this section used to
|
|
||||||
say** (V-629, `docs/plans/21-persisting-the-routing-trace.md`). The old rule was
|
|
||||||
that nothing persists, because a turn record is read minutes later or never. The
|
|
||||||
owner reversed it: the routing heads (V-546) cannot be fitted or calibrated
|
|
||||||
without real utterances, and 9 of the 31 modes in `internal/modes` have no seed
|
|
||||||
example at all. The ring did not move. It is still what `/trace` reads and still
|
|
||||||
what a test with no store gets. `cmd/mavend/routingtrace.go` is a second sink
|
|
||||||
beside it, writing `routing_traces` (migration #23). The utterance is stored in
|
|
||||||
clear, because a 384-dimension vector of a short sentence is substantially
|
|
||||||
recoverable and storing vectors instead would be a privacy claim we cannot
|
|
||||||
support. What makes it safe is the same thing that makes the fact store safe.
|
|
||||||
Retention is 14 days, enforced on write and again on start, so a box that goes
|
|
||||||
quiet does not keep every row. Nothing reads it outward, and the rule
|
|
||||||
that his notes and facts are never search input covers this table. `Store.Wipe`
|
|
||||||
deletes it with everything else. A correction (V-630) is promoted out into a
|
|
||||||
seed-shaped row in `routing_labels` (migration #24) and kept, because a label is
|
|
||||||
not a transcript. The transcript still expires. The gesture that writes one is
|
|
||||||
two buttons beside the reply on `/chat`, reached over `ipc.CorrectTurn` and the
|
|
||||||
trace id that now rides back on `ipc.ChatReply`. A turn marked wrong with no
|
|
||||||
target is a usable negative, so naming the intent is never required. The target
|
|
||||||
is one of the seven intents and never free text. **All three reaches offer it as
|
|
||||||
of 06-08-2026**, and this section used to say only `/chat` did. Voice is the
|
|
||||||
`repair` rung, which has read spoken corrections since V-455 and now writes the
|
|
||||||
durable label beside the classifier seed it always wrote; a spoken negative with
|
|
||||||
no target is its own rung, `repair-negative` (V-636, `docs/plans/22-correcting-a-turn.md`).
|
|
||||||
Telegram is an inline keyboard under the reply, and it needed the chat to become
|
|
||||||
readable first — **telegram is no longer outbound only** (V-637,
|
|
||||||
`docs/plans/23-inbound-telegram.md`). The poller is dark unless the `telegram`
|
|
||||||
block says `intake`, it long-polls because the box takes no inbound connections,
|
|
||||||
it accepts `chat_id` and no other sender, and it drops whatever queued while the
|
|
||||||
daemon was down. It reaches the daemon through `ipc.CoreAPI` alone, so a chat
|
|
||||||
turn takes the path `POST /api/chat` takes. Note that the turn source is still
|
|
||||||
`tap:text` for both, so provenance cannot tell a chat turn from a typed one.
|
|
||||||
Adding a rung to the ladder
|
|
||||||
in `runTurn` means adding its name to `preRouteLadder` in
|
|
||||||
`cmd/mavend/decisiontrace.go`, or that rung is silently missing from the record.
|
|
||||||
|
|
||||||
**A route now says where the answer lives, not only that the turn is a question**
|
|
||||||
(V-655, 07-08-2026). `query` was a shrug. The cascade sorted an utterance into one of
|
|
||||||
seven intents, with stage 0, the resident model and the classifier behind it. Then
|
|
||||||
`IntentQuery` handed the turn to `querySources` in the daemon. That is twenty-two branches
|
|
||||||
deciding by seed similarity in a fixed order. It has no fixture and no accuracy
|
|
||||||
number, no model arm and no floor. `Decision.Source` (`internal/router/source.go`) is
|
|
||||||
the second half of the route. Twelve destinations, not twenty-two. The three recall
|
|
||||||
passes plus `fact-by-key` are one destination from outside. So are search, Kiwix and
|
|
||||||
the URL reader.
|
|
||||||
|
|
||||||
**`SourceUnknown` is a real value and it is the floor.** Nothing named a destination,
|
|
||||||
so the daemon walks the whole chain. That is byte-for-byte what shipped before the
|
|
||||||
field existed. The classifier arm names nothing, so a box whose model is down routes
|
|
||||||
queries exactly as it did.
|
|
||||||
|
|
||||||
`queryWalk` in `cmd/mavend/actions_query.go` takes sources **out** and moves none.
|
|
||||||
That is the safety argument and it is not negotiable. The table's order is
|
|
||||||
load-bearing. Every comment on it argues a reason between two sources, and above all
|
|
||||||
it carries "the owner's data first, then the world". Naming `SourceWorld` does not
|
|
||||||
send the turn outside on its own. His notes and his facts still run first, because
|
|
||||||
they look rather than guess.
|
|
||||||
|
|
||||||
**The personal boundary is the one exception and it is deliberate.** It guesses,
|
|
||||||
so naming `SourceWorld` drops it. That is what stops it answering "кто такой
|
|
||||||
Линус Торвальдс?" with "не нашла у тебя такой записи", which it did on
|
|
||||||
2026-08-07. `TestNamingRecallKeepsTheBoundary` pins the other half: naming
|
|
||||||
`SourceRecall` keeps the boundary in front of the world.
|
|
||||||
|
|
||||||
**Only a stage 0 grammar may drop it** (owner's call, 09-08-2026, V-666). The
|
|
||||||
question of who is allowed to was open until then. Three deciders name a
|
|
||||||
destination and two of them infer it: the routing heads and the resident model.
|
|
||||||
An inferred `SourceWorld` on a question about him would reach SearXNG, and that
|
|
||||||
widens what is asked rather than costing a local answer. So `Decision.SourceAnchored`
|
|
||||||
carries the provenance. It is a field and not `Stage == 0`. Stage 0 also means
|
|
||||||
confidence 1.0 and an anchored claim band, and one of those could stop implying
|
|
||||||
the others. `queryWalk` reads it for the source marked `boundary: true` and for
|
|
||||||
no other. So every other guesser still comes off the turn, whoever named the
|
|
||||||
destination. `TestOnlyAGrammarMayDropTheBoundary` pins both directions.
|
|
||||||
`definitionQueryPattern` claims "кто такой X", so the 2026-08-07 case is still
|
|
||||||
anchored and still answered.
|
|
||||||
|
|
||||||
What comes out is only the sources that **guess**. Those decide a turn is theirs by
|
|
||||||
cosine against frozen seeds, then answer whatever they claimed. They hold no table
|
|
||||||
that could come back empty. Weather is the pure case and has no local data at
|
|
||||||
all. It was measured on the box on 2026-08-07
|
|
||||||
(`docs/evals/2026-08-07-week-of-usage.md` section 4). It answered both "что такое
|
|
||||||
TCP?" and "сколько будет 17 на 23?" with "для какого города?". The feed answered "какой у меня любимый язык?" with kernel headlines.
|
|
||||||
The personal boundary answered "кто такой Линус Торвальдс?" with "не нашла у тебя
|
|
||||||
такой записи". A source that guesses is marked `guesses: true` in the table. One that
|
|
||||||
looks is not, and it is always asked.
|
|
||||||
|
|
||||||
Stage 0 fills the destination where a rule already knows it. `WorldQueryGrammars()`
|
|
||||||
(`internal/router/worldquery.go`) claims "что такое X" and "сколько будет 17 на 23".
|
|
||||||
It is wired after the agenda rules and **before** the feed and list rules.
|
|
||||||
"что такое лента" is a definition question, and the feed rule would take it on the
|
|
||||||
noun alone.
|
|
||||||
`calendar-query` and `event-time-query` name the calendar. The possessive agenda rules
|
|
||||||
deliberately do not. "что у меня в списке покупок" matches `agenda-query`, and naming
|
|
||||||
the calendar there would take the list source off the turn.
|
|
||||||
|
|
||||||
Fixture unchanged at **69/91 classifier+ONNX**, measured both sides. That is the
|
|
||||||
expected result, because it scores intent and no case here changes intent.
|
|
||||||
|
|
||||||
**The destination has its own fixture and its own number as of 08-08-2026**
|
|
||||||
(V-659, `docs/evals/2026-08-08-destination-fixture.md`). This section used to say
|
|
||||||
it had neither. `want_source` on `eval.Case` is a pointer, because the destination
|
|
||||||
has three states and a bare string has two. Absent is every intent but query,
|
|
||||||
which never reaches `queryWalk`. Present and empty is the `SourceUnknown`
|
|
||||||
contract: name nothing and walk the chain. Present and named is a destination the
|
|
||||||
route must produce. Thirty-three of ninety-six cases carry one.
|
|
||||||
|
|
||||||
A destination miss does **not** fail the case. It lands in `Outcome.SourceReason`
|
|
||||||
and never in `Reasons`, so `Accuracy` and `IntentAccuracy` mean what they meant
|
|
||||||
and `SourceAccuracy` is a second number over the labelled cases only. Intent and
|
|
||||||
destination are two decisions, and one number hides which one moved. A route that
|
|
||||||
lost its intent scores no destination hit, or a clarify would satisfy an empty
|
|
||||||
label for free.
|
|
||||||
|
|
||||||
Measured classifier+ONNX: intent **73/96 (76.0%)**, destination **12/33 (36.4%)**.
|
|
||||||
The split is the finding. World is 5/5, because a stage 0 rule names it. The
|
|
||||||
`SourceUnknown` floor is 5/7. Calendar is 2/6, because the possessive agenda
|
|
||||||
rules deliberately do not name it. And **recall is 0/15, because nothing
|
|
||||||
anywhere names it**. Those turns are still answered, since the chain walks
|
|
||||||
recall early. Recall is the number the fourth head has to move.
|
|
||||||
|
|
||||||
Seven cases assert the floor and five of them are homelab operations. They
|
|
||||||
cluster because `SourceRecall`, `SourceNetwork` and `SourceAttention` overlap on
|
|
||||||
every question about the box. The other two are `ru-query-005` and
|
|
||||||
`ru-query-014`. No query source reads the reminder store, and a deadline could
|
|
||||||
sit in tasks, the calendar or Praxis. `mavpoll` writes its netdata and uptime-kuma
|
|
||||||
observations into the fact store recall reads. That is a finding about the enum,
|
|
||||||
not a gap in the labelling. The owner confirmed all seven floor labels on
|
|
||||||
08-08-2026, so they are a decision rather than an agent's guess.
|
|
||||||
|
|
||||||
`baselineGrammars` in `eval_test.go` mirrors `buildRouter` and had drifted:
|
|
||||||
`WorldQueryGrammars` was wired into the daemon by V-655 and not into the mirror,
|
|
||||||
so the fixture scored a grammar set nobody runs. Fixed by V-659, worth 3 points of
|
|
||||||
destination and nothing else. Check that function when adding a grammar.
|
|
||||||
|
|
||||||
**The model arm landed the same day** (V-660,
|
|
||||||
`docs/evals/2026-08-08-destination-model-arm.md`). `routeGrammar` carries a
|
|
||||||
`source` rule closed over `router.Sources` plus the empty floor, so the model
|
|
||||||
cannot emit a destination that does not exist. The prompt lists the twelve in
|
|
||||||
Russian and says `""` is a normal answer to give often. `LLMRouter.Route` reads it
|
|
||||||
back through `ValidSource` and on `IntentQuery` alone. Against gemma-4-12b on the
|
|
||||||
workstation the cascade scores destination **24/33 (72.7%)** with intent unmoved
|
|
||||||
at 84.4%, and **recall goes 0/15 to 14/15**. The resident Qwen3-1.7B is
|
|
||||||
unmeasured, because it binds `--port 0` inside the container.
|
|
||||||
|
|
||||||
**Stage 0 now costs four destination points.** It did not before. The four cases
|
|
||||||
the cascade loses and the model alone wins are all calendar. The possessive
|
|
||||||
agenda rules claim them first and name nothing on purpose. That caution was free
|
|
||||||
while nothing downstream could name anything either. It is not free now, and the
|
|
||||||
fix is the owner's call rather than a quiet edit.
|
|
||||||
|
|
||||||
The last arm is V-546. Intent, mood and BIO slot tags were already three heads on
|
|
||||||
one forward pass of the resident e5-small. Destination is a fourth head on the
|
|
||||||
same pass, and 72.7% from a 12B teacher is the label source for training it.
|
|
||||||
|
|
||||||
## LLM output contract
|
|
||||||
|
|
||||||
All phrasing paths emit `{"response":"...","mood":"..."}`, with fallback to plain text when
|
|
||||||
the model skips the JSON. **One parser, `parseResponseMood` in
|
|
||||||
`internal/phraser/parse.go`**, and every path reaches it: the six `LLMPhraser` methods,
|
|
||||||
`PhraseWorld`, and `Replier.PhraseReply`, which `cmd/mavend/replier_llm.go` wraps — that file
|
|
||||||
holds the stub fallback and no parsing of its own. The legacy `{"body","summary"}` fallback
|
|
||||||
was deleted on 2026-08-06 (V-397): it was the contract before `{"response","mood"}` replaced
|
|
||||||
it, no prompt asks for that shape, the GBNF cannot emit it, and no test covered it.
|
|
||||||
Mood is a fixed enum. Router prompt is a separate contract:
|
|
||||||
`[{"intent":<enum>, key?, value?, text?, verb?}, ...]`, 7 intents (`fact, reminder,
|
|
||||||
note, query, act, chat, system`). `llm/check_prompt_parity.py` in the training
|
|
||||||
workspace enforces that the Go and relabelling prompts remain identical.
|
|
||||||
|
|
||||||
## Russian patterns — three mechanisms, no fourth
|
|
||||||
|
|
||||||
Hand-written Russian stem patterns were swept out on 2026-08-04 (owner's call: not a
|
|
||||||
pattern, and the resident model cannot be asked per turn either). A regex whose output is a
|
|
||||||
fact or a route is the defect; a regex over structured input — HTML, MIME, JSON, a URL, an
|
|
||||||
argv list — is not. Before writing a Russian word list, pick one of these:
|
|
||||||
|
|
||||||
- **`internal/lexicon`** — closed classes, in `lexicon_ru_v1.json`. Interrogatives,
|
|
||||||
capture verbs, reminder verbs, cardinals, day offsets, parts of day, weekdays, months,
|
|
||||||
spoken hours. Editing a word is a data change, and there is exactly one copy: months used
|
|
||||||
to live in three files. Cardinals carry the oblique forms, because a spoken time declines
|
|
||||||
and `в семь` / `к семи` are one hour.
|
|
||||||
- **`internal/morph`** — grammar, from the vendored golem Russian dictionary. `IsVerbForm`
|
|
||||||
and `SameWord`. Note that lemma matching is BROADER than stem-plus-one-ending, so a verb
|
|
||||||
slot that means the imperative must be matched exactly — `говори` and `говорил` are one
|
|
||||||
lemma and only one of them is a command (`cmd/mavend/quiet_toggle.go`).
|
|
||||||
- **`cmd/mavend/topics.go` and the embedder** — open sets, where the question is what a
|
|
||||||
turn is ABOUT. Frozen seeds per subject plus a real `other` class, scored against the
|
|
||||||
turn's own query vector. Same shape as the personal boundary in `personalboundary.go`,
|
|
||||||
with one difference: a topic must clear the runner-up by `topicMargin`, because a false
|
|
||||||
claim here spends a network scan rather than one honest "не знаю". The old keyword tests
|
|
||||||
stay as the offline floor and may remain narrow, since they are no longer the only answer.
|
|
||||||
- **The ecosystem trio** — when the answer is not in the utterance at all. Identity is
|
|
||||||
Nexus's, never a local pattern.
|
|
||||||
|
|
||||||
Seeds are scoring data. Editing one moves a recogniser and must be re-measured against the
|
|
||||||
`TestONNX*` tests, not eyeballed.
|
|
||||||
|
|
||||||
## Non-goals (hard constraints)
|
|
||||||
|
|
||||||
Not a nag, not autonomous. Maven's persona is **feminine** — Russian
|
|
||||||
self-reference must use feminine forms — `рада`, not `рад`; `поняла`, not `понял`. The owner
|
|
||||||
is male and is addressed informally: "ты", singular, never "вы"/"ваш" and never "он"/"его"
|
|
||||||
(she talks TO the owner, not about the owner). Pet names ("милый", "дорогой") are forbidden; the name
|
|
||||||
("Ками") is not. The eval enforces this: `CheckAddress`, `CheckFeminine` and `CheckCringe` in
|
|
||||||
`internal/phraser/eval/checks.go`, scored by `make eval-phrasing`.
|
|
||||||
|
|
||||||
**"Never phones home" is DEPRECATED** (owner's call, 2026-07-31). It used to be a hard
|
|
||||||
constraint and it is not one any more: a 0.8B — and a 1.7B — does not know enough to answer
|
|
||||||
world questions, so she needs to read external sources. What replaces it:
|
|
||||||
|
|
||||||
- **No telemetry, no cloud model, no third-party account.** That part never changes. Nothing
|
|
||||||
about Maven is reported to anyone, and inference stays on the box.
|
|
||||||
- **The owner's data first, then the world.** Every source that reads the owner's facts, notes, calendar,
|
|
||||||
tasks or house runs before anything outside, and the personal boundary sits between them.
|
|
||||||
Reading beats recalling for a small model.
|
|
||||||
- **In the world, live search leads and the ZIMs are the fallback** (owner's call,
|
|
||||||
2026-08-02). A self-hosted SearXNG (`search` block) answers first; the Kiwix ZIMs on
|
|
||||||
homesrv answer when the search is empty, unreachable, or the line is down.
|
|
||||||
**Verified with the line down on 2026-08-05** (V-508,
|
|
||||||
`docs/evals/2026-08-05-kiwix-offline-fallback.md`): a stopped SearXNG costs nothing,
|
|
||||||
the ZIM answers in the same turn budget. A blackholed host cost 8 seconds the owner waited
|
|
||||||
through. So the connect phase alone is capped at `dialTimeout` (1.5s), while a slow
|
|
||||||
instance that did connect keeps the full 8. **A Russian question reads
|
|
||||||
`wikipedia_ru_all_maxi_2026-02` verbatim** through `kiwix.book_ru`. The RU→EN rewriter
|
|
||||||
is the workaround for an English book and is skipped there. Kiwix catalog names come
|
|
||||||
from the filename, not the `<name>` field.
|
|
||||||
**That verbatim path sent the whole sentence to a keyword engine until 09-08-2026**
|
|
||||||
(V-668, `docs/evals/2026-08-09-kiwix-topic-retrieval.md`). Kiwix ranks by keyword
|
|
||||||
overlap, so the question words outrank the one word naming the article. "что такое TCP"
|
|
||||||
returned "Перехват TCP-соединения". "кто написал Войну и мир" returned an episode of
|
|
||||||
Doctor Who. `kiwix.Topic` drops the narrative request, the interrogative and a verb
|
|
||||||
behind one. `kiwix.TitlePath` tries the exact article first, since a ZIM is addressable
|
|
||||||
by title and a wrong title is a 404. Five of eight questions reach the right article
|
|
||||||
where they did not, one was already right, and nothing regressed. The title needs its
|
|
||||||
leading capital, so `TitleCandidates` tries the spoken form and then the capitalized
|
|
||||||
one. **"столица Франции" is answered by a title redirect to Париж**, which is the case
|
|
||||||
the 2026-08-05 measurement named as unreachable by any lexical signal. Both apply on the
|
|
||||||
verbatim path alone. The rewriter already reduces a question, and reducing twice takes
|
|
||||||
the topic off its input.
|
|
||||||
`Response.Empty()` is the whole gate and there is no quality threshold in front of it:
|
|
||||||
the three signals one could read were measured on 2026-08-05 and none of them separate a
|
|
||||||
real question from an invented one. Token overlap would cost "столица Франции" its
|
|
||||||
answer, because the answer is Париж and that word is not in the question. See
|
|
||||||
`docs/evals/2026-08-05-search-quality-signals.md` (V-539). **The embedder is not a
|
|
||||||
fourth signal**, measured 2026-08-09 (V-668). Query-to-passage cosine scores 0.79 to
|
|
||||||
0.91 on answerable questions and 0.75 to 0.84 on unanswerable ones, and the sets
|
|
||||||
overlap. The wrong TCP article scored 0.8653, above five of six unanswerable rows. It
|
|
||||||
measures topic and not whether the passage answers, so no threshold splits them. **Which query source claimed
|
|
||||||
a turn is readable on `/chat`** as a badge beside the reply, carried on
|
|
||||||
`ipc.ChatReply.Source` and noted by `noteQuerySource` in `cmd/mavend/querysource.go`. It
|
|
||||||
rides the context, so `handleText` keeps the one string signature the mic, telegram and
|
|
||||||
the web share.
|
|
||||||
- **External search is allowed and off unless configured**, like the weather and telegram
|
|
||||||
capabilities. The code default is still off. `deploy/mavend.json` now ships a `search`
|
|
||||||
block (owner's call, 2026-08-02), so it is on for this box and deleting the block turns
|
|
||||||
it off again.
|
|
||||||
- **The owner's notes and facts are never search input.** Looking up why the sky is blue and
|
|
||||||
sending the owner's stored personal notes to an upstream engine are different acts. Only the utterance goes
|
|
||||||
out, never the persona block, history, or matched notes.
|
|
||||||
|
|
||||||
## Web UI conventions
|
|
||||||
|
|
||||||
Server-rendered pages share `cmd/mavweb/static/ui.css` (served at `/ui.css`) and the shell
|
|
||||||
partial in `cmd/mavweb/shell.html`: a page opens with `{{template "shellTop" "<page-key>"}}`
|
|
||||||
and closes with `{{template "shellBottom"}}`, and the key marks the active sidebar link.
|
|
||||||
Every page is its own embedded `.html` file next to `main.go` — no page markup lives in Go,
|
|
||||||
and the sidebar is data (`sidebarSections`, `pageIcon`) the template renders. No
|
|
||||||
per-page `<style>` beyond true one-offs. Wrap every table in `<div class=scroll>` so wide
|
|
||||||
data pans on a phone. Local preview + headless screenshot recipe is in `AGENTS.md`.
|
|
||||||
|
|
||||||
## Vikunja
|
|
||||||
|
|
||||||
This repo is project **Maven** (ID 2) in Vikunja. MCP: `http://localhost:9100/mcp` (or
|
|
||||||
`http://192.168.1.104:9100/mcp` from workpc). Feature/bug/deploy tasks go there.
|
|
||||||
|
|
||||||
Vikunja is the durable task store. A task holds the goal, the constraints and the
|
|
||||||
assumption ledger. Work without a task id is work nobody can resume, so a session that
|
|
||||||
has no id asks for one before it starts.
|
|
||||||
|
|
||||||
The MCP tool schemas are deferred, so load the four you actually use in ONE call at the
|
|
||||||
start of a session rather than one lookup per first use:
|
|
||||||
|
|
||||||
```text
|
|
||||||
ToolSearch("select:mcp__vikunja__list_tasks,mcp__vikunja__get_task_details,mcp__vikunja__create_task,mcp__vikunja__update_task")
|
|
||||||
```
|
|
||||||
|
|
||||||
**Close a finished task with `done: true` and nothing else** (owner's call, 07-08-2026).
|
|
||||||
Do not write a completion summary into the description on the way out. It is lost anyway,
|
|
||||||
and the durable record is the commit messages and the merged PR. Note that `update_task`
|
|
||||||
carrying a `description` resets `done` to false, which is why a write-up ever took two
|
|
||||||
calls.
|
|
||||||
|
|
||||||
## Session workflow
|
## Session workflow
|
||||||
|
|
||||||
`~/.local/bin/task` owns the branch, the commit identity and the PR. One task, one
|
`docs/workflow.md` carries the five stores, the doc tiers and the guards. One
|
||||||
session, one PR.
|
task, one session, one PR. `/pickup` opens a session and `/wrap` closes it. Wrap
|
||||||
|
at roughly half context rather than letting the session compact.
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
task start <vikunja-id> # branch off origin/master, write TASK.md, fetch review comments
|
task start <vikunja-id> # branch off origin/master, write TASK.md, fetch review comments
|
||||||
task pr # push, open or refresh the PR, label Vikunja, notify
|
task pr # push, open or refresh the PR, label Vikunja, notify
|
||||||
task comments # re-pull this branch's review comments into .task/
|
ToolSearch("select:mcp__vikunja__list_tasks,mcp__vikunja__get_task_details,mcp__vikunja__create_task,mcp__vikunja__update_task")
|
||||||
```
|
```
|
||||||
|
|
||||||
Around that, `/pickup` opens a session and `/wrap` closes it. Wrap at roughly half
|
- This repo is Vikunja project **Maven** (ID 2), MCP at
|
||||||
context rather than letting the session compact.
|
`http://localhost:9100/mcp`, or `http://192.168.1.104:9100/mcp` from workpc.
|
||||||
|
- **A session with no task id asks for one before it starts**, because work
|
||||||
Five stores, and each one owns something the others must not hold:
|
without one is work nobody can resume.
|
||||||
|
- **Close a finished task with `done: true` and nothing else** (owner's call,
|
||||||
| Store | Holds | Lifetime |
|
2026-08-07). `update_task` carrying a `description` resets `done` to false.
|
||||||
|---|---|---|
|
- **`pre-commit` refuses master** and more than 300 changed lines in
|
||||||
| Vikunja task | goal, constraints, assumption ledger, status | durable |
|
non-markdown files. Markdown is exempt and may land as one batch.
|
||||||
| `CLAUDE.md`, `AGENTS.md` | what an agent must know before touching code | durable |
|
- **`commit-msg` requires the subject to end with `(V-<id>)`.** `V-` and not
|
||||||
| `docs/` | design, measurements, decisions | durable |
|
`#`, because Gitea autolinks `#123` to the wrong tracker.
|
||||||
| `TASK.md` | the brief for this branch, written by `task start`, immutable | one branch |
|
- **`diff-budget.sh` blocks edits past 600 changed lines** on a `task/` branch.
|
||||||
| `HANDOFF.md` | only what the next agent needs to resume | one session |
|
- **`--no-verify` exists.** Using it means saying why in the commit body.
|
||||||
|
|
||||||
`TASK.md` and `.task/` are excluded through `.git/info/exclude`. `HANDOFF.md` is
|
|
||||||
gitignored and injected at session start. If a line in the handoff would still matter
|
|
||||||
next week, it is in the wrong file.
|
|
||||||
|
|
||||||
Docs are tiered by path, so staleness is visible from the filename. Files directly under
|
|
||||||
`docs/` are living and carry a `Last verified: <date> @ <sha>` line. Files under
|
|
||||||
`docs/evals/` are dated measurements and are never edited after the day, so a newer
|
|
||||||
number is a new file. Files under `docs/archive/` are dead and read by nobody by default.
|
|
||||||
|
|
||||||
## Git guards
|
|
||||||
|
|
||||||
Two hooks in `.githooks/`, tracked, wired with `core.hooksPath`. Fresh clone:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
git config core.hooksPath .githooks
|
|
||||||
```
|
|
||||||
|
|
||||||
- `pre-commit` refuses master, and refuses more than 300 changed lines in non-markdown
|
|
||||||
files. Markdown is exempt and may land as one batch.
|
|
||||||
- `commit-msg` requires the subject to end with `(V-<id>)`. `V-` and not `#`, because
|
|
||||||
Gitea autolinks `#123` to a Gitea issue, which is the wrong tracker.
|
|
||||||
|
|
||||||
Two more guards live outside the repo, in `~/.claude/hooks/`. `diff-budget.sh` blocks
|
|
||||||
further edits past 600 changed lines on a `task/` branch. `prose_lint_hook.py` checks
|
|
||||||
prose on every write. Both measure against `origin/master`, so a local master that is
|
|
||||||
ahead of the remote makes the diff budget read high.
|
|
||||||
|
|
||||||
`--no-verify` exists. Using it means saying why in the commit body.
|
|
||||||
|
|||||||
+47
-5
@@ -12,10 +12,17 @@
|
|||||||
// samples and the capture frame is 480, so silero.go re-chunks. This comment
|
// samples and the capture frame is 480, so silero.go re-chunks. This comment
|
||||||
// used to say the two matched, which was true of silero v4.
|
// used to say the two matched, which was true of silero v4.
|
||||||
//
|
//
|
||||||
// There is still no wake-word model, so anything spoken near the microphone
|
// The keyword is "Мэйвен" and it is required, when -wake-model points at the
|
||||||
// becomes a turn (V-487 stage two). The SurfaceVoice auth layer caps all
|
// head (V-487 stage two). Without it anything spoken near the microphone
|
||||||
// commands at L0 (no destructive acts), which is what makes an accidental
|
// becomes a turn, which the SurfaceVoice auth layer makes safe rather than
|
||||||
// trigger safe rather than expensive.
|
// expensive: it caps all commands at L0, no destructive acts. It does not cap
|
||||||
|
// reading, so an open gate still lets the room hear his facts read back.
|
||||||
|
// wakeword.go holds the cadence and wakefeatures.go the three models.
|
||||||
|
//
|
||||||
|
// The conn carries both directions. mavwaked sends utterances and receives
|
||||||
|
// proactive nudges on it, and it is opened at startup rather than at the first
|
||||||
|
// utterance, because mavend registers a voice session on accept. See nudge.go
|
||||||
|
// for why a nudge that is not heard is worse than one that is not delivered.
|
||||||
//
|
//
|
||||||
// While a reply is playing the capture side is muted (half-duplex): without
|
// While a reply is playing the capture side is muted (half-duplex): without
|
||||||
// it, Maven's own voice comes back in through the mic and she answers
|
// it, Maven's own voice comes back in through the mic and she answers
|
||||||
@@ -57,6 +64,12 @@ const (
|
|||||||
defaultAddr = "127.0.0.1:9100"
|
defaultAddr = "127.0.0.1:9100"
|
||||||
defaultLang = "ru"
|
defaultLang = "ru"
|
||||||
defaultReadSize = 4096 // max PCM bytes per read from arecord (fits multiple frames)
|
defaultReadSize = 4096 // max PCM bytes per read from arecord (fits multiple frames)
|
||||||
|
|
||||||
|
// defaultWakeWindowMs — how long the keyword stays good for. He says
|
||||||
|
// "Мэйвен" and then a sentence, and the VAD does not close the utterance
|
||||||
|
// until he stops, so this has to outlive the word by the length of what
|
||||||
|
// follows it. It is spent on dispatch: one keyword, one turn.
|
||||||
|
defaultWakeWindowMs = 8000
|
||||||
)
|
)
|
||||||
|
|
||||||
func main() {
|
func main() {
|
||||||
@@ -81,12 +94,18 @@ func run(args []string) error {
|
|||||||
vadModel := flag.String("vad-model", "", "silero-vad onnx file; empty runs the energy threshold instead")
|
vadModel := flag.String("vad-model", "", "silero-vad onnx file; empty runs the energy threshold instead")
|
||||||
vadThreshold := flag.Float64("vad-threshold", defaultSileroThreshold, "speech probability a frame must clear")
|
vadThreshold := flag.Float64("vad-threshold", defaultSileroThreshold, "speech probability a frame must clear")
|
||||||
onnxLib := flag.String("onnx-lib", os.Getenv("MAVEN_ONNX_LIB"), "libonnxruntime.so, needed with -vad-model")
|
onnxLib := flag.String("onnx-lib", os.Getenv("MAVEN_ONNX_LIB"), "libonnxruntime.so, needed with -vad-model")
|
||||||
|
wakeModel := flag.String("wake-model", "", "keyword head onnx; empty ships every utterance, as before V-487")
|
||||||
|
wakeMel := flag.String("wake-mel", "", "melspectrogram.onnx, required with -wake-model")
|
||||||
|
wakeEmbed := flag.String("wake-embed", "", "embedding_model.onnx, required with -wake-model")
|
||||||
|
wakeThreshold := flag.Float64("wake-threshold", defaultWakeThreshold, "score the keyword must clear")
|
||||||
|
wakeWindowMs := flag.Int("wake-window-ms", defaultWakeWindowMs, "ms an utterance may still start after the keyword")
|
||||||
flag.CommandLine.Parse(args)
|
flag.CommandLine.Parse(args)
|
||||||
|
|
||||||
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM, syscall.SIGHUP)
|
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM, syscall.SIGHUP)
|
||||||
defer stop()
|
defer stop()
|
||||||
|
|
||||||
// Voice client — reused across utterances; SendRequest reconnects on error.
|
// Voice client — one conn carrying both directions. SendRequest reconnects
|
||||||
|
// on error, and the push receiver redials on its own clock.
|
||||||
vc := voice.Dial(*addr)
|
vc := voice.Dial(*addr)
|
||||||
defer vc.Close()
|
defer vc.Close()
|
||||||
|
|
||||||
@@ -165,6 +184,29 @@ func run(args []string) error {
|
|||||||
}
|
}
|
||||||
sess := newSession(vad, newAplayPlayer(), &voiceSender{vc: vc}, *lang, barge)
|
sess := newSession(vad, newAplayPlayer(), &voiceSender{vc: vc}, *lang, barge)
|
||||||
|
|
||||||
|
// Keyword gate. A model that will not load is logged and not fatal, for
|
||||||
|
// the same reason silero's is not: an open gate is the daemon he had
|
||||||
|
// yesterday, and a daemon that refuses to start is not.
|
||||||
|
if *wakeModel != "" {
|
||||||
|
w, err := newWakeWord(*wakeMel, *wakeEmbed, *wakeModel, *onnxLib, *wakeThreshold)
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("mavwaked: wake word unavailable, every utterance is a turn: %v", err)
|
||||||
|
} else {
|
||||||
|
defer w.Close()
|
||||||
|
sess.UseWakeWord(w, time.Duration(*wakeWindowMs)*time.Millisecond)
|
||||||
|
log.Printf("mavwaked: wake word from %s, threshold %.3f, window %dms",
|
||||||
|
*wakeModel, *wakeThreshold, *wakeWindowMs)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Listen for nudges alongside capture. Connect eagerly so mavend has a
|
||||||
|
// voice session before he has said anything: without one, a nudge routed
|
||||||
|
// to voice finds nobody home and goes to the away channels instead.
|
||||||
|
if err := vc.Connect(ctx); err != nil {
|
||||||
|
log.Printf("mavwaked: voice server not reachable yet, retrying in background: %v", err)
|
||||||
|
}
|
||||||
|
go runNudgeReceiver(ctx, vc, sess)
|
||||||
|
|
||||||
return captureLoop(ctx, src, sess)
|
return captureLoop(ctx, src, sess)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,79 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
// The receiving half of the voice reach (V-671).
|
||||||
|
//
|
||||||
|
// mavwaked used to send and never listen. It wired no PushHandler, and
|
||||||
|
// SendRequest discards a push frame when there is none. The consequence was
|
||||||
|
// not a missing feature but a silent one: mavend routes a nudge to the voice
|
||||||
|
// session that spoke most recently, and once mavwaked had spoken once it WAS
|
||||||
|
// that session. PushToMostRecent succeeded, the dispatcher counted the nudge
|
||||||
|
// delivered and stopped rerouting to telegram and ntfy, and mavwaked threw the
|
||||||
|
// audio away. He heard nothing, anywhere.
|
||||||
|
//
|
||||||
|
// So the connection is opened at startup rather than at the first utterance,
|
||||||
|
// and it is held open. A client that has never connected has no session, and
|
||||||
|
// the dispatcher must be able to tell "he is not at the machine" from "he is,
|
||||||
|
// and she has nothing to say".
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"log"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/kami/maven/internal/voice"
|
||||||
|
)
|
||||||
|
|
||||||
|
// nudgeRetry is how long to wait before dialling again after the conn ends.
|
||||||
|
// mavend restarts on every deploy, and a listener that gives up then is a
|
||||||
|
// listener that is deaf until the next reboot.
|
||||||
|
const nudgeRetry = 5 * time.Second
|
||||||
|
|
||||||
|
// nudgeHandler decodes a push and hands the audio to the session, which
|
||||||
|
// speaks it through the same player the reply path uses. It does not play
|
||||||
|
// anything itself: the half-duplex gate and barge-in live on the capture
|
||||||
|
// loop, and a nudge has to sit under both.
|
||||||
|
type nudgeHandler struct{ sess *session }
|
||||||
|
|
||||||
|
func (h *nudgeHandler) OnPush(p voice.Push) {
|
||||||
|
if p.Kind != voice.PushKindAudioNudge {
|
||||||
|
log.Printf("mavwaked: ignoring push of unknown kind %q", p.Kind)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
var ap voice.AudioNudgePush
|
||||||
|
if err := json.Unmarshal(p.Params, &ap); err != nil {
|
||||||
|
log.Printf("mavwaked: nudge: decode: %v", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
log.Printf("mavwaked: nudge from rule %q (severity %d): %q (%.2fs audio)",
|
||||||
|
ap.RuleName, ap.Severity, ap.Text, ap.Audio.Duration())
|
||||||
|
if len(ap.Audio.Bytes) == 0 {
|
||||||
|
// mavttsd was down or the text was empty. Say so rather than going
|
||||||
|
// quiet: the dispatcher already counted this one as delivered.
|
||||||
|
log.Printf("mavwaked: nudge %q carried no audio, nothing to speak", ap.RuleName)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.sess.Nudge(ap.Audio)
|
||||||
|
}
|
||||||
|
|
||||||
|
// runNudgeReceiver keeps a push handler wired for as long as ctx lives,
|
||||||
|
// redialling whenever the conn ends. Returns when ctx is cancelled.
|
||||||
|
func runNudgeReceiver(ctx context.Context, vc *voice.Client, sess *session) {
|
||||||
|
h := &nudgeHandler{sess: sess}
|
||||||
|
for {
|
||||||
|
err := vc.RunPushReceiver(ctx, h)
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("mavwaked: nudge receiver: %v", err)
|
||||||
|
} else {
|
||||||
|
log.Printf("mavwaked: voice connection ended, reconnecting in %s", nudgeRetry)
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
case <-time.After(nudgeRetry):
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,170 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
// The receiving half: a nudge pushed by mavend has to reach the speaker, and
|
||||||
|
// it has to obey the same two gates a reply obeys (V-671).
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/kami/maven/internal/audio"
|
||||||
|
"github.com/kami/maven/internal/voice"
|
||||||
|
)
|
||||||
|
|
||||||
|
func nudgeAudio() audio.Audio {
|
||||||
|
return audio.Audio{Format: audio.PCM16kMono, Bytes: make([]byte, 8000)}
|
||||||
|
}
|
||||||
|
|
||||||
|
// pushFrame builds the frame mavend's voicesink sends.
|
||||||
|
func pushFrame(t *testing.T, a audio.Audio) voice.Push {
|
||||||
|
t.Helper()
|
||||||
|
body, err := json.Marshal(voice.AudioNudgePush{
|
||||||
|
RuleName: "test-rule",
|
||||||
|
Severity: 3,
|
||||||
|
Audio: a,
|
||||||
|
Text: "пора пить воду",
|
||||||
|
Ts: time.Unix(0, 0),
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("marshal push: %v", err)
|
||||||
|
}
|
||||||
|
return voice.Push{Kind: voice.PushKindAudioNudge, Params: body}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The defect itself: the push arrived and nothing came out of the speaker.
|
||||||
|
func TestNudgeReachesThePlayer(t *testing.T) {
|
||||||
|
sess, p, snd := newTestSession(bargeInConfig{})
|
||||||
|
(&nudgeHandler{sess: sess}).OnPush(pushFrame(t, nudgeAudio()))
|
||||||
|
|
||||||
|
if p.plays != 0 {
|
||||||
|
t.Fatal("nudge played from the push goroutine; it must wait for the capture loop")
|
||||||
|
}
|
||||||
|
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||||
|
t.Fatalf("feed: %v", err)
|
||||||
|
}
|
||||||
|
if p.plays != 1 {
|
||||||
|
t.Fatalf("plays = %d, want 1", p.plays)
|
||||||
|
}
|
||||||
|
if len(p.last.Bytes) != 8000 {
|
||||||
|
t.Errorf("played %d bytes, want the nudge audio", len(p.last.Bytes))
|
||||||
|
}
|
||||||
|
if sess.nudges != 1 {
|
||||||
|
t.Errorf("nudges = %d, want 1", sess.nudges)
|
||||||
|
}
|
||||||
|
if len(snd.sent) != 0 {
|
||||||
|
t.Errorf("a nudge must not be shipped back to the daemon as an utterance")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A push of some other kind, or one carrying no audio, must not reach the
|
||||||
|
// player and must not wedge the one that follows.
|
||||||
|
func TestNudgeIgnoresUnusablePushes(t *testing.T) {
|
||||||
|
sess, p, _ := newTestSession(bargeInConfig{})
|
||||||
|
h := &nudgeHandler{sess: sess}
|
||||||
|
|
||||||
|
h.OnPush(voice.Push{Kind: "something-else", Params: json.RawMessage(`{}`)})
|
||||||
|
h.OnPush(voice.Push{Kind: voice.PushKindAudioNudge, Params: json.RawMessage(`not json`)})
|
||||||
|
h.OnPush(pushFrame(t, audio.Audio{Format: audio.PCM16kMono}))
|
||||||
|
|
||||||
|
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||||
|
t.Fatalf("feed: %v", err)
|
||||||
|
}
|
||||||
|
if p.plays != 0 {
|
||||||
|
t.Fatalf("plays = %d, want 0", p.plays)
|
||||||
|
}
|
||||||
|
|
||||||
|
h.OnPush(pushFrame(t, nudgeAudio()))
|
||||||
|
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||||
|
t.Fatalf("feed: %v", err)
|
||||||
|
}
|
||||||
|
if p.plays != 1 {
|
||||||
|
t.Fatalf("plays after a usable nudge = %d, want 1", p.plays)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The half-duplex gate covers a nudge exactly as it covers a reply: she does
|
||||||
|
// not start one over herself, and the mic stays muted while it runs.
|
||||||
|
func TestNudgeWaitsForTheReplyToFinish(t *testing.T) {
|
||||||
|
sess, p, _ := newTestSession(bargeInConfig{})
|
||||||
|
speakThenPause(t, sess)
|
||||||
|
if !p.Playing() {
|
||||||
|
t.Fatal("expected the reply to be playing")
|
||||||
|
}
|
||||||
|
plays := p.plays
|
||||||
|
|
||||||
|
(&nudgeHandler{sess: sess}).OnPush(pushFrame(t, nudgeAudio()))
|
||||||
|
for i := 0; i < 20; i++ {
|
||||||
|
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||||
|
t.Fatalf("feed: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if p.plays != plays {
|
||||||
|
t.Fatalf("nudge cut across the reply: plays = %d, want %d", p.plays, plays)
|
||||||
|
}
|
||||||
|
|
||||||
|
p.playing = false
|
||||||
|
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||||
|
t.Fatalf("feed: %v", err)
|
||||||
|
}
|
||||||
|
if p.plays != plays+1 {
|
||||||
|
t.Fatalf("nudge never played after the reply ended: plays = %d", p.plays)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Speaking a nudge must not leave half a sentence in the VAD. The frames
|
||||||
|
// captured before it are pre-nudge speech, and splicing them onto whatever he
|
||||||
|
// says afterwards ships one utterance that is two.
|
||||||
|
func TestNudgeResetsTheVAD(t *testing.T) {
|
||||||
|
sess, p, snd := newTestSession(bargeInConfig{})
|
||||||
|
loud := frameAt(0.35)
|
||||||
|
speechFrames := (defaultSpeechMs + defaultFrameMs - 1) / defaultFrameMs
|
||||||
|
for i := 0; i < speechFrames+5; i++ {
|
||||||
|
if err := sess.feed(context.Background(), loud); err != nil {
|
||||||
|
t.Fatalf("feed: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
(&nudgeHandler{sess: sess}).OnPush(pushFrame(t, nudgeAudio()))
|
||||||
|
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||||
|
t.Fatalf("feed: %v", err)
|
||||||
|
}
|
||||||
|
if p.plays != 1 {
|
||||||
|
t.Fatalf("nudge did not play: plays = %d", p.plays)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Playback ends, silence follows. The half-formed utterance must be gone
|
||||||
|
// rather than closing on the first quiet frame.
|
||||||
|
p.playing = false
|
||||||
|
silenceFrames := (defaultSilenceMs+defaultFrameMs-1)/defaultFrameMs + 2
|
||||||
|
for i := 0; i < silenceFrames; i++ {
|
||||||
|
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||||
|
t.Fatalf("feed: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(snd.sent) != 0 {
|
||||||
|
t.Fatalf("sent %d utterances after a nudge, want 0", len(snd.sent))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Two nudges queued back to back: the newer one is what he hears. The
|
||||||
|
// PushHandler contract in internal/voice says the next nudge replaces the
|
||||||
|
// stale one rather than dogpiling on it.
|
||||||
|
func TestNudgeReplacesAnUnspokenOne(t *testing.T) {
|
||||||
|
sess, p, _ := newTestSession(bargeInConfig{})
|
||||||
|
h := &nudgeHandler{sess: sess}
|
||||||
|
|
||||||
|
h.OnPush(pushFrame(t, audio.Audio{Format: audio.PCM16kMono, Bytes: make([]byte, 4000)}))
|
||||||
|
h.OnPush(pushFrame(t, audio.Audio{Format: audio.PCM16kMono, Bytes: make([]byte, 12000)}))
|
||||||
|
|
||||||
|
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||||
|
t.Fatalf("feed: %v", err)
|
||||||
|
}
|
||||||
|
if p.plays != 1 {
|
||||||
|
t.Fatalf("plays = %d, want 1", p.plays)
|
||||||
|
}
|
||||||
|
if len(p.last.Bytes) != 12000 {
|
||||||
|
t.Errorf("played %d bytes, want the newer nudge", len(p.last.Bytes))
|
||||||
|
}
|
||||||
|
}
|
||||||
+136
-1
@@ -7,6 +7,7 @@ package main
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"log"
|
"log"
|
||||||
|
"sync"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/kami/maven/internal/audio"
|
"github.com/kami/maven/internal/audio"
|
||||||
@@ -19,6 +20,15 @@ type utteranceSender interface {
|
|||||||
Send(ctx context.Context, utt audio.Audio, lang string) (audio.Audio, error)
|
Send(ctx context.Context, utt audio.Audio, lang string) (audio.Audio, error)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// keywordGate answers whether the keyword has just been spoken. The
|
||||||
|
// production one is wakeWord; tests substitute a recorder, because a gate that
|
||||||
|
// can only be exercised with three ONNX files is a gate nobody tests.
|
||||||
|
type keywordGate interface {
|
||||||
|
Feed(frame []int16) bool
|
||||||
|
Reset()
|
||||||
|
Score() float64
|
||||||
|
}
|
||||||
|
|
||||||
// bargeInConfig holds the two numbers barge-in needs. Zero Frames disables
|
// bargeInConfig holds the two numbers barge-in needs. Zero Frames disables
|
||||||
// barge-in entirely — the half-duplex gate still runs.
|
// barge-in entirely — the half-duplex gate still runs.
|
||||||
type bargeInConfig struct {
|
type bargeInConfig struct {
|
||||||
@@ -62,11 +72,29 @@ type session struct {
|
|||||||
// whenever playback ends.
|
// whenever playback ends.
|
||||||
loudFrames int
|
loudFrames int
|
||||||
|
|
||||||
|
// wake is the keyword gate, or nil when no model was loaded. wakeUntil is
|
||||||
|
// how long a keyword stays good for: he says "Мэйвен" and then a sentence,
|
||||||
|
// and the VAD does not close the utterance until he stops, so the window
|
||||||
|
// has to outlive the word by the length of what follows it.
|
||||||
|
wake keywordGate
|
||||||
|
wakeWindow time.Duration
|
||||||
|
wakeUntil time.Time
|
||||||
|
|
||||||
|
// pending holds a nudge the push receiver handed over, waiting for the
|
||||||
|
// capture loop to speak it. It is the one field written from another
|
||||||
|
// goroutine, hence the mutex; everything else in this struct belongs to
|
||||||
|
// the capture loop alone.
|
||||||
|
nudgeMu sync.Mutex
|
||||||
|
pending *audio.Audio
|
||||||
|
|
||||||
// counters, read by tests and logged on the way out.
|
// counters, read by tests and logged on the way out.
|
||||||
suppressed int // frames dropped because she was speaking
|
suppressed int // frames dropped because she was speaking
|
||||||
dropped int // frames dropped as round-trip backlog
|
dropped int // frames dropped as round-trip backlog
|
||||||
bargeIns int // times playback was cut because he spoke over her
|
bargeIns int // times playback was cut because he spoke over her
|
||||||
sent int // utterances shipped to the daemon
|
sent int // utterances shipped to the daemon
|
||||||
|
nudges int // proactive pushes spoken through the speaker
|
||||||
|
wakes int // times the keyword opened the gate
|
||||||
|
ignored int // complete utterances dropped because the keyword was absent
|
||||||
|
|
||||||
// loudSum and loudSeen accumulate the energy of suppressed frames, so
|
// loudSum and loudSeen accumulate the energy of suppressed frames, so
|
||||||
// the operator can read what the room actually measures and set
|
// the operator can read what the room actually measures and set
|
||||||
@@ -79,6 +107,12 @@ func newSession(vad *VAD, p player, s utteranceSender, lang string, barge bargeI
|
|||||||
return &session{vad: vad, player: p, sender: s, lang: lang, barge: barge, now: time.Now}
|
return &session{vad: vad, player: p, sender: s, lang: lang, barge: barge, now: time.Now}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// UseWakeWord puts the keyword gate in front of dispatch. Without it every
|
||||||
|
// utterance is shipped, which is what mavwaked did before V-487 stage two.
|
||||||
|
func (s *session) UseWakeWord(w keywordGate, window time.Duration) {
|
||||||
|
s.wake, s.wakeWindow = w, window
|
||||||
|
}
|
||||||
|
|
||||||
// frameDuration is the wall time one captured frame represents.
|
// frameDuration is the wall time one captured frame represents.
|
||||||
const frameDuration = defaultFrameMs * time.Millisecond
|
const frameDuration = defaultFrameMs * time.Millisecond
|
||||||
|
|
||||||
@@ -138,6 +172,7 @@ func (s *session) feed(ctx context.Context, frame []byte) error {
|
|||||||
s.bargeIns++
|
s.bargeIns++
|
||||||
s.loudFrames = 0
|
s.loudFrames = 0
|
||||||
s.vad.Reset()
|
s.vad.Reset()
|
||||||
|
s.resetWake()
|
||||||
log.Printf("mavwaked: barge-in — stopped playback")
|
log.Printf("mavwaked: barge-in — stopped playback")
|
||||||
s.replayRecent()
|
s.replayRecent()
|
||||||
return nil
|
return nil
|
||||||
@@ -148,15 +183,101 @@ func (s *session) feed(ctx context.Context, frame []byte) error {
|
|||||||
if s.loudFrames != 0 {
|
if s.loudFrames != 0 {
|
||||||
s.loudFrames = 0
|
s.loudFrames = 0
|
||||||
s.vad.Reset()
|
s.vad.Reset()
|
||||||
|
// The wake word saw nothing during playback, so what it holds is from
|
||||||
|
// before she spoke. Judging what he says next on it would score a
|
||||||
|
// sentence that ended a reply ago.
|
||||||
|
s.resetWake()
|
||||||
}
|
}
|
||||||
|
|
||||||
utt, state := s.vad.Feed(PCMToI16(frame))
|
if s.startPendingNudge() {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// The keyword is scored on the same frames the VAD sees, and only on the
|
||||||
|
// ones that reach here: every path above returns while she is speaking, so
|
||||||
|
// her own voice saying "Мэйвен" cannot wake her.
|
||||||
|
pcm := PCMToI16(frame)
|
||||||
|
if s.wake != nil && s.wake.Feed(pcm) {
|
||||||
|
s.wakes++
|
||||||
|
s.wakeUntil = s.now().Add(s.wakeWindow)
|
||||||
|
log.Printf("mavwaked: keyword heard (score %.3f), listening for %s",
|
||||||
|
s.wake.Score(), s.wakeWindow)
|
||||||
|
}
|
||||||
|
|
||||||
|
utt, state := s.vad.Feed(pcm)
|
||||||
if state == StateSpeech || utt.Bytes == nil {
|
if state == StateSpeech || utt.Bytes == nil {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
return s.dispatch(ctx, utt)
|
return s.dispatch(ctx, utt)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Nudge hands proactive audio to the session, to be spoken as soon as the
|
||||||
|
// capture loop finds a quiet moment. Safe to call from the push receiver
|
||||||
|
// goroutine; nothing else here is.
|
||||||
|
//
|
||||||
|
// A nudge arriving while one is already waiting REPLACES it. That is the
|
||||||
|
// contract internal/voice states for PushHandler: the next nudge replaces the
|
||||||
|
// stale one in his attention rather than dogpiling on it.
|
||||||
|
func (s *session) Nudge(a audio.Audio) {
|
||||||
|
if len(a.Bytes) == 0 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.nudgeMu.Lock()
|
||||||
|
if s.pending != nil {
|
||||||
|
log.Printf("mavwaked: nudge replaced one still waiting to be spoken")
|
||||||
|
}
|
||||||
|
s.pending = &a
|
||||||
|
s.nudgeMu.Unlock()
|
||||||
|
}
|
||||||
|
|
||||||
|
// takeNudge removes and returns the waiting nudge, or nil.
|
||||||
|
func (s *session) takeNudge() *audio.Audio {
|
||||||
|
s.nudgeMu.Lock()
|
||||||
|
defer s.nudgeMu.Unlock()
|
||||||
|
a := s.pending
|
||||||
|
s.pending = nil
|
||||||
|
return a
|
||||||
|
}
|
||||||
|
|
||||||
|
// startPendingNudge speaks a waiting nudge and reports whether it started
|
||||||
|
// one. It runs on the capture loop, past the half-duplex gate, so a nudge
|
||||||
|
// never cuts across a reply and never plays into a backlog drain.
|
||||||
|
//
|
||||||
|
// The VAD is reset first. Playback is about to suppress every frame until it
|
||||||
|
// ends, and a half-heard sentence left in the VAD would splice onto whatever
|
||||||
|
// he says afterwards. Barge-in needs no special case: it reads the player,
|
||||||
|
// and the player does not care which audio it is playing.
|
||||||
|
func (s *session) startPendingNudge() bool {
|
||||||
|
a := s.takeNudge()
|
||||||
|
if a == nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
s.vad.Reset()
|
||||||
|
s.nudges++
|
||||||
|
log.Printf("mavwaked: speaking nudge (%.2fs audio)", a.Duration())
|
||||||
|
s.player.Play(*a)
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
// awake reports whether an utterance ending now was addressed to her.
|
||||||
|
//
|
||||||
|
// With no wake word loaded every utterance is, which is exactly what mavwaked
|
||||||
|
// did before this gate existed. An operator with no model file gets the old
|
||||||
|
// daemon rather than a daemon that refuses to hear anything.
|
||||||
|
func (s *session) awake() bool {
|
||||||
|
if s.wake == nil {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return s.now().Before(s.wakeUntil)
|
||||||
|
}
|
||||||
|
|
||||||
|
// resetWake drops the gate's streaming state when there is a gate.
|
||||||
|
func (s *session) resetWake() {
|
||||||
|
if s.wake != nil {
|
||||||
|
s.wake.Reset()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// keepRecent stores a copy of one barge-in trigger frame, keeping at most
|
// keepRecent stores a copy of one barge-in trigger frame, keeping at most
|
||||||
// barge.Frames of them.
|
// barge.Frames of them.
|
||||||
func (s *session) keepRecent(frame []byte) {
|
func (s *session) keepRecent(frame []byte) {
|
||||||
@@ -197,6 +318,19 @@ func (s *session) replayRecent() {
|
|||||||
// whole backlog straight into the VAD, and a Send error did the same on every
|
// whole backlog straight into the VAD, and a Send error did the same on every
|
||||||
// failed turn, so a dead socket drove a retry loop off nothing but backlog.
|
// failed turn, so a dead socket drove a retry loop off nothing but backlog.
|
||||||
func (s *session) dispatch(ctx context.Context, utt audio.Audio) error {
|
func (s *session) dispatch(ctx context.Context, utt audio.Audio) error {
|
||||||
|
if !s.awake() {
|
||||||
|
s.ignored++
|
||||||
|
log.Printf("mavwaked: utterance ignored, keyword not heard (%.2fs, %d ignored so far)",
|
||||||
|
utt.Duration(), s.ignored)
|
||||||
|
s.vad.Reset()
|
||||||
|
s.resetWake()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
// One keyword, one turn. A window that renewed itself on every reply would
|
||||||
|
// leave the microphone open for as long as he kept talking, which is the
|
||||||
|
// state this gate exists to end.
|
||||||
|
s.wakeUntil = time.Time{}
|
||||||
|
|
||||||
log.Printf("mavwaked: utterance complete (%.2fs, %d bytes), sending...", utt.Duration(), len(utt.Bytes))
|
log.Printf("mavwaked: utterance complete (%.2fs, %d bytes), sending...", utt.Duration(), len(utt.Bytes))
|
||||||
start := s.now()
|
start := s.now()
|
||||||
reply, err := s.sender.Send(ctx, utt, s.lang)
|
reply, err := s.sender.Send(ctx, utt, s.lang)
|
||||||
@@ -225,6 +359,7 @@ func (s *session) dispatch(ctx context.Context, utt audio.Audio) error {
|
|||||||
// recorded before she started speaking.
|
// recorded before she started speaking.
|
||||||
func (s *session) dropBacklog(start time.Time) {
|
func (s *session) dropBacklog(start time.Time) {
|
||||||
s.vad.Reset()
|
s.vad.Reset()
|
||||||
|
s.resetWake()
|
||||||
s.loudFrames = 0
|
s.loudFrames = 0
|
||||||
s.recent = s.recent[:0]
|
s.recent = s.recent[:0]
|
||||||
if elapsed := s.now().Sub(start); elapsed > 0 {
|
if elapsed := s.now().Sub(start); elapsed > 0 {
|
||||||
|
|||||||
@@ -0,0 +1,202 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
// The three models behind the wake word (V-487 stage two).
|
||||||
|
//
|
||||||
|
// openWakeWord's pipeline, run in a row:
|
||||||
|
//
|
||||||
|
// audio -> melspectrogram.onnx -> 32-bin mel frames, one per 10ms
|
||||||
|
// 76 frames -> embedding_model.onnx -> one 96-dim embedding per 80ms
|
||||||
|
// 16 embeds -> maven_wakeword.onnx -> one score
|
||||||
|
//
|
||||||
|
// The first two are frozen and pretrained. Only the last was trained here,
|
||||||
|
// which is why it is 100KB and the other two are megabytes. The shapes are
|
||||||
|
// not guesses: 2.0s of 16kHz audio measures 197 mel frames, and 76-frame
|
||||||
|
// windows at stride 8 give exactly the 16 embeddings the head was fitted on.
|
||||||
|
//
|
||||||
|
// This file knows ONNX and nothing about the 80ms cadence. wakeword.go knows
|
||||||
|
// the cadence and nothing about tensors.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
|
||||||
|
ort "github.com/yalue/onnxruntime_go"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
// melHop — samples per mel frame. 10ms at 16kHz.
|
||||||
|
melHop = 160
|
||||||
|
// melBins — mel bins per frame, fixed by melspectrogram.onnx.
|
||||||
|
melBins = 32
|
||||||
|
// embedFrames — mel frames one embedding is computed over, 760ms.
|
||||||
|
embedFrames = 76
|
||||||
|
// embedStride — mel frames between embeddings, 80ms.
|
||||||
|
embedStride = 8
|
||||||
|
// embedDim — the embedding width.
|
||||||
|
embedDim = 96
|
||||||
|
// headWindow — embeddings the head scores at once, 1.28s of audio.
|
||||||
|
headWindow = 16
|
||||||
|
|
||||||
|
// melContext — samples of history prepended to each incremental mel
|
||||||
|
// call, chosen so the eight frames this call yields continue exactly
|
||||||
|
// where the previous call's eight stopped.
|
||||||
|
//
|
||||||
|
// melspectrogram.onnx returns N/160-3 frames for N samples, and frame i
|
||||||
|
// covers [i*160, i*160+400). With 480 samples of history the buffer is
|
||||||
|
// 1760 samples, which is 8 frames, and the oldest of them starts one hop
|
||||||
|
// after the newest of the previous call. Less history leaves a gap: the
|
||||||
|
// first frames of a bare chunk would be computed against silence.
|
||||||
|
melContext = 480
|
||||||
|
|
||||||
|
// chunkSamples — audio per embedding step, 80ms.
|
||||||
|
chunkSamples = embedStride * melHop
|
||||||
|
)
|
||||||
|
|
||||||
|
// wakeModels holds the three ONNX sessions. It runs on CPU threads beside
|
||||||
|
// silero and never touches the GPU. That is a rule, not a result: a wake word
|
||||||
|
// that waits on card admission is not a wake word.
|
||||||
|
type wakeModels struct {
|
||||||
|
mel *ort.DynamicAdvancedSession
|
||||||
|
emb *ort.DynamicAdvancedSession
|
||||||
|
head *ort.DynamicAdvancedSession
|
||||||
|
}
|
||||||
|
|
||||||
|
// newWakeModels loads all three. melPath and embedPath are openWakeWord's
|
||||||
|
// frozen feature models; headPath is the keyword head trained for "Мэйвен".
|
||||||
|
func newWakeModels(melPath, embedPath, headPath, libPath string) (*wakeModels, error) {
|
||||||
|
if !ort.IsInitialized() {
|
||||||
|
if libPath != "" {
|
||||||
|
ort.SetSharedLibraryPath(libPath)
|
||||||
|
}
|
||||||
|
if err := ort.InitializeEnvironment(); err != nil {
|
||||||
|
return nil, fmt.Errorf("wake word: onnx runtime: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// One thread per session, not the default of every core. Measured on
|
||||||
|
// workpc: the default took mavwaked from 68% of one core to 335% of
|
||||||
|
// three, for three graphs that each run in well under 80ms single
|
||||||
|
// threaded. An always-on gate that eats a quarter of the workstation is
|
||||||
|
// not a gate he will leave running.
|
||||||
|
opts, err := ort.NewSessionOptions()
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("wake word: session options: %w", err)
|
||||||
|
}
|
||||||
|
defer opts.Destroy()
|
||||||
|
if err := opts.SetIntraOpNumThreads(1); err != nil {
|
||||||
|
return nil, fmt.Errorf("wake word: intra-op threads: %w", err)
|
||||||
|
}
|
||||||
|
if err := opts.SetInterOpNumThreads(1); err != nil {
|
||||||
|
return nil, fmt.Errorf("wake word: inter-op threads: %w", err)
|
||||||
|
}
|
||||||
|
open := func(p string, in, out []string) (*ort.DynamicAdvancedSession, error) {
|
||||||
|
s, err := ort.NewDynamicAdvancedSession(p, in, out, opts)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("wake word: load %s: %w", p, err)
|
||||||
|
}
|
||||||
|
return s, nil
|
||||||
|
}
|
||||||
|
m := &wakeModels{}
|
||||||
|
if m.mel, err = open(melPath, []string{"input"}, []string{"output"}); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if m.emb, err = open(embedPath, []string{"input_1"}, []string{"conv2d_19"}); err != nil {
|
||||||
|
m.Close()
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if m.head, err = open(headPath, []string{"embeddings"}, []string{"score"}); err != nil {
|
||||||
|
m.Close()
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return m, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Close releases the three sessions.
|
||||||
|
func (m *wakeModels) Close() {
|
||||||
|
if m == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
for _, s := range []*ort.DynamicAdvancedSession{m.mel, m.emb, m.head} {
|
||||||
|
if s != nil {
|
||||||
|
s.Destroy()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
m.mel, m.emb, m.head = nil, nil, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// melFrames runs one buffer of samples and returns the mel frames it yielded.
|
||||||
|
func (m *wakeModels) melFrames(buf []float32) ([][melBins]float32, error) {
|
||||||
|
in, err := ort.NewTensor(ort.NewShape(1, int64(len(buf))), buf)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer in.Destroy()
|
||||||
|
|
||||||
|
n := int64(len(buf)/melHop - 3)
|
||||||
|
if n < 1 {
|
||||||
|
return nil, fmt.Errorf("wake word: %d samples yield no mel frames", len(buf))
|
||||||
|
}
|
||||||
|
out, err := ort.NewEmptyTensor[float32](ort.NewShape(1, 1, n, melBins))
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer out.Destroy()
|
||||||
|
|
||||||
|
if err := m.mel.Run([]ort.Value{in}, []ort.Value{out}); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
data := out.GetData()
|
||||||
|
frames := make([][melBins]float32, n)
|
||||||
|
for i := range frames {
|
||||||
|
for j := 0; j < melBins; j++ {
|
||||||
|
// The scaling openWakeWord applies between the two feature
|
||||||
|
// models, and the head was fitted on its output.
|
||||||
|
frames[i][j] = data[i*melBins+j]/10.0 + 2.0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return frames, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// embedding runs embedFrames mel frames through the frozen embedder.
|
||||||
|
func (m *wakeModels) embedding(mels [][melBins]float32) ([embedDim]float32, error) {
|
||||||
|
var e [embedDim]float32
|
||||||
|
flat := make([]float32, 0, embedFrames*melBins)
|
||||||
|
for _, f := range mels {
|
||||||
|
flat = append(flat, f[:]...)
|
||||||
|
}
|
||||||
|
in, err := ort.NewTensor(ort.NewShape(1, embedFrames, melBins, 1), flat)
|
||||||
|
if err != nil {
|
||||||
|
return e, err
|
||||||
|
}
|
||||||
|
defer in.Destroy()
|
||||||
|
out, err := ort.NewEmptyTensor[float32](ort.NewShape(1, 1, 1, embedDim))
|
||||||
|
if err != nil {
|
||||||
|
return e, err
|
||||||
|
}
|
||||||
|
defer out.Destroy()
|
||||||
|
if err := m.emb.Run([]ort.Value{in}, []ort.Value{out}); err != nil {
|
||||||
|
return e, err
|
||||||
|
}
|
||||||
|
copy(e[:], out.GetData())
|
||||||
|
return e, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// score runs the trained head over headWindow embeddings.
|
||||||
|
func (m *wakeModels) score(embeds [][embedDim]float32) (float64, error) {
|
||||||
|
flat := make([]float32, 0, headWindow*embedDim)
|
||||||
|
for _, e := range embeds {
|
||||||
|
flat = append(flat, e[:]...)
|
||||||
|
}
|
||||||
|
in, err := ort.NewTensor(ort.NewShape(1, headWindow, embedDim), flat)
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
defer in.Destroy()
|
||||||
|
out, err := ort.NewEmptyTensor[float32](ort.NewShape(1, 1))
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
defer out.Destroy()
|
||||||
|
if err := m.head.Run([]ort.Value{in}, []ort.Value{out}); err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
return float64(out.GetData()[0]), nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,195 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
// The wake word, "Мэйвен" (V-487 stage two).
|
||||||
|
//
|
||||||
|
// Silero answers "is this frame speech". It does not answer "was this said to
|
||||||
|
// her", and until this file existed nothing did: every utterance near the
|
||||||
|
// microphone became a turn. What made that safe rather than expensive was
|
||||||
|
// SurfaceVoice capping acts at L0, and L0 does not cap reading, so the room
|
||||||
|
// could still hear his facts read back.
|
||||||
|
//
|
||||||
|
// This file owns the 80ms cadence and the three rings of state between the
|
||||||
|
// models. wakefeatures.go owns the tensors.
|
||||||
|
//
|
||||||
|
// Nil is a working value, and it is the CLOSED gate rather than the open one.
|
||||||
|
// Feed on a nil receiver reports no keyword; session.go asks separately
|
||||||
|
// whether a gate exists at all. That split is deliberate: a nil that answers
|
||||||
|
// "yes, keyword" reads as a working wake word in every log line it produces.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"log"
|
||||||
|
"sync"
|
||||||
|
)
|
||||||
|
|
||||||
|
// defaultWakeThreshold — score above which the keyword was said.
|
||||||
|
//
|
||||||
|
// Picked from the false-accept rate on held-out Russian speech, not from
|
||||||
|
// accuracy: a miss costs him a repeat, a false accept costs a turn nobody
|
||||||
|
// asked for. Over 65 minutes of Common Voice, 0.99 woke her three times and
|
||||||
|
// 0.999 once, and the difference in recall was one render out of 126. So the
|
||||||
|
// default is the strict one. `docs/evals/2026-08-09-wake-word.md` has both
|
||||||
|
// tables.
|
||||||
|
const defaultWakeThreshold = 0.999
|
||||||
|
|
||||||
|
// wakeWord is the streaming state around wakeModels. It is fed the same
|
||||||
|
// capture frames the VAD sees and answers whether the keyword has just been
|
||||||
|
// spoken.
|
||||||
|
type wakeWord struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
m *wakeModels
|
||||||
|
|
||||||
|
threshold float64
|
||||||
|
|
||||||
|
// pending holds captured samples not yet part of a full 80ms chunk, and
|
||||||
|
// history holds the melContext samples before them.
|
||||||
|
pending []float32
|
||||||
|
history []float32
|
||||||
|
|
||||||
|
// mels is the newest embedFrames mel frames, oldest first.
|
||||||
|
mels [][melBins]float32
|
||||||
|
// embeds is the newest headWindow embeddings, oldest first.
|
||||||
|
embeds [][embedDim]float32
|
||||||
|
|
||||||
|
last float64 // most recent score, held between chunks
|
||||||
|
}
|
||||||
|
|
||||||
|
// newWakeWord loads the models and wraps them in the streaming gate.
|
||||||
|
func newWakeWord(melPath, embedPath, headPath, libPath string, threshold float64) (*wakeWord, error) {
|
||||||
|
m, err := newWakeModels(melPath, embedPath, headPath, libPath)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if threshold <= 0 {
|
||||||
|
threshold = defaultWakeThreshold
|
||||||
|
}
|
||||||
|
return &wakeWord{m: m, threshold: threshold}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Close releases the models.
|
||||||
|
func (w *wakeWord) Close() {
|
||||||
|
if w == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
w.mu.Lock()
|
||||||
|
defer w.mu.Unlock()
|
||||||
|
w.m.Close()
|
||||||
|
w.m = nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Feed takes one capture frame and reports whether the keyword was heard on
|
||||||
|
// it. A nil wakeWord hears nothing.
|
||||||
|
func (w *wakeWord) Feed(frame []int16) bool {
|
||||||
|
if w == nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
w.mu.Lock()
|
||||||
|
defer w.mu.Unlock()
|
||||||
|
|
||||||
|
for _, v := range frame {
|
||||||
|
w.pending = append(w.pending, float32(v)/32768.0)
|
||||||
|
}
|
||||||
|
fired := false
|
||||||
|
for len(w.pending) >= chunkSamples {
|
||||||
|
chunk := w.pending[:chunkSamples]
|
||||||
|
if w.step(chunk) {
|
||||||
|
fired = true
|
||||||
|
}
|
||||||
|
w.history = append(w.history[:0], tailFloat32(append(w.history, chunk...), melContext)...)
|
||||||
|
// Slide the remainder to the front rather than reslicing. This runs
|
||||||
|
// every 80ms for as long as the daemon lives.
|
||||||
|
w.pending = append(w.pending[:0], w.pending[chunkSamples:]...)
|
||||||
|
}
|
||||||
|
return fired
|
||||||
|
}
|
||||||
|
|
||||||
|
// Reset drops the streaming state, so a fresh utterance is not judged on audio
|
||||||
|
// from before it. Called after every dispatch and after barge-in, for the same
|
||||||
|
// reason silero is: echo-era history must not score the next sentence, and her
|
||||||
|
// own voice saying the keyword must not wake her.
|
||||||
|
func (w *wakeWord) Reset() {
|
||||||
|
if w == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
w.mu.Lock()
|
||||||
|
defer w.mu.Unlock()
|
||||||
|
w.pending, w.history = w.pending[:0], w.history[:0]
|
||||||
|
w.mels, w.embeds = nil, nil
|
||||||
|
w.last = 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// Score returns the most recent score, for the operator to read out of the
|
||||||
|
// journal when picking a threshold for his room.
|
||||||
|
func (w *wakeWord) Score() float64 {
|
||||||
|
if w == nil {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
w.mu.Lock()
|
||||||
|
defer w.mu.Unlock()
|
||||||
|
return w.last
|
||||||
|
}
|
||||||
|
|
||||||
|
// step runs one 80ms chunk through all three models. It returns true when the
|
||||||
|
// score crosses the threshold on this chunk.
|
||||||
|
func (w *wakeWord) step(chunk []float32) bool {
|
||||||
|
buf := make([]float32, 0, melContext+len(chunk))
|
||||||
|
if pad := melContext - len(w.history); pad > 0 {
|
||||||
|
buf = append(buf, make([]float32, pad)...)
|
||||||
|
}
|
||||||
|
buf = append(buf, tailFloat32(w.history, melContext)...)
|
||||||
|
buf = append(buf, chunk...)
|
||||||
|
|
||||||
|
frames, err := w.m.melFrames(buf)
|
||||||
|
if err != nil {
|
||||||
|
// A failed inference must not silence the microphone. Hold the last
|
||||||
|
// score and let the next chunk try again.
|
||||||
|
log.Printf("mavwaked: wake word: mel: %v", err)
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
w.mels = tailMel(append(w.mels, frames...), embedFrames)
|
||||||
|
if len(w.mels) < embedFrames {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
e, err := w.m.embedding(w.mels)
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("mavwaked: wake word: embedding: %v", err)
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
w.embeds = tailEmbed(append(w.embeds, e), headWindow)
|
||||||
|
if len(w.embeds) < headWindow {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
score, err := w.m.score(w.embeds)
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("mavwaked: wake word: head: %v", err)
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
// Report the crossing, not the state. A keyword held above the threshold
|
||||||
|
// for a second is one wake, and firing on every chunk of it would make the
|
||||||
|
// gate look open when it is merely slow to fall.
|
||||||
|
crossed := score >= w.threshold && w.last < w.threshold
|
||||||
|
w.last = score
|
||||||
|
return crossed
|
||||||
|
}
|
||||||
|
|
||||||
|
// The three rings. Each keeps the newest n entries and nothing older.
|
||||||
|
|
||||||
|
func tailFloat32(s []float32, n int) []float32 {
|
||||||
|
if len(s) <= n {
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
return s[len(s)-n:]
|
||||||
|
}
|
||||||
|
|
||||||
|
func tailMel(s [][melBins]float32, n int) [][melBins]float32 {
|
||||||
|
if len(s) <= n {
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
return append(s[:0], s[len(s)-n:]...)
|
||||||
|
}
|
||||||
|
|
||||||
|
func tailEmbed(s [][embedDim]float32, n int) [][embedDim]float32 {
|
||||||
|
if len(s) <= n {
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
return append(s[:0], s[len(s)-n:]...)
|
||||||
|
}
|
||||||
@@ -0,0 +1,174 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// fakeGate fires on demand instead of running three ONNX models. The gate's
|
||||||
|
// own arithmetic is measured on real audio in docs/evals; what these tests
|
||||||
|
// cover is the thing that decides whether an utterance is shipped.
|
||||||
|
type fakeGate struct {
|
||||||
|
fireOn int // fire when this many frames have been fed, 0 never fires
|
||||||
|
fed int
|
||||||
|
resets int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (g *fakeGate) Feed(_ []int16) bool {
|
||||||
|
g.fed++
|
||||||
|
return g.fireOn > 0 && g.fed == g.fireOn
|
||||||
|
}
|
||||||
|
func (g *fakeGate) Reset() { g.resets++ }
|
||||||
|
func (g *fakeGate) Score() float64 { return 1 }
|
||||||
|
|
||||||
|
// wakingSession wires a session whose gate fires on the first frame it sees.
|
||||||
|
func wakingSession(fireOn int, window time.Duration) (*session, *fakePlayer, *fakeSender, *fakeGate) {
|
||||||
|
sess, p, snd := newTestSession(bargeInConfig{})
|
||||||
|
g := &fakeGate{fireOn: fireOn}
|
||||||
|
sess.UseWakeWord(g, window)
|
||||||
|
return sess, p, snd, g
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestKeywordlessSpeechNeverReachesSTT(t *testing.T) {
|
||||||
|
sess, p, snd, g := wakingSession(0, 8*time.Second)
|
||||||
|
speakThenPause(t, sess)
|
||||||
|
|
||||||
|
if len(snd.sent) != 0 {
|
||||||
|
t.Fatalf("sent %d utterances, want 0 — this is the whole point of V-487", len(snd.sent))
|
||||||
|
}
|
||||||
|
if sess.ignored != 1 {
|
||||||
|
t.Errorf("ignored = %d, want 1", sess.ignored)
|
||||||
|
}
|
||||||
|
if p.plays != 0 {
|
||||||
|
t.Errorf("plays = %d, want 0", p.plays)
|
||||||
|
}
|
||||||
|
if g.fed == 0 {
|
||||||
|
t.Error("the gate was never fed a frame")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestKeywordOpensTheGate(t *testing.T) {
|
||||||
|
sess, p, snd, _ := wakingSession(1, 8*time.Second)
|
||||||
|
speakThenPause(t, sess)
|
||||||
|
|
||||||
|
if len(snd.sent) != 1 {
|
||||||
|
t.Fatalf("sent %d utterances, want 1", len(snd.sent))
|
||||||
|
}
|
||||||
|
if sess.wakes != 1 {
|
||||||
|
t.Errorf("wakes = %d, want 1", sess.wakes)
|
||||||
|
}
|
||||||
|
if sess.ignored != 0 {
|
||||||
|
t.Errorf("ignored = %d, want 0", sess.ignored)
|
||||||
|
}
|
||||||
|
if p.plays != 1 {
|
||||||
|
t.Errorf("plays = %d, want 1", p.plays)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// One keyword buys one turn. Without this the microphone stays open for as
|
||||||
|
// long as he keeps talking, which is the state the gate exists to end.
|
||||||
|
func TestOneKeywordBuysOneTurn(t *testing.T) {
|
||||||
|
sess, p, snd, _ := wakingSession(1, 8*time.Second)
|
||||||
|
speakThenPause(t, sess)
|
||||||
|
p.Stop() // she finished her reply
|
||||||
|
sess.discard = 0 // the backlog drain is not what this measures
|
||||||
|
speakThenPause(t, sess)
|
||||||
|
|
||||||
|
if len(snd.sent) != 1 {
|
||||||
|
t.Fatalf("sent %d utterances, want 1: the second had no keyword", len(snd.sent))
|
||||||
|
}
|
||||||
|
if sess.ignored != 1 {
|
||||||
|
t.Errorf("ignored = %d, want 1", sess.ignored)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The keyword is heard, then he says nothing for longer than the window. What
|
||||||
|
// he says after that is not addressed to her.
|
||||||
|
func TestTheKeywordExpires(t *testing.T) {
|
||||||
|
sess, _, snd, _ := wakingSession(1, 500*time.Millisecond)
|
||||||
|
now := time.Unix(1750000000, 0)
|
||||||
|
sess.now = func() time.Time { return now }
|
||||||
|
|
||||||
|
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||||
|
t.Fatalf("feed: %v", err)
|
||||||
|
}
|
||||||
|
if sess.wakes != 1 {
|
||||||
|
t.Fatalf("wakes = %d, want 1", sess.wakes)
|
||||||
|
}
|
||||||
|
now = now.Add(2 * time.Second)
|
||||||
|
speakThenPause(t, sess)
|
||||||
|
|
||||||
|
if len(snd.sent) != 0 {
|
||||||
|
t.Fatalf("sent %d utterances, want 0 — the keyword had expired", len(snd.sent))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Barge-in cuts her off whether or not the keyword was heard. What he says
|
||||||
|
// after cutting her off still has to carry it.
|
||||||
|
func TestBargeInStillInterruptsHer(t *testing.T) {
|
||||||
|
sess, p, _, g := wakingSession(0, 8*time.Second)
|
||||||
|
sess.barge = bargeInConfig{RMS: 0.2, Frames: 3}
|
||||||
|
p.playing = true
|
||||||
|
loud := frameAt(0.35)
|
||||||
|
for i := 0; i < 4; i++ {
|
||||||
|
if err := sess.feed(context.Background(), loud); err != nil {
|
||||||
|
t.Fatalf("feed %d: %v", i, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if sess.bargeIns != 1 {
|
||||||
|
t.Fatalf("bargeIns = %d, want 1", sess.bargeIns)
|
||||||
|
}
|
||||||
|
if p.stops != 1 {
|
||||||
|
t.Errorf("stops = %d, want 1", p.stops)
|
||||||
|
}
|
||||||
|
if g.resets == 0 {
|
||||||
|
t.Error("barge-in left pre-playback audio in the gate")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Her own reply must not wake her. Frames captured while the player runs never
|
||||||
|
// reach the gate, and the gate is cleared when playback ends.
|
||||||
|
func TestHerOwnVoiceNeverReachesTheGate(t *testing.T) {
|
||||||
|
sess, p, _, g := wakingSession(1, 8*time.Second)
|
||||||
|
p.playing = true
|
||||||
|
for i := 0; i < 10; i++ {
|
||||||
|
if err := sess.feed(context.Background(), frameAt(0.35)); err != nil {
|
||||||
|
t.Fatalf("feed: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if g.fed != 0 {
|
||||||
|
t.Fatalf("gate was fed %d frames while she was speaking, want 0", g.fed)
|
||||||
|
}
|
||||||
|
if sess.wakes != 0 {
|
||||||
|
t.Errorf("wakes = %d, want 0", sess.wakes)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// No model, no gate: the daemon behaves exactly as it did before V-487 stage
|
||||||
|
// two. An operator with a missing file gets yesterday's mavwaked, not one that
|
||||||
|
// refuses to hear anything.
|
||||||
|
func TestNoGateShipsEveryUtterance(t *testing.T) {
|
||||||
|
sess, _, snd := newTestSession(bargeInConfig{})
|
||||||
|
speakThenPause(t, sess)
|
||||||
|
|
||||||
|
if len(snd.sent) != 1 {
|
||||||
|
t.Fatalf("sent %d utterances, want 1", len(snd.sent))
|
||||||
|
}
|
||||||
|
if sess.ignored != 0 {
|
||||||
|
t.Errorf("ignored = %d, want 0", sess.ignored)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A nil *wakeWord is the closed gate, not a crash and not an open one.
|
||||||
|
func TestNilWakeWordHearsNothing(t *testing.T) {
|
||||||
|
var w *wakeWord
|
||||||
|
if w.Feed([]int16{0, 0, 0}) {
|
||||||
|
t.Error("a nil wake word reported the keyword")
|
||||||
|
}
|
||||||
|
if w.Score() != 0 {
|
||||||
|
t.Error("a nil wake word reported a score")
|
||||||
|
}
|
||||||
|
w.Reset()
|
||||||
|
w.Close()
|
||||||
|
}
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
# maven-voice-tunnel — the ssh leg that carries the voice wire to homesrv.
|
||||||
|
#
|
||||||
|
# Runs on workpc, as a user unit (`systemctl --user`), beside mavgpud.service.
|
||||||
|
#
|
||||||
|
# WHY THIS EXISTS AT ALL. internal/voice is plaintext and unauthenticated.
|
||||||
|
# Its own server doc says production binds inside the wg tunnel, because "the
|
||||||
|
# wg layer IS the L0 floor". workpc is not a wg peer, it sits on wlan0. So ssh
|
||||||
|
# is the substitute floor: it authenticates with his key and encrypts the leg,
|
||||||
|
# and mavend's published port stays on homesrv loopback (127.0.0.1:9110).
|
||||||
|
# Nothing about this puts a Maven port on the LAN.
|
||||||
|
#
|
||||||
|
# Do not replace this with a LAN bind. SurfaceVoice caps acts at L0, so an
|
||||||
|
# unauthorized speaker could not run a destructive tool. It would still hear
|
||||||
|
# his facts, his notes and his calendar read back, and L0 does not cap reading.
|
||||||
|
#
|
||||||
|
# install: cp to ~/.config/systemd/user/ on workpc
|
||||||
|
# systemctl --user enable --now maven-voice-tunnel.service
|
||||||
|
|
||||||
|
[Unit]
|
||||||
|
Description=SSH tunnel to mavend's voice wire on homesrv
|
||||||
|
After=network-online.target
|
||||||
|
Wants=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
# -N: no remote command, forwarding only.
|
||||||
|
# ExitOnForwardFailure: fail loudly rather than sit up with a dead forward,
|
||||||
|
# which is what makes Restart meaningful.
|
||||||
|
# ServerAlive*: a laptop that suspends drops the tunnel silently otherwise.
|
||||||
|
ExecStart=/usr/bin/ssh -N \
|
||||||
|
-o ExitOnForwardFailure=yes \
|
||||||
|
-o ServerAliveInterval=30 \
|
||||||
|
-o ServerAliveCountMax=3 \
|
||||||
|
-o BatchMode=yes \
|
||||||
|
-L 127.0.0.1:9100:127.0.0.1:9110 \
|
||||||
|
kami@192.168.1.104
|
||||||
|
Restart=always
|
||||||
|
RestartSec=5
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=default.target
|
||||||
@@ -0,0 +1,76 @@
|
|||||||
|
# mavwaked — always-on listening, on workpc where the microphone is.
|
||||||
|
#
|
||||||
|
# User unit, beside mavgpud.service and maven-voice-tunnel.service. It is a
|
||||||
|
# user unit because it needs his ALSA session and his ssh agent, and because
|
||||||
|
# it should stop when he logs out.
|
||||||
|
#
|
||||||
|
# The keyword is "Мэйвен" and the three -wake- flags are what require it
|
||||||
|
# (V-487 stage two). Without them anything spoken near the fifine becomes a
|
||||||
|
# turn, which voiceSender makes safe rather than expensive: it sends
|
||||||
|
# Surface=SurfaceVoice, capping every command at L0. That does not stop her
|
||||||
|
# answering out loud, which is the whole reason the keyword exists.
|
||||||
|
#
|
||||||
|
# The threshold is 0.999 and it is the binary's default, so it is not passed.
|
||||||
|
# It came from 65 minutes of held-out Russian speech through this same binary:
|
||||||
|
# 0.9 false wakes an hour against 2.8 at 0.99, for one lost render out of 126
|
||||||
|
# (docs/evals/2026-08-09-wake-word.md). If the room proves noisier than the
|
||||||
|
# corpus, read the scores out of this unit's journal and pass -wake-threshold.
|
||||||
|
# Do not lower it by guessing.
|
||||||
|
#
|
||||||
|
# A keyword shorter than 1.32s can be heard too late to be used, because the
|
||||||
|
# head scores 1.28s of audio and the VAD has closed the utterance by then.
|
||||||
|
# "Мэйвен, <request>" is unaffected. A bare "Мэйвен" is the case that fails.
|
||||||
|
#
|
||||||
|
# -vad-model is passed on purpose. Silero answers "is this frame speech" where
|
||||||
|
# the energy floor answers "is this frame loud". It declines white noise at
|
||||||
|
# the same RMS 0 frames to 68-99, and still hears all four spoken fixtures
|
||||||
|
# (docs/evals/2026-08-09-silero-vad.md). It costs 509us a frame, 1.7% of one
|
||||||
|
# core, and never touches the GPU. Drop the flag and the energy floor is back.
|
||||||
|
#
|
||||||
|
# -barge-in is NOT passed. The threshold is room-specific and this room has no
|
||||||
|
# number yet. Turn it on only after reading the "suppressed while speaking"
|
||||||
|
# means out of this unit's own journal, never by guessing.
|
||||||
|
#
|
||||||
|
# install: cp to ~/.config/systemd/user/ on workpc
|
||||||
|
# systemctl --user enable --now mavwaked.service
|
||||||
|
|
||||||
|
[Unit]
|
||||||
|
Description=Maven always-on listening (silero VAD, "Мэйвен" keyword)
|
||||||
|
# The tunnel is the only path to mavend and the only thing authenticating it.
|
||||||
|
Requires=maven-voice-tunnel.service
|
||||||
|
After=maven-voice-tunnel.service
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
# The Scarlett Solo 4th Gen, and not the fifine. The fifine was the device
|
||||||
|
# here for three days and mavwaked never logged one utterance in them, because
|
||||||
|
# it returns RMS 0.00004 with its capture switch on and its ALSA volume at the
|
||||||
|
# full 496 of 496. That silence is in the hardware, so no flag reaches it.
|
||||||
|
#
|
||||||
|
# Named CARD=Gen and not card 4, because a USB card number moves when
|
||||||
|
# something else is replugged and this daemon must not change ears quietly.
|
||||||
|
# Not "default" either: that follows whatever pipewire last decided.
|
||||||
|
#
|
||||||
|
# plughw and not hw. mavwaked asks arecord for 16kHz mono, which is what the
|
||||||
|
# whole pipeline is canonical in. Neither microphone offers it, so bare hw
|
||||||
|
# dies on "Channels count non available" before a frame is read. plughw puts
|
||||||
|
# ALSA's downmix and resampler in front. Any replacement wants the same.
|
||||||
|
#
|
||||||
|
# The Scarlett measured RMS 0.003 against 0.14 on the onboard input, so its
|
||||||
|
# front-panel gain is the thing to raise if she mishears. That is a knob, not
|
||||||
|
# a control ALSA exposes. The two loud devices, the onboard ALC897 and the
|
||||||
|
# camera, both clip at peak 1.0 and are worse candidates, not better ones.
|
||||||
|
Environment=LD_LIBRARY_PATH=%h/.local/lib
|
||||||
|
ExecStart=%h/.local/bin/mavwaked \
|
||||||
|
-device plughw:CARD=Gen,DEV=0 \
|
||||||
|
-addr 127.0.0.1:9100 \
|
||||||
|
-lang ru \
|
||||||
|
-vad-model %h/.local/share/maven/models/silero_vad.onnx \
|
||||||
|
-wake-model %h/.local/share/maven/models/maven_wakeword.onnx \
|
||||||
|
-wake-mel %h/.local/share/maven/models/melspectrogram.onnx \
|
||||||
|
-wake-embed %h/.local/share/maven/models/embedding_model.onnx \
|
||||||
|
-onnx-lib %h/.local/lib/libonnxruntime.so
|
||||||
|
Restart=on-failure
|
||||||
|
RestartSec=5
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=default.target
|
||||||
@@ -53,6 +53,19 @@ services:
|
|||||||
# the decrypted working copy lives in RAM (see db_tmpfs in mavend.json).
|
# the decrypted working copy lives in RAM (see db_tmpfs in mavend.json).
|
||||||
tmpfs:
|
tmpfs:
|
||||||
- /dev/shm
|
- /dev/shm
|
||||||
|
# the voice wire, for mavwaked and mavenclient on workpc (V-515).
|
||||||
|
#
|
||||||
|
# LOOPBACK ONLY, and that is the whole security argument. internal/voice
|
||||||
|
# is plaintext with no auth: its own server doc says production binds
|
||||||
|
# inside the wg tunnel, "the wg layer IS the L0 floor". workpc is not a wg
|
||||||
|
# peer, it is on wlan0. So the tunnel is ssh instead, terminated on this
|
||||||
|
# loopback address, and nothing new is on the LAN. Anyone who could reach
|
||||||
|
# a LAN-bound port here could push audio and hear his facts read back.
|
||||||
|
# SurfaceVoice caps acts at L0; it does not cap reading.
|
||||||
|
#
|
||||||
|
# Host 9100 is Vikunja's MCP, hence 9110. The container side stays 9100
|
||||||
|
# so mavweb keeps reaching mavend:9100 by name.
|
||||||
|
ports: ["127.0.0.1:9110:9100"]
|
||||||
|
|
||||||
mavsttd:
|
mavsttd:
|
||||||
<<: *image
|
<<: *image
|
||||||
|
|||||||
@@ -0,0 +1,208 @@
|
|||||||
|
# Deployment: the boxes, the models, the daemons
|
||||||
|
|
||||||
|
*Last verified: 2026-08-09 @ a9b480a*
|
||||||
|
|
||||||
|
What runs where, and why each choice was made. `CLAUDE.md` carries only the
|
||||||
|
rules. This file carries the reasoning.
|
||||||
|
|
||||||
|
## The two boxes
|
||||||
|
|
||||||
|
**homesrv** is a Ryzen 5 5600U laptop and the deploy target. It offloads to the
|
||||||
|
Vega iGPU over Vulkan (`n_gpu_layers: 99`). Compose passes `/dev/dri` and the
|
||||||
|
render gid (993), and without both Vulkan enumerates zero devices and
|
||||||
|
llama-server falls back to CPU silently.
|
||||||
|
|
||||||
|
**workpc** is the workstation, 16GB of VRAM, reached as `kami@workpc` at
|
||||||
|
192.168.1.105. Model work moved there on 2026-08-02 by the owner's call, because
|
||||||
|
homesrv cannot grow a GPU.
|
||||||
|
|
||||||
|
Three rules govern the seam:
|
||||||
|
|
||||||
|
- **The workstation is never assumed up.**
|
||||||
|
- **Fall back silently** when it would only do the job better.
|
||||||
|
- **Name the gap** when the resident model cannot do the job at all.
|
||||||
|
|
||||||
|
A world question goes through `LLMPhraser.PhraseWorld` and returns `worldGap`
|
||||||
|
(`cmd/mavend/worldmodel.go`) rather than an invented answer. A box with no
|
||||||
|
`workstation` block behaves exactly as it did before the seam. `docs/offload.md`
|
||||||
|
says which caller is which.
|
||||||
|
|
||||||
|
## The resident model
|
||||||
|
|
||||||
|
**Qwen3-1.7B** (`UD-Q4_K_XL`), stock, not yet the CPT'd one. It is a Thinking
|
||||||
|
variant, so `n_ctx` is 4096. Reasoning tokens need the room, and 4096 is what
|
||||||
|
every score was measured at.
|
||||||
|
|
||||||
|
The target is the locally CPT'd Qwen3-1.7B (V-122, training in flight). Stock
|
||||||
|
already speaks good Russian. What it gets wrong is the persona. It writes `я рад`
|
||||||
|
where Maven needs `рада`.
|
||||||
|
|
||||||
|
**Do not bother with sub-500M models.** LFM2.5-230M and 350M were measured on
|
||||||
|
2026-07-31 and both are unusable in Russian
|
||||||
|
(`docs/evals/2026-07-31-model-bakeoff.md`). Their published IFEval and BFCL
|
||||||
|
numbers are English-only.
|
||||||
|
|
||||||
|
Model files live in `/mnt/hdd1/llms`, bind-mounted to `/opt/maven/models/llm`.
|
||||||
|
That **shadows** the repo's `models/llm/`, so a gguf sitting there is not loaded
|
||||||
|
by anything. Swapping the resident model is a one-line change to
|
||||||
|
`phraser.model_path` in `deploy/mavend.json`.
|
||||||
|
|
||||||
|
The workstation model is gemma-4-E4B as of 2026-08-09, replacing the 12B by the
|
||||||
|
owner's call. Keep the 12B gguf. It is the better teacher for label runs, at
|
||||||
|
72.7% destination against E4B's 57.6%.
|
||||||
|
|
||||||
|
## The embedder
|
||||||
|
|
||||||
|
**It stays on homesrv permanently**, because it backs the floor. It is
|
||||||
|
multilingual-e5-small, quantized and asymmetric. `EmbedQuery` and `EmbedPassage`
|
||||||
|
apply the `query:` and `passage:` prefixes it was trained with. Calling plain
|
||||||
|
`Embed` on a note is a bug. See `docs/evals/2026-08-04-recall-e5-small.md`.
|
||||||
|
|
||||||
|
The vendored onnxruntime under `deps/` has two copies, and the stale one is
|
||||||
|
1.17.1. The live runtime is 1.26.0, and the Go binding asks for API 26. Anything
|
||||||
|
shipped to another box needs `deps/onnxruntime-linux-x64-1.26.0`.
|
||||||
|
|
||||||
|
## Speech-to-text
|
||||||
|
|
||||||
|
`sttSeam` in `cmd/mavend/voicewire.go` builds an `stt.Pair` beside `modelSeam`.
|
||||||
|
It prefers CrisperWhisper 2.0 turbo on workpc with mavsttd as the floor. It takes
|
||||||
|
only the silent half of the rule, because a worse transcript is still a turn. So
|
||||||
|
`stt.Pair` has no `TranscribeRemote` and the fallback is never spoken.
|
||||||
|
|
||||||
|
CW2 turbo scores 10.4% WER in Russian against 27.5% for the `ggml-small.bin`
|
||||||
|
mavsttd loads, over 200 Golos clips
|
||||||
|
(`docs/evals/2026-08-09-crisperwhisper2-russian-wer.md`).
|
||||||
|
|
||||||
|
**whisper.cpp cannot load CW2 at all.** It reads its language count off the
|
||||||
|
vocabulary size. CW2's 51897 tokens shift seven special token ids. So CW2 is its
|
||||||
|
own transformers service on port 8081 (`deploy/cw2/serve.py`).
|
||||||
|
`stt.HTTPTranscriber` posts raw PCM to it with a bearer token, because audio is
|
||||||
|
the most sensitive thing that crosses this seam. The switch is `workstation.stt`
|
||||||
|
in `deploy/mavend.json`, and deleting the block sends every utterance to mavsttd.
|
||||||
|
|
||||||
|
**mavgpud runs that service as a second child.** This is not an optimisation.
|
||||||
|
CW2 is a ROCm process on the same card, so it registers on the KFD like any
|
||||||
|
contender. Under its own systemd unit it made mavgpud evict llama-server every
|
||||||
|
few seconds. That took the model arm down for eight minutes on 2026-08-09. The
|
||||||
|
card needs one owner. CW2 is on the yield clock and not the idle one. At 1.6GB
|
||||||
|
it denies the card to nobody.
|
||||||
|
|
||||||
|
Text-to-speech has not moved. piper on homesrv is the only synthesizer.
|
||||||
|
|
||||||
|
## The daemons
|
||||||
|
|
||||||
|
| Binary | Role |
|
||||||
|
|---|---|
|
||||||
|
| `mavend` | **Core.** Router, phraser, memory, reminders, digestion tick. Owns the DB and IPC socket. |
|
||||||
|
| `mavweb` | HTTP UI and PWA (`/dash`, `/history`, `/trace`, `/notifications`, `/tools`), WebAuthn auth. |
|
||||||
|
| `mavsttd` | Speech-to-text (whisper.cpp, CGO). |
|
||||||
|
| `mavttsd` | Text-to-speech (piper subprocess). |
|
||||||
|
| `mavwaked` | Wake-word and VAD gate. Runs on workpc. |
|
||||||
|
| `mavenclient` | Voice loop client (mic, stt, core, tts). Not deployed. |
|
||||||
|
| `mavpoll` | Environment poller: netdata alarms, uptime-kuma, zenmoney, wireguard presence. Writes facts, sends nothing. Telegram is `internal/delivery/telegramsink`. |
|
||||||
|
| `mavcaldav` | CalDAV calendar sync. |
|
||||||
|
| `mavmaild` | Mail reader (IMAP, read-only). Holds the IMAP password, core never sees it. |
|
||||||
|
| `mavgpud` | GPU supervisor. **Runs on workpc**, own unit `deploy/mavgpud.service`. Keeps llama-server loaded while the card is free (V-488). Maven never asks it for anything and reads `/health` through `llm.Pair`. |
|
||||||
|
| `mavupdate` | Not a daemon. Operator CLI a human runs on the box to deploy a new build. |
|
||||||
|
|
||||||
|
Two binaries have no Makefile target and neither is deployed. `mavseal` encrypts
|
||||||
|
a live tmpfs working copy back to the ciphertext file when mavend was killed
|
||||||
|
before `defer st.Close()` sealed it. `labelgen` runs the stage 0 grammars over
|
||||||
|
utterances and prints JSONL, the training data for the routing heads.
|
||||||
|
|
||||||
|
Daemons are wired socket-to-socket, not linked. `internal/ipc` is the wire
|
||||||
|
protocol. `deploy/mavend.json` sets socket paths, model paths and the phraser and
|
||||||
|
embedder blocks, with `${VAR}` expansion from gitignored `deploy/telegram.env`.
|
||||||
|
|
||||||
|
### Who is in compose, and who is not
|
||||||
|
|
||||||
|
**`docker-compose.yml` runs five**: `mavend`, `mavsttd`, `mavttsd`, `mavweb`,
|
||||||
|
`mavpoll`. Count against compose, not against the table above.
|
||||||
|
|
||||||
|
`mavmaild` and `mavcaldav` are commented out, each with the reason beside it. The
|
||||||
|
first needs a mail account and the second a CalDAV account, and this box has
|
||||||
|
neither. Two things ride on the CalDAV absence (V-644). Agenda questions route to
|
||||||
|
`IntentQuery` at stage 0, and the `calendar` query source then reads a table
|
||||||
|
nobody writes. And `loop.State.CalendarBusy` is fed by the same facts, so the
|
||||||
|
gate's "do not nag mid-meeting" is permanently false.
|
||||||
|
|
||||||
|
`mavenclient` is still absent. `mavwaked` moved to workpc on 2026-08-09 (V-515).
|
||||||
|
|
||||||
|
### The voice wire
|
||||||
|
|
||||||
|
`internal/voice` is plaintext with no auth. Its own server doc says production
|
||||||
|
binds inside the wg tunnel, because the wg layer is the L0 floor. workpc is not
|
||||||
|
a wg peer, it sits on wlan0. So the tunnel is ssh instead.
|
||||||
|
|
||||||
|
mavend publishes the voice port to homesrv loopback only, `127.0.0.1:9110`.
|
||||||
|
Host 9100 is Vikunja's MCP, hence 9110. The container side stays 9100 so mavweb
|
||||||
|
keeps reaching `mavend:9100` by name. `deploy/maven-voice-tunnel.service` on
|
||||||
|
workpc forwards it over his key.
|
||||||
|
|
||||||
|
**Do not replace this with a LAN bind.** `SurfaceVoice` caps acts at L0, so an
|
||||||
|
unauthorized speaker could not run a destructive tool. L0 does not cap reading,
|
||||||
|
so they would still hear his facts, notes and calendar read back.
|
||||||
|
|
||||||
|
Both `mavwaked` and `mavenclient` speak `voice.Dial`, not `ipc.Dial`. The
|
||||||
|
`netaddr` token guards the daemon-to-daemon IPC seam and never touches this one.
|
||||||
|
`ipc.Dial` does take `tcp://host:port?token=...`, which is why V-515 was filed
|
||||||
|
as a config change. That premise was wrong, and the ssh leg is the correction.
|
||||||
|
|
||||||
|
The voice loop belongs on a client machine where the owner is standing, and that
|
||||||
|
machine is workpc (V-463, `docs/plans/17-where-the-voice-loop-runs.md`). homesrv
|
||||||
|
has a microphone, because it is a laptop, but it is in the wrong room.
|
||||||
|
|
||||||
|
### mavwaked on workpc
|
||||||
|
|
||||||
|
`deploy/mavwaked.service`, a user unit beside `mavgpud.service`. Two flags are
|
||||||
|
deliberate.
|
||||||
|
|
||||||
|
`-vad-model` is passed. Silero answers "is this frame speech" where the energy
|
||||||
|
floor answers "is this frame loud". It declines white noise at the same RMS, 0
|
||||||
|
frames against 68 to 99, and still hears all four spoken fixtures
|
||||||
|
(`docs/evals/2026-08-09-silero-vad.md`). It costs 509µs a frame and never touches
|
||||||
|
the GPU. A model that will not load is logged and not fatal.
|
||||||
|
|
||||||
|
`-barge-in` is not passed. The threshold is room-specific and this room has no
|
||||||
|
number yet. Read the "suppressed while speaking" means out of the journal first.
|
||||||
|
|
||||||
|
The device is `plughw:0,0` and not `hw:0,0`. The fifine offers 2 channels at
|
||||||
|
44100 or 48000 and nothing else, and mavwaked asks arecord for 16kHz mono. Bare
|
||||||
|
`hw` dies on "Channels count non available" before a frame is read.
|
||||||
|
|
||||||
|
The three `-wake-` flags require the keyword "Мэйвен" (V-487 stage two). Drop
|
||||||
|
them and the loop runs open, which is what it did before. The threshold is the
|
||||||
|
binary's default of 0.999 and is not passed. Over 65 minutes of held-out
|
||||||
|
Russian speech it woke her 0.9 times an hour against 2.8 at 0.99. That cost one
|
||||||
|
lost render out of 126 (`docs/evals/2026-08-09-wake-word.md`).
|
||||||
|
|
||||||
|
The three sessions are pinned to one thread each. onnxruntime otherwise sizes
|
||||||
|
its pool to every core and spins between runs, which took mavwaked from 68% of
|
||||||
|
one core to 335%. With the cap it sits at 81%, so the gate costs about 13%.
|
||||||
|
|
||||||
|
A keyword shorter than 1.32s can be heard too late to be used. The head scores
|
||||||
|
1.28s of audio, and the VAD has closed the utterance by then.
|
||||||
|
"Мэйвен, <request>" is unaffected. A bare "Мэйвен" is the case that fails.
|
||||||
|
|
||||||
|
mavwaked connects at startup and holds the conn, so a nudge routed to voice
|
||||||
|
reaches the speaker before he has said anything (V-671). It used to connect
|
||||||
|
lazily, which made the failure silent rather than absent: after one utterance
|
||||||
|
the session existed, `PushToMostRecent` succeeded, the dispatcher stopped
|
||||||
|
rerouting to telegram and ntfy, and mavwaked discarded the audio.
|
||||||
|
|
||||||
|
**Passwords are read from files, never taken as flag values.** `mavcaldav` uses
|
||||||
|
`-pass-file` and `-render-pass-file`. `mavpoll` and `mavmaild` follow the same
|
||||||
|
rule.
|
||||||
|
|
||||||
|
## Web UI conventions
|
||||||
|
|
||||||
|
Server-rendered pages share `cmd/mavweb/static/ui.css` (served at `/ui.css`) and
|
||||||
|
the shell partial in `cmd/mavweb/shell.html`. A page opens with
|
||||||
|
`{{template "shellTop" "<page-key>"}}` and closes with `{{template "shellBottom"}}`,
|
||||||
|
and the key marks the active sidebar link.
|
||||||
|
|
||||||
|
Every page is its own embedded `.html` file next to `main.go`. No page markup
|
||||||
|
lives in Go, and the sidebar is data (`sidebarSections`, `pageIcon`) the template
|
||||||
|
renders. No per-page `<style>` beyond true one-offs. Wrap every table in
|
||||||
|
`<div class=scroll>` so wide data pans on a phone. Local preview and headless
|
||||||
|
screenshot recipes are in `AGENTS.md`.
|
||||||
@@ -30,6 +30,12 @@ Maven is the user-facing control center, but not the source of truth for identit
|
|||||||
|
|
||||||
Maven provides the human interface over the other systems.
|
Maven provides the human interface over the other systems.
|
||||||
|
|
||||||
|
| Service | Owns | Maven's client | Configured at |
|
||||||
|
|---|---|---|---|
|
||||||
|
| **Nexus** | Canonical entity ids, names, aliases, relationships. Projects, services, devices, people, pets, places. | `nexusClient` in `cmd/mavend/ecosystem.go`, `POST /api/v1/resolve` | `nexus.url` (`http://nexus:9740`) |
|
||||||
|
| **Praxis** | Operational attention and item lifecycle. What needs looking at, what changed, what is unresolved. | `praxisClient`, the HTTP tools API under `/api/v1/tools/` | `praxis.url` (`http://praxis:8989`) |
|
||||||
|
| **Hexis** | The capability registry and the only path to executing anything. | vendored `github.com/kami/hexis/pkg/client` | `hexis.url` (`http://hexis:9741`) |
|
||||||
|
|
||||||
It is responsible for:
|
It is responsible for:
|
||||||
|
|
||||||
- interpreting Russian and English utterances
|
- interpreting Russian and English utterances
|
||||||
|
|||||||
@@ -0,0 +1,112 @@
|
|||||||
|
# The "Мэйвен" wake word: what it hears and what it invents
|
||||||
|
|
||||||
|
*Measured 2026-08-09 on workpc and homesrv. V-487, stage two of two.*
|
||||||
|
|
||||||
|
Stage one gave mavwaked silero-vad, which answers "is this frame speech".
|
||||||
|
Nothing answered "was this said to her", so every utterance near the
|
||||||
|
microphone became a turn. SurfaceVoice caps acts at L0, which made that safe
|
||||||
|
rather than expensive. L0 does not cap reading, so the room could still hear
|
||||||
|
his facts read back.
|
||||||
|
|
||||||
|
The keyword is "Мэйвен". openWakeWord's two frozen feature models do the
|
||||||
|
hearing and a 100KB head trained here draws the boundary. It runs on CPU
|
||||||
|
beside silero and never touches the GPU.
|
||||||
|
|
||||||
|
## Why a per-window accuracy is not a number anyone can act on
|
||||||
|
|
||||||
|
The gate scores every 80ms. A 1.7% false-accept rate per window sounds small
|
||||||
|
and means a wake every few seconds. The useful question is how many times an hour
|
||||||
|
it wakes on speech that was not the keyword. So every table below counts
|
||||||
|
threshold crossings over whole clips and divides by the audio duration.
|
||||||
|
|
||||||
|
A crossing, not a window above the threshold. A keyword held high for half a
|
||||||
|
second is one wake, not six.
|
||||||
|
|
||||||
|
## The data
|
||||||
|
|
||||||
|
Positives are 600 silero TTS renders of three stressings of the keyword, six
|
||||||
|
speakers, ten trailing phrases, augmented eight ways each. Hard negatives are
|
||||||
|
560 renders of confusable Russian words. Real speech is Common Voice ru and Golos.
|
||||||
|
The 74257 Common Voice clips were already on workpc from the CrisperWhisper
|
||||||
|
work. The 200 Golos clips came from the CW2 WER eval.
|
||||||
|
|
||||||
|
Splits are by source file. Augmented copies of one render on both sides of a
|
||||||
|
split would measure memorisation.
|
||||||
|
|
||||||
|
Golos was never trained on at any stage, so it answers the harder question:
|
||||||
|
does this survive a change of speakers and rooms.
|
||||||
|
|
||||||
|
## Three heads
|
||||||
|
|
||||||
|
Each row is a full retrain. The false-accept column is 8.89 hours of Common
|
||||||
|
Voice that no stage of training had seen.
|
||||||
|
|
||||||
|
| trained on | recall (window) | false wakes/hour @0.99 |
|
||||||
|
|---|---|---|
|
||||||
|
| TTS + 13.7 min of Golos | 0.869 | not measurable |
|
||||||
|
| + 4000 Common Voice clips | 0.836 | 21.9 |
|
||||||
|
| + 3837 mined hard negatives | 0.784 | 4.2 |
|
||||||
|
| + 753 more mined | 0.810 | 3.4 |
|
||||||
|
|
||||||
|
The first row is why the second exists. Thirteen minutes of held-out speech
|
||||||
|
cannot measure a rate for a gate that scores twelve times a second. A head
|
||||||
|
trained only against TTS learns to tell TTS from not-TTS.
|
||||||
|
|
||||||
|
Mining is the whole story after that. Random negatives teach the head what
|
||||||
|
most speech sounds like. They do not teach it the few syllable sequences that
|
||||||
|
score high, because 4000 clips barely contain them. So the current head was
|
||||||
|
run over 20000 fresh clips, keeping every window it scored above 0.05. That
|
||||||
|
found 3837 windows in 855512. Repeating those ten times in the next training
|
||||||
|
run cut the rate five-fold.
|
||||||
|
|
||||||
|
The second round found 753 in 852240, a fifth of the yield, and bought a
|
||||||
|
further 20%. It also recovered recall, which the first round had cost. Whether
|
||||||
|
a third round is worth 25 minutes of workpc is untested.
|
||||||
|
|
||||||
|
## Where the threshold came from
|
||||||
|
|
||||||
|
Both columns are held out. Positives are the 126 renders in the test split.
|
||||||
|
Speech is 65.1 minutes of Common Voice, disjoint from every training and
|
||||||
|
mining pool. Both were run through the built `mavwaked` binary reading PCM from a
|
||||||
|
file, not through the python that trained the head.
|
||||||
|
|
||||||
|
| threshold | renders shipped | false wakes/hour |
|
||||||
|
|---|---|---|
|
||||||
|
| 0.99 | 116 / 126 | 2.8 |
|
||||||
|
| 0.999 | 115 / 126 | 0.9 |
|
||||||
|
|
||||||
|
One render against a third of the false wakes. `defaultWakeThreshold` is
|
||||||
|
0.999.
|
||||||
|
|
||||||
|
Golos disagrees. It gave 2 wakes in 14 minutes at every threshold, which is
|
||||||
|
8.7 per hour. Two events is not a rate. What it does say is that a handful of real utterances score above 0.999
|
||||||
|
and no threshold will move them.
|
||||||
|
|
||||||
|
## What it costs him
|
||||||
|
|
||||||
|
Ten of the 126 held-out renders were heard and still dropped, and every one
|
||||||
|
was an utterance shorter than 1.32s. The head scores 16 embeddings, or 1.28s of
|
||||||
|
audio. The score therefore peaks up to a second after a short keyword ends.
|
||||||
|
By then the VAD has closed the utterance and dispatch has already asked.
|
||||||
|
|
||||||
|
Real commands are "Мэйвен, <request>" and run past two seconds, which gives
|
||||||
|
the head the whole request to peak during. A bare "Мэйвен" with nothing after
|
||||||
|
it is the case that fails. One fix would hold an ignored utterance for a grace
|
||||||
|
period and ship it if the keyword lands late. It is not built.
|
||||||
|
|
||||||
|
## What it costs the workstation
|
||||||
|
|
||||||
|
Under systemd on workpc, mavwaked sat at 335% of a core with the gate on and
|
||||||
|
68% with only silero. onnxruntime sizes its thread pool to every core and spins
|
||||||
|
between runs, and this gate runs three graphs twelve times a second. Pinning
|
||||||
|
all three sessions to one thread brought it to 81%, so the keyword costs about
|
||||||
|
13% of one core. The three graphs each finish in well under 80ms that way.
|
||||||
|
|
||||||
|
## What was not measured
|
||||||
|
|
||||||
|
No room recordings. Every negative above is a clean corpus clip. This gate
|
||||||
|
will live among a television, a fan and the far side of a kitchen. None of
|
||||||
|
those are in these numbers.
|
||||||
|
|
||||||
|
No measurement of him. Training on his voice means copying his transcripts off
|
||||||
|
homesrv, which is his call and has not been asked.
|
||||||
@@ -0,0 +1,74 @@
|
|||||||
|
# Language: what the model emits, and how Russian is matched
|
||||||
|
|
||||||
|
*Last verified: 2026-08-09 @ a9b480a*
|
||||||
|
|
||||||
|
Two contracts live here. What a model call is allowed to return, and which
|
||||||
|
mechanism is allowed to recognise a Russian word.
|
||||||
|
|
||||||
|
## The LLM output contract
|
||||||
|
|
||||||
|
All phrasing paths emit `{"response":"...","mood":"..."}`. They fall back to
|
||||||
|
plain text when the model skips the JSON.
|
||||||
|
|
||||||
|
**One parser, `parseResponseMood` in `internal/phraser/parse.go`.** Every path
|
||||||
|
reaches it: the six `LLMPhraser` methods, `PhraseWorld`, and
|
||||||
|
`Replier.PhraseReply`. `cmd/mavend/replier_llm.go` wraps the last of those,
|
||||||
|
holds the stub fallback, and parses nothing itself. Mood is a fixed enum.
|
||||||
|
|
||||||
|
The router prompt is a separate contract:
|
||||||
|
|
||||||
|
```text
|
||||||
|
[{"intent":<enum>, key?, value?, text?, verb?}, ...]
|
||||||
|
```
|
||||||
|
|
||||||
|
over 7 intents: `fact, reminder, note, query, act, chat, system`.
|
||||||
|
`llm/check_prompt_parity.py` in the training workspace enforces that the Go
|
||||||
|
prompt and the relabelling prompt stay identical. They diverged once, and the
|
||||||
|
relabelled set then taught a head the Go router never asks for.
|
||||||
|
|
||||||
|
## Russian patterns: three mechanisms, no fourth
|
||||||
|
|
||||||
|
Hand-written Russian stem patterns were swept out on 2026-08-04 by the owner's
|
||||||
|
call. A regex whose output is a fact or a route is the defect. A regex over
|
||||||
|
structured input, such as HTML, MIME, JSON, a URL or an argv list, is not.
|
||||||
|
|
||||||
|
Before writing a Russian word list, pick one of these.
|
||||||
|
|
||||||
|
### `internal/lexicon`, for closed classes
|
||||||
|
|
||||||
|
`lexicon_ru_v1.json` holds interrogatives, capture verbs, reminder verbs,
|
||||||
|
cardinals, day offsets, parts of day, weekdays, months and spoken hours.
|
||||||
|
Editing a word is a data change and there is exactly one copy. Months used to
|
||||||
|
live in three files and drifted between them.
|
||||||
|
|
||||||
|
Cardinals carry the oblique forms, because a spoken time declines. `в семь` and
|
||||||
|
`к семи` are one hour.
|
||||||
|
|
||||||
|
### `internal/morph`, for grammar
|
||||||
|
|
||||||
|
From the vendored golem Russian dictionary. `IsVerbForm` and `SameWord`.
|
||||||
|
|
||||||
|
Lemma matching is BROADER than stem-plus-one-ending. A verb slot that means the
|
||||||
|
imperative must be matched exactly. `говори` and `говорил` share a
|
||||||
|
lemma and only one of them is a command
|
||||||
|
(`cmd/mavend/quiet_toggle.go`).
|
||||||
|
|
||||||
|
### `cmd/mavend/topics.go` and the embedder, for open sets
|
||||||
|
|
||||||
|
Use these when the question is what a turn is ABOUT. Frozen seeds per subject
|
||||||
|
plus a real `other` class, scored against the turn's own query vector.
|
||||||
|
|
||||||
|
Same shape as the personal boundary in `personalboundary.go`, with one
|
||||||
|
difference. A topic must clear the runner-up by `topicMargin`, because a false
|
||||||
|
claim here spends a network scan rather than one honest "не знаю". The old
|
||||||
|
keyword tests stay as the offline floor.
|
||||||
|
|
||||||
|
### The ecosystem trio
|
||||||
|
|
||||||
|
Use it when the answer is not in the utterance at all. Identity is Nexus's,
|
||||||
|
never a local pattern.
|
||||||
|
|
||||||
|
## Seeds are scoring data
|
||||||
|
|
||||||
|
Editing one moves a recogniser. Re-measure against the `TestONNX*` tests rather
|
||||||
|
than eyeballing the change.
|
||||||
+462
@@ -0,0 +1,462 @@
|
|||||||
|
# Routing
|
||||||
|
|
||||||
|
*Last verified: 2026-08-09 @ 31b5093*
|
||||||
|
|
||||||
|
How an utterance becomes a `Decision`, why each stage exists, and what every
|
||||||
|
stage has measured. `CLAUDE.md` carries the rules an agent must not break. This
|
||||||
|
file carries the reasoning and the history behind them.
|
||||||
|
|
||||||
|
Two decisions come out of a route. **Intent** is one of seven values. **Source**
|
||||||
|
is where the answer lives, and it is read on `IntentQuery` alone. They are scored
|
||||||
|
separately, because one number hides which one moved.
|
||||||
|
|
||||||
|
## The cascade
|
||||||
|
|
||||||
|
| Stage | What it is | Where |
|
||||||
|
|---|---|---|
|
||||||
|
| 0 | Deterministic grammars over the utterance | `stage0.go`, `praxis.go`, `worldquery.go` |
|
||||||
|
| 0b | Four ONNX heads on one e5-small forward pass | `heads.go` |
|
||||||
|
| 1 | The resident model, GBNF-constrained JSON | `llmrouter.go` |
|
||||||
|
| 2 | Nearest neighbour over frozen seed phrases | `classifier.go`, `embedder.go` |
|
||||||
|
|
||||||
|
Every stage may decline, and the next one answers. Any error at stage 0b or 1
|
||||||
|
falls through, so a turn never breaks on a model.
|
||||||
|
|
||||||
|
The resident model arm is wired at `voice.go:214` through
|
||||||
|
`pickLLMRouter(cfg.Voice.UseLLMRouter(), llmClient)`. The flag is
|
||||||
|
`voice.llm_router` in `config.go`, `DefaultLLMRouter` is on, and
|
||||||
|
`deploy/mavend.json` sets it `true`. With no llama-server to talk to,
|
||||||
|
`pickLLMRouter` logs that and degrades to the classifier.
|
||||||
|
|
||||||
|
The classifier is the floor and not dead code. It runs when the resident model
|
||||||
|
is off, when there is no llama-server to talk to, and on any per-turn error.
|
||||||
|
Routing by seed similarity is the known cause of weak Russian queries. Deleting
|
||||||
|
it would make a model outage a broken turn.
|
||||||
|
|
||||||
|
### Why there are two engines at all
|
||||||
|
|
||||||
|
The original design was the classifier alone. `docs/rearchitecture.md` replaced
|
||||||
|
it with a model that emits structured JSON. The same weights phrase the reply.
|
||||||
|
That demoted the embedder from a routing gate to a hint for recall. The model
|
||||||
|
became the default on 2026-07-31.
|
||||||
|
|
||||||
|
The gap it buys is smaller than the design assumed. Measured 2026-08-02 on the
|
||||||
|
77-case Russian fixture, the classifier scores 68.8% full accuracy at p50 16.6µs.
|
||||||
|
Qwen3-1.7B scores 72.7% through the cascade. Four points, not a doubling.
|
||||||
|
|
||||||
|
An older figure of 36.8% for the classifier stood in `CLAUDE.md` until then. It
|
||||||
|
predates the stage 0 rules and the seed additions. Both now score inside the
|
||||||
|
classifier baseline.
|
||||||
|
|
||||||
|
Latency was misreported the same way. A figure of 2.7 seconds stood for two
|
||||||
|
days and was contention rather than the model.
|
||||||
|
`docs/evals/2026-07-31-routing.md` line 61 measures the router at p50 825ms and
|
||||||
|
the cascade at p50 0.80s to 1.04s.
|
||||||
|
|
||||||
|
## Numbers
|
||||||
|
|
||||||
|
Three arms answer, so three numbers are live. Judge a routing change against the
|
||||||
|
classifier and the resident model, since those are what always answer.
|
||||||
|
|
||||||
|
| Arm | Intent | Destination | p50 | Measured |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| classifier + ONNX | 76.0% (73/96) | 36.4% (12/33) | 16.6µs | 2026-08-08 |
|
||||||
|
| resident Qwen3-1.7B, cascade | 80.2% | not measured | 1.19s | 2026-08-05 |
|
||||||
|
| routing heads, cascade | 96.9% | 75.8% | 27.9ms | 2026-08-08 |
|
||||||
|
| gemma-4-12b, cascade | 84.4% | 72.7% | 329ms | 2026-08-02 |
|
||||||
|
| gemma-4-E4B, cascade | 89.6% | 57.6% (19/33) | 294ms | 2026-08-09 |
|
||||||
|
|
||||||
|
The fixture grew from 77 cases to 91 to 96. So a number is comparable only to
|
||||||
|
another number on the same fixture. Sources:
|
||||||
|
`docs/evals/2026-08-05-routing-resident-model.md`,
|
||||||
|
`docs/evals/2026-08-02-workstation-gemma4-12b.md`,
|
||||||
|
`docs/evals/2026-08-09-e4b-vs-12b-routing.md`,
|
||||||
|
`docs/evals/2026-08-08-routing-heads-in-go.md`,
|
||||||
|
`docs/evals/2026-08-08-destination-fixture.md`.
|
||||||
|
|
||||||
|
The resident model alone scores 37.4% full against 61.5% intent-only. The gap is
|
||||||
|
slots and not routing. It routes `reminder` and leaves the time to the daemon,
|
||||||
|
which is what the contract asks.
|
||||||
|
|
||||||
|
To re-run the resident model as router, start a **second** llama-server on a
|
||||||
|
fixed host port. The resident one binds `--port 0` inside the container and no
|
||||||
|
host process can reach it.
|
||||||
|
|
||||||
|
### The workstation is not the better router any more
|
||||||
|
|
||||||
|
It was, from 2026-08-02 until the heads landed. gemma-4-12b beat everything on
|
||||||
|
the box at 84.4% intent and 72.7% destination. The heads beat it on both at a
|
||||||
|
twelfth of the latency. The workstation stays the better phraser.
|
||||||
|
|
||||||
|
E4B replaced the 12B on 2026-08-09 by the owner's call. It is a step down on
|
||||||
|
routing. Against a same-session 12B control it costs four destination cases and
|
||||||
|
buys 50ms. Read destination as the finding. It names nothing where the 12B names
|
||||||
|
`recall` or `calendar`, which is safe but walks the whole chain. It has no MTP
|
||||||
|
and cannot be given any here. The only `gemma4-assistant` draft on disk is
|
||||||
|
trained against the 12B's hidden states.
|
||||||
|
|
||||||
|
## Stage 0: what the grammars claim, and why
|
||||||
|
|
||||||
|
A rule at this stage is a claim. Either the model gets this wrong, or it wastes a
|
||||||
|
second getting it right. Every rule was added against a measurement.
|
||||||
|
|
||||||
|
- **Agenda questions** (`AgendaQueryGrammars`, 2026-08-01). "что у меня сегодня",
|
||||||
|
"во сколько у меня встреча" and anything naming a calendar go to `IntentQuery`.
|
||||||
|
They were going to `IntentSystem`, where `replySystem` has no agenda arm and
|
||||||
|
answered "пока не умею". Worth 2.6 points of full accuracy and calendar 0/2 to
|
||||||
|
2/2.
|
||||||
|
- **Rest of day and narrative** (V-498, 2026-08-04). `rest-of-day-query` claims
|
||||||
|
"что дальше?". `NarrativeQueryGrammar` claims "расскажи про X", "объясни X" and
|
||||||
|
"опиши X". Neither carries a question mark or an interrogative, so the model
|
||||||
|
called both `IntentFact`. `IsQuestionShaped` caught the write downstream, so
|
||||||
|
this was a latency and fixture defect rather than a correctness one. The
|
||||||
|
narrative rule declines `chatNarrativeTopics`, because the query chain has no
|
||||||
|
source that answers a joke or a bedtime story.
|
||||||
|
- **Praxis** (V-516, 2026-08-05). `PraxisGrammars()` fills `Slots.Fn` with a
|
||||||
|
capability name. These grammars are the **only** path to Praxis and not a
|
||||||
|
faster one. The model reaches Praxis 0/12 alone, the same as the classifier.
|
||||||
|
Nothing in the router prompt names a Praxis capability, so there is no string
|
||||||
|
for it to write. Through the cascade it is 11/12. Measured overall 16/30 to
|
||||||
|
27/30, lifecycle 0/5 to 5/5
|
||||||
|
(`docs/evals/2026-08-05-praxis-reach.md`,
|
||||||
|
`docs/evals/2026-08-05-reach-llm-router.md`).
|
||||||
|
`handlePraxisAct` compares `Slots.Fn` to a capability alias. Otherwise that
|
||||||
|
slot is filled from the deployment's enabled tool names, and no Praxis alias
|
||||||
|
is on that list.
|
||||||
|
- **World questions** (`WorldQueryGrammars`, V-655, 2026-08-07). "что такое X"
|
||||||
|
and "сколько будет 17 на 23". Wired after the agenda rules and **before** the
|
||||||
|
feed and list rules. "что такое лента" is a definition question, and the feed
|
||||||
|
rule would take it on the noun alone.
|
||||||
|
|
||||||
|
`calendar-query` and `event-time-query` name the calendar as the destination.
|
||||||
|
The possessive agenda rules deliberately do not. "что у меня в списке покупок"
|
||||||
|
matches `agenda-query`, and naming the calendar there would take the list source
|
||||||
|
off the turn. That caution now costs four destination cases. See the model arm
|
||||||
|
below.
|
||||||
|
|
||||||
|
Go's `\b` is ASCII-only and never fires after a Cyrillic letter. A pattern needs
|
||||||
|
an explicit `(\s|[?!.]|$)`.
|
||||||
|
|
||||||
|
`baselineGrammars` in `eval_test.go` mirrors `buildRouter` and has drifted before.
|
||||||
|
`WorldQueryGrammars` was wired into the daemon by V-655 and not into the mirror,
|
||||||
|
so the fixture scored a grammar set nobody runs. Fixed by V-659, worth 3 points
|
||||||
|
of destination.
|
||||||
|
|
||||||
|
### Praxis lifecycle rules
|
||||||
|
|
||||||
|
A **stative** lifecycle word ("готово", "принято") needs an item named beside it.
|
||||||
|
A bare **imperative** ("закрывай") may ask which one. It also requires a sentence
|
||||||
|
naming no object of its own. Otherwise "закрой шторы в комнате" goes to Praxis
|
||||||
|
instead of the house. A demonstrative ("отметь это как сделанное") resolves
|
||||||
|
against `h.surfacedItems` only when exactly one item was spoken. Otherwise the
|
||||||
|
turn goes back to the cascade rather than transitioning the wrong item.
|
||||||
|
|
||||||
|
### Slots on a stage 0 decision
|
||||||
|
|
||||||
|
`fillMatchedSlots` runs the stage 2 extractor over whatever a grammar built
|
||||||
|
(V-572, 2026-08-06). It fills only the slots the grammar left empty. A matched
|
||||||
|
value always wins, because the rule read a literal pattern and the extractor
|
||||||
|
guesses.
|
||||||
|
|
||||||
|
It did not run before. So `ReminderGrammar` handed the daemon `HasTime: false`
|
||||||
|
for "напомни в 11:00 позвонить маме", and `missingFor` read the silence as
|
||||||
|
absence and asked "Когда?". It is inert for every grammar but the reminder:
|
||||||
|
`Extract` fills Time, Fn and Key and nothing else. A stage 0 query costs 3.7µs
|
||||||
|
against 3.9µs before, benchmarked at 20000x.
|
||||||
|
|
||||||
|
`Slots.Text` is deliberately not filled. A grammar that left it empty meant it,
|
||||||
|
and `agendaQueryBuild` hands the query chain the utterance itself.
|
||||||
|
|
||||||
|
## Stage 0b: the routing heads
|
||||||
|
|
||||||
|
Routing has a bounded output space, so it is classification rather than
|
||||||
|
generation (owner's call, V-546,
|
||||||
|
`docs/plans/18-routing-heads-on-e5-small.md`). The 118M multilingual-e5-small is
|
||||||
|
already resident. A softmax cannot emit a value that does not exist, so no
|
||||||
|
grammar is needed. Max softmax is a calibratable confidence, where
|
||||||
|
`Confidence: 1.0` was a hardcode. Training costs roughly 5e15 FLOPs, so 10 to 30
|
||||||
|
minutes on the workstation. A 100M decoder from scratch is 10 to 20 GPU hours.
|
||||||
|
|
||||||
|
**Fine-tune a copy of the weights.** The resident embedder backs memory recall.
|
||||||
|
Training it in place couples routing accuracy to recall@1, with nothing in the
|
||||||
|
suite to name the trade.
|
||||||
|
|
||||||
|
Four heads share one masked mean pool, trained over three days. The measurements
|
||||||
|
are `docs/evals/2026-08-08-routing-heads-two-head.md`,
|
||||||
|
`docs/evals/2026-08-08-slot-head-three-head.md`,
|
||||||
|
`docs/evals/2026-08-08-clarify-head-four-head.md` and
|
||||||
|
`docs/evals/2026-08-08-massive-warm-start.md`.
|
||||||
|
|
||||||
|
| Head | Score | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| intent | 92.8% mean over 3 seeds | fixture is the 88 cases carrying an intent |
|
||||||
|
| destination | 80.8% mean, best 29/33 | beats the 12B teacher it was distilled from |
|
||||||
|
| slot BIO tags | 72.4% span F1 | still climbing when epoch selection stops it |
|
||||||
|
| clarify | catches 7.0 of 8, 2.3 false of 88 | parity with the cascade, no rules in front |
|
||||||
|
|
||||||
|
Read the best destination run as one seed and not a headline. One case is 3
|
||||||
|
points on a fixture this small. Head intent accuracy is **not** comparable to the
|
||||||
|
cascade's 76.0% and 84.4%. A softmax has no clarify class, so the head's fixture
|
||||||
|
is 88 cases and not 96.
|
||||||
|
|
||||||
|
Recall is 15/15 and world is 5/5.
|
||||||
|
|
||||||
|
**Mood is cut, not deferred.** The enum describes her own reply state, not the
|
||||||
|
speaker's emotion, and no dataset maps onto it.
|
||||||
|
|
||||||
|
### The clarify head
|
||||||
|
|
||||||
|
Clarify is not a value of intent, so a softmax cannot emit it. It is a second
|
||||||
|
question over the same pooled vector: can Maven act on this at all. Accuracy is
|
||||||
|
the wrong number here and a head that never asks scores 91.7%.
|
||||||
|
|
||||||
|
Confidence is the other half. Max softmax over the intent head reads 0.851 where
|
||||||
|
it is right and 0.604 where it is wrong. It ranks right above wrong in 83.4% of
|
||||||
|
pairs.
|
||||||
|
|
||||||
|
It is not free the way the slot head was. Intent, destination and slot F1 each
|
||||||
|
move down one to four points, inside the seed spread. `поужинал` is a false
|
||||||
|
clarify on every seed. That is the same defect `thinSingleToken` was narrowed for
|
||||||
|
on 2026-08-01.
|
||||||
|
|
||||||
|
The corpus is generated, because every existing row is answerable by
|
||||||
|
construction. The router-prompt agreement filter cannot work here. `routeGrammar`
|
||||||
|
has no clarify value, and a generated line always agrees with itself. A gemma
|
||||||
|
judge replaces it. The first judge called 24 of 40 answerable rows underspecified.
|
||||||
|
It judged against a generic assistant rather than against Maven's contract.
|
||||||
|
|
||||||
|
### The slot head
|
||||||
|
|
||||||
|
BIO slot tags had no Maven-domain corpus. That was true of found corpora and
|
||||||
|
false of made ones. `label_slots.py` distils spans out of gemma-4-12b under a
|
||||||
|
GBNF closed over Maven's own five slots. A span survives only when it is a
|
||||||
|
literal substring of the utterance, so the agreement filter costs no second call.
|
||||||
|
2178 spans over 1702 rows, 37 dropped, nothing unparsed.
|
||||||
|
|
||||||
|
Epoch selection reads the intent dev slice alone. That costs the slot head about
|
||||||
|
4 points.
|
||||||
|
|
||||||
|
### Warm start and the floor
|
||||||
|
|
||||||
|
The MASSIVE warm-start of step 2 is worth nothing here. Stock e5-small ties it on
|
||||||
|
intent and leads by a third of a case on destination. Nothing argues for keeping
|
||||||
|
that step.
|
||||||
|
|
||||||
|
The floor was a corpus defect and it is fixed. The first 120 floor rows carried
|
||||||
|
one sentence shape, so the head named a destination where the fixture says walk
|
||||||
|
the chain. Rotating six shapes took the floor 3/7 to 6/7 and destination 75.8% to
|
||||||
|
80.8%.
|
||||||
|
|
||||||
|
What is left is calendar at 3/6 on every seed, which training cannot move. The
|
||||||
|
possessive agenda rules claim those cases at stage 0 and name nothing, so no
|
||||||
|
label reaches the head.
|
||||||
|
|
||||||
|
### Reading them in Go
|
||||||
|
|
||||||
|
`RouterHeads` in `internal/router/heads.go` loads `router_heads.onnx` (V-664,
|
||||||
|
2026-08-08). It reads intent, destination and clarify off one forward pass.
|
||||||
|
|
||||||
|
Three rules around it, each measured:
|
||||||
|
|
||||||
|
- The **clarify head decides first**, before the intent threshold. It answers a
|
||||||
|
different question. A thin utterance scores low intent by construction, so
|
||||||
|
gating it cost 6 of 8 ambiguous cases.
|
||||||
|
- The **destination head is read on `IntentQuery` only**, since no other intent
|
||||||
|
reaches `queryWalk`.
|
||||||
|
- `headsThreshold` is 0.6, the measured knee. Every value up to 0.85 drops right
|
||||||
|
answers and keeps the same two wrong ones.
|
||||||
|
|
||||||
|
`voice.embedder.heads_path` is the whole switch. Empty, missing or unloadable
|
||||||
|
means the heads are nil. The cascade is then byte-for-byte what shipped before
|
||||||
|
them.
|
||||||
|
|
||||||
|
### The tokenizer bug the heads found
|
||||||
|
|
||||||
|
`encodeWord` in `onnxembedder.go` read every long word backwards until 2026-08-08.
|
||||||
|
It cost recall@1 7.4 points and recall@3 11.1. Nothing caught it, because seeds
|
||||||
|
and queries were mangled the same way and cosine survived. The heads found it.
|
||||||
|
They are trained through transformers and read through this.
|
||||||
|
|
||||||
|
The embedder id now carries a tokenizer revision (`@384/tok2`). So fixing the
|
||||||
|
tokenizer triggers `ReembedAll` the way swapping the model file does. Bump
|
||||||
|
`tokenizerRev` on any change to what it emits.
|
||||||
|
|
||||||
|
## Clarify
|
||||||
|
|
||||||
|
`Confidence: 1.0` was hardcoded in `llmrouter.go`. So the model path could never
|
||||||
|
ask for clarification, and it missed 6 of 6 refusal cases (V-359). The bug had a
|
||||||
|
second half. The model branch never consulted `r.threshold` at all, so a correct
|
||||||
|
low confidence would have been discarded anyway.
|
||||||
|
|
||||||
|
Fixed 2026-07-31 with structural signal feeding the same stage 3 gate the
|
||||||
|
classifier path already had (`gateLLMDecision` in `router.go`). Three signals: a
|
||||||
|
single-token utterance, a keyless fact, an act with no allowlisted fn.
|
||||||
|
|
||||||
|
Re-measured: missed clarify 6/6 to 1, at the cost of 3 false clarifies and 2.6
|
||||||
|
points of full accuracy. Two of the three false clarifies are acts the model
|
||||||
|
mis-routed and the gate caught. Asking beats wrongly executing, so the fixture
|
||||||
|
and the daemon disagree about what is correct there.
|
||||||
|
|
||||||
|
The third, `поужинал`, was a real defect. The single-token rule was an English
|
||||||
|
intuition. It does not transfer to Russian, where one word is routinely a whole
|
||||||
|
sentence.
|
||||||
|
|
||||||
|
Narrowed 2026-08-01. `thinSingleToken` (`internal/router/singletoken.go`) still
|
||||||
|
thins a bare one-word nominal. It spares two classes. One is a closed lexicon of
|
||||||
|
social and control singles ("привет", "стоп", "yes"). The other is any token
|
||||||
|
carrying a Russian verb ending, because a verb already contains its subject. Both
|
||||||
|
tests are offline and cost nothing. False clarifies 3 to 2, intent-only 74.0% to
|
||||||
|
75.3%.
|
||||||
|
|
||||||
|
The two remaining false clarifies are the act-with-no-allowlisted-fn arm of the
|
||||||
|
gate, not this rule.
|
||||||
|
|
||||||
|
## The destination
|
||||||
|
|
||||||
|
`query` was a shrug. The cascade sorted an utterance into one of seven intents,
|
||||||
|
then `IntentQuery` handed the turn to `querySources` in the daemon. That is
|
||||||
|
twenty-two branches deciding by seed similarity in a fixed order. It had no
|
||||||
|
fixture, no accuracy number, no model arm and no floor.
|
||||||
|
|
||||||
|
`Decision.Source` (`internal/router/source.go`) is the second half of the route
|
||||||
|
(V-655, 2026-08-07). Twelve destinations, not twenty-two. The three recall passes
|
||||||
|
plus `fact-by-key` are one destination from outside. So are search, Kiwix and the
|
||||||
|
URL reader.
|
||||||
|
|
||||||
|
`queryWalk` in `cmd/mavend/actions_query.go` takes sources **out** and moves none.
|
||||||
|
That is the safety argument. The table's order is load-bearing. Every comment on
|
||||||
|
it argues a reason between two sources. Above all it carries "the owner's data
|
||||||
|
first, then the world". Naming `SourceWorld` does not send the turn outside on its
|
||||||
|
own.
|
||||||
|
|
||||||
|
What comes out is only the sources that **guess**. Those decide a turn is theirs
|
||||||
|
by cosine against frozen seeds, then answer whatever they claimed. They hold no
|
||||||
|
table that could come back empty. Weather is the pure case and has no local data
|
||||||
|
at all. It was measured on the box 2026-08-07
|
||||||
|
(`docs/evals/2026-08-07-week-of-usage.md` section 4). It answered both "что такое
|
||||||
|
TCP?" and "сколько будет 17 на 23?" with "для какого города?". The feed answered
|
||||||
|
"какой у меня любимый язык?" with kernel headlines.
|
||||||
|
|
||||||
|
### Who may drop the personal boundary
|
||||||
|
|
||||||
|
The personal boundary guesses, so naming `SourceWorld` drops it. That is what
|
||||||
|
stops it answering "кто такой Линус Торвальдс?" with "не нашла у тебя такой
|
||||||
|
записи", which it did on 2026-08-07.
|
||||||
|
|
||||||
|
Three deciders name a destination and two of them infer it: the heads and the
|
||||||
|
resident model. An inferred `SourceWorld` on a question about him would reach
|
||||||
|
SearXNG. That widens what is asked rather than costing a local answer. So only a
|
||||||
|
stage 0 grammar may drop it (owner's call, V-666, 2026-08-09).
|
||||||
|
|
||||||
|
`Decision.SourceAnchored` carries the provenance. It is a field and not
|
||||||
|
`Stage == 0`. Stage 0 also means confidence 1.0 and an anchored claim band, and
|
||||||
|
one of those could stop implying the others. `definitionQueryPattern` claims "кто
|
||||||
|
такой X", so the 2026-08-07 case is still anchored and still answered.
|
||||||
|
|
||||||
|
`queryWalk` reads `SourceAnchored` for the query source marked `boundary: true`
|
||||||
|
and no other. Every other guesser still comes off the turn, whoever named the
|
||||||
|
destination. `TestOnlyAGrammarMayDropTheBoundary` and
|
||||||
|
`TestNamingRecallKeepsTheBoundary` pin both directions.
|
||||||
|
|
||||||
|
### The destination fixture
|
||||||
|
|
||||||
|
`want_source` on `eval.Case` is a pointer, because the destination has three
|
||||||
|
states and a bare string has two. Absent is every intent but query. Present and
|
||||||
|
empty is the `SourceUnknown` contract: name nothing and walk the chain. Present
|
||||||
|
and named is a destination the route must produce. Thirty-three of ninety-six
|
||||||
|
cases carry one.
|
||||||
|
|
||||||
|
A destination miss does **not** fail the case. It lands in `Outcome.SourceReason`
|
||||||
|
and never in `Reasons`, so `Accuracy` and `IntentAccuracy` mean what they meant.
|
||||||
|
`SourceAccuracy` is a second number over the labelled cases alone.
|
||||||
|
A route that lost its intent scores no destination hit. Otherwise a clarify would
|
||||||
|
satisfy an empty label for free.
|
||||||
|
|
||||||
|
Seven cases assert the floor and five of them are homelab operations. They cluster
|
||||||
|
because `SourceRecall`, `SourceNetwork` and `SourceAttention` overlap on every
|
||||||
|
question about the box. `mavpoll` writes its netdata and uptime-kuma observations
|
||||||
|
into the fact store recall reads. That is a finding about the enum, not a gap in
|
||||||
|
the labelling. The other two are `ru-query-005` and `ru-query-014`. No query
|
||||||
|
source reads the reminder store, and a deadline could sit in tasks, the calendar
|
||||||
|
or Praxis. The owner confirmed all seven floor labels on 2026-08-08.
|
||||||
|
|
||||||
|
### The model arm
|
||||||
|
|
||||||
|
`routeGrammar` carries a `source` rule closed over `router.Sources` plus the
|
||||||
|
empty floor (V-660, 2026-08-08). So the model cannot emit a destination that does
|
||||||
|
not exist. The prompt lists the twelve in Russian and says `""` is a normal answer
|
||||||
|
to give often. `LLMRouter.Route` reads it back through `ValidSource` and on
|
||||||
|
`IntentQuery` alone.
|
||||||
|
|
||||||
|
Against gemma-4-12b the cascade scores destination 24/33 with intent unmoved, and
|
||||||
|
recall goes 0/15 to 14/15.
|
||||||
|
|
||||||
|
**Stage 0 now costs four destination points.** It did not before. The four cases
|
||||||
|
the cascade loses and the model alone wins are all calendar. The possessive agenda
|
||||||
|
rules claim them first and name nothing on purpose. That caution was free while
|
||||||
|
nothing downstream could name anything either. It is not free now, and the fix is
|
||||||
|
the owner's call (V-660 open).
|
||||||
|
|
||||||
|
## The decision trace
|
||||||
|
|
||||||
|
Arbitration between the claimants on the utterance stream is order. It is
|
||||||
|
hardcoded in the pre-route resolver ladder, in `buildRouter` and in
|
||||||
|
`querySources`. Nothing recorded who lost until V-564.
|
||||||
|
|
||||||
|
`internal/decision` records one `Record` per turn. It holds every claimant, what
|
||||||
|
it would have made the turn, the score it reported, and how it ended. A claimant
|
||||||
|
won, declined, lost on score, was thinned by a gate or was **never asked**.
|
||||||
|
|
||||||
|
The record rides the context, the same seam `querysource.go` uses. So a claim
|
||||||
|
site cannot change a route, and a context with no record costs nothing. It is
|
||||||
|
installed in `runTurn`, so the mic, telegram and the web leave the same trail.
|
||||||
|
|
||||||
|
Adding a rung to the ladder in `runTurn` means adding its name to `preRouteLadder`
|
||||||
|
in `cmd/mavend/decisiontrace.go`. Otherwise that rung is silently missing from the
|
||||||
|
record.
|
||||||
|
|
||||||
|
### Why it persists now
|
||||||
|
|
||||||
|
The original rule was that nothing persists, because a turn record is read minutes
|
||||||
|
later or never. Storage was a 25-turn in-memory ring read over `ipc.TurnDecisions`
|
||||||
|
and rendered on `/trace`.
|
||||||
|
|
||||||
|
The owner reversed it on 2026-08-06 (V-629,
|
||||||
|
`docs/plans/21-persisting-the-routing-trace.md`). The routing heads cannot be
|
||||||
|
fitted or calibrated without real utterances. And 9 of the 31 modes in
|
||||||
|
`internal/modes` have no seed example at all.
|
||||||
|
|
||||||
|
The ring did not move. `cmd/mavend/routingtrace.go` is a second sink beside it,
|
||||||
|
writing `routing_traces` (migration #23). The utterance is stored in clear. A
|
||||||
|
384-dimension vector of a short sentence is substantially recoverable, so storing
|
||||||
|
vectors instead would be a privacy claim we cannot support. What makes it safe is
|
||||||
|
the same thing that makes the fact store safe. Retention is 14 days, enforced on
|
||||||
|
write and again on start, so a box that goes quiet does not keep every row.
|
||||||
|
Nothing reads it outward. `Store.Wipe` deletes it with everything else.
|
||||||
|
|
||||||
|
### Corrections
|
||||||
|
|
||||||
|
A correction is promoted out into a seed-shaped row in `routing_labels`
|
||||||
|
(migration #24) and kept, because a label is not a transcript. The transcript
|
||||||
|
still expires.
|
||||||
|
|
||||||
|
A turn marked wrong with no target is a usable negative, so naming the intent is
|
||||||
|
never required. The target is one of the seven intents and never free text.
|
||||||
|
|
||||||
|
All three reaches offer it as of 2026-08-06:
|
||||||
|
|
||||||
|
- `/chat` offers two buttons beside the reply, over `ipc.CorrectTurn` and the
|
||||||
|
trace id that rides back on `ipc.ChatReply`.
|
||||||
|
- Voice offers the `repair` rung, which has read spoken corrections since V-455.
|
||||||
|
It now writes the durable label beside the classifier seed it always wrote. A
|
||||||
|
spoken negative with no target is its own rung, `repair-negative` (V-636,
|
||||||
|
`docs/plans/22-correcting-a-turn.md`).
|
||||||
|
- Telegram offers an inline keyboard under the reply. It needed the chat to become
|
||||||
|
readable first (V-637, `docs/plans/23-inbound-telegram.md`). The poller is dark
|
||||||
|
unless the `telegram` block says `intake`. It long-polls, because the box takes
|
||||||
|
no inbound connections. It accepts `chat_id` and no other sender, and it drops
|
||||||
|
whatever queued while the daemon was down. It reaches the daemon through
|
||||||
|
`ipc.CoreAPI` alone.
|
||||||
|
|
||||||
|
The turn source is still `tap:text` for both telegram and the web. So provenance
|
||||||
|
cannot tell a chat turn from a typed one.
|
||||||
@@ -0,0 +1,80 @@
|
|||||||
|
# Session workflow: the five stores and the guards
|
||||||
|
|
||||||
|
*Last verified: 2026-08-09 @ a9b480a*
|
||||||
|
|
||||||
|
How a session starts, where each kind of writing belongs, and what the hooks
|
||||||
|
refuse. `CLAUDE.md` carries the commands. This file carries the reasoning.
|
||||||
|
|
||||||
|
## Five stores
|
||||||
|
|
||||||
|
Each owns something the others must not hold.
|
||||||
|
|
||||||
|
| Store | Holds | Lifetime |
|
||||||
|
|---|---|---|
|
||||||
|
| Vikunja task | goal, constraints, assumption ledger, status | durable |
|
||||||
|
| `CLAUDE.md`, `AGENTS.md` | what an agent must know before touching code | durable |
|
||||||
|
| `docs/` | design, measurements, decisions | durable |
|
||||||
|
| `TASK.md` | the brief for this branch, written by `task start`, immutable | one branch |
|
||||||
|
| `HANDOFF.md` | only what the next agent needs to resume | one session |
|
||||||
|
|
||||||
|
`TASK.md` and `.task/` are excluded through `.git/info/exclude`. `HANDOFF.md` is
|
||||||
|
gitignored and injected at session start. If a line in the handoff would still
|
||||||
|
matter next week, it is in the wrong file.
|
||||||
|
|
||||||
|
## Doc tiers
|
||||||
|
|
||||||
|
Tiered by path, so staleness is visible from the filename.
|
||||||
|
|
||||||
|
- Files directly under `docs/` are living. They carry a
|
||||||
|
`Last verified: <date> @ <sha>` line and are corrected in place.
|
||||||
|
- Files under `docs/evals/` are dated measurements and are never edited after
|
||||||
|
the day. A newer number is a new file, not an edit.
|
||||||
|
- Files under `docs/archive/` are dead and read by nobody by default.
|
||||||
|
|
||||||
|
## Vikunja
|
||||||
|
|
||||||
|
This repo is project **Maven** (ID 2). MCP at `http://localhost:9100/mcp`, or
|
||||||
|
`http://192.168.1.104:9100/mcp` from workpc. Feature, bug and deploy tasks go
|
||||||
|
there.
|
||||||
|
|
||||||
|
A task holds the goal, the constraints and the assumption ledger. A session
|
||||||
|
without a task id cannot be resumed by anyone, so a session with none asks for
|
||||||
|
one first.
|
||||||
|
|
||||||
|
**Close a finished task with `done: true` and nothing else** (owner's call,
|
||||||
|
2026-08-07). Do not write a completion summary into the description on the way
|
||||||
|
out. It is lost anyway, and the durable record is the commit messages and the
|
||||||
|
merged PR. `update_task` carrying a `description` resets `done` to false, which
|
||||||
|
is why a write-up ever took two calls.
|
||||||
|
|
||||||
|
## The branch tool
|
||||||
|
|
||||||
|
`~/.local/bin/task` owns the branch, the commit identity and the PR. One task,
|
||||||
|
one session, one PR.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
task start <vikunja-id> # branch off origin/master, write TASK.md, fetch review comments
|
||||||
|
task pr # push, open or refresh the PR, label Vikunja, notify
|
||||||
|
task comments # re-pull this branch's review comments into .task/
|
||||||
|
```
|
||||||
|
|
||||||
|
`/pickup` opens a session and `/wrap` closes it. Wrap at roughly half context
|
||||||
|
rather than letting the session compact.
|
||||||
|
|
||||||
|
## Guards
|
||||||
|
|
||||||
|
Two hooks in `.githooks/`, tracked, wired with `core.hooksPath`. A fresh clone
|
||||||
|
needs `git config core.hooksPath .githooks`.
|
||||||
|
|
||||||
|
- `pre-commit` refuses master, and refuses more than 300 changed lines in
|
||||||
|
non-markdown files. Markdown is exempt and may land as one batch.
|
||||||
|
- `commit-msg` requires the subject to end with `(V-<id>)`. `V-` and not `#`,
|
||||||
|
because Gitea autolinks `#123` to a Gitea issue, which is the wrong tracker.
|
||||||
|
|
||||||
|
Two more guards live outside the repo, in `~/.claude/hooks/`. `diff-budget.sh`
|
||||||
|
blocks further edits past 600 changed lines on a `task/` branch.
|
||||||
|
`prose_lint_hook.py` checks prose on every write. Both measure against
|
||||||
|
`origin/master`, so a local master that is ahead of the remote makes the diff
|
||||||
|
budget read high.
|
||||||
|
|
||||||
|
`--no-verify` exists. Using it means saying why in the commit body.
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
# The world chain
|
||||||
|
|
||||||
|
*Last verified: 2026-08-09 @ a9b480a*
|
||||||
|
|
||||||
|
What happens when the answer is not his. `CLAUDE.md` carries the boundary rule.
|
||||||
|
This file carries the mechanism and the measurements behind it.
|
||||||
|
|
||||||
|
## What replaced "never phones home"
|
||||||
|
|
||||||
|
That promise was deprecated on 2026-07-31 by the owner's call. A 1.7B does not
|
||||||
|
know enough to answer world questions, so she reads external sources. Four rules
|
||||||
|
replaced it:
|
||||||
|
|
||||||
|
- **No telemetry, no cloud model, no third-party account.** That part never
|
||||||
|
changes. Nothing about Maven is reported to anyone and inference stays on the
|
||||||
|
box.
|
||||||
|
- **The owner's data first, then the world.** Every source reading his facts,
|
||||||
|
notes, calendar, tasks or house runs before anything outside. The personal
|
||||||
|
boundary sits between them. Reading beats recalling for a small model.
|
||||||
|
- **The owner's notes and facts are never search input.** Only the utterance
|
||||||
|
goes out. Never the persona block, the history, or matched notes.
|
||||||
|
- **External search is allowed and off unless configured**, like weather and
|
||||||
|
telegram. The code default is off. `deploy/mavend.json` ships a `search`
|
||||||
|
block, so it is on for this box and deleting the block turns it off again.
|
||||||
|
|
||||||
|
**Live search leads and the ZIMs are the fallback** (owner's call, 2026-08-02).
|
||||||
|
A self-hosted SearXNG answers first. The Kiwix ZIMs on homesrv answer when the
|
||||||
|
search is empty, unreachable, or the line is down.
|
||||||
|
|
||||||
|
## The gate is emptiness and nothing else
|
||||||
|
|
||||||
|
`Response.Empty()` is the whole gate. There is no quality threshold in front of
|
||||||
|
it. Four signals were tried and none separates a real question from an invented
|
||||||
|
one.
|
||||||
|
|
||||||
|
Token overlap was the closest and it still fails. "столица Франции" would lose
|
||||||
|
its answer, because the answer is Париж and that word is not in the question
|
||||||
|
(`docs/evals/2026-08-05-search-quality-signals.md`).
|
||||||
|
|
||||||
|
**The embedder is not a fifth signal.** Query-to-passage cosine measures topic
|
||||||
|
and not whether the passage answers. The two sets overlap
|
||||||
|
(`docs/evals/2026-08-09-kiwix-topic-retrieval.md`).
|
||||||
|
|
||||||
|
## Timeouts
|
||||||
|
|
||||||
|
The connect phase alone is capped at `dialTimeout`, 1.5s, because a blackholed
|
||||||
|
host once cost the owner 8 seconds. A slow instance that did connect keeps the
|
||||||
|
full 8 (`docs/evals/2026-08-05-kiwix-offline-fallback.md`).
|
||||||
|
|
||||||
|
## Kiwix
|
||||||
|
|
||||||
|
**A Russian question reads `wikipedia_ru_all_maxi_2026-02` verbatim** through
|
||||||
|
`kiwix.book_ru`. The RU→EN rewriter is the workaround for an English book and is
|
||||||
|
skipped there. Kiwix catalog names come from the filename, not the `<name>`
|
||||||
|
field.
|
||||||
|
|
||||||
|
**Kiwix ranks by keyword overlap.** Never send it a whole sentence. `kiwix.Topic`
|
||||||
|
drops the narrative request, the interrogative and a verb behind one.
|
||||||
|
|
||||||
|
`kiwix.TitlePath` tries the exact article first, since a ZIM is addressable by
|
||||||
|
title and a wrong title is a 404. `TitleCandidates` tries the spoken form and
|
||||||
|
then the capitalized one. Both apply on the verbatim path alone. The rewriter
|
||||||
|
already reduces a question, and reducing twice takes the topic off its input
|
||||||
|
(V-668).
|
||||||
|
|
||||||
|
## Which source answered
|
||||||
|
|
||||||
|
**The claiming query source is readable on `/chat`** as a badge beside the
|
||||||
|
reply. It is carried on `ipc.ChatReply.Source` and noted by `noteQuerySource` in
|
||||||
|
`cmd/mavend/querysource.go`. It rides the context, so `handleText` keeps the one
|
||||||
|
string signature the mic, telegram and the web share.
|
||||||
+104
-51
@@ -9,15 +9,18 @@
|
|||||||
//
|
//
|
||||||
// The wire is symmetric: a Request from the client is answered by a
|
// The wire is symmetric: a Request from the client is answered by a
|
||||||
// Response with a matching ID, OR a server-initiated Push frame (no ID)
|
// Response with a matching ID, OR a server-initiated Push frame (no ID)
|
||||||
// may arrive interleaved. SendRequest loops reading frames, drops Push
|
// may arrive interleaved. One reader goroutine per connection owns the
|
||||||
// frames to the harness if a receiver is running (or silently if not),
|
// socket. It hands each Response to whichever SendRequest is waiting on
|
||||||
// and returns the first Response with the matching ID.
|
// that ID and each Push to the handler, so a client may send and listen
|
||||||
|
// at the same time on one conn. mavwaked needs exactly that: it speaks
|
||||||
|
// utterances and it must hear nudges, and the server routes a nudge to
|
||||||
|
// the session that spoke most recently, so a second listening conn would
|
||||||
|
// never be picked (V-671).
|
||||||
package voice
|
package voice
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"errors"
|
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"net"
|
"net"
|
||||||
@@ -41,13 +44,22 @@ type PushHandler interface {
|
|||||||
// Client — one connection to the voice.Server.
|
// Client — one connection to the voice.Server.
|
||||||
type Client struct {
|
type Client struct {
|
||||||
addr string
|
addr string
|
||||||
mu sync.Mutex
|
|
||||||
c net.Conn
|
|
||||||
nextID atomic.Uint64
|
nextID atomic.Uint64
|
||||||
|
|
||||||
// pushCh fan-out: a reader goroutine (started by RunPushReceiver)
|
mu sync.Mutex
|
||||||
// writes Push frames here; SendRequest also drains it when no reader
|
c net.Conn
|
||||||
// is running (drops the frame in that case).
|
// pending holds one channel per in-flight request, keyed by frame id.
|
||||||
|
// The reader goroutine delivers the Response here and deletes the entry.
|
||||||
|
pending map[uint64]chan *Response
|
||||||
|
// dead is closed by the reader goroutine when this conn ends, so a
|
||||||
|
// waiting SendRequest fails at once instead of at its own deadline.
|
||||||
|
dead chan struct{}
|
||||||
|
|
||||||
|
// wmu serialises writes. Frames must not interleave on the wire.
|
||||||
|
wmu sync.Mutex
|
||||||
|
|
||||||
|
// pushH is set by RunPushReceiver and survives a reconnect, because the
|
||||||
|
// client that wants pushes wants them on whatever conn it ends up with.
|
||||||
pushMu sync.Mutex
|
pushMu sync.Mutex
|
||||||
pushH PushHandler
|
pushH PushHandler
|
||||||
}
|
}
|
||||||
@@ -55,6 +67,15 @@ type Client struct {
|
|||||||
// Dial returns a Client that will connect to addr on first use.
|
// Dial returns a Client that will connect to addr on first use.
|
||||||
func Dial(addr string) *Client { return &Client{addr: addr} }
|
func Dial(addr string) *Client { return &Client{addr: addr} }
|
||||||
|
|
||||||
|
// Connect opens the conn now rather than on the first request. mavwaked calls
|
||||||
|
// it at startup: the server registers a session on accept, and a client that
|
||||||
|
// has never connected cannot be sent a nudge.
|
||||||
|
func (c *Client) Connect(ctx context.Context) error {
|
||||||
|
c.mu.Lock()
|
||||||
|
defer c.mu.Unlock()
|
||||||
|
return c.ensureConnLocked(ctx)
|
||||||
|
}
|
||||||
|
|
||||||
// Close releases the conn. Idempotent.
|
// Close releases the conn. Idempotent.
|
||||||
func (c *Client) Close() error {
|
func (c *Client) Close() error {
|
||||||
c.mu.Lock()
|
c.mu.Lock()
|
||||||
@@ -75,12 +96,14 @@ func (c *Client) PushToTalk(ctx context.Context, a audio.Audio, lang string) (Pu
|
|||||||
return out, err
|
return out, err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// requestTimeout bounds a round-trip with no deadline on its context. It is
|
||||||
|
// generous because the far end runs speech-to-text, a router and a voice.
|
||||||
|
const requestTimeout = 120 * time.Second
|
||||||
|
|
||||||
// SendRequest sends one Request frame and waits for the matching Response.
|
// SendRequest sends one Request frame and waits for the matching Response.
|
||||||
// Push frames received while waiting are dropped on the floor UNLESS a
|
// Push frames arriving meanwhile go to the handler on the reader goroutine,
|
||||||
// PushHandler has been wired via RunPushReceiver, in which case the handler
|
// so listening and sending on one Client is supported rather than merely
|
||||||
// is invoked inline (still synchronous with the SendRequest caller's
|
// tolerated.
|
||||||
// read). For sanity, the reference client runs either one-shot (no
|
|
||||||
// receiver) or interactive (RunPushReceiver, no concurrent SendRequest).
|
|
||||||
func (c *Client) SendRequest(ctx context.Context, m Method, params any, out any) error {
|
func (c *Client) SendRequest(ctx context.Context, m Method, params any, out any) error {
|
||||||
body, err := marshalParams(params)
|
body, err := marshalParams(params)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -94,33 +117,38 @@ func (c *Client) SendRequest(ctx context.Context, m Method, params any, out any)
|
|||||||
c.mu.Unlock()
|
c.mu.Unlock()
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
conn := c.c
|
conn, dead := c.c, c.dead
|
||||||
|
ch := make(chan *Response, 1)
|
||||||
|
c.pending[id] = ch
|
||||||
c.mu.Unlock()
|
c.mu.Unlock()
|
||||||
|
|
||||||
if dl, ok := ctx.Deadline(); ok {
|
c.wmu.Lock()
|
||||||
_ = conn.SetDeadline(dl)
|
err = writeFrame(conn, &req)
|
||||||
} else {
|
c.wmu.Unlock()
|
||||||
_ = conn.SetDeadline(time.Now().Add(120 * time.Second))
|
|
||||||
}
|
|
||||||
defer conn.SetDeadline(time.Time{})
|
|
||||||
|
|
||||||
if err := writeFrame(conn, &req); err != nil {
|
|
||||||
c.teardown()
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
for {
|
|
||||||
resp, push, err := readOneFrame(conn)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
c.teardown()
|
c.forget(id)
|
||||||
|
c.teardownConn(conn)
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
if push != nil {
|
|
||||||
c.deliverPush(*push)
|
timer := time.NewTimer(requestTimeout)
|
||||||
continue
|
defer timer.Stop()
|
||||||
}
|
|
||||||
if resp.ID != id {
|
var resp *Response
|
||||||
continue // not ours; ignore (singleplex ⇒ shouldn't happen)
|
select {
|
||||||
|
case resp = <-ch:
|
||||||
|
case <-dead:
|
||||||
|
c.forget(id)
|
||||||
|
return fmt.Errorf("voice: connection closed before reply")
|
||||||
|
case <-ctx.Done():
|
||||||
|
c.forget(id)
|
||||||
|
return ctx.Err()
|
||||||
|
case <-timer.C:
|
||||||
|
c.forget(id)
|
||||||
|
c.teardownConn(conn)
|
||||||
|
return fmt.Errorf("voice: no reply within %s", requestTimeout)
|
||||||
}
|
}
|
||||||
|
|
||||||
if resp.Error != nil {
|
if resp.Error != nil {
|
||||||
return hydrate(resp.Error)
|
return hydrate(resp.Error)
|
||||||
}
|
}
|
||||||
@@ -131,21 +159,27 @@ func (c *Client) SendRequest(ctx context.Context, m Method, params any, out any)
|
|||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// forget drops an abandoned request so a late Response is discarded rather
|
||||||
|
// than delivered to nobody.
|
||||||
|
func (c *Client) forget(id uint64) {
|
||||||
|
c.mu.Lock()
|
||||||
|
delete(c.pending, id)
|
||||||
|
c.mu.Unlock()
|
||||||
}
|
}
|
||||||
|
|
||||||
// RunPushReceiver spawns a reader goroutine that delivers Push frames to h
|
// RunPushReceiver wires h and blocks until the conn ends or ctx is
|
||||||
// until the conn closes or Close is called. Today's reference client uses
|
// cancelled. Frames are read by the per-conn reader goroutine, so a client
|
||||||
// this in -listen mode (proactive voice playback). SendRequest and
|
// may call SendRequest on the same Client while this is running. Returns nil
|
||||||
// RunPushReceiver SHOULD NOT be used concurrently on the same Client — the
|
// when the conn ended, so a caller that wants to stay reachable reconnects
|
||||||
// wire is singleplex at the reference client's scale; production picks one
|
// and calls it again.
|
||||||
// mode per conn. Returns when the goroutine ends (ctx cancel or conn close).
|
|
||||||
func (c *Client) RunPushReceiver(ctx context.Context, h PushHandler) error {
|
func (c *Client) RunPushReceiver(ctx context.Context, h PushHandler) error {
|
||||||
c.mu.Lock()
|
c.mu.Lock()
|
||||||
if err := c.ensureConnLocked(ctx); err != nil {
|
if err := c.ensureConnLocked(ctx); err != nil {
|
||||||
c.mu.Unlock()
|
c.mu.Unlock()
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
conn := c.c
|
dead := c.dead
|
||||||
c.mu.Unlock()
|
c.mu.Unlock()
|
||||||
|
|
||||||
c.pushMu.Lock()
|
c.pushMu.Lock()
|
||||||
@@ -158,21 +192,34 @@ func (c *Client) RunPushReceiver(ctx context.Context, h PushHandler) error {
|
|||||||
c.pushMu.Unlock()
|
c.pushMu.Unlock()
|
||||||
}()
|
}()
|
||||||
|
|
||||||
for {
|
|
||||||
select {
|
select {
|
||||||
case <-ctx.Done():
|
case <-ctx.Done():
|
||||||
return ctx.Err()
|
return ctx.Err()
|
||||||
default:
|
case <-dead:
|
||||||
}
|
|
||||||
_, push, err := readOneFrame(conn)
|
|
||||||
if err != nil {
|
|
||||||
if errors.Is(err, io.EOF) || errors.Is(err, net.ErrClosed) {
|
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
return err
|
}
|
||||||
|
|
||||||
|
// readLoop owns conn for its whole life. It ends on any read error, which is
|
||||||
|
// how a closed conn, a killed server and a cancelled dial all arrive here.
|
||||||
|
func (c *Client) readLoop(conn net.Conn, dead chan struct{}) {
|
||||||
|
defer close(dead)
|
||||||
|
for {
|
||||||
|
resp, push, err := readOneFrame(conn)
|
||||||
|
if err != nil {
|
||||||
|
c.teardownConn(conn)
|
||||||
|
return
|
||||||
}
|
}
|
||||||
if push != nil {
|
if push != nil {
|
||||||
c.deliverPush(*push)
|
c.deliverPush(*push)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
c.mu.Lock()
|
||||||
|
ch := c.pending[resp.ID]
|
||||||
|
delete(c.pending, resp.ID)
|
||||||
|
c.mu.Unlock()
|
||||||
|
if ch != nil {
|
||||||
|
ch <- resp
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -196,13 +243,19 @@ func (c *Client) ensureConnLocked(ctx context.Context) error {
|
|||||||
return fmt.Errorf("voice: dial %s: %w", c.addr, err)
|
return fmt.Errorf("voice: dial %s: %w", c.addr, err)
|
||||||
}
|
}
|
||||||
c.c = conn
|
c.c = conn
|
||||||
|
c.pending = make(map[uint64]chan *Response)
|
||||||
|
c.dead = make(chan struct{})
|
||||||
|
go c.readLoop(conn, c.dead)
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *Client) teardown() {
|
// teardownConn closes conn and forgets it, but only if it is still the live
|
||||||
|
// one. A reconnect may already have replaced it, and closing the new conn
|
||||||
|
// because the old one died takes the client down on every hiccup.
|
||||||
|
func (c *Client) teardownConn(conn net.Conn) {
|
||||||
c.mu.Lock()
|
c.mu.Lock()
|
||||||
defer c.mu.Unlock()
|
defer c.mu.Unlock()
|
||||||
if c.c != nil {
|
if c.c != nil && c.c == conn {
|
||||||
_ = c.c.Close()
|
_ = c.c.Close()
|
||||||
c.c = nil
|
c.c = nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -221,3 +221,113 @@ func TestClientListenModeReceivesPush(t *testing.T) {
|
|||||||
type pushHandlerFunc func(Push)
|
type pushHandlerFunc func(Push)
|
||||||
|
|
||||||
func (f pushHandlerFunc) OnPush(p Push) { f(p) }
|
func (f pushHandlerFunc) OnPush(p Push) { f(p) }
|
||||||
|
|
||||||
|
// The whole point of the per-conn reader (V-671): mavwaked speaks utterances
|
||||||
|
// and must hear nudges, and the server routes a nudge to the session that
|
||||||
|
// spoke most recently. A second listening conn would never be picked, so both
|
||||||
|
// directions have to share one conn.
|
||||||
|
func TestClientSendsAndListensOnOneConn(t *testing.T) {
|
||||||
|
l := newTestListener(t)
|
||||||
|
sess := NewSessions()
|
||||||
|
h := &stubHandler{}
|
||||||
|
srv := NewServer(l.Addr().String(), h, sess)
|
||||||
|
if err := srv.Listen(); err != nil {
|
||||||
|
t.Fatalf("listen: %v", err)
|
||||||
|
}
|
||||||
|
defer func() {
|
||||||
|
_ = srv.Close()
|
||||||
|
waitPort()
|
||||||
|
}()
|
||||||
|
go func() { _ = srv.Serve() }()
|
||||||
|
|
||||||
|
c := Dial(l.Addr().String())
|
||||||
|
defer c.Close()
|
||||||
|
|
||||||
|
got := make(chan audio.Audio, 4)
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
go func() {
|
||||||
|
_ = c.RunPushReceiver(ctx, pushHandlerFunc(func(p Push) {
|
||||||
|
var ap AudioNudgePush
|
||||||
|
if err := json.Unmarshal(p.Params, &ap); err == nil {
|
||||||
|
got <- ap.Audio
|
||||||
|
}
|
||||||
|
}))
|
||||||
|
}()
|
||||||
|
|
||||||
|
for i := 0; i < 100 && sess.Active() < 1; i++ {
|
||||||
|
time.Sleep(10 * time.Millisecond)
|
||||||
|
}
|
||||||
|
if sess.Active() != 1 {
|
||||||
|
t.Fatalf("active sessions = %d, want exactly 1", sess.Active())
|
||||||
|
}
|
||||||
|
|
||||||
|
// A round-trip while the receiver is running. Before the reader owned the
|
||||||
|
// conn, this and the receiver raced for every frame.
|
||||||
|
resp, err := c.PushToTalk(context.Background(), audio.Audio{Format: audio.PCM16kMono, Bytes: []byte("hello")}, "ru")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("PushToTalk with a receiver running: %v", err)
|
||||||
|
}
|
||||||
|
if resp.ReplyText != "got it" {
|
||||||
|
t.Fatalf("ReplyText = %q, want %q", resp.ReplyText, "got it")
|
||||||
|
}
|
||||||
|
|
||||||
|
// And the nudge still arrives, on the session that just spoke.
|
||||||
|
err = sess.PushToMostRecent(context.Background(), AudioNudgePush{
|
||||||
|
RuleName: "after-speaking",
|
||||||
|
Audio: audio.Audio{Format: audio.PCM16kMono, Bytes: []byte("proactive")},
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("PushToMostRecent: %v", err)
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case a := <-got:
|
||||||
|
if string(a.Bytes) != "proactive" {
|
||||||
|
t.Fatalf("received %q, want the nudge audio", string(a.Bytes))
|
||||||
|
}
|
||||||
|
case <-time.After(2 * time.Second):
|
||||||
|
t.Fatal("nudge never reached the handler after the client had spoken")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A request abandoned by its context must not leave its slot behind, or a
|
||||||
|
// long-running client leaks one channel per timeout.
|
||||||
|
func TestClientForgetsAbandonedRequests(t *testing.T) {
|
||||||
|
l := newTestListener(t)
|
||||||
|
sess := NewSessions()
|
||||||
|
srv := NewServer(l.Addr().String(), &blockingHandler{}, sess)
|
||||||
|
if err := srv.Listen(); err != nil {
|
||||||
|
t.Fatalf("listen: %v", err)
|
||||||
|
}
|
||||||
|
defer func() {
|
||||||
|
_ = srv.Close()
|
||||||
|
waitPort()
|
||||||
|
}()
|
||||||
|
go func() { _ = srv.Serve() }()
|
||||||
|
|
||||||
|
c := Dial(l.Addr().String())
|
||||||
|
defer c.Close()
|
||||||
|
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
|
||||||
|
defer cancel()
|
||||||
|
_, err := c.PushToTalk(ctx, audio.Audio{Format: audio.PCM16kMono, Bytes: []byte("x")}, "ru")
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected the round-trip to fail on its context")
|
||||||
|
}
|
||||||
|
|
||||||
|
c.mu.Lock()
|
||||||
|
n := len(c.pending)
|
||||||
|
c.mu.Unlock()
|
||||||
|
if n != 0 {
|
||||||
|
t.Fatalf("pending = %d after an abandoned request, want 0", n)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// blockingHandler never answers, so the client's context is what ends the
|
||||||
|
// round-trip.
|
||||||
|
type blockingHandler struct{}
|
||||||
|
|
||||||
|
func (blockingHandler) HandlePushToTalk(ctx context.Context, _ PushToTalkReq, _ uint64) (PushToTalkResp, error) {
|
||||||
|
<-ctx.Done()
|
||||||
|
return PushToTalkResp{}, ctx.Err()
|
||||||
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user