Compare commits
42 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 240d53a96a | |||
| d8efb667c7 | |||
| 25ed201c4d | |||
| a926383827 | |||
| 557f5a3acc | |||
| 17e6195aeb | |||
| 14f2725452 | |||
| f8beee8416 | |||
| 634f82717c | |||
| 353b8f5a16 | |||
| d1b8519239 | |||
| c0f4074a5d | |||
| 9bb342569b | |||
| 5596cdddbc | |||
| 1c13d2265b | |||
| 95e7427153 | |||
| a1d018dc47 | |||
| 9f714b7ae8 | |||
| ef3ee1e00a | |||
| 99e73ea653 | |||
| ab1784f5e1 | |||
| 2c73493bf8 | |||
| ff202c0c35 | |||
| 02d96e611d | |||
| 62eef01c18 | |||
| 1a8aed35b8 | |||
| ce6a6821a9 | |||
| 479b0c4475 | |||
| 877b1fd4f8 | |||
| 21a42cb3e6 | |||
| b8279f6a22 | |||
| 8c30971a96 | |||
| d0ea927ac3 | |||
| 9c7bafd5b1 | |||
| 1f1e002789 | |||
| ce91d20ac8 | |||
| 50130cdffb | |||
| a9b480a78f | |||
| c5264deb46 | |||
| 1cb269886e | |||
| 7a9b9cc669 | |||
| 31b5093403 |
@@ -1,805 +1,211 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
Guidance for Claude Code (claude.ai/code) working in this repository.
|
||||
|
||||
Maven is a self-hosted, privacy-first voice assistant (Russian + English). Go daemons
|
||||
talking over unix sockets; one resident small model for routing + phrasing; whisper.cpp STT, piper TTS.
|
||||
Deploy target is a Ryzen laptop (homesrv) with Vulkan offload to the Vega iGPU (`n_gpu_layers: 99`,
|
||||
compose passes `/dev/dri` + the render gid) — the resident model stays ≤1.7B either way.
|
||||
**This is a rules file.** It loads into every session, so it carries only what
|
||||
changes what an agent does. A measurement belongs in `docs/evals/`, dated and
|
||||
never edited after the day. A subsystem's reasoning belongs in its living doc
|
||||
under `docs/`. Read that doc before changing the subsystem.
|
||||
|
||||
**Resident model:** currently **Qwen3-1.7B** (`UD-Q4_K_XL`), stock — not yet the CPT'd one.
|
||||
It replaced Qwen3.5-0.8B on 2026-07-31 because it measured better on both fixtures we have:
|
||||
67.5% vs 59.7% intent-only on the 77-case RU routing fixture, and 20/27 vs 11-17/27 on the
|
||||
talk fixture. See `docs/evals/2026-07-31-model-bakeoff.md`. It is a Thinking variant, so `n_ctx` is 4096
|
||||
— reasoning tokens need the room, and 4096 is what the scores above were measured at.
|
||||
| Read this | Before |
|
||||
|---|---|
|
||||
| `docs/routing.md` | touching `internal/router/` or `queryWalk` |
|
||||
| `docs/deployment.md` | touching a daemon, compose, a systemd unit or the web UI |
|
||||
| `docs/offload.md` | touching a daemon seam or adding a model caller |
|
||||
| `docs/world.md` | touching search, Kiwix or the world chain |
|
||||
| `docs/language.md` | changing a prompt contract or a Russian word list |
|
||||
| `docs/ecosystem.md` | touching Nexus, Praxis or Hexis |
|
||||
| `docs/rearchitecture.md`, `docs/design.md` | changing the shape of anything |
|
||||
| `docs/workflow.md` | the five stores, the doc tiers, the guards |
|
||||
| `docs/caveats/` | a known limit, its task id and its revisit trigger |
|
||||
| `docs/CLAUDE.md` | which tier a doc belongs in, and what each one holds |
|
||||
| `AGENTS.md` | local preview, screenshots, model downloads |
|
||||
|
||||
The **target** is still the locally CPT'd **Qwen3-1.7B** (Vikunja #122, training in flight).
|
||||
Stock already speaks good Russian; what it gets wrong is the persona — it writes `я рад`,
|
||||
masculine, where Maven needs `рада`. That is what the CPT is for.
|
||||
## What Maven is
|
||||
|
||||
**Do not bother with sub-500M models.** LFM2.5-230M and 350M were measured on 2026-07-31 and
|
||||
both are unusable in Russian: the 350M routes at 5.2% (worse than guessing) and answers
|
||||
"столица Франции?" with the invented non-word "Сторзит"; the 230M replies to Russian in
|
||||
Spanish. Their strong published IFEval/BFCL numbers are English-only. Model files live in
|
||||
`/mnt/hdd1/llms`, bind-mounted to `/opt/maven/models/llm` — which **shadows** the repo's
|
||||
`models/llm/`, so the LFM2.5 gguf sitting there is not loaded by anything. Swapping the resident
|
||||
model is a one-line change to `phraser.model_path` in `deploy/mavend.json`.
|
||||
A self-hosted, privacy-first voice assistant in Russian and English. Go daemons
|
||||
talk over unix sockets. One resident small model routes and phrases. whisper.cpp
|
||||
does speech-to-text and piper does text-to-speech.
|
||||
|
||||
See `docs/rearchitecture.md` for the target architecture, `docs/design.md` for the folded design spec, and
|
||||
`AGENTS.md` for local-preview + model-download recipes.
|
||||
The resident model is **Qwen3-1.7B** (`UD-Q4_K_XL`) on homesrv, a Thinking
|
||||
variant at `n_ctx` 4096. Keep it at 1.7B or under. Sub-500M models are unusable
|
||||
in Russian (`docs/evals/2026-07-31-model-bakeoff.md`). Model files live in
|
||||
`/mnt/hdd1/llms`, bind-mounted over the repo's `models/llm/`, so a gguf sitting
|
||||
in the repo is loaded by nothing.
|
||||
|
||||
**Model work is moving to the workstation** (owner's call, 2026-08-02). homesrv cannot grow a
|
||||
GPU and the workstation has 16GB of VRAM. So the resident model, STT and TTS become preferred
|
||||
remotes with a floor on homesrv. The workstation is never assumed up. Fall back silently when
|
||||
it would only do the job better. Name the gap when the 1.7B cannot do it at all. The embedder
|
||||
stays on homesrv permanently, because it backs that floor. It is multilingual-e5-small,
|
||||
quantized and asymmetric — `EmbedQuery` and `EmbedPassage` apply the `query:`/`passage:`
|
||||
prefixes it was trained with, and calling plain `Embed` on a note is a bug. It replaced
|
||||
MiniLM and bought ten points of recall@1 and 2.5× the speed; see
|
||||
`docs/evals/2026-08-04-recall-e5-small.md`. Read `docs/offload.md` before
|
||||
touching a daemon seam or adding a model caller. Vikunja #483 is the umbrella, #484 to #487
|
||||
are the work.
|
||||
The workstation is workpc and it holds the remote model and speech-to-text.
|
||||
**It is never assumed up.** **Fall back silently** when it would only do the job
|
||||
better. **Name the gap** when the resident model cannot do the job at all.
|
||||
|
||||
Both halves are wired as of 2026-08-03. Routing and replies prefer the workstation silently
|
||||
through `modelSeam`; nudge and reminder phrasing prefer it silently inside the phraser. A
|
||||
world question goes through `LLMPhraser.PhraseWorld` and names the gap when the card is not
|
||||
free — `worldGap` in `cmd/mavend/worldmodel.go`, which the owner hears instead of an invented
|
||||
answer. A box with no `workstation` block behaves exactly as it did before the seam: naming
|
||||
a gap requires a gap. The offload table in `docs/offload.md` says which caller is which.
|
||||
**The embedder stays on homesrv permanently**, because it backs that floor.
|
||||
`EmbedQuery` and `EmbedPassage` apply the `query:` and `passage:` prefixes
|
||||
multilingual-e5-small was trained with. Calling plain `Embed` on a note is a bug.
|
||||
|
||||
**Speech-to-text moved on 2026-08-09** (V-486). `sttSeam` in `cmd/mavend/voicewire.go`
|
||||
builds an `stt.Pair` beside `modelSeam`, preferring CrisperWhisper 2.0 turbo on workpc
|
||||
with mavsttd as the floor. It takes only the silent half of the rule. A worse
|
||||
transcript is still a turn, so `stt.Pair` has no `TranscribeRemote`. The fallback is
|
||||
never spoken. CW2 turbo scores **10.4% WER in Russian against 27.5%** for the `ggml-small.bin`
|
||||
mavsttd loads, over 200 Golos clips
|
||||
(`docs/evals/2026-08-09-crisperwhisper2-russian-wer.md`). It runs in Intended mode, not
|
||||
Verbatim, though that corpus cannot separate the two.
|
||||
**whisper.cpp cannot load CW2 at all.** It reads its language count off the vocabulary
|
||||
size, and CW2's 51897 tokens shift seven special token ids. So it is not a second
|
||||
endpoint on mavgpud. It is its own transformers service on port 8081
|
||||
(`deploy/cw2/serve.py`), which Maven reaches directly. `stt.HTTPTranscriber`
|
||||
posts raw PCM to it with a bearer token, because audio is the most sensitive thing that
|
||||
crosses this seam. The switch is `workstation.stt` in
|
||||
`deploy/mavend.json`, and deleting the block sends every utterance to mavsttd.
|
||||
**mavgpud runs that service as a second child.** That is not an optimisation. CW2 is a
|
||||
ROCm process on the same card, so it registers on the KFD like any contender. Under its own
|
||||
systemd unit it made mavgpud evict llama-server every few seconds. That took the
|
||||
gemma-4-12b arm down for eight minutes on 2026-08-09 before anyone noticed. The card needs
|
||||
one owner. Any GPU service added beside this daemon has the same defect, so add it to
|
||||
`cmd/mavgpud` and not to systemd. CW2 is on the yield clock and not the idle one. At 1.6GB
|
||||
it denies the card to nobody, and unloading it would only send the next voice turn to the
|
||||
homesrv floor.
|
||||
Text-to-speech has not moved and piper on homesrv is still the only synthesizer.
|
||||
## Build and test
|
||||
|
||||
## Build & test
|
||||
|
||||
CGO daemons (`mavend`, `mavsttd`, `mavttsd`, `mavenclient`) need the vendored toolchain
|
||||
and libs wired through the Makefile — **do not** call `go build` on them bare, use `make`:
|
||||
CGO daemons (`mavend`, `mavsttd`, `mavttsd`, `mavenclient`) need the vendored
|
||||
toolchain wired through the Makefile. **Do not call `go build` on them bare**,
|
||||
and **do not hand-write the CGO preamble**. This box runs zsh, so an unquoted
|
||||
`-run Test*` dies on "no matches found" before `go` is reached. `make t` also
|
||||
carries `-count=1` and sets `MAVEN_ONNX_LIB`. Without that variable the four
|
||||
`TestONNX*` measurements self-skip and the run still prints `ok`.
|
||||
|
||||
```sh
|
||||
make build # all 11 binaries
|
||||
make build-web # single daemon (pure-Go ones: web/waked/poll/caldav build without CGO)
|
||||
make test # go test -race across ./internal/... ./cmd/... with CGO env set
|
||||
```
|
||||
|
||||
Run one package or one test with `make t`. **Do not hand-write the CGO preamble.**
|
||||
Past sessions pasted it about 390 times. That is where the shell-quoting failures
|
||||
came from. This box runs zsh, so an unquoted `-run Test*` or `--include=*.go`
|
||||
dies on "no matches found" before `go` is ever reached.
|
||||
|
||||
```sh
|
||||
make t PKG=./internal/router/
|
||||
make t PKG=./cmd/mavend/ RUN=TestSimulator
|
||||
make build # all 11 binaries. make build-web for one (web/waked/poll/caldav skip CGO)
|
||||
make test # go test -race across ./internal/... ./cmd/... with CGO env set
|
||||
make t PKG=./internal/router/eval/ RUN='TestONNX' V=1 # V=1 for -v, RACE=0 to drop -race
|
||||
make analyze # staticcheck, deadcode and govulncheck. Not in `test`: all three need the network
|
||||
```
|
||||
|
||||
`t` carries `-race`, so a green `make t` cannot turn red under `make test`. It carries
|
||||
`-count=1`, so a cached PASS from before your edit is never mistaken for a result.
|
||||
**The static gates pass against a baseline, not against zero**
|
||||
(`scripts/analyzers/*.baseline`, reasoning in `docs/workflow.md`). A fix must
|
||||
delete its baseline entry, because the gate also fails on an entry whose finding
|
||||
is gone. **`make audit` is a git-grep inventory, not analysis.** Do not cite it
|
||||
as a reachability check.
|
||||
|
||||
It also sets `MAVEN_ONNX_LIB`, which the hand-written recipe did not. The four
|
||||
`TestONNX*` measurements self-skip when that variable is unset. The run still prints
|
||||
`ok`. So every targeted eval done the old way reported the hash ratchet while reading
|
||||
as a real embedder score.
|
||||
## The daemons
|
||||
|
||||
Pure-Go packages (`router`, `memory`, `mavweb`, …) also run under a plain `go test ./pkg/`,
|
||||
but `make t` works everywhere and is one thing to remember.
|
||||
Eleven binaries under `cmd/`, wired socket-to-socket over `internal/ipc`, not
|
||||
linked. `mavend` is the core and owns the DB and the IPC socket.
|
||||
`deploy/mavend.json` sets sockets, model paths and the phraser and embedder
|
||||
blocks, with `${VAR}` expansion from gitignored `deploy/telegram.env`.
|
||||
**`docker-compose.yml` runs five**: `mavend`, `mavsttd`, `mavttsd`, `mavweb`,
|
||||
`mavpoll`. Count against compose, not against `make build`. `mavwaked` runs on
|
||||
workpc under systemd. `docs/deployment.md` says who else is absent and why.
|
||||
|
||||
## The daemons (`cmd/`)
|
||||
|
||||
| Binary | Role |
|
||||
|---|---|
|
||||
| `mavend` | **Core.** Router, phraser, memory, reminders, digestion tick. Owns the DB + IPC socket. |
|
||||
| `mavweb` | HTTP UI + PWA (`/dash`, `/history`, `/trace`, `/notifications`, `/tools`); WebAuthn auth. Connects to mavend's socket. |
|
||||
| `mavsttd` | Speech-to-text (whisper.cpp, CGO). |
|
||||
| `mavttsd` | Text-to-speech (piper subprocess). |
|
||||
| `mavwaked` | Wake-word / VAD gate. **Not on homesrv** — see below. |
|
||||
| `mavenclient` | Voice loop client (mic → stt → core → tts). **Not on homesrv** — see below. |
|
||||
| `mavpoll` | Environment poller: netdata alarms, uptime-kuma, zenmoney, wireguard presence. Writes facts, sends nothing. Telegram is `internal/delivery/telegramsink`, not this. |
|
||||
| `mavcaldav` | CalDAV calendar sync. |
|
||||
| `mavmaild` | Mail reader (IMAP, read-only). Holds the IMAP password; core never sees it. |
|
||||
| `mavgpud` | GPU supervisor. **Runs on workpc, not homesrv** — own unit, `deploy/mavgpud.service`. Keeps llama-server loaded while the card is free (V-488). Maven never asks it for anything, it reads `/health` through `llm.Pair`. |
|
||||
| `mavupdate` | Not a daemon. Operator CLI a human runs on the box to deploy a new build. |
|
||||
|
||||
Two more binaries have no Makefile target and are built with `go run` or `go build` when
|
||||
they are needed. Neither is deployed.
|
||||
|
||||
| Binary | Role |
|
||||
|---|---|
|
||||
| `mavseal` | Recovery tool. Encrypts a live tmpfs working copy back to the ciphertext file when mavend was killed before `defer st.Close()` sealed it. |
|
||||
| `labelgen` | Runs the stage 0 grammars over utterances and prints JSONL, the training data for the routing heads (V-546). |
|
||||
|
||||
Daemons are wired socket-to-socket, not linked. `internal/ipc` is the client/server wire
|
||||
protocol; the config in `deploy/mavend.json` (with `${VAR}` env expansion from gitignored
|
||||
`deploy/telegram.env`) sets socket paths, model paths, and the phraser/embedder blocks.
|
||||
|
||||
**`docker-compose.yml` runs five: `mavend`, `mavsttd`, `mavttsd`, `mavweb`, `mavpoll`.**
|
||||
Count against compose, not against the table. Four of the nine daemons are absent, and each
|
||||
absence has a different reason.
|
||||
|
||||
`mavmaild` and `mavcaldav` are commented out in compose, each with the reason written
|
||||
beside it: the first needs a mail account, the second a CalDAV account, and this box has
|
||||
neither. `mavcaldav` used to appear nowhere at all, which was an oversight; it became a
|
||||
recorded decision on 07-08-2026 (V-644). Two things ride on that absence and the block
|
||||
names them. Agenda questions route to `IntentQuery` at stage 0 (V-498) and the `calendar`
|
||||
query source then reads a table nobody writes. And `loop.State.CalendarBusy` is fed by the
|
||||
same facts, so the gate's "do not nag mid-meeting" is permanently false. Its password is
|
||||
read from a file (`-pass-file`, and `-render-pass-file` for the render collection), never
|
||||
taken as a flag value, which is the rule `mavpoll` and `mavmaild` follow too.
|
||||
|
||||
**`mavwaked` and `mavenclient` are absent by decision, not oversight** (Vikunja #463,
|
||||
`docs/plans/17-where-the-voice-loop-runs.md`).
|
||||
homesrv has a microphone — it is a laptop — but it is in the wrong room, so a wake-word
|
||||
daemon there listens to nobody. They belong on a client machine where the owner is standing.
|
||||
|
||||
**That machine is workpc** (owner's correction, 2026-08-05). This section used to say no
|
||||
such machine existed, which was written when the workstation was only a model host. It is
|
||||
where he sits most of the day and it has the microphone. `ipc.Dial` already takes
|
||||
`tcp://host:port?token=...` through the netaddr seam, so the two daemons need deploying,
|
||||
not building. V-515 is that deployment.
|
||||
|
||||
Until they are deployed, **the wake word and the VAD gate are covered by unit tests and by
|
||||
nothing else**, and push-to-talk through `/dash` is what QA actually covers. Note that
|
||||
deploying them does not by itself prove a wake word: `mavwaked` gates on energy and has no
|
||||
keyword model (V-487), so the loop runs open until that lands.
|
||||
- **Passwords are read from files, never taken as flag values.**
|
||||
- **The voice wire is plaintext with no auth.** mavend's voice port stays on
|
||||
homesrv loopback and reaches workpc over ssh. Do not LAN-bind it.
|
||||
`SurfaceVoice` caps acts at L0, and L0 does not cap reading.
|
||||
- **A GPU service added beside mavgpud goes in `cmd/mavgpud`, never in systemd.**
|
||||
The card needs one owner. A second unit made mavgpud evict llama-server every
|
||||
few seconds and took the model arm down for eight minutes.
|
||||
|
||||
## The ecosystem: Nexus, Praxis, Hexis
|
||||
|
||||
Maven is one of four services. It owns conversation and personal memory. It does not
|
||||
Nexus identifies, Praxis observes, Hexis acts, Maven understands. Maven does not
|
||||
own identity, operational state, or execution. Full contract in
|
||||
`docs/ecosystem.md`.
|
||||
`docs/ecosystem.md`. All three are `nil` unless configured and each degrades
|
||||
alone. An outage means a named gap, never a broken turn or a guess.
|
||||
|
||||
```text
|
||||
Nexus identifies. Praxis observes. Hexis acts. Maven understands and coordinates.
|
||||
```
|
||||
|
||||
| Service | Owns | Maven's client | Configured at |
|
||||
|---|---|---|---|
|
||||
| **Nexus** | Canonical entity ids, names, aliases, relationships. Projects, services, devices, people, pets, places. | `nexusClient` in `cmd/mavend/ecosystem.go`, `POST /api/v1/resolve` | `nexus.url` (`http://nexus:9740`) |
|
||||
| **Praxis** | Operational attention and item lifecycle. What needs looking at, what changed, what is still unresolved. | `praxisClient`, the HTTP tools API under `/api/v1/tools/` | `praxis.url` (`http://praxis:8989`) |
|
||||
| **Hexis** | The capability registry and the only path to executing anything. | vendored `github.com/kami/hexis/pkg/client` | `hexis.url` (`http://hexis:9741`) |
|
||||
|
||||
All three are `nil` unless configured, and every one of them degrades on its own.
|
||||
An outage means a named gap in the answer, never a broken turn and never a guess.
|
||||
|
||||
Rules that are not negotiable:
|
||||
|
||||
- **No component reads another component's database.** Praxis attention comes over
|
||||
HTTP, never from its SQLite file.
|
||||
- **Identity lives in Nexus.** Do not invent a local fact key for something Nexus
|
||||
resolves. `actionFact` already sets `Subject`, and `cmd/mavend/factenrichment.go`
|
||||
resolves it in the background against Nexus.
|
||||
- **Free text never reaches a mutating Hexis call.** Resolve to a canonical entity id
|
||||
first. Ambiguous resolution asks the owner, it does not pick.
|
||||
- **No component reads another component's database.** Praxis attention comes
|
||||
over HTTP, never from its SQLite file.
|
||||
- **Identity lives in Nexus.** Do not invent a local fact key for something
|
||||
Nexus resolves. `cmd/mavend/factenrichment.go` resolves `actionFact.Subject`.
|
||||
- **Free text never reaches a mutating Hexis call.** Resolve to a canonical
|
||||
entity id first. Ambiguous resolution asks the owner, it does not pick.
|
||||
- **LLM output is not authorization.** Confirmation binds capability id, target
|
||||
entity, arguments, requester and expiry. See `cmd/mavend/confirm.go`.
|
||||
- **Praxis lifecycle words mean different things.** Surfaced is not acknowledged,
|
||||
acknowledged is not resolved, execution success is not recovery. Reading an item
|
||||
aloud calls `Surface`, never `Acknowledge`.
|
||||
- **No automatic attention-to-action path.** Digestion may summarise Praxis. It may
|
||||
not call Hexis.
|
||||
entity, arguments, requester and expiry (`cmd/mavend/confirm.go`).
|
||||
- **Praxis lifecycle words differ.** Surfaced is not acknowledged, acknowledged
|
||||
is not resolved, execution success is not recovery. Reading an item aloud
|
||||
calls `Surface`, never `Acknowledge`.
|
||||
- **No automatic attention-to-action path.** Digestion may summarise Praxis and
|
||||
may not call Hexis.
|
||||
- Every cross-service call carries a correlation id minted once per action
|
||||
(`withCorrelationID`), a contract version header, and `X-Requested-By: maven`.
|
||||
|
||||
Every cross-service call carries a correlation id minted once per action
|
||||
(`withCorrelationID`), a contract version header, and `X-Requested-By: maven`.
|
||||
## Routing
|
||||
|
||||
## Routing — read this before touching the router
|
||||
**Read `docs/routing.md` before touching `internal/router/` or `queryWalk`.** It
|
||||
carries the reasoning, the measurements and every rule's why. A route produces
|
||||
two decisions. **Intent** is one of seven values. **Source** is where the answer
|
||||
lives and is read on `IntentQuery` alone. Score them separately. The cascade is
|
||||
stage 0 grammars, then the routing heads, then the resident model, then the
|
||||
classifier. Every stage may decline and the next one answers.
|
||||
|
||||
`internal/router/` has TWO layered engines. **The LLM router is now the default and it is
|
||||
on in deploy** — this section used to say it was wired `nil`, which stopped being true on
|
||||
2026-07-31.
|
||||
- **The classifier is the floor, not dead code.** It answers when the resident
|
||||
model is off, absent, or erroring. **Any model error falls through.**
|
||||
- **The stage 0 set lives in `router.StageZeroGrammars`**, and both `buildRouter`
|
||||
and the eval fixture call it. Add a grammar there, in the right place, and read
|
||||
the comment above the line you insert after. Do not restate the list anywhere.
|
||||
- **Go's `\b` is ASCII-only** and never fires after a Cyrillic letter. A Russian
|
||||
pattern needs an explicit `(\s|[?!.]|$)`.
|
||||
- **`PraxisGrammars()` is the only path to Praxis**, not a faster one.
|
||||
- **`voice.embedder.heads_path` must never point at `model_path`.** Recall
|
||||
depends on the resident e5-small scoring what it scored. Fine-tune a copy.
|
||||
Refused at config load since V-692, symlinks included.
|
||||
- **Routing traces are retained 14 days**, enforced on write and again on start.
|
||||
- **Bump `tokenizerRev` on any change to what `encodeWord` emits**, so a
|
||||
tokenizer fix triggers `ReembedAll` the way swapping the model file does.
|
||||
- **A new rung in the `runTurn` ladder needs its name in `preRouteLadder`**
|
||||
(`cmd/mavend/decisiontrace.go`), or it is missing from the decision record.
|
||||
|
||||
- **LLM router (the intended design, docs/rearchitecture.md):** the resident Qwen3-1.7B (`llmrouter.go`)
|
||||
emits GBNF-constrained structured JSON, and the SAME model phrases replies. Embedder is
|
||||
demoted from a routing gate to a RAG hint. Wired at `voice.go:214` via
|
||||
`pickLLMRouter(cfg.Voice.UseLLMRouter(), llmClient)`; the flag is `voice.llm_router`
|
||||
(`config.go`), `DefaultLLMRouter` is **on**, and `deploy/mavend.json` sets it `true`.
|
||||
- **Classifier cascade (the failure floor, not dead code):** `classifier.go` +
|
||||
`embedder.go` nearest-neighbour over frozen seed phrases. It runs when the LLM router is
|
||||
off, when there is no llama-server to talk to (`pickLLMRouter` logs that and degrades),
|
||||
and on any per-turn LLM error. Do not delete it — routing by seed similarity is the known
|
||||
cause of weak RU query handling, but a turn must never break on the model.
|
||||
**`queryWalk` takes query sources out and moves none** (`actions_query.go`). That
|
||||
is the safety argument and it is not negotiable. The table's order is
|
||||
load-bearing and carries "the owner's data first, then the world".
|
||||
`SourceUnknown` is the floor and walks the whole chain. A named destination
|
||||
removes only the sources marked `guesses: true`, so a source that looks rather
|
||||
than guesses is always asked. **The personal boundary is the one exception and
|
||||
it is deliberate.** It guesses, so naming `SourceWorld` drops it. **Only a stage
|
||||
0 grammar may drop it** (owner's call, V-666). `queryWalk` reads
|
||||
`Decision.SourceAnchored` for the source marked `boundary: true` and no other.
|
||||
|
||||
Cascade order: `stage0.go` exact-match fast-path → LLM router (when non-nil) → classifier
|
||||
fallback. Any LLM error falls through to the classifier so a turn never breaks on the model.
|
||||
Judge a routing change against the classifier (76.0% intent, 36.4% destination)
|
||||
and the resident model (80.2% intent), since those always answer. The fixture
|
||||
has grown from 77 cases to 96, so a number compares only to another number on
|
||||
the same fixture.
|
||||
|
||||
**A stage-0 decision is slot-extracted too, since 06-08-2026** (V-572). `fillMatchedSlots`
|
||||
in `router.go` runs the stage-2 extractor over whatever a grammar built and fills only the
|
||||
slots it left empty — a matched value always wins, because the rule read a literal pattern
|
||||
and the extractor guesses. It did not run before, so `ReminderGrammar` handed the daemon
|
||||
`HasTime: false` for "напомни в 11:00 позвонить маме" and `missingFor` read the silence as
|
||||
absence and asked "Когда?". It applies to every grammar and is inert for all but the
|
||||
reminder: `Extract` fills Time, Fn and Key and nothing else, and the query, clock, agenda,
|
||||
feed, list, task and narrative rules all emit intents with no such slot. Benchmarked at
|
||||
20000x, a stage-0 query costs 3.7µs against 3.9µs before. **`Slots.Text` is deliberately not
|
||||
filled** — a grammar that left it empty meant it, and `agendaQueryBuild` hands the query
|
||||
chain the utterance itself. Fixture unchanged at 64/91, with "slots deferred to daemon"
|
||||
6 → 0.
|
||||
## Language: model output and Russian
|
||||
|
||||
Measured on the 77-case RU fixture. **Re-measured 2026-08-02: the classifier scores 68.8%
|
||||
full accuracy at p50 16.6µs**, not the 36.8% at p50 31ms that stood here from
|
||||
`docs/evals/2026-07-31-model-bakeoff.md`. That older figure predates the stage 0 rules and the
|
||||
seed additions, both of which now score inside the classifier baseline. Qwen3-1.7B scores
|
||||
77.9% intent-only / 72.7% through the cascade. So the router buys about 4 points of accuracy,
|
||||
not a doubling, and the trade is worth re-arguing rather than assuming. **The ≈2.7s figure
|
||||
that stood here until 2026-08-02 was contention, not the model.** See `docs/evals/2026-07-31-routing.md` line 61, which measures the LLM router at
|
||||
p50 825ms / p95 1.2s / max 3.0s and the full cascade at p50 0.80-1.04s. Do not plan latency
|
||||
work off the bakeoff table.
|
||||
Both contracts are in `docs/language.md`. What must not be broken:
|
||||
|
||||
**Re-measured 2026-08-05 on the fixture as it now stands, 91 cases** (V-320 item 2,
|
||||
`docs/evals/2026-08-05-routing-resident-model.md`): cascade + resident model scores
|
||||
**75.8% full / 80.2% intent-only at p50 1.19s / p95 1.65s**. That is a new baseline and not
|
||||
a movement, because 14 cases were added since the 77-case number above. The model alone
|
||||
scores 37.4% full against 61.5% intent-only, and the gap is slots rather than routing: it
|
||||
routes `reminder` and leaves the time to the daemon, which is what the contract asks. To
|
||||
re-run it, start a **second** llama-server on a fixed host port — the resident one binds
|
||||
`--port 0` inside the container and no host process can reach it.
|
||||
- **One parser for model text, `parseResponseMood`** in
|
||||
`internal/phraser/parse.go`. Every phrasing path reaches it. Mood is an enum.
|
||||
- **The router prompt is a separate contract** over 7 intents, and
|
||||
`llm/check_prompt_parity.py` keeps the Go and relabelling copies identical.
|
||||
- **Russian words are matched by three mechanisms and no fourth**:
|
||||
`internal/lexicon` for closed classes, `internal/morph` for grammar, and
|
||||
`cmd/mavend/topics.go` with the embedder for open sets. A regex whose output
|
||||
is a fact or a route is the defect. A regex over structured input is not.
|
||||
- **Seeds are scoring data.** Editing one moves a recogniser and must be
|
||||
re-measured against the `TestONNX*` tests, not eyeballed.
|
||||
|
||||
**The numbers above are the homesrv floor, not the ceiling.** With the workstation up, routing
|
||||
completes through `llm.Pair` against the model mavgpud holds, which is better than the resident
|
||||
model and about 2.5× faster. gemma-4-12b scored **84.4% full / 93.5% intent-only at p50 329ms**
|
||||
(`docs/evals/2026-08-02-workstation-gemma4-12b.md`, Vikunja #485). The workstation is never
|
||||
assumed up, so both sets of numbers are live. Judge a
|
||||
routing change against the classifier and the resident model, since those are what always answer.
|
||||
## Non-goals and hard constraints
|
||||
|
||||
**The workstation runs gemma-4-E4B since 2026-08-09** (owner's call), and it is a
|
||||
step down measured the same day (`docs/evals/2026-08-09-e4b-vs-12b-routing.md`).
|
||||
Against a same-session 12B control it scores **83.3% full / 89.6% intent-only,
|
||||
destination 19/33 against 23/33, at p50 294ms against 344ms**. So it costs four
|
||||
destination cases and buys 50ms. Read destination as the finding: it names nothing
|
||||
where the 12B names `recall` or `calendar`, which is safe but walks the whole chain.
|
||||
It also has no MTP and cannot be given any here. The only `gemma4-assistant`
|
||||
draft on disk is trained against the 12B's hidden states.
|
||||
**Phrasing was the unmeasured half and it is measured now**
|
||||
(`docs/evals/2026-08-09-e4b-phrasing.md`). E4B scores nudges 15/15 and the
|
||||
36-case talk fixture **29/36 at p50 516ms**, against the resident model's 25/36
|
||||
at p50 2.97s. Persona is clean: `lang`, `feminine` and `address` are all 36/36,
|
||||
where the resident model loses three on `address`. Every failure is `ontopic`
|
||||
and none is a parse error. The 2026-08-05 temperature sweep put this fixture's
|
||||
ceiling at 30/36, because two reply cases fail at every temperature (V-537), and
|
||||
both are in E4B's failure list. So the swap costs nothing here. One defect no
|
||||
check catches: in chat E4B claims "Я записала несколько идей!" when nothing was
|
||||
stored, which is a wrong claim about state.
|
||||
Not a nag, not autonomous.
|
||||
**The persona is feminine.** Russian self-reference takes feminine forms: `рада`
|
||||
not `рад`, `поняла` not `понял`. The owner is male and she speaks to him
|
||||
informally. Use "ты", singular, never "вы" or "ваш", and never "он" or "его".
|
||||
She talks TO the owner, not about him. Pet names such as "милый" are forbidden.
|
||||
The name "Ками" is not. `CheckAddress`, `CheckFeminine` and `CheckCringe` in
|
||||
`internal/phraser/eval/checks.go` enforce this, scored by `make eval-phrasing`.
|
||||
|
||||
**The intended third engine is not a generative model** (owner's call, 05-08-2026, V-546,
|
||||
`docs/plans/18-routing-heads-on-e5-small.md`). Routing has a bounded output space, so it is
|
||||
classification, and the 118M multilingual-e5-small is already resident. Three heads on one
|
||||
forward pass: intent, mood, and BIO slot tags. Roughly 5e15 FLOPs to train, so 10 to 30
|
||||
minutes on the workstation. A 100M decoder from scratch is 10 to 20 GPU hours. Two things
|
||||
it buys that a decoder cannot. No grammar is needed, because a softmax cannot emit a value
|
||||
that does not exist. And max softmax is a calibratable confidence, where `Confidence: 1.0`
|
||||
was a hardcode. **Fine-tune a copy of the weights.** The resident embedder backs memory
|
||||
recall. Training it in place couples routing accuracy to recall@1, with nothing in the
|
||||
suite to name the trade.
|
||||
**"Never phones home" is deprecated** (owner's call, 2026-07-31). She reads
|
||||
external sources, and `docs/world.md` carries that chain. What holds regardless:
|
||||
|
||||
**Two of those heads are trained as of 08-08-2026, and they are not the three
|
||||
above** (V-661, `docs/evals/2026-08-08-routing-heads-two-head.md`). Intent and
|
||||
destination share one masked mean pool. Destination scores a mean **80.8%** over
|
||||
three seeds, best **29/33 (87.9%)**. The classifier cascade scores 12/33 and the
|
||||
cascade with gemma-4-12b scores 24/33, so a 118M encoder beats the 12B teacher it
|
||||
was distilled from. Read the best run as one seed and not a headline, because one
|
||||
case is 3 points on a fixture this small.
|
||||
Recall is 15/15 and world is 5/5. Intent is 93.6% mean over three seeds. That is
|
||||
**not** comparable to the 76.0% and 84.4% those two arms scored: a softmax has no
|
||||
clarify class, so the head's fixture is the 88 cases carrying an intent.
|
||||
|
||||
**A fourth head asks instead of guessing, same day** (V-661,
|
||||
`docs/evals/2026-08-08-clarify-head-four-head.md`). Clarify is not a value
|
||||
of intent, so a softmax cannot emit it. It is a second question over the
|
||||
same pooled vector: can Maven act on this at all. That is why the head's
|
||||
fixture was 88 cases and not 96. Over three seeds it catches **7.0 of the 8
|
||||
`want_clarify` cases and produces 2.3 false clarifies of 88**. The cascade
|
||||
today misses 1 and produces 2, so this is parity with no rules in front of
|
||||
it. Accuracy is the wrong number here and a head that never asks scores
|
||||
91.7%. Confidence is the other half. Max softmax over the intent head reads
|
||||
**0.851 where it is right against 0.604 where it is wrong**, ranking right
|
||||
above wrong in 83.4% of pairs. `Confidence: 1.0` was a hardcode, and this
|
||||
replaces it with a signal. The two are not the same signal: one says which
|
||||
intent is unclear, the other says the utterance carries too little to act
|
||||
on. **The fourth head is not free the way the third was.** Intent,
|
||||
destination and slot F1 each move down one to four points, inside the seed
|
||||
spread. `поужинал` is a false clarify on every seed, which is the same
|
||||
defect `thinSingleToken` was narrowed for on 2026-08-01.
|
||||
|
||||
The corpus for it is generated, because every existing row is answerable by
|
||||
construction. **The router-prompt agreement filter cannot work here**, since
|
||||
`routeGrammar` has no clarify value and a generated line always agrees with
|
||||
itself. A gemma judge replaces it. The first judge called 24 of 40
|
||||
answerable rows underspecified, because it judged against a generic
|
||||
assistant rather than against Maven's contract.
|
||||
|
||||
**Mood is cut, not deferred.** The enum describes her own reply state, not the
|
||||
speaker's emotion, and no dataset maps onto it.
|
||||
|
||||
**A third head landed the same day** (`docs/evals/2026-08-08-slot-head-three-head.md`).
|
||||
BIO slot tags had no Maven-domain corpus, which was true of found corpora and
|
||||
false of made ones. `label_slots.py` distils spans out of gemma-4-12b under a
|
||||
GBNF closed over Maven's own five slots. A span survives only when it is a
|
||||
literal substring of the utterance, so the agreement filter costs no second
|
||||
call. 2178 spans over 1702 rows, 37 dropped, nothing unparsed. Three heads score
|
||||
intent **92.8%**, destination **82.8%** and slot span F1 **72.4%** over three
|
||||
seeds. The slot head is free: both other numbers move less than their own seed
|
||||
spread. Epoch selection reads the intent dev slice alone. Slot F1 is still
|
||||
climbing when it stops, which costs about 4 points.
|
||||
|
||||
The MASSIVE warm-start of step 2 is worth nothing here. Stock e5-small ties it on
|
||||
intent and leads by a third of a case on destination. Nothing argues for keeping
|
||||
that step.
|
||||
|
||||
The floor was a corpus defect and it is fixed. The first 120 floor rows carried
|
||||
one sentence shape, so the head named a destination where the fixture says walk
|
||||
the chain. Rotating six shapes took the floor 3/7 to 6/7 and destination 75.8% to
|
||||
80.8%. What is left is calendar at 3/6 on every seed, which training cannot move:
|
||||
the possessive agenda rules claim those cases at stage 0 and name nothing, so no
|
||||
label reaches the head. That is the same trade V-660 flagged and it wants the
|
||||
owner's call.
|
||||
|
||||
**The heads run in Go and route every turn, since 08-08-2026** (V-664,
|
||||
`docs/evals/2026-08-08-routing-heads-in-go.md`). This section used to say
|
||||
nothing of it ran. `RouterHeads` in `internal/router/heads.go` loads
|
||||
`router_heads.onnx` and reads intent, destination and clarify off one forward
|
||||
pass. It is stage 0b: after the grammars, **before** the resident model, and the
|
||||
classifier is still behind both. Through the cascade it scores intent **96.9%**
|
||||
and destination **75.8%** at p50 27.9ms. That beats the gemma-4-12b cascade,
|
||||
84.4% and 72.7%, at a twelfth of its 329ms. The workstation stays the better
|
||||
phraser and is no longer the better router.
|
||||
|
||||
Three rules around it. The **clarify head decides first**, before the intent
|
||||
threshold. It answers a different question. A thin utterance scores low
|
||||
intent by construction, so gating it cost 6 of 8 ambiguous cases. The
|
||||
**destination head is read on `IntentQuery` only**, since no other intent
|
||||
reaches `queryWalk`. And `headsThreshold` is 0.6, the measured knee: every value
|
||||
to 0.85 drops right answers and keeps the same two wrong ones.
|
||||
|
||||
`voice.embedder.heads_path` is the whole switch. Empty, missing or unloadable
|
||||
means the heads are nil and the cascade is byte-for-byte what shipped before
|
||||
them. **It must never be pointed at `model_path`.** The resident e5-small must
|
||||
not be replaced by the fine-tuned copy. Recall depends on that file scoring
|
||||
what it scored.
|
||||
|
||||
**The hand-written tokenizer read every long word backwards** until this task
|
||||
(`encodeWord`, `onnxembedder.go`). It cost recall@1 7.4 points and recall@3 11.1.
|
||||
Nothing caught it because seeds and queries were mangled the same way, so cosine
|
||||
survived. The heads found it. They are trained through transformers and read
|
||||
through this. The embedder id now carries a tokenizer revision
|
||||
(`@384/tok2`), so fixing the tokenizer triggers `ReembedAll` the way swapping the
|
||||
model file does. Bump `tokenizerRev` on any change to what it emits.
|
||||
|
||||
`Confidence: 1.0` used to be hardcoded in `llmrouter.go`, so the LLM
|
||||
path could never ask for clarification (6/6 refusal cases missed on the fixture) — Vikunja
|
||||
#359. Fixed 31-07-2026 with structural signal (single-token utterance, keyless fact, act with
|
||||
no allowlisted fn) feeding the same stage-3 gate the classifier path already had — see
|
||||
`gateLLMDecision` in `router.go`. Note the second half of that bug: the LLM branch never
|
||||
consulted `r.threshold` at all, so a correct low confidence would have been discarded anyway.
|
||||
|
||||
Re-measured on the fixture after the fix: **missed clarify 6/6 → 1**, at the cost of 3 false
|
||||
clarifies and 2.6pt of full accuracy (72.7% → 70.1%, intent-only 67.5% → 74.0%). Two of the
|
||||
three false clarifies are acts the model mis-routed and the gate caught — asking beats wrongly
|
||||
executing, so the fixture and the daemon disagree about what is correct there. The third,
|
||||
`"поужинал"`, was a real defect: the single-token rule was an English intuition and does not
|
||||
transfer to Russian, where one word is routinely a whole sentence.
|
||||
|
||||
Narrowed 01-08-2026. `thinSingleToken` (`internal/router/singletoken.go`) still thins a bare
|
||||
one-word nominal — "вода", "бэкап" — but spares two classes: a closed lexicon of social and
|
||||
control singles ("привет", "спасибо", "стоп", "yes"), and any token carrying a Russian verb
|
||||
ending (past tense, 2nd person, reflexive), because a verb already contains its subject. Both
|
||||
tests are offline and cost nothing. Re-measured: **false clarifies 3 → 2, intent-only 74.0% →
|
||||
75.3%, full accuracy unchanged at 70.1%, missed clarify still 1.** The two remaining false
|
||||
clarifies are the act-with-no-allowlisted-fn arm of the gate, not this rule.
|
||||
|
||||
Agenda questions taken off the model, 01-08-2026. `AgendaQueryGrammars` (`stage0.go`, wired
|
||||
after the clock rules in `buildRouter`) routes "что у меня сегодня", "во сколько у меня
|
||||
встреча" and anything naming a calendar to `IntentQuery` at stage 0. They were going to
|
||||
`IntentSystem`, where `replySystem` has no agenda arm and answered "пока не умею" — the
|
||||
fixture had said `query` since ru-query-019 was written. Measured: **full accuracy 70.1% →
|
||||
72.7%, intent-only 75.3% → 77.9%, calendar 0/2 → 2/2**, clarify counts unchanged. Note that
|
||||
Go's `\b` is ASCII-only and never fires after a Cyrillic letter; the pattern needs an
|
||||
explicit `(\s|[?!.]|$)`.
|
||||
|
||||
Two more shapes taken off the model, 04-08-2026 (V-498). `rest-of-day-query` inside
|
||||
`AgendaQueryGrammars` claims "что дальше?" / "what's next", and `NarrativeQueryGrammar`
|
||||
(`stage0.go`, wired **last** in `buildRouter`, after the capture marker) claims "расскажи про
|
||||
X", "объясни X", "опиши X". Neither carries a question mark or an interrogative, so the model
|
||||
called both `IntentFact`; the write was caught downstream by `IsQuestionShaped`, so this was a
|
||||
latency and fixture defect, not a correctness one. The narrative rule reads the same
|
||||
`narrativeRequests` lexicon `IsQuestionShaped` reads, and declines `chatNarrativeTopics` — a
|
||||
joke, a bedtime story, herself — because the query chain has no source that answers those.
|
||||
New fixture cases ru-query-024 and ru-query-025. Classifier + ONNX baseline **56/80 (70.0%) →
|
||||
58/82 (70.7%)**, no case regressed, no new false clarify. The LLM arm was not measured (no
|
||||
llama-server in that run), so judge it again before quoting a cascade number.
|
||||
|
||||
Praxis taken off the model, 05-08-2026 (V-516). `PraxisGrammars()`
|
||||
(`internal/router/praxis.go`, wired in `buildRouter` before the capture marker because
|
||||
"отметь" is a capture verb) fills `Slots.Fn` with a Praxis capability name.
|
||||
**These grammars are the only path to Praxis, not a faster one.** Measured
|
||||
2026-08-05 with the resident model as router (V-517,
|
||||
`docs/evals/2026-08-05-reach-llm-router.md`): the model alone reaches Praxis
|
||||
**0/12**, the same as the classifier alone, because nothing in the router
|
||||
prompt names a Praxis capability and there is no string for it to write.
|
||||
Through the cascade it is 11/12. Deleting these rules costs every point. Praxis reach
|
||||
was **0/12 and structurally so**: `handlePraxisAct` compares `Slots.Fn` to a capability
|
||||
alias, and that slot is filled from the deployment's enabled tool names, which no Praxis
|
||||
alias is on. Measured **16/30 → 27/30 overall, praxis 0/12 → 11/12, lifecycle 0/5 → 5/5**
|
||||
(`docs/evals/2026-08-05-praxis-reach.md`). Two rules to know before editing: a **stative**
|
||||
lifecycle word ("готово", "принято") needs an item named beside it, while a bare
|
||||
**imperative** ("закрывай") may ask which one. The bare arm additionally requires that
|
||||
the sentence name no object of its own, or "закрой шторы в комнате" goes to Praxis instead
|
||||
of the house. A demonstrative ("отметь это как сделанное") resolves against
|
||||
`h.surfacedItems` only when exactly one item was spoken. Otherwise the turn goes back to
|
||||
the cascade rather than transitioning the wrong item.
|
||||
|
||||
**Who claimed a turn is now recorded, and so is who did not** (V-564, umbrella
|
||||
V-558). Arbitration between the claimants on the utterance stream is order,
|
||||
hardcoded in the pre-route resolver ladder, in `buildRouter` and in
|
||||
`querySources`. `internal/decision` records one `Record` per turn: every
|
||||
claimant, what it would have made the turn, the score it reported, and whether
|
||||
it won, declined, lost on score, was thinned by a gate or was **never asked**.
|
||||
The record rides the context, the same seam `querysource.go` uses, so a claim
|
||||
site cannot change a route and a context with no record costs nothing. It is
|
||||
installed in `runTurn`, so the mic, telegram and the web all leave the same
|
||||
trail. Storage is a 25-turn in-memory ring on the handler (`decision.Ring`),
|
||||
read over `ipc.TurnDecisions` and rendered as the second table on `/trace`.
|
||||
**It also persists, since 06-08-2026, and that reverses what this section used to
|
||||
say** (V-629, `docs/plans/21-persisting-the-routing-trace.md`). The old rule was
|
||||
that nothing persists, because a turn record is read minutes later or never. The
|
||||
owner reversed it: the routing heads (V-546) cannot be fitted or calibrated
|
||||
without real utterances, and 9 of the 31 modes in `internal/modes` have no seed
|
||||
example at all. The ring did not move. It is still what `/trace` reads and still
|
||||
what a test with no store gets. `cmd/mavend/routingtrace.go` is a second sink
|
||||
beside it, writing `routing_traces` (migration #23). The utterance is stored in
|
||||
clear, because a 384-dimension vector of a short sentence is substantially
|
||||
recoverable and storing vectors instead would be a privacy claim we cannot
|
||||
support. What makes it safe is the same thing that makes the fact store safe.
|
||||
Retention is 14 days, enforced on write and again on start, so a box that goes
|
||||
quiet does not keep every row. Nothing reads it outward, and the rule
|
||||
that his notes and facts are never search input covers this table. `Store.Wipe`
|
||||
deletes it with everything else. A correction (V-630) is promoted out into a
|
||||
seed-shaped row in `routing_labels` (migration #24) and kept, because a label is
|
||||
not a transcript. The transcript still expires. The gesture that writes one is
|
||||
two buttons beside the reply on `/chat`, reached over `ipc.CorrectTurn` and the
|
||||
trace id that now rides back on `ipc.ChatReply`. A turn marked wrong with no
|
||||
target is a usable negative, so naming the intent is never required. The target
|
||||
is one of the seven intents and never free text. **All three reaches offer it as
|
||||
of 06-08-2026**, and this section used to say only `/chat` did. Voice is the
|
||||
`repair` rung, which has read spoken corrections since V-455 and now writes the
|
||||
durable label beside the classifier seed it always wrote; a spoken negative with
|
||||
no target is its own rung, `repair-negative` (V-636, `docs/plans/22-correcting-a-turn.md`).
|
||||
Telegram is an inline keyboard under the reply, and it needed the chat to become
|
||||
readable first — **telegram is no longer outbound only** (V-637,
|
||||
`docs/plans/23-inbound-telegram.md`). The poller is dark unless the `telegram`
|
||||
block says `intake`, it long-polls because the box takes no inbound connections,
|
||||
it accepts `chat_id` and no other sender, and it drops whatever queued while the
|
||||
daemon was down. It reaches the daemon through `ipc.CoreAPI` alone, so a chat
|
||||
turn takes the path `POST /api/chat` takes. Note that the turn source is still
|
||||
`tap:text` for both, so provenance cannot tell a chat turn from a typed one.
|
||||
Adding a rung to the ladder
|
||||
in `runTurn` means adding its name to `preRouteLadder` in
|
||||
`cmd/mavend/decisiontrace.go`, or that rung is silently missing from the record.
|
||||
|
||||
**A route now says where the answer lives, not only that the turn is a question**
|
||||
(V-655, 07-08-2026). `query` was a shrug. The cascade sorted an utterance into one of
|
||||
seven intents, with stage 0, the resident model and the classifier behind it. Then
|
||||
`IntentQuery` handed the turn to `querySources` in the daemon. That is twenty-two branches
|
||||
deciding by seed similarity in a fixed order. It has no fixture and no accuracy
|
||||
number, no model arm and no floor. `Decision.Source` (`internal/router/source.go`) is
|
||||
the second half of the route. Twelve destinations, not twenty-two. The three recall
|
||||
passes plus `fact-by-key` are one destination from outside. So are search, Kiwix and
|
||||
the URL reader.
|
||||
|
||||
**`SourceUnknown` is a real value and it is the floor.** Nothing named a destination,
|
||||
so the daemon walks the whole chain. That is byte-for-byte what shipped before the
|
||||
field existed. The classifier arm names nothing, so a box whose model is down routes
|
||||
queries exactly as it did.
|
||||
|
||||
`queryWalk` in `cmd/mavend/actions_query.go` takes sources **out** and moves none.
|
||||
That is the safety argument and it is not negotiable. The table's order is
|
||||
load-bearing. Every comment on it argues a reason between two sources, and above all
|
||||
it carries "the owner's data first, then the world". Naming `SourceWorld` does not
|
||||
send the turn outside on its own. His notes and his facts still run first, because
|
||||
they look rather than guess.
|
||||
|
||||
**The personal boundary is the one exception and it is deliberate.** It guesses,
|
||||
so naming `SourceWorld` drops it. That is what stops it answering "кто такой
|
||||
Линус Торвальдс?" with "не нашла у тебя такой записи", which it did on
|
||||
2026-08-07. `TestNamingRecallKeepsTheBoundary` pins the other half: naming
|
||||
`SourceRecall` keeps the boundary in front of the world.
|
||||
|
||||
**Only a stage 0 grammar may drop it** (owner's call, 09-08-2026, V-666). The
|
||||
question of who is allowed to was open until then. Three deciders name a
|
||||
destination and two of them infer it: the routing heads and the resident model.
|
||||
An inferred `SourceWorld` on a question about him would reach SearXNG, and that
|
||||
widens what is asked rather than costing a local answer. So `Decision.SourceAnchored`
|
||||
carries the provenance. It is a field and not `Stage == 0`. Stage 0 also means
|
||||
confidence 1.0 and an anchored claim band, and one of those could stop implying
|
||||
the others. `queryWalk` reads it for the source marked `boundary: true` and for
|
||||
no other. So every other guesser still comes off the turn, whoever named the
|
||||
destination. `TestOnlyAGrammarMayDropTheBoundary` pins both directions.
|
||||
`definitionQueryPattern` claims "кто такой X", so the 2026-08-07 case is still
|
||||
anchored and still answered.
|
||||
|
||||
What comes out is only the sources that **guess**. Those decide a turn is theirs by
|
||||
cosine against frozen seeds, then answer whatever they claimed. They hold no table
|
||||
that could come back empty. Weather is the pure case and has no local data at
|
||||
all. It was measured on the box on 2026-08-07
|
||||
(`docs/evals/2026-08-07-week-of-usage.md` section 4). It answered both "что такое
|
||||
TCP?" and "сколько будет 17 на 23?" with "для какого города?". The feed answered "какой у меня любимый язык?" with kernel headlines.
|
||||
The personal boundary answered "кто такой Линус Торвальдс?" with "не нашла у тебя
|
||||
такой записи". A source that guesses is marked `guesses: true` in the table. One that
|
||||
looks is not, and it is always asked.
|
||||
|
||||
Stage 0 fills the destination where a rule already knows it. `WorldQueryGrammars()`
|
||||
(`internal/router/worldquery.go`) claims "что такое X" and "сколько будет 17 на 23".
|
||||
It is wired after the agenda rules and **before** the feed and list rules.
|
||||
"что такое лента" is a definition question, and the feed rule would take it on the
|
||||
noun alone.
|
||||
`calendar-query` and `event-time-query` name the calendar. The possessive agenda rules
|
||||
deliberately do not. "что у меня в списке покупок" matches `agenda-query`, and naming
|
||||
the calendar there would take the list source off the turn.
|
||||
|
||||
Fixture unchanged at **69/91 classifier+ONNX**, measured both sides. That is the
|
||||
expected result, because it scores intent and no case here changes intent.
|
||||
|
||||
**The destination has its own fixture and its own number as of 08-08-2026**
|
||||
(V-659, `docs/evals/2026-08-08-destination-fixture.md`). This section used to say
|
||||
it had neither. `want_source` on `eval.Case` is a pointer, because the destination
|
||||
has three states and a bare string has two. Absent is every intent but query,
|
||||
which never reaches `queryWalk`. Present and empty is the `SourceUnknown`
|
||||
contract: name nothing and walk the chain. Present and named is a destination the
|
||||
route must produce. Thirty-three of ninety-six cases carry one.
|
||||
|
||||
A destination miss does **not** fail the case. It lands in `Outcome.SourceReason`
|
||||
and never in `Reasons`, so `Accuracy` and `IntentAccuracy` mean what they meant
|
||||
and `SourceAccuracy` is a second number over the labelled cases only. Intent and
|
||||
destination are two decisions, and one number hides which one moved. A route that
|
||||
lost its intent scores no destination hit, or a clarify would satisfy an empty
|
||||
label for free.
|
||||
|
||||
Measured classifier+ONNX: intent **73/96 (76.0%)**, destination **12/33 (36.4%)**.
|
||||
The split is the finding. World is 5/5, because a stage 0 rule names it. The
|
||||
`SourceUnknown` floor is 5/7. Calendar is 2/6, because the possessive agenda
|
||||
rules deliberately do not name it. And **recall is 0/15, because nothing
|
||||
anywhere names it**. Those turns are still answered, since the chain walks
|
||||
recall early. Recall is the number the fourth head has to move.
|
||||
|
||||
Seven cases assert the floor and five of them are homelab operations. They
|
||||
cluster because `SourceRecall`, `SourceNetwork` and `SourceAttention` overlap on
|
||||
every question about the box. The other two are `ru-query-005` and
|
||||
`ru-query-014`. No query source reads the reminder store, and a deadline could
|
||||
sit in tasks, the calendar or Praxis. `mavpoll` writes its netdata and uptime-kuma
|
||||
observations into the fact store recall reads. That is a finding about the enum,
|
||||
not a gap in the labelling. The owner confirmed all seven floor labels on
|
||||
08-08-2026, so they are a decision rather than an agent's guess.
|
||||
|
||||
`baselineGrammars` in `eval_test.go` mirrors `buildRouter` and had drifted:
|
||||
`WorldQueryGrammars` was wired into the daemon by V-655 and not into the mirror,
|
||||
so the fixture scored a grammar set nobody runs. Fixed by V-659, worth 3 points of
|
||||
destination and nothing else. Check that function when adding a grammar.
|
||||
|
||||
**The model arm landed the same day** (V-660,
|
||||
`docs/evals/2026-08-08-destination-model-arm.md`). `routeGrammar` carries a
|
||||
`source` rule closed over `router.Sources` plus the empty floor, so the model
|
||||
cannot emit a destination that does not exist. The prompt lists the twelve in
|
||||
Russian and says `""` is a normal answer to give often. `LLMRouter.Route` reads it
|
||||
back through `ValidSource` and on `IntentQuery` alone. Against gemma-4-12b on the
|
||||
workstation the cascade scores destination **24/33 (72.7%)** with intent unmoved
|
||||
at 84.4%, and **recall goes 0/15 to 14/15**. The resident Qwen3-1.7B is
|
||||
unmeasured, because it binds `--port 0` inside the container.
|
||||
|
||||
**Stage 0 now costs four destination points.** It did not before. The four cases
|
||||
the cascade loses and the model alone wins are all calendar. The possessive
|
||||
agenda rules claim them first and name nothing on purpose. That caution was free
|
||||
while nothing downstream could name anything either. It is not free now, and the
|
||||
fix is the owner's call rather than a quiet edit.
|
||||
|
||||
The last arm is V-546. Intent, mood and BIO slot tags were already three heads on
|
||||
one forward pass of the resident e5-small. Destination is a fourth head on the
|
||||
same pass, and 72.7% from a 12B teacher is the label source for training it.
|
||||
|
||||
## LLM output contract
|
||||
|
||||
All phrasing paths emit `{"response":"...","mood":"..."}`, with fallback to plain text when
|
||||
the model skips the JSON. **One parser, `parseResponseMood` in
|
||||
`internal/phraser/parse.go`**, and every path reaches it: the six `LLMPhraser` methods,
|
||||
`PhraseWorld`, and `Replier.PhraseReply`, which `cmd/mavend/replier_llm.go` wraps — that file
|
||||
holds the stub fallback and no parsing of its own. The legacy `{"body","summary"}` fallback
|
||||
was deleted on 2026-08-06 (V-397): it was the contract before `{"response","mood"}` replaced
|
||||
it, no prompt asks for that shape, the GBNF cannot emit it, and no test covered it.
|
||||
Mood is a fixed enum. Router prompt is a separate contract:
|
||||
`[{"intent":<enum>, key?, value?, text?, verb?}, ...]`, 7 intents (`fact, reminder,
|
||||
note, query, act, chat, system`). `llm/check_prompt_parity.py` in the training
|
||||
workspace enforces that the Go and relabelling prompts remain identical.
|
||||
|
||||
## Russian patterns — three mechanisms, no fourth
|
||||
|
||||
Hand-written Russian stem patterns were swept out on 2026-08-04 (owner's call: not a
|
||||
pattern, and the resident model cannot be asked per turn either). A regex whose output is a
|
||||
fact or a route is the defect; a regex over structured input — HTML, MIME, JSON, a URL, an
|
||||
argv list — is not. Before writing a Russian word list, pick one of these:
|
||||
|
||||
- **`internal/lexicon`** — closed classes, in `lexicon_ru_v1.json`. Interrogatives,
|
||||
capture verbs, reminder verbs, cardinals, day offsets, parts of day, weekdays, months,
|
||||
spoken hours. Editing a word is a data change, and there is exactly one copy: months used
|
||||
to live in three files. Cardinals carry the oblique forms, because a spoken time declines
|
||||
and `в семь` / `к семи` are one hour.
|
||||
- **`internal/morph`** — grammar, from the vendored golem Russian dictionary. `IsVerbForm`
|
||||
and `SameWord`. Note that lemma matching is BROADER than stem-plus-one-ending, so a verb
|
||||
slot that means the imperative must be matched exactly — `говори` and `говорил` are one
|
||||
lemma and only one of them is a command (`cmd/mavend/quiet_toggle.go`).
|
||||
- **`cmd/mavend/topics.go` and the embedder** — open sets, where the question is what a
|
||||
turn is ABOUT. Frozen seeds per subject plus a real `other` class, scored against the
|
||||
turn's own query vector. Same shape as the personal boundary in `personalboundary.go`,
|
||||
with one difference: a topic must clear the runner-up by `topicMargin`, because a false
|
||||
claim here spends a network scan rather than one honest "не знаю". The old keyword tests
|
||||
stay as the offline floor and may remain narrow, since they are no longer the only answer.
|
||||
- **The ecosystem trio** — when the answer is not in the utterance at all. Identity is
|
||||
Nexus's, never a local pattern.
|
||||
|
||||
Seeds are scoring data. Editing one moves a recogniser and must be re-measured against the
|
||||
`TestONNX*` tests, not eyeballed.
|
||||
|
||||
## Non-goals (hard constraints)
|
||||
|
||||
Not a nag, not autonomous. Maven's persona is **feminine** — Russian
|
||||
self-reference must use feminine forms — `рада`, not `рад`; `поняла`, not `понял`. The owner
|
||||
is male and is addressed informally: "ты", singular, never "вы"/"ваш" and never "он"/"его"
|
||||
(she talks TO the owner, not about the owner). Pet names ("милый", "дорогой") are forbidden; the name
|
||||
("Ками") is not. The eval enforces this: `CheckAddress`, `CheckFeminine` and `CheckCringe` in
|
||||
`internal/phraser/eval/checks.go`, scored by `make eval-phrasing`.
|
||||
|
||||
**"Never phones home" is DEPRECATED** (owner's call, 2026-07-31). It used to be a hard
|
||||
constraint and it is not one any more: a 0.8B — and a 1.7B — does not know enough to answer
|
||||
world questions, so she needs to read external sources. What replaces it:
|
||||
|
||||
- **No telemetry, no cloud model, no third-party account.** That part never changes. Nothing
|
||||
about Maven is reported to anyone, and inference stays on the box.
|
||||
- **The owner's data first, then the world.** Every source that reads the owner's facts, notes, calendar,
|
||||
tasks or house runs before anything outside, and the personal boundary sits between them.
|
||||
Reading beats recalling for a small model.
|
||||
- **In the world, live search leads and the ZIMs are the fallback** (owner's call,
|
||||
2026-08-02). A self-hosted SearXNG (`search` block) answers first; the Kiwix ZIMs on
|
||||
homesrv answer when the search is empty, unreachable, or the line is down.
|
||||
**Verified with the line down on 2026-08-05** (V-508,
|
||||
`docs/evals/2026-08-05-kiwix-offline-fallback.md`): a stopped SearXNG costs nothing,
|
||||
the ZIM answers in the same turn budget. A blackholed host cost 8 seconds the owner waited
|
||||
through. So the connect phase alone is capped at `dialTimeout` (1.5s), while a slow
|
||||
instance that did connect keeps the full 8. **A Russian question reads
|
||||
`wikipedia_ru_all_maxi_2026-02` verbatim** through `kiwix.book_ru`. The RU→EN rewriter
|
||||
is the workaround for an English book and is skipped there. Kiwix catalog names come
|
||||
from the filename, not the `<name>` field.
|
||||
**That verbatim path sent the whole sentence to a keyword engine until 09-08-2026**
|
||||
(V-668, `docs/evals/2026-08-09-kiwix-topic-retrieval.md`). Kiwix ranks by keyword
|
||||
overlap, so the question words outrank the one word naming the article. "что такое TCP"
|
||||
returned "Перехват TCP-соединения". "кто написал Войну и мир" returned an episode of
|
||||
Doctor Who. `kiwix.Topic` drops the narrative request, the interrogative and a verb
|
||||
behind one. `kiwix.TitlePath` tries the exact article first, since a ZIM is addressable
|
||||
by title and a wrong title is a 404. Five of eight questions reach the right article
|
||||
where they did not, one was already right, and nothing regressed. The title needs its
|
||||
leading capital, so `TitleCandidates` tries the spoken form and then the capitalized
|
||||
one. **"столица Франции" is answered by a title redirect to Париж**, which is the case
|
||||
the 2026-08-05 measurement named as unreachable by any lexical signal. Both apply on the
|
||||
verbatim path alone. The rewriter already reduces a question, and reducing twice takes
|
||||
the topic off its input.
|
||||
`Response.Empty()` is the whole gate and there is no quality threshold in front of it:
|
||||
the three signals one could read were measured on 2026-08-05 and none of them separate a
|
||||
real question from an invented one. Token overlap would cost "столица Франции" its
|
||||
answer, because the answer is Париж and that word is not in the question. See
|
||||
`docs/evals/2026-08-05-search-quality-signals.md` (V-539). **The embedder is not a
|
||||
fourth signal**, measured 2026-08-09 (V-668). Query-to-passage cosine scores 0.79 to
|
||||
0.91 on answerable questions and 0.75 to 0.84 on unanswerable ones, and the sets
|
||||
overlap. The wrong TCP article scored 0.8653, above five of six unanswerable rows. It
|
||||
measures topic and not whether the passage answers, so no threshold splits them. **Which query source claimed
|
||||
a turn is readable on `/chat`** as a badge beside the reply, carried on
|
||||
`ipc.ChatReply.Source` and noted by `noteQuerySource` in `cmd/mavend/querysource.go`. It
|
||||
rides the context, so `handleText` keeps the one string signature the mic, telegram and
|
||||
the web share.
|
||||
- **External search is allowed and off unless configured**, like the weather and telegram
|
||||
capabilities. The code default is still off. `deploy/mavend.json` now ships a `search`
|
||||
block (owner's call, 2026-08-02), so it is on for this box and deleting the block turns
|
||||
it off again.
|
||||
- **The owner's notes and facts are never search input.** Looking up why the sky is blue and
|
||||
sending the owner's stored personal notes to an upstream engine are different acts. Only the utterance goes
|
||||
out, never the persona block, history, or matched notes.
|
||||
|
||||
## Web UI conventions
|
||||
|
||||
Server-rendered pages share `cmd/mavweb/static/ui.css` (served at `/ui.css`) and the shell
|
||||
partial in `cmd/mavweb/shell.html`: a page opens with `{{template "shellTop" "<page-key>"}}`
|
||||
and closes with `{{template "shellBottom"}}`, and the key marks the active sidebar link.
|
||||
Every page is its own embedded `.html` file next to `main.go` — no page markup lives in Go,
|
||||
and the sidebar is data (`sidebarSections`, `pageIcon`) the template renders. No
|
||||
per-page `<style>` beyond true one-offs. Wrap every table in `<div class=scroll>` so wide
|
||||
data pans on a phone. Local preview + headless screenshot recipe is in `AGENTS.md`.
|
||||
|
||||
## Vikunja
|
||||
|
||||
This repo is project **Maven** (ID 2) in Vikunja. MCP: `http://localhost:9100/mcp` (or
|
||||
`http://192.168.1.104:9100/mcp` from workpc). Feature/bug/deploy tasks go there.
|
||||
|
||||
Vikunja is the durable task store. A task holds the goal, the constraints and the
|
||||
assumption ledger. Work without a task id is work nobody can resume, so a session that
|
||||
has no id asks for one before it starts.
|
||||
|
||||
The MCP tool schemas are deferred, so load the four you actually use in ONE call at the
|
||||
start of a session rather than one lookup per first use:
|
||||
|
||||
```text
|
||||
ToolSearch("select:mcp__vikunja__list_tasks,mcp__vikunja__get_task_details,mcp__vikunja__create_task,mcp__vikunja__update_task")
|
||||
```
|
||||
|
||||
**Close a finished task with `done: true` and nothing else** (owner's call, 07-08-2026).
|
||||
Do not write a completion summary into the description on the way out. It is lost anyway,
|
||||
and the durable record is the commit messages and the merged PR. Note that `update_task`
|
||||
carrying a `description` resets `done` to false, which is why a write-up ever took two
|
||||
calls.
|
||||
- **No telemetry, no cloud model, no third-party account.** Inference stays on
|
||||
the box and nothing about Maven is reported to anyone.
|
||||
- **The owner's data first, then the world.** Every source reading his facts,
|
||||
notes, calendar, tasks or house runs first, and the personal boundary sits
|
||||
between them and anything outside.
|
||||
- **His notes and facts are never search input.** Only the utterance goes out,
|
||||
never the persona block, the history, or matched notes.
|
||||
- **External search is allowed and off unless configured.** Deleting the
|
||||
`search` block in `deploy/mavend.json` turns it off.
|
||||
- **`Response.Empty()` is the whole gate** on a world answer. There is no
|
||||
quality threshold in front of it and four candidate signals all failed.
|
||||
|
||||
## Session workflow
|
||||
|
||||
`~/.local/bin/task` owns the branch, the commit identity and the PR. One task, one
|
||||
session, one PR.
|
||||
`docs/workflow.md` carries the five stores, the doc tiers and the guards. One
|
||||
task, one session, one PR. `/pickup` opens a session and `/wrap` closes it. Wrap
|
||||
at roughly half context rather than letting the session compact.
|
||||
|
||||
```sh
|
||||
task start <vikunja-id> # branch off origin/master, write TASK.md, fetch review comments
|
||||
task pr # push, open or refresh the PR, label Vikunja, notify
|
||||
task comments # re-pull this branch's review comments into .task/
|
||||
ToolSearch("select:mcp__vikunja__list_tasks,mcp__vikunja__get_task_details,mcp__vikunja__create_task,mcp__vikunja__update_task")
|
||||
```
|
||||
|
||||
Around that, `/pickup` opens a session and `/wrap` closes it. Wrap at roughly half
|
||||
context rather than letting the session compact.
|
||||
|
||||
Five stores, and each one owns something the others must not hold:
|
||||
|
||||
| Store | Holds | Lifetime |
|
||||
|---|---|---|
|
||||
| Vikunja task | goal, constraints, assumption ledger, status | durable |
|
||||
| `CLAUDE.md`, `AGENTS.md` | what an agent must know before touching code | durable |
|
||||
| `docs/` | design, measurements, decisions | durable |
|
||||
| `TASK.md` | the brief for this branch, written by `task start`, immutable | one branch |
|
||||
| `HANDOFF.md` | only what the next agent needs to resume | one session |
|
||||
|
||||
`TASK.md` and `.task/` are excluded through `.git/info/exclude`. `HANDOFF.md` is
|
||||
gitignored and injected at session start. If a line in the handoff would still matter
|
||||
next week, it is in the wrong file.
|
||||
|
||||
Docs are tiered by path, so staleness is visible from the filename. Files directly under
|
||||
`docs/` are living and carry a `Last verified: <date> @ <sha>` line. Files under
|
||||
`docs/evals/` are dated measurements and are never edited after the day, so a newer
|
||||
number is a new file. Files under `docs/archive/` are dead and read by nobody by default.
|
||||
|
||||
## Git guards
|
||||
|
||||
Two hooks in `.githooks/`, tracked, wired with `core.hooksPath`. Fresh clone:
|
||||
|
||||
```sh
|
||||
git config core.hooksPath .githooks
|
||||
```
|
||||
|
||||
- `pre-commit` refuses master, and refuses more than 300 changed lines in non-markdown
|
||||
files. Markdown is exempt and may land as one batch.
|
||||
- `commit-msg` requires the subject to end with `(V-<id>)`. `V-` and not `#`, because
|
||||
Gitea autolinks `#123` to a Gitea issue, which is the wrong tracker.
|
||||
|
||||
Two more guards live outside the repo, in `~/.claude/hooks/`. `diff-budget.sh` blocks
|
||||
further edits past 600 changed lines on a `task/` branch. `prose_lint_hook.py` checks
|
||||
prose on every write. Both measure against `origin/master`, so a local master that is
|
||||
ahead of the remote makes the diff budget read high.
|
||||
|
||||
`--no-verify` exists. Using it means saying why in the commit body.
|
||||
- This repo is Vikunja project **Maven** (ID 2), MCP at
|
||||
`http://localhost:9100/mcp`, or `http://192.168.1.104:9100/mcp` from workpc.
|
||||
- **A session with no task id asks for one before it starts**, because work
|
||||
without one is work nobody can resume.
|
||||
- **Close a finished task with `done: true` and nothing else** (owner's call,
|
||||
2026-08-07). `update_task` carrying a `description` resets `done` to false.
|
||||
- **`pre-commit` refuses master** and more than 300 changed lines in
|
||||
non-markdown files. Markdown is exempt and may land as one batch.
|
||||
- **`commit-msg` requires the subject to end with `(V-<id>)`.** `V-` and not
|
||||
`#`, because Gitea autolinks `#123` to the wrong tracker.
|
||||
- **`diff-budget.sh` blocks edits past 600 changed lines** on a `task/` branch.
|
||||
- **`--no-verify` exists.** Using it means saying why in the commit body.
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
# `test` below fail on the two packages that have no test files. deps-go builds
|
||||
# the missing tools in, so the vendored tree is self-sufficient. Keep the version
|
||||
# here in step with the `go` directive in go.mod.
|
||||
GO_VERSION := 1.25.5
|
||||
GO_VERSION := 1.25.12
|
||||
GO := $(shell pwd)/deps/go/go/bin/go
|
||||
export GOTOOLCHAIN := local
|
||||
GOFLAGS :=
|
||||
@@ -16,7 +16,7 @@ PIPER_BIN := $(shell pwd)/deps/piper/piper
|
||||
PIPER_MODEL := $(shell pwd)/models/tts/ru_RU-irina-medium.onnx
|
||||
PIPER_ESPEAK := $(shell pwd)/deps/piper/espeak-ng-data
|
||||
|
||||
.PHONY: t audit simulate stt-fixtures test-stt-golden all build build-stt build-tts build-daemon build-client build-waked build-web build-poll build-caldav clean test fmt-check vet run-stt run-tts run-web download-embedder deps-go deps-sentinel tidy eval-router eval-reach eval-recall eval-phrasing eval-models build-gpud
|
||||
.PHONY: t audit simulate stt-fixtures test-stt-golden all build build-stt build-tts build-daemon build-client build-waked build-web build-poll build-caldav clean test fmt-check vet run-stt run-tts run-web download-embedder deps-go deps-sentinel deps-vuln vuln deps-lint lint deadcode analyze tidy eval-router eval-reach eval-recall eval-phrasing eval-models build-gpud
|
||||
|
||||
all: build
|
||||
|
||||
@@ -73,7 +73,7 @@ run-web: build-web
|
||||
# builds them on demand, but `go test -coverprofile` calls covdata through
|
||||
# base.Tool(), which only stats pkg/tool and exits. So build them in once here.
|
||||
GO_TARBALL := go$(GO_VERSION).linux-amd64.tar.gz
|
||||
GO_SHA256 := 9e9b755d63b36acf30c12a9a3fc379243714c1c6d3dd72861da637f336ebb35b
|
||||
GO_SHA256 := 234828b7a89e0e303d2556310ee549fbcf253d28de937bac3da13d6294262ac1
|
||||
deps-go: deps-sentinel
|
||||
@mkdir -p deps/go
|
||||
cd deps/go && curl -fLO 'https://go.dev/dl/$(GO_TARBALL)'
|
||||
@@ -95,6 +95,70 @@ deps-sentinel:
|
||||
@mkdir -p deps
|
||||
@printf 'module github.com/kami/maven/deps\n\ngo 1.21\n' > deps/go.mod
|
||||
|
||||
# vuln — the advisory gate the 2026-08-10 audit found missing (V-682). It reads
|
||||
# the published database over the network, so it is not part of `test`, which
|
||||
# has to pass on a box with no route out. Run it before a toolchain or
|
||||
# dependency bump lands, because that is what it grades: on 2026-08-11 the
|
||||
# pinned Go 1.25.5 and x/text 0.14.0 carried 20 reachable advisories and the
|
||||
# bumped pair carries none.
|
||||
#
|
||||
# govulncheck is a tool and not a dependency, so it is installed into deps/ like
|
||||
# the toolchain rather than added to go.mod. The version is pinned here for the
|
||||
# same reason GO_VERSION is: a gate that moves on its own is not a gate.
|
||||
GOVULNCHECK_VERSION := v1.6.0
|
||||
GOVULNCHECK := $(shell pwd)/deps/bin/govulncheck
|
||||
|
||||
deps-vuln: deps-sentinel
|
||||
@mkdir -p deps/bin
|
||||
GOTOOLCHAIN=local GOBIN=$(shell pwd)/deps/bin \
|
||||
$(GO) install golang.org/x/vuln/cmd/govulncheck@$(GOVULNCHECK_VERSION)
|
||||
|
||||
# The CGO env is the same one `test` carries: govulncheck loads the packages,
|
||||
# and the four CGO daemons do not load without it.
|
||||
vuln: deps-vuln
|
||||
CGO_CFLAGS="$(CGO_CFLAGS)" CGO_LDFLAGS="$(CGO_LDFLAGS)" LD_LIBRARY_PATH="$(shell pwd)/deps/lib" \
|
||||
PATH="$(shell pwd)/deps/go/go/bin:$$PATH" GOTOOLCHAIN=local $(GOVULNCHECK) ./...
|
||||
|
||||
# lint and deadcode — the other two analyzers the 2026-08-10 audit asked for
|
||||
# (V-694). They are not part of `test` for the same reason `vuln` is not: they
|
||||
# install over the network, and they are slow enough that a change to one Go
|
||||
# file should not pay for them.
|
||||
#
|
||||
# Neither reports zero, so neither fails on its own output. The accepted set
|
||||
# lives in scripts/analyzers/*.baseline and scripts/analyzer-gate.sh decides.
|
||||
# What is new fails, and so does a baseline entry whose finding is gone.
|
||||
#
|
||||
# deadcode runs with -test, so a test file is a root. Without it the report is
|
||||
# 172 lines, most of internal/router/eval, and none of it is a mistake.
|
||||
STATICCHECK_VERSION := v0.7.0
|
||||
DEADCODE_VERSION := v0.48.0
|
||||
STATICCHECK := $(shell pwd)/deps/bin/staticcheck
|
||||
DEADCODE := $(shell pwd)/deps/bin/deadcode
|
||||
|
||||
deps-lint: deps-sentinel
|
||||
@mkdir -p deps/bin
|
||||
GOTOOLCHAIN=local GOBIN=$(shell pwd)/deps/bin \
|
||||
$(GO) install honnef.co/go/tools/cmd/staticcheck@$(STATICCHECK_VERSION)
|
||||
GOTOOLCHAIN=local GOBIN=$(shell pwd)/deps/bin \
|
||||
$(GO) install golang.org/x/tools/cmd/deadcode@$(DEADCODE_VERSION)
|
||||
|
||||
# Both load the packages, so both carry the CGO env `test` carries. Without it
|
||||
# the four CGO daemons do not load and the analyzer reports a build error
|
||||
# instead of a finding -- which analyzer-gate.sh fails on rather than filters.
|
||||
ANALYZER_ENV = CGO_CFLAGS="$(CGO_CFLAGS)" CGO_LDFLAGS="$(CGO_LDFLAGS)" \
|
||||
LD_LIBRARY_PATH="$(shell pwd)/deps/lib" \
|
||||
PATH="$(shell pwd)/deps/go/go/bin:$$PATH" GOTOOLCHAIN=local
|
||||
|
||||
lint: deps-lint
|
||||
@$(ANALYZER_ENV) $(STATICCHECK) ./... | scripts/analyzer-gate.sh staticcheck
|
||||
|
||||
deadcode: deps-lint
|
||||
@$(ANALYZER_ENV) $(DEADCODE) -test ./... | scripts/analyzer-gate.sh deadcode
|
||||
|
||||
# Every static gate in one command. Not `check`, because it is not the thing to
|
||||
# run before a commit: vuln reads the network and all three are slow.
|
||||
analyze: lint deadcode vuln
|
||||
|
||||
# Run the tidy the sentinel makes possible. Not part of `test`: it rewrites
|
||||
# go.mod, and a build target that edits the module file is a surprise.
|
||||
# vendor/ is committed, so a tidy that drops a requirement must be followed by
|
||||
|
||||
+9
-40
@@ -386,8 +386,13 @@ func modelSeam(cfg *config.Config, resident *llm.Client) (router.Completer, *llm
|
||||
return resident, nil
|
||||
}
|
||||
ws := cfg.Workstation
|
||||
remote := llm.New(ws.URL, time.Duration(ws.Timeout))
|
||||
remote.SetToken(ws.Token)
|
||||
if ws.Token == "" {
|
||||
log.Printf("voice: no workstation.token — mavgpud refuses an unauthenticated request, so this reads as a card that is always busy")
|
||||
}
|
||||
pair := llm.NewPair(
|
||||
llm.New(ws.URL, time.Duration(ws.Timeout)),
|
||||
remote,
|
||||
resident,
|
||||
ws.Health,
|
||||
time.Duration(ws.Probe),
|
||||
@@ -464,45 +469,9 @@ func buildRouter(emb router.Embedder, acts router.ActMatcher, threshold float64,
|
||||
llmR *router.LLMRouter, heads *router.RouterHeads) *router.Router {
|
||||
cls := router.NewClassifier(emb)
|
||||
seedClassifier(cls)
|
||||
grammars := router.DefaultGrammars(acts)
|
||||
grammars = append(grammars, router.SystemTimeDateGrammars()...)
|
||||
// After the time/date rules on purpose: "какой сегодня день" is a clock
|
||||
// question and must keep reaching replySystem, while "что у меня сегодня"
|
||||
// is an agenda question and must not.
|
||||
grammars = append(grammars, router.AgendaQueryGrammars()...)
|
||||
// Same reason as the agenda rules, for the feeds: "что нового в лентах?"
|
||||
// routed system and answered "пока не умею" (Vikunja #474).
|
||||
// After the agenda rules, which are the narrower claim, and BEFORE the feed
|
||||
// and list rules, which are not: "что такое лента" is a definition question
|
||||
// and the feed rule would take it on the noun alone (V-655).
|
||||
grammars = append(grammars, router.WorldQueryGrammars()...)
|
||||
grammars = append(grammars, router.FeedQueryGrammar())
|
||||
// The list side of the same exposure: a phrasing with no possessive in it
|
||||
// ("список дел") routed system and never reached queryTasks (Vikunja #467).
|
||||
grammars = append(grammars, router.TaskListGrammar())
|
||||
grammars = append(grammars, router.ListGrammars()...)
|
||||
grammars = append(grammars, router.ReminderGrammar())
|
||||
// Before the capture marker, because "отметь" is a capture verb and "отметь
|
||||
// второй пункт" is not a note. The Praxis rules are the narrower claim — a
|
||||
// lifecycle verb AND an item named — so they get first refusal (Vikunja #516).
|
||||
grammars = append(grammars, router.PraxisGrammars()...)
|
||||
// Last, and it matches any utterance shape — its Build is the filter. An
|
||||
// explicit capture marker beats the model, which called it an act and
|
||||
// rewrote the task text (Vikunja #467). After the rules above because a
|
||||
// marker never collides with a clock or agenda question.
|
||||
// After Praxis, whose bare "закрой" claim this rule cannot reach (it needs the
|
||||
// board noun), and before the capture marker, which would otherwise read
|
||||
// "убери из задач купить молоко" as a new task (Vikunja #512).
|
||||
grammars = append(grammars, router.TaskStatusGrammar())
|
||||
// Before the capture markers, which all need an object. A capture verb
|
||||
// alone is a fact with no key, and the clarify path asks for it rather than
|
||||
// letting the model invent an answer (Vikunja #557).
|
||||
grammars = append(grammars, router.BareCaptureGrammar()...)
|
||||
grammars = append(grammars, router.TaskCaptureGrammar())
|
||||
// After the capture marker, so "запиши" still wins over "расскажи", and
|
||||
// last overall because it matches on the first word alone: "расскажи про
|
||||
// X" is a world question the model called a fact (Vikunja #498).
|
||||
grammars = append(grammars, router.NarrativeQueryGrammars()...)
|
||||
// The stage 0 set, in the router package, so the eval fixture runs the rules
|
||||
// the daemon runs (V-693). Order and reasoning live with the list.
|
||||
grammars := router.StageZeroGrammars(acts)
|
||||
return router.New(router.Config{
|
||||
Grammars: grammars,
|
||||
Classifier: cls,
|
||||
|
||||
@@ -0,0 +1,145 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"crypto/subtle"
|
||||
"fmt"
|
||||
"net"
|
||||
"net/http"
|
||||
"os"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// The boundary in front of the card.
|
||||
//
|
||||
// mavgpud has to listen on the LAN, because homesrv is the client and a
|
||||
// loopback default takes the model arm down. That makes this the one hop on the
|
||||
// workstation anything on the network could reach, and until 2026-08-11 it
|
||||
// reverse-proxied every path to llama-server unauthenticated: any client could
|
||||
// spend the card, hold the model resident by touching the idle clock, and read
|
||||
// /slots, which carries the prompts of whoever else was using it.
|
||||
//
|
||||
// So: a bearer token every request must carry, read from a file, and a path
|
||||
// allowlist so a token that leaks buys the model API and not the admin one. The
|
||||
// CW2 transcriber beside this daemon has worked this way since it shipped; this
|
||||
// is the same arrangement, not a new one.
|
||||
|
||||
// readToken loads the bearer token. The file holds the token and nothing else,
|
||||
// trailing newline allowed. A path that is set and unreadable is fatal to the
|
||||
// caller: a supervisor that silently ran without its boundary is the failure
|
||||
// this exists to prevent.
|
||||
func readToken(path string) (string, error) {
|
||||
b, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("token_file: %w", err)
|
||||
}
|
||||
tok := strings.TrimSpace(string(b))
|
||||
if tok == "" {
|
||||
return "", fmt.Errorf("token_file %s is empty", path)
|
||||
}
|
||||
return tok, nil
|
||||
}
|
||||
|
||||
// loopbackListen reports whether addr can only be reached from this machine.
|
||||
// An empty or wildcard host is not loopback, which is the case that matters:
|
||||
// ":8080" is the shipped default and it answers the whole LAN.
|
||||
func loopbackListen(addr string) bool {
|
||||
host, _, err := net.SplitHostPort(addr)
|
||||
if err != nil {
|
||||
host = addr
|
||||
}
|
||||
host = strings.Trim(host, "[]")
|
||||
if host == "" {
|
||||
return false
|
||||
}
|
||||
if host == "localhost" {
|
||||
return true
|
||||
}
|
||||
ip := net.ParseIP(host)
|
||||
return ip != nil && ip.IsLoopback()
|
||||
}
|
||||
|
||||
// allowed is what a token buys. Everything llama-server exposes beyond this is
|
||||
// refused, because the endpoints Maven does not call are the expensive ones to
|
||||
// hand out: /slots returns other callers' prompts, and its save/restore actions
|
||||
// write files chosen by the request.
|
||||
//
|
||||
// Adding a caller means adding its path here. That is deliberate — the list is
|
||||
// short because Maven's use of the workstation is.
|
||||
var allowed = map[string]string{
|
||||
"/v1/chat/completions": http.MethodPost,
|
||||
"/v1/completions": http.MethodPost,
|
||||
"/v1/embeddings": http.MethodPost,
|
||||
"/v1/models": http.MethodGet,
|
||||
"/props": http.MethodGet,
|
||||
}
|
||||
|
||||
// requireToken authenticates, then bounds. Order matters: an unauthenticated
|
||||
// client must not be able to make this daemon allocate a body buffer.
|
||||
//
|
||||
// /health is not exempt. It reports whether the card is loaded and free, which
|
||||
// is exactly what someone deciding whether to take it from him would ask.
|
||||
func requireToken(token string, maxBody int64, next http.Handler) http.Handler {
|
||||
want := sha256.Sum256([]byte(token))
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
got := sha256.Sum256([]byte(bearer(r)))
|
||||
if subtle.ConstantTimeCompare(got[:], want[:]) != 1 {
|
||||
w.Header().Set("WWW-Authenticate", "Bearer")
|
||||
http.Error(w, "unauthorized", http.StatusUnauthorized)
|
||||
return
|
||||
}
|
||||
r.Body = http.MaxBytesReader(w, r.Body, maxBody)
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
|
||||
// bearer pulls the credential out of the header. A malformed header yields the
|
||||
// empty string, which fails the comparison like any other wrong token — there
|
||||
// is no separate error for it, because telling a caller *how* it was wrong is
|
||||
// the only thing a probe learns from a 401.
|
||||
func bearer(r *http.Request) string {
|
||||
h := r.Header.Get("Authorization")
|
||||
const prefix = "Bearer "
|
||||
if len(h) <= len(prefix) || !strings.EqualFold(h[:len(prefix)], prefix) {
|
||||
return ""
|
||||
}
|
||||
return strings.TrimSpace(h[len(prefix):])
|
||||
}
|
||||
|
||||
// allowlist refuses a path the model arm does not use. It answers 404 rather
|
||||
// than 403 so a scan cannot map llama-server's surface through this hop.
|
||||
func allowlist(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
method, ok := allowed[r.URL.Path]
|
||||
if !ok {
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
if r.Method != method {
|
||||
w.Header().Set("Allow", method)
|
||||
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
|
||||
return
|
||||
}
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
|
||||
// limitInflight caps concurrent proxied requests. A waiter leaves when its own
|
||||
// context ends, so a client that gave up does not keep a slot: llama-server
|
||||
// runs with -np 1 and queueing here is cheaper than queueing inside the child
|
||||
// with a body held in memory on both sides.
|
||||
func limitInflight(n int, next http.Handler) http.Handler {
|
||||
if n <= 0 {
|
||||
return next
|
||||
}
|
||||
slots := make(chan struct{}, n)
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
select {
|
||||
case slots <- struct{}{}:
|
||||
defer func() { <-slots }()
|
||||
next.ServeHTTP(w, r)
|
||||
case <-r.Context().Done():
|
||||
http.Error(w, "client went away", http.StatusServiceUnavailable)
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,180 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// ok is what the boundary is protecting: anything that reaches it has spent
|
||||
// the card.
|
||||
func ok(w http.ResponseWriter, r *http.Request) { w.WriteHeader(http.StatusTeapot) }
|
||||
|
||||
func TestRequireTokenRefusesEveryWrongCredential(t *testing.T) {
|
||||
h := requireToken("s3cret", 1<<20, http.HandlerFunc(ok))
|
||||
cases := []struct {
|
||||
name string
|
||||
auth string
|
||||
want int
|
||||
}{
|
||||
{"no header", "", http.StatusUnauthorized},
|
||||
{"wrong token", "Bearer wrong", http.StatusUnauthorized},
|
||||
{"prefix of the token", "Bearer s3cre", http.StatusUnauthorized},
|
||||
{"token with no scheme", "s3cret", http.StatusUnauthorized},
|
||||
{"basic auth", "Basic czNjcmV0", http.StatusUnauthorized},
|
||||
{"right token", "Bearer s3cret", http.StatusTeapot},
|
||||
{"scheme is case-insensitive", "bearer s3cret", http.StatusTeapot},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
r := httptest.NewRequest(http.MethodGet, "/health", nil)
|
||||
if tc.auth != "" {
|
||||
r.Header.Set("Authorization", tc.auth)
|
||||
}
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, r)
|
||||
if w.Code != tc.want {
|
||||
t.Errorf("status %d, want %d", w.Code, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The 401 must not say which part was wrong. A probe that can tell a malformed
|
||||
// header from a wrong token learns the header shape for free.
|
||||
func TestUnauthorizedSaysNothingUseful(t *testing.T) {
|
||||
h := requireToken("s3cret", 1<<20, http.HandlerFunc(ok))
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, httptest.NewRequest(http.MethodGet, "/health", nil))
|
||||
if got := strings.TrimSpace(w.Body.String()); got != "unauthorized" {
|
||||
t.Errorf("body %q, want %q", got, "unauthorized")
|
||||
}
|
||||
if got := w.Header().Get("WWW-Authenticate"); got != "Bearer" {
|
||||
t.Errorf("WWW-Authenticate %q, want Bearer", got)
|
||||
}
|
||||
}
|
||||
|
||||
// The body cap applies to an authenticated request. An unauthenticated one
|
||||
// never gets far enough to allocate anything.
|
||||
func TestRequireTokenCapsTheBody(t *testing.T) {
|
||||
var read error
|
||||
h := requireToken("t", 8, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
buf := make([]byte, 64)
|
||||
for read == nil {
|
||||
if _, read = r.Body.Read(buf); read != nil {
|
||||
break
|
||||
}
|
||||
}
|
||||
}))
|
||||
r := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(strings.Repeat("x", 4096)))
|
||||
r.Header.Set("Authorization", "Bearer t")
|
||||
h.ServeHTTP(httptest.NewRecorder(), r)
|
||||
if read == nil || !strings.Contains(read.Error(), "too large") {
|
||||
t.Errorf("read error %v, want the body cap", read)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAllowlistRefusesWhatMavenDoesNotCall(t *testing.T) {
|
||||
h := allowlist(http.HandlerFunc(ok))
|
||||
cases := []struct {
|
||||
method, path string
|
||||
want int
|
||||
}{
|
||||
{http.MethodPost, "/v1/chat/completions", http.StatusTeapot},
|
||||
{http.MethodGet, "/v1/models", http.StatusTeapot},
|
||||
// /slots returns the prompts of whoever else is using the card, and
|
||||
// its actions write files the request names.
|
||||
{http.MethodGet, "/slots", http.StatusNotFound},
|
||||
{http.MethodPost, "/slots/0?action=save", http.StatusNotFound},
|
||||
{http.MethodGet, "/", http.StatusNotFound},
|
||||
{http.MethodGet, "/v1/chat/completions", http.StatusMethodNotAllowed},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.method+" "+tc.path, func(t *testing.T) {
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, httptest.NewRequest(tc.method, tc.path, nil))
|
||||
if w.Code != tc.want {
|
||||
t.Errorf("status %d, want %d", w.Code, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestLimitInflightCapsConcurrency(t *testing.T) {
|
||||
const cap = 2
|
||||
var mu sync.Mutex
|
||||
now, peak := 0, 0
|
||||
release := make(chan struct{})
|
||||
h := limitInflight(cap, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
mu.Lock()
|
||||
now++
|
||||
if now > peak {
|
||||
peak = now
|
||||
}
|
||||
mu.Unlock()
|
||||
<-release
|
||||
mu.Lock()
|
||||
now--
|
||||
mu.Unlock()
|
||||
}))
|
||||
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 8; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
h.ServeHTTP(httptest.NewRecorder(), httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil))
|
||||
}()
|
||||
}
|
||||
// Let the first wave arrive, then drain. The assertion is the peak, and a
|
||||
// peak that never reached the cap still cannot exceed it.
|
||||
close(release)
|
||||
wg.Wait()
|
||||
if peak > cap {
|
||||
t.Errorf("%d requests in flight at once, cap is %d", peak, cap)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoopbackListen(t *testing.T) {
|
||||
cases := map[string]bool{
|
||||
":8080": false, // the shipped default, and the whole LAN
|
||||
"0.0.0.0:8080": false,
|
||||
"[::]:8080": false,
|
||||
"192.168.1.105:8080": false,
|
||||
"127.0.0.1:8080": true,
|
||||
"[::1]:8080": true,
|
||||
"localhost:8080": true,
|
||||
}
|
||||
for addr, want := range cases {
|
||||
if got := loopbackListen(addr); got != want {
|
||||
t.Errorf("loopbackListen(%q) = %v, want %v", addr, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestReadToken(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
good := filepath.Join(dir, "tok")
|
||||
if err := os.WriteFile(good, []byte(" abc123\n"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := readToken(good)
|
||||
if err != nil || got != "abc123" {
|
||||
t.Errorf("readToken = %q, %v; want abc123", got, err)
|
||||
}
|
||||
|
||||
blank := filepath.Join(dir, "blank")
|
||||
if err := os.WriteFile(blank, []byte("\n\n"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := readToken(blank); err == nil {
|
||||
t.Error("an empty token file is not a token")
|
||||
}
|
||||
if _, err := readToken(filepath.Join(dir, "absent")); err == nil {
|
||||
t.Error("a missing token file is not a token")
|
||||
}
|
||||
}
|
||||
@@ -104,9 +104,11 @@ func TestHealthAndProxyRefuseWhenNotReady(t *testing.T) {
|
||||
s := &supervisor{run: newRunner("fake", "/bin/true", nil, "")}
|
||||
h := s.handler(mustURL(t, "http://127.0.0.1:1"))
|
||||
|
||||
for _, path := range []string{"/health", "/v1/chat/completions"} {
|
||||
// The completion is a POST because the allowlist is in front of the
|
||||
// readiness check now, and it answers 405 to a method it never serves.
|
||||
for path, method := range map[string]string{"/health": http.MethodGet, "/v1/chat/completions": http.MethodPost} {
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, httptest.NewRequest(http.MethodGet, path, nil))
|
||||
h.ServeHTTP(w, httptest.NewRequest(method, path, nil))
|
||||
if w.Code != http.StatusServiceUnavailable {
|
||||
t.Errorf("%s with no model: got %d, want 503", path, w.Code)
|
||||
}
|
||||
|
||||
+47
-3
@@ -36,6 +36,21 @@ type config struct {
|
||||
Listen string `json:"listen"` // what Maven talks to
|
||||
LlamaAddr string `json:"llama_addr"` // where llama-server binds
|
||||
LlamaBin string `json:"llama_bin"`
|
||||
|
||||
// TokenFile holds the bearer token every request must carry. It is a path
|
||||
// and never the token itself, the rule mavpoll, mavmaild and the CW2
|
||||
// transcriber already follow: a secret in a committed config is a secret
|
||||
// in the history. Empty is allowed only on a loopback Listen, and
|
||||
// requireToken is where that is decided.
|
||||
TokenFile string `json:"token_file,omitempty"`
|
||||
|
||||
// MaxBody bounds a proxied request body. A completion is a prompt, and a
|
||||
// prompt that does not fit here would not fit the context window either.
|
||||
MaxBody int64 `json:"max_body_bytes,omitempty"`
|
||||
// MaxInflight bounds how many proxied requests reach llama-server at once.
|
||||
// It runs with -np 1, so anything above a handful only queues inside the
|
||||
// child while holding a connection and a body in memory here.
|
||||
MaxInflight int `json:"max_inflight,omitempty"`
|
||||
// LlamaArgs must include the flags that bind LlamaAddr. They are passed
|
||||
// through untouched so the model, context size and layer count stay the
|
||||
// owner's business and not this daemon's schema.
|
||||
@@ -88,6 +103,8 @@ func defaults() config {
|
||||
MinFreeVRAM: 15 << 30,
|
||||
EvictAfter: 2,
|
||||
StartAfter: 5,
|
||||
MaxBody: 8 << 20,
|
||||
MaxInflight: 4,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -123,6 +140,20 @@ func main() {
|
||||
log.Fatal("mavgpud: llama_bin is required")
|
||||
}
|
||||
|
||||
// A LAN listener with no token is refused rather than downgraded to
|
||||
// loopback. Downgrading would look like a safe default and would take the
|
||||
// model arm down instead: homesrv is the client and it is on the LAN.
|
||||
var token string
|
||||
if cfg.TokenFile != "" {
|
||||
var err error
|
||||
if token, err = readToken(cfg.TokenFile); err != nil {
|
||||
log.Fatalf("mavgpud: %v", err)
|
||||
}
|
||||
} else if !loopbackListen(cfg.Listen) {
|
||||
log.Fatalf("mavgpud: listen %s is reachable from the network and token_file is unset — "+
|
||||
"set token_file, or listen on 127.0.0.1 and accept that Maven cannot reach it", cfg.Listen)
|
||||
}
|
||||
|
||||
base := "http://" + cfg.LlamaAddr
|
||||
run := newRunner("llama-server", cfg.LlamaBin, cfg.LlamaArgs, base+"/health")
|
||||
sup := &supervisor{
|
||||
@@ -145,7 +176,20 @@ func main() {
|
||||
if err != nil {
|
||||
log.Fatalf("mavgpud: llama_addr: %v", err)
|
||||
}
|
||||
srv := &http.Server{Addr: cfg.Listen, Handler: sup.handler(target)}
|
||||
var h http.Handler = sup.handler(target)
|
||||
if token != "" {
|
||||
h = requireToken(token, cfg.MaxBody, h)
|
||||
}
|
||||
srv := &http.Server{
|
||||
Addr: cfg.Listen,
|
||||
Handler: h,
|
||||
// A slow-loris client holds a connection and a header buffer for free
|
||||
// otherwise. No ReadTimeout or WriteTimeout: a completion legitimately
|
||||
// takes minutes on this card, and either one would cut it off.
|
||||
ReadHeaderTimeout: 10 * time.Second,
|
||||
IdleTimeout: 60 * time.Second,
|
||||
MaxHeaderBytes: 1 << 16,
|
||||
}
|
||||
go func() {
|
||||
log.Printf("mavgpud: listening on %s, model %s", cfg.Listen, cfg.LlamaBin)
|
||||
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
|
||||
@@ -203,14 +247,14 @@ func (s *supervisor) handler(target *url.URL) http.Handler {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_, _ = w.Write([]byte(`{"status":"ok"}`))
|
||||
})
|
||||
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
|
||||
mux.Handle("/", allowlist(limitInflight(s.cfg.MaxInflight, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if !s.run.isReady() {
|
||||
http.Error(w, "model not loaded", http.StatusServiceUnavailable)
|
||||
return
|
||||
}
|
||||
s.touch()
|
||||
proxy.ServeHTTP(w, r)
|
||||
})
|
||||
}))))
|
||||
return mux
|
||||
}
|
||||
|
||||
|
||||
+47
-5
@@ -12,10 +12,17 @@
|
||||
// samples and the capture frame is 480, so silero.go re-chunks. This comment
|
||||
// used to say the two matched, which was true of silero v4.
|
||||
//
|
||||
// There is still no wake-word model, so anything spoken near the microphone
|
||||
// becomes a turn (V-487 stage two). The SurfaceVoice auth layer caps all
|
||||
// commands at L0 (no destructive acts), which is what makes an accidental
|
||||
// trigger safe rather than expensive.
|
||||
// The keyword is "Мэйвен" and it is required, when -wake-model points at the
|
||||
// head (V-487 stage two). Without it anything spoken near the microphone
|
||||
// becomes a turn, which the SurfaceVoice auth layer makes safe rather than
|
||||
// expensive: it caps all commands at L0, no destructive acts. It does not cap
|
||||
// reading, so an open gate still lets the room hear his facts read back.
|
||||
// wakeword.go holds the cadence and wakefeatures.go the three models.
|
||||
//
|
||||
// The conn carries both directions. mavwaked sends utterances and receives
|
||||
// proactive nudges on it, and it is opened at startup rather than at the first
|
||||
// utterance, because mavend registers a voice session on accept. See nudge.go
|
||||
// for why a nudge that is not heard is worse than one that is not delivered.
|
||||
//
|
||||
// While a reply is playing the capture side is muted (half-duplex): without
|
||||
// it, Maven's own voice comes back in through the mic and she answers
|
||||
@@ -57,6 +64,12 @@ const (
|
||||
defaultAddr = "127.0.0.1:9100"
|
||||
defaultLang = "ru"
|
||||
defaultReadSize = 4096 // max PCM bytes per read from arecord (fits multiple frames)
|
||||
|
||||
// defaultWakeWindowMs — how long the keyword stays good for. He says
|
||||
// "Мэйвен" and then a sentence, and the VAD does not close the utterance
|
||||
// until he stops, so this has to outlive the word by the length of what
|
||||
// follows it. It is spent on dispatch: one keyword, one turn.
|
||||
defaultWakeWindowMs = 8000
|
||||
)
|
||||
|
||||
func main() {
|
||||
@@ -81,12 +94,18 @@ func run(args []string) error {
|
||||
vadModel := flag.String("vad-model", "", "silero-vad onnx file; empty runs the energy threshold instead")
|
||||
vadThreshold := flag.Float64("vad-threshold", defaultSileroThreshold, "speech probability a frame must clear")
|
||||
onnxLib := flag.String("onnx-lib", os.Getenv("MAVEN_ONNX_LIB"), "libonnxruntime.so, needed with -vad-model")
|
||||
wakeModel := flag.String("wake-model", "", "keyword head onnx; empty ships every utterance, as before V-487")
|
||||
wakeMel := flag.String("wake-mel", "", "melspectrogram.onnx, required with -wake-model")
|
||||
wakeEmbed := flag.String("wake-embed", "", "embedding_model.onnx, required with -wake-model")
|
||||
wakeThreshold := flag.Float64("wake-threshold", defaultWakeThreshold, "score the keyword must clear")
|
||||
wakeWindowMs := flag.Int("wake-window-ms", defaultWakeWindowMs, "ms an utterance may still start after the keyword")
|
||||
flag.CommandLine.Parse(args)
|
||||
|
||||
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM, syscall.SIGHUP)
|
||||
defer stop()
|
||||
|
||||
// Voice client — reused across utterances; SendRequest reconnects on error.
|
||||
// Voice client — one conn carrying both directions. SendRequest reconnects
|
||||
// on error, and the push receiver redials on its own clock.
|
||||
vc := voice.Dial(*addr)
|
||||
defer vc.Close()
|
||||
|
||||
@@ -165,6 +184,29 @@ func run(args []string) error {
|
||||
}
|
||||
sess := newSession(vad, newAplayPlayer(), &voiceSender{vc: vc}, *lang, barge)
|
||||
|
||||
// Keyword gate. A model that will not load is logged and not fatal, for
|
||||
// the same reason silero's is not: an open gate is the daemon he had
|
||||
// yesterday, and a daemon that refuses to start is not.
|
||||
if *wakeModel != "" {
|
||||
w, err := newWakeWord(*wakeMel, *wakeEmbed, *wakeModel, *onnxLib, *wakeThreshold)
|
||||
if err != nil {
|
||||
log.Printf("mavwaked: wake word unavailable, every utterance is a turn: %v", err)
|
||||
} else {
|
||||
defer w.Close()
|
||||
sess.UseWakeWord(w, time.Duration(*wakeWindowMs)*time.Millisecond)
|
||||
log.Printf("mavwaked: wake word from %s, threshold %.3f, window %dms",
|
||||
*wakeModel, *wakeThreshold, *wakeWindowMs)
|
||||
}
|
||||
}
|
||||
|
||||
// Listen for nudges alongside capture. Connect eagerly so mavend has a
|
||||
// voice session before he has said anything: without one, a nudge routed
|
||||
// to voice finds nobody home and goes to the away channels instead.
|
||||
if err := vc.Connect(ctx); err != nil {
|
||||
log.Printf("mavwaked: voice server not reachable yet, retrying in background: %v", err)
|
||||
}
|
||||
go runNudgeReceiver(ctx, vc, sess)
|
||||
|
||||
return captureLoop(ctx, src, sess)
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
package main
|
||||
|
||||
// The receiving half of the voice reach (V-671).
|
||||
//
|
||||
// mavwaked used to send and never listen. It wired no PushHandler, and
|
||||
// SendRequest discards a push frame when there is none. The consequence was
|
||||
// not a missing feature but a silent one: mavend routes a nudge to the voice
|
||||
// session that spoke most recently, and once mavwaked had spoken once it WAS
|
||||
// that session. PushToMostRecent succeeded, the dispatcher counted the nudge
|
||||
// delivered and stopped rerouting to telegram and ntfy, and mavwaked threw the
|
||||
// audio away. He heard nothing, anywhere.
|
||||
//
|
||||
// So the connection is opened at startup rather than at the first utterance,
|
||||
// and it is held open. A client that has never connected has no session, and
|
||||
// the dispatcher must be able to tell "he is not at the machine" from "he is,
|
||||
// and she has nothing to say".
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"log"
|
||||
"time"
|
||||
|
||||
"github.com/kami/maven/internal/voice"
|
||||
)
|
||||
|
||||
// nudgeRetry is how long to wait before dialling again after the conn ends.
|
||||
// mavend restarts on every deploy, and a listener that gives up then is a
|
||||
// listener that is deaf until the next reboot.
|
||||
const nudgeRetry = 5 * time.Second
|
||||
|
||||
// nudgeHandler decodes a push and hands the audio to the session, which
|
||||
// speaks it through the same player the reply path uses. It does not play
|
||||
// anything itself: the half-duplex gate and barge-in live on the capture
|
||||
// loop, and a nudge has to sit under both.
|
||||
type nudgeHandler struct{ sess *session }
|
||||
|
||||
func (h *nudgeHandler) OnPush(p voice.Push) {
|
||||
if p.Kind != voice.PushKindAudioNudge {
|
||||
log.Printf("mavwaked: ignoring push of unknown kind %q", p.Kind)
|
||||
return
|
||||
}
|
||||
var ap voice.AudioNudgePush
|
||||
if err := json.Unmarshal(p.Params, &ap); err != nil {
|
||||
log.Printf("mavwaked: nudge: decode: %v", err)
|
||||
return
|
||||
}
|
||||
log.Printf("mavwaked: nudge from rule %q (severity %d): %q (%.2fs audio)",
|
||||
ap.RuleName, ap.Severity, ap.Text, ap.Audio.Duration())
|
||||
if len(ap.Audio.Bytes) == 0 {
|
||||
// mavttsd was down or the text was empty. Say so rather than going
|
||||
// quiet: the dispatcher already counted this one as delivered.
|
||||
log.Printf("mavwaked: nudge %q carried no audio, nothing to speak", ap.RuleName)
|
||||
return
|
||||
}
|
||||
h.sess.Nudge(ap.Audio)
|
||||
}
|
||||
|
||||
// runNudgeReceiver keeps a push handler wired for as long as ctx lives,
|
||||
// redialling whenever the conn ends. Returns when ctx is cancelled.
|
||||
func runNudgeReceiver(ctx context.Context, vc *voice.Client, sess *session) {
|
||||
h := &nudgeHandler{sess: sess}
|
||||
for {
|
||||
err := vc.RunPushReceiver(ctx, h)
|
||||
if ctx.Err() != nil {
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
log.Printf("mavwaked: nudge receiver: %v", err)
|
||||
} else {
|
||||
log.Printf("mavwaked: voice connection ended, reconnecting in %s", nudgeRetry)
|
||||
}
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-time.After(nudgeRetry):
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,170 @@
|
||||
package main
|
||||
|
||||
// The receiving half: a nudge pushed by mavend has to reach the speaker, and
|
||||
// it has to obey the same two gates a reply obeys (V-671).
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/kami/maven/internal/audio"
|
||||
"github.com/kami/maven/internal/voice"
|
||||
)
|
||||
|
||||
func nudgeAudio() audio.Audio {
|
||||
return audio.Audio{Format: audio.PCM16kMono, Bytes: make([]byte, 8000)}
|
||||
}
|
||||
|
||||
// pushFrame builds the frame mavend's voicesink sends.
|
||||
func pushFrame(t *testing.T, a audio.Audio) voice.Push {
|
||||
t.Helper()
|
||||
body, err := json.Marshal(voice.AudioNudgePush{
|
||||
RuleName: "test-rule",
|
||||
Severity: 3,
|
||||
Audio: a,
|
||||
Text: "пора пить воду",
|
||||
Ts: time.Unix(0, 0),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("marshal push: %v", err)
|
||||
}
|
||||
return voice.Push{Kind: voice.PushKindAudioNudge, Params: body}
|
||||
}
|
||||
|
||||
// The defect itself: the push arrived and nothing came out of the speaker.
|
||||
func TestNudgeReachesThePlayer(t *testing.T) {
|
||||
sess, p, snd := newTestSession(bargeInConfig{})
|
||||
(&nudgeHandler{sess: sess}).OnPush(pushFrame(t, nudgeAudio()))
|
||||
|
||||
if p.plays != 0 {
|
||||
t.Fatal("nudge played from the push goroutine; it must wait for the capture loop")
|
||||
}
|
||||
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
if p.plays != 1 {
|
||||
t.Fatalf("plays = %d, want 1", p.plays)
|
||||
}
|
||||
if len(p.last.Bytes) != 8000 {
|
||||
t.Errorf("played %d bytes, want the nudge audio", len(p.last.Bytes))
|
||||
}
|
||||
if sess.nudges != 1 {
|
||||
t.Errorf("nudges = %d, want 1", sess.nudges)
|
||||
}
|
||||
if len(snd.sent) != 0 {
|
||||
t.Errorf("a nudge must not be shipped back to the daemon as an utterance")
|
||||
}
|
||||
}
|
||||
|
||||
// A push of some other kind, or one carrying no audio, must not reach the
|
||||
// player and must not wedge the one that follows.
|
||||
func TestNudgeIgnoresUnusablePushes(t *testing.T) {
|
||||
sess, p, _ := newTestSession(bargeInConfig{})
|
||||
h := &nudgeHandler{sess: sess}
|
||||
|
||||
h.OnPush(voice.Push{Kind: "something-else", Params: json.RawMessage(`{}`)})
|
||||
h.OnPush(voice.Push{Kind: voice.PushKindAudioNudge, Params: json.RawMessage(`not json`)})
|
||||
h.OnPush(pushFrame(t, audio.Audio{Format: audio.PCM16kMono}))
|
||||
|
||||
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
if p.plays != 0 {
|
||||
t.Fatalf("plays = %d, want 0", p.plays)
|
||||
}
|
||||
|
||||
h.OnPush(pushFrame(t, nudgeAudio()))
|
||||
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
if p.plays != 1 {
|
||||
t.Fatalf("plays after a usable nudge = %d, want 1", p.plays)
|
||||
}
|
||||
}
|
||||
|
||||
// The half-duplex gate covers a nudge exactly as it covers a reply: she does
|
||||
// not start one over herself, and the mic stays muted while it runs.
|
||||
func TestNudgeWaitsForTheReplyToFinish(t *testing.T) {
|
||||
sess, p, _ := newTestSession(bargeInConfig{})
|
||||
speakThenPause(t, sess)
|
||||
if !p.Playing() {
|
||||
t.Fatal("expected the reply to be playing")
|
||||
}
|
||||
plays := p.plays
|
||||
|
||||
(&nudgeHandler{sess: sess}).OnPush(pushFrame(t, nudgeAudio()))
|
||||
for i := 0; i < 20; i++ {
|
||||
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
}
|
||||
if p.plays != plays {
|
||||
t.Fatalf("nudge cut across the reply: plays = %d, want %d", p.plays, plays)
|
||||
}
|
||||
|
||||
p.playing = false
|
||||
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
if p.plays != plays+1 {
|
||||
t.Fatalf("nudge never played after the reply ended: plays = %d", p.plays)
|
||||
}
|
||||
}
|
||||
|
||||
// Speaking a nudge must not leave half a sentence in the VAD. The frames
|
||||
// captured before it are pre-nudge speech, and splicing them onto whatever he
|
||||
// says afterwards ships one utterance that is two.
|
||||
func TestNudgeResetsTheVAD(t *testing.T) {
|
||||
sess, p, snd := newTestSession(bargeInConfig{})
|
||||
loud := frameAt(0.35)
|
||||
speechFrames := (defaultSpeechMs + defaultFrameMs - 1) / defaultFrameMs
|
||||
for i := 0; i < speechFrames+5; i++ {
|
||||
if err := sess.feed(context.Background(), loud); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
(&nudgeHandler{sess: sess}).OnPush(pushFrame(t, nudgeAudio()))
|
||||
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
if p.plays != 1 {
|
||||
t.Fatalf("nudge did not play: plays = %d", p.plays)
|
||||
}
|
||||
|
||||
// Playback ends, silence follows. The half-formed utterance must be gone
|
||||
// rather than closing on the first quiet frame.
|
||||
p.playing = false
|
||||
silenceFrames := (defaultSilenceMs+defaultFrameMs-1)/defaultFrameMs + 2
|
||||
for i := 0; i < silenceFrames; i++ {
|
||||
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
}
|
||||
if len(snd.sent) != 0 {
|
||||
t.Fatalf("sent %d utterances after a nudge, want 0", len(snd.sent))
|
||||
}
|
||||
}
|
||||
|
||||
// Two nudges queued back to back: the newer one is what he hears. The
|
||||
// PushHandler contract in internal/voice says the next nudge replaces the
|
||||
// stale one rather than dogpiling on it.
|
||||
func TestNudgeReplacesAnUnspokenOne(t *testing.T) {
|
||||
sess, p, _ := newTestSession(bargeInConfig{})
|
||||
h := &nudgeHandler{sess: sess}
|
||||
|
||||
h.OnPush(pushFrame(t, audio.Audio{Format: audio.PCM16kMono, Bytes: make([]byte, 4000)}))
|
||||
h.OnPush(pushFrame(t, audio.Audio{Format: audio.PCM16kMono, Bytes: make([]byte, 12000)}))
|
||||
|
||||
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
if p.plays != 1 {
|
||||
t.Fatalf("plays = %d, want 1", p.plays)
|
||||
}
|
||||
if len(p.last.Bytes) != 12000 {
|
||||
t.Errorf("played %d bytes, want the newer nudge", len(p.last.Bytes))
|
||||
}
|
||||
}
|
||||
+136
-1
@@ -7,6 +7,7 @@ package main
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/kami/maven/internal/audio"
|
||||
@@ -19,6 +20,15 @@ type utteranceSender interface {
|
||||
Send(ctx context.Context, utt audio.Audio, lang string) (audio.Audio, error)
|
||||
}
|
||||
|
||||
// keywordGate answers whether the keyword has just been spoken. The
|
||||
// production one is wakeWord; tests substitute a recorder, because a gate that
|
||||
// can only be exercised with three ONNX files is a gate nobody tests.
|
||||
type keywordGate interface {
|
||||
Feed(frame []int16) bool
|
||||
Reset()
|
||||
Score() float64
|
||||
}
|
||||
|
||||
// bargeInConfig holds the two numbers barge-in needs. Zero Frames disables
|
||||
// barge-in entirely — the half-duplex gate still runs.
|
||||
type bargeInConfig struct {
|
||||
@@ -62,11 +72,29 @@ type session struct {
|
||||
// whenever playback ends.
|
||||
loudFrames int
|
||||
|
||||
// wake is the keyword gate, or nil when no model was loaded. wakeUntil is
|
||||
// how long a keyword stays good for: he says "Мэйвен" and then a sentence,
|
||||
// and the VAD does not close the utterance until he stops, so the window
|
||||
// has to outlive the word by the length of what follows it.
|
||||
wake keywordGate
|
||||
wakeWindow time.Duration
|
||||
wakeUntil time.Time
|
||||
|
||||
// pending holds a nudge the push receiver handed over, waiting for the
|
||||
// capture loop to speak it. It is the one field written from another
|
||||
// goroutine, hence the mutex; everything else in this struct belongs to
|
||||
// the capture loop alone.
|
||||
nudgeMu sync.Mutex
|
||||
pending *audio.Audio
|
||||
|
||||
// counters, read by tests and logged on the way out.
|
||||
suppressed int // frames dropped because she was speaking
|
||||
dropped int // frames dropped as round-trip backlog
|
||||
bargeIns int // times playback was cut because he spoke over her
|
||||
sent int // utterances shipped to the daemon
|
||||
nudges int // proactive pushes spoken through the speaker
|
||||
wakes int // times the keyword opened the gate
|
||||
ignored int // complete utterances dropped because the keyword was absent
|
||||
|
||||
// loudSum and loudSeen accumulate the energy of suppressed frames, so
|
||||
// the operator can read what the room actually measures and set
|
||||
@@ -79,6 +107,12 @@ func newSession(vad *VAD, p player, s utteranceSender, lang string, barge bargeI
|
||||
return &session{vad: vad, player: p, sender: s, lang: lang, barge: barge, now: time.Now}
|
||||
}
|
||||
|
||||
// UseWakeWord puts the keyword gate in front of dispatch. Without it every
|
||||
// utterance is shipped, which is what mavwaked did before V-487 stage two.
|
||||
func (s *session) UseWakeWord(w keywordGate, window time.Duration) {
|
||||
s.wake, s.wakeWindow = w, window
|
||||
}
|
||||
|
||||
// frameDuration is the wall time one captured frame represents.
|
||||
const frameDuration = defaultFrameMs * time.Millisecond
|
||||
|
||||
@@ -138,6 +172,7 @@ func (s *session) feed(ctx context.Context, frame []byte) error {
|
||||
s.bargeIns++
|
||||
s.loudFrames = 0
|
||||
s.vad.Reset()
|
||||
s.resetWake()
|
||||
log.Printf("mavwaked: barge-in — stopped playback")
|
||||
s.replayRecent()
|
||||
return nil
|
||||
@@ -148,15 +183,101 @@ func (s *session) feed(ctx context.Context, frame []byte) error {
|
||||
if s.loudFrames != 0 {
|
||||
s.loudFrames = 0
|
||||
s.vad.Reset()
|
||||
// The wake word saw nothing during playback, so what it holds is from
|
||||
// before she spoke. Judging what he says next on it would score a
|
||||
// sentence that ended a reply ago.
|
||||
s.resetWake()
|
||||
}
|
||||
|
||||
utt, state := s.vad.Feed(PCMToI16(frame))
|
||||
if s.startPendingNudge() {
|
||||
return nil
|
||||
}
|
||||
|
||||
// The keyword is scored on the same frames the VAD sees, and only on the
|
||||
// ones that reach here: every path above returns while she is speaking, so
|
||||
// her own voice saying "Мэйвен" cannot wake her.
|
||||
pcm := PCMToI16(frame)
|
||||
if s.wake != nil && s.wake.Feed(pcm) {
|
||||
s.wakes++
|
||||
s.wakeUntil = s.now().Add(s.wakeWindow)
|
||||
log.Printf("mavwaked: keyword heard (score %.3f), listening for %s",
|
||||
s.wake.Score(), s.wakeWindow)
|
||||
}
|
||||
|
||||
utt, state := s.vad.Feed(pcm)
|
||||
if state == StateSpeech || utt.Bytes == nil {
|
||||
return nil
|
||||
}
|
||||
return s.dispatch(ctx, utt)
|
||||
}
|
||||
|
||||
// Nudge hands proactive audio to the session, to be spoken as soon as the
|
||||
// capture loop finds a quiet moment. Safe to call from the push receiver
|
||||
// goroutine; nothing else here is.
|
||||
//
|
||||
// A nudge arriving while one is already waiting REPLACES it. That is the
|
||||
// contract internal/voice states for PushHandler: the next nudge replaces the
|
||||
// stale one in his attention rather than dogpiling on it.
|
||||
func (s *session) Nudge(a audio.Audio) {
|
||||
if len(a.Bytes) == 0 {
|
||||
return
|
||||
}
|
||||
s.nudgeMu.Lock()
|
||||
if s.pending != nil {
|
||||
log.Printf("mavwaked: nudge replaced one still waiting to be spoken")
|
||||
}
|
||||
s.pending = &a
|
||||
s.nudgeMu.Unlock()
|
||||
}
|
||||
|
||||
// takeNudge removes and returns the waiting nudge, or nil.
|
||||
func (s *session) takeNudge() *audio.Audio {
|
||||
s.nudgeMu.Lock()
|
||||
defer s.nudgeMu.Unlock()
|
||||
a := s.pending
|
||||
s.pending = nil
|
||||
return a
|
||||
}
|
||||
|
||||
// startPendingNudge speaks a waiting nudge and reports whether it started
|
||||
// one. It runs on the capture loop, past the half-duplex gate, so a nudge
|
||||
// never cuts across a reply and never plays into a backlog drain.
|
||||
//
|
||||
// The VAD is reset first. Playback is about to suppress every frame until it
|
||||
// ends, and a half-heard sentence left in the VAD would splice onto whatever
|
||||
// he says afterwards. Barge-in needs no special case: it reads the player,
|
||||
// and the player does not care which audio it is playing.
|
||||
func (s *session) startPendingNudge() bool {
|
||||
a := s.takeNudge()
|
||||
if a == nil {
|
||||
return false
|
||||
}
|
||||
s.vad.Reset()
|
||||
s.nudges++
|
||||
log.Printf("mavwaked: speaking nudge (%.2fs audio)", a.Duration())
|
||||
s.player.Play(*a)
|
||||
return true
|
||||
}
|
||||
|
||||
// awake reports whether an utterance ending now was addressed to her.
|
||||
//
|
||||
// With no wake word loaded every utterance is, which is exactly what mavwaked
|
||||
// did before this gate existed. An operator with no model file gets the old
|
||||
// daemon rather than a daemon that refuses to hear anything.
|
||||
func (s *session) awake() bool {
|
||||
if s.wake == nil {
|
||||
return true
|
||||
}
|
||||
return s.now().Before(s.wakeUntil)
|
||||
}
|
||||
|
||||
// resetWake drops the gate's streaming state when there is a gate.
|
||||
func (s *session) resetWake() {
|
||||
if s.wake != nil {
|
||||
s.wake.Reset()
|
||||
}
|
||||
}
|
||||
|
||||
// keepRecent stores a copy of one barge-in trigger frame, keeping at most
|
||||
// barge.Frames of them.
|
||||
func (s *session) keepRecent(frame []byte) {
|
||||
@@ -197,6 +318,19 @@ func (s *session) replayRecent() {
|
||||
// whole backlog straight into the VAD, and a Send error did the same on every
|
||||
// failed turn, so a dead socket drove a retry loop off nothing but backlog.
|
||||
func (s *session) dispatch(ctx context.Context, utt audio.Audio) error {
|
||||
if !s.awake() {
|
||||
s.ignored++
|
||||
log.Printf("mavwaked: utterance ignored, keyword not heard (%.2fs, %d ignored so far)",
|
||||
utt.Duration(), s.ignored)
|
||||
s.vad.Reset()
|
||||
s.resetWake()
|
||||
return nil
|
||||
}
|
||||
// One keyword, one turn. A window that renewed itself on every reply would
|
||||
// leave the microphone open for as long as he kept talking, which is the
|
||||
// state this gate exists to end.
|
||||
s.wakeUntil = time.Time{}
|
||||
|
||||
log.Printf("mavwaked: utterance complete (%.2fs, %d bytes), sending...", utt.Duration(), len(utt.Bytes))
|
||||
start := s.now()
|
||||
reply, err := s.sender.Send(ctx, utt, s.lang)
|
||||
@@ -225,6 +359,7 @@ func (s *session) dispatch(ctx context.Context, utt audio.Audio) error {
|
||||
// recorded before she started speaking.
|
||||
func (s *session) dropBacklog(start time.Time) {
|
||||
s.vad.Reset()
|
||||
s.resetWake()
|
||||
s.loudFrames = 0
|
||||
s.recent = s.recent[:0]
|
||||
if elapsed := s.now().Sub(start); elapsed > 0 {
|
||||
|
||||
@@ -0,0 +1,202 @@
|
||||
package main
|
||||
|
||||
// The three models behind the wake word (V-487 stage two).
|
||||
//
|
||||
// openWakeWord's pipeline, run in a row:
|
||||
//
|
||||
// audio -> melspectrogram.onnx -> 32-bin mel frames, one per 10ms
|
||||
// 76 frames -> embedding_model.onnx -> one 96-dim embedding per 80ms
|
||||
// 16 embeds -> maven_wakeword.onnx -> one score
|
||||
//
|
||||
// The first two are frozen and pretrained. Only the last was trained here,
|
||||
// which is why it is 100KB and the other two are megabytes. The shapes are
|
||||
// not guesses: 2.0s of 16kHz audio measures 197 mel frames, and 76-frame
|
||||
// windows at stride 8 give exactly the 16 embeddings the head was fitted on.
|
||||
//
|
||||
// This file knows ONNX and nothing about the 80ms cadence. wakeword.go knows
|
||||
// the cadence and nothing about tensors.
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
ort "github.com/yalue/onnxruntime_go"
|
||||
)
|
||||
|
||||
const (
|
||||
// melHop — samples per mel frame. 10ms at 16kHz.
|
||||
melHop = 160
|
||||
// melBins — mel bins per frame, fixed by melspectrogram.onnx.
|
||||
melBins = 32
|
||||
// embedFrames — mel frames one embedding is computed over, 760ms.
|
||||
embedFrames = 76
|
||||
// embedStride — mel frames between embeddings, 80ms.
|
||||
embedStride = 8
|
||||
// embedDim — the embedding width.
|
||||
embedDim = 96
|
||||
// headWindow — embeddings the head scores at once, 1.28s of audio.
|
||||
headWindow = 16
|
||||
|
||||
// melContext — samples of history prepended to each incremental mel
|
||||
// call, chosen so the eight frames this call yields continue exactly
|
||||
// where the previous call's eight stopped.
|
||||
//
|
||||
// melspectrogram.onnx returns N/160-3 frames for N samples, and frame i
|
||||
// covers [i*160, i*160+400). With 480 samples of history the buffer is
|
||||
// 1760 samples, which is 8 frames, and the oldest of them starts one hop
|
||||
// after the newest of the previous call. Less history leaves a gap: the
|
||||
// first frames of a bare chunk would be computed against silence.
|
||||
melContext = 480
|
||||
|
||||
// chunkSamples — audio per embedding step, 80ms.
|
||||
chunkSamples = embedStride * melHop
|
||||
)
|
||||
|
||||
// wakeModels holds the three ONNX sessions. It runs on CPU threads beside
|
||||
// silero and never touches the GPU. That is a rule, not a result: a wake word
|
||||
// that waits on card admission is not a wake word.
|
||||
type wakeModels struct {
|
||||
mel *ort.DynamicAdvancedSession
|
||||
emb *ort.DynamicAdvancedSession
|
||||
head *ort.DynamicAdvancedSession
|
||||
}
|
||||
|
||||
// newWakeModels loads all three. melPath and embedPath are openWakeWord's
|
||||
// frozen feature models; headPath is the keyword head trained for "Мэйвен".
|
||||
func newWakeModels(melPath, embedPath, headPath, libPath string) (*wakeModels, error) {
|
||||
if !ort.IsInitialized() {
|
||||
if libPath != "" {
|
||||
ort.SetSharedLibraryPath(libPath)
|
||||
}
|
||||
if err := ort.InitializeEnvironment(); err != nil {
|
||||
return nil, fmt.Errorf("wake word: onnx runtime: %w", err)
|
||||
}
|
||||
}
|
||||
// One thread per session, not the default of every core. Measured on
|
||||
// workpc: the default took mavwaked from 68% of one core to 335% of
|
||||
// three, for three graphs that each run in well under 80ms single
|
||||
// threaded. An always-on gate that eats a quarter of the workstation is
|
||||
// not a gate he will leave running.
|
||||
opts, err := ort.NewSessionOptions()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("wake word: session options: %w", err)
|
||||
}
|
||||
defer opts.Destroy()
|
||||
if err := opts.SetIntraOpNumThreads(1); err != nil {
|
||||
return nil, fmt.Errorf("wake word: intra-op threads: %w", err)
|
||||
}
|
||||
if err := opts.SetInterOpNumThreads(1); err != nil {
|
||||
return nil, fmt.Errorf("wake word: inter-op threads: %w", err)
|
||||
}
|
||||
open := func(p string, in, out []string) (*ort.DynamicAdvancedSession, error) {
|
||||
s, err := ort.NewDynamicAdvancedSession(p, in, out, opts)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("wake word: load %s: %w", p, err)
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
m := &wakeModels{}
|
||||
if m.mel, err = open(melPath, []string{"input"}, []string{"output"}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if m.emb, err = open(embedPath, []string{"input_1"}, []string{"conv2d_19"}); err != nil {
|
||||
m.Close()
|
||||
return nil, err
|
||||
}
|
||||
if m.head, err = open(headPath, []string{"embeddings"}, []string{"score"}); err != nil {
|
||||
m.Close()
|
||||
return nil, err
|
||||
}
|
||||
return m, nil
|
||||
}
|
||||
|
||||
// Close releases the three sessions.
|
||||
func (m *wakeModels) Close() {
|
||||
if m == nil {
|
||||
return
|
||||
}
|
||||
for _, s := range []*ort.DynamicAdvancedSession{m.mel, m.emb, m.head} {
|
||||
if s != nil {
|
||||
s.Destroy()
|
||||
}
|
||||
}
|
||||
m.mel, m.emb, m.head = nil, nil, nil
|
||||
}
|
||||
|
||||
// melFrames runs one buffer of samples and returns the mel frames it yielded.
|
||||
func (m *wakeModels) melFrames(buf []float32) ([][melBins]float32, error) {
|
||||
in, err := ort.NewTensor(ort.NewShape(1, int64(len(buf))), buf)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer in.Destroy()
|
||||
|
||||
n := int64(len(buf)/melHop - 3)
|
||||
if n < 1 {
|
||||
return nil, fmt.Errorf("wake word: %d samples yield no mel frames", len(buf))
|
||||
}
|
||||
out, err := ort.NewEmptyTensor[float32](ort.NewShape(1, 1, n, melBins))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer out.Destroy()
|
||||
|
||||
if err := m.mel.Run([]ort.Value{in}, []ort.Value{out}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
data := out.GetData()
|
||||
frames := make([][melBins]float32, n)
|
||||
for i := range frames {
|
||||
for j := 0; j < melBins; j++ {
|
||||
// The scaling openWakeWord applies between the two feature
|
||||
// models, and the head was fitted on its output.
|
||||
frames[i][j] = data[i*melBins+j]/10.0 + 2.0
|
||||
}
|
||||
}
|
||||
return frames, nil
|
||||
}
|
||||
|
||||
// embedding runs embedFrames mel frames through the frozen embedder.
|
||||
func (m *wakeModels) embedding(mels [][melBins]float32) ([embedDim]float32, error) {
|
||||
var e [embedDim]float32
|
||||
flat := make([]float32, 0, embedFrames*melBins)
|
||||
for _, f := range mels {
|
||||
flat = append(flat, f[:]...)
|
||||
}
|
||||
in, err := ort.NewTensor(ort.NewShape(1, embedFrames, melBins, 1), flat)
|
||||
if err != nil {
|
||||
return e, err
|
||||
}
|
||||
defer in.Destroy()
|
||||
out, err := ort.NewEmptyTensor[float32](ort.NewShape(1, 1, 1, embedDim))
|
||||
if err != nil {
|
||||
return e, err
|
||||
}
|
||||
defer out.Destroy()
|
||||
if err := m.emb.Run([]ort.Value{in}, []ort.Value{out}); err != nil {
|
||||
return e, err
|
||||
}
|
||||
copy(e[:], out.GetData())
|
||||
return e, nil
|
||||
}
|
||||
|
||||
// score runs the trained head over headWindow embeddings.
|
||||
func (m *wakeModels) score(embeds [][embedDim]float32) (float64, error) {
|
||||
flat := make([]float32, 0, headWindow*embedDim)
|
||||
for _, e := range embeds {
|
||||
flat = append(flat, e[:]...)
|
||||
}
|
||||
in, err := ort.NewTensor(ort.NewShape(1, headWindow, embedDim), flat)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
defer in.Destroy()
|
||||
out, err := ort.NewEmptyTensor[float32](ort.NewShape(1, 1))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
defer out.Destroy()
|
||||
if err := m.head.Run([]ort.Value{in}, []ort.Value{out}); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return float64(out.GetData()[0]), nil
|
||||
}
|
||||
@@ -0,0 +1,195 @@
|
||||
package main
|
||||
|
||||
// The wake word, "Мэйвен" (V-487 stage two).
|
||||
//
|
||||
// Silero answers "is this frame speech". It does not answer "was this said to
|
||||
// her", and until this file existed nothing did: every utterance near the
|
||||
// microphone became a turn. What made that safe rather than expensive was
|
||||
// SurfaceVoice capping acts at L0, and L0 does not cap reading, so the room
|
||||
// could still hear his facts read back.
|
||||
//
|
||||
// This file owns the 80ms cadence and the three rings of state between the
|
||||
// models. wakefeatures.go owns the tensors.
|
||||
//
|
||||
// Nil is a working value, and it is the CLOSED gate rather than the open one.
|
||||
// Feed on a nil receiver reports no keyword; session.go asks separately
|
||||
// whether a gate exists at all. That split is deliberate: a nil that answers
|
||||
// "yes, keyword" reads as a working wake word in every log line it produces.
|
||||
|
||||
import (
|
||||
"log"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// defaultWakeThreshold — score above which the keyword was said.
|
||||
//
|
||||
// Picked from the false-accept rate on held-out Russian speech, not from
|
||||
// accuracy: a miss costs him a repeat, a false accept costs a turn nobody
|
||||
// asked for. Over 65 minutes of Common Voice, 0.99 woke her three times and
|
||||
// 0.999 once, and the difference in recall was one render out of 126. So the
|
||||
// default is the strict one. `docs/evals/2026-08-09-wake-word.md` has both
|
||||
// tables.
|
||||
const defaultWakeThreshold = 0.999
|
||||
|
||||
// wakeWord is the streaming state around wakeModels. It is fed the same
|
||||
// capture frames the VAD sees and answers whether the keyword has just been
|
||||
// spoken.
|
||||
type wakeWord struct {
|
||||
mu sync.Mutex
|
||||
m *wakeModels
|
||||
|
||||
threshold float64
|
||||
|
||||
// pending holds captured samples not yet part of a full 80ms chunk, and
|
||||
// history holds the melContext samples before them.
|
||||
pending []float32
|
||||
history []float32
|
||||
|
||||
// mels is the newest embedFrames mel frames, oldest first.
|
||||
mels [][melBins]float32
|
||||
// embeds is the newest headWindow embeddings, oldest first.
|
||||
embeds [][embedDim]float32
|
||||
|
||||
last float64 // most recent score, held between chunks
|
||||
}
|
||||
|
||||
// newWakeWord loads the models and wraps them in the streaming gate.
|
||||
func newWakeWord(melPath, embedPath, headPath, libPath string, threshold float64) (*wakeWord, error) {
|
||||
m, err := newWakeModels(melPath, embedPath, headPath, libPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if threshold <= 0 {
|
||||
threshold = defaultWakeThreshold
|
||||
}
|
||||
return &wakeWord{m: m, threshold: threshold}, nil
|
||||
}
|
||||
|
||||
// Close releases the models.
|
||||
func (w *wakeWord) Close() {
|
||||
if w == nil {
|
||||
return
|
||||
}
|
||||
w.mu.Lock()
|
||||
defer w.mu.Unlock()
|
||||
w.m.Close()
|
||||
w.m = nil
|
||||
}
|
||||
|
||||
// Feed takes one capture frame and reports whether the keyword was heard on
|
||||
// it. A nil wakeWord hears nothing.
|
||||
func (w *wakeWord) Feed(frame []int16) bool {
|
||||
if w == nil {
|
||||
return false
|
||||
}
|
||||
w.mu.Lock()
|
||||
defer w.mu.Unlock()
|
||||
|
||||
for _, v := range frame {
|
||||
w.pending = append(w.pending, float32(v)/32768.0)
|
||||
}
|
||||
fired := false
|
||||
for len(w.pending) >= chunkSamples {
|
||||
chunk := w.pending[:chunkSamples]
|
||||
if w.step(chunk) {
|
||||
fired = true
|
||||
}
|
||||
w.history = append(w.history[:0], tailFloat32(append(w.history, chunk...), melContext)...)
|
||||
// Slide the remainder to the front rather than reslicing. This runs
|
||||
// every 80ms for as long as the daemon lives.
|
||||
w.pending = append(w.pending[:0], w.pending[chunkSamples:]...)
|
||||
}
|
||||
return fired
|
||||
}
|
||||
|
||||
// Reset drops the streaming state, so a fresh utterance is not judged on audio
|
||||
// from before it. Called after every dispatch and after barge-in, for the same
|
||||
// reason silero is: echo-era history must not score the next sentence, and her
|
||||
// own voice saying the keyword must not wake her.
|
||||
func (w *wakeWord) Reset() {
|
||||
if w == nil {
|
||||
return
|
||||
}
|
||||
w.mu.Lock()
|
||||
defer w.mu.Unlock()
|
||||
w.pending, w.history = w.pending[:0], w.history[:0]
|
||||
w.mels, w.embeds = nil, nil
|
||||
w.last = 0
|
||||
}
|
||||
|
||||
// Score returns the most recent score, for the operator to read out of the
|
||||
// journal when picking a threshold for his room.
|
||||
func (w *wakeWord) Score() float64 {
|
||||
if w == nil {
|
||||
return 0
|
||||
}
|
||||
w.mu.Lock()
|
||||
defer w.mu.Unlock()
|
||||
return w.last
|
||||
}
|
||||
|
||||
// step runs one 80ms chunk through all three models. It returns true when the
|
||||
// score crosses the threshold on this chunk.
|
||||
func (w *wakeWord) step(chunk []float32) bool {
|
||||
buf := make([]float32, 0, melContext+len(chunk))
|
||||
if pad := melContext - len(w.history); pad > 0 {
|
||||
buf = append(buf, make([]float32, pad)...)
|
||||
}
|
||||
buf = append(buf, tailFloat32(w.history, melContext)...)
|
||||
buf = append(buf, chunk...)
|
||||
|
||||
frames, err := w.m.melFrames(buf)
|
||||
if err != nil {
|
||||
// A failed inference must not silence the microphone. Hold the last
|
||||
// score and let the next chunk try again.
|
||||
log.Printf("mavwaked: wake word: mel: %v", err)
|
||||
return false
|
||||
}
|
||||
w.mels = tailMel(append(w.mels, frames...), embedFrames)
|
||||
if len(w.mels) < embedFrames {
|
||||
return false
|
||||
}
|
||||
e, err := w.m.embedding(w.mels)
|
||||
if err != nil {
|
||||
log.Printf("mavwaked: wake word: embedding: %v", err)
|
||||
return false
|
||||
}
|
||||
w.embeds = tailEmbed(append(w.embeds, e), headWindow)
|
||||
if len(w.embeds) < headWindow {
|
||||
return false
|
||||
}
|
||||
score, err := w.m.score(w.embeds)
|
||||
if err != nil {
|
||||
log.Printf("mavwaked: wake word: head: %v", err)
|
||||
return false
|
||||
}
|
||||
// Report the crossing, not the state. A keyword held above the threshold
|
||||
// for a second is one wake, and firing on every chunk of it would make the
|
||||
// gate look open when it is merely slow to fall.
|
||||
crossed := score >= w.threshold && w.last < w.threshold
|
||||
w.last = score
|
||||
return crossed
|
||||
}
|
||||
|
||||
// The three rings. Each keeps the newest n entries and nothing older.
|
||||
|
||||
func tailFloat32(s []float32, n int) []float32 {
|
||||
if len(s) <= n {
|
||||
return s
|
||||
}
|
||||
return s[len(s)-n:]
|
||||
}
|
||||
|
||||
func tailMel(s [][melBins]float32, n int) [][melBins]float32 {
|
||||
if len(s) <= n {
|
||||
return s
|
||||
}
|
||||
return append(s[:0], s[len(s)-n:]...)
|
||||
}
|
||||
|
||||
func tailEmbed(s [][embedDim]float32, n int) [][embedDim]float32 {
|
||||
if len(s) <= n {
|
||||
return s
|
||||
}
|
||||
return append(s[:0], s[len(s)-n:]...)
|
||||
}
|
||||
@@ -0,0 +1,174 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// fakeGate fires on demand instead of running three ONNX models. The gate's
|
||||
// own arithmetic is measured on real audio in docs/evals; what these tests
|
||||
// cover is the thing that decides whether an utterance is shipped.
|
||||
type fakeGate struct {
|
||||
fireOn int // fire when this many frames have been fed, 0 never fires
|
||||
fed int
|
||||
resets int
|
||||
}
|
||||
|
||||
func (g *fakeGate) Feed(_ []int16) bool {
|
||||
g.fed++
|
||||
return g.fireOn > 0 && g.fed == g.fireOn
|
||||
}
|
||||
func (g *fakeGate) Reset() { g.resets++ }
|
||||
func (g *fakeGate) Score() float64 { return 1 }
|
||||
|
||||
// wakingSession wires a session whose gate fires on the first frame it sees.
|
||||
func wakingSession(fireOn int, window time.Duration) (*session, *fakePlayer, *fakeSender, *fakeGate) {
|
||||
sess, p, snd := newTestSession(bargeInConfig{})
|
||||
g := &fakeGate{fireOn: fireOn}
|
||||
sess.UseWakeWord(g, window)
|
||||
return sess, p, snd, g
|
||||
}
|
||||
|
||||
func TestKeywordlessSpeechNeverReachesSTT(t *testing.T) {
|
||||
sess, p, snd, g := wakingSession(0, 8*time.Second)
|
||||
speakThenPause(t, sess)
|
||||
|
||||
if len(snd.sent) != 0 {
|
||||
t.Fatalf("sent %d utterances, want 0 — this is the whole point of V-487", len(snd.sent))
|
||||
}
|
||||
if sess.ignored != 1 {
|
||||
t.Errorf("ignored = %d, want 1", sess.ignored)
|
||||
}
|
||||
if p.plays != 0 {
|
||||
t.Errorf("plays = %d, want 0", p.plays)
|
||||
}
|
||||
if g.fed == 0 {
|
||||
t.Error("the gate was never fed a frame")
|
||||
}
|
||||
}
|
||||
|
||||
func TestKeywordOpensTheGate(t *testing.T) {
|
||||
sess, p, snd, _ := wakingSession(1, 8*time.Second)
|
||||
speakThenPause(t, sess)
|
||||
|
||||
if len(snd.sent) != 1 {
|
||||
t.Fatalf("sent %d utterances, want 1", len(snd.sent))
|
||||
}
|
||||
if sess.wakes != 1 {
|
||||
t.Errorf("wakes = %d, want 1", sess.wakes)
|
||||
}
|
||||
if sess.ignored != 0 {
|
||||
t.Errorf("ignored = %d, want 0", sess.ignored)
|
||||
}
|
||||
if p.plays != 1 {
|
||||
t.Errorf("plays = %d, want 1", p.plays)
|
||||
}
|
||||
}
|
||||
|
||||
// One keyword buys one turn. Without this the microphone stays open for as
|
||||
// long as he keeps talking, which is the state the gate exists to end.
|
||||
func TestOneKeywordBuysOneTurn(t *testing.T) {
|
||||
sess, p, snd, _ := wakingSession(1, 8*time.Second)
|
||||
speakThenPause(t, sess)
|
||||
p.Stop() // she finished her reply
|
||||
sess.discard = 0 // the backlog drain is not what this measures
|
||||
speakThenPause(t, sess)
|
||||
|
||||
if len(snd.sent) != 1 {
|
||||
t.Fatalf("sent %d utterances, want 1: the second had no keyword", len(snd.sent))
|
||||
}
|
||||
if sess.ignored != 1 {
|
||||
t.Errorf("ignored = %d, want 1", sess.ignored)
|
||||
}
|
||||
}
|
||||
|
||||
// The keyword is heard, then he says nothing for longer than the window. What
|
||||
// he says after that is not addressed to her.
|
||||
func TestTheKeywordExpires(t *testing.T) {
|
||||
sess, _, snd, _ := wakingSession(1, 500*time.Millisecond)
|
||||
now := time.Unix(1750000000, 0)
|
||||
sess.now = func() time.Time { return now }
|
||||
|
||||
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
if sess.wakes != 1 {
|
||||
t.Fatalf("wakes = %d, want 1", sess.wakes)
|
||||
}
|
||||
now = now.Add(2 * time.Second)
|
||||
speakThenPause(t, sess)
|
||||
|
||||
if len(snd.sent) != 0 {
|
||||
t.Fatalf("sent %d utterances, want 0 — the keyword had expired", len(snd.sent))
|
||||
}
|
||||
}
|
||||
|
||||
// Barge-in cuts her off whether or not the keyword was heard. What he says
|
||||
// after cutting her off still has to carry it.
|
||||
func TestBargeInStillInterruptsHer(t *testing.T) {
|
||||
sess, p, _, g := wakingSession(0, 8*time.Second)
|
||||
sess.barge = bargeInConfig{RMS: 0.2, Frames: 3}
|
||||
p.playing = true
|
||||
loud := frameAt(0.35)
|
||||
for i := 0; i < 4; i++ {
|
||||
if err := sess.feed(context.Background(), loud); err != nil {
|
||||
t.Fatalf("feed %d: %v", i, err)
|
||||
}
|
||||
}
|
||||
if sess.bargeIns != 1 {
|
||||
t.Fatalf("bargeIns = %d, want 1", sess.bargeIns)
|
||||
}
|
||||
if p.stops != 1 {
|
||||
t.Errorf("stops = %d, want 1", p.stops)
|
||||
}
|
||||
if g.resets == 0 {
|
||||
t.Error("barge-in left pre-playback audio in the gate")
|
||||
}
|
||||
}
|
||||
|
||||
// Her own reply must not wake her. Frames captured while the player runs never
|
||||
// reach the gate, and the gate is cleared when playback ends.
|
||||
func TestHerOwnVoiceNeverReachesTheGate(t *testing.T) {
|
||||
sess, p, _, g := wakingSession(1, 8*time.Second)
|
||||
p.playing = true
|
||||
for i := 0; i < 10; i++ {
|
||||
if err := sess.feed(context.Background(), frameAt(0.35)); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
}
|
||||
if g.fed != 0 {
|
||||
t.Fatalf("gate was fed %d frames while she was speaking, want 0", g.fed)
|
||||
}
|
||||
if sess.wakes != 0 {
|
||||
t.Errorf("wakes = %d, want 0", sess.wakes)
|
||||
}
|
||||
}
|
||||
|
||||
// No model, no gate: the daemon behaves exactly as it did before V-487 stage
|
||||
// two. An operator with a missing file gets yesterday's mavwaked, not one that
|
||||
// refuses to hear anything.
|
||||
func TestNoGateShipsEveryUtterance(t *testing.T) {
|
||||
sess, _, snd := newTestSession(bargeInConfig{})
|
||||
speakThenPause(t, sess)
|
||||
|
||||
if len(snd.sent) != 1 {
|
||||
t.Fatalf("sent %d utterances, want 1", len(snd.sent))
|
||||
}
|
||||
if sess.ignored != 0 {
|
||||
t.Errorf("ignored = %d, want 0", sess.ignored)
|
||||
}
|
||||
}
|
||||
|
||||
// A nil *wakeWord is the closed gate, not a crash and not an open one.
|
||||
func TestNilWakeWordHearsNothing(t *testing.T) {
|
||||
var w *wakeWord
|
||||
if w.Feed([]int16{0, 0, 0}) {
|
||||
t.Error("a nil wake word reported the keyword")
|
||||
}
|
||||
if w.Score() != 0 {
|
||||
t.Error("a nil wake word reported a score")
|
||||
}
|
||||
w.Reset()
|
||||
w.Close()
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
# maven-voice-tunnel — the ssh leg that carries the voice wire to homesrv.
|
||||
#
|
||||
# Runs on workpc, as a user unit (`systemctl --user`), beside mavgpud.service.
|
||||
#
|
||||
# WHY THIS EXISTS AT ALL. internal/voice is plaintext and unauthenticated.
|
||||
# Its own server doc says production binds inside the wg tunnel, because "the
|
||||
# wg layer IS the L0 floor". workpc is not a wg peer, it sits on wlan0. So ssh
|
||||
# is the substitute floor: it authenticates with his key and encrypts the leg,
|
||||
# and mavend's published port stays on homesrv loopback (127.0.0.1:9110).
|
||||
# Nothing about this puts a Maven port on the LAN.
|
||||
#
|
||||
# Do not replace this with a LAN bind. SurfaceVoice caps acts at L0, so an
|
||||
# unauthorized speaker could not run a destructive tool. It would still hear
|
||||
# his facts, his notes and his calendar read back, and L0 does not cap reading.
|
||||
#
|
||||
# install: cp to ~/.config/systemd/user/ on workpc
|
||||
# systemctl --user enable --now maven-voice-tunnel.service
|
||||
|
||||
[Unit]
|
||||
Description=SSH tunnel to mavend's voice wire on homesrv
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
# -N: no remote command, forwarding only.
|
||||
# ExitOnForwardFailure: fail loudly rather than sit up with a dead forward,
|
||||
# which is what makes Restart meaningful.
|
||||
# ServerAlive*: a laptop that suspends drops the tunnel silently otherwise.
|
||||
ExecStart=/usr/bin/ssh -N \
|
||||
-o ExitOnForwardFailure=yes \
|
||||
-o ServerAliveInterval=30 \
|
||||
-o ServerAliveCountMax=3 \
|
||||
-o BatchMode=yes \
|
||||
-L 127.0.0.1:9100:127.0.0.1:9110 \
|
||||
kami@192.168.1.104
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -94,6 +94,7 @@
|
||||
],
|
||||
"workstation": {
|
||||
"url": "http://192.168.1.105:8080",
|
||||
"token": "${MAVEN_GPU_TOKEN}",
|
||||
"probe": "15s",
|
||||
"timeout": "90s",
|
||||
"stt": {
|
||||
|
||||
@@ -1,5 +1,14 @@
|
||||
{
|
||||
"listen": ":8080",
|
||||
"//token_file": [
|
||||
"The bearer token every request must carry. homesrv is the client and it",
|
||||
"is on the LAN, so this port cannot be loopback and the token is what",
|
||||
"stops anything else on the network spending the card or reading /slots.",
|
||||
"A path, never the token: mavgpud refuses to start when listen is",
|
||||
"reachable from the network and this is unset.",
|
||||
"Same value as MAVEN_GPU_TOKEN in homesrv's deploy/telegram.env."
|
||||
],
|
||||
"token_file": "/home/kami/.config/mavgpud.token",
|
||||
"llama_addr": "127.0.0.1:10000",
|
||||
"llama_bin": "llama-server",
|
||||
"//llama_args": [
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
# mavwaked — always-on listening, on workpc where the microphone is.
|
||||
#
|
||||
# User unit, beside mavgpud.service and maven-voice-tunnel.service. It is a
|
||||
# user unit because it needs his ALSA session and his ssh agent, and because
|
||||
# it should stop when he logs out.
|
||||
#
|
||||
# The keyword is "Мэйвен" and the three -wake- flags are what require it
|
||||
# (V-487 stage two). Without them anything spoken near the fifine becomes a
|
||||
# turn, which voiceSender makes safe rather than expensive: it sends
|
||||
# Surface=SurfaceVoice, capping every command at L0. That does not stop her
|
||||
# answering out loud, which is the whole reason the keyword exists.
|
||||
#
|
||||
# The threshold is 0.999 and it is the binary's default, so it is not passed.
|
||||
# It came from 65 minutes of held-out Russian speech through this same binary:
|
||||
# 0.9 false wakes an hour against 2.8 at 0.99, for one lost render out of 126
|
||||
# (docs/evals/2026-08-09-wake-word.md). If the room proves noisier than the
|
||||
# corpus, read the scores out of this unit's journal and pass -wake-threshold.
|
||||
# Do not lower it by guessing.
|
||||
#
|
||||
# A keyword shorter than 1.32s can be heard too late to be used, because the
|
||||
# head scores 1.28s of audio and the VAD has closed the utterance by then.
|
||||
# "Мэйвен, <request>" is unaffected. A bare "Мэйвен" is the case that fails.
|
||||
#
|
||||
# -vad-model is passed on purpose. Silero answers "is this frame speech" where
|
||||
# the energy floor answers "is this frame loud". It declines white noise at
|
||||
# the same RMS 0 frames to 68-99, and still hears all four spoken fixtures
|
||||
# (docs/evals/2026-08-09-silero-vad.md). It costs 509us a frame, 1.7% of one
|
||||
# core, and never touches the GPU. Drop the flag and the energy floor is back.
|
||||
#
|
||||
# -barge-in is NOT passed. The threshold is room-specific and this room has no
|
||||
# number yet. Turn it on only after reading the "suppressed while speaking"
|
||||
# means out of this unit's own journal, never by guessing.
|
||||
#
|
||||
# install: cp to ~/.config/systemd/user/ on workpc
|
||||
# systemctl --user enable --now mavwaked.service
|
||||
|
||||
[Unit]
|
||||
Description=Maven always-on listening (silero VAD, "Мэйвен" keyword)
|
||||
# The tunnel is the only path to mavend and the only thing authenticating it.
|
||||
Requires=maven-voice-tunnel.service
|
||||
After=maven-voice-tunnel.service
|
||||
|
||||
[Service]
|
||||
# The Scarlett Solo 4th Gen, and not the fifine. The fifine was the device
|
||||
# here for three days and mavwaked never logged one utterance in them, because
|
||||
# it returns RMS 0.00004 with its capture switch on and its ALSA volume at the
|
||||
# full 496 of 496. That silence is in the hardware, so no flag reaches it.
|
||||
#
|
||||
# Named CARD=Gen and not card 4, because a USB card number moves when
|
||||
# something else is replugged and this daemon must not change ears quietly.
|
||||
# Not "default" either: that follows whatever pipewire last decided.
|
||||
#
|
||||
# plughw and not hw. mavwaked asks arecord for 16kHz mono, which is what the
|
||||
# whole pipeline is canonical in. Neither microphone offers it, so bare hw
|
||||
# dies on "Channels count non available" before a frame is read. plughw puts
|
||||
# ALSA's downmix and resampler in front. Any replacement wants the same.
|
||||
#
|
||||
# The Scarlett measured RMS 0.003 against 0.14 on the onboard input, so its
|
||||
# front-panel gain is the thing to raise if she mishears. That is a knob, not
|
||||
# a control ALSA exposes. The two loud devices, the onboard ALC897 and the
|
||||
# camera, both clip at peak 1.0 and are worse candidates, not better ones.
|
||||
Environment=LD_LIBRARY_PATH=%h/.local/lib
|
||||
ExecStart=%h/.local/bin/mavwaked \
|
||||
-device plughw:CARD=Gen,DEV=0 \
|
||||
-addr 127.0.0.1:9100 \
|
||||
-lang ru \
|
||||
-vad-model %h/.local/share/maven/models/silero_vad.onnx \
|
||||
-wake-model %h/.local/share/maven/models/maven_wakeword.onnx \
|
||||
-wake-mel %h/.local/share/maven/models/melspectrogram.onnx \
|
||||
-wake-embed %h/.local/share/maven/models/embedding_model.onnx \
|
||||
-onnx-lib %h/.local/lib/libonnxruntime.so
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -10,3 +10,12 @@ TELEGRAM_CHAT_ID=
|
||||
# ntfy token add --expires=never maven
|
||||
# Read access is not needed — mavend publishes and never subscribes.
|
||||
NTFY_TOKEN=
|
||||
|
||||
# Bearer token for mavgpud, the workstation's GPU supervisor (V-673). It fronts
|
||||
# the big model on a LAN port, so the token is the whole boundary in front of
|
||||
# the card. Any long random string; mint one with:
|
||||
# openssl rand -hex 32
|
||||
# The same value goes in a file on workpc, named by token_file in
|
||||
# deploy/mavgpud.json. Unset here and every workstation turn falls back to the
|
||||
# resident model, because mavgpud answers 401 and Maven reads that as down.
|
||||
MAVEN_GPU_TOKEN=
|
||||
|
||||
@@ -53,6 +53,19 @@ services:
|
||||
# the decrypted working copy lives in RAM (see db_tmpfs in mavend.json).
|
||||
tmpfs:
|
||||
- /dev/shm
|
||||
# the voice wire, for mavwaked and mavenclient on workpc (V-515).
|
||||
#
|
||||
# LOOPBACK ONLY, and that is the whole security argument. internal/voice
|
||||
# is plaintext with no auth: its own server doc says production binds
|
||||
# inside the wg tunnel, "the wg layer IS the L0 floor". workpc is not a wg
|
||||
# peer, it is on wlan0. So the tunnel is ssh instead, terminated on this
|
||||
# loopback address, and nothing new is on the LAN. Anyone who could reach
|
||||
# a LAN-bound port here could push audio and hear his facts read back.
|
||||
# SurfaceVoice caps acts at L0; it does not cap reading.
|
||||
#
|
||||
# Host 9100 is Vikunja's MCP, hence 9110. The container side stays 9100
|
||||
# so mavweb keeps reaching mavend:9100 by name.
|
||||
ports: ["127.0.0.1:9110:9100"]
|
||||
|
||||
mavsttd:
|
||||
<<: *image
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
# docs/
|
||||
|
||||
Everything an agent needs that is not a rule and not code. `CLAUDE.md` at the
|
||||
root carries the rules and points here. Nothing here restates a rule.
|
||||
|
||||
The tier is the path, so staleness is visible from the filename.
|
||||
|
||||
| path | holds | lifetime |
|
||||
| --- | --- | --- |
|
||||
| `docs/*.md` | living. One file per subsystem: the reasoning, corrected in place. Each carries `Last verified: <date> @ <sha>`. | until it is wrong |
|
||||
| `docs/evals/` | dated measurements, one file per measurement. **Never edited after the day.** A newer number is a new file. | forever |
|
||||
| `docs/caveats/` | known limits, one entry per limit, each with a task id and a revisit trigger. Indexed in `docs/caveats/CLAUDE.md`. | until fixed, then deleted |
|
||||
| `docs/plans/` | the plan for one piece of work, frozen once it starts | until the work lands |
|
||||
| `docs/archive/` | dead. Read by nobody by default. | forever |
|
||||
|
||||
## Rules for this directory
|
||||
|
||||
* One fact, one home. A measurement is cited from a living doc, never copied
|
||||
into it. The two drift the moment they are both edited.
|
||||
* A living doc is corrected in place and its `Last verified` line moves with the
|
||||
correction. Do not append a changelog to it.
|
||||
* A number in prose with no `docs/evals/` file behind it is an opinion.
|
||||
* Fixing something deletes its caveat. It does not edit the eval that found it.
|
||||
|
||||
## Where a subsystem's reasoning lives
|
||||
|
||||
`routing.md`, `language.md`, `world.md`, `offload.md`, `deployment.md`,
|
||||
`ecosystem.md`, `workflow.md`, `design.md`, `rearchitecture.md`,
|
||||
`determinism.md`, `protocol.md`, `handler-wiring.md`, `operations.md`, `qa.md`.
|
||||
The root `CLAUDE.md` says which one to read before touching what.
|
||||
@@ -0,0 +1,50 @@
|
||||
# docs/caveats/
|
||||
|
||||
One entry per known limit: something broken, deferred or unsafe that a session
|
||||
will otherwise walk into. An entry names what fails, who it costs, and the
|
||||
condition that makes it worth fixing.
|
||||
|
||||
Two things do not belong here. The evidence is a dated file under `docs/evals/`.
|
||||
The reasoning behind a subsystem is its living doc directly under `docs/`. A
|
||||
caveat is the pointer between them plus the trigger.
|
||||
|
||||
## Rules for this directory
|
||||
|
||||
* One file per area, one `##` section per limit, each carrying its task id.
|
||||
* **A caveat with no revisit trigger is a complaint.** Give it one or delete it.
|
||||
* Closing a limit deletes its entry. It does not edit it to say "fixed", and it
|
||||
never edits the frozen measurement it came from. The durable record of a fix
|
||||
is the commit and the subsystem's living doc.
|
||||
* An entry whose task is closed but whose limit is still live is the failure
|
||||
mode to watch for. The id joins the two directions, so check both.
|
||||
|
||||
## Index
|
||||
|
||||
Every entry below came from the 2026-08-10 deep audit
|
||||
(`docs/evals/2026-08-10-repo-audit.md`), except the last, which came from wiring
|
||||
the gate the audit asked for. Five of the twenty findings are fixed and have no
|
||||
entry. The unauthenticated mavgpud proxy was V-673. The 20 reachable advisories
|
||||
in the toolchain and `x/text` were V-682. The missing analyzers were V-694, and
|
||||
what they now report is the baseline entry under V-701. The two unguarded
|
||||
invariants were V-692 and V-693, and their guards are described in
|
||||
`docs/routing.md`.
|
||||
|
||||
| limit | severity |
|
||||
| --- | --- |
|
||||
| [Anyone past the proxy can enroll a passkey](security.md#enrollment) | high |
|
||||
| [Passkey credentials are rewritten in place](security.md#credentials) | medium |
|
||||
| [An empty STT transcript reads as a successful one](external-inputs.md#stt) | medium |
|
||||
| [Open-Meteo's empty body becomes 0°C](external-inputs.md#weather) | medium |
|
||||
| [Dialogue persistence errors are swallowed](storage.md#dialogue) | medium |
|
||||
| [The reminder transition is a lost update](storage.md#reminders) | medium |
|
||||
| [A recall miss scans two whole tables](storage.md#recall) | medium |
|
||||
| [Closing a TCP listener can strand Accept](transport.md#accept) | medium |
|
||||
| [PTT reads an unbounded body](transport.md#ptt) | medium |
|
||||
| [mavweb errors cannot be traced](transport.md#errors) | medium |
|
||||
| [Fact enrichment is a 20-call serial waterfall](workers.md#enrichment) | medium |
|
||||
| [A suppressed nudge is phrased anyway](workers.md#nudges) | medium |
|
||||
| [Committed absolute paths pin the build to this box](config.md#paths) | medium |
|
||||
| [The env example omits deployed variables](config.md#secrets) | medium |
|
||||
| [The analyzers pass against a baseline, not zero](dependencies.md#baseline) | medium |
|
||||
| [Domain packages depend on store and IPC types](layering.md#dtos) | low |
|
||||
| [Eleven symbols are unreachable](layering.md#deadcode) | low |
|
||||
@@ -0,0 +1,21 @@
|
||||
# Configuration and environment
|
||||
|
||||
## Committed absolute paths pin the build to this box [#690] {#paths}
|
||||
|
||||
Costs: `go.mod` replaces Hexis with `/home/kami/apps/hexis`, `start-maven.sh`
|
||||
hardcodes the checkout and the data directory, and `deploy/mavgpud.json` holds
|
||||
workstation model and Python paths. Vendoring hides the `go.mod` problem for an
|
||||
ordinary build. `-mod=mod`, `go mod tidy` and a fresh checkout all fail.
|
||||
Revisit when: anyone clones this repo elsewhere, or a `tidy` is needed.
|
||||
Workaround: build only from this checkout, with the vendor directory.
|
||||
|
||||
## The env example omits deployed variables [#691] {#secrets}
|
||||
|
||||
Costs: a fresh deploy can lose remote speech-to-text or ambient authentication
|
||||
and run on fallback behaviour with an apparently valid config. Three variables
|
||||
are referenced and undocumented: `MAVEN_STT_TOKEN`, `MAVEN_AMBIENT_TOKEN` and
|
||||
`CW2_TOKEN`. V-673 added `MAVEN_GPU_TOKEN` to the example.
|
||||
It is not silent. The loader logs which variables were unset and says whatever
|
||||
they configure is off. What is missing is a startup failure.
|
||||
Revisit when: the box is redeployed from scratch, or a new secret is added.
|
||||
Workaround: read that log line at startup.
|
||||
@@ -0,0 +1,13 @@
|
||||
# Dependencies
|
||||
|
||||
## The analyzers pass against a baseline, not against zero [#701] {#baseline}
|
||||
|
||||
Costs: `make lint` and `make deadcode` are wired and green (V-694), but green
|
||||
means "nothing new since 2026-08-11". The accepted set is 19 staticcheck
|
||||
findings and 13 unreachable symbols, listed with a reason each in
|
||||
`scripts/analyzers/*.baseline`. Three of the unreachable symbols must stay:
|
||||
[layering.md](layering.md#deadcode). One accepted staticcheck finding is V-687.
|
||||
Revisit when: V-701 sweeps the baseline, or a fix deletes an entry. The gate
|
||||
fails on an entry whose finding is gone, so the deletion is not optional.
|
||||
Workaround: none needed. Reachability claims are checkable now. Read the
|
||||
baseline before trusting that a target reporting clean means the tree is clean.
|
||||
@@ -0,0 +1,20 @@
|
||||
# External inputs
|
||||
|
||||
What arrives from a service Maven does not run, and what happens when it
|
||||
arrives malformed. The shared shape: a JSON decode into value fields cannot
|
||||
tell "absent" from "zero", so a degraded response becomes a confident answer.
|
||||
|
||||
## An empty STT transcript reads as a successful one [#675] {#stt}
|
||||
|
||||
Costs: one dropped voice turn per malformed 200 from workpc. The mavsttd floor
|
||||
is never asked, because `stt.Pair` falls back on a non-nil error alone.
|
||||
Revisit when: CW2 returns a 200 with no text. Sooner if a proxy is put between
|
||||
homesrv and port 8081.
|
||||
Workaround: none. It is silent by design and the fallback is never spoken.
|
||||
|
||||
## Open-Meteo's empty body becomes 0°C [#676] {#weather}
|
||||
|
||||
Costs: he is told the weather is clear and 0°C when the service answered
|
||||
nothing. Distinct from V-589, which covered the HTTP status and not the body.
|
||||
Revisit when: a weather answer is reported as wrong, or the geocoder changes.
|
||||
Workaround: none.
|
||||
@@ -0,0 +1,25 @@
|
||||
# Layering and dead surface
|
||||
|
||||
Neither entry breaks anything today. Both make a later change cost more than it
|
||||
should, which is why they are low and not medium.
|
||||
|
||||
## Domain packages depend on store and IPC types [#685] {#dtos}
|
||||
|
||||
Costs: dialogue exposes `store.DialogueSessionRow` in its port, the pure
|
||||
morning planner takes a `store.Fact`, and auth policy imports IPC method and
|
||||
caller types. There is no Go import cycle. A schema change reaches further than
|
||||
it should.
|
||||
Revisit when: the dialogue or fact schema changes, or a second transport
|
||||
appears beside IPC.
|
||||
Workaround: none needed. It compiles and it is correct.
|
||||
|
||||
## Eleven symbols are unreachable [#686] {#deadcode}
|
||||
|
||||
Costs: extra API and test surface, and comments that claim callers which no
|
||||
longer exist. Three of the eleven must not be deleted. `HisGender` is a
|
||||
documented seam tied to V-399. `AudioDuration` duplicates `internal/audio` and
|
||||
should call it. `CountWord` is a one-line alias nobody uses and can go.
|
||||
Revisit when: `deadcode` is wired into the audit gate, which needs the
|
||||
allowlist this entry describes. **An unannotated list invites deleting the
|
||||
three above.**
|
||||
Workaround: none needed.
|
||||
@@ -0,0 +1,26 @@
|
||||
# Security
|
||||
|
||||
Both entries are mavweb's passkey seam. Assertion itself is sound and is not
|
||||
the problem: `userVerification` is required, and a sign count that does not
|
||||
increase is rejected.
|
||||
|
||||
## Anyone past the proxy can enroll a passkey [#683] {#enrollment}
|
||||
|
||||
Costs: registration is gated on nothing, so any client that reaches mavweb can
|
||||
enroll its own key and become him. Step-up is worse than per-client: one
|
||||
process-global `assertedAt` means every client inherits the same five-minute
|
||||
window after any successful assertion. The voice WebSocket accepts every
|
||||
origin, which makes cross-site use easier. V-317 covers which routes are gated
|
||||
and V-605 covers challenge-map growth. Neither covers this.
|
||||
Revisit when: mavweb is reachable from anything but the tunnel, and before any
|
||||
new credential is enrolled.
|
||||
Workaround: the reverse proxy is the only boundary today. That is the finding.
|
||||
|
||||
## Passkey credentials are rewritten in place [#684] {#credentials}
|
||||
|
||||
Costs: `os.WriteFile` over the live file. A crash, a full disk or an
|
||||
interrupted write corrupts every enrolled credential at once, and mavweb will
|
||||
not start afterwards.
|
||||
Revisit when: a second credential is enrolled, since the blast radius grows
|
||||
with the count. Sooner if the box loses power unexpectedly.
|
||||
Workaround: back the file up before enrolling.
|
||||
@@ -0,0 +1,30 @@
|
||||
# Storage
|
||||
|
||||
The DB seam: what it loses quietly, and what it reads more of than it needs.
|
||||
|
||||
## Dialogue persistence errors are swallowed [#677] {#dialogue}
|
||||
|
||||
Costs: restart continuity can vanish with nothing in the log, and a failed
|
||||
delete can bring stale conversation state back. Current-turn dialogue is
|
||||
unaffected, which is why this has never been noticed.
|
||||
Revisit when: a restart is reported as losing context. Sooner if a turn starts
|
||||
reading dialogue rows back to him.
|
||||
Workaround: none. The failure is invisible from outside.
|
||||
|
||||
## The reminder transition is a lost update [#678] {#reminders}
|
||||
|
||||
Costs: a concurrent fire and cancel both succeed and the last writer wins.
|
||||
Medium today because cancellation has no surface. High the moment V-622 adds
|
||||
one, and V-622 does not describe this invariant.
|
||||
Revisit when: V-622 starts, whichever comes first.
|
||||
Workaround: none, but the window is small while nothing can cancel.
|
||||
|
||||
## A recall miss scans two whole tables [#681] {#recall}
|
||||
|
||||
Costs: every missed recall reads all of `memory_vectors` and then decodes and
|
||||
sorts every note vector. Not an N+1, and the memory scan is cheap per losing
|
||||
row on purpose. The duplicated decode is the legacy notes path alone.
|
||||
Revisit when: the note count makes a miss measurably slow. Also when the two
|
||||
exclusion filters are proven to agree. `QueryNotes` uses `notHisWordsSQL` and
|
||||
`Search` uses `memory.NonRecallPrefix`. The fallback cannot go until they match.
|
||||
Workaround: none needed at today's row counts.
|
||||
@@ -0,0 +1,34 @@
|
||||
# Transport
|
||||
|
||||
The HTTP and socket seams. What a client can do to them, and what a shutdown
|
||||
can do to us.
|
||||
|
||||
## Closing a TCP listener can strand Accept [#679] {#accept}
|
||||
|
||||
Costs: during close, both `errc` and `done` are ready in `acceptLoop`'s select.
|
||||
Go picks uniformly. So roughly one close in two leaves a waiting `Accept`
|
||||
blocked forever on a TCP seam. Unix sockets are unaffected.
|
||||
Revisit when: a daemon is seen hanging on shutdown, or before any new TCP
|
||||
listener is added.
|
||||
Workaround: the process usually exits anyway, which hides it.
|
||||
|
||||
## PTT reads an unbounded body [#688] {#ptt}
|
||||
|
||||
Costs: `handlePTT` does an unlimited `io.ReadAll`, and mavweb sets no header or
|
||||
idle timeouts. A client can force unbounded allocation or hold a connection
|
||||
open. mavgpud's half of this was fixed in V-673.
|
||||
Revisit when: mavweb is reachable from anything but the tunnel.
|
||||
Workaround: mavweb is not LAN-exposed today.
|
||||
|
||||
`/ws` rides along with this entry. It never calls `SetReadLimit`, so the
|
||||
dependency default of 32,768 bytes applies, about a second of audio. Nothing
|
||||
reaches it: the browser posts PCM to `/api/ptt`, and only `handlers_test.go`
|
||||
opens `/ws`. It gets a caller and a real limit, or it gets deleted.
|
||||
|
||||
## mavweb errors cannot be traced [#689] {#errors}
|
||||
|
||||
Costs: some handlers return the raw internal error, which discloses internals.
|
||||
Others return a generic one with no identifier, which cannot be joined to its
|
||||
log line. There is no request-id middleware to join them.
|
||||
Revisit when: a reported UI failure cannot be found in the log.
|
||||
Workaround: read the log by timestamp.
|
||||
@@ -0,0 +1,23 @@
|
||||
# Background workers
|
||||
|
||||
Both entries are a tick doing expensive work it did not need to do.
|
||||
|
||||
## Fact enrichment is a 20-call serial waterfall [#680] {#enrichment}
|
||||
|
||||
Costs: each fact is resolved in turn and each ecosystem call can spend ten
|
||||
seconds. A slow but reachable Nexus holds one tick for minutes, so the worker
|
||||
stops observing its configured interval. V-647 covered the duplicate queue
|
||||
scan, not this.
|
||||
Revisit when: Nexus gets slow, or when a batch-resolution endpoint exists.
|
||||
Workaround: an unreachable Nexus is fine. It is the slow-but-answering case
|
||||
that hurts.
|
||||
|
||||
## A suppressed nudge is phrased anyway [#687] {#nudges}
|
||||
|
||||
Costs: `PhraseNudge` runs before the dedupe is known. The `continue` meant to
|
||||
skip it is the last statement in the loop body. Every tick that
|
||||
keeps suppressing the same rule pays the resident model again. The comment
|
||||
above it claims the opposite.
|
||||
Revisit when: digestion ticks show up in the model's load, or when nudge rules
|
||||
grow past a handful.
|
||||
Workaround: none.
|
||||
@@ -0,0 +1,220 @@
|
||||
# Deployment: the boxes, the models, the daemons
|
||||
|
||||
*Last verified: 2026-08-09 @ a9b480a*
|
||||
|
||||
What runs where, and why each choice was made. `CLAUDE.md` carries only the
|
||||
rules. This file carries the reasoning.
|
||||
|
||||
## The two boxes
|
||||
|
||||
**homesrv** is a Ryzen 5 5600U laptop and the deploy target. It offloads to the
|
||||
Vega iGPU over Vulkan (`n_gpu_layers: 99`). Compose passes `/dev/dri` and the
|
||||
render gid (993), and without both Vulkan enumerates zero devices and
|
||||
llama-server falls back to CPU silently.
|
||||
|
||||
**workpc** is the workstation, 16GB of VRAM, reached as `kami@workpc` at
|
||||
192.168.1.105. Model work moved there on 2026-08-02 by the owner's call, because
|
||||
homesrv cannot grow a GPU.
|
||||
|
||||
Three rules govern the seam:
|
||||
|
||||
- **The workstation is never assumed up.**
|
||||
- **Fall back silently** when it would only do the job better.
|
||||
- **Name the gap** when the resident model cannot do the job at all.
|
||||
|
||||
A world question goes through `LLMPhraser.PhraseWorld` and returns `worldGap`
|
||||
(`cmd/mavend/worldmodel.go`) rather than an invented answer. A box with no
|
||||
`workstation` block behaves exactly as it did before the seam. `docs/offload.md`
|
||||
says which caller is which.
|
||||
|
||||
## The resident model
|
||||
|
||||
**Qwen3-1.7B** (`UD-Q4_K_XL`), stock, not yet the CPT'd one. It is a Thinking
|
||||
variant, so `n_ctx` is 4096. Reasoning tokens need the room, and 4096 is what
|
||||
every score was measured at.
|
||||
|
||||
The target is the locally CPT'd Qwen3-1.7B (V-122, training in flight). Stock
|
||||
already speaks good Russian. What it gets wrong is the persona. It writes `я рад`
|
||||
where Maven needs `рада`.
|
||||
|
||||
**Do not bother with sub-500M models.** LFM2.5-230M and 350M were measured on
|
||||
2026-07-31 and both are unusable in Russian
|
||||
(`docs/evals/2026-07-31-model-bakeoff.md`). Their published IFEval and BFCL
|
||||
numbers are English-only.
|
||||
|
||||
Model files live in `/mnt/hdd1/llms`, bind-mounted to `/opt/maven/models/llm`.
|
||||
That **shadows** the repo's `models/llm/`, so a gguf sitting there is not loaded
|
||||
by anything. Swapping the resident model is a one-line change to
|
||||
`phraser.model_path` in `deploy/mavend.json`.
|
||||
|
||||
The workstation model is gemma-4-E4B as of 2026-08-09, replacing the 12B by the
|
||||
owner's call. Keep the 12B gguf. It is the better teacher for label runs, at
|
||||
72.7% destination against E4B's 57.6%.
|
||||
|
||||
## The embedder
|
||||
|
||||
**It stays on homesrv permanently**, because it backs the floor. It is
|
||||
multilingual-e5-small, quantized and asymmetric. `EmbedQuery` and `EmbedPassage`
|
||||
apply the `query:` and `passage:` prefixes it was trained with. Calling plain
|
||||
`Embed` on a note is a bug. See `docs/evals/2026-08-04-recall-e5-small.md`.
|
||||
|
||||
The vendored onnxruntime under `deps/` has two copies, and the stale one is
|
||||
1.17.1. The live runtime is 1.26.0, and the Go binding asks for API 26. Anything
|
||||
shipped to another box needs `deps/onnxruntime-linux-x64-1.26.0`.
|
||||
|
||||
## Speech-to-text
|
||||
|
||||
`sttSeam` in `cmd/mavend/voicewire.go` builds an `stt.Pair` beside `modelSeam`.
|
||||
It prefers CrisperWhisper 2.0 turbo on workpc with mavsttd as the floor. It takes
|
||||
only the silent half of the rule, because a worse transcript is still a turn. So
|
||||
`stt.Pair` has no `TranscribeRemote` and the fallback is never spoken.
|
||||
|
||||
CW2 turbo scores 10.4% WER in Russian against 27.5% for the `ggml-small.bin`
|
||||
mavsttd loads, over 200 Golos clips
|
||||
(`docs/evals/2026-08-09-crisperwhisper2-russian-wer.md`).
|
||||
|
||||
**whisper.cpp cannot load CW2 at all.** It reads its language count off the
|
||||
vocabulary size. CW2's 51897 tokens shift seven special token ids. So CW2 is its
|
||||
own transformers service on port 8081 (`deploy/cw2/serve.py`).
|
||||
`stt.HTTPTranscriber` posts raw PCM to it with a bearer token, because audio is
|
||||
the most sensitive thing that crosses this seam. The switch is `workstation.stt`
|
||||
in `deploy/mavend.json`, and deleting the block sends every utterance to mavsttd.
|
||||
|
||||
**mavgpud runs that service as a second child.** This is not an optimisation.
|
||||
CW2 is a ROCm process on the same card, so it registers on the KFD like any
|
||||
contender. Under its own systemd unit it made mavgpud evict llama-server every
|
||||
few seconds. That took the model arm down for eight minutes on 2026-08-09. The
|
||||
card needs one owner. CW2 is on the yield clock and not the idle one. At 1.6GB
|
||||
it denies the card to nobody.
|
||||
|
||||
Text-to-speech has not moved. piper on homesrv is the only synthesizer.
|
||||
|
||||
## The daemons
|
||||
|
||||
| Binary | Role |
|
||||
|---|---|
|
||||
| `mavend` | **Core.** Router, phraser, memory, reminders, digestion tick. Owns the DB and IPC socket. |
|
||||
| `mavweb` | HTTP UI and PWA (`/dash`, `/history`, `/trace`, `/notifications`, `/tools`), WebAuthn auth. |
|
||||
| `mavsttd` | Speech-to-text (whisper.cpp, CGO). |
|
||||
| `mavttsd` | Text-to-speech (piper subprocess). |
|
||||
| `mavwaked` | Wake-word and VAD gate. Runs on workpc. |
|
||||
| `mavenclient` | Voice loop client (mic, stt, core, tts). Not deployed. |
|
||||
| `mavpoll` | Environment poller: netdata alarms, uptime-kuma, zenmoney, wireguard presence. Writes facts, sends nothing. Telegram is `internal/delivery/telegramsink`. |
|
||||
| `mavcaldav` | CalDAV calendar sync. |
|
||||
| `mavmaild` | Mail reader (IMAP, read-only). Holds the IMAP password, core never sees it. |
|
||||
| `mavgpud` | GPU supervisor. **Runs on workpc**, own unit `deploy/mavgpud.service`. Keeps llama-server loaded while the card is free (V-488). Maven never asks it for anything and reads `/health` through `llm.Pair`. Its LAN port needs a bearer token in the file named by `token_file`, matching `MAVEN_GPU_TOKEN` on homesrv, or it refuses to start (V-673). |
|
||||
| `mavupdate` | Not a daemon. Operator CLI a human runs on the box to deploy a new build. |
|
||||
|
||||
Two binaries have no Makefile target and neither is deployed. `mavseal` encrypts
|
||||
a live tmpfs working copy back to the ciphertext file when mavend was killed
|
||||
before `defer st.Close()` sealed it. `labelgen` runs the stage 0 grammars over
|
||||
utterances and prints JSONL, the training data for the routing heads.
|
||||
|
||||
Daemons are wired socket-to-socket, not linked. `internal/ipc` is the wire
|
||||
protocol. `deploy/mavend.json` sets socket paths, model paths and the phraser and
|
||||
embedder blocks, with `${VAR}` expansion from gitignored `deploy/telegram.env`.
|
||||
|
||||
### Who is in compose, and who is not
|
||||
|
||||
**`docker-compose.yml` runs five**: `mavend`, `mavsttd`, `mavttsd`, `mavweb`,
|
||||
`mavpoll`. Count against compose, not against the table above.
|
||||
|
||||
`mavmaild` and `mavcaldav` are commented out, each with the reason beside it. The
|
||||
first needs a mail account and the second a CalDAV account, and this box has
|
||||
neither. Two things ride on the CalDAV absence (V-644). Agenda questions route to
|
||||
`IntentQuery` at stage 0, and the `calendar` query source then reads a table
|
||||
nobody writes. And `loop.State.CalendarBusy` is fed by the same facts, so the
|
||||
gate's "do not nag mid-meeting" is permanently false.
|
||||
|
||||
`mavenclient` is still absent. `mavwaked` moved to workpc on 2026-08-09 (V-515).
|
||||
|
||||
### The voice wire
|
||||
|
||||
`internal/voice` is plaintext with no auth. Its own server doc says production
|
||||
binds inside the wg tunnel, because the wg layer is the L0 floor. workpc is not
|
||||
a wg peer, it sits on wlan0. So the tunnel is ssh instead.
|
||||
|
||||
mavend publishes the voice port to homesrv loopback only, `127.0.0.1:9110`.
|
||||
Host 9100 is Vikunja's MCP, hence 9110. The container side stays 9100 so mavweb
|
||||
keeps reaching `mavend:9100` by name. `deploy/maven-voice-tunnel.service` on
|
||||
workpc forwards it over his key.
|
||||
|
||||
**Do not replace this with a LAN bind.** `SurfaceVoice` caps acts at L0, so an
|
||||
unauthorized speaker could not run a destructive tool. L0 does not cap reading,
|
||||
so they would still hear his facts, notes and calendar read back.
|
||||
|
||||
Both `mavwaked` and `mavenclient` speak `voice.Dial`, not `ipc.Dial`. The
|
||||
`netaddr` token guards the daemon-to-daemon IPC seam and never touches this one.
|
||||
`ipc.Dial` does take `tcp://host:port?token=...`, which is why V-515 was filed
|
||||
as a config change. That premise was wrong, and the ssh leg is the correction.
|
||||
|
||||
The voice loop belongs on a client machine where the owner is standing, and that
|
||||
machine is workpc (V-463, `docs/plans/17-where-the-voice-loop-runs.md`). homesrv
|
||||
has a microphone, because it is a laptop, but it is in the wrong room.
|
||||
|
||||
### mavwaked on workpc
|
||||
|
||||
`deploy/mavwaked.service`, a user unit beside `mavgpud.service`. Two flags are
|
||||
deliberate.
|
||||
|
||||
`-vad-model` is passed. Silero answers "is this frame speech" where the energy
|
||||
floor answers "is this frame loud". It declines white noise at the same RMS, 0
|
||||
frames against 68 to 99, and still hears all four spoken fixtures
|
||||
(`docs/evals/2026-08-09-silero-vad.md`). It costs 509µs a frame and never touches
|
||||
the GPU. A model that will not load is logged and not fatal.
|
||||
|
||||
`-barge-in` is not passed. The threshold is room-specific and this room has no
|
||||
number yet. Read the "suppressed while speaking" means out of the journal first.
|
||||
|
||||
The device is `plughw:CARD=Gen,DEV=0`, the Scarlett Solo. It is `plughw` and
|
||||
not `hw` because mavwaked asks arecord for 16kHz mono. No microphone here
|
||||
offers that, so bare `hw` dies on "Channels count non available" before a
|
||||
frame is read. It is named `CARD=Gen` and not card 4 because a USB card number
|
||||
moves when something else is replugged.
|
||||
|
||||
It used to be the fifine on card 0, and that cost three days. mavwaked logged
|
||||
zero completed utterances across them, before the wake word existed and after.
|
||||
The fifine returns RMS 0.00004 with its capture switch on and its ALSA volume
|
||||
at the full 496 of 496. That silence is in the hardware and no flag reaches
|
||||
it. Over the same eight seconds of speech the onboard ALC897 read 0.142 and
|
||||
the camera 0.289, both clipping at peak 1.0. The Scarlett read 0.003 clean.
|
||||
|
||||
Check the level before blaming the gate. Stop the unit, run `arecord` against
|
||||
the device for five seconds, and measure. A live room floor reads near 0.001.
|
||||
|
||||
The three `-wake-` flags require the keyword "Мэйвен" (V-487 stage two). Drop
|
||||
them and the loop runs open, which is what it did before. The threshold is the
|
||||
binary's default of 0.999 and is not passed. Over 65 minutes of held-out
|
||||
Russian speech it woke her 0.9 times an hour against 2.8 at 0.99. That cost one
|
||||
lost render out of 126 (`docs/evals/2026-08-09-wake-word.md`).
|
||||
|
||||
The three sessions are pinned to one thread each. onnxruntime otherwise sizes
|
||||
its pool to every core and spins between runs, which took mavwaked from 68% of
|
||||
one core to 335%. With the cap it sits at 81%, so the gate costs about 13%.
|
||||
|
||||
A keyword shorter than 1.32s can be heard too late to be used. The head scores
|
||||
1.28s of audio, and the VAD has closed the utterance by then.
|
||||
"Мэйвен, <request>" is unaffected. A bare "Мэйвен" is the case that fails.
|
||||
|
||||
mavwaked connects at startup and holds the conn, so a nudge routed to voice
|
||||
reaches the speaker before he has said anything (V-671). It used to connect
|
||||
lazily, which made the failure silent rather than absent: after one utterance
|
||||
the session existed, `PushToMostRecent` succeeded, the dispatcher stopped
|
||||
rerouting to telegram and ntfy, and mavwaked discarded the audio.
|
||||
|
||||
**Passwords are read from files, never taken as flag values.** `mavcaldav` uses
|
||||
`-pass-file` and `-render-pass-file`. `mavpoll` and `mavmaild` follow the same
|
||||
rule.
|
||||
|
||||
## Web UI conventions
|
||||
|
||||
Server-rendered pages share `cmd/mavweb/static/ui.css` (served at `/ui.css`) and
|
||||
the shell partial in `cmd/mavweb/shell.html`. A page opens with
|
||||
`{{template "shellTop" "<page-key>"}}` and closes with `{{template "shellBottom"}}`,
|
||||
and the key marks the active sidebar link.
|
||||
|
||||
Every page is its own embedded `.html` file next to `main.go`. No page markup
|
||||
lives in Go, and the sidebar is data (`sidebarSections`, `pageIcon`) the template
|
||||
renders. No per-page `<style>` beyond true one-offs. Wrap every table in
|
||||
`<div class=scroll>` so wide data pans on a phone. Local preview and headless
|
||||
screenshot recipes are in `AGENTS.md`.
|
||||
@@ -30,6 +30,12 @@ Maven is the user-facing control center, but not the source of truth for identit
|
||||
|
||||
Maven provides the human interface over the other systems.
|
||||
|
||||
| Service | Owns | Maven's client | Configured at |
|
||||
|---|---|---|---|
|
||||
| **Nexus** | Canonical entity ids, names, aliases, relationships. Projects, services, devices, people, pets, places. | `nexusClient` in `cmd/mavend/ecosystem.go`, `POST /api/v1/resolve` | `nexus.url` (`http://nexus:9740`) |
|
||||
| **Praxis** | Operational attention and item lifecycle. What needs looking at, what changed, what is unresolved. | `praxisClient`, the HTTP tools API under `/api/v1/tools/` | `praxis.url` (`http://praxis:8989`) |
|
||||
| **Hexis** | The capability registry and the only path to executing anything. | vendored `github.com/kami/hexis/pkg/client` | `hexis.url` (`http://hexis:9741`) |
|
||||
|
||||
It is responsible for:
|
||||
|
||||
- interpreting Russian and English utterances
|
||||
|
||||
@@ -0,0 +1,112 @@
|
||||
# The "Мэйвен" wake word: what it hears and what it invents
|
||||
|
||||
*Measured 2026-08-09 on workpc and homesrv. V-487, stage two of two.*
|
||||
|
||||
Stage one gave mavwaked silero-vad, which answers "is this frame speech".
|
||||
Nothing answered "was this said to her", so every utterance near the
|
||||
microphone became a turn. SurfaceVoice caps acts at L0, which made that safe
|
||||
rather than expensive. L0 does not cap reading, so the room could still hear
|
||||
his facts read back.
|
||||
|
||||
The keyword is "Мэйвен". openWakeWord's two frozen feature models do the
|
||||
hearing and a 100KB head trained here draws the boundary. It runs on CPU
|
||||
beside silero and never touches the GPU.
|
||||
|
||||
## Why a per-window accuracy is not a number anyone can act on
|
||||
|
||||
The gate scores every 80ms. A 1.7% false-accept rate per window sounds small
|
||||
and means a wake every few seconds. The useful question is how many times an hour
|
||||
it wakes on speech that was not the keyword. So every table below counts
|
||||
threshold crossings over whole clips and divides by the audio duration.
|
||||
|
||||
A crossing, not a window above the threshold. A keyword held high for half a
|
||||
second is one wake, not six.
|
||||
|
||||
## The data
|
||||
|
||||
Positives are 600 silero TTS renders of three stressings of the keyword, six
|
||||
speakers, ten trailing phrases, augmented eight ways each. Hard negatives are
|
||||
560 renders of confusable Russian words. Real speech is Common Voice ru and Golos.
|
||||
The 74257 Common Voice clips were already on workpc from the CrisperWhisper
|
||||
work. The 200 Golos clips came from the CW2 WER eval.
|
||||
|
||||
Splits are by source file. Augmented copies of one render on both sides of a
|
||||
split would measure memorisation.
|
||||
|
||||
Golos was never trained on at any stage, so it answers the harder question:
|
||||
does this survive a change of speakers and rooms.
|
||||
|
||||
## Three heads
|
||||
|
||||
Each row is a full retrain. The false-accept column is 8.89 hours of Common
|
||||
Voice that no stage of training had seen.
|
||||
|
||||
| trained on | recall (window) | false wakes/hour @0.99 |
|
||||
|---|---|---|
|
||||
| TTS + 13.7 min of Golos | 0.869 | not measurable |
|
||||
| + 4000 Common Voice clips | 0.836 | 21.9 |
|
||||
| + 3837 mined hard negatives | 0.784 | 4.2 |
|
||||
| + 753 more mined | 0.810 | 3.4 |
|
||||
|
||||
The first row is why the second exists. Thirteen minutes of held-out speech
|
||||
cannot measure a rate for a gate that scores twelve times a second. A head
|
||||
trained only against TTS learns to tell TTS from not-TTS.
|
||||
|
||||
Mining is the whole story after that. Random negatives teach the head what
|
||||
most speech sounds like. They do not teach it the few syllable sequences that
|
||||
score high, because 4000 clips barely contain them. So the current head was
|
||||
run over 20000 fresh clips, keeping every window it scored above 0.05. That
|
||||
found 3837 windows in 855512. Repeating those ten times in the next training
|
||||
run cut the rate five-fold.
|
||||
|
||||
The second round found 753 in 852240, a fifth of the yield, and bought a
|
||||
further 20%. It also recovered recall, which the first round had cost. Whether
|
||||
a third round is worth 25 minutes of workpc is untested.
|
||||
|
||||
## Where the threshold came from
|
||||
|
||||
Both columns are held out. Positives are the 126 renders in the test split.
|
||||
Speech is 65.1 minutes of Common Voice, disjoint from every training and
|
||||
mining pool. Both were run through the built `mavwaked` binary reading PCM from a
|
||||
file, not through the python that trained the head.
|
||||
|
||||
| threshold | renders shipped | false wakes/hour |
|
||||
|---|---|---|
|
||||
| 0.99 | 116 / 126 | 2.8 |
|
||||
| 0.999 | 115 / 126 | 0.9 |
|
||||
|
||||
One render against a third of the false wakes. `defaultWakeThreshold` is
|
||||
0.999.
|
||||
|
||||
Golos disagrees. It gave 2 wakes in 14 minutes at every threshold, which is
|
||||
8.7 per hour. Two events is not a rate. What it does say is that a handful of real utterances score above 0.999
|
||||
and no threshold will move them.
|
||||
|
||||
## What it costs him
|
||||
|
||||
Ten of the 126 held-out renders were heard and still dropped, and every one
|
||||
was an utterance shorter than 1.32s. The head scores 16 embeddings, or 1.28s of
|
||||
audio. The score therefore peaks up to a second after a short keyword ends.
|
||||
By then the VAD has closed the utterance and dispatch has already asked.
|
||||
|
||||
Real commands are "Мэйвен, <request>" and run past two seconds, which gives
|
||||
the head the whole request to peak during. A bare "Мэйвен" with nothing after
|
||||
it is the case that fails. One fix would hold an ignored utterance for a grace
|
||||
period and ship it if the keyword lands late. It is not built.
|
||||
|
||||
## What it costs the workstation
|
||||
|
||||
Under systemd on workpc, mavwaked sat at 335% of a core with the gate on and
|
||||
68% with only silero. onnxruntime sizes its thread pool to every core and spins
|
||||
between runs, and this gate runs three graphs twelve times a second. Pinning
|
||||
all three sessions to one thread brought it to 81%, so the keyword costs about
|
||||
13% of one core. The three graphs each finish in well under 80ms that way.
|
||||
|
||||
## What was not measured
|
||||
|
||||
No room recordings. Every negative above is a clean corpus clip. This gate
|
||||
will live among a television, a fan and the far side of a kitchen. None of
|
||||
those are in these numbers.
|
||||
|
||||
No measurement of him. Training on his voice means copying his transcripts off
|
||||
homesrv, which is his call and has not been asked.
|
||||
@@ -0,0 +1,261 @@
|
||||
# Repository deep-audit report
|
||||
|
||||
Date: 2026-08-10
|
||||
Revised: 2026-08-11, a verification pass over every cited line. Six claims were
|
||||
wrong as first written and are corrected in place. Two findings were added.
|
||||
|
||||
Frozen 2026-08-11, V-674. A dated measurement is never edited after the day,
|
||||
and that holds for a finding which later gets fixed. The live state of each one
|
||||
sits in `docs/caveats/` with its task id. The fix sits in the subsystem's living
|
||||
doc under `docs/`. Read this file for the evidence, not for what is still true.
|
||||
|
||||
Read-only audit complete. I found **20 issue-specific candidates not represented by a matching Vikunja task**: **3 High, 15 Medium, 2 Low**. No code or Vikunja tasks were changed.
|
||||
|
||||
I compared all 366 tasks in Maven project 2. Existing items such as V-317, V-589, V-597, V-603, V-605, V-608, V-643 and V-647 were excluded except where a new finding is clearly separate.
|
||||
|
||||
## 1. Unhandled edge cases and silent failures
|
||||
|
||||
### Empty STT responses suppress the local fallback
|
||||
|
||||
**A — Location:** `internal/stt/http.go:85` accepts `{}` as a successful transcript and returns empty text at line 89. `internal/stt/pair.go:143` only falls back when `err != nil`.
|
||||
|
||||
**B — Severity:** Medium. A malformed `200 OK` from the workstation silently drops a voice turn instead of invoking `mavsttd`.
|
||||
|
||||
**C — Proposed fix:** Require nonblank transcript text and a valid confidence range; treat missing required fields as an error so `Pair` falls back. Bound the response decoder and add `{}`, `{"text":""}`, oversized-body and invalid-confidence tests.
|
||||
|
||||
### Open-Meteo can report invented zero-degree weather
|
||||
|
||||
**A — Location:** `internal/weather/openmeteo.go:47` uses value fields, while line 95 accepts `{}` and line 100 interprets it as WMO 0 and 0°C.
|
||||
|
||||
**B — Severity:** Medium. A valid JSON error/degraded response becomes plausible but false weather. This is separate from V-589, which covered HTTP status handling.
|
||||
|
||||
**C — Proposed fix:** Make `current_weather` and its required members pointer/nullable fields, validate presence and ranges, and cap both forecast and geocoder bodies.
|
||||
|
||||
### Dialogue persistence errors disappear completely
|
||||
|
||||
**A — Location:** Corrupt rows are silently skipped at `internal/dialogue/session.go:127`; delete failures are discarded at line 191; marshal and save failures are swallowed at lines 201 and 209.
|
||||
|
||||
**B — Severity:** Medium. Restart continuity can silently vanish, while failed deletes can resurrect stale conversation state.
|
||||
|
||||
**C — Proposed fix:** Return or report persistence errors with session ID and operation; quarantine/delete corrupt rows; use bounded contexts instead of `context.Background()`. Keep current-turn availability, but expose the lost restart guarantee through logs/metrics.
|
||||
|
||||
## 2. Concurrency and race conditions
|
||||
|
||||
### Reminder state transition is a read-then-write lost update
|
||||
|
||||
**A — Location:** `internal/store/reminders.go:172` reads `pending`, then line 182 updates without checking the old state.
|
||||
|
||||
**B — Severity:** Medium now; High once V-622 adds cancellation surfaces. Concurrent fire/cancel operations can both succeed and the last writer wins. This invariant is not described in V-622.
|
||||
|
||||
**C — Proposed fix:** Use one conditional statement: `UPDATE ... WHERE id=? AND status='pending'`; inspect `RowsAffected`, then distinguish not-found from invalid transition. Add simultaneous fired/cancelled tests under `-race`.
|
||||
|
||||
### TCP listener shutdown can leave `Accept` blocked forever
|
||||
|
||||
**A — Location:** `internal/netaddr/netaddr.go:180` waits only on `conns` and `errc`. During close, line 201 may select the already-closed `done` branch without publishing the listener error; `Close` at line 225 does not wake `Accept`.
|
||||
|
||||
**B — Severity:** Medium. During close both `errc` and `done` are ready in `acceptLoop`'s select and Go picks uniformly, so roughly one close in two strands a waiting `Accept` forever on a TCP seam.
|
||||
|
||||
**C — Proposed fix:** Add `case <-l.done: return nil, net.ErrClosed` to `Accept`, and make error/channel closure ownership explicit. Test an in-flight `Accept` concurrently with `Close`.
|
||||
|
||||
## 3. Data fetching: waterfalls and duplicated scans
|
||||
|
||||
### Fact enrichment performs a serial 20-call network waterfall
|
||||
|
||||
**A — Location:** `cmd/mavend/factenrichment.go:160` resolves each fact sequentially; line 223 makes the Nexus call. Each request can consume ten seconds at `cmd/mavend/ecosystem.go:63`.
|
||||
|
||||
**B — Severity:** Medium. A slow-but-reachable Nexus can hold one tick for roughly 20 × 10s, preventing the worker from observing its intended interval. V-647 only covered the duplicate queue scan.
|
||||
|
||||
**C — Proposed fix:** Prefer a Nexus batch-resolution endpoint. Otherwise use bounded concurrency, such as four workers, while preserving per-fact backoff and the 20-attempt ceiling.
|
||||
|
||||
### A recall miss scans two whole tables
|
||||
|
||||
**A — Location:** The query source table runs embed, memory and notes in order at `cmd/mavend/actions_query.go:161` through line 164. `MemoryStore.Search` scans every row of `memory_vectors` at `internal/store/memory.go:82`; a miss then calls the legacy notes query at `cmd/mavend/actions_query.go:718`, which decodes every note vector at `internal/store/notes.go:63` and sorts the whole table at line 69.
|
||||
|
||||
**B — Severity:** Medium at scale. This is not an N+1. It is two full scans per missed recall. The memory scan is already cheap per losing row on purpose, costing one dot product read off the stored bytes with no `[]float32` materialized, so the duplicated decode cost is the notes path alone. Both recall widths are tiny (`memoryRecallWidth` 3, `noteRecallWidth` 5), so the work is in the scan, not the result set. V-581 is a generic sweep of this file, but does not identify this issue.
|
||||
|
||||
**C — Proposed fix:** Establish the invariant that every recallable note exists in `memory_vectors`, then remove the fallback. `internal/store/backfill.go` already rewrites note rows into the unified index, so the backfill exists; what is missing is proof that the two exclusion filters agree, since `QueryNotes` filters on `notHisWordsSQL` while `Search` filters on `memory.NonRecallPrefix`. Until they do, query only notes missing from the unified index and rank with the existing bounded top-K heap.
|
||||
|
||||
## 4. Dependency health and version pinning
|
||||
|
||||
### Reachable published vulnerabilities in the pinned toolchain and `x/text`
|
||||
|
||||
**A — Location:** `go.mod:3` and `Makefile:7` pin Go 1.25.5; `go.mod:25` pins `x/text` 0.14.0. Reachable traces include normalization at `internal/router/onnxembedder.go:364`, HTML rendering at `cmd/mavweb/shell.go:154`, email header decoding at `internal/email/message.go:244`, and reverse proxying at `cmd/mavgpud/main.go:212`.
|
||||
|
||||
**B — Severity:** High. `govulncheck` found **20 reachable advisories**: one in `x/text` and 19 in the Go standard library, including template XSS, parser complexity/DoS and TLS issues. The official database says `x/text` before 0.39.0 can loop on invalid UTF-8; Go 1.25.12 contains the accumulated security corrections. See [GO-2026-5970](https://pkg.go.dev/vuln/GO-2026-5970), [GO-2026-4980](https://pkg.go.dev/vuln/GO-2026-4980), and the [Go release history](https://go.dev/doc/devel/release#go1.25.0).
|
||||
|
||||
**C — Proposed fix:** Upgrade the vendored toolchain to at least 1.25.12, preferably current 1.26.5 after compatibility testing; upgrade `x/text` to at least 0.39.0/current 0.40.0; tidy and re-vendor. Add `govulncheck ./...` to the repository gate.
|
||||
|
||||
No dependency was three major versions behind. The remaining direct updates were minor/patch releases. `.opencode`'s `npm audit` reported zero vulnerabilities and no peer conflicts.
|
||||
|
||||
## 5. Security exposure
|
||||
|
||||
### WebAuthn enrollment is open and step-up state is process-global
|
||||
|
||||
**A — Location:** Registration endpoints have no existing-credential or bootstrap authorization at `cmd/mavweb/webauthn.go:92` and line 103. `RegisterBegin` also answers GET, while `RegisterFinish` requires POST. A single server-wide session is created at `cmd/mavweb/main.go:170`, backed by one `assertedAt` timestamp at `internal/webauthn/session.go:20`. The voice WebSocket accepts every origin at `cmd/mavweb/voiceproxy.go:48`.
|
||||
|
||||
**B — Severity:** High, and scoped to enrollment and session binding. Assertion itself is sound. `internal/webauthn/webauthn.go:221` requests `userVerification: "required"`, and line 304 rejects a sign count that did not increase. What is broken is that any client past the reverse proxy can enroll its own key, and that after any successful assertion every client inherits the same five-minute step-up window. The wildcard WebSocket origin makes cross-site use easier. V-317 covers which routes are gated, and V-605 covers challenge-map growth, not enrollment or session binding.
|
||||
|
||||
**C — Proposed fix:** Permit first enrollment only through a local/one-time bootstrap ceremony; require an already-authenticated credential for subsequent enrollment. Bind step-up to a signed, HttpOnly, SameSite browser session and exact RP origin. Restrict WebSocket origins and add CSRF/origin validation to mutating routes.
|
||||
|
||||
### `mavgpud` exposes an unauthenticated GPU/model proxy on the LAN
|
||||
|
||||
**Closed 2026-08-11, V-673.** The reasoning now lives in `docs/offload.md`,
|
||||
beside the rest of the workstation seam, and `docs/deployment.md` carries the
|
||||
operational line in the daemon table. This block is a pointer, not a second
|
||||
home: read those, not this.
|
||||
|
||||
- Boundary and limits: `cmd/mavgpud/auth.go`, wired in `cmd/mavgpud/main.go`.
|
||||
- Client half: `internal/llm/client.go`, `internal/llm/remote.go`,
|
||||
`internal/config/workstation.go`, `cmd/mavend/voicewire.go`.
|
||||
- Deploy: `token_file` in `deploy/mavgpud.json`, `MAVEN_GPU_TOKEN` in
|
||||
`deploy/telegram.env.example`.
|
||||
- Commits: `95e7427`, `1c13d22`, `5596cdd`, `9bb3425`.
|
||||
|
||||
**Deploy step, not yet done:** write the token to
|
||||
`/home/kami/.config/mavgpud.token` on workpc and put the same value in
|
||||
`MAVEN_GPU_TOKEN` on homesrv, before restarting either side. mavgpud refuses to
|
||||
start without it, and a homesrv missing it falls back to the resident model.
|
||||
|
||||
### Passkey credential persistence is not crash-atomic
|
||||
|
||||
**A — Location:** `cmd/mavweb/credentials.go:36` serializes the full credential map and overwrites the live file directly with `os.WriteFile` at line 41.
|
||||
|
||||
**B — Severity:** Medium. A crash, disk-full event or interrupted write can corrupt every enrolled credential and prevent mavweb from starting.
|
||||
|
||||
**C — Proposed fix:** Write a `0600` temporary file in the same directory, `fsync`, rename atomically, then sync the directory. Preserve the last known-good file and test simulated write failures.
|
||||
|
||||
No new tracked hardcoded keys, raw user-concatenated SQL, `eval`, or shell execution of untrusted strings were found. The historical DB-key exposure is already covered by V-12.
|
||||
|
||||
## 6. Circular dependencies and layering
|
||||
|
||||
### Domain packages depend directly on storage/wire DTOs
|
||||
|
||||
**A — Location:** Dialogue imports `store` and exposes `store.DialogueSessionRow` in its port at `internal/dialogue/session.go:9` and line 75. The pure morning planner accepts `store.Fact` at `internal/morning/plan.go:72`. Auth policy imports IPC method and caller types at `internal/auth/policy.go:8` and `internal/auth/scope.go:33`.
|
||||
|
||||
**B — Severity:** Low. There is no current Go import cycle, but domain changes are coupled to database and IPC schema changes.
|
||||
|
||||
**C — Proposed fix:** Make domain packages own their DTOs and ports—dialogue persistence records, morning evidence, auth operation/caller identity—and adapt them in store/IPC/cmd wiring.
|
||||
|
||||
No circular Go imports were found; compilation and `go vet` both succeeded.
|
||||
|
||||
## 7. Dead code and zombie endpoints
|
||||
|
||||
### Eleven production symbols are unreachable
|
||||
|
||||
**A — Location:** `deadcode` found:
|
||||
|
||||
- `cmd/mavwaked/vad.go:244` — `PCMToF32`
|
||||
- `cmd/mavwaked/vad.go:265` — `AudioDuration`
|
||||
- `internal/crawl/watch.go:86` — `Watcher.Watches`
|
||||
- `internal/phraser/confirm.go:137` — `IsC`
|
||||
- `internal/phraser/plural.go:13` — `CountWord`
|
||||
- `internal/phraser/eval/checks.go:80` — `HisGender`
|
||||
- `internal/update/update.go:363` — `WithClock`
|
||||
- `internal/voice/errors.go:72` — `jsonMarshal`
|
||||
- `internal/voice/errors.go:73` — `jsonUnmarshal`
|
||||
- `internal/webauthn/cbor.go:98` — `cborValue.At`
|
||||
- `internal/worker/server.go:64` — `Server.SetSynthesizer`
|
||||
|
||||
Three of the eleven do not want deleting, and the 2026-08-11 pass checked each:
|
||||
|
||||
- `internal/phraser/eval/checks.go:80` `HisGender` is deliberately exposed and deliberately uncalled. The comment at line 72 ties the trio to V-399, and `cmd/mavend/personaguard.go:94` states why this one is not run on a phrased message. Deleting it removes a documented seam.
|
||||
- `cmd/mavwaked/vad.go:265` `AudioDuration` duplicates what `internal/audio` already computes. Call that instead of deleting the body.
|
||||
- `internal/phraser/plural.go:13` `CountWord` is a one-line alias for `say.CountWord`, and every real caller already uses `say` directly. Safe to delete outright.
|
||||
|
||||
**B — Severity:** Low. They increase API and test surface, and some comments claim callers that no longer exist.
|
||||
|
||||
**C — Proposed fix:** Delete the genuinely obsolete symbols. Where one is an intended extension seam, add the actual caller and a contract test, or record why it stays. Add `deadcode ./...` with an explicit allowlist to the audit gate, since an unannotated list invites deleting the three above.
|
||||
|
||||
The repository history begins on 2026-07-03, so a six-month rotten-feature-flag check is not yet applicable. One zombie HTTP route does exist: `/ws` is wired at `cmd/mavweb/main.go:235` and no shipped client reaches it, since the browser posts to `/api/ptt`. Section 8 carries the detail.
|
||||
|
||||
## 8. Performance hot paths and memory/resource leaks
|
||||
|
||||
### Digest dedupe happens after paying the LLM cost
|
||||
|
||||
**A — Location:** `cmd/mavend/tick_digest.go:147` calls `PhraseNudge` before `EnqueueDigestEntry` reports the dedupe at line 153. The `else if deduped { continue }` is the last statement in the loop body, so it changes nothing.
|
||||
|
||||
**B — Severity:** Medium. Every tick that continues suppressing the same rule can invoke the model again, contrary to the cache claim in the preceding comment.
|
||||
|
||||
**C — Proposed fix:** Check for a live pending entry by stable rule/candidate fingerprint before phrasing, or persist/cache the phrased result with a TTL. Add a test asserting one phraser call across repeated suppressed ticks.
|
||||
|
||||
### PTT reads an unbounded body and neither server sets header or idle limits
|
||||
|
||||
**A — Location:** `handlePTT` performs an unlimited `io.ReadAll` at `cmd/mavweb/voiceproxy.go:125`. Mavweb and mavgpud construct servers without header or idle limits at `cmd/mavweb/main.go:242` and `cmd/mavgpud/main.go:148`. Separately, `handleWS` never calls `conn.SetReadLimit`, so the dependency default of 32,768 bytes applies at `vendor/github.com/coder/websocket/read.go:92`, about one second of 16kHz mono PCM.
|
||||
|
||||
**B — Severity:** Medium for the HTTP side. A client can force unbounded body allocation or hold a connection open indefinitely. Low for the WebSocket read limit, because `/ws` has no caller: the browser client posts PCM to `/api/ptt` at `cmd/mavweb/static/app.js:114`, and `/ws` is wired at `cmd/mavweb/main.go:235` but reached only from `handlers_test.go`. The 64MiB `maxFrame` at `cmd/mavweb/voiceproxy.go:27` is not an unapplied declaration. It caps the mavend voice wire at line 194 and line 212, which is the "either direction" its comment names.
|
||||
|
||||
**C — Proposed fix:** Use `http.MaxBytesReader` for PTT and return 413 on overflow. Configure `ReadHeaderTimeout`, `IdleTimeout` and header limits on both servers. Decide `/ws` separately: either give it an audio-duration read limit and a client, or delete it. See section 7.
|
||||
|
||||
## 9. Error propagation and user feedback
|
||||
|
||||
### Mavweb has no consistent, traceable error contract
|
||||
|
||||
**A — Location:** Some handlers expose raw internal errors, such as `cmd/mavweb/tools.go:57`, `cmd/mavweb/routines.go:58` and `cmd/mavweb/webauthn.go:122`. Others return generic errors without a request/incident identifier, such as `cmd/mavweb/facts.go:90`. No HTTP request-ID middleware was found.
|
||||
|
||||
**B — Severity:** Medium. Raw errors can disclose implementation details, while generic errors cannot be correlated with the correct log entry.
|
||||
|
||||
**C — Proposed fix:** Add a central `writeProblem`/error-page helper with a stable error code and generated request ID; log the full wrapped error server-side and return only a sanitized message plus the ID. Carry the ID into IPC/ecosystem correlation where possible.
|
||||
|
||||
## 10. Configuration drift and environment assumptions
|
||||
|
||||
### Committed absolute paths make builds and deployment host-specific
|
||||
|
||||
**A — Location:** `go.mod:31` replaces Hexis with `/home/kami/apps/hexis`. `start-maven.sh:11` hardcodes the Maven checkout and line 40 hardcodes the data directory. `deploy/mavgpud.json:12` and line 35 contain workstation-specific model/Python paths.
|
||||
|
||||
**B — Severity:** Medium. Vendoring masks the `go.mod` problem for ordinary builds, but `-mod=mod`, tidy and fresh non-Kami checkouts fail. Deployment files cannot be reused safely on another host.
|
||||
|
||||
**C — Proposed fix:** Pin a real Hexis module revision; keep local replacement in an uncommitted `go.work`. Derive script root from the script location and make data paths configurable. Split mavgpud into a committed template plus host-local override.
|
||||
|
||||
### Environment examples do not cover deployed variables
|
||||
|
||||
**A — Location:** Active config references `MAVEN_STT_TOKEN` at `deploy/mavend.json:101`, Compose references `MAVEN_AMBIENT_TOKEN` at `docker-compose.yml:109`, and the GPU service expects `CW2_TOKEN` at `deploy/mavgpud.service:15`. `deploy/telegram.env.example:5` documents only Telegram and ntfy. The loader deliberately converts missing variables to empty settings at `internal/config/config.go:333`.
|
||||
|
||||
**B — Severity:** Medium. A fresh deployment can lose remote STT or ambient authentication and run on fallback behavior despite apparently valid config. It is not silent: `internal/config/config.go:340` logs which variables were unset and states that whatever they configure is off. What is missing is a startup failure and an example file naming them.
|
||||
|
||||
**C — Proposed fix:** Maintain one canonical secret manifest/example covering every referenced variable, or service-specific examples with validation. Fail startup when an enabled integration lacks its required secret; permit empty variables only for explicitly disabled blocks.
|
||||
|
||||
No production/staging debug-mode or mock-gateway drift was found.
|
||||
|
||||
## 11. Declared invariants with no guard
|
||||
|
||||
`CLAUDE.md` names several rules as load-bearing. Two of them are enforced by
|
||||
nothing, which the first pass missed because it audited generic categories only.
|
||||
|
||||
### `heads_path` may equal `model_path` and nothing objects
|
||||
|
||||
**A — Location:** `cmd/mavend/voicewire.go:168` reads `cfg.Voice.Embedder.HeadsPath` and loads it without comparing it to the model path. The rule is stated at `internal/config/voice.go:41`, which says the heads graph is a fine-tuned COPY, and again in `CLAUDE.md`.
|
||||
|
||||
**B — Severity:** Medium. Pointing both keys at the same file degrades recall, because the routing heads then score with the same graph the resident e5-small uses. There is no error and no log line, so the failure looks like ordinary recall drift.
|
||||
|
||||
**C — Proposed fix:** Reject the config at load when `heads_path` equals `model_path` after path cleaning. A daemon that cannot route well should refuse to start rather than answer worse.
|
||||
|
||||
### `baselineGrammars` mirrors `buildRouter` by hand
|
||||
|
||||
**A — Location:** `internal/router/eval/eval_test.go:263` restates the stage 0 rule set in the daemon's order, and its own comment says so. `claims_test.go:30`, `heads_test.go:76` and `eval_test.go:221` all score against it. Nothing compares the two lists.
|
||||
|
||||
**B — Severity:** Medium. A grammar added to `buildRouter` and not to the fixture means every routing measurement scores a set nobody runs, which is the failure mode `CLAUDE.md` warns about by name.
|
||||
|
||||
**C — Proposed fix:** Export the grammar set from one place and have both `buildRouter` and the fixture consume it, or add a test that diffs the two by grammar name and fails on drift.
|
||||
|
||||
`tokenizerRev` and `preRouteLadder` were checked and need nothing.
|
||||
`internal/router/onnxembedder.go:91` bakes the rev into the embedder key, so a
|
||||
bump changes the key and triggers re-embedding. `cmd/mavend/voice.go:278` passes
|
||||
`preRouteLadder` to `decision.Expect`, so a missing rung is observable.
|
||||
|
||||
## Cross-cutting subsystem candidates
|
||||
|
||||
The recurring findings suggest five reusable patterns:
|
||||
|
||||
- A bounded, required-field-validating JSON client for STT, weather, ecosystem and model calls.
|
||||
- Authenticated remote-service middleware providing token checks, body limits, concurrency limits and correlation IDs.
|
||||
- Atomic state-transition helpers using conditional SQL and `RowsAffected`.
|
||||
- A uniform HTTP problem/error envelope.
|
||||
- A repository health gate combining `staticcheck`, `govulncheck`, `deadcode` and dependency audits.
|
||||
|
||||
## Validation
|
||||
|
||||
- `make fmt-check` and `make vet` passed. `make audit` passed too, but it is a git-grep inventory over loc, todo, stubs, docs, tests and gaps (`scripts/audit.sh`), not a static-analysis gate. Do not read it as one.
|
||||
- `staticcheck`, `govulncheck`, `deadcode`, tracked-secret and history scans and npm audit were run out of tree. None of the three Go analyzers is installed on this box or wired into any make target, which is the argument for section 4's proposed gate.
|
||||
- The advisory version numbers in section 4 could not be re-checked offline on 2026-08-11. `deps/go/go/VERSION` reads `go1.25.5`, built 2025-11-26, so the eight-month gap behind current supports the upgrade claim.
|
||||
- The first whole-tree race run failed once in `cmd/mavend` while analyzers were compiling concurrently; a fresh isolated `go test -race -count=1 ./cmd/mavend` passed in 100.6 seconds, so the transient result was not counted as a defect.
|
||||
- The pre-existing `deploy/mavwaked.service` modification and untracked `deploy/asoundrc` remained untouched.
|
||||
@@ -0,0 +1,74 @@
|
||||
# Language: what the model emits, and how Russian is matched
|
||||
|
||||
*Last verified: 2026-08-09 @ a9b480a*
|
||||
|
||||
Two contracts live here. What a model call is allowed to return, and which
|
||||
mechanism is allowed to recognise a Russian word.
|
||||
|
||||
## The LLM output contract
|
||||
|
||||
All phrasing paths emit `{"response":"...","mood":"..."}`. They fall back to
|
||||
plain text when the model skips the JSON.
|
||||
|
||||
**One parser, `parseResponseMood` in `internal/phraser/parse.go`.** Every path
|
||||
reaches it: the six `LLMPhraser` methods, `PhraseWorld`, and
|
||||
`Replier.PhraseReply`. `cmd/mavend/replier_llm.go` wraps the last of those,
|
||||
holds the stub fallback, and parses nothing itself. Mood is a fixed enum.
|
||||
|
||||
The router prompt is a separate contract:
|
||||
|
||||
```text
|
||||
[{"intent":<enum>, key?, value?, text?, verb?}, ...]
|
||||
```
|
||||
|
||||
over 7 intents: `fact, reminder, note, query, act, chat, system`.
|
||||
`llm/check_prompt_parity.py` in the training workspace enforces that the Go
|
||||
prompt and the relabelling prompt stay identical. They diverged once, and the
|
||||
relabelled set then taught a head the Go router never asks for.
|
||||
|
||||
## Russian patterns: three mechanisms, no fourth
|
||||
|
||||
Hand-written Russian stem patterns were swept out on 2026-08-04 by the owner's
|
||||
call. A regex whose output is a fact or a route is the defect. A regex over
|
||||
structured input, such as HTML, MIME, JSON, a URL or an argv list, is not.
|
||||
|
||||
Before writing a Russian word list, pick one of these.
|
||||
|
||||
### `internal/lexicon`, for closed classes
|
||||
|
||||
`lexicon_ru_v1.json` holds interrogatives, capture verbs, reminder verbs,
|
||||
cardinals, day offsets, parts of day, weekdays, months and spoken hours.
|
||||
Editing a word is a data change and there is exactly one copy. Months used to
|
||||
live in three files and drifted between them.
|
||||
|
||||
Cardinals carry the oblique forms, because a spoken time declines. `в семь` and
|
||||
`к семи` are one hour.
|
||||
|
||||
### `internal/morph`, for grammar
|
||||
|
||||
From the vendored golem Russian dictionary. `IsVerbForm` and `SameWord`.
|
||||
|
||||
Lemma matching is BROADER than stem-plus-one-ending. A verb slot that means the
|
||||
imperative must be matched exactly. `говори` and `говорил` share a
|
||||
lemma and only one of them is a command
|
||||
(`cmd/mavend/quiet_toggle.go`).
|
||||
|
||||
### `cmd/mavend/topics.go` and the embedder, for open sets
|
||||
|
||||
Use these when the question is what a turn is ABOUT. Frozen seeds per subject
|
||||
plus a real `other` class, scored against the turn's own query vector.
|
||||
|
||||
Same shape as the personal boundary in `personalboundary.go`, with one
|
||||
difference. A topic must clear the runner-up by `topicMargin`, because a false
|
||||
claim here spends a network scan rather than one honest "не знаю". The old
|
||||
keyword tests stay as the offline floor.
|
||||
|
||||
### The ecosystem trio
|
||||
|
||||
Use it when the answer is not in the utterance at all. Identity is Nexus's,
|
||||
never a local pattern.
|
||||
|
||||
## Seeds are scoring data
|
||||
|
||||
Editing one moves a recogniser. Re-measure against the `TestONNX*` tests rather
|
||||
than eyeballing the change.
|
||||
@@ -105,6 +105,19 @@ how we find out whether the blind spot is real.
|
||||
untouched. The model, the context size, the layer count and the MTP flags are the
|
||||
owner's business and not this daemon's schema.
|
||||
|
||||
**The port carries a bearer token and cannot be loopback** (V-673). homesrv is
|
||||
the client, so this hop is on the LAN. Until 2026-08-11 anything on the network
|
||||
could spend the card, hold the model resident by touching the idle clock, and
|
||||
read `/slots`, which returns other callers' prompts. mavgpud now reads
|
||||
`token_file` and refuses to start when the listen address is reachable from the
|
||||
network without one. Downgrading to loopback instead would look safe and take
|
||||
the model arm down. Maven sends the same token from `workstation.token`, on the
|
||||
completion and on the `/health` probe alike. An unsigned probe answers 401,
|
||||
which Pair reads as a busy card, so a missing token degrades to the resident
|
||||
model rather than breaking a turn. The proxy also
|
||||
allowlists the five paths Maven calls, so a leaked token buys the model API and
|
||||
not llama-server's admin surface.
|
||||
|
||||
**Every GPU service on that box belongs under this supervisor**, added to
|
||||
`cmd/mavgpud` rather than to systemd beside it. The rule was learned on
|
||||
2026-08-09. The CW2 transcriber ran as its own user unit and registered on the
|
||||
|
||||
+474
@@ -0,0 +1,474 @@
|
||||
# Routing
|
||||
|
||||
*Last verified: 2026-08-11 @ 25ed201*
|
||||
|
||||
How an utterance becomes a `Decision`, why each stage exists, and what every
|
||||
stage has measured. `CLAUDE.md` carries the rules an agent must not break. This
|
||||
file carries the reasoning and the history behind them.
|
||||
|
||||
Two decisions come out of a route. **Intent** is one of seven values. **Source**
|
||||
is where the answer lives, and it is read on `IntentQuery` alone. They are scored
|
||||
separately, because one number hides which one moved.
|
||||
|
||||
## The cascade
|
||||
|
||||
| Stage | What it is | Where |
|
||||
|---|---|---|
|
||||
| 0 | Deterministic grammars over the utterance | `stage0.go`, `praxis.go`, `worldquery.go` |
|
||||
| 0b | Four ONNX heads on one e5-small forward pass | `heads.go` |
|
||||
| 1 | The resident model, GBNF-constrained JSON | `llmrouter.go` |
|
||||
| 2 | Nearest neighbour over frozen seed phrases | `classifier.go`, `embedder.go` |
|
||||
|
||||
Every stage may decline, and the next one answers. Any error at stage 0b or 1
|
||||
falls through, so a turn never breaks on a model.
|
||||
|
||||
The resident model arm is wired at `voice.go:214` through
|
||||
`pickLLMRouter(cfg.Voice.UseLLMRouter(), llmClient)`. The flag is
|
||||
`voice.llm_router` in `config.go`, `DefaultLLMRouter` is on, and
|
||||
`deploy/mavend.json` sets it `true`. With no llama-server to talk to,
|
||||
`pickLLMRouter` logs that and degrades to the classifier.
|
||||
|
||||
The classifier is the floor and not dead code. It runs when the resident model
|
||||
is off, when there is no llama-server to talk to, and on any per-turn error.
|
||||
Routing by seed similarity is the known cause of weak Russian queries. Deleting
|
||||
it would make a model outage a broken turn.
|
||||
|
||||
### Why there are two engines at all
|
||||
|
||||
The original design was the classifier alone. `docs/rearchitecture.md` replaced
|
||||
it with a model that emits structured JSON. The same weights phrase the reply.
|
||||
That demoted the embedder from a routing gate to a hint for recall. The model
|
||||
became the default on 2026-07-31.
|
||||
|
||||
The gap it buys is smaller than the design assumed. Measured 2026-08-02 on the
|
||||
77-case Russian fixture, the classifier scores 68.8% full accuracy at p50 16.6µs.
|
||||
Qwen3-1.7B scores 72.7% through the cascade. Four points, not a doubling.
|
||||
|
||||
An older figure of 36.8% for the classifier stood in `CLAUDE.md` until then. It
|
||||
predates the stage 0 rules and the seed additions. Both now score inside the
|
||||
classifier baseline.
|
||||
|
||||
Latency was misreported the same way. A figure of 2.7 seconds stood for two
|
||||
days and was contention rather than the model.
|
||||
`docs/evals/2026-07-31-routing.md` line 61 measures the router at p50 825ms and
|
||||
the cascade at p50 0.80s to 1.04s.
|
||||
|
||||
## Numbers
|
||||
|
||||
Three arms answer, so three numbers are live. Judge a routing change against the
|
||||
classifier and the resident model, since those are what always answer.
|
||||
|
||||
| Arm | Intent | Destination | p50 | Measured |
|
||||
|---|---|---|---|---|
|
||||
| classifier + ONNX | 76.0% (73/96) | 36.4% (12/33) | 16.6µs | 2026-08-08 |
|
||||
| resident Qwen3-1.7B, cascade | 80.2% | not measured | 1.19s | 2026-08-05 |
|
||||
| routing heads, cascade | 96.9% | 75.8% | 27.9ms | 2026-08-08 |
|
||||
| gemma-4-12b, cascade | 84.4% | 72.7% | 329ms | 2026-08-02 |
|
||||
| gemma-4-E4B, cascade | 89.6% | 57.6% (19/33) | 294ms | 2026-08-09 |
|
||||
|
||||
The fixture grew from 77 cases to 91 to 96. So a number is comparable only to
|
||||
another number on the same fixture. Sources:
|
||||
`docs/evals/2026-08-05-routing-resident-model.md`,
|
||||
`docs/evals/2026-08-02-workstation-gemma4-12b.md`,
|
||||
`docs/evals/2026-08-09-e4b-vs-12b-routing.md`,
|
||||
`docs/evals/2026-08-08-routing-heads-in-go.md`,
|
||||
`docs/evals/2026-08-08-destination-fixture.md`.
|
||||
|
||||
The resident model alone scores 37.4% full against 61.5% intent-only. The gap is
|
||||
slots and not routing. It routes `reminder` and leaves the time to the daemon,
|
||||
which is what the contract asks.
|
||||
|
||||
To re-run the resident model as router, start a **second** llama-server on a
|
||||
fixed host port. The resident one binds `--port 0` inside the container and no
|
||||
host process can reach it.
|
||||
|
||||
### The workstation is not the better router any more
|
||||
|
||||
It was, from 2026-08-02 until the heads landed. gemma-4-12b beat everything on
|
||||
the box at 84.4% intent and 72.7% destination. The heads beat it on both at a
|
||||
twelfth of the latency. The workstation stays the better phraser.
|
||||
|
||||
E4B replaced the 12B on 2026-08-09 by the owner's call. It is a step down on
|
||||
routing. Against a same-session 12B control it costs four destination cases and
|
||||
buys 50ms. Read destination as the finding. It names nothing where the 12B names
|
||||
`recall` or `calendar`, which is safe but walks the whole chain. It has no MTP
|
||||
and cannot be given any here. The only `gemma4-assistant` draft on disk is
|
||||
trained against the 12B's hidden states.
|
||||
|
||||
## Stage 0: what the grammars claim, and why
|
||||
|
||||
A rule at this stage is a claim. Either the model gets this wrong, or it wastes a
|
||||
second getting it right. Every rule was added against a measurement.
|
||||
|
||||
- **Agenda questions** (`AgendaQueryGrammars`, 2026-08-01). "что у меня сегодня",
|
||||
"во сколько у меня встреча" and anything naming a calendar go to `IntentQuery`.
|
||||
They were going to `IntentSystem`, where `replySystem` has no agenda arm and
|
||||
answered "пока не умею". Worth 2.6 points of full accuracy and calendar 0/2 to
|
||||
2/2.
|
||||
- **Rest of day and narrative** (V-498, 2026-08-04). `rest-of-day-query` claims
|
||||
"что дальше?". `NarrativeQueryGrammar` claims "расскажи про X", "объясни X" and
|
||||
"опиши X". Neither carries a question mark or an interrogative, so the model
|
||||
called both `IntentFact`. `IsQuestionShaped` caught the write downstream, so
|
||||
this was a latency and fixture defect rather than a correctness one. The
|
||||
narrative rule declines `chatNarrativeTopics`, because the query chain has no
|
||||
source that answers a joke or a bedtime story.
|
||||
- **Praxis** (V-516, 2026-08-05). `PraxisGrammars()` fills `Slots.Fn` with a
|
||||
capability name. These grammars are the **only** path to Praxis and not a
|
||||
faster one. The model reaches Praxis 0/12 alone, the same as the classifier.
|
||||
Nothing in the router prompt names a Praxis capability, so there is no string
|
||||
for it to write. Through the cascade it is 11/12. Measured overall 16/30 to
|
||||
27/30, lifecycle 0/5 to 5/5
|
||||
(`docs/evals/2026-08-05-praxis-reach.md`,
|
||||
`docs/evals/2026-08-05-reach-llm-router.md`).
|
||||
`handlePraxisAct` compares `Slots.Fn` to a capability alias. Otherwise that
|
||||
slot is filled from the deployment's enabled tool names, and no Praxis alias
|
||||
is on that list.
|
||||
- **World questions** (`WorldQueryGrammars`, V-655, 2026-08-07). "что такое X"
|
||||
and "сколько будет 17 на 23". Wired after the agenda rules and **before** the
|
||||
feed and list rules. "что такое лента" is a definition question, and the feed
|
||||
rule would take it on the noun alone.
|
||||
|
||||
`calendar-query` and `event-time-query` name the calendar as the destination.
|
||||
The possessive agenda rules deliberately do not. "что у меня в списке покупок"
|
||||
matches `agenda-query`, and naming the calendar there would take the list source
|
||||
off the turn. That caution now costs four destination cases. See the model arm
|
||||
below.
|
||||
|
||||
Go's `\b` is ASCII-only and never fires after a Cyrillic letter. A pattern needs
|
||||
an explicit `(\s|[?!.]|$)`.
|
||||
|
||||
The stage 0 set lives in `router.StageZeroGrammars` (`internal/router/stagezero.go`).
|
||||
Both `buildRouter` and the eval fixture call it. The daemon and the measurement
|
||||
cannot disagree about which rules exist, or in what order.
|
||||
|
||||
It was two lists until V-693 and it drifted twice. V-655 wired
|
||||
`WorldQueryGrammars` into the daemon and not into the fixture. That cost 3 points
|
||||
of destination and V-659 fixed it. `BareCaptureGrammar` then did the same thing,
|
||||
from V-557 until V-693 found it. That one moved no number, which is the point:
|
||||
the fixture had been scoring a set nobody ran and nothing said so.
|
||||
|
||||
### Praxis lifecycle rules
|
||||
|
||||
A **stative** lifecycle word ("готово", "принято") needs an item named beside it.
|
||||
A bare **imperative** ("закрывай") may ask which one. It also requires a sentence
|
||||
naming no object of its own. Otherwise "закрой шторы в комнате" goes to Praxis
|
||||
instead of the house. A demonstrative ("отметь это как сделанное") resolves
|
||||
against `h.surfacedItems` only when exactly one item was spoken. Otherwise the
|
||||
turn goes back to the cascade rather than transitioning the wrong item.
|
||||
|
||||
### Slots on a stage 0 decision
|
||||
|
||||
`fillMatchedSlots` runs the stage 2 extractor over whatever a grammar built
|
||||
(V-572, 2026-08-06). It fills only the slots the grammar left empty. A matched
|
||||
value always wins, because the rule read a literal pattern and the extractor
|
||||
guesses.
|
||||
|
||||
It did not run before. So `ReminderGrammar` handed the daemon `HasTime: false`
|
||||
for "напомни в 11:00 позвонить маме", and `missingFor` read the silence as
|
||||
absence and asked "Когда?". It is inert for every grammar but the reminder:
|
||||
`Extract` fills Time, Fn and Key and nothing else. A stage 0 query costs 3.7µs
|
||||
against 3.9µs before, benchmarked at 20000x.
|
||||
|
||||
`Slots.Text` is deliberately not filled. A grammar that left it empty meant it,
|
||||
and `agendaQueryBuild` hands the query chain the utterance itself.
|
||||
|
||||
## Stage 0b: the routing heads
|
||||
|
||||
Routing has a bounded output space, so it is classification rather than
|
||||
generation (owner's call, V-546,
|
||||
`docs/plans/18-routing-heads-on-e5-small.md`). The 118M multilingual-e5-small is
|
||||
already resident. A softmax cannot emit a value that does not exist, so no
|
||||
grammar is needed. Max softmax is a calibratable confidence, where
|
||||
`Confidence: 1.0` was a hardcode. Training costs roughly 5e15 FLOPs, so 10 to 30
|
||||
minutes on the workstation. A 100M decoder from scratch is 10 to 20 GPU hours.
|
||||
|
||||
**Fine-tune a copy of the weights.** The resident embedder backs memory recall.
|
||||
Training it in place couples routing accuracy to recall@1, with nothing in the
|
||||
suite to name the trade.
|
||||
|
||||
Four heads share one masked mean pool, trained over three days. The measurements
|
||||
are `docs/evals/2026-08-08-routing-heads-two-head.md`,
|
||||
`docs/evals/2026-08-08-slot-head-three-head.md`,
|
||||
`docs/evals/2026-08-08-clarify-head-four-head.md` and
|
||||
`docs/evals/2026-08-08-massive-warm-start.md`.
|
||||
|
||||
| Head | Score | Notes |
|
||||
|---|---|---|
|
||||
| intent | 92.8% mean over 3 seeds | fixture is the 88 cases carrying an intent |
|
||||
| destination | 80.8% mean, best 29/33 | beats the 12B teacher it was distilled from |
|
||||
| slot BIO tags | 72.4% span F1 | still climbing when epoch selection stops it |
|
||||
| clarify | catches 7.0 of 8, 2.3 false of 88 | parity with the cascade, no rules in front |
|
||||
|
||||
Read the best destination run as one seed and not a headline. One case is 3
|
||||
points on a fixture this small. Head intent accuracy is **not** comparable to the
|
||||
cascade's 76.0% and 84.4%. A softmax has no clarify class, so the head's fixture
|
||||
is 88 cases and not 96.
|
||||
|
||||
Recall is 15/15 and world is 5/5.
|
||||
|
||||
**Mood is cut, not deferred.** The enum describes her own reply state, not the
|
||||
speaker's emotion, and no dataset maps onto it.
|
||||
|
||||
### The clarify head
|
||||
|
||||
Clarify is not a value of intent, so a softmax cannot emit it. It is a second
|
||||
question over the same pooled vector: can Maven act on this at all. Accuracy is
|
||||
the wrong number here and a head that never asks scores 91.7%.
|
||||
|
||||
Confidence is the other half. Max softmax over the intent head reads 0.851 where
|
||||
it is right and 0.604 where it is wrong. It ranks right above wrong in 83.4% of
|
||||
pairs.
|
||||
|
||||
It is not free the way the slot head was. Intent, destination and slot F1 each
|
||||
move down one to four points, inside the seed spread. `поужинал` is a false
|
||||
clarify on every seed. That is the same defect `thinSingleToken` was narrowed for
|
||||
on 2026-08-01.
|
||||
|
||||
The corpus is generated, because every existing row is answerable by
|
||||
construction. The router-prompt agreement filter cannot work here. `routeGrammar`
|
||||
has no clarify value, and a generated line always agrees with itself. A gemma
|
||||
judge replaces it. The first judge called 24 of 40 answerable rows underspecified.
|
||||
It judged against a generic assistant rather than against Maven's contract.
|
||||
|
||||
### The slot head
|
||||
|
||||
BIO slot tags had no Maven-domain corpus. That was true of found corpora and
|
||||
false of made ones. `label_slots.py` distils spans out of gemma-4-12b under a
|
||||
GBNF closed over Maven's own five slots. A span survives only when it is a
|
||||
literal substring of the utterance, so the agreement filter costs no second call.
|
||||
2178 spans over 1702 rows, 37 dropped, nothing unparsed.
|
||||
|
||||
Epoch selection reads the intent dev slice alone. That costs the slot head about
|
||||
4 points.
|
||||
|
||||
### Warm start and the floor
|
||||
|
||||
The MASSIVE warm-start of step 2 is worth nothing here. Stock e5-small ties it on
|
||||
intent and leads by a third of a case on destination. Nothing argues for keeping
|
||||
that step.
|
||||
|
||||
The floor was a corpus defect and it is fixed. The first 120 floor rows carried
|
||||
one sentence shape, so the head named a destination where the fixture says walk
|
||||
the chain. Rotating six shapes took the floor 3/7 to 6/7 and destination 75.8% to
|
||||
80.8%.
|
||||
|
||||
What is left is calendar at 3/6 on every seed, which training cannot move. The
|
||||
possessive agenda rules claim those cases at stage 0 and name nothing, so no
|
||||
label reaches the head.
|
||||
|
||||
### Reading them in Go
|
||||
|
||||
`RouterHeads` in `internal/router/heads.go` loads `router_heads.onnx` (V-664,
|
||||
2026-08-08). It reads intent, destination and clarify off one forward pass.
|
||||
|
||||
Three rules around it, each measured:
|
||||
|
||||
- The **clarify head decides first**, before the intent threshold. It answers a
|
||||
different question. A thin utterance scores low intent by construction, so
|
||||
gating it cost 6 of 8 ambiguous cases.
|
||||
- The **destination head is read on `IntentQuery` only**, since no other intent
|
||||
reaches `queryWalk`.
|
||||
- `headsThreshold` is 0.6, the measured knee. Every value up to 0.85 drops right
|
||||
answers and keeps the same two wrong ones.
|
||||
|
||||
`voice.embedder.heads_path` is the whole switch. Empty, missing or unloadable
|
||||
means the heads are nil. The cascade is then byte-for-byte what shipped before
|
||||
them.
|
||||
|
||||
Pointing it at `model_path` is refused at config load (V-692). An unloadable
|
||||
weights file is not fatal, because the heads are an accelerator. A working file
|
||||
in the wrong role is a different thing. The heads then score with the graph the
|
||||
resident embedder scored with, and recall degrades with no log line. The check
|
||||
cleans and absolutises both paths, then compares them with `os.SameFile`, so a
|
||||
symlinked copy is caught too.
|
||||
|
||||
### The tokenizer bug the heads found
|
||||
|
||||
`encodeWord` in `onnxembedder.go` read every long word backwards until 2026-08-08.
|
||||
It cost recall@1 7.4 points and recall@3 11.1. Nothing caught it, because seeds
|
||||
and queries were mangled the same way and cosine survived. The heads found it.
|
||||
They are trained through transformers and read through this.
|
||||
|
||||
The embedder id now carries a tokenizer revision (`@384/tok2`). So fixing the
|
||||
tokenizer triggers `ReembedAll` the way swapping the model file does. Bump
|
||||
`tokenizerRev` on any change to what it emits.
|
||||
|
||||
## Clarify
|
||||
|
||||
`Confidence: 1.0` was hardcoded in `llmrouter.go`. So the model path could never
|
||||
ask for clarification, and it missed 6 of 6 refusal cases (V-359). The bug had a
|
||||
second half. The model branch never consulted `r.threshold` at all, so a correct
|
||||
low confidence would have been discarded anyway.
|
||||
|
||||
Fixed 2026-07-31 with structural signal feeding the same stage 3 gate the
|
||||
classifier path already had (`gateLLMDecision` in `router.go`). Three signals: a
|
||||
single-token utterance, a keyless fact, an act with no allowlisted fn.
|
||||
|
||||
Re-measured: missed clarify 6/6 to 1, at the cost of 3 false clarifies and 2.6
|
||||
points of full accuracy. Two of the three false clarifies are acts the model
|
||||
mis-routed and the gate caught. Asking beats wrongly executing, so the fixture
|
||||
and the daemon disagree about what is correct there.
|
||||
|
||||
The third, `поужинал`, was a real defect. The single-token rule was an English
|
||||
intuition. It does not transfer to Russian, where one word is routinely a whole
|
||||
sentence.
|
||||
|
||||
Narrowed 2026-08-01. `thinSingleToken` (`internal/router/singletoken.go`) still
|
||||
thins a bare one-word nominal. It spares two classes. One is a closed lexicon of
|
||||
social and control singles ("привет", "стоп", "yes"). The other is any token
|
||||
carrying a Russian verb ending, because a verb already contains its subject. Both
|
||||
tests are offline and cost nothing. False clarifies 3 to 2, intent-only 74.0% to
|
||||
75.3%.
|
||||
|
||||
The two remaining false clarifies are the act-with-no-allowlisted-fn arm of the
|
||||
gate, not this rule.
|
||||
|
||||
## The destination
|
||||
|
||||
`query` was a shrug. The cascade sorted an utterance into one of seven intents,
|
||||
then `IntentQuery` handed the turn to `querySources` in the daemon. That is
|
||||
twenty-two branches deciding by seed similarity in a fixed order. It had no
|
||||
fixture, no accuracy number, no model arm and no floor.
|
||||
|
||||
`Decision.Source` (`internal/router/source.go`) is the second half of the route
|
||||
(V-655, 2026-08-07). Twelve destinations, not twenty-two. The three recall passes
|
||||
plus `fact-by-key` are one destination from outside. So are search, Kiwix and the
|
||||
URL reader.
|
||||
|
||||
`queryWalk` in `cmd/mavend/actions_query.go` takes sources **out** and moves none.
|
||||
That is the safety argument. The table's order is load-bearing. Every comment on
|
||||
it argues a reason between two sources. Above all it carries "the owner's data
|
||||
first, then the world". Naming `SourceWorld` does not send the turn outside on its
|
||||
own.
|
||||
|
||||
What comes out is only the sources that **guess**. Those decide a turn is theirs
|
||||
by cosine against frozen seeds, then answer whatever they claimed. They hold no
|
||||
table that could come back empty. Weather is the pure case and has no local data
|
||||
at all. It was measured on the box 2026-08-07
|
||||
(`docs/evals/2026-08-07-week-of-usage.md` section 4). It answered both "что такое
|
||||
TCP?" and "сколько будет 17 на 23?" with "для какого города?". The feed answered
|
||||
"какой у меня любимый язык?" with kernel headlines.
|
||||
|
||||
### Who may drop the personal boundary
|
||||
|
||||
The personal boundary guesses, so naming `SourceWorld` drops it. That is what
|
||||
stops it answering "кто такой Линус Торвальдс?" with "не нашла у тебя такой
|
||||
записи", which it did on 2026-08-07.
|
||||
|
||||
Three deciders name a destination and two of them infer it: the heads and the
|
||||
resident model. An inferred `SourceWorld` on a question about him would reach
|
||||
SearXNG. That widens what is asked rather than costing a local answer. So only a
|
||||
stage 0 grammar may drop it (owner's call, V-666, 2026-08-09).
|
||||
|
||||
`Decision.SourceAnchored` carries the provenance. It is a field and not
|
||||
`Stage == 0`. Stage 0 also means confidence 1.0 and an anchored claim band, and
|
||||
one of those could stop implying the others. `definitionQueryPattern` claims "кто
|
||||
такой X", so the 2026-08-07 case is still anchored and still answered.
|
||||
|
||||
`queryWalk` reads `SourceAnchored` for the query source marked `boundary: true`
|
||||
and no other. Every other guesser still comes off the turn, whoever named the
|
||||
destination. `TestOnlyAGrammarMayDropTheBoundary` and
|
||||
`TestNamingRecallKeepsTheBoundary` pin both directions.
|
||||
|
||||
### The destination fixture
|
||||
|
||||
`want_source` on `eval.Case` is a pointer, because the destination has three
|
||||
states and a bare string has two. Absent is every intent but query. Present and
|
||||
empty is the `SourceUnknown` contract: name nothing and walk the chain. Present
|
||||
and named is a destination the route must produce. Thirty-three of ninety-six
|
||||
cases carry one.
|
||||
|
||||
A destination miss does **not** fail the case. It lands in `Outcome.SourceReason`
|
||||
and never in `Reasons`, so `Accuracy` and `IntentAccuracy` mean what they meant.
|
||||
`SourceAccuracy` is a second number over the labelled cases alone.
|
||||
A route that lost its intent scores no destination hit. Otherwise a clarify would
|
||||
satisfy an empty label for free.
|
||||
|
||||
Seven cases assert the floor and five of them are homelab operations. They cluster
|
||||
because `SourceRecall`, `SourceNetwork` and `SourceAttention` overlap on every
|
||||
question about the box. `mavpoll` writes its netdata and uptime-kuma observations
|
||||
into the fact store recall reads. That is a finding about the enum, not a gap in
|
||||
the labelling. The other two are `ru-query-005` and `ru-query-014`. No query
|
||||
source reads the reminder store, and a deadline could sit in tasks, the calendar
|
||||
or Praxis. The owner confirmed all seven floor labels on 2026-08-08.
|
||||
|
||||
### The model arm
|
||||
|
||||
`routeGrammar` carries a `source` rule closed over `router.Sources` plus the
|
||||
empty floor (V-660, 2026-08-08). So the model cannot emit a destination that does
|
||||
not exist. The prompt lists the twelve in Russian and says `""` is a normal answer
|
||||
to give often. `LLMRouter.Route` reads it back through `ValidSource` and on
|
||||
`IntentQuery` alone.
|
||||
|
||||
Against gemma-4-12b the cascade scores destination 24/33 with intent unmoved, and
|
||||
recall goes 0/15 to 14/15.
|
||||
|
||||
**Stage 0 now costs four destination points.** It did not before. The four cases
|
||||
the cascade loses and the model alone wins are all calendar. The possessive agenda
|
||||
rules claim them first and name nothing on purpose. That caution was free while
|
||||
nothing downstream could name anything either. It is not free now, and the fix is
|
||||
the owner's call (V-660 open).
|
||||
|
||||
## The decision trace
|
||||
|
||||
Arbitration between the claimants on the utterance stream is order. It is
|
||||
hardcoded in the pre-route resolver ladder, in `buildRouter` and in
|
||||
`querySources`. Nothing recorded who lost until V-564.
|
||||
|
||||
`internal/decision` records one `Record` per turn. It holds every claimant, what
|
||||
it would have made the turn, the score it reported, and how it ended. A claimant
|
||||
won, declined, lost on score, was thinned by a gate or was **never asked**.
|
||||
|
||||
The record rides the context, the same seam `querysource.go` uses. So a claim
|
||||
site cannot change a route, and a context with no record costs nothing. It is
|
||||
installed in `runTurn`, so the mic, telegram and the web leave the same trail.
|
||||
|
||||
Adding a rung to the ladder in `runTurn` means adding its name to `preRouteLadder`
|
||||
in `cmd/mavend/decisiontrace.go`. Otherwise that rung is silently missing from the
|
||||
record.
|
||||
|
||||
### Why it persists now
|
||||
|
||||
The original rule was that nothing persists, because a turn record is read minutes
|
||||
later or never. Storage was a 25-turn in-memory ring read over `ipc.TurnDecisions`
|
||||
and rendered on `/trace`.
|
||||
|
||||
The owner reversed it on 2026-08-06 (V-629,
|
||||
`docs/plans/21-persisting-the-routing-trace.md`). The routing heads cannot be
|
||||
fitted or calibrated without real utterances. And 9 of the 31 modes in
|
||||
`internal/modes` have no seed example at all.
|
||||
|
||||
The ring did not move. `cmd/mavend/routingtrace.go` is a second sink beside it,
|
||||
writing `routing_traces` (migration #23). The utterance is stored in clear. A
|
||||
384-dimension vector of a short sentence is substantially recoverable, so storing
|
||||
vectors instead would be a privacy claim we cannot support. What makes it safe is
|
||||
the same thing that makes the fact store safe. Retention is 14 days, enforced on
|
||||
write and again on start, so a box that goes quiet does not keep every row.
|
||||
Nothing reads it outward. `Store.Wipe` deletes it with everything else.
|
||||
|
||||
### Corrections
|
||||
|
||||
A correction is promoted out into a seed-shaped row in `routing_labels`
|
||||
(migration #24) and kept, because a label is not a transcript. The transcript
|
||||
still expires.
|
||||
|
||||
A turn marked wrong with no target is a usable negative, so naming the intent is
|
||||
never required. The target is one of the seven intents and never free text.
|
||||
|
||||
All three reaches offer it as of 2026-08-06:
|
||||
|
||||
- `/chat` offers two buttons beside the reply, over `ipc.CorrectTurn` and the
|
||||
trace id that rides back on `ipc.ChatReply`.
|
||||
- Voice offers the `repair` rung, which has read spoken corrections since V-455.
|
||||
It now writes the durable label beside the classifier seed it always wrote. A
|
||||
spoken negative with no target is its own rung, `repair-negative` (V-636,
|
||||
`docs/plans/22-correcting-a-turn.md`).
|
||||
- Telegram offers an inline keyboard under the reply. It needed the chat to become
|
||||
readable first (V-637, `docs/plans/23-inbound-telegram.md`). The poller is dark
|
||||
unless the `telegram` block says `intake`. It long-polls, because the box takes
|
||||
no inbound connections. It accepts `chat_id` and no other sender, and it drops
|
||||
whatever queued while the daemon was down. It reaches the daemon through
|
||||
`ipc.CoreAPI` alone.
|
||||
|
||||
The turn source is still `tap:text` for both telegram and the web. So provenance
|
||||
cannot tell a chat turn from a typed one.
|
||||
@@ -0,0 +1,117 @@
|
||||
# Session workflow: the five stores and the guards
|
||||
|
||||
*Last verified: 2026-08-11 @ 557f5a3*
|
||||
|
||||
How a session starts, where each kind of writing belongs, and what the hooks
|
||||
refuse. `CLAUDE.md` carries the commands. This file carries the reasoning.
|
||||
|
||||
## Five stores
|
||||
|
||||
Each owns something the others must not hold.
|
||||
|
||||
| Store | Holds | Lifetime |
|
||||
|---|---|---|
|
||||
| Vikunja task | goal, constraints, assumption ledger, status | durable |
|
||||
| `CLAUDE.md`, `AGENTS.md` | what an agent must know before touching code | durable |
|
||||
| `docs/` | design, measurements, decisions | durable |
|
||||
| `TASK.md` | the brief for this branch, written by `task start`, immutable | one branch |
|
||||
| `HANDOFF.md` | only what the next agent needs to resume | one session |
|
||||
|
||||
`TASK.md` and `.task/` are excluded through `.git/info/exclude`. `HANDOFF.md` is
|
||||
gitignored and injected at session start. If a line in the handoff would still
|
||||
matter next week, it is in the wrong file.
|
||||
|
||||
## Doc tiers
|
||||
|
||||
Tiered by path, so staleness is visible from the filename.
|
||||
|
||||
- Files directly under `docs/` are living. They carry a
|
||||
`Last verified: <date> @ <sha>` line and are corrected in place.
|
||||
- Files under `docs/evals/` are dated measurements and are never edited after
|
||||
the day. A newer number is a new file, not an edit.
|
||||
- Files under `docs/archive/` are dead and read by nobody by default.
|
||||
|
||||
## Vikunja
|
||||
|
||||
This repo is project **Maven** (ID 2). MCP at `http://localhost:9100/mcp`, or
|
||||
`http://192.168.1.104:9100/mcp` from workpc. Feature, bug and deploy tasks go
|
||||
there.
|
||||
|
||||
A task holds the goal, the constraints and the assumption ledger. A session
|
||||
without a task id cannot be resumed by anyone, so a session with none asks for
|
||||
one first.
|
||||
|
||||
**Close a finished task with `done: true` and nothing else** (owner's call,
|
||||
2026-08-07). Do not write a completion summary into the description on the way
|
||||
out. It is lost anyway, and the durable record is the commit messages and the
|
||||
merged PR. `update_task` carrying a `description` resets `done` to false, which
|
||||
is why a write-up ever took two calls.
|
||||
|
||||
## The branch tool
|
||||
|
||||
`~/.local/bin/task` owns the branch, the commit identity and the PR. One task,
|
||||
one session, one PR.
|
||||
|
||||
```sh
|
||||
task start <vikunja-id> # branch off origin/master, write TASK.md, fetch review comments
|
||||
task pr # push, open or refresh the PR, label Vikunja, notify
|
||||
task comments # re-pull this branch's review comments into .task/
|
||||
```
|
||||
|
||||
`/pickup` opens a session and `/wrap` closes it. Wrap at roughly half context
|
||||
rather than letting the session compact.
|
||||
|
||||
## Guards
|
||||
|
||||
Two hooks in `.githooks/`, tracked, wired with `core.hooksPath`. A fresh clone
|
||||
needs `git config core.hooksPath .githooks`.
|
||||
|
||||
- `pre-commit` refuses master, and refuses more than 300 changed lines in
|
||||
non-markdown files. Markdown is exempt and may land as one batch.
|
||||
- `commit-msg` requires the subject to end with `(V-<id>)`. `V-` and not `#`,
|
||||
because Gitea autolinks `#123` to a Gitea issue, which is the wrong tracker.
|
||||
|
||||
Two more guards live outside the repo, in `~/.claude/hooks/`. `diff-budget.sh`
|
||||
blocks further edits past 600 changed lines on a `task/` branch.
|
||||
`prose_lint_hook.py` checks prose on every write. Both measure against
|
||||
`origin/master`, so a local master that is ahead of the remote makes the diff
|
||||
budget read high.
|
||||
|
||||
`--no-verify` exists. Using it means saying why in the commit body.
|
||||
|
||||
## Static gates
|
||||
|
||||
Three analyzers, one target each, and `make analyze` for all three. The
|
||||
2026-08-10 audit asked for them because none was installed on the box and
|
||||
`make audit` is a git-grep inventory, not analysis. Do not read `make audit` as
|
||||
a gate.
|
||||
|
||||
- `make vuln`, govulncheck over `./...` (V-682).
|
||||
- `make lint`, staticcheck over `./...` (V-694).
|
||||
- `make deadcode`, deadcode with `-test` over `./...` (V-694).
|
||||
|
||||
None of the three joins `make test`. All three install over the network, and
|
||||
`test` has to pass on a box with no route out. `vuln` reads the advisory
|
||||
database at run time as well. Run `make analyze` before a dependency or
|
||||
toolchain bump lands, and before calling a symbol unreachable.
|
||||
|
||||
Each tool is pinned in the Makefile beside `GO_VERSION`. A gate that moves on
|
||||
its own is not a gate. Each installs into `deps/bin`, because a tool is not a
|
||||
dependency of the module.
|
||||
|
||||
**staticcheck and deadcode pass against a baseline, not against zero.** The
|
||||
accepted findings live in `scripts/analyzers/*.baseline`, one line each. A key
|
||||
holds file, check id and message, never a line number. A line number goes stale
|
||||
on the next edit above it. The output then reports moved findings as new ones,
|
||||
and the reader learns to skip it.
|
||||
|
||||
`scripts/analyzer-gate.sh` gives the verdict. A finding absent from the baseline
|
||||
fails. So does a baseline entry whose finding is gone, which is what stops the
|
||||
accepted set from outliving the repo. Deleting the entry is part of each fix.
|
||||
|
||||
`deadcode` runs with `-test` because a test is a caller. Without the flag the
|
||||
report is 172 lines, most of `internal/router/eval`, none of it a mistake.
|
||||
|
||||
A baseline entry carries the reason it stays. Three reasons appear. Another task
|
||||
owns the finding. The check cannot see through a false positive. A cosmetic
|
||||
finding waits for a sweep.
|
||||
@@ -0,0 +1,71 @@
|
||||
# The world chain
|
||||
|
||||
*Last verified: 2026-08-09 @ a9b480a*
|
||||
|
||||
What happens when the answer is not his. `CLAUDE.md` carries the boundary rule.
|
||||
This file carries the mechanism and the measurements behind it.
|
||||
|
||||
## What replaced "never phones home"
|
||||
|
||||
That promise was deprecated on 2026-07-31 by the owner's call. A 1.7B does not
|
||||
know enough to answer world questions, so she reads external sources. Four rules
|
||||
replaced it:
|
||||
|
||||
- **No telemetry, no cloud model, no third-party account.** That part never
|
||||
changes. Nothing about Maven is reported to anyone and inference stays on the
|
||||
box.
|
||||
- **The owner's data first, then the world.** Every source reading his facts,
|
||||
notes, calendar, tasks or house runs before anything outside. The personal
|
||||
boundary sits between them. Reading beats recalling for a small model.
|
||||
- **The owner's notes and facts are never search input.** Only the utterance
|
||||
goes out. Never the persona block, the history, or matched notes.
|
||||
- **External search is allowed and off unless configured**, like weather and
|
||||
telegram. The code default is off. `deploy/mavend.json` ships a `search`
|
||||
block, so it is on for this box and deleting the block turns it off again.
|
||||
|
||||
**Live search leads and the ZIMs are the fallback** (owner's call, 2026-08-02).
|
||||
A self-hosted SearXNG answers first. The Kiwix ZIMs on homesrv answer when the
|
||||
search is empty, unreachable, or the line is down.
|
||||
|
||||
## The gate is emptiness and nothing else
|
||||
|
||||
`Response.Empty()` is the whole gate. There is no quality threshold in front of
|
||||
it. Four signals were tried and none separates a real question from an invented
|
||||
one.
|
||||
|
||||
Token overlap was the closest and it still fails. "столица Франции" would lose
|
||||
its answer, because the answer is Париж and that word is not in the question
|
||||
(`docs/evals/2026-08-05-search-quality-signals.md`).
|
||||
|
||||
**The embedder is not a fifth signal.** Query-to-passage cosine measures topic
|
||||
and not whether the passage answers. The two sets overlap
|
||||
(`docs/evals/2026-08-09-kiwix-topic-retrieval.md`).
|
||||
|
||||
## Timeouts
|
||||
|
||||
The connect phase alone is capped at `dialTimeout`, 1.5s, because a blackholed
|
||||
host once cost the owner 8 seconds. A slow instance that did connect keeps the
|
||||
full 8 (`docs/evals/2026-08-05-kiwix-offline-fallback.md`).
|
||||
|
||||
## Kiwix
|
||||
|
||||
**A Russian question reads `wikipedia_ru_all_maxi_2026-02` verbatim** through
|
||||
`kiwix.book_ru`. The RU→EN rewriter is the workaround for an English book and is
|
||||
skipped there. Kiwix catalog names come from the filename, not the `<name>`
|
||||
field.
|
||||
|
||||
**Kiwix ranks by keyword overlap.** Never send it a whole sentence. `kiwix.Topic`
|
||||
drops the narrative request, the interrogative and a verb behind one.
|
||||
|
||||
`kiwix.TitlePath` tries the exact article first, since a ZIM is addressable by
|
||||
title and a wrong title is a 404. `TitleCandidates` tries the spoken form and
|
||||
then the capitalized one. Both apply on the verbatim path alone. The rewriter
|
||||
already reduces a question, and reducing twice takes the topic off its input
|
||||
(V-668).
|
||||
|
||||
## Which source answered
|
||||
|
||||
**The claiming query source is readable on `/chat`** as a badge beside the
|
||||
reply. It is carried on `ipc.ChatReply.Source` and noted by `noteQuerySource` in
|
||||
`cmd/mavend/querysource.go`. It rides the context, so `handleText` keeps the one
|
||||
string signature the mic, telegram and the web share.
|
||||
@@ -1,6 +1,6 @@
|
||||
module github.com/kami/maven
|
||||
|
||||
go 1.25.5
|
||||
go 1.25.12
|
||||
|
||||
require (
|
||||
github.com/coder/websocket v1.8.12
|
||||
@@ -22,7 +22,7 @@ require (
|
||||
github.com/mattn/go-isatty v0.0.20 // indirect
|
||||
github.com/ncruces/go-strftime v1.0.0 // indirect
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
|
||||
golang.org/x/text v0.14.0
|
||||
golang.org/x/text v0.40.0
|
||||
modernc.org/libc v1.74.1 // indirect
|
||||
modernc.org/mathutil v1.7.1 // indirect
|
||||
modernc.org/memory v1.11.0 // indirect
|
||||
|
||||
@@ -25,13 +25,13 @@ github.com/yalue/onnxruntime_go v1.31.0 h1:1ln4YW1SFOFfGJZXe3jNOb2JUSt+l2pEneZfV
|
||||
github.com/yalue/onnxruntime_go v1.31.0/go.mod h1:b4X26A8pekNb1ACJ58wAXgNKeUCGEAQ9dmACut9Sm/4=
|
||||
golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ=
|
||||
golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0=
|
||||
golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM=
|
||||
golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
|
||||
golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
|
||||
golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
|
||||
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
|
||||
golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
|
||||
golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
|
||||
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
|
||||
golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
|
||||
golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
|
||||
golang.org/x/tools v0.47.0 h1:7Kn5x/d1svx/PzryTsqeoZN4TZwqeH5pGWjefhLi/1Q=
|
||||
golang.org/x/tools v0.47.0/go.mod h1:dFHnyTvFWY212G+h7ZY4Vsp/K3U4/7W9TyVaAul8uCA=
|
||||
modernc.org/cc/v4 v4.29.0 h1:CXgwL8cvxmyzBQZzbSl/6xFtMCryb6u8IOqDci39cgc=
|
||||
|
||||
@@ -2,6 +2,9 @@ package config
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"time"
|
||||
)
|
||||
|
||||
@@ -141,10 +144,55 @@ func (c *Config) validateVoice() error {
|
||||
if e.ModelPath == "" || e.TokenizerPath == "" || e.LibPath == "" {
|
||||
return errors.New("voice.embedder: all three of model_path, tokenizer_path, lib_path must be set, or remove embedder to use the floor stub")
|
||||
}
|
||||
if err := e.checkHeadsDistinct(); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// checkHeadsDistinct refuses a heads graph that is the embedder's own file
|
||||
// (V-692). The rule is stated on HeadsPath above and in CLAUDE.md, and until
|
||||
// now nothing enforced it: the daemon loaded whatever the key pointed at, so
|
||||
// pointing both keys at one file cost recall with no error and no log line. It
|
||||
// reads as ordinary drift, which is the worst kind of misconfiguration.
|
||||
//
|
||||
// Refusing to start is the right trade here. The heads are an accelerator and a
|
||||
// broken weights file is deliberately not fatal in voicewire.go, but this is not
|
||||
// a broken file. It is a working file in the wrong role, and a daemon that
|
||||
// cannot route well should say so rather than answer worse.
|
||||
//
|
||||
// Cleaned and made absolute first, so "./m.onnx" and "$PWD/m.onnx" are one
|
||||
// path. Then SameFile, which catches the copy that is a symlink or a hard link
|
||||
// to the original. A path that does not stat is left to the loader, which fails
|
||||
// on it with a better message than this can give.
|
||||
func (e *EmbedderConfig) checkHeadsDistinct() error {
|
||||
if e.HeadsPath == "" || e.ModelPath == "" {
|
||||
return nil
|
||||
}
|
||||
heads, model := absClean(e.HeadsPath), absClean(e.ModelPath)
|
||||
same := heads == model
|
||||
if !same {
|
||||
hi, herr := os.Stat(heads)
|
||||
mi, merr := os.Stat(model)
|
||||
same = herr == nil && merr == nil && os.SameFile(hi, mi)
|
||||
}
|
||||
if same {
|
||||
return fmt.Errorf("voice.embedder: heads_path and model_path are the same file (%s) — the heads graph is a fine-tuned copy, and scoring recall with it degrades what the resident embedder already stored", heads)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// absClean — the comparable form of a path. Abs fails only when the working
|
||||
// directory is unreadable, and a cleaned relative path is still worth comparing,
|
||||
// so the error falls back rather than propagating.
|
||||
func absClean(p string) string {
|
||||
if abs, err := filepath.Abs(p); err == nil {
|
||||
return abs
|
||||
}
|
||||
return filepath.Clean(p)
|
||||
}
|
||||
|
||||
// VoiceConfig — the client↔core TCP surface + the stt/tts worker-module
|
||||
// seams.
|
||||
//
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The heads graph is a fine-tuned copy of the embedder, and pointing both keys
|
||||
// at one file degrades recall with no error and no log line (V-692). These are
|
||||
// the shapes that used to boot clean.
|
||||
func TestHeadsPathMustNotBeTheModelFile(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
model := filepath.Join(dir, "model.onnx")
|
||||
heads := filepath.Join(dir, "heads.onnx")
|
||||
for _, p := range []string{model, heads} {
|
||||
if err := os.WriteFile(p, []byte("onnx"), 0o600); err != nil {
|
||||
t.Fatalf("write %s: %v", p, err)
|
||||
}
|
||||
}
|
||||
link := filepath.Join(dir, "link.onnx")
|
||||
if err := os.Symlink(model, link); err != nil {
|
||||
t.Fatalf("symlink: %v", err)
|
||||
}
|
||||
|
||||
// A path that stats and one that does not, because the guard compares the
|
||||
// cleaned string before it stats anything.
|
||||
for name, headsPath := range map[string]string{
|
||||
"the same path": model,
|
||||
"a symlink to it": link,
|
||||
"an uncleaned path": filepath.Join(dir, ".", "sub", "..", "model.onnx"),
|
||||
"a path on no disk": filepath.Join(dir, "absent.onnx"),
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
same := headsPath != filepath.Join(dir, "absent.onnx")
|
||||
err := voiceConfigWith(t, model, headsPath)
|
||||
if same && err == nil {
|
||||
t.Fatal("want a startup error, got a daemon that routes worse in silence")
|
||||
}
|
||||
if same && !strings.Contains(err.Error(), "heads_path") {
|
||||
t.Fatalf("the error does not name the key: %v", err)
|
||||
}
|
||||
if !same && err != nil {
|
||||
t.Fatalf("a distinct heads_path was refused: %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
if err := voiceConfigWith(t, model, heads); err != nil {
|
||||
t.Fatalf("two distinct files were refused: %v", err)
|
||||
}
|
||||
if err := voiceConfigWith(t, model, ""); err != nil {
|
||||
t.Fatalf("no heads at all was refused: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// voiceConfigWith loads a minimal enabled voice block through the real Load, so
|
||||
// the test exercises the startup path and not just the check in isolation.
|
||||
func voiceConfigWith(t *testing.T, model, heads string) error {
|
||||
t.Helper()
|
||||
body := `{"voice":{"enabled":true,"bind":"127.0.0.1:9100","embedder":{` +
|
||||
`"model_path":"` + model + `","tokenizer_path":"/t.json","lib_path":"/l.so"`
|
||||
if heads != "" {
|
||||
body += `,"heads_path":"` + heads + `"`
|
||||
}
|
||||
body += `}}}`
|
||||
_, err := Load(writeConfig(t, body))
|
||||
return err
|
||||
}
|
||||
@@ -27,6 +27,13 @@ type WorkstationConfig struct {
|
||||
// signal, so it must be the supervisor's endpoint and not llama-server's.
|
||||
Health string `json:"health,omitempty"`
|
||||
|
||||
// Token — the bearer credential mavgpud requires, expanded from the
|
||||
// environment like every other secret here. It is what stops anything on
|
||||
// the LAN spending the card, so a URL that is not loopback needs one.
|
||||
// Wrong or missing reads as a workstation that is down, and Maven falls
|
||||
// back to the resident model.
|
||||
Token string `json:"token,omitempty"`
|
||||
|
||||
// Probe — how often admission is re-checked. 0 ⇒ DefaultWorkstationProbe.
|
||||
// Nothing on the hot path waits for it: the answer is cached and read
|
||||
// atomically, so this only sets how late Maven notices the card came back.
|
||||
|
||||
@@ -51,6 +51,10 @@ type Client struct {
|
||||
base string
|
||||
swap SwapGate
|
||||
http *http.Client
|
||||
// token — the bearer credential for a server that asks for one. Empty for
|
||||
// the resident model, which is reached over loopback on the same box.
|
||||
// mavgpud on the workstation requires it: that hop is on the LAN.
|
||||
token string
|
||||
|
||||
// gate / background — priority on the single llama-server slot. Set once
|
||||
// at wiring time (SetGate), read on every request. nil gate ⇒ no gating,
|
||||
@@ -101,6 +105,27 @@ func New(baseURL string, timeout time.Duration) *Client {
|
||||
return &Client{base: baseURL, http: &http.Client{Timeout: timeout}}
|
||||
}
|
||||
|
||||
// SetToken installs the bearer credential this client sends. Wiring-time, like
|
||||
// SetGate: an empty token means the server is not asking for one.
|
||||
func (c *Client) SetToken(t string) {
|
||||
c.mu.Lock()
|
||||
c.token = t
|
||||
c.mu.Unlock()
|
||||
}
|
||||
|
||||
// authorize adds the credential when there is one. Exported to the package so
|
||||
// the Pair prober signs /health with the same token as the completion — a
|
||||
// probe that answers 401 would otherwise read as a workstation that is down,
|
||||
// and Maven would fall back forever without saying why.
|
||||
func (c *Client) authorize(req *http.Request) {
|
||||
c.mu.RLock()
|
||||
t := c.token
|
||||
c.mu.RUnlock()
|
||||
if t != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+t)
|
||||
}
|
||||
}
|
||||
|
||||
// SetBaseURL re-points the client at another llama-server. Safe to call while
|
||||
// requests are in flight: a request that already read the old base finishes
|
||||
// against the old base (or fails, and every caller of Complete has a fallback),
|
||||
@@ -196,6 +221,7 @@ func (c *Client) Complete(ctx context.Context, r Req) (string, error) {
|
||||
return "", err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
c.authorize(req)
|
||||
httpResp, err := c.http.Do(req)
|
||||
if err != nil {
|
||||
return "", err
|
||||
|
||||
@@ -132,6 +132,9 @@ func (p *Pair) probe(ctx context.Context) {
|
||||
p.set(false)
|
||||
return
|
||||
}
|
||||
if p.remote != nil {
|
||||
p.remote.authorize(req)
|
||||
}
|
||||
resp, err := p.http.Do(req)
|
||||
if err != nil {
|
||||
p.set(false)
|
||||
|
||||
@@ -250,3 +250,35 @@ func TestNoFloorIsAnError(t *testing.T) {
|
||||
t.Fatalf("err = %v, want ErrNoFloor", err)
|
||||
}
|
||||
}
|
||||
|
||||
// The workstation is behind mavgpud, which requires a bearer token on the
|
||||
// completion and on /health alike. A probe that did not carry it would answer
|
||||
// 401, Pair would read that as a card that is busy, and every turn would fall
|
||||
// back to the resident model with nothing in the log naming why.
|
||||
func TestPairSignsTheProbeAndTheCompletion(t *testing.T) {
|
||||
seen := make(chan string, 2)
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
seen <- r.Header.Get("Authorization")
|
||||
if r.URL.Path == "/health" {
|
||||
return
|
||||
}
|
||||
_, _ = w.Write([]byte(`{"choices":[{"message":{"content":"ok"}}]}`))
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
remote := New(srv.URL, time.Second)
|
||||
remote.SetToken("s3cret")
|
||||
p := NewPair(remote, New(srv.URL, time.Second), srv.URL+"/health", time.Hour)
|
||||
p.probe(context.Background())
|
||||
if !p.Available() {
|
||||
t.Fatal("the probe did not admit an answering workstation")
|
||||
}
|
||||
if _, err := p.Complete(context.Background(), Req{User: "привет"}); err != nil {
|
||||
t.Fatalf("complete: %v", err)
|
||||
}
|
||||
for i := 0; i < 2; i++ {
|
||||
if got := <-seen; got != "Bearer s3cret" {
|
||||
t.Errorf("request %d carried %q, want the bearer token", i, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -255,35 +255,12 @@ func newBaselineClassifier(t *testing.T, emb router.Embedder) *router.Classifier
|
||||
return cls
|
||||
}
|
||||
|
||||
// baselineGrammars — the stage-0 rule set in the daemon's order (buildRouter in
|
||||
// cmd/mavend/voicewire.go). Split out of newBaselineRouter so the claim
|
||||
// measurement can run the same rules one at a time and see which of them
|
||||
// contend for the same utterance, which the cascade hides by stopping at the
|
||||
// first match.
|
||||
// baselineGrammars — the stage 0 rule set the daemon runs, from the one place
|
||||
// it is written down (V-693). It used to restate the list by hand, and by the
|
||||
// time the guard was written the two had already drifted by one grammar.
|
||||
// Kept as a name because the claim measurement reads it as "the baseline set".
|
||||
func baselineGrammars(acts router.ActMatcher) []router.Grammar {
|
||||
grammars := router.DefaultGrammars(acts)
|
||||
grammars = append(grammars, router.SystemTimeDateGrammars()...)
|
||||
// Same order as buildRouter (voicewire.go). The fixture is only worth
|
||||
// anything while its grammar set is the daemon's grammar set.
|
||||
grammars = append(grammars, router.AgendaQueryGrammars()...)
|
||||
// After the agenda rules and before the feed and list rules, same as
|
||||
// voicewire.go: "что такое лента" is a definition question and the feed
|
||||
// rule would claim it on the noun alone (V-655). Missing here until V-659,
|
||||
// so the fixture was scoring a grammar set the daemon does not run.
|
||||
grammars = append(grammars, router.WorldQueryGrammars()...)
|
||||
grammars = append(grammars, router.FeedQueryGrammar())
|
||||
// The list side of the same exposure: a phrasing with no possessive in it
|
||||
// ("список дел") routed system and never reached queryTasks (Vikunja #467).
|
||||
grammars = append(grammars, router.TaskListGrammar())
|
||||
grammars = append(grammars, router.ListGrammars()...)
|
||||
grammars = append(grammars, router.ReminderGrammar())
|
||||
grammars = append(grammars, router.PraxisGrammars()...)
|
||||
grammars = append(grammars, router.TaskStatusGrammar())
|
||||
grammars = append(grammars, router.TaskCaptureGrammar())
|
||||
// "расскажи про X" is a world question the model called a fact, and the
|
||||
// rule goes last because it matches on the first word alone (Vikunja #498).
|
||||
grammars = append(grammars, router.NarrativeQueryGrammars()...)
|
||||
return grammars
|
||||
return router.StageZeroGrammars(acts)
|
||||
}
|
||||
|
||||
// seedOrder — fixed iteration order over the corpus. Not cosmetic: a few
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
package router
|
||||
|
||||
// StageZeroGrammars — the stage 0 rule set, in the order the daemon runs it.
|
||||
//
|
||||
// It lives here because it used to live in two places (V-693). `buildRouter` in
|
||||
// cmd/mavend/voicewire.go held the real set and `baselineGrammars` in
|
||||
// internal/router/eval/eval_test.go restated it by hand, in the daemon's order,
|
||||
// with its own comment saying so. Three test files score against the fixture,
|
||||
// and nothing compared the two lists. By 2026-08-11 they had already drifted:
|
||||
// BareCaptureGrammar was in the daemon and not in the fixture, so every routing
|
||||
// measurement scored a set nobody ran. That is the failure CLAUDE.md warned
|
||||
// about by name, and a diff test would have caught it one grammar late. One
|
||||
// list cannot drift from itself.
|
||||
//
|
||||
// The order is the contract, not the membership. Each rule below says why it
|
||||
// sits where it sits, and a rule inserted in the wrong place changes which
|
||||
// utterances the cascade never reaches. Read the comment above a line before
|
||||
// moving it, and read docs/routing.md before adding one.
|
||||
//
|
||||
// The classifier, the extractor, the threshold and the model arm are the
|
||||
// daemon's to assemble. This function returns the rules and nothing else, so
|
||||
// the fixture can also run them one at a time and see which of them contend for
|
||||
// the same utterance, which the cascade hides by stopping at the first match.
|
||||
func StageZeroGrammars(acts ActMatcher) []Grammar {
|
||||
grammars := DefaultGrammars(acts)
|
||||
grammars = append(grammars, SystemTimeDateGrammars()...)
|
||||
// After the time/date rules on purpose: "какой сегодня день" is a clock
|
||||
// question and must keep reaching replySystem, while "что у меня сегодня"
|
||||
// is an agenda question and must not.
|
||||
grammars = append(grammars, AgendaQueryGrammars()...)
|
||||
// Same reason as the agenda rules, for the feeds: "что нового в лентах?"
|
||||
// routed system and answered "пока не умею" (Vikunja #474).
|
||||
// After the agenda rules, which are the narrower claim, and BEFORE the feed
|
||||
// and list rules, which are not: "что такое лента" is a definition question
|
||||
// and the feed rule would take it on the noun alone (V-655).
|
||||
grammars = append(grammars, WorldQueryGrammars()...)
|
||||
grammars = append(grammars, FeedQueryGrammar())
|
||||
// The list side of the same exposure: a phrasing with no possessive in it
|
||||
// ("список дел") routed system and never reached queryTasks (Vikunja #467).
|
||||
grammars = append(grammars, TaskListGrammar())
|
||||
grammars = append(grammars, ListGrammars()...)
|
||||
grammars = append(grammars, ReminderGrammar())
|
||||
// Before the capture marker, because "отметь" is a capture verb and "отметь
|
||||
// второй пункт" is not a note. The Praxis rules are the narrower claim — a
|
||||
// lifecycle verb AND an item named — so they get first refusal (Vikunja #516).
|
||||
grammars = append(grammars, PraxisGrammars()...)
|
||||
// After Praxis, whose bare "закрой" claim this rule cannot reach (it needs the
|
||||
// board noun), and before the capture marker, which would otherwise read
|
||||
// "убери из задач купить молоко" as a new task (Vikunja #512).
|
||||
grammars = append(grammars, TaskStatusGrammar())
|
||||
// Before the capture markers, which all need an object. A capture verb
|
||||
// alone is a fact with no key, and the clarify path asks for it rather than
|
||||
// letting the model invent an answer (Vikunja #557).
|
||||
grammars = append(grammars, BareCaptureGrammar()...)
|
||||
// Last, and it matches any utterance shape — its Build is the filter. An
|
||||
// explicit capture marker beats the model, which called it an act and
|
||||
// rewrote the task text (Vikunja #467). After the rules above because a
|
||||
// marker never collides with a clock or agenda question.
|
||||
grammars = append(grammars, TaskCaptureGrammar())
|
||||
// After the capture marker, so "запиши" still wins over "расскажи", and
|
||||
// last overall because it matches on the first word alone: "расскажи про
|
||||
// X" is a world question the model called a fact (Vikunja #498).
|
||||
grammars = append(grammars, NarrativeQueryGrammars()...)
|
||||
return grammars
|
||||
}
|
||||
+118
-65
@@ -9,15 +9,18 @@
|
||||
//
|
||||
// The wire is symmetric: a Request from the client is answered by a
|
||||
// Response with a matching ID, OR a server-initiated Push frame (no ID)
|
||||
// may arrive interleaved. SendRequest loops reading frames, drops Push
|
||||
// frames to the harness if a receiver is running (or silently if not),
|
||||
// and returns the first Response with the matching ID.
|
||||
// may arrive interleaved. One reader goroutine per connection owns the
|
||||
// socket. It hands each Response to whichever SendRequest is waiting on
|
||||
// that ID and each Push to the handler, so a client may send and listen
|
||||
// at the same time on one conn. mavwaked needs exactly that: it speaks
|
||||
// utterances and it must hear nudges, and the server routes a nudge to
|
||||
// the session that spoke most recently, so a second listening conn would
|
||||
// never be picked (V-671).
|
||||
package voice
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net"
|
||||
@@ -41,13 +44,22 @@ type PushHandler interface {
|
||||
// Client — one connection to the voice.Server.
|
||||
type Client struct {
|
||||
addr string
|
||||
mu sync.Mutex
|
||||
c net.Conn
|
||||
nextID atomic.Uint64
|
||||
|
||||
// pushCh fan-out: a reader goroutine (started by RunPushReceiver)
|
||||
// writes Push frames here; SendRequest also drains it when no reader
|
||||
// is running (drops the frame in that case).
|
||||
mu sync.Mutex
|
||||
c net.Conn
|
||||
// pending holds one channel per in-flight request, keyed by frame id.
|
||||
// The reader goroutine delivers the Response here and deletes the entry.
|
||||
pending map[uint64]chan *Response
|
||||
// dead is closed by the reader goroutine when this conn ends, so a
|
||||
// waiting SendRequest fails at once instead of at its own deadline.
|
||||
dead chan struct{}
|
||||
|
||||
// wmu serialises writes. Frames must not interleave on the wire.
|
||||
wmu sync.Mutex
|
||||
|
||||
// pushH is set by RunPushReceiver and survives a reconnect, because the
|
||||
// client that wants pushes wants them on whatever conn it ends up with.
|
||||
pushMu sync.Mutex
|
||||
pushH PushHandler
|
||||
}
|
||||
@@ -55,6 +67,15 @@ type Client struct {
|
||||
// Dial returns a Client that will connect to addr on first use.
|
||||
func Dial(addr string) *Client { return &Client{addr: addr} }
|
||||
|
||||
// Connect opens the conn now rather than on the first request. mavwaked calls
|
||||
// it at startup: the server registers a session on accept, and a client that
|
||||
// has never connected cannot be sent a nudge.
|
||||
func (c *Client) Connect(ctx context.Context) error {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
return c.ensureConnLocked(ctx)
|
||||
}
|
||||
|
||||
// Close releases the conn. Idempotent.
|
||||
func (c *Client) Close() error {
|
||||
c.mu.Lock()
|
||||
@@ -75,12 +96,14 @@ func (c *Client) PushToTalk(ctx context.Context, a audio.Audio, lang string) (Pu
|
||||
return out, err
|
||||
}
|
||||
|
||||
// requestTimeout bounds a round-trip with no deadline on its context. It is
|
||||
// generous because the far end runs speech-to-text, a router and a voice.
|
||||
const requestTimeout = 120 * time.Second
|
||||
|
||||
// SendRequest sends one Request frame and waits for the matching Response.
|
||||
// Push frames received while waiting are dropped on the floor UNLESS a
|
||||
// PushHandler has been wired via RunPushReceiver, in which case the handler
|
||||
// is invoked inline (still synchronous with the SendRequest caller's
|
||||
// read). For sanity, the reference client runs either one-shot (no
|
||||
// receiver) or interactive (RunPushReceiver, no concurrent SendRequest).
|
||||
// Push frames arriving meanwhile go to the handler on the reader goroutine,
|
||||
// so listening and sending on one Client is supported rather than merely
|
||||
// tolerated.
|
||||
func (c *Client) SendRequest(ctx context.Context, m Method, params any, out any) error {
|
||||
body, err := marshalParams(params)
|
||||
if err != nil {
|
||||
@@ -94,58 +117,69 @@ func (c *Client) SendRequest(ctx context.Context, m Method, params any, out any)
|
||||
c.mu.Unlock()
|
||||
return err
|
||||
}
|
||||
conn := c.c
|
||||
conn, dead := c.c, c.dead
|
||||
ch := make(chan *Response, 1)
|
||||
c.pending[id] = ch
|
||||
c.mu.Unlock()
|
||||
|
||||
if dl, ok := ctx.Deadline(); ok {
|
||||
_ = conn.SetDeadline(dl)
|
||||
} else {
|
||||
_ = conn.SetDeadline(time.Now().Add(120 * time.Second))
|
||||
}
|
||||
defer conn.SetDeadline(time.Time{})
|
||||
|
||||
if err := writeFrame(conn, &req); err != nil {
|
||||
c.teardown()
|
||||
c.wmu.Lock()
|
||||
err = writeFrame(conn, &req)
|
||||
c.wmu.Unlock()
|
||||
if err != nil {
|
||||
c.forget(id)
|
||||
c.teardownConn(conn)
|
||||
return err
|
||||
}
|
||||
for {
|
||||
resp, push, err := readOneFrame(conn)
|
||||
if err != nil {
|
||||
c.teardown()
|
||||
return err
|
||||
}
|
||||
if push != nil {
|
||||
c.deliverPush(*push)
|
||||
continue
|
||||
}
|
||||
if resp.ID != id {
|
||||
continue // not ours; ignore (singleplex ⇒ shouldn't happen)
|
||||
}
|
||||
if resp.Error != nil {
|
||||
return hydrate(resp.Error)
|
||||
}
|
||||
if out != nil {
|
||||
if err := json.Unmarshal(resp.Result, out); err != nil {
|
||||
return fmt.Errorf("voice: unmarshal result: %w", err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
|
||||
timer := time.NewTimer(requestTimeout)
|
||||
defer timer.Stop()
|
||||
|
||||
var resp *Response
|
||||
select {
|
||||
case resp = <-ch:
|
||||
case <-dead:
|
||||
c.forget(id)
|
||||
return fmt.Errorf("voice: connection closed before reply")
|
||||
case <-ctx.Done():
|
||||
c.forget(id)
|
||||
return ctx.Err()
|
||||
case <-timer.C:
|
||||
c.forget(id)
|
||||
c.teardownConn(conn)
|
||||
return fmt.Errorf("voice: no reply within %s", requestTimeout)
|
||||
}
|
||||
|
||||
if resp.Error != nil {
|
||||
return hydrate(resp.Error)
|
||||
}
|
||||
if out != nil {
|
||||
if err := json.Unmarshal(resp.Result, out); err != nil {
|
||||
return fmt.Errorf("voice: unmarshal result: %w", err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// RunPushReceiver spawns a reader goroutine that delivers Push frames to h
|
||||
// until the conn closes or Close is called. Today's reference client uses
|
||||
// this in -listen mode (proactive voice playback). SendRequest and
|
||||
// RunPushReceiver SHOULD NOT be used concurrently on the same Client — the
|
||||
// wire is singleplex at the reference client's scale; production picks one
|
||||
// mode per conn. Returns when the goroutine ends (ctx cancel or conn close).
|
||||
// forget drops an abandoned request so a late Response is discarded rather
|
||||
// than delivered to nobody.
|
||||
func (c *Client) forget(id uint64) {
|
||||
c.mu.Lock()
|
||||
delete(c.pending, id)
|
||||
c.mu.Unlock()
|
||||
}
|
||||
|
||||
// RunPushReceiver wires h and blocks until the conn ends or ctx is
|
||||
// cancelled. Frames are read by the per-conn reader goroutine, so a client
|
||||
// may call SendRequest on the same Client while this is running. Returns nil
|
||||
// when the conn ended, so a caller that wants to stay reachable reconnects
|
||||
// and calls it again.
|
||||
func (c *Client) RunPushReceiver(ctx context.Context, h PushHandler) error {
|
||||
c.mu.Lock()
|
||||
if err := c.ensureConnLocked(ctx); err != nil {
|
||||
c.mu.Unlock()
|
||||
return err
|
||||
}
|
||||
conn := c.c
|
||||
dead := c.dead
|
||||
c.mu.Unlock()
|
||||
|
||||
c.pushMu.Lock()
|
||||
@@ -158,21 +192,34 @@ func (c *Client) RunPushReceiver(ctx context.Context, h PushHandler) error {
|
||||
c.pushMu.Unlock()
|
||||
}()
|
||||
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case <-dead:
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// readLoop owns conn for its whole life. It ends on any read error, which is
|
||||
// how a closed conn, a killed server and a cancelled dial all arrive here.
|
||||
func (c *Client) readLoop(conn net.Conn, dead chan struct{}) {
|
||||
defer close(dead)
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
default:
|
||||
}
|
||||
_, push, err := readOneFrame(conn)
|
||||
resp, push, err := readOneFrame(conn)
|
||||
if err != nil {
|
||||
if errors.Is(err, io.EOF) || errors.Is(err, net.ErrClosed) {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
c.teardownConn(conn)
|
||||
return
|
||||
}
|
||||
if push != nil {
|
||||
c.deliverPush(*push)
|
||||
continue
|
||||
}
|
||||
c.mu.Lock()
|
||||
ch := c.pending[resp.ID]
|
||||
delete(c.pending, resp.ID)
|
||||
c.mu.Unlock()
|
||||
if ch != nil {
|
||||
ch <- resp
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -196,13 +243,19 @@ func (c *Client) ensureConnLocked(ctx context.Context) error {
|
||||
return fmt.Errorf("voice: dial %s: %w", c.addr, err)
|
||||
}
|
||||
c.c = conn
|
||||
c.pending = make(map[uint64]chan *Response)
|
||||
c.dead = make(chan struct{})
|
||||
go c.readLoop(conn, c.dead)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *Client) teardown() {
|
||||
// teardownConn closes conn and forgets it, but only if it is still the live
|
||||
// one. A reconnect may already have replaced it, and closing the new conn
|
||||
// because the old one died takes the client down on every hiccup.
|
||||
func (c *Client) teardownConn(conn net.Conn) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
if c.c != nil {
|
||||
if c.c != nil && c.c == conn {
|
||||
_ = c.c.Close()
|
||||
c.c = nil
|
||||
}
|
||||
|
||||
@@ -221,3 +221,113 @@ func TestClientListenModeReceivesPush(t *testing.T) {
|
||||
type pushHandlerFunc func(Push)
|
||||
|
||||
func (f pushHandlerFunc) OnPush(p Push) { f(p) }
|
||||
|
||||
// The whole point of the per-conn reader (V-671): mavwaked speaks utterances
|
||||
// and must hear nudges, and the server routes a nudge to the session that
|
||||
// spoke most recently. A second listening conn would never be picked, so both
|
||||
// directions have to share one conn.
|
||||
func TestClientSendsAndListensOnOneConn(t *testing.T) {
|
||||
l := newTestListener(t)
|
||||
sess := NewSessions()
|
||||
h := &stubHandler{}
|
||||
srv := NewServer(l.Addr().String(), h, sess)
|
||||
if err := srv.Listen(); err != nil {
|
||||
t.Fatalf("listen: %v", err)
|
||||
}
|
||||
defer func() {
|
||||
_ = srv.Close()
|
||||
waitPort()
|
||||
}()
|
||||
go func() { _ = srv.Serve() }()
|
||||
|
||||
c := Dial(l.Addr().String())
|
||||
defer c.Close()
|
||||
|
||||
got := make(chan audio.Audio, 4)
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
go func() {
|
||||
_ = c.RunPushReceiver(ctx, pushHandlerFunc(func(p Push) {
|
||||
var ap AudioNudgePush
|
||||
if err := json.Unmarshal(p.Params, &ap); err == nil {
|
||||
got <- ap.Audio
|
||||
}
|
||||
}))
|
||||
}()
|
||||
|
||||
for i := 0; i < 100 && sess.Active() < 1; i++ {
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
}
|
||||
if sess.Active() != 1 {
|
||||
t.Fatalf("active sessions = %d, want exactly 1", sess.Active())
|
||||
}
|
||||
|
||||
// A round-trip while the receiver is running. Before the reader owned the
|
||||
// conn, this and the receiver raced for every frame.
|
||||
resp, err := c.PushToTalk(context.Background(), audio.Audio{Format: audio.PCM16kMono, Bytes: []byte("hello")}, "ru")
|
||||
if err != nil {
|
||||
t.Fatalf("PushToTalk with a receiver running: %v", err)
|
||||
}
|
||||
if resp.ReplyText != "got it" {
|
||||
t.Fatalf("ReplyText = %q, want %q", resp.ReplyText, "got it")
|
||||
}
|
||||
|
||||
// And the nudge still arrives, on the session that just spoke.
|
||||
err = sess.PushToMostRecent(context.Background(), AudioNudgePush{
|
||||
RuleName: "after-speaking",
|
||||
Audio: audio.Audio{Format: audio.PCM16kMono, Bytes: []byte("proactive")},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("PushToMostRecent: %v", err)
|
||||
}
|
||||
select {
|
||||
case a := <-got:
|
||||
if string(a.Bytes) != "proactive" {
|
||||
t.Fatalf("received %q, want the nudge audio", string(a.Bytes))
|
||||
}
|
||||
case <-time.After(2 * time.Second):
|
||||
t.Fatal("nudge never reached the handler after the client had spoken")
|
||||
}
|
||||
}
|
||||
|
||||
// A request abandoned by its context must not leave its slot behind, or a
|
||||
// long-running client leaks one channel per timeout.
|
||||
func TestClientForgetsAbandonedRequests(t *testing.T) {
|
||||
l := newTestListener(t)
|
||||
sess := NewSessions()
|
||||
srv := NewServer(l.Addr().String(), &blockingHandler{}, sess)
|
||||
if err := srv.Listen(); err != nil {
|
||||
t.Fatalf("listen: %v", err)
|
||||
}
|
||||
defer func() {
|
||||
_ = srv.Close()
|
||||
waitPort()
|
||||
}()
|
||||
go func() { _ = srv.Serve() }()
|
||||
|
||||
c := Dial(l.Addr().String())
|
||||
defer c.Close()
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
|
||||
defer cancel()
|
||||
_, err := c.PushToTalk(ctx, audio.Audio{Format: audio.PCM16kMono, Bytes: []byte("x")}, "ru")
|
||||
if err == nil {
|
||||
t.Fatal("expected the round-trip to fail on its context")
|
||||
}
|
||||
|
||||
c.mu.Lock()
|
||||
n := len(c.pending)
|
||||
c.mu.Unlock()
|
||||
if n != 0 {
|
||||
t.Fatalf("pending = %d after an abandoned request, want 0", n)
|
||||
}
|
||||
}
|
||||
|
||||
// blockingHandler never answers, so the client's context is what ends the
|
||||
// round-trip.
|
||||
type blockingHandler struct{}
|
||||
|
||||
func (blockingHandler) HandlePushToTalk(ctx context.Context, _ PushToTalkReq, _ uint64) (PushToTalkResp, error) {
|
||||
<-ctx.Done()
|
||||
return PushToTalkResp{}, ctx.Err()
|
||||
}
|
||||
|
||||
Executable
+97
@@ -0,0 +1,97 @@
|
||||
#!/usr/bin/env bash
|
||||
# analyzer-gate.sh — turn an analyzer's output into a pass/fail verdict.
|
||||
#
|
||||
# The 2026-08-10 audit asked for staticcheck, govulncheck and deadcode
|
||||
# (V-694). govulncheck needed no gate of this shape because it already
|
||||
# reported zero after the toolchain bump. The other two do not: staticcheck
|
||||
# reports 20 findings today and deadcode reports 11 unreachable symbols, and
|
||||
# three of those eleven are deliberate. A target that fails on the first run
|
||||
# is not a gate, it is a target nobody runs. So the accepted set is written
|
||||
# down, and only what is NOT in it fails.
|
||||
#
|
||||
# staticcheck ./... | scripts/analyzer-gate.sh staticcheck
|
||||
# deadcode -test ./... | scripts/analyzer-gate.sh deadcode
|
||||
#
|
||||
# The baseline is keyed on file, check id and message, never on line number.
|
||||
# A key carrying a line number goes stale on the next edit above it and then
|
||||
# reports moved findings as new ones, which trains the reader to ignore it.
|
||||
# The cost of dropping the line is that two identical findings in one file
|
||||
# share one key, so the second is accepted with the first. That is the right
|
||||
# way round: the same check firing twice on the same file is one thing to fix.
|
||||
#
|
||||
# A baseline entry with no finding left also fails. Fixing something and
|
||||
# leaving its entry behind is how the accepted set stops describing the repo.
|
||||
# The fix is one line: delete the entry the failure names.
|
||||
#
|
||||
# Reads stdin, writes a report, never writes a file.
|
||||
|
||||
set -uo pipefail
|
||||
cd "$(dirname "$0")/.." || exit 1
|
||||
|
||||
tool="${1:?usage: analyzer-gate.sh <staticcheck|deadcode>}"
|
||||
baseline="scripts/analyzers/$tool.baseline"
|
||||
[ -f "$baseline" ] || { printf 'analyzer-gate: no baseline at %s\n' "$baseline" >&2; exit 2; }
|
||||
|
||||
# Normalise to "<file>\t<id>\t<message>". Anything that does not parse is an
|
||||
# analyzer error, not a finding, and it fails without consulting the baseline.
|
||||
# staticcheck: path.go:12:34: message (SA1234)
|
||||
# deadcode: path.go:12:34: unreachable func: Symbol
|
||||
found=$(mktemp) || exit 2
|
||||
malformed=$(mktemp) || exit 2
|
||||
trap 'rm -f "$found" "$malformed"' EXIT
|
||||
|
||||
while IFS= read -r line; do
|
||||
[ -n "$line" ] || continue
|
||||
case "$tool" in
|
||||
staticcheck)
|
||||
if [[ "$line" =~ ^([^:]+):[0-9]+:[0-9]+:\ (.*)\ \(([A-Z]+[0-9]+)\)$ ]]; then
|
||||
# SA1019 ends its message with a space. Trim, so no baseline entry
|
||||
# depends on trailing whitespace surviving an editor.
|
||||
msg="${BASH_REMATCH[2]}"
|
||||
printf '%s\t%s\t%s\n' "${BASH_REMATCH[1]}" "${BASH_REMATCH[3]}" "${msg%"${msg##*[![:space:]]}"}" >>"$found"
|
||||
else
|
||||
printf '%s\n' "$line" >>"$malformed"
|
||||
fi
|
||||
;;
|
||||
deadcode)
|
||||
if [[ "$line" =~ ^([^:]+):[0-9]+:[0-9]+:\ unreachable\ func:\ (.*)$ ]]; then
|
||||
printf '%s\tunreachable\t%s\n' "${BASH_REMATCH[1]}" "${BASH_REMATCH[2]}" >>"$found"
|
||||
else
|
||||
printf '%s\n' "$line" >>"$malformed"
|
||||
fi
|
||||
;;
|
||||
*) printf 'analyzer-gate: unknown tool %s\n' "$tool" >&2; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ -s "$malformed" ]; then
|
||||
printf '%s: the analyzer said something that is not a finding:\n' "$tool" >&2
|
||||
sed 's/^/ /' "$malformed" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
accepted=$(mktemp) || exit 2
|
||||
trap 'rm -f "$found" "$malformed" "$accepted"' EXIT
|
||||
grep -v '^[[:space:]]*\(#\|$\)' "$baseline" | sort -u >"$accepted"
|
||||
sort -u "$found" -o "$found"
|
||||
|
||||
new=$(comm -23 "$found" "$accepted")
|
||||
gone=$(comm -13 "$found" "$accepted")
|
||||
status=0
|
||||
|
||||
if [ -n "$new" ]; then
|
||||
printf '%s: %d finding(s) not in %s:\n' "$tool" "$(printf '%s\n' "$new" | wc -l)" "$baseline"
|
||||
printf '%s\n' "$new" | sed 's/^/ /'
|
||||
printf 'Fix it, or add the line to the baseline with the reason it stays.\n'
|
||||
status=1
|
||||
fi
|
||||
|
||||
if [ -n "$gone" ]; then
|
||||
printf '%s: %d baseline entry/entries no longer found:\n' "$tool" "$(printf '%s\n' "$gone" | wc -l)"
|
||||
printf '%s\n' "$gone" | sed 's/^/ /'
|
||||
printf 'Delete them from %s.\n' "$baseline"
|
||||
status=1
|
||||
fi
|
||||
|
||||
[ "$status" -eq 0 ] && printf '%s: clean against %d accepted finding(s)\n' "$tool" "$(wc -l <"$accepted")"
|
||||
exit "$status"
|
||||
@@ -0,0 +1,32 @@
|
||||
# deadcode — the unreachable symbols this repo accepts today.
|
||||
#
|
||||
# Keyed "<file>\t unreachable \t<symbol>", tab separated, no line numbers.
|
||||
# Generated from the first gated run on 2026-08-11 and edited by hand since.
|
||||
# `make deadcode` fails on anything absent here and on any entry left behind
|
||||
# after its symbol is deleted.
|
||||
#
|
||||
# The gate runs with -test, so a test file counts as a root. Without it the
|
||||
# whole of internal/router/eval is unreachable and the report is 172 lines of
|
||||
# fixtures nobody wrote by mistake.
|
||||
#
|
||||
# Eleven of these are V-686, from the 2026-08-10 audit. Three of the eleven
|
||||
# must stay and the audit says why: HisGender is a documented seam tied to
|
||||
# V-399, AudioDuration should call internal/audio rather than be deleted, and
|
||||
# CountWord is a safe delete. Read docs/caveats/layering.md#deadcode before
|
||||
# removing any of them.
|
||||
cmd/mavwaked/vad.go unreachable AudioDuration
|
||||
cmd/mavwaked/vad.go unreachable PCMToF32
|
||||
internal/crawl/watch.go unreachable Watcher.Watches
|
||||
internal/phraser/confirm.go unreachable IsC
|
||||
internal/phraser/eval/checks.go unreachable HisGender
|
||||
internal/phraser/plural.go unreachable CountWord
|
||||
internal/update/update.go unreachable WithClock
|
||||
internal/voice/errors.go unreachable jsonMarshal
|
||||
internal/voice/errors.go unreachable jsonUnmarshal
|
||||
internal/webauthn/cbor.go unreachable cborValue.At
|
||||
internal/worker/server.go unreachable Server.SetSynthesizer
|
||||
|
||||
# Two test helpers the audit did not count, because it listed production
|
||||
# symbols only. A helper no test calls is dead the same way.
|
||||
cmd/mavend/replier_llm_test.go unreachable assertStub
|
||||
cmd/mavwaked/vad_test.go unreachable frameRMSQuick
|
||||
@@ -0,0 +1,46 @@
|
||||
# staticcheck — the findings this repo accepts today.
|
||||
#
|
||||
# Keyed "<file>\t<check>\t<message>", tab separated, no line numbers.
|
||||
# Generated from the first gated run on 2026-08-11 and edited by hand since.
|
||||
# `make lint` fails on anything absent here and on any entry left behind after
|
||||
# its finding is fixed, so emptying this file is done one line at a time.
|
||||
#
|
||||
# The sweep that empties it is V-701, which carries the judgement on each
|
||||
# entry. What follows is the short reason only.
|
||||
|
||||
# V-687. The dedupe check runs after the phraser has already been paid.
|
||||
cmd/mavend/tick_digest.go SA4006 this value of deduped is never used
|
||||
|
||||
# V-686, the eleven unreachable symbols the 2026-08-10 audit listed, seen from
|
||||
# the other side. Three of them must stay: docs/caveats/layering.md#deadcode.
|
||||
cmd/mavend/replier_llm_test.go U1000 func assertStub is unused
|
||||
cmd/mavwaked/vad_test.go U1000 func frameRMSQuick is unused
|
||||
cmd/mavweb/handlers_test.go U1000 field signalErr is unused
|
||||
internal/voice/errors.go U1000 func jsonMarshal is unused
|
||||
internal/voice/errors.go U1000 func jsonUnmarshal is unused
|
||||
|
||||
# False positives, checked. The code is right and the check cannot see why.
|
||||
# RenderICal is called twice because rendering twice is the assertion. The
|
||||
# morning loop reads the first rune after the hedge and breaks on purpose. The
|
||||
# task_phrases line is prose about //go:embed and the real directive is below it.
|
||||
internal/calendar/ical_render_test.go SA4000 identical expressions on the left and right side of the '!=' operator
|
||||
internal/morning/plan_test.go SA4004 the surrounding loop is unconditionally terminated
|
||||
internal/router/task_phrases.go SA9009 ineffectual compiler directive due to extraneous space: "// go:embed, so the single-binary deploy is unchanged: the JSON is compiled into"
|
||||
|
||||
# At EOF the wake loop trims partial and returns, so audio past one frame is
|
||||
# dropped. Harmless where it sits, misleading to read. V-701.
|
||||
cmd/mavwaked/main.go SA4006 this value of partial is never used
|
||||
|
||||
# Cosmetic and mechanical. V-701 sweeps them.
|
||||
cmd/mavweb/voiceproxy.go ST1013 should use constant http.StatusMethodNotAllowed instead of numeric literal 405
|
||||
cmd/mavweb/voiceproxy.go ST1013 should use constant http.StatusServiceUnavailable instead of numeric literal 503
|
||||
internal/ipc/client.go S1016 should convert r (type chatResp) to ChatReply instead of using struct literal
|
||||
internal/ipc/server.go S1016 should convert reply (type ChatReply) to chatResp instead of using struct literal
|
||||
internal/memory/behavior_test.go S1011 should replace loop with obs = append(obs, habitHistory("calendar_event_20260804_standup", time.Tuesday, 10, 0, 3, now)...)
|
||||
internal/memory/behavior_test.go S1011 should replace loop with obs = append(obs, habitHistory("cooldown:water", time.Tuesday, 9, 0, 3, now)...)
|
||||
cmd/mavend/continuation_test.go SA1012 do not pass a nil Context, even if a function permits it; pass context.TODO if you are unsure about which Context to use
|
||||
|
||||
# Deprecated since Go 1.25. Replacing it means rewriting both guards on
|
||||
# golang.org/x/tools/go/packages, which is a decision and not a sweep.
|
||||
internal/ipc/maperr_test.go SA1019 parser.ParseDir has been deprecated since Go 1.25 and an alternative has been available since Go 1.11: ParseDir does not consider build tags when associating files with packages. For precise information about the relationship between packages and files, use golang.org/x/tools/go/packages, which can also optionally parse and type-check the files too.
|
||||
internal/phraser/persona_floor_test.go SA1019 parser.ParseDir has been deprecated since Go 1.25 and an alternative has been available since Go 1.11: ParseDir does not consider build tags when associating files with packages. For precise information about the relationship between packages and files, use golang.org/x/tools/go/packages, which can also optionally parse and type-check the files too.
|
||||
+2
-2
@@ -1,4 +1,4 @@
|
||||
Copyright (c) 2009 The Go Authors. All rights reserved.
|
||||
Copyright 2009 The Go Authors.
|
||||
|
||||
Redistribution and use in source and binary forms, with or without
|
||||
modification, are permitted provided that the following conditions are
|
||||
@@ -10,7 +10,7 @@ notice, this list of conditions and the following disclaimer.
|
||||
copyright notice, this list of conditions and the following disclaimer
|
||||
in the documentation and/or other materials provided with the
|
||||
distribution.
|
||||
* Neither the name of Google Inc. nor the names of its
|
||||
* Neither the name of Google LLC nor the names of its
|
||||
contributors may be used to endorse or promote products derived from
|
||||
this software without specific prior written permission.
|
||||
|
||||
|
||||
+26
-9
@@ -13,15 +13,18 @@ import "encoding/binary"
|
||||
// a rune to a uint16. The values take two forms. For v >= 0x8000:
|
||||
// bits
|
||||
// 15: 1 (inverse of NFD_QC bit of qcInfo)
|
||||
// 13..7: qcInfo (see below). isYesD is always true (no decomposition).
|
||||
// 12..7: qcInfo (see below). isYesD is always true (no decomposition).
|
||||
// 6..0: ccc (compressed CCC value).
|
||||
// For v < 0x8000, the respective rune has a decomposition and v is an index
|
||||
// into a byte array of UTF-8 decomposition sequences and additional info and
|
||||
// has the form:
|
||||
// <header> <decomp_byte>* [<tccc> [<lccc>]]
|
||||
// The header contains the number of bytes in the decomposition (excluding this
|
||||
// length byte). The two most significant bits of this length byte correspond
|
||||
// to bit 5 and 4 of qcInfo (see below). The byte sequence itself starts at v+1.
|
||||
// length byte), with 33 mapped to 31 to fit in 5 bits.
|
||||
// (If any 31- or 32-byte decompositions come along, we could switch to using
|
||||
// use a general lookup table as long as there are at most 32 distinct lengths.)
|
||||
// The three most significant bits of this length byte correspond
|
||||
// to bit 5, 4, and 3 of qcInfo (see below). The byte sequence itself starts at v+1.
|
||||
// The byte sequence is followed by a trailing and leading CCC if the values
|
||||
// for these are not zero. The value of v determines which ccc are appended
|
||||
// to the sequences. For v < firstCCC, there are none, for v >= firstCCC,
|
||||
@@ -32,8 +35,8 @@ import "encoding/binary"
|
||||
|
||||
const (
|
||||
qcInfoMask = 0x3F // to clear all but the relevant bits in a qcInfo
|
||||
headerLenMask = 0x3F // extract the length value from the header byte
|
||||
headerFlagsMask = 0xC0 // extract the qcInfo bits from the header byte
|
||||
headerLenMask = 0x1F // extract the length value from the header byte (31 => 33)
|
||||
headerFlagsMask = 0xE0 // extract the qcInfo bits from the header byte
|
||||
)
|
||||
|
||||
// Properties provides access to normalization properties of a rune.
|
||||
@@ -109,17 +112,21 @@ func (p Properties) BoundaryAfter() bool {
|
||||
return p.isInert()
|
||||
}
|
||||
|
||||
// We pack quick check data in 4 bits:
|
||||
// We pack quick check data in 6 bits:
|
||||
//
|
||||
// 5: Combines forward (0 == false, 1 == true)
|
||||
// 4..3: NFC_QC Yes(00), No (10), or Maybe (11)
|
||||
// 2: NFD_QC Yes (0) or No (1). No also means there is a decomposition.
|
||||
// 1..0: Number of trailing non-starters.
|
||||
//
|
||||
// When all 4 bits are zero, the character is inert, meaning it is never
|
||||
// When all 6 bits are zero, the character is inert, meaning it is never
|
||||
// influenced by normalization.
|
||||
//
|
||||
// We set flags to 0x80 (high bit 7 unused in quick check data) to indicate an invalid rune.
|
||||
type qcInfo uint8
|
||||
|
||||
func (p Properties) isInvalid() bool { return p.flags == 0x80 }
|
||||
|
||||
func (p Properties) isYesC() bool { return p.flags&0x10 == 0 }
|
||||
func (p Properties) isYesD() bool { return p.flags&0x4 == 0 }
|
||||
|
||||
@@ -152,6 +159,9 @@ func (p Properties) Decomposition() []byte {
|
||||
}
|
||||
i := p.index
|
||||
n := decomps[i] & headerLenMask
|
||||
if n == 31 {
|
||||
n = 33
|
||||
}
|
||||
i++
|
||||
return decomps[i : i+uint16(n)]
|
||||
}
|
||||
@@ -241,6 +251,9 @@ func (f Form) PropertiesString(s string) Properties {
|
||||
// to a Properties. See the comment at the top of the file
|
||||
// for more information on the format.
|
||||
func compInfo(v uint16, sz int) Properties {
|
||||
if sz == 0 {
|
||||
return Properties{flags: 0x80, size: 1}
|
||||
}
|
||||
if v == 0 {
|
||||
return Properties{size: uint8(sz)}
|
||||
} else if v >= 0x8000 {
|
||||
@@ -248,7 +261,7 @@ func compInfo(v uint16, sz int) Properties {
|
||||
size: uint8(sz),
|
||||
ccc: uint8(v),
|
||||
tccc: uint8(v),
|
||||
flags: qcInfo(v >> 8),
|
||||
flags: qcInfo(v>>8) & 0x3f,
|
||||
}
|
||||
if p.ccc > 0 || p.combinesBackward() {
|
||||
p.nLead = uint8(p.flags & 0x3)
|
||||
@@ -260,7 +273,11 @@ func compInfo(v uint16, sz int) Properties {
|
||||
f := (qcInfo(h&headerFlagsMask) >> 2) | 0x4
|
||||
p := Properties{size: uint8(sz), flags: f, index: v}
|
||||
if v >= firstCCC {
|
||||
v += uint16(h&headerLenMask) + 1
|
||||
n := uint16(h & headerLenMask)
|
||||
if n == 31 {
|
||||
n = 33
|
||||
}
|
||||
v += n + 1
|
||||
c := decomps[v]
|
||||
p.tccc = c >> 2
|
||||
p.flags |= qcInfo(c & 0x3)
|
||||
|
||||
+2
-6
@@ -376,16 +376,12 @@ func nextComposed(i *Iter) []byte {
|
||||
goto doNorm
|
||||
}
|
||||
prevCC = i.info.tccc
|
||||
sz := int(i.info.size)
|
||||
if sz == 0 {
|
||||
sz = 1 // illegal rune: copy byte-by-byte
|
||||
}
|
||||
p := outp + sz
|
||||
p := outp + int(i.info.size)
|
||||
if p > len(i.buf) {
|
||||
break
|
||||
}
|
||||
outp = p
|
||||
i.p += sz
|
||||
i.p += int(i.info.size)
|
||||
if i.p >= i.rb.nsrc {
|
||||
i.setDone()
|
||||
break
|
||||
|
||||
+10
-10
@@ -148,7 +148,7 @@ func (f Form) IsNormalString(s string) bool {
|
||||
// patched buffer and whether the decomposition is still in progress.
|
||||
func patchTail(rb *reorderBuffer) bool {
|
||||
info, p := lastRuneStart(&rb.f, rb.out)
|
||||
if p == -1 || info.size == 0 {
|
||||
if p == -1 || info.isInvalid() {
|
||||
return true
|
||||
}
|
||||
end := p + int(info.size)
|
||||
@@ -225,7 +225,7 @@ func doAppend(rb *reorderBuffer, out []byte, p int) []byte {
|
||||
}
|
||||
fd := &rb.f
|
||||
if doMerge {
|
||||
var info Properties
|
||||
info := Properties{flags: 0x80, size: 1} // invalid rune
|
||||
if p < n {
|
||||
info = fd.info(src, p)
|
||||
if !info.BoundaryBefore() || info.nLeadingNonStarters() > 0 {
|
||||
@@ -235,7 +235,7 @@ func doAppend(rb *reorderBuffer, out []byte, p int) []byte {
|
||||
p = decomposeSegment(rb, p, true)
|
||||
}
|
||||
}
|
||||
if info.size == 0 {
|
||||
if info.isInvalid() {
|
||||
rb.doFlush()
|
||||
// Append incomplete UTF-8 encoding.
|
||||
return src.appendSlice(rb.out, p, n)
|
||||
@@ -314,7 +314,7 @@ func (f *formInfo) quickSpan(src input, i, end int, atEOF bool) (n int, ok bool)
|
||||
continue
|
||||
}
|
||||
info := f.info(src, i)
|
||||
if info.size == 0 {
|
||||
if info.isInvalid() {
|
||||
if atEOF {
|
||||
// include incomplete runes
|
||||
return n, true
|
||||
@@ -379,7 +379,7 @@ func (f Form) firstBoundary(src input, nsrc int) int {
|
||||
// CGJ insertion points correctly. Luckily it doesn't have to.
|
||||
for {
|
||||
info := fd.info(src, i)
|
||||
if info.size == 0 {
|
||||
if info.isInvalid() {
|
||||
return -1
|
||||
}
|
||||
if s := ss.next(info); s != ssSuccess {
|
||||
@@ -424,7 +424,7 @@ func (f Form) nextBoundary(src input, nsrc int, atEOF bool) int {
|
||||
}
|
||||
fd := formTable[f]
|
||||
info := fd.info(src, 0)
|
||||
if info.size == 0 {
|
||||
if info.isInvalid() {
|
||||
if atEOF {
|
||||
return 1
|
||||
}
|
||||
@@ -435,7 +435,7 @@ func (f Form) nextBoundary(src input, nsrc int, atEOF bool) int {
|
||||
|
||||
for i := int(info.size); i < nsrc; i += int(info.size) {
|
||||
info = fd.info(src, i)
|
||||
if info.size == 0 {
|
||||
if info.isInvalid() {
|
||||
if atEOF {
|
||||
return i
|
||||
}
|
||||
@@ -465,7 +465,7 @@ func lastBoundary(fd *formInfo, b []byte) int {
|
||||
if p == -1 {
|
||||
return -1
|
||||
}
|
||||
if info.size == 0 { // ends with incomplete rune
|
||||
if info.isInvalid() { // ends with incomplete rune
|
||||
if p == 0 { // starts with incomplete rune
|
||||
return -1
|
||||
}
|
||||
@@ -504,7 +504,7 @@ func lastBoundary(fd *formInfo, b []byte) int {
|
||||
func decomposeSegment(rb *reorderBuffer, sp int, atEOF bool) int {
|
||||
// Force one character to be consumed.
|
||||
info := rb.f.info(rb.src, sp)
|
||||
if info.size == 0 {
|
||||
if info.isInvalid() {
|
||||
return 0
|
||||
}
|
||||
if s := rb.ss.next(info); s == ssStarter {
|
||||
@@ -528,7 +528,7 @@ func decomposeSegment(rb *reorderBuffer, sp int, atEOF bool) int {
|
||||
break
|
||||
}
|
||||
info = rb.f.info(rb.src, sp)
|
||||
if info.size == 0 {
|
||||
if info.isInvalid() {
|
||||
if !atEOF {
|
||||
return int(iShortSrc)
|
||||
}
|
||||
|
||||
-7657
File diff suppressed because it is too large
Load Diff
-7693
File diff suppressed because it is too large
Load Diff
-7710
File diff suppressed because it is too large
Load Diff
+1478
-1478
File diff suppressed because it is too large
Load Diff
Generated
Vendored
+3532
-3188
File diff suppressed because it is too large
Load Diff
-7637
File diff suppressed because it is too large
Load Diff
Vendored
+2
-2
@@ -40,8 +40,8 @@ github.com/yalue/onnxruntime_go
|
||||
## explicit; go 1.25.0
|
||||
golang.org/x/sys/unix
|
||||
golang.org/x/sys/windows
|
||||
# golang.org/x/text v0.14.0
|
||||
## explicit; go 1.18
|
||||
# golang.org/x/text v0.40.0
|
||||
## explicit; go 1.25.0
|
||||
golang.org/x/text/transform
|
||||
golang.org/x/text/unicode/norm
|
||||
# modernc.org/libc v1.74.1
|
||||
|
||||
Reference in New Issue
Block a user