Compare commits
66 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 9f714b7ae8 | |||
| ef3ee1e00a | |||
| 99e73ea653 | |||
| ab1784f5e1 | |||
| 2c73493bf8 | |||
| ff202c0c35 | |||
| 02d96e611d | |||
| 62eef01c18 | |||
| 1a8aed35b8 | |||
| ce6a6821a9 | |||
| 479b0c4475 | |||
| 877b1fd4f8 | |||
| 21a42cb3e6 | |||
| b8279f6a22 | |||
| 8c30971a96 | |||
| d0ea927ac3 | |||
| 9c7bafd5b1 | |||
| 1f1e002789 | |||
| ce91d20ac8 | |||
| 50130cdffb | |||
| a9b480a78f | |||
| c5264deb46 | |||
| 1cb269886e | |||
| 7a9b9cc669 | |||
| 31b5093403 | |||
| 229890abd7 | |||
| 0db9ca084c | |||
| 96d97e8964 | |||
| 02f6e8ad4a | |||
| 999a5ad562 | |||
| 2ea39a3d41 | |||
| 8fb6f2154d | |||
| c938148619 | |||
| 2512d686a1 | |||
| f44abcc526 | |||
| a99932b427 | |||
| 6d5801bb1f | |||
| 8aba4845bf | |||
| 2ec92ee8bf | |||
| 7c77a378c1 | |||
| 672eabc134 | |||
| a1a2fa3704 | |||
| 22a4978459 | |||
| 1456336652 | |||
| b975716759 | |||
| 4b1edb0617 | |||
| 944e553669 | |||
| cc32c2c4ab | |||
| a1e97c94ac | |||
| c7f59e48f4 | |||
| 4666057066 | |||
| 7138086c3f | |||
| 83e168f326 | |||
| a4abcdefa3 | |||
| 68a3c85186 | |||
| 88c086482e | |||
| feabf9f350 | |||
| 50c6637c1b | |||
| ee9d55ca95 | |||
| a886217223 | |||
| de9884e063 | |||
| bbefda66e2 | |||
| a37c4138a1 | |||
| d6f391430f | |||
| 68b2aa9137 | |||
| f8fa0d1b44 |
@@ -70,3 +70,5 @@ coverage.out
|
||||
|
||||
# root .env — MAVEN_AMBIENT_TOKEN and friends, same class as deploy/telegram.env
|
||||
.env
|
||||
# silero-vad, downloaded (see AGENTS.md)
|
||||
/models/vad/
|
||||
|
||||
@@ -95,6 +95,22 @@ model: the code puts `query: ` in front of a question and `passage: ` in front
|
||||
of a stored note, which is how e5 was trained. The quantized file is the one
|
||||
that is downloaded, deployed and measured.
|
||||
|
||||
## Voice activity model for mavwaked
|
||||
|
||||
`mavwaked` decides an utterance has started with silero-vad when `-vad-model`
|
||||
points at it, and with an energy threshold when it does not. The model is 2.3MB
|
||||
and is not committed:
|
||||
|
||||
```sh
|
||||
mkdir -p models/vad
|
||||
curl -sL -o models/vad/silero_vad.onnx \
|
||||
https://github.com/snakers4/silero-vad/raw/master/src/silero_vad/data/silero_vad.onnx
|
||||
```
|
||||
|
||||
It needs the same `libonnxruntime.so` the embedder needs, passed as `-onnx-lib`
|
||||
or read from `MAVEN_ONNX_LIB`. The measurement is
|
||||
`docs/evals/2026-08-09-silero-vad.md`, and the tests skip without the file.
|
||||
|
||||
**Also need ONNX Runtime** (`libonnxruntime.so`):
|
||||
|
||||
```sh
|
||||
|
||||
@@ -1,708 +1,200 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
Guidance for Claude Code (claude.ai/code) working in this repository.
|
||||
|
||||
Maven is a self-hosted, privacy-first voice assistant (Russian + English). Go daemons
|
||||
talking over unix sockets; one resident small model for routing + phrasing; whisper.cpp STT, piper TTS.
|
||||
Deploy target is a Ryzen laptop (homesrv) with Vulkan offload to the Vega iGPU (`n_gpu_layers: 99`,
|
||||
compose passes `/dev/dri` + the render gid) — the resident model stays ≤1.7B either way.
|
||||
**This is a rules file.** It loads into every session, so it carries only what
|
||||
changes what an agent does. A measurement belongs in `docs/evals/`, dated and
|
||||
never edited after the day. A subsystem's reasoning belongs in its living doc
|
||||
under `docs/`. Read that doc before changing the subsystem.
|
||||
|
||||
**Resident model:** currently **Qwen3-1.7B** (`UD-Q4_K_XL`), stock — not yet the CPT'd one.
|
||||
It replaced Qwen3.5-0.8B on 2026-07-31 because it measured better on both fixtures we have:
|
||||
67.5% vs 59.7% intent-only on the 77-case RU routing fixture, and 20/27 vs 11-17/27 on the
|
||||
talk fixture. See `docs/evals/2026-07-31-model-bakeoff.md`. It is a Thinking variant, so `n_ctx` is 4096
|
||||
— reasoning tokens need the room, and 4096 is what the scores above were measured at.
|
||||
| Read this | Before |
|
||||
|---|---|
|
||||
| `docs/routing.md` | touching `internal/router/` or `queryWalk` |
|
||||
| `docs/deployment.md` | touching a daemon, compose, a systemd unit or the web UI |
|
||||
| `docs/offload.md` | touching a daemon seam or adding a model caller |
|
||||
| `docs/world.md` | touching search, Kiwix or the world chain |
|
||||
| `docs/language.md` | changing a prompt contract or a Russian word list |
|
||||
| `docs/ecosystem.md` | touching Nexus, Praxis or Hexis |
|
||||
| `docs/rearchitecture.md`, `docs/design.md` | changing the shape of anything |
|
||||
| `docs/workflow.md` | the five stores, the doc tiers, the guards |
|
||||
| `AGENTS.md` | local preview, screenshots, model downloads |
|
||||
|
||||
The **target** is still the locally CPT'd **Qwen3-1.7B** (Vikunja #122, training in flight).
|
||||
Stock already speaks good Russian; what it gets wrong is the persona — it writes `я рад`,
|
||||
masculine, where Maven needs `рада`. That is what the CPT is for.
|
||||
## What Maven is
|
||||
|
||||
**Do not bother with sub-500M models.** LFM2.5-230M and 350M were measured on 2026-07-31 and
|
||||
both are unusable in Russian: the 350M routes at 5.2% (worse than guessing) and answers
|
||||
"столица Франции?" with the invented non-word "Сторзит"; the 230M replies to Russian in
|
||||
Spanish. Their strong published IFEval/BFCL numbers are English-only. Model files live in
|
||||
`/mnt/hdd1/llms`, bind-mounted to `/opt/maven/models/llm` — which **shadows** the repo's
|
||||
`models/llm/`, so the LFM2.5 gguf sitting there is not loaded by anything. Swapping the resident
|
||||
model is a one-line change to `phraser.model_path` in `deploy/mavend.json`.
|
||||
A self-hosted, privacy-first voice assistant in Russian and English. Go daemons
|
||||
talk over unix sockets. One resident small model routes and phrases. whisper.cpp
|
||||
does speech-to-text and piper does text-to-speech.
|
||||
|
||||
See `docs/rearchitecture.md` for the target architecture, `docs/design.md` for the folded design spec, and
|
||||
`AGENTS.md` for local-preview + model-download recipes.
|
||||
The resident model is **Qwen3-1.7B** (`UD-Q4_K_XL`) on homesrv, a Thinking
|
||||
variant at `n_ctx` 4096. Keep it at 1.7B or under. Sub-500M models are unusable
|
||||
in Russian (`docs/evals/2026-07-31-model-bakeoff.md`). Model files live in
|
||||
`/mnt/hdd1/llms`, bind-mounted over the repo's `models/llm/`, so a gguf sitting
|
||||
in the repo is loaded by nothing.
|
||||
|
||||
**Model work is moving to the workstation** (owner's call, 2026-08-02). homesrv cannot grow a
|
||||
GPU and the workstation has 16GB of VRAM. So the resident model, STT and TTS become preferred
|
||||
remotes with a floor on homesrv. The workstation is never assumed up. Fall back silently when
|
||||
it would only do the job better. Name the gap when the 1.7B cannot do it at all. The embedder
|
||||
stays on homesrv permanently, because it backs that floor. It is multilingual-e5-small,
|
||||
quantized and asymmetric — `EmbedQuery` and `EmbedPassage` apply the `query:`/`passage:`
|
||||
prefixes it was trained with, and calling plain `Embed` on a note is a bug. It replaced
|
||||
MiniLM and bought ten points of recall@1 and 2.5× the speed; see
|
||||
`docs/evals/2026-08-04-recall-e5-small.md`. Read `docs/offload.md` before
|
||||
touching a daemon seam or adding a model caller. Vikunja #483 is the umbrella, #484 to #487
|
||||
are the work.
|
||||
The workstation is workpc and it holds the remote model and speech-to-text.
|
||||
**It is never assumed up.** **Fall back silently** when it would only do the job
|
||||
better. **Name the gap** when the resident model cannot do the job at all.
|
||||
|
||||
Both halves are wired as of 2026-08-03. Routing and replies prefer the workstation silently
|
||||
through `modelSeam`; nudge and reminder phrasing prefer it silently inside the phraser. A
|
||||
world question goes through `LLMPhraser.PhraseWorld` and names the gap when the card is not
|
||||
free — `worldGap` in `cmd/mavend/worldmodel.go`, which the owner hears instead of an invented
|
||||
answer. A box with no `workstation` block behaves exactly as it did before the seam: naming
|
||||
a gap requires a gap. The offload table in `docs/offload.md` says which caller is which.
|
||||
**The embedder stays on homesrv permanently**, because it backs that floor.
|
||||
`EmbedQuery` and `EmbedPassage` apply the `query:` and `passage:` prefixes
|
||||
multilingual-e5-small was trained with. Calling plain `Embed` on a note is a bug.
|
||||
|
||||
## Build & test
|
||||
## Build and test
|
||||
|
||||
CGO daemons (`mavend`, `mavsttd`, `mavttsd`, `mavenclient`) need the vendored toolchain
|
||||
and libs wired through the Makefile — **do not** call `go build` on them bare, use `make`:
|
||||
CGO daemons (`mavend`, `mavsttd`, `mavttsd`, `mavenclient`) need the vendored
|
||||
toolchain wired through the Makefile. **Do not call `go build` on them bare**,
|
||||
and **do not hand-write the CGO preamble**. This box runs zsh, so an unquoted
|
||||
`-run Test*` dies on "no matches found" before `go` is reached. `make t` also
|
||||
carries `-count=1` and sets `MAVEN_ONNX_LIB`. Without that variable the four
|
||||
`TestONNX*` measurements self-skip and the run still prints `ok`.
|
||||
|
||||
```sh
|
||||
make build # all 11 binaries
|
||||
make build-web # single daemon (pure-Go ones: web/waked/poll/caldav build without CGO)
|
||||
make test # go test -race across ./internal/... ./cmd/... with CGO env set
|
||||
```
|
||||
|
||||
Run one package or one test with `make t`. **Do not hand-write the CGO preamble.**
|
||||
Past sessions pasted it about 390 times. That is where the shell-quoting failures
|
||||
came from. This box runs zsh, so an unquoted `-run Test*` or `--include=*.go`
|
||||
dies on "no matches found" before `go` is ever reached.
|
||||
|
||||
```sh
|
||||
make t PKG=./internal/router/
|
||||
make t PKG=./cmd/mavend/ RUN=TestSimulator
|
||||
make build # all 11 binaries. make build-web for one (web/waked/poll/caldav skip CGO)
|
||||
make test # go test -race across ./internal/... ./cmd/... with CGO env set
|
||||
make t PKG=./internal/router/eval/ RUN='TestONNX' V=1 # V=1 for -v, RACE=0 to drop -race
|
||||
```
|
||||
|
||||
`t` carries `-race`, so a green `make t` cannot turn red under `make test`. It carries
|
||||
`-count=1`, so a cached PASS from before your edit is never mistaken for a result.
|
||||
## The daemons
|
||||
|
||||
It also sets `MAVEN_ONNX_LIB`, which the hand-written recipe did not. The four
|
||||
`TestONNX*` measurements self-skip when that variable is unset. The run still prints
|
||||
`ok`. So every targeted eval done the old way reported the hash ratchet while reading
|
||||
as a real embedder score.
|
||||
Eleven binaries under `cmd/`, wired socket-to-socket over `internal/ipc`, not
|
||||
linked. `mavend` is the core and owns the DB and the IPC socket.
|
||||
`deploy/mavend.json` sets sockets, model paths and the phraser and embedder
|
||||
blocks, with `${VAR}` expansion from gitignored `deploy/telegram.env`.
|
||||
**`docker-compose.yml` runs five**: `mavend`, `mavsttd`, `mavttsd`, `mavweb`,
|
||||
`mavpoll`. Count against compose, not against `make build`. `mavwaked` runs on
|
||||
workpc under systemd. `docs/deployment.md` says who else is absent and why.
|
||||
|
||||
Pure-Go packages (`router`, `memory`, `mavweb`, …) also run under a plain `go test ./pkg/`,
|
||||
but `make t` works everywhere and is one thing to remember.
|
||||
|
||||
## The daemons (`cmd/`)
|
||||
|
||||
| Binary | Role |
|
||||
|---|---|
|
||||
| `mavend` | **Core.** Router, phraser, memory, reminders, digestion tick. Owns the DB + IPC socket. |
|
||||
| `mavweb` | HTTP UI + PWA (`/dash`, `/history`, `/trace`, `/notifications`, `/tools`); WebAuthn auth. Connects to mavend's socket. |
|
||||
| `mavsttd` | Speech-to-text (whisper.cpp, CGO). |
|
||||
| `mavttsd` | Text-to-speech (piper subprocess). |
|
||||
| `mavwaked` | Wake-word / VAD gate. **Not on homesrv** — see below. |
|
||||
| `mavenclient` | Voice loop client (mic → stt → core → tts). **Not on homesrv** — see below. |
|
||||
| `mavpoll` | Environment poller: netdata alarms, uptime-kuma, zenmoney, wireguard presence. Writes facts, sends nothing. Telegram is `internal/delivery/telegramsink`, not this. |
|
||||
| `mavcaldav` | CalDAV calendar sync. |
|
||||
| `mavmaild` | Mail reader (IMAP, read-only). Holds the IMAP password; core never sees it. |
|
||||
| `mavgpud` | GPU supervisor. **Runs on workpc, not homesrv** — own unit, `deploy/mavgpud.service`. Keeps llama-server loaded while the card is free (V-488). Maven never asks it for anything, it reads `/health` through `llm.Pair`. |
|
||||
| `mavupdate` | Not a daemon. Operator CLI a human runs on the box to deploy a new build. |
|
||||
|
||||
Two more binaries have no Makefile target and are built with `go run` or `go build` when
|
||||
they are needed. Neither is deployed.
|
||||
|
||||
| Binary | Role |
|
||||
|---|---|
|
||||
| `mavseal` | Recovery tool. Encrypts a live tmpfs working copy back to the ciphertext file when mavend was killed before `defer st.Close()` sealed it. |
|
||||
| `labelgen` | Runs the stage 0 grammars over utterances and prints JSONL, the training data for the routing heads (V-546). |
|
||||
|
||||
Daemons are wired socket-to-socket, not linked. `internal/ipc` is the client/server wire
|
||||
protocol; the config in `deploy/mavend.json` (with `${VAR}` env expansion from gitignored
|
||||
`deploy/telegram.env`) sets socket paths, model paths, and the phraser/embedder blocks.
|
||||
|
||||
**`docker-compose.yml` runs five: `mavend`, `mavsttd`, `mavttsd`, `mavweb`, `mavpoll`.**
|
||||
Count against compose, not against the table. Four of the nine daemons are absent, and each
|
||||
absence has a different reason.
|
||||
|
||||
`mavmaild` and `mavcaldav` are commented out in compose, each with the reason written
|
||||
beside it: the first needs a mail account, the second a CalDAV account, and this box has
|
||||
neither. `mavcaldav` used to appear nowhere at all, which was an oversight; it became a
|
||||
recorded decision on 07-08-2026 (V-644). Two things ride on that absence and the block
|
||||
names them. Agenda questions route to `IntentQuery` at stage 0 (V-498) and the `calendar`
|
||||
query source then reads a table nobody writes. And `loop.State.CalendarBusy` is fed by the
|
||||
same facts, so the gate's "do not nag mid-meeting" is permanently false. Its password is
|
||||
read from a file (`-pass-file`, and `-render-pass-file` for the render collection), never
|
||||
taken as a flag value, which is the rule `mavpoll` and `mavmaild` follow too.
|
||||
|
||||
**`mavwaked` and `mavenclient` are absent by decision, not oversight** (Vikunja #463,
|
||||
`docs/plans/17-where-the-voice-loop-runs.md`).
|
||||
homesrv has a microphone — it is a laptop — but it is in the wrong room, so a wake-word
|
||||
daemon there listens to nobody. They belong on a client machine where the owner is standing.
|
||||
|
||||
**That machine is workpc** (owner's correction, 2026-08-05). This section used to say no
|
||||
such machine existed, which was written when the workstation was only a model host. It is
|
||||
where he sits most of the day and it has the microphone. `ipc.Dial` already takes
|
||||
`tcp://host:port?token=...` through the netaddr seam, so the two daemons need deploying,
|
||||
not building. V-515 is that deployment.
|
||||
|
||||
Until they are deployed, **the wake word and the VAD gate are covered by unit tests and by
|
||||
nothing else**, and push-to-talk through `/dash` is what QA actually covers. Note that
|
||||
deploying them does not by itself prove a wake word: `mavwaked` gates on energy and has no
|
||||
keyword model (V-487), so the loop runs open until that lands.
|
||||
- **Passwords are read from files, never taken as flag values.**
|
||||
- **The voice wire is plaintext with no auth.** mavend's voice port stays on
|
||||
homesrv loopback and reaches workpc over ssh. Do not LAN-bind it.
|
||||
`SurfaceVoice` caps acts at L0, and L0 does not cap reading.
|
||||
- **A GPU service added beside mavgpud goes in `cmd/mavgpud`, never in systemd.**
|
||||
The card needs one owner. A second unit made mavgpud evict llama-server every
|
||||
few seconds and took the model arm down for eight minutes.
|
||||
|
||||
## The ecosystem: Nexus, Praxis, Hexis
|
||||
|
||||
Maven is one of four services. It owns conversation and personal memory. It does not
|
||||
Nexus identifies, Praxis observes, Hexis acts, Maven understands. Maven does not
|
||||
own identity, operational state, or execution. Full contract in
|
||||
`docs/ecosystem.md`.
|
||||
`docs/ecosystem.md`. All three are `nil` unless configured and each degrades
|
||||
alone. An outage means a named gap, never a broken turn or a guess.
|
||||
|
||||
```text
|
||||
Nexus identifies. Praxis observes. Hexis acts. Maven understands and coordinates.
|
||||
```
|
||||
|
||||
| Service | Owns | Maven's client | Configured at |
|
||||
|---|---|---|---|
|
||||
| **Nexus** | Canonical entity ids, names, aliases, relationships. Projects, services, devices, people, pets, places. | `nexusClient` in `cmd/mavend/ecosystem.go`, `POST /api/v1/resolve` | `nexus.url` (`http://nexus:9740`) |
|
||||
| **Praxis** | Operational attention and item lifecycle. What needs looking at, what changed, what is still unresolved. | `praxisClient`, the HTTP tools API under `/api/v1/tools/` | `praxis.url` (`http://praxis:8989`) |
|
||||
| **Hexis** | The capability registry and the only path to executing anything. | vendored `github.com/kami/hexis/pkg/client` | `hexis.url` (`http://hexis:9741`) |
|
||||
|
||||
All three are `nil` unless configured, and every one of them degrades on its own.
|
||||
An outage means a named gap in the answer, never a broken turn and never a guess.
|
||||
|
||||
Rules that are not negotiable:
|
||||
|
||||
- **No component reads another component's database.** Praxis attention comes over
|
||||
HTTP, never from its SQLite file.
|
||||
- **Identity lives in Nexus.** Do not invent a local fact key for something Nexus
|
||||
resolves. `actionFact` already sets `Subject`, and `cmd/mavend/factenrichment.go`
|
||||
resolves it in the background against Nexus.
|
||||
- **Free text never reaches a mutating Hexis call.** Resolve to a canonical entity id
|
||||
first. Ambiguous resolution asks the owner, it does not pick.
|
||||
- **No component reads another component's database.** Praxis attention comes
|
||||
over HTTP, never from its SQLite file.
|
||||
- **Identity lives in Nexus.** Do not invent a local fact key for something
|
||||
Nexus resolves. `cmd/mavend/factenrichment.go` resolves `actionFact.Subject`.
|
||||
- **Free text never reaches a mutating Hexis call.** Resolve to a canonical
|
||||
entity id first. Ambiguous resolution asks the owner, it does not pick.
|
||||
- **LLM output is not authorization.** Confirmation binds capability id, target
|
||||
entity, arguments, requester and expiry. See `cmd/mavend/confirm.go`.
|
||||
- **Praxis lifecycle words mean different things.** Surfaced is not acknowledged,
|
||||
acknowledged is not resolved, execution success is not recovery. Reading an item
|
||||
aloud calls `Surface`, never `Acknowledge`.
|
||||
- **No automatic attention-to-action path.** Digestion may summarise Praxis. It may
|
||||
not call Hexis.
|
||||
entity, arguments, requester and expiry (`cmd/mavend/confirm.go`).
|
||||
- **Praxis lifecycle words differ.** Surfaced is not acknowledged, acknowledged
|
||||
is not resolved, execution success is not recovery. Reading an item aloud
|
||||
calls `Surface`, never `Acknowledge`.
|
||||
- **No automatic attention-to-action path.** Digestion may summarise Praxis and
|
||||
may not call Hexis.
|
||||
- Every cross-service call carries a correlation id minted once per action
|
||||
(`withCorrelationID`), a contract version header, and `X-Requested-By: maven`.
|
||||
|
||||
Every cross-service call carries a correlation id minted once per action
|
||||
(`withCorrelationID`), a contract version header, and `X-Requested-By: maven`.
|
||||
## Routing
|
||||
|
||||
## Routing — read this before touching the router
|
||||
**Read `docs/routing.md` before touching `internal/router/` or `queryWalk`.** It
|
||||
carries the reasoning, the measurements and every rule's why. A route produces
|
||||
two decisions. **Intent** is one of seven values. **Source** is where the answer
|
||||
lives and is read on `IntentQuery` alone. Score them separately. The cascade is
|
||||
stage 0 grammars, then the routing heads, then the resident model, then the
|
||||
classifier. Every stage may decline and the next one answers.
|
||||
|
||||
`internal/router/` has TWO layered engines. **The LLM router is now the default and it is
|
||||
on in deploy** — this section used to say it was wired `nil`, which stopped being true on
|
||||
2026-07-31.
|
||||
- **The classifier is the floor, not dead code.** It answers when the resident
|
||||
model is off, absent, or erroring. **Any model error falls through.**
|
||||
- **`baselineGrammars` in `eval_test.go` mirrors `buildRouter`.** A grammar
|
||||
added to one belongs in both, or the fixture scores a set nobody runs.
|
||||
- **Go's `\b` is ASCII-only** and never fires after a Cyrillic letter. A Russian
|
||||
pattern needs an explicit `(\s|[?!.]|$)`.
|
||||
- **`PraxisGrammars()` is the only path to Praxis**, not a faster one.
|
||||
- **`voice.embedder.heads_path` must never point at `model_path`.** Recall
|
||||
depends on the resident e5-small scoring what it scored. Fine-tune a copy.
|
||||
- **Routing traces are retained 14 days**, enforced on write and again on start.
|
||||
- **Bump `tokenizerRev` on any change to what `encodeWord` emits**, so a
|
||||
tokenizer fix triggers `ReembedAll` the way swapping the model file does.
|
||||
- **A new rung in the `runTurn` ladder needs its name in `preRouteLadder`**
|
||||
(`cmd/mavend/decisiontrace.go`), or it is missing from the decision record.
|
||||
|
||||
- **LLM router (the intended design, docs/rearchitecture.md):** the resident Qwen3-1.7B (`llmrouter.go`)
|
||||
emits GBNF-constrained structured JSON, and the SAME model phrases replies. Embedder is
|
||||
demoted from a routing gate to a RAG hint. Wired at `voice.go:214` via
|
||||
`pickLLMRouter(cfg.Voice.UseLLMRouter(), llmClient)`; the flag is `voice.llm_router`
|
||||
(`config.go`), `DefaultLLMRouter` is **on**, and `deploy/mavend.json` sets it `true`.
|
||||
- **Classifier cascade (the failure floor, not dead code):** `classifier.go` +
|
||||
`embedder.go` nearest-neighbour over frozen seed phrases. It runs when the LLM router is
|
||||
off, when there is no llama-server to talk to (`pickLLMRouter` logs that and degrades),
|
||||
and on any per-turn LLM error. Do not delete it — routing by seed similarity is the known
|
||||
cause of weak RU query handling, but a turn must never break on the model.
|
||||
**`queryWalk` takes query sources out and moves none** (`actions_query.go`). That
|
||||
is the safety argument and it is not negotiable. The table's order is
|
||||
load-bearing and carries "the owner's data first, then the world".
|
||||
`SourceUnknown` is the floor and walks the whole chain. A named destination
|
||||
removes only the sources marked `guesses: true`, so a source that looks rather
|
||||
than guesses is always asked. **The personal boundary is the one exception and
|
||||
it is deliberate.** It guesses, so naming `SourceWorld` drops it. **Only a stage
|
||||
0 grammar may drop it** (owner's call, V-666). `queryWalk` reads
|
||||
`Decision.SourceAnchored` for the source marked `boundary: true` and no other.
|
||||
|
||||
Cascade order: `stage0.go` exact-match fast-path → LLM router (when non-nil) → classifier
|
||||
fallback. Any LLM error falls through to the classifier so a turn never breaks on the model.
|
||||
Judge a routing change against the classifier (76.0% intent, 36.4% destination)
|
||||
and the resident model (80.2% intent), since those always answer. The fixture
|
||||
has grown from 77 cases to 96, so a number compares only to another number on
|
||||
the same fixture.
|
||||
|
||||
**A stage-0 decision is slot-extracted too, since 06-08-2026** (V-572). `fillMatchedSlots`
|
||||
in `router.go` runs the stage-2 extractor over whatever a grammar built and fills only the
|
||||
slots it left empty — a matched value always wins, because the rule read a literal pattern
|
||||
and the extractor guesses. It did not run before, so `ReminderGrammar` handed the daemon
|
||||
`HasTime: false` for "напомни в 11:00 позвонить маме" and `missingFor` read the silence as
|
||||
absence and asked "Когда?". It applies to every grammar and is inert for all but the
|
||||
reminder: `Extract` fills Time, Fn and Key and nothing else, and the query, clock, agenda,
|
||||
feed, list, task and narrative rules all emit intents with no such slot. Benchmarked at
|
||||
20000x, a stage-0 query costs 3.7µs against 3.9µs before. **`Slots.Text` is deliberately not
|
||||
filled** — a grammar that left it empty meant it, and `agendaQueryBuild` hands the query
|
||||
chain the utterance itself. Fixture unchanged at 64/91, with "slots deferred to daemon"
|
||||
6 → 0.
|
||||
## Language: model output and Russian
|
||||
|
||||
Measured on the 77-case RU fixture. **Re-measured 2026-08-02: the classifier scores 68.8%
|
||||
full accuracy at p50 16.6µs**, not the 36.8% at p50 31ms that stood here from
|
||||
`docs/evals/2026-07-31-model-bakeoff.md`. That older figure predates the stage 0 rules and the
|
||||
seed additions, both of which now score inside the classifier baseline. Qwen3-1.7B scores
|
||||
77.9% intent-only / 72.7% through the cascade. So the router buys about 4 points of accuracy,
|
||||
not a doubling, and the trade is worth re-arguing rather than assuming. **The ≈2.7s figure
|
||||
that stood here until 2026-08-02 was contention, not the model.** See `docs/evals/2026-07-31-routing.md` line 61, which measures the LLM router at
|
||||
p50 825ms / p95 1.2s / max 3.0s and the full cascade at p50 0.80-1.04s. Do not plan latency
|
||||
work off the bakeoff table.
|
||||
Both contracts are in `docs/language.md`. What must not be broken:
|
||||
|
||||
**Re-measured 2026-08-05 on the fixture as it now stands, 91 cases** (V-320 item 2,
|
||||
`docs/evals/2026-08-05-routing-resident-model.md`): cascade + resident model scores
|
||||
**75.8% full / 80.2% intent-only at p50 1.19s / p95 1.65s**. That is a new baseline and not
|
||||
a movement, because 14 cases were added since the 77-case number above. The model alone
|
||||
scores 37.4% full against 61.5% intent-only, and the gap is slots rather than routing: it
|
||||
routes `reminder` and leaves the time to the daemon, which is what the contract asks. To
|
||||
re-run it, start a **second** llama-server on a fixed host port — the resident one binds
|
||||
`--port 0` inside the container and no host process can reach it.
|
||||
- **One parser for model text, `parseResponseMood`** in
|
||||
`internal/phraser/parse.go`. Every phrasing path reaches it. Mood is an enum.
|
||||
- **The router prompt is a separate contract** over 7 intents, and
|
||||
`llm/check_prompt_parity.py` keeps the Go and relabelling copies identical.
|
||||
- **Russian words are matched by three mechanisms and no fourth**:
|
||||
`internal/lexicon` for closed classes, `internal/morph` for grammar, and
|
||||
`cmd/mavend/topics.go` with the embedder for open sets. A regex whose output
|
||||
is a fact or a route is the defect. A regex over structured input is not.
|
||||
- **Seeds are scoring data.** Editing one moves a recogniser and must be
|
||||
re-measured against the `TestONNX*` tests, not eyeballed.
|
||||
|
||||
**The numbers above are the homesrv floor, not the ceiling.** With the workstation up, routing
|
||||
completes through `llm.Pair` against gemma-4-12b and scores **84.4% full / 93.5% intent-only at
|
||||
p50 329ms** — better than the resident model and about 2.5× faster (`docs/evals/2026-08-02-workstation-gemma4-12b.md`,
|
||||
Vikunja #485). The workstation is never assumed up, so both sets of numbers are live. Judge a
|
||||
routing change against the classifier and the resident model, since those are what always answer.
|
||||
## Non-goals and hard constraints
|
||||
|
||||
**The intended third engine is not a generative model** (owner's call, 05-08-2026, V-546,
|
||||
`docs/plans/18-routing-heads-on-e5-small.md`). Routing has a bounded output space, so it is
|
||||
classification, and the 118M multilingual-e5-small is already resident. Three heads on one
|
||||
forward pass: intent, mood, and BIO slot tags. Roughly 5e15 FLOPs to train, so 10 to 30
|
||||
minutes on the workstation. A 100M decoder from scratch is 10 to 20 GPU hours. Two things
|
||||
it buys that a decoder cannot. No grammar is needed, because a softmax cannot emit a value
|
||||
that does not exist. And max softmax is a calibratable confidence, where `Confidence: 1.0`
|
||||
was a hardcode. **Fine-tune a copy of the weights.** The resident embedder backs memory
|
||||
recall. Training it in place couples routing accuracy to recall@1, with nothing in the
|
||||
suite to name the trade.
|
||||
Not a nag, not autonomous.
|
||||
**The persona is feminine.** Russian self-reference takes feminine forms: `рада`
|
||||
not `рад`, `поняла` not `понял`. The owner is male and she speaks to him
|
||||
informally. Use "ты", singular, never "вы" or "ваш", and never "он" or "его".
|
||||
She talks TO the owner, not about him. Pet names such as "милый" are forbidden.
|
||||
The name "Ками" is not. `CheckAddress`, `CheckFeminine` and `CheckCringe` in
|
||||
`internal/phraser/eval/checks.go` enforce this, scored by `make eval-phrasing`.
|
||||
|
||||
**Two of those heads are trained as of 08-08-2026, and they are not the three
|
||||
above** (V-661, `docs/evals/2026-08-08-routing-heads-two-head.md`). Intent and
|
||||
destination share one masked mean pool. Destination scores a mean **80.8%** over
|
||||
three seeds, best **29/33 (87.9%)**. The classifier cascade scores 12/33 and the
|
||||
cascade with gemma-4-12b scores 24/33, so a 118M encoder beats the 12B teacher it
|
||||
was distilled from. Read the best run as one seed and not a headline, because one
|
||||
case is 3 points on a fixture this small.
|
||||
Recall is 15/15 and world is 5/5. Intent is 93.6% mean over three seeds. That is
|
||||
**not** comparable to the 76.0% and 84.4% those two arms scored: a softmax has no
|
||||
clarify class, so the head's fixture is the 88 cases carrying an intent.
|
||||
**"Never phones home" is deprecated** (owner's call, 2026-07-31). She reads
|
||||
external sources, and `docs/world.md` carries that chain. What holds regardless:
|
||||
|
||||
**A fourth head asks instead of guessing, same day** (V-661,
|
||||
`docs/evals/2026-08-08-clarify-head-four-head.md`). Clarify is not a value
|
||||
of intent, so a softmax cannot emit it. It is a second question over the
|
||||
same pooled vector: can Maven act on this at all. That is why the head's
|
||||
fixture was 88 cases and not 96. Over three seeds it catches **7.0 of the 8
|
||||
`want_clarify` cases and produces 2.3 false clarifies of 88**. The cascade
|
||||
today misses 1 and produces 2, so this is parity with no rules in front of
|
||||
it. Accuracy is the wrong number here and a head that never asks scores
|
||||
91.7%. Confidence is the other half. Max softmax over the intent head reads
|
||||
**0.851 where it is right against 0.604 where it is wrong**, ranking right
|
||||
above wrong in 83.4% of pairs. `Confidence: 1.0` was a hardcode, and this
|
||||
replaces it with a signal. The two are not the same signal: one says which
|
||||
intent is unclear, the other says the utterance carries too little to act
|
||||
on. **The fourth head is not free the way the third was.** Intent,
|
||||
destination and slot F1 each move down one to four points, inside the seed
|
||||
spread. `поужинал` is a false clarify on every seed, which is the same
|
||||
defect `thinSingleToken` was narrowed for on 2026-08-01.
|
||||
|
||||
The corpus for it is generated, because every existing row is answerable by
|
||||
construction. **The router-prompt agreement filter cannot work here**, since
|
||||
`routeGrammar` has no clarify value and a generated line always agrees with
|
||||
itself. A gemma judge replaces it. The first judge called 24 of 40
|
||||
answerable rows underspecified, because it judged against a generic
|
||||
assistant rather than against Maven's contract.
|
||||
|
||||
**Mood is cut, not deferred.** The enum describes her own reply state, not the
|
||||
speaker's emotion, and no dataset maps onto it.
|
||||
|
||||
**A third head landed the same day** (`docs/evals/2026-08-08-slot-head-three-head.md`).
|
||||
BIO slot tags had no Maven-domain corpus, which was true of found corpora and
|
||||
false of made ones. `label_slots.py` distils spans out of gemma-4-12b under a
|
||||
GBNF closed over Maven's own five slots. A span survives only when it is a
|
||||
literal substring of the utterance, so the agreement filter costs no second
|
||||
call. 2178 spans over 1702 rows, 37 dropped, nothing unparsed. Three heads score
|
||||
intent **92.8%**, destination **82.8%** and slot span F1 **72.4%** over three
|
||||
seeds. The slot head is free: both other numbers move less than their own seed
|
||||
spread. Epoch selection reads the intent dev slice alone. Slot F1 is still
|
||||
climbing when it stops, which costs about 4 points.
|
||||
|
||||
The MASSIVE warm-start of step 2 is worth nothing here. Stock e5-small ties it on
|
||||
intent and leads by a third of a case on destination. Nothing argues for keeping
|
||||
that step.
|
||||
|
||||
The floor was a corpus defect and it is fixed. The first 120 floor rows carried
|
||||
one sentence shape, so the head named a destination where the fixture says walk
|
||||
the chain. Rotating six shapes took the floor 3/7 to 6/7 and destination 75.8% to
|
||||
80.8%. What is left is calendar at 3/6 on every seed, which training cannot move:
|
||||
the possessive agenda rules claim those cases at stage 0 and name nothing, so no
|
||||
label reaches the head. That is the same trade V-660 flagged and it wants the
|
||||
owner's call.
|
||||
|
||||
**Nothing of this runs in Go.** The weights are `heads.pt` and `out/body_heads/`
|
||||
on workpc. Reaching the daemon needs an ONNX export and a caller. The resident
|
||||
e5-small must not be replaced by the copy, because recall depends on that file.
|
||||
|
||||
`Confidence: 1.0` used to be hardcoded in `llmrouter.go`, so the LLM
|
||||
path could never ask for clarification (6/6 refusal cases missed on the fixture) — Vikunja
|
||||
#359. Fixed 31-07-2026 with structural signal (single-token utterance, keyless fact, act with
|
||||
no allowlisted fn) feeding the same stage-3 gate the classifier path already had — see
|
||||
`gateLLMDecision` in `router.go`. Note the second half of that bug: the LLM branch never
|
||||
consulted `r.threshold` at all, so a correct low confidence would have been discarded anyway.
|
||||
|
||||
Re-measured on the fixture after the fix: **missed clarify 6/6 → 1**, at the cost of 3 false
|
||||
clarifies and 2.6pt of full accuracy (72.7% → 70.1%, intent-only 67.5% → 74.0%). Two of the
|
||||
three false clarifies are acts the model mis-routed and the gate caught — asking beats wrongly
|
||||
executing, so the fixture and the daemon disagree about what is correct there. The third,
|
||||
`"поужинал"`, was a real defect: the single-token rule was an English intuition and does not
|
||||
transfer to Russian, where one word is routinely a whole sentence.
|
||||
|
||||
Narrowed 01-08-2026. `thinSingleToken` (`internal/router/singletoken.go`) still thins a bare
|
||||
one-word nominal — "вода", "бэкап" — but spares two classes: a closed lexicon of social and
|
||||
control singles ("привет", "спасибо", "стоп", "yes"), and any token carrying a Russian verb
|
||||
ending (past tense, 2nd person, reflexive), because a verb already contains its subject. Both
|
||||
tests are offline and cost nothing. Re-measured: **false clarifies 3 → 2, intent-only 74.0% →
|
||||
75.3%, full accuracy unchanged at 70.1%, missed clarify still 1.** The two remaining false
|
||||
clarifies are the act-with-no-allowlisted-fn arm of the gate, not this rule.
|
||||
|
||||
Agenda questions taken off the model, 01-08-2026. `AgendaQueryGrammars` (`stage0.go`, wired
|
||||
after the clock rules in `buildRouter`) routes "что у меня сегодня", "во сколько у меня
|
||||
встреча" and anything naming a calendar to `IntentQuery` at stage 0. They were going to
|
||||
`IntentSystem`, where `replySystem` has no agenda arm and answered "пока не умею" — the
|
||||
fixture had said `query` since ru-query-019 was written. Measured: **full accuracy 70.1% →
|
||||
72.7%, intent-only 75.3% → 77.9%, calendar 0/2 → 2/2**, clarify counts unchanged. Note that
|
||||
Go's `\b` is ASCII-only and never fires after a Cyrillic letter; the pattern needs an
|
||||
explicit `(\s|[?!.]|$)`.
|
||||
|
||||
Two more shapes taken off the model, 04-08-2026 (V-498). `rest-of-day-query` inside
|
||||
`AgendaQueryGrammars` claims "что дальше?" / "what's next", and `NarrativeQueryGrammar`
|
||||
(`stage0.go`, wired **last** in `buildRouter`, after the capture marker) claims "расскажи про
|
||||
X", "объясни X", "опиши X". Neither carries a question mark or an interrogative, so the model
|
||||
called both `IntentFact`; the write was caught downstream by `IsQuestionShaped`, so this was a
|
||||
latency and fixture defect, not a correctness one. The narrative rule reads the same
|
||||
`narrativeRequests` lexicon `IsQuestionShaped` reads, and declines `chatNarrativeTopics` — a
|
||||
joke, a bedtime story, herself — because the query chain has no source that answers those.
|
||||
New fixture cases ru-query-024 and ru-query-025. Classifier + ONNX baseline **56/80 (70.0%) →
|
||||
58/82 (70.7%)**, no case regressed, no new false clarify. The LLM arm was not measured (no
|
||||
llama-server in that run), so judge it again before quoting a cascade number.
|
||||
|
||||
Praxis taken off the model, 05-08-2026 (V-516). `PraxisGrammars()`
|
||||
(`internal/router/praxis.go`, wired in `buildRouter` before the capture marker because
|
||||
"отметь" is a capture verb) fills `Slots.Fn` with a Praxis capability name.
|
||||
**These grammars are the only path to Praxis, not a faster one.** Measured
|
||||
2026-08-05 with the resident model as router (V-517,
|
||||
`docs/evals/2026-08-05-reach-llm-router.md`): the model alone reaches Praxis
|
||||
**0/12**, the same as the classifier alone, because nothing in the router
|
||||
prompt names a Praxis capability and there is no string for it to write.
|
||||
Through the cascade it is 11/12. Deleting these rules costs every point. Praxis reach
|
||||
was **0/12 and structurally so**: `handlePraxisAct` compares `Slots.Fn` to a capability
|
||||
alias, and that slot is filled from the deployment's enabled tool names, which no Praxis
|
||||
alias is on. Measured **16/30 → 27/30 overall, praxis 0/12 → 11/12, lifecycle 0/5 → 5/5**
|
||||
(`docs/evals/2026-08-05-praxis-reach.md`). Two rules to know before editing: a **stative**
|
||||
lifecycle word ("готово", "принято") needs an item named beside it, while a bare
|
||||
**imperative** ("закрывай") may ask which one. The bare arm additionally requires that
|
||||
the sentence name no object of its own, or "закрой шторы в комнате" goes to Praxis instead
|
||||
of the house. A demonstrative ("отметь это как сделанное") resolves against
|
||||
`h.surfacedItems` only when exactly one item was spoken. Otherwise the turn goes back to
|
||||
the cascade rather than transitioning the wrong item.
|
||||
|
||||
**Who claimed a turn is now recorded, and so is who did not** (V-564, umbrella
|
||||
V-558). Arbitration between the claimants on the utterance stream is order,
|
||||
hardcoded in the pre-route resolver ladder, in `buildRouter` and in
|
||||
`querySources`. `internal/decision` records one `Record` per turn: every
|
||||
claimant, what it would have made the turn, the score it reported, and whether
|
||||
it won, declined, lost on score, was thinned by a gate or was **never asked**.
|
||||
The record rides the context, the same seam `querysource.go` uses, so a claim
|
||||
site cannot change a route and a context with no record costs nothing. It is
|
||||
installed in `runTurn`, so the mic, telegram and the web all leave the same
|
||||
trail. Storage is a 25-turn in-memory ring on the handler (`decision.Ring`),
|
||||
read over `ipc.TurnDecisions` and rendered as the second table on `/trace`.
|
||||
**It also persists, since 06-08-2026, and that reverses what this section used to
|
||||
say** (V-629, `docs/plans/21-persisting-the-routing-trace.md`). The old rule was
|
||||
that nothing persists, because a turn record is read minutes later or never. The
|
||||
owner reversed it: the routing heads (V-546) cannot be fitted or calibrated
|
||||
without real utterances, and 9 of the 31 modes in `internal/modes` have no seed
|
||||
example at all. The ring did not move. It is still what `/trace` reads and still
|
||||
what a test with no store gets. `cmd/mavend/routingtrace.go` is a second sink
|
||||
beside it, writing `routing_traces` (migration #23). The utterance is stored in
|
||||
clear, because a 384-dimension vector of a short sentence is substantially
|
||||
recoverable and storing vectors instead would be a privacy claim we cannot
|
||||
support. What makes it safe is the same thing that makes the fact store safe.
|
||||
Retention is 14 days, enforced on write and again on start, so a box that goes
|
||||
quiet does not keep every row. Nothing reads it outward, and the rule
|
||||
that his notes and facts are never search input covers this table. `Store.Wipe`
|
||||
deletes it with everything else. A correction (V-630) is promoted out into a
|
||||
seed-shaped row in `routing_labels` (migration #24) and kept, because a label is
|
||||
not a transcript. The transcript still expires. The gesture that writes one is
|
||||
two buttons beside the reply on `/chat`, reached over `ipc.CorrectTurn` and the
|
||||
trace id that now rides back on `ipc.ChatReply`. A turn marked wrong with no
|
||||
target is a usable negative, so naming the intent is never required. The target
|
||||
is one of the seven intents and never free text. **All three reaches offer it as
|
||||
of 06-08-2026**, and this section used to say only `/chat` did. Voice is the
|
||||
`repair` rung, which has read spoken corrections since V-455 and now writes the
|
||||
durable label beside the classifier seed it always wrote; a spoken negative with
|
||||
no target is its own rung, `repair-negative` (V-636, `docs/plans/22-correcting-a-turn.md`).
|
||||
Telegram is an inline keyboard under the reply, and it needed the chat to become
|
||||
readable first — **telegram is no longer outbound only** (V-637,
|
||||
`docs/plans/23-inbound-telegram.md`). The poller is dark unless the `telegram`
|
||||
block says `intake`, it long-polls because the box takes no inbound connections,
|
||||
it accepts `chat_id` and no other sender, and it drops whatever queued while the
|
||||
daemon was down. It reaches the daemon through `ipc.CoreAPI` alone, so a chat
|
||||
turn takes the path `POST /api/chat` takes. Note that the turn source is still
|
||||
`tap:text` for both, so provenance cannot tell a chat turn from a typed one.
|
||||
Adding a rung to the ladder
|
||||
in `runTurn` means adding its name to `preRouteLadder` in
|
||||
`cmd/mavend/decisiontrace.go`, or that rung is silently missing from the record.
|
||||
|
||||
**A route now says where the answer lives, not only that the turn is a question**
|
||||
(V-655, 07-08-2026). `query` was a shrug. The cascade sorted an utterance into one of
|
||||
seven intents, with stage 0, the resident model and the classifier behind it. Then
|
||||
`IntentQuery` handed the turn to `querySources` in the daemon. That is twenty-two branches
|
||||
deciding by seed similarity in a fixed order. It has no fixture and no accuracy
|
||||
number, no model arm and no floor. `Decision.Source` (`internal/router/source.go`) is
|
||||
the second half of the route. Twelve destinations, not twenty-two. The three recall
|
||||
passes plus `fact-by-key` are one destination from outside. So are search, Kiwix and
|
||||
the URL reader.
|
||||
|
||||
**`SourceUnknown` is a real value and it is the floor.** Nothing named a destination,
|
||||
so the daemon walks the whole chain. That is byte-for-byte what shipped before the
|
||||
field existed. The classifier arm names nothing, so a box whose model is down routes
|
||||
queries exactly as it did.
|
||||
|
||||
`queryWalk` in `cmd/mavend/actions_query.go` takes sources **out** and moves none.
|
||||
That is the safety argument and it is not negotiable. The table's order is
|
||||
load-bearing. Every comment on it argues a reason between two sources, and above all
|
||||
it carries "the owner's data first, then the world". Naming `SourceWorld` does not
|
||||
send the turn outside on its own. His notes and his facts still run first, because
|
||||
they look rather than guess.
|
||||
|
||||
**The personal boundary is the one exception and it is deliberate.** It guesses,
|
||||
so naming `SourceWorld` drops it. That is what stops it answering "кто такой
|
||||
Линус Торвальдс?" with "не нашла у тебя такой записи", which it did on
|
||||
2026-08-07. The cost is that a destination a model wrote can now take the
|
||||
boundary off a turn. A question about him that the model calls `world` reaches
|
||||
SearXNG, where today the boundary stops it. Only the utterance leaves the box,
|
||||
never his notes or history, so this widens what is asked and not what is sent.
|
||||
`TestNamingRecallKeepsTheBoundary` pins the other half: naming `SourceRecall`
|
||||
keeps the boundary in front of the world. Whether a model may drop it at all is
|
||||
the owner's call and has not been made.
|
||||
|
||||
What comes out is only the sources that **guess**. Those decide a turn is theirs by
|
||||
cosine against frozen seeds, then answer whatever they claimed. They hold no table
|
||||
that could come back empty. Weather is the pure case and has no local data at
|
||||
all. It was measured on the box on 2026-08-07
|
||||
(`docs/evals/2026-08-07-week-of-usage.md` section 4). It answered both "что такое
|
||||
TCP?" and "сколько будет 17 на 23?" with "для какого города?". The feed answered "какой у меня любимый язык?" with kernel headlines.
|
||||
The personal boundary answered "кто такой Линус Торвальдс?" with "не нашла у тебя
|
||||
такой записи". A source that guesses is marked `guesses: true` in the table. One that
|
||||
looks is not, and it is always asked.
|
||||
|
||||
Stage 0 fills the destination where a rule already knows it. `WorldQueryGrammars()`
|
||||
(`internal/router/worldquery.go`) claims "что такое X" and "сколько будет 17 на 23".
|
||||
It is wired after the agenda rules and **before** the feed and list rules.
|
||||
"что такое лента" is a definition question, and the feed rule would take it on the
|
||||
noun alone.
|
||||
`calendar-query` and `event-time-query` name the calendar. The possessive agenda rules
|
||||
deliberately do not. "что у меня в списке покупок" matches `agenda-query`, and naming
|
||||
the calendar there would take the list source off the turn.
|
||||
|
||||
Fixture unchanged at **69/91 classifier+ONNX**, measured both sides. That is the
|
||||
expected result, because it scores intent and no case here changes intent.
|
||||
|
||||
**The destination has its own fixture and its own number as of 08-08-2026**
|
||||
(V-659, `docs/evals/2026-08-08-destination-fixture.md`). This section used to say
|
||||
it had neither. `want_source` on `eval.Case` is a pointer, because the destination
|
||||
has three states and a bare string has two. Absent is every intent but query,
|
||||
which never reaches `queryWalk`. Present and empty is the `SourceUnknown`
|
||||
contract: name nothing and walk the chain. Present and named is a destination the
|
||||
route must produce. Thirty-three of ninety-six cases carry one.
|
||||
|
||||
A destination miss does **not** fail the case. It lands in `Outcome.SourceReason`
|
||||
and never in `Reasons`, so `Accuracy` and `IntentAccuracy` mean what they meant
|
||||
and `SourceAccuracy` is a second number over the labelled cases only. Intent and
|
||||
destination are two decisions, and one number hides which one moved. A route that
|
||||
lost its intent scores no destination hit, or a clarify would satisfy an empty
|
||||
label for free.
|
||||
|
||||
Measured classifier+ONNX: intent **73/96 (76.0%)**, destination **12/33 (36.4%)**.
|
||||
The split is the finding. World is 5/5, because a stage 0 rule names it. The
|
||||
`SourceUnknown` floor is 5/7. Calendar is 2/6, because the possessive agenda
|
||||
rules deliberately do not name it. And **recall is 0/15, because nothing
|
||||
anywhere names it**. Those turns are still answered, since the chain walks
|
||||
recall early. Recall is the number the fourth head has to move.
|
||||
|
||||
Seven cases assert the floor and five of them are homelab operations. They
|
||||
cluster because `SourceRecall`, `SourceNetwork` and `SourceAttention` overlap on
|
||||
every question about the box. The other two are `ru-query-005` and
|
||||
`ru-query-014`. No query source reads the reminder store, and a deadline could
|
||||
sit in tasks, the calendar or Praxis. `mavpoll` writes its netdata and uptime-kuma
|
||||
observations into the fact store recall reads. That is a finding about the enum,
|
||||
not a gap in the labelling. The owner confirmed all seven floor labels on
|
||||
08-08-2026, so they are a decision rather than an agent's guess.
|
||||
|
||||
`baselineGrammars` in `eval_test.go` mirrors `buildRouter` and had drifted:
|
||||
`WorldQueryGrammars` was wired into the daemon by V-655 and not into the mirror,
|
||||
so the fixture scored a grammar set nobody runs. Fixed by V-659, worth 3 points of
|
||||
destination and nothing else. Check that function when adding a grammar.
|
||||
|
||||
**The model arm landed the same day** (V-660,
|
||||
`docs/evals/2026-08-08-destination-model-arm.md`). `routeGrammar` carries a
|
||||
`source` rule closed over `router.Sources` plus the empty floor, so the model
|
||||
cannot emit a destination that does not exist. The prompt lists the twelve in
|
||||
Russian and says `""` is a normal answer to give often. `LLMRouter.Route` reads it
|
||||
back through `ValidSource` and on `IntentQuery` alone. Against gemma-4-12b on the
|
||||
workstation the cascade scores destination **24/33 (72.7%)** with intent unmoved
|
||||
at 84.4%, and **recall goes 0/15 to 14/15**. The resident Qwen3-1.7B is
|
||||
unmeasured, because it binds `--port 0` inside the container.
|
||||
|
||||
**Stage 0 now costs four destination points.** It did not before. The four cases
|
||||
the cascade loses and the model alone wins are all calendar. The possessive
|
||||
agenda rules claim them first and name nothing on purpose. That caution was free
|
||||
while nothing downstream could name anything either. It is not free now, and the
|
||||
fix is the owner's call rather than a quiet edit.
|
||||
|
||||
The last arm is V-546. Intent, mood and BIO slot tags were already three heads on
|
||||
one forward pass of the resident e5-small. Destination is a fourth head on the
|
||||
same pass, and 72.7% from a 12B teacher is the label source for training it.
|
||||
|
||||
## LLM output contract
|
||||
|
||||
All phrasing paths emit `{"response":"...","mood":"..."}`, with fallback to plain text when
|
||||
the model skips the JSON. **One parser, `parseResponseMood` in
|
||||
`internal/phraser/parse.go`**, and every path reaches it: the six `LLMPhraser` methods,
|
||||
`PhraseWorld`, and `Replier.PhraseReply`, which `cmd/mavend/replier_llm.go` wraps — that file
|
||||
holds the stub fallback and no parsing of its own. The legacy `{"body","summary"}` fallback
|
||||
was deleted on 2026-08-06 (V-397): it was the contract before `{"response","mood"}` replaced
|
||||
it, no prompt asks for that shape, the GBNF cannot emit it, and no test covered it.
|
||||
Mood is a fixed enum. Router prompt is a separate contract:
|
||||
`[{"intent":<enum>, key?, value?, text?, verb?}, ...]`, 7 intents (`fact, reminder,
|
||||
note, query, act, chat, system`). `llm/check_prompt_parity.py` in the training
|
||||
workspace enforces that the Go and relabelling prompts remain identical.
|
||||
|
||||
## Russian patterns — three mechanisms, no fourth
|
||||
|
||||
Hand-written Russian stem patterns were swept out on 2026-08-04 (owner's call: not a
|
||||
pattern, and the resident model cannot be asked per turn either). A regex whose output is a
|
||||
fact or a route is the defect; a regex over structured input — HTML, MIME, JSON, a URL, an
|
||||
argv list — is not. Before writing a Russian word list, pick one of these:
|
||||
|
||||
- **`internal/lexicon`** — closed classes, in `lexicon_ru_v1.json`. Interrogatives,
|
||||
capture verbs, reminder verbs, cardinals, day offsets, parts of day, weekdays, months,
|
||||
spoken hours. Editing a word is a data change, and there is exactly one copy: months used
|
||||
to live in three files. Cardinals carry the oblique forms, because a spoken time declines
|
||||
and `в семь` / `к семи` are one hour.
|
||||
- **`internal/morph`** — grammar, from the vendored golem Russian dictionary. `IsVerbForm`
|
||||
and `SameWord`. Note that lemma matching is BROADER than stem-plus-one-ending, so a verb
|
||||
slot that means the imperative must be matched exactly — `говори` and `говорил` are one
|
||||
lemma and only one of them is a command (`cmd/mavend/quiet_toggle.go`).
|
||||
- **`cmd/mavend/topics.go` and the embedder** — open sets, where the question is what a
|
||||
turn is ABOUT. Frozen seeds per subject plus a real `other` class, scored against the
|
||||
turn's own query vector. Same shape as the personal boundary in `personalboundary.go`,
|
||||
with one difference: a topic must clear the runner-up by `topicMargin`, because a false
|
||||
claim here spends a network scan rather than one honest "не знаю". The old keyword tests
|
||||
stay as the offline floor and may remain narrow, since they are no longer the only answer.
|
||||
- **The ecosystem trio** — when the answer is not in the utterance at all. Identity is
|
||||
Nexus's, never a local pattern.
|
||||
|
||||
Seeds are scoring data. Editing one moves a recogniser and must be re-measured against the
|
||||
`TestONNX*` tests, not eyeballed.
|
||||
|
||||
## Non-goals (hard constraints)
|
||||
|
||||
Not a nag, not autonomous. Maven's persona is **feminine** — Russian
|
||||
self-reference must use feminine forms — `рада`, not `рад`; `поняла`, not `понял`. The owner
|
||||
is male and is addressed informally: "ты", singular, never "вы"/"ваш" and never "он"/"его"
|
||||
(she talks TO the owner, not about the owner). Pet names ("милый", "дорогой") are forbidden; the name
|
||||
("Ками") is not. The eval enforces this: `CheckAddress`, `CheckFeminine` and `CheckCringe` in
|
||||
`internal/phraser/eval/checks.go`, scored by `make eval-phrasing`.
|
||||
|
||||
**"Never phones home" is DEPRECATED** (owner's call, 2026-07-31). It used to be a hard
|
||||
constraint and it is not one any more: a 0.8B — and a 1.7B — does not know enough to answer
|
||||
world questions, so she needs to read external sources. What replaces it:
|
||||
|
||||
- **No telemetry, no cloud model, no third-party account.** That part never changes. Nothing
|
||||
about Maven is reported to anyone, and inference stays on the box.
|
||||
- **The owner's data first, then the world.** Every source that reads the owner's facts, notes, calendar,
|
||||
tasks or house runs before anything outside, and the personal boundary sits between them.
|
||||
Reading beats recalling for a small model.
|
||||
- **In the world, live search leads and the ZIMs are the fallback** (owner's call,
|
||||
2026-08-02). A self-hosted SearXNG (`search` block) answers first; the Kiwix ZIMs on
|
||||
homesrv answer when the search is empty, unreachable, or the line is down.
|
||||
**Verified with the line down on 2026-08-05** (V-508,
|
||||
`docs/evals/2026-08-05-kiwix-offline-fallback.md`): a stopped SearXNG costs nothing,
|
||||
the ZIM answers in the same turn budget. A blackholed host cost 8 seconds the owner waited
|
||||
through. So the connect phase alone is capped at `dialTimeout` (1.5s), while a slow
|
||||
instance that did connect keeps the full 8. **A Russian question reads
|
||||
`wikipedia_ru_all_maxi_2026-02` verbatim** through `kiwix.book_ru`. The RU→EN rewriter
|
||||
is the workaround for an English book and is skipped there. Kiwix catalog names come
|
||||
from the filename, not the `<name>` field.
|
||||
`Response.Empty()` is the whole gate and there is no quality threshold in front of it:
|
||||
the three signals one could read were measured on 2026-08-05 and none of them separate a
|
||||
real question from an invented one. Token overlap would cost "столица Франции" its
|
||||
answer, because the answer is Париж and that word is not in the question. See
|
||||
`docs/evals/2026-08-05-search-quality-signals.md` (V-539). **Which query source claimed
|
||||
a turn is readable on `/chat`** as a badge beside the reply, carried on
|
||||
`ipc.ChatReply.Source` and noted by `noteQuerySource` in `cmd/mavend/querysource.go`. It
|
||||
rides the context, so `handleText` keeps the one string signature the mic, telegram and
|
||||
the web share.
|
||||
- **External search is allowed and off unless configured**, like the weather and telegram
|
||||
capabilities. The code default is still off. `deploy/mavend.json` now ships a `search`
|
||||
block (owner's call, 2026-08-02), so it is on for this box and deleting the block turns
|
||||
it off again.
|
||||
- **The owner's notes and facts are never search input.** Looking up why the sky is blue and
|
||||
sending the owner's stored personal notes to an upstream engine are different acts. Only the utterance goes
|
||||
out, never the persona block, history, or matched notes.
|
||||
|
||||
## Web UI conventions
|
||||
|
||||
Server-rendered pages share `cmd/mavweb/static/ui.css` (served at `/ui.css`) and the shell
|
||||
partial in `cmd/mavweb/shell.html`: a page opens with `{{template "shellTop" "<page-key>"}}`
|
||||
and closes with `{{template "shellBottom"}}`, and the key marks the active sidebar link.
|
||||
Every page is its own embedded `.html` file next to `main.go` — no page markup lives in Go,
|
||||
and the sidebar is data (`sidebarSections`, `pageIcon`) the template renders. No
|
||||
per-page `<style>` beyond true one-offs. Wrap every table in `<div class=scroll>` so wide
|
||||
data pans on a phone. Local preview + headless screenshot recipe is in `AGENTS.md`.
|
||||
|
||||
## Vikunja
|
||||
|
||||
This repo is project **Maven** (ID 2) in Vikunja. MCP: `http://localhost:9100/mcp` (or
|
||||
`http://192.168.1.104:9100/mcp` from workpc). Feature/bug/deploy tasks go there.
|
||||
|
||||
Vikunja is the durable task store. A task holds the goal, the constraints and the
|
||||
assumption ledger. Work without a task id is work nobody can resume, so a session that
|
||||
has no id asks for one before it starts.
|
||||
|
||||
The MCP tool schemas are deferred, so load the four you actually use in ONE call at the
|
||||
start of a session rather than one lookup per first use:
|
||||
|
||||
```text
|
||||
ToolSearch("select:mcp__vikunja__list_tasks,mcp__vikunja__get_task_details,mcp__vikunja__create_task,mcp__vikunja__update_task")
|
||||
```
|
||||
|
||||
**Close a finished task with `done: true` and nothing else** (owner's call, 07-08-2026).
|
||||
Do not write a completion summary into the description on the way out. It is lost anyway,
|
||||
and the durable record is the commit messages and the merged PR. Note that `update_task`
|
||||
carrying a `description` resets `done` to false, which is why a write-up ever took two
|
||||
calls.
|
||||
- **No telemetry, no cloud model, no third-party account.** Inference stays on
|
||||
the box and nothing about Maven is reported to anyone.
|
||||
- **The owner's data first, then the world.** Every source reading his facts,
|
||||
notes, calendar, tasks or house runs first, and the personal boundary sits
|
||||
between them and anything outside.
|
||||
- **His notes and facts are never search input.** Only the utterance goes out,
|
||||
never the persona block, the history, or matched notes.
|
||||
- **External search is allowed and off unless configured.** Deleting the
|
||||
`search` block in `deploy/mavend.json` turns it off.
|
||||
- **`Response.Empty()` is the whole gate** on a world answer. There is no
|
||||
quality threshold in front of it and four candidate signals all failed.
|
||||
|
||||
## Session workflow
|
||||
|
||||
`~/.local/bin/task` owns the branch, the commit identity and the PR. One task, one
|
||||
session, one PR.
|
||||
`docs/workflow.md` carries the five stores, the doc tiers and the guards. One
|
||||
task, one session, one PR. `/pickup` opens a session and `/wrap` closes it. Wrap
|
||||
at roughly half context rather than letting the session compact.
|
||||
|
||||
```sh
|
||||
task start <vikunja-id> # branch off origin/master, write TASK.md, fetch review comments
|
||||
task pr # push, open or refresh the PR, label Vikunja, notify
|
||||
task comments # re-pull this branch's review comments into .task/
|
||||
ToolSearch("select:mcp__vikunja__list_tasks,mcp__vikunja__get_task_details,mcp__vikunja__create_task,mcp__vikunja__update_task")
|
||||
```
|
||||
|
||||
Around that, `/pickup` opens a session and `/wrap` closes it. Wrap at roughly half
|
||||
context rather than letting the session compact.
|
||||
|
||||
Five stores, and each one owns something the others must not hold:
|
||||
|
||||
| Store | Holds | Lifetime |
|
||||
|---|---|---|
|
||||
| Vikunja task | goal, constraints, assumption ledger, status | durable |
|
||||
| `CLAUDE.md`, `AGENTS.md` | what an agent must know before touching code | durable |
|
||||
| `docs/` | design, measurements, decisions | durable |
|
||||
| `TASK.md` | the brief for this branch, written by `task start`, immutable | one branch |
|
||||
| `HANDOFF.md` | only what the next agent needs to resume | one session |
|
||||
|
||||
`TASK.md` and `.task/` are excluded through `.git/info/exclude`. `HANDOFF.md` is
|
||||
gitignored and injected at session start. If a line in the handoff would still matter
|
||||
next week, it is in the wrong file.
|
||||
|
||||
Docs are tiered by path, so staleness is visible from the filename. Files directly under
|
||||
`docs/` are living and carry a `Last verified: <date> @ <sha>` line. Files under
|
||||
`docs/evals/` are dated measurements and are never edited after the day, so a newer
|
||||
number is a new file. Files under `docs/archive/` are dead and read by nobody by default.
|
||||
|
||||
## Git guards
|
||||
|
||||
Two hooks in `.githooks/`, tracked, wired with `core.hooksPath`. Fresh clone:
|
||||
|
||||
```sh
|
||||
git config core.hooksPath .githooks
|
||||
```
|
||||
|
||||
- `pre-commit` refuses master, and refuses more than 300 changed lines in non-markdown
|
||||
files. Markdown is exempt and may land as one batch.
|
||||
- `commit-msg` requires the subject to end with `(V-<id>)`. `V-` and not `#`, because
|
||||
Gitea autolinks `#123` to a Gitea issue, which is the wrong tracker.
|
||||
|
||||
Two more guards live outside the repo, in `~/.claude/hooks/`. `diff-budget.sh` blocks
|
||||
further edits past 600 changed lines on a `task/` branch. `prose_lint_hook.py` checks
|
||||
prose on every write. Both measure against `origin/master`, so a local master that is
|
||||
ahead of the remote makes the diff budget read high.
|
||||
|
||||
`--no-verify` exists. Using it means saying why in the commit body.
|
||||
- This repo is Vikunja project **Maven** (ID 2), MCP at
|
||||
`http://localhost:9100/mcp`, or `http://192.168.1.104:9100/mcp` from workpc.
|
||||
- **A session with no task id asks for one before it starts**, because work
|
||||
without one is work nobody can resume.
|
||||
- **Close a finished task with `done: true` and nothing else** (owner's call,
|
||||
2026-08-07). `update_task` carrying a `description` resets `done` to false.
|
||||
- **`pre-commit` refuses master** and more than 300 changed lines in
|
||||
non-markdown files. Markdown is exempt and may land as one batch.
|
||||
- **`commit-msg` requires the subject to end with `(V-<id>)`.** `V-` and not
|
||||
`#`, because Gitea autolinks `#123` to the wrong tracker.
|
||||
- **`diff-budget.sh` blocks edits past 600 changed lines** on a `task/` branch.
|
||||
- **`--no-verify` exists.** Using it means saying why in the commit body.
|
||||
|
||||
@@ -13,6 +13,7 @@ import (
|
||||
"github.com/kami/maven/internal/crawl"
|
||||
"github.com/kami/maven/internal/decision"
|
||||
"github.com/kami/maven/internal/ipc"
|
||||
"github.com/kami/maven/internal/kiwix"
|
||||
"github.com/kami/maven/internal/memory"
|
||||
"github.com/kami/maven/internal/morning"
|
||||
"github.com/kami/maven/internal/phraser"
|
||||
@@ -79,6 +80,15 @@ type querySource struct {
|
||||
// not named do not get to try. The lookups still run, because a named
|
||||
// destination is evidence and not a promise.
|
||||
guesses bool
|
||||
|
||||
// boundary — dropping this source widens what leaves the box, so only a
|
||||
// literal pattern may do it (V-666, owner's call of 2026-08-09).
|
||||
//
|
||||
// Every other guesser costs an answer when it is wrongly taken off a turn.
|
||||
// This one costs the rule that a question about him never reaches an
|
||||
// upstream engine. A grammar read the words to name a destination. A model
|
||||
// and a softmax both inferred one, and neither may spend that.
|
||||
boundary bool
|
||||
}
|
||||
|
||||
// querySources is the ordered chain actionQuery walks; first source to claim
|
||||
@@ -156,7 +166,7 @@ var querySources = []querySource{
|
||||
// below answers from the world's. A question about him that got this far
|
||||
// has no answer in his data, and no outside source can supply one, so this
|
||||
// stops the walk rather than let the encyclopedia and the model guess.
|
||||
{name: "personal", answer: (*reactiveHandler).queryPersonal, dest: router.SourceRecall, guesses: true},
|
||||
{name: "personal", answer: (*reactiveHandler).queryPersonal, dest: router.SourceRecall, guesses: true, boundary: true},
|
||||
// The world, read live. Owner's ruling of 2026-08-02: a metasearch hit beats
|
||||
// a frozen ZIM, so SearXNG asks before Kiwix does. Nothing of his is at
|
||||
// stake by this point — the boundary above already stopped every question
|
||||
@@ -198,12 +208,17 @@ var querySources = []querySource{
|
||||
// No destination named ⇒ the table exactly as written, which is what shipped
|
||||
// before the field existed. That is the floor. The classifier arm names
|
||||
// nothing, so a box whose model is down routes queries the way it always did.
|
||||
func queryWalk(dest router.Source) (walk, skipped []querySource) {
|
||||
// The personal boundary is the one exception, and anchored is what buys it
|
||||
// (V-666). A grammar matched a literal pattern to name the destination. The
|
||||
// routing heads and the resident model inferred one, and an inferred SourceWorld
|
||||
// takes the boundary off a question about him. That widens what is asked
|
||||
// upstream rather than costing a local answer, so those two keep it.
|
||||
func queryWalk(dest router.Source, anchored bool) (walk, skipped []querySource) {
|
||||
if dest == router.SourceUnknown {
|
||||
return querySources, nil
|
||||
}
|
||||
for _, s := range querySources {
|
||||
if s.guesses && s.dest != dest {
|
||||
if s.guesses && s.dest != dest && (anchored || !s.boundary) {
|
||||
skipped = append(skipped, s)
|
||||
continue
|
||||
}
|
||||
@@ -219,7 +234,7 @@ func (h *reactiveHandler) actionQuery(ctx context.Context, dec router.Decision)
|
||||
// (V-564). Finish names everyone below the winner.
|
||||
decision.Expect(ctx, decision.StageQuery, querySourceNames())
|
||||
rec := decision.From(ctx)
|
||||
walk, skipped := queryWalk(dec.Source)
|
||||
walk, skipped := queryWalk(dec.Source, dec.SourceAnchored)
|
||||
for _, src := range skipped {
|
||||
rec.Note(decision.Claim{
|
||||
Stage: decision.StageQuery, Claimant: src.name, Outcome: decision.NeverAsked,
|
||||
@@ -912,6 +927,28 @@ func (h *reactiveHandler) queryKiwix(ctx context.Context, t *queryTurn) (string,
|
||||
}
|
||||
}
|
||||
|
||||
// The topic, not the sentence (V-668). Kiwix ranks by keyword overlap, so
|
||||
// the question words outrank the one word that names the article: measured
|
||||
// on 2026-08-09, "что такое TCP" returns "Перехват TCP-соединения" and
|
||||
// "TCP" returns TCP. Only the verbatim path needs this. The rewriter
|
||||
// already reduces a question to English keywords, and reducing twice would
|
||||
// take the topic off the input it reads.
|
||||
if verbatim {
|
||||
if topic := kiwix.Topic(pattern); topic != "" {
|
||||
// The article named exactly, before any ranking runs. A ZIM is
|
||||
// addressable by title and a wrong title is a 404, so this either
|
||||
// answers or costs one request that says nothing.
|
||||
for _, cand := range kiwix.TitleCandidates(topic) {
|
||||
page, err := h.kiwix.client.Article(ctxK, kiwix.TitlePath(book, cand), h.kiwix.runes)
|
||||
if err == nil && page.Text != "" {
|
||||
log.Printf("voice: kiwix: %q in %q → title hit %q", topic, book, page.Title)
|
||||
return h.kiwixReply(ctx, t, page.Title, page.Text)
|
||||
}
|
||||
}
|
||||
pattern = topic
|
||||
}
|
||||
}
|
||||
|
||||
hits, err := h.kiwix.client.Search(ctxK, pattern, book, h.kiwix.max)
|
||||
if err != nil {
|
||||
log.Printf("voice: kiwix: search %q: %v", pattern, err)
|
||||
@@ -944,14 +981,18 @@ func (h *reactiveHandler) queryKiwix(ctx context.Context, t *queryTurn) (string,
|
||||
}
|
||||
page = crawl.Page{Title: top.Title, Text: top.Snippet}
|
||||
}
|
||||
// Handed over the same way a note or a page is: context for the question he
|
||||
// asked, not something to recite.
|
||||
snippet := top.Title + "\n" + crawl.TrimRunes(page.Text, h.kiwix.runes)
|
||||
return h.kiwixReply(ctx, t, top.Title, page.Text)
|
||||
}
|
||||
|
||||
// kiwixReply hands one article over the same way a note or a page is handed
|
||||
// over: context for the question he asked, not something to recite.
|
||||
func (h *reactiveHandler) kiwixReply(ctx context.Context, t *queryTurn, title, text string) (string, bool) {
|
||||
snippet := title + "\n" + crawl.TrimRunes(text, h.kiwix.runes)
|
||||
reply := h.phraseSource(ctx, "kiwix", t.dec.Utterance, []string{snippet})
|
||||
if reply == "" {
|
||||
// No phraser, or it failed. Read back the best hit rather than pretend
|
||||
// the search did not happen.
|
||||
return readBack(top.Title + " — " + page.Text), true
|
||||
return readBack(title + " — " + text), true
|
||||
}
|
||||
return reply, true
|
||||
}
|
||||
|
||||
@@ -19,7 +19,7 @@ func TestChatAnswersWithNoLlamaServer(t *testing.T) {
|
||||
dead := llm.New("http://127.0.0.1:1", 500*time.Millisecond)
|
||||
emb := router.NewHashEmbedder(1024)
|
||||
h.recall.embedder = emb
|
||||
h.router = buildRouter(emb, h.matcher, 0.55, pickLLMRouter(true, dead))
|
||||
h.router = buildRouter(emb, h.matcher, 0.55, pickLLMRouter(true, dead), nil)
|
||||
h.replier = newLLMReplier(dead, nil)
|
||||
|
||||
ctx := withDialogueID(context.Background(), dialogueIDFor(sourceText, "web"))
|
||||
|
||||
+12
-2
@@ -489,15 +489,19 @@ func (h *reactiveHandler) noteSuspended(ctx context.Context, q *dialogue.Pending
|
||||
if !q.CanResume() {
|
||||
h.clarifyStore.Delete(dialogueIDOf(ctx))
|
||||
h.noteDropped(ctx)
|
||||
log.Printf("voice: clarify — the question about %s stepped aside %d times; letting the request go", q.Missing[0], q.Suspends)
|
||||
log.Printf("voice: clarify — letting the question about %s go: %d asides in a row, %d rides in all", q.Missing[0], q.Suspends, q.Rides)
|
||||
return
|
||||
}
|
||||
q.Suspends++
|
||||
// Rides is the same event counted without the reset (V-663). Incremented
|
||||
// beside Suspends and never anywhere else, so the two cannot disagree about
|
||||
// what happened, only about how much of it they remember.
|
||||
q.Rides++
|
||||
q.Asked = h.now()
|
||||
h.clarifyStore.Put(dialogueIDOf(ctx), q)
|
||||
rt.resume = question
|
||||
rt.suspended = true
|
||||
log.Printf("voice: clarify — is its own request; suspending the question about %s and resuming it in the same reply (suspend %d of %d)", q.Missing[0], q.Suspends, dialogue.MaxSuspends)
|
||||
log.Printf("voice: clarify — is its own request; suspending the question about %s and resuming it in the same reply (suspend %d of %d, ride %d of %d)", q.Missing[0], q.Suspends, dialogue.MaxSuspends, q.Rides, dialogue.MaxRides)
|
||||
}
|
||||
|
||||
// foldAnswerIntoUtterance appends an answered subject to the original words,
|
||||
@@ -540,6 +544,11 @@ func (h *reactiveHandler) askRemainingGap(ctx context.Context, q *dialogue.Pendi
|
||||
// Suspends is not carried, and by this point it is already zero: the answer
|
||||
// path resets it (V-654). Left off the literal so the zero is stated where
|
||||
// the struct is built, rather than inherited from a field nobody names.
|
||||
//
|
||||
// Rides IS carried, and that is the whole point of it (V-663). This is the
|
||||
// same request under a second question, not a new one, so the turns it has
|
||||
// already ridden still count against it. Dropping the field here is exactly
|
||||
// the re-basing that let one question ride twenty-six replies.
|
||||
h.clarifyStore.Put(dialogueIDOf(ctx), &dialogue.PendingQuestion{
|
||||
Intent: q.Intent,
|
||||
Slots: merged,
|
||||
@@ -550,6 +559,7 @@ func (h *reactiveHandler) askRemainingGap(ctx context.Context, q *dialogue.Pendi
|
||||
TTL: clarifyTTL,
|
||||
Attempts: q.Attempts + 1,
|
||||
MaxAttempts: q.MaxAttempts,
|
||||
Rides: q.Rides,
|
||||
})
|
||||
log.Printf("voice: clarify — one gap filled, still missing %s for intent=%s, asking again (attempt %d)", remaining[0], intent, q.Attempts+1)
|
||||
return question, true
|
||||
|
||||
@@ -317,7 +317,7 @@ func TestClarifyExpiryIsAnnouncedAndWordsStillRoute(t *testing.T) {
|
||||
h, _, now := newClarifyHandler(t)
|
||||
emb := router.NewHashEmbedder(1024)
|
||||
h.recall.embedder = emb
|
||||
h.router = buildRouter(emb, h.matcher, 0.55, nil)
|
||||
h.router = buildRouter(emb, h.matcher, 0.55, nil, nil)
|
||||
|
||||
if _, asked := h.askClarify(ctx, clarifyDec(router.IntentReminder, router.Slots{Text: "напомни"}, "напомни")); !asked {
|
||||
t.Fatal("expected a question")
|
||||
@@ -671,7 +671,7 @@ func TestUnresolvedActSaysItDoesNotKnowTheCommand(t *testing.T) {
|
||||
func newRoutingClarifyHandler(t *testing.T) (*reactiveHandler, *store.Store) {
|
||||
t.Helper()
|
||||
h, st, _ := newClarifyHandler(t)
|
||||
h.router = buildRouter(router.NewHashEmbedder(1024), h.matcher, 0.55, nil)
|
||||
h.router = buildRouter(router.NewHashEmbedder(1024), h.matcher, 0.55, nil, nil)
|
||||
h.recall = recallWiring{embedder: router.NewHashEmbedder(1024), memStore: memory.NewInMemoryStore()}
|
||||
return h, st
|
||||
}
|
||||
|
||||
@@ -25,7 +25,7 @@ func traceHandler(t *testing.T, ring *decision.Ring) *reactiveHandler {
|
||||
return &reactiveHandler{
|
||||
api: api,
|
||||
recall: recallWiring{embedder: emb, memStore: memory.NewInMemoryStore()},
|
||||
router: buildRouter(emb, tool.NewMatcher(api), 0.55, nil),
|
||||
router: buildRouter(emb, tool.NewMatcher(api), 0.55, nil, nil),
|
||||
replier: voice.NewStubReplier(),
|
||||
now: func() time.Time { return now },
|
||||
dataStore: st,
|
||||
|
||||
@@ -171,7 +171,7 @@ func newDialogueHandler(t *testing.T) (*reactiveHandler, *store.Store, *time.Tim
|
||||
// and never a coincidence (V-577, V-579). checkEnd refuses any reminder
|
||||
// landing on it, and at 09:00 the row that answers "на 9" would trip that.
|
||||
*now = time.Date(2026, 7, 31, 9, 17, 0, 0, time.UTC)
|
||||
h.router = buildRouter(router.NewHashEmbedder(1024), h.matcher, 0.55, nil)
|
||||
h.router = buildRouter(router.NewHashEmbedder(1024), h.matcher, 0.55, nil, nil)
|
||||
h.recall = recallWiring{embedder: router.NewHashEmbedder(1024), memStore: memory.NewInMemoryStore()}
|
||||
return h, st, now
|
||||
}
|
||||
|
||||
@@ -24,7 +24,7 @@ func TestApplyAction_FactCapture_QueuesEntityResolution(t *testing.T) {
|
||||
|
||||
emb := router.NewHashEmbedder(1024)
|
||||
matcher := tool.NewMatcher(api)
|
||||
rtr := buildRouter(emb, matcher, 0.55, nil)
|
||||
rtr := buildRouter(emb, matcher, 0.55, nil, nil)
|
||||
|
||||
h := &reactiveHandler{
|
||||
api: api,
|
||||
|
||||
@@ -20,7 +20,7 @@ func newFactGateHandler(t *testing.T, now time.Time) (*reactiveHandler, ipc.Core
|
||||
h := &reactiveHandler{
|
||||
api: api,
|
||||
recall: recallWiring{embedder: emb, memStore: memory.NewInMemoryStore()},
|
||||
router: buildRouter(emb, tool.NewMatcher(api), 0.55, nil),
|
||||
router: buildRouter(emb, tool.NewMatcher(api), 0.55, nil, nil),
|
||||
replier: voice.NewStubReplier(),
|
||||
now: func() time.Time { return now },
|
||||
dataStore: st,
|
||||
|
||||
@@ -45,7 +45,7 @@ func newNoteHandler(t *testing.T) (*reactiveHandler, *store.Store) {
|
||||
h := &reactiveHandler{
|
||||
api: api,
|
||||
recall: recallWiring{embedder: emb, memStore: memory.NewInMemoryStore()},
|
||||
router: buildRouter(emb, tool.NewMatcher(api), 0.55, nil),
|
||||
router: buildRouter(emb, tool.NewMatcher(api), 0.55, nil, nil),
|
||||
replier: voice.NewStubReplier(),
|
||||
now: func() time.Time { return now },
|
||||
dataStore: st,
|
||||
|
||||
@@ -10,7 +10,7 @@ import (
|
||||
// whose model is down names nothing, and naming nothing has to walk the chain
|
||||
// the way it walked before the field existed.
|
||||
func TestNoDestinationWalksTheWholeChain(t *testing.T) {
|
||||
walk, skipped := queryWalk(router.SourceUnknown)
|
||||
walk, skipped := queryWalk(router.SourceUnknown, false)
|
||||
if len(skipped) != 0 {
|
||||
t.Errorf("skipped %d sources with no destination named, want none", len(skipped))
|
||||
}
|
||||
@@ -32,15 +32,16 @@ func TestANamedDestinationSilencesTheOtherGuessers(t *testing.T) {
|
||||
dest router.Source
|
||||
utterance string
|
||||
silenced string
|
||||
anchored bool // a stage 0 grammar named the destination
|
||||
}{
|
||||
{router.SourceWorld, "что такое TCP?", "weather"},
|
||||
{router.SourceWorld, "сколько будет 17 на 23?", "weather"},
|
||||
{router.SourceWorld, "кто такой Линус Торвальдс?", "personal"},
|
||||
{router.SourceRecall, "какой у меня любимый язык?", "feeds"},
|
||||
{router.SourceCalendar, "что в календаре на завтра?", "weather"},
|
||||
{router.SourceWorld, "что такое TCP?", "weather", true},
|
||||
{router.SourceWorld, "сколько будет 17 на 23?", "weather", true},
|
||||
{router.SourceWorld, "кто такой Линус Торвальдс?", "personal", true},
|
||||
{router.SourceRecall, "какой у меня любимый язык?", "feeds", false},
|
||||
{router.SourceCalendar, "что в календаре на завтра?", "weather", true},
|
||||
}
|
||||
for _, c := range cases {
|
||||
walk, skipped := queryWalk(c.dest)
|
||||
walk, skipped := queryWalk(c.dest, c.anchored)
|
||||
if inWalk(walk, c.silenced) {
|
||||
t.Errorf("%q named %q: %q is still asked", c.utterance, c.dest, c.silenced)
|
||||
}
|
||||
@@ -56,7 +57,7 @@ func TestANamedDestinationSilencesTheOtherGuessers(t *testing.T) {
|
||||
// data first, then the world", and a destination a model wrote must not be able
|
||||
// to reverse it.
|
||||
func TestNamingTheWorldStillReadsHisDataFirst(t *testing.T) {
|
||||
walk, _ := queryWalk(router.SourceWorld)
|
||||
walk, _ := queryWalk(router.SourceWorld, true)
|
||||
for _, look := range []string{"fact-by-key", "embed", "memory", "notes"} {
|
||||
if !inWalk(walk, look) {
|
||||
t.Errorf("%q was dropped; only the sources that guess may be dropped", look)
|
||||
@@ -74,7 +75,7 @@ func TestNamingTheWorldStillReadsHisDataFirst(t *testing.T) {
|
||||
// makes "какой у меня любимый язык?" answer "не нашла у тебя такой записи"
|
||||
// rather than reaching SearXNG once nothing local had it.
|
||||
func TestNamingRecallKeepsTheBoundary(t *testing.T) {
|
||||
walk, _ := queryWalk(router.SourceRecall)
|
||||
walk, _ := queryWalk(router.SourceRecall, true)
|
||||
if !inWalk(walk, "personal") {
|
||||
t.Fatal("the personal boundary was skipped on a turn named for his own data")
|
||||
}
|
||||
@@ -83,12 +84,33 @@ func TestNamingRecallKeepsTheBoundary(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// The owner's call of 2026-08-09 (V-666): only a stage 0 grammar may take the
|
||||
// personal boundary off a turn. The routing heads and the resident model both
|
||||
// name a destination by inference, and an inferred SourceWorld would send a
|
||||
// question about him upstream. Every other guesser still goes.
|
||||
func TestOnlyAGrammarMayDropTheBoundary(t *testing.T) {
|
||||
walk, skipped := queryWalk(router.SourceWorld, false)
|
||||
if !inWalk(walk, "personal") {
|
||||
t.Error("an inferred destination took the boundary off the turn")
|
||||
}
|
||||
if !inWalk(skipped, "weather") {
|
||||
t.Error("weather is still asked; the rule covers the boundary alone")
|
||||
}
|
||||
if posOf(walk, "personal") > posOf(walk, "search") {
|
||||
t.Error("the boundary no longer sits in front of the world")
|
||||
}
|
||||
if anchored, _ := queryWalk(router.SourceWorld, true); inWalk(anchored, "personal") {
|
||||
t.Error(`a grammar named the world and the boundary stayed: ` +
|
||||
`"кто такой Линус Торвальдс?" is answered "не нашла у тебя такой записи" again`)
|
||||
}
|
||||
}
|
||||
|
||||
// Whatever the destination, the walk is a subsequence of the table. Every
|
||||
// comment on that table argues an order between two sources, and none of those
|
||||
// reasons is about this field.
|
||||
func TestTheWalkNeverReordersTheTable(t *testing.T) {
|
||||
for _, dest := range append([]router.Source{router.SourceUnknown}, router.Sources...) {
|
||||
walk, skipped := queryWalk(dest)
|
||||
walk, skipped := queryWalk(dest, true)
|
||||
if len(walk)+len(skipped) != len(querySources) {
|
||||
t.Errorf("%q: %d walked + %d skipped, want %d", dest, len(walk), len(skipped), len(querySources))
|
||||
}
|
||||
|
||||
@@ -22,7 +22,7 @@ func TestReactiveNotesReminders(t *testing.T) {
|
||||
|
||||
emb := router.NewHashEmbedder(1024)
|
||||
matcher := tool.NewMatcher(api)
|
||||
rtr := buildRouter(emb, matcher, 0.55, nil)
|
||||
rtr := buildRouter(emb, matcher, 0.55, nil, nil)
|
||||
|
||||
h := &reactiveHandler{
|
||||
api: api,
|
||||
@@ -104,7 +104,7 @@ func TestSpokenTaskCaptureFilesATask(t *testing.T) {
|
||||
h := &reactiveHandler{
|
||||
api: api,
|
||||
recall: recallWiring{embedder: emb, memStore: memory.NewInMemoryStore()},
|
||||
router: buildRouter(emb, matcher, 0.55, nil),
|
||||
router: buildRouter(emb, matcher, 0.55, nil, nil),
|
||||
replier: voice.NewStubReplier(),
|
||||
now: func() time.Time { return now },
|
||||
dataStore: st,
|
||||
|
||||
@@ -474,7 +474,7 @@ func newSimWorld(t *testing.T, sc scenario) *simWorld {
|
||||
// used to be built on a nil API, which meant any scenario that produced an
|
||||
// act panicked the moment the matcher was consulted.
|
||||
matcher := tool.NewMatcher(api)
|
||||
rtr := buildRouter(emb, matcher, config.DefaultRouterThreshold, router.NewLLMRouter(scripted))
|
||||
rtr := buildRouter(emb, matcher, config.DefaultRouterThreshold, router.NewLLMRouter(scripted), nil)
|
||||
|
||||
w.handler = &reactiveHandler{
|
||||
stt: simTranscriber{},
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/kami/maven/internal/config"
|
||||
"github.com/kami/maven/internal/stt"
|
||||
)
|
||||
|
||||
// A box with no workstation.stt block transcribes exactly as it did before the
|
||||
// seam existed: the floor is handed back untouched, and nothing probes.
|
||||
func TestSttSeamWithNoBlockIsTheFloor(t *testing.T) {
|
||||
floor := stt.NewStub()
|
||||
got, pair := sttSeam(&config.Config{}, floor)
|
||||
if pair != nil {
|
||||
t.Fatal("no block must build no pair")
|
||||
}
|
||||
if got != stt.Transcriber(floor) {
|
||||
t.Fatal("no block must hand back the floor itself")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSttSeamPrefersTheWorkstation(t *testing.T) {
|
||||
cfg := &config.Config{Workstation: &config.WorkstationConfig{
|
||||
URL: "http://127.0.0.1:1",
|
||||
Stt: &config.WorkstationSttConfig{
|
||||
URL: "http://127.0.0.1:2/transcribe",
|
||||
Health: "http://127.0.0.1:2/health",
|
||||
},
|
||||
}}
|
||||
got, pair := sttSeam(cfg, stt.NewStub())
|
||||
if pair == nil {
|
||||
t.Fatal("a configured block must build a pair")
|
||||
}
|
||||
defer pair.Stop()
|
||||
if got != stt.Transcriber(pair) {
|
||||
t.Fatal("the pair is what callers must transcribe through")
|
||||
}
|
||||
// Nothing answers on port 2, so the seam is the floor until it does.
|
||||
if pair.Available() {
|
||||
t.Fatal("an unreachable workstation must not be available")
|
||||
}
|
||||
}
|
||||
@@ -186,6 +186,28 @@ func carriesReminderVerb(text string) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
// isPleasantry matches the WHOLE utterance against lexicon.Pleasantries, after
|
||||
// lowercasing and dropping the punctuation a greeting carries.
|
||||
//
|
||||
// Whole utterance and not tokens. Every token rule tried here was wrong on
|
||||
// something: "вечер" answers "это утра или вечера?", "нет" answers a confirm,
|
||||
// and "спокойной" alone is not an utterance at all. A greeting is a fixed
|
||||
// phrase, so matching it as one costs nothing and claims nothing else.
|
||||
func isPleasantry(text string) bool {
|
||||
t := strings.ToLower(strings.TrimSpace(text))
|
||||
t = strings.Trim(t, " .,!?…")
|
||||
t = strings.Join(strings.Fields(t), " ")
|
||||
if t == "" {
|
||||
return false
|
||||
}
|
||||
for _, p := range lexicon.Pleasantries() {
|
||||
if t == p {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// offlineOwnRequest is the shape half of the evidence: the offline token tests,
|
||||
// which cost nothing and never depend on the model that produced the routing.
|
||||
// It is also the whole answer when there is no route to read — the classifier
|
||||
@@ -225,6 +247,16 @@ func classifyTurnRole(q *dialogue.PendingQuestion, text string, answer dialogue.
|
||||
// hour, and no route saying "question" changes that. It works because the
|
||||
// extractor no longer reads a day word as the current clock, so a sentence
|
||||
// that names no hour now fills nothing to weigh.
|
||||
// A pleasantry is neither (V-663). "спасибо" and "привет" fell through to
|
||||
// roleAnswer, so a question about a reminder's DAY was re-asked at a man
|
||||
// saying thank you, and the retry it spent was one of the three bounds
|
||||
// meant to end the ride. It is an aside: answered as itself, the question
|
||||
// resumed on the tail, no attempt spent, one ride counted. Placed above the
|
||||
// content gate because "доброе утро" has content and states nothing, so
|
||||
// neither half of the evidence below can reach it.
|
||||
if q != nil && isPleasantry(text) {
|
||||
return roleAside
|
||||
}
|
||||
own := false
|
||||
if len(ownContent(text)) > 0 {
|
||||
own = offlineOwnRequest(text) || (ok && carriesOwnRequest(routed, text))
|
||||
|
||||
@@ -372,4 +372,69 @@ func TestAnAnsweredGapResetsTheSuspendBudget(t *testing.T) {
|
||||
if q.Suspends != 0 {
|
||||
t.Fatalf("answering a gap must reset the suspend budget: suspends = %d", q.Suspends)
|
||||
}
|
||||
// The ride it already took is carried across the re-park (V-663). Resetting
|
||||
// both counters here is what let one question ride twenty-six replies.
|
||||
if q.Rides != 1 {
|
||||
t.Fatalf("the aside it already took was forgotten: rides = %d", q.Rides)
|
||||
}
|
||||
}
|
||||
|
||||
// TestTwoBoundsCannotRearmEachOther — V-663.
|
||||
//
|
||||
// MaxSuspends landed and the measurement did not move: twenty-six of 140 turns
|
||||
// carried a tail before it and twenty-six after. This is the shape it misses,
|
||||
// taken from the 2026-08-08 run, where one question rode turns 7 to 13.
|
||||
//
|
||||
// An aside spends no attempt, so MaxAttempts never reaches it. A turn that
|
||||
// reads as a failed answer zeroes Suspends, so MaxSuspends never reaches the
|
||||
// asides either. Alternating the two rearms each bound with the other's
|
||||
// traffic. Rides counts both kinds and is never reset, so it is what ends this.
|
||||
func TestTwoBoundsCannotRearmEachOther(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
h, _ := newRoutingClarifyHandler(t)
|
||||
id := dialogueIDFor(sourceText, "web")
|
||||
resumed, _ := clarifyResumedFor(dialogue.SlotTime)
|
||||
|
||||
if reply := h.handleText(ctx, "web", "напомни позвонить маме"); !strings.Contains(reply, "?") {
|
||||
t.Fatalf("expected the time question, got %q", reply)
|
||||
}
|
||||
|
||||
// Two asides. Each one rides and neither spends an attempt.
|
||||
for i := 0; i < 2; i++ {
|
||||
reply := h.handleText(ctx, "web", "какие у меня напоминания?")
|
||||
if !strings.HasSuffix(reply, resumed) {
|
||||
t.Fatalf("aside %d: the question must come back, got %q", i+1, reply)
|
||||
}
|
||||
}
|
||||
q := h.clarifyStore.Get(id, h.now())
|
||||
if q == nil || q.Rides != 2 || q.Suspends != 2 {
|
||||
t.Fatalf("after two asides: %+v", q)
|
||||
}
|
||||
|
||||
// A pleasantry. It used to read as a failed answer, so she re-asked the
|
||||
// question at a man saying thank you and spent an attempt doing it. Now it
|
||||
// is an aside: answered as itself, question on the tail, one more ride.
|
||||
reply := h.handleText(ctx, "web", "спасибо")
|
||||
if !strings.HasSuffix(reply, resumed) {
|
||||
t.Fatalf("a pleasantry lost the parked question: %q", reply)
|
||||
}
|
||||
q = h.clarifyStore.Get(id, h.now())
|
||||
if q == nil || q.Attempts != 1 {
|
||||
t.Fatalf("a pleasantry spent an attempt: %+v", q)
|
||||
}
|
||||
if q.Rides != 3 {
|
||||
t.Fatalf("a pleasantry rode free: %+v", q)
|
||||
}
|
||||
|
||||
// One more ride of any kind and the request goes, out loud.
|
||||
reply = h.handleText(ctx, "web", "какие у меня напоминания?")
|
||||
if !strings.Contains(reply, clarifyDropped) {
|
||||
t.Fatalf("the question rode four asides and was let go in silence: %q", reply)
|
||||
}
|
||||
if strings.HasSuffix(reply, resumed) {
|
||||
t.Fatalf("a question she has let go must not be asked again: %q", reply)
|
||||
}
|
||||
if h.clarifyStore.Get(id, h.now()) != nil {
|
||||
t.Fatal("the question must be gone once she has said she let it go")
|
||||
}
|
||||
}
|
||||
|
||||
+76
-4
@@ -36,7 +36,9 @@ type voiceWiring struct {
|
||||
sessions *voice.Sessions
|
||||
voiceSink delivery.Sink
|
||||
embedder router.Embedder
|
||||
handler *reactiveHandler // the reactive handler for IPC Chat
|
||||
// heads — the routing heads, nil unless embedder.heads_path is set.
|
||||
heads *router.RouterHeads
|
||||
handler *reactiveHandler // the reactive handler for IPC Chat
|
||||
// worker clients (set when configured as Remote): closed on shutdown so
|
||||
// mavsttd / mavttsd don't keep a stale conn into a restarting daemon.
|
||||
sttClient *worker.Client
|
||||
@@ -53,7 +55,11 @@ type voiceWiring struct {
|
||||
// unless a `workstation` block names an address. Held here only so the
|
||||
// prober is stopped on shutdown; callers were handed it at build time.
|
||||
pair *llm.Pair
|
||||
mcp *mcpWiring
|
||||
// sttPair — CrisperWhisper 2.0 on the workstation with mavsttd as the
|
||||
// floor, nil unless the `workstation.stt` block names an address. Held for
|
||||
// the same reason as pair: to stop its prober on shutdown.
|
||||
sttPair *stt.Pair
|
||||
mcp *mcpWiring
|
||||
// home — the Home Assistant client, nil unless the `smarthome` block is
|
||||
// enabled (Vikunja #256). Its devices land in the same allowlist as every
|
||||
// other act, so nothing else here has to know about it.
|
||||
@@ -72,6 +78,9 @@ func (w *voiceWiring) close() {
|
||||
if w.embedder != nil {
|
||||
_ = w.embedder.Close()
|
||||
}
|
||||
if w.heads != nil {
|
||||
_ = w.heads.Close()
|
||||
}
|
||||
if w.server != nil {
|
||||
_ = w.server.Close()
|
||||
}
|
||||
@@ -84,6 +93,9 @@ func (w *voiceWiring) close() {
|
||||
if w.pair != nil {
|
||||
w.pair.Stop()
|
||||
}
|
||||
if w.sttPair != nil {
|
||||
w.sttPair.Stop()
|
||||
}
|
||||
w.mcp.close()
|
||||
}
|
||||
|
||||
@@ -112,6 +124,7 @@ func wireVoice(cfg *config.Config, coreAPI ipc.CoreAPI, phr phraser.Phraser, mem
|
||||
} else {
|
||||
transcriber = stt.NewStub()
|
||||
}
|
||||
transcriber, w.sttPair = sttSeam(cfg, transcriber)
|
||||
w.transcriber = transcriber
|
||||
|
||||
// ----- tts (Stub in-process OR Remote) -----
|
||||
@@ -147,6 +160,24 @@ func wireVoice(cfg *config.Config, coreAPI ipc.CoreAPI, phr phraser.Phraser, mem
|
||||
emb = router.NewHashEmbedder(1024)
|
||||
}
|
||||
w.embedder = emb
|
||||
|
||||
// ----- router: routing heads (only when configured, and never fatal) -----
|
||||
// A missing or broken weights file logs and leaves w.heads nil, which is
|
||||
// byte-for-byte the cascade that shipped before V-664. Refusing to start
|
||||
// over a routing accelerator would trade a working box for a better one.
|
||||
if cfg.Voice.Embedder != nil && cfg.Voice.Embedder.HeadsPath != "" {
|
||||
h, err := router.NewRouterHeads(
|
||||
cfg.Voice.Embedder.HeadsPath,
|
||||
cfg.Voice.Embedder.TokenizerPath,
|
||||
)
|
||||
if err != nil {
|
||||
log.Printf("voice: routing heads unavailable, cascade unchanged: %v", err)
|
||||
} else {
|
||||
log.Printf("voice: routing heads loaded from %s", cfg.Voice.Embedder.HeadsPath)
|
||||
w.heads = h
|
||||
}
|
||||
}
|
||||
|
||||
repairFactVectors(dataStore, emb)
|
||||
checkStoredEmbedder(dataStore, emb)
|
||||
// Retention is enforced on write, which is not enough on its own: a box that
|
||||
@@ -223,7 +254,8 @@ func wireVoice(cfg *config.Config, coreAPI ipc.CoreAPI, phr phraser.Phraser, mem
|
||||
// against the classifier's 50.0%, at about 1s a turn instead of 30ms (see
|
||||
// config.VoiceConfig.LLMRouter). The classifier always stays wired as the
|
||||
// fallback, so a model error never breaks a turn.
|
||||
rtr := buildRouter(emb, matcher, threshold, pickLLMRouter(cfg.Voice.UseLLMRouter(), hot))
|
||||
rtr := buildRouter(emb, matcher, threshold,
|
||||
pickLLMRouter(cfg.Voice.UseLLMRouter(), hot), w.heads)
|
||||
|
||||
// ----- sessions registry (shared with voicesink) -----
|
||||
sessions := voice.NewSessions()
|
||||
@@ -366,6 +398,44 @@ func modelSeam(cfg *config.Config, resident *llm.Client) (router.Completer, *llm
|
||||
return pair, pair
|
||||
}
|
||||
|
||||
// sttSeam builds the transcription seam the voice path and the meeting
|
||||
// recorder share. It is modelSeam for audio and follows the same rule.
|
||||
//
|
||||
// With no `workstation.stt` block it hands back the floor untouched, which is
|
||||
// today's deploy exactly. With one, it is an stt.Pair preferring CrisperWhisper
|
||||
// 2.0 on workpc, which scores 10.4% WER in Russian against the floor's 27.5%
|
||||
// (docs/evals/2026-08-09-crisperwhisper2-russian-wer.md).
|
||||
//
|
||||
// Only the silent half of the degradation rule applies here. A worse transcript
|
||||
// is still a turn, so there is nothing to name a gap about and the fallback is
|
||||
// never spoken. That is why stt.Pair has no TranscribeRemote.
|
||||
func sttSeam(cfg *config.Config, floor stt.Transcriber) (stt.Transcriber, *stt.Pair) {
|
||||
if cfg.Workstation == nil || cfg.Workstation.Stt == nil {
|
||||
return floor, nil
|
||||
}
|
||||
s := cfg.Workstation.Stt
|
||||
lang := ""
|
||||
if cfg.Voice != nil {
|
||||
lang = cfg.Voice.Lang
|
||||
if cfg.Voice.Stt != nil && cfg.Voice.Stt.Lang != "" {
|
||||
lang = cfg.Voice.Stt.Lang
|
||||
}
|
||||
}
|
||||
pair := stt.NewPair(
|
||||
stt.NewHTTPTranscriber(s.URL, s.Token, lang, time.Duration(s.Timeout)),
|
||||
floor,
|
||||
s.Health,
|
||||
time.Duration(s.Probe),
|
||||
)
|
||||
pair.Start(context.Background())
|
||||
if s.Token == "" {
|
||||
log.Print("voice: the workstation transcriber has no token, so anything on the LAN can post audio to it")
|
||||
}
|
||||
log.Printf("voice: workstation transcriber at %s, probed every %s, mavsttd as the floor",
|
||||
s.URL, time.Duration(s.Probe))
|
||||
return pair, pair
|
||||
}
|
||||
|
||||
func pickLLMRouter(enabled bool, c router.Completer) *router.LLMRouter {
|
||||
if !enabled {
|
||||
return nil
|
||||
@@ -390,7 +460,8 @@ func pickLLMRouter(enabled bool, c router.Completer) *router.LLMRouter {
|
||||
// intent from seedDir (models/seeds/<intent>.txt) — see seedClassifier
|
||||
// below for the current intent list and file names.
|
||||
// - Threshold is from voice.router_threshold config (default 0.55).
|
||||
func buildRouter(emb router.Embedder, acts router.ActMatcher, threshold float64, llmR *router.LLMRouter) *router.Router {
|
||||
func buildRouter(emb router.Embedder, acts router.ActMatcher, threshold float64,
|
||||
llmR *router.LLMRouter, heads *router.RouterHeads) *router.Router {
|
||||
cls := router.NewClassifier(emb)
|
||||
seedClassifier(cls)
|
||||
grammars := router.DefaultGrammars(acts)
|
||||
@@ -442,6 +513,7 @@ func buildRouter(emb router.Embedder, acts router.ActMatcher, threshold float64,
|
||||
},
|
||||
Threshold: threshold,
|
||||
LLM: llmR,
|
||||
Heads: heads,
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
+13
-4
@@ -34,22 +34,31 @@ type probe struct {
|
||||
drmDev string
|
||||
}
|
||||
|
||||
// foreign lists every ROCm process that is not ours. selfPID is the supervisor's
|
||||
// llama-server child, or 0 when it is not running.
|
||||
// foreign lists every ROCm process that is not ours. self holds the pids of the
|
||||
// supervisor's own children, and a child that is not running contributes 0.
|
||||
//
|
||||
// There is more than one child since 09-08-2026. CW2 registers on the KFD like
|
||||
// any ROCm job, so a supervisor that excluded only llama-server would read its
|
||||
// own transcriber as a contender, yield the card to it, and never keep a model
|
||||
// loaded again.
|
||||
//
|
||||
// An unreadable kfd tree returns no processes and no error. That is deliberate
|
||||
// and it is the safe direction only because startVRAM also has to agree before
|
||||
// anything launches: a supervisor that cannot see the KFD never sees free VRAM
|
||||
// either, because the CPT run holding the card shows up in the drm totals.
|
||||
func (p probe) foreign(selfPID int) []gpuProc {
|
||||
func (p probe) foreign(self ...int) []gpuProc {
|
||||
entries, err := os.ReadDir(p.kfdRoot)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
mine := make(map[int]bool, len(self))
|
||||
for _, pid := range self {
|
||||
mine[pid] = true
|
||||
}
|
||||
var out []gpuProc
|
||||
for _, e := range entries {
|
||||
pid, err := strconv.Atoi(e.Name())
|
||||
if err != nil || pid == selfPID {
|
||||
if err != nil || mine[pid] {
|
||||
continue
|
||||
}
|
||||
out = append(out, gpuProc{
|
||||
|
||||
+46
-1
@@ -1,6 +1,7 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
@@ -8,6 +9,7 @@ import (
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// fakeKFD builds the sysfs shape the workstation actually has: one directory
|
||||
@@ -47,6 +49,24 @@ func TestForeignExcludesOurChild(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// The transcriber is a ROCm process on the same card, so it registers on the
|
||||
// KFD exactly like a contender does. Reading it as one is what happened on
|
||||
// 2026-08-09 while CW2 ran under its own systemd unit: mavgpud yielded, waited
|
||||
// five polls, loaded the model, yielded again, and never held it for a whole
|
||||
// minute. Excluding every child is the fix and this is the test of it.
|
||||
func TestForeignExcludesEveryChild(t *testing.T) {
|
||||
p := probe{kfdRoot: fakeKFD(t, map[int]int64{478104: 12791693312, 999: 4096, 1001: 1717986918})}
|
||||
|
||||
ours := p.foreign(999, 1001)
|
||||
if len(ours) != 1 || ours[0].PID != 478104 {
|
||||
t.Fatalf("only the CPT run is a contender, got %+v", ours)
|
||||
}
|
||||
// A child that is not running reports pid 0, which must exclude nothing.
|
||||
if got := p.foreign(999, 0); len(got) != 2 {
|
||||
t.Errorf("a stopped child excludes nobody: got %d contenders, want 2", len(got))
|
||||
}
|
||||
}
|
||||
|
||||
// An empty KFD tree is the state that permits a start, so it must read as empty
|
||||
// rather than as an error the caller has to interpret.
|
||||
func TestForeignEmptyAndMissing(t *testing.T) {
|
||||
@@ -81,7 +101,7 @@ func TestFreeVRAM(t *testing.T) {
|
||||
// rather than hanging or proxying into a closed port. Maven reads this endpoint
|
||||
// on a timer forever, including while the workstation is busy.
|
||||
func TestHealthAndProxyRefuseWhenNotReady(t *testing.T) {
|
||||
s := &supervisor{run: newRunner("/bin/true", nil, "")}
|
||||
s := &supervisor{run: newRunner("fake", "/bin/true", nil, "")}
|
||||
h := s.handler(mustURL(t, "http://127.0.0.1:1"))
|
||||
|
||||
for _, path := range []string{"/health", "/v1/chat/completions"} {
|
||||
@@ -101,3 +121,28 @@ func mustURL(t *testing.T, s string) *url.URL {
|
||||
}
|
||||
return u
|
||||
}
|
||||
|
||||
// Yielding is all or nothing. A CPT run wants the whole card, so handing back
|
||||
// the language model while the transcriber keeps 1.6GB mapped would leave the
|
||||
// other job failing its allocation, which is the outcome yielding exists to
|
||||
// prevent.
|
||||
func TestYieldStopsEveryChild(t *testing.T) {
|
||||
idle := "while : ; do sleep 1 ; done"
|
||||
s := &supervisor{
|
||||
cfg: config{EvictAfter: 1, StopGrace: duration(2 * time.Second)},
|
||||
probe: probe{kfdRoot: fakeKFD(t, map[int]int64{478104: 12791693312})},
|
||||
run: newRunner("llama-server", fakeServer(t, idle), nil, ""),
|
||||
stt: newRunner("cw2", fakeServer(t, idle), nil, ""),
|
||||
}
|
||||
for _, r := range s.children() {
|
||||
if err := r.start(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
s.tick(context.Background())
|
||||
for _, r := range s.children() {
|
||||
if r.running() {
|
||||
t.Errorf("%s outlived the yield", r.name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+90
-17
@@ -10,6 +10,11 @@
|
||||
// the card. Not on demand, because a 7-14B takes tens of seconds to load and a
|
||||
// world question would be answered by a gap every time the card had been quiet.
|
||||
// Not always on, because that holds 16GB against the owner's own jobs.
|
||||
//
|
||||
// It supervises a second child since 09-08-2026, the CW2 transcriber, and for
|
||||
// one reason only: it is a ROCm process on the same card. Any GPU service the
|
||||
// owner leaves running beside this daemon reads as a contender and evicts the
|
||||
// model, so the card needs one owner rather than two neighbours.
|
||||
package main
|
||||
|
||||
import (
|
||||
@@ -36,6 +41,10 @@ type config struct {
|
||||
// owner's business and not this daemon's schema.
|
||||
LlamaArgs []string `json:"llama_args"`
|
||||
|
||||
// Stt is optional. Without it mavgpud supervises llama-server alone, which
|
||||
// is everything it did before 09-08-2026.
|
||||
Stt *sttConfig `json:"stt,omitempty"`
|
||||
|
||||
KFDRoot string `json:"kfd_root"`
|
||||
DRMDevice string `json:"drm_device"`
|
||||
|
||||
@@ -51,6 +60,22 @@ type config struct {
|
||||
StartAfter int `json:"start_after_polls"`
|
||||
}
|
||||
|
||||
// sttConfig is the CW2 transcriber, which mavgpud runs for one reason: it is a
|
||||
// ROCm process on this card. Left to its own systemd unit it registers on the
|
||||
// KFD, the supervisor reads it as a contender, and llama-server is evicted
|
||||
// within two polls and restarted five polls later, forever. That thrash was
|
||||
// observed on 2026-08-09 and it is what folded the service in here.
|
||||
//
|
||||
// Maven talks to it directly, not through this daemon. There is no proxy and no
|
||||
// idle timer: at 1.6GB it denies the card to nobody, and unloading it would only
|
||||
// send the next voice turn to the homesrv floor for no gain.
|
||||
type sttConfig struct {
|
||||
// Addr is where the service binds, and it is read only to probe /health.
|
||||
Addr string `json:"addr"`
|
||||
Bin string `json:"bin"`
|
||||
Args []string `json:"args"`
|
||||
}
|
||||
|
||||
func defaults() config {
|
||||
return config{
|
||||
Listen: ":8080",
|
||||
@@ -99,12 +124,18 @@ func main() {
|
||||
}
|
||||
|
||||
base := "http://" + cfg.LlamaAddr
|
||||
run := newRunner(cfg.LlamaBin, cfg.LlamaArgs, base+"/health")
|
||||
run := newRunner("llama-server", cfg.LlamaBin, cfg.LlamaArgs, base+"/health")
|
||||
sup := &supervisor{
|
||||
cfg: cfg,
|
||||
probe: probe{kfdRoot: cfg.KFDRoot, drmDev: cfg.DRMDevice},
|
||||
run: run,
|
||||
}
|
||||
if s := cfg.Stt; s != nil {
|
||||
if s.Bin == "" || s.Addr == "" {
|
||||
log.Fatal("mavgpud: stt needs both bin and addr")
|
||||
}
|
||||
sup.stt = newRunner("cw2", s.Bin, s.Args, "http://"+s.Addr+"/health")
|
||||
}
|
||||
sup.touch()
|
||||
|
||||
ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
|
||||
@@ -129,13 +160,17 @@ func main() {
|
||||
shut, done := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer done()
|
||||
_ = srv.Shutdown(shut)
|
||||
run.stop(time.Duration(cfg.StopGrace))
|
||||
for _, r := range sup.children() {
|
||||
r.stop(time.Duration(cfg.StopGrace))
|
||||
}
|
||||
}
|
||||
|
||||
type supervisor struct {
|
||||
cfg config
|
||||
probe probe
|
||||
run *runner
|
||||
// stt is the CW2 transcriber, or nil when the config names none.
|
||||
stt *runner
|
||||
|
||||
lastReq atomic.Int64 // unix nanos of the last request Maven sent
|
||||
|
||||
@@ -198,7 +233,11 @@ func (s *supervisor) loop(ctx context.Context) {
|
||||
// allocates, so we see a contender during its startup rather than after it has
|
||||
// already failed to get the memory it wanted.
|
||||
func (s *supervisor) tick(ctx context.Context) {
|
||||
others := s.probe.foreign(s.run.pid())
|
||||
var pids []int
|
||||
for _, r := range s.children() {
|
||||
pids = append(pids, r.pid())
|
||||
}
|
||||
others := s.probe.foreign(pids...)
|
||||
if len(others) > 0 {
|
||||
s.foreignStreak++
|
||||
s.clearStreak = 0
|
||||
@@ -207,31 +246,65 @@ func (s *supervisor) tick(ctx context.Context) {
|
||||
s.clearStreak++
|
||||
}
|
||||
|
||||
if s.run.running() {
|
||||
s.run.refreshReady(ctx)
|
||||
switch {
|
||||
case s.foreignStreak >= s.cfg.EvictAfter:
|
||||
log.Printf("mavgpud: yielding the card to %s", describe(others))
|
||||
s.run.stop(time.Duration(s.cfg.StopGrace))
|
||||
case s.idle() > time.Duration(s.cfg.IdleTimeout):
|
||||
log.Printf("mavgpud: idle for %s, unloading", s.idle().Round(time.Second))
|
||||
s.run.stop(time.Duration(s.cfg.StopGrace))
|
||||
// Yielding is all or nothing. A CPT run wants the whole card, and handing
|
||||
// back 8GB while holding 1.6GB is the shape of a failed allocation.
|
||||
if s.foreignStreak >= s.cfg.EvictAfter && s.anyRunning() {
|
||||
log.Printf("mavgpud: yielding the card to %s", describe(others))
|
||||
for _, r := range s.children() {
|
||||
r.stop(time.Duration(s.cfg.StopGrace))
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
if s.clearStreak < s.cfg.StartAfter {
|
||||
clear := s.clearStreak >= s.cfg.StartAfter
|
||||
|
||||
if s.run.running() {
|
||||
s.run.refreshReady(ctx)
|
||||
if s.idle() > time.Duration(s.cfg.IdleTimeout) {
|
||||
log.Printf("mavgpud: idle for %s, unloading", s.idle().Round(time.Second))
|
||||
s.run.stop(time.Duration(s.cfg.StopGrace))
|
||||
}
|
||||
} else if clear && s.probe.freeVRAM() >= s.cfg.MinFreeVRAM {
|
||||
s.touch() // the idle clock starts at load, not at the last request before it
|
||||
if err := s.run.start(); err != nil {
|
||||
log.Printf("mavgpud: start llama-server: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
if s.stt == nil {
|
||||
return
|
||||
}
|
||||
if free := s.probe.freeVRAM(); free < s.cfg.MinFreeVRAM {
|
||||
if s.stt.running() {
|
||||
s.stt.refreshReady(ctx)
|
||||
return
|
||||
}
|
||||
s.touch() // the idle clock starts at load, not at the last request before it
|
||||
if err := s.run.start(); err != nil {
|
||||
log.Printf("mavgpud: start llama-server: %v", err)
|
||||
// No VRAM precondition here, unlike llama-server. That check exists because
|
||||
// a 12B refuses to load when the card is short, and 1.6GB fits wherever the
|
||||
// KFD is clear. Reading free VRAM would also block the transcriber for good
|
||||
// once the language model was resident, since it holds more than the floor.
|
||||
if clear {
|
||||
if err := s.stt.start(); err != nil {
|
||||
log.Printf("mavgpud: start cw2: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (s *supervisor) children() []*runner {
|
||||
if s.stt == nil {
|
||||
return []*runner{s.run}
|
||||
}
|
||||
return []*runner{s.run, s.stt}
|
||||
}
|
||||
|
||||
func (s *supervisor) anyRunning() bool {
|
||||
for _, r := range s.children() {
|
||||
if r.running() {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// describe names the contenders in the log. This log is the instrument for the
|
||||
// open question in #488: whether polling the KFD misses a job that wants the
|
||||
// card without registering there.
|
||||
|
||||
+17
-13
@@ -10,14 +10,18 @@ import (
|
||||
"time"
|
||||
)
|
||||
|
||||
// runner owns one llama-server process. Owning it is the point of the daemon:
|
||||
// the workstation cannot keep a 7-14B resident, because that holds 16GB against
|
||||
// runner owns one GPU process. Owning it is the point of the daemon: the
|
||||
// workstation cannot keep a 7-14B resident, because that holds 16GB against
|
||||
// the owner's CPT runs, Correx and the manga-recap pipeline. So the thing that
|
||||
// stays up is this, which costs no VRAM, and the model comes and goes under it.
|
||||
//
|
||||
// There are two of them since 09-08-2026: llama-server and the CW2 transcriber.
|
||||
// name is what the log calls this one.
|
||||
type runner struct {
|
||||
name string
|
||||
bin string
|
||||
args []string
|
||||
// ready is llama-server's own /health, which answers "is a model loaded".
|
||||
// ready is the child's own /health, which answers "is a model loaded".
|
||||
// Loading a 7-14B takes tens of seconds, so started is not ready.
|
||||
readyURL string
|
||||
|
||||
@@ -32,9 +36,9 @@ type runner struct {
|
||||
http *http.Client
|
||||
}
|
||||
|
||||
func newRunner(bin string, args []string, readyURL string) *runner {
|
||||
func newRunner(name, bin string, args []string, readyURL string) *runner {
|
||||
return &runner{
|
||||
bin: bin, args: args, readyURL: readyURL,
|
||||
name: name, bin: bin, args: args, readyURL: readyURL,
|
||||
http: &http.Client{Timeout: 2 * time.Second},
|
||||
}
|
||||
}
|
||||
@@ -60,7 +64,7 @@ func (r *runner) isReady() bool {
|
||||
return r.ready
|
||||
}
|
||||
|
||||
// start launches llama-server. It returns as soon as the process exists, not
|
||||
// start launches the child. It returns as soon as the process exists, not
|
||||
// when the model is loaded.
|
||||
func (r *runner) start() error {
|
||||
r.mu.Lock()
|
||||
@@ -76,7 +80,7 @@ func (r *runner) start() error {
|
||||
return err
|
||||
}
|
||||
r.cmd, r.ready, r.yielding = cmd, false, false
|
||||
log.Printf("mavgpud: started llama-server pid=%d", cmd.Process.Pid)
|
||||
log.Printf("mavgpud: started %s pid=%d", r.name, cmd.Process.Pid)
|
||||
go func() {
|
||||
err := cmd.Wait()
|
||||
r.mu.Lock()
|
||||
@@ -84,15 +88,15 @@ func (r *runner) start() error {
|
||||
r.cmd, r.ready, r.yielding = nil, false, false
|
||||
r.mu.Unlock()
|
||||
if yielded {
|
||||
log.Printf("mavgpud: llama-server stopped, card yielded (%v)", err)
|
||||
log.Printf("mavgpud: %s stopped, card yielded (%v)", r.name, err)
|
||||
return
|
||||
}
|
||||
log.Printf("mavgpud: llama-server exited: %v", err)
|
||||
log.Printf("mavgpud: %s exited: %v", r.name, err)
|
||||
}()
|
||||
return nil
|
||||
}
|
||||
|
||||
// stop ends llama-server and waits for the VRAM to come back. SIGTERM first so
|
||||
// stop ends the child and waits for the VRAM to come back. SIGTERM first so
|
||||
// it unmaps cleanly, SIGKILL after the grace window. Returning before the
|
||||
// process is gone would let the supervisor report a free card while 14GB is
|
||||
// still mapped, which is the one lie that would make yielding useless.
|
||||
@@ -117,11 +121,11 @@ func (r *runner) stop(grace time.Duration) {
|
||||
}
|
||||
time.Sleep(100 * time.Millisecond)
|
||||
}
|
||||
log.Printf("mavgpud: llama-server did not exit in %s, killing", grace)
|
||||
log.Printf("mavgpud: %s did not exit in %s, killing", r.name, grace)
|
||||
_ = syscall.Kill(pgid, syscall.SIGKILL)
|
||||
}
|
||||
|
||||
// refreshReady asks llama-server whether the model is loaded. Called once per
|
||||
// refreshReady asks the child whether the model is loaded. Called once per
|
||||
// supervisor tick, never per request.
|
||||
func (r *runner) refreshReady(ctx context.Context) {
|
||||
if !r.running() {
|
||||
@@ -141,6 +145,6 @@ func (r *runner) refreshReady(ctx context.Context) {
|
||||
r.ready = ok
|
||||
r.mu.Unlock()
|
||||
if ok && !was {
|
||||
log.Printf("mavgpud: model ready")
|
||||
log.Printf("mavgpud: %s ready", r.name)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -25,7 +25,7 @@ func fakeServer(t *testing.T, body string) string {
|
||||
// status of a routine yield is identical to that of a real crash. Reading the
|
||||
// mavgpud log, the two were indistinguishable (Vikunja #491).
|
||||
func TestStopMarksTheExitAsAYield(t *testing.T) {
|
||||
r := newRunner(fakeServer(t, "while : ; do sleep 1 ; done"), nil, "")
|
||||
r := newRunner("fake", fakeServer(t, "while : ; do sleep 1 ; done"), nil, "")
|
||||
if err := r.start(); err != nil {
|
||||
t.Fatalf("start: %v", err)
|
||||
}
|
||||
@@ -49,7 +49,7 @@ func TestStopMarksTheExitAsAYield(t *testing.T) {
|
||||
// Stopping when nothing is running must not arm the flag for the next child.
|
||||
// The next exit after that would be a real crash logged as a yield.
|
||||
func TestStopWithNoChildDoesNotArmTheFlag(t *testing.T) {
|
||||
r := newRunner("/nonexistent", nil, "")
|
||||
r := newRunner("fake", "/nonexistent", nil, "")
|
||||
r.stop(10 * time.Millisecond)
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
|
||||
+70
-8
@@ -5,12 +5,24 @@
|
||||
// is detected sends it as a PushToTalk frame to the voice server. The reply
|
||||
// audio is played back through aplay(1).
|
||||
//
|
||||
// No wake-word model yet (MVP uses voice-activity-only trigger). The
|
||||
// SurfaceVoice auth layer caps all commands at L0 (no destructive acts),
|
||||
// making accidental triggers safe by design. A proper wake-word engine
|
||||
// (openWakeWord / Silero VAD ONNX) is the planned upgrade — the VAD shape
|
||||
// (30ms frames, 16kHz PCM) matches silero-vad's input interface exactly, so
|
||||
// swapping energy-threshold for ONNX-inference is a local change in vad.go.
|
||||
// Voice activity is silero-vad when -vad-model points at the graph, and an
|
||||
// energy threshold when it does not. Silero declines noise the threshold
|
||||
// accepts: 0 frames against 68 to 99 on the four fixtures, measured in
|
||||
// docs/evals/2026-08-09-silero-vad.md. Note that the model window is 512
|
||||
// samples and the capture frame is 480, so silero.go re-chunks. This comment
|
||||
// used to say the two matched, which was true of silero v4.
|
||||
//
|
||||
// The keyword is "Мэйвен" and it is required, when -wake-model points at the
|
||||
// head (V-487 stage two). Without it anything spoken near the microphone
|
||||
// becomes a turn, which the SurfaceVoice auth layer makes safe rather than
|
||||
// expensive: it caps all commands at L0, no destructive acts. It does not cap
|
||||
// reading, so an open gate still lets the room hear his facts read back.
|
||||
// wakeword.go holds the cadence and wakefeatures.go the three models.
|
||||
//
|
||||
// The conn carries both directions. mavwaked sends utterances and receives
|
||||
// proactive nudges on it, and it is opened at startup rather than at the first
|
||||
// utterance, because mavend registers a voice session on accept. See nudge.go
|
||||
// for why a nudge that is not heard is worse than one that is not delivered.
|
||||
//
|
||||
// While a reply is playing the capture side is muted (half-duplex): without
|
||||
// it, Maven's own voice comes back in through the mic and she answers
|
||||
@@ -52,6 +64,12 @@ const (
|
||||
defaultAddr = "127.0.0.1:9100"
|
||||
defaultLang = "ru"
|
||||
defaultReadSize = 4096 // max PCM bytes per read from arecord (fits multiple frames)
|
||||
|
||||
// defaultWakeWindowMs — how long the keyword stays good for. He says
|
||||
// "Мэйвен" and then a sentence, and the VAD does not close the utterance
|
||||
// until he stops, so this has to outlive the word by the length of what
|
||||
// follows it. It is spent on dispatch: one keyword, one turn.
|
||||
defaultWakeWindowMs = 8000
|
||||
)
|
||||
|
||||
func main() {
|
||||
@@ -73,17 +91,38 @@ func run(args []string) error {
|
||||
bargeIn := flag.Bool("barge-in", false, "cut Maven off when he talks over her (needs a room-tuned -barge-in-rms)")
|
||||
bargeRMS := flag.Int("barge-in-rms", defaultBargeRMS, "RMS x10000 a frame must clear to count as barge-in")
|
||||
bargeFrames := flag.Int("barge-in-frames", defaultBargeFrames, "consecutive frames over -barge-in-rms before playback is cut")
|
||||
vadModel := flag.String("vad-model", "", "silero-vad onnx file; empty runs the energy threshold instead")
|
||||
vadThreshold := flag.Float64("vad-threshold", defaultSileroThreshold, "speech probability a frame must clear")
|
||||
onnxLib := flag.String("onnx-lib", os.Getenv("MAVEN_ONNX_LIB"), "libonnxruntime.so, needed with -vad-model")
|
||||
wakeModel := flag.String("wake-model", "", "keyword head onnx; empty ships every utterance, as before V-487")
|
||||
wakeMel := flag.String("wake-mel", "", "melspectrogram.onnx, required with -wake-model")
|
||||
wakeEmbed := flag.String("wake-embed", "", "embedding_model.onnx, required with -wake-model")
|
||||
wakeThreshold := flag.Float64("wake-threshold", defaultWakeThreshold, "score the keyword must clear")
|
||||
wakeWindowMs := flag.Int("wake-window-ms", defaultWakeWindowMs, "ms an utterance may still start after the keyword")
|
||||
flag.CommandLine.Parse(args)
|
||||
|
||||
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM, syscall.SIGHUP)
|
||||
defer stop()
|
||||
|
||||
// Voice client — reused across utterances; SendRequest reconnects on error.
|
||||
// Voice client — one conn carrying both directions. SendRequest reconnects
|
||||
// on error, and the push receiver redials on its own clock.
|
||||
vc := voice.Dial(*addr)
|
||||
defer vc.Close()
|
||||
|
||||
// VAD engine.
|
||||
// VAD engine. A model that will not load is logged and not fatal: the
|
||||
// energy threshold is worse, and it is a great deal better than a
|
||||
// listening client that refuses to start.
|
||||
vad := NewVAD(*minRMS, *speechMs, *silenceMs, *maxMs)
|
||||
if *vadModel != "" {
|
||||
s, err := newSileroVAD(*vadModel, *onnxLib)
|
||||
if err != nil {
|
||||
log.Printf("mavwaked: silero unavailable, energy threshold unchanged: %v", err)
|
||||
} else {
|
||||
defer s.Close()
|
||||
vad.UseSilero(s, *vadThreshold)
|
||||
log.Printf("mavwaked: silero-vad from %s, threshold %.2f", *vadModel, *vadThreshold)
|
||||
}
|
||||
}
|
||||
|
||||
// Audio source.
|
||||
var src io.ReadCloser
|
||||
@@ -145,6 +184,29 @@ func run(args []string) error {
|
||||
}
|
||||
sess := newSession(vad, newAplayPlayer(), &voiceSender{vc: vc}, *lang, barge)
|
||||
|
||||
// Keyword gate. A model that will not load is logged and not fatal, for
|
||||
// the same reason silero's is not: an open gate is the daemon he had
|
||||
// yesterday, and a daemon that refuses to start is not.
|
||||
if *wakeModel != "" {
|
||||
w, err := newWakeWord(*wakeMel, *wakeEmbed, *wakeModel, *onnxLib, *wakeThreshold)
|
||||
if err != nil {
|
||||
log.Printf("mavwaked: wake word unavailable, every utterance is a turn: %v", err)
|
||||
} else {
|
||||
defer w.Close()
|
||||
sess.UseWakeWord(w, time.Duration(*wakeWindowMs)*time.Millisecond)
|
||||
log.Printf("mavwaked: wake word from %s, threshold %.3f, window %dms",
|
||||
*wakeModel, *wakeThreshold, *wakeWindowMs)
|
||||
}
|
||||
}
|
||||
|
||||
// Listen for nudges alongside capture. Connect eagerly so mavend has a
|
||||
// voice session before he has said anything: without one, a nudge routed
|
||||
// to voice finds nobody home and goes to the away channels instead.
|
||||
if err := vc.Connect(ctx); err != nil {
|
||||
log.Printf("mavwaked: voice server not reachable yet, retrying in background: %v", err)
|
||||
}
|
||||
go runNudgeReceiver(ctx, vc, sess)
|
||||
|
||||
return captureLoop(ctx, src, sess)
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
package main
|
||||
|
||||
// The receiving half of the voice reach (V-671).
|
||||
//
|
||||
// mavwaked used to send and never listen. It wired no PushHandler, and
|
||||
// SendRequest discards a push frame when there is none. The consequence was
|
||||
// not a missing feature but a silent one: mavend routes a nudge to the voice
|
||||
// session that spoke most recently, and once mavwaked had spoken once it WAS
|
||||
// that session. PushToMostRecent succeeded, the dispatcher counted the nudge
|
||||
// delivered and stopped rerouting to telegram and ntfy, and mavwaked threw the
|
||||
// audio away. He heard nothing, anywhere.
|
||||
//
|
||||
// So the connection is opened at startup rather than at the first utterance,
|
||||
// and it is held open. A client that has never connected has no session, and
|
||||
// the dispatcher must be able to tell "he is not at the machine" from "he is,
|
||||
// and she has nothing to say".
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"log"
|
||||
"time"
|
||||
|
||||
"github.com/kami/maven/internal/voice"
|
||||
)
|
||||
|
||||
// nudgeRetry is how long to wait before dialling again after the conn ends.
|
||||
// mavend restarts on every deploy, and a listener that gives up then is a
|
||||
// listener that is deaf until the next reboot.
|
||||
const nudgeRetry = 5 * time.Second
|
||||
|
||||
// nudgeHandler decodes a push and hands the audio to the session, which
|
||||
// speaks it through the same player the reply path uses. It does not play
|
||||
// anything itself: the half-duplex gate and barge-in live on the capture
|
||||
// loop, and a nudge has to sit under both.
|
||||
type nudgeHandler struct{ sess *session }
|
||||
|
||||
func (h *nudgeHandler) OnPush(p voice.Push) {
|
||||
if p.Kind != voice.PushKindAudioNudge {
|
||||
log.Printf("mavwaked: ignoring push of unknown kind %q", p.Kind)
|
||||
return
|
||||
}
|
||||
var ap voice.AudioNudgePush
|
||||
if err := json.Unmarshal(p.Params, &ap); err != nil {
|
||||
log.Printf("mavwaked: nudge: decode: %v", err)
|
||||
return
|
||||
}
|
||||
log.Printf("mavwaked: nudge from rule %q (severity %d): %q (%.2fs audio)",
|
||||
ap.RuleName, ap.Severity, ap.Text, ap.Audio.Duration())
|
||||
if len(ap.Audio.Bytes) == 0 {
|
||||
// mavttsd was down or the text was empty. Say so rather than going
|
||||
// quiet: the dispatcher already counted this one as delivered.
|
||||
log.Printf("mavwaked: nudge %q carried no audio, nothing to speak", ap.RuleName)
|
||||
return
|
||||
}
|
||||
h.sess.Nudge(ap.Audio)
|
||||
}
|
||||
|
||||
// runNudgeReceiver keeps a push handler wired for as long as ctx lives,
|
||||
// redialling whenever the conn ends. Returns when ctx is cancelled.
|
||||
func runNudgeReceiver(ctx context.Context, vc *voice.Client, sess *session) {
|
||||
h := &nudgeHandler{sess: sess}
|
||||
for {
|
||||
err := vc.RunPushReceiver(ctx, h)
|
||||
if ctx.Err() != nil {
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
log.Printf("mavwaked: nudge receiver: %v", err)
|
||||
} else {
|
||||
log.Printf("mavwaked: voice connection ended, reconnecting in %s", nudgeRetry)
|
||||
}
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-time.After(nudgeRetry):
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,170 @@
|
||||
package main
|
||||
|
||||
// The receiving half: a nudge pushed by mavend has to reach the speaker, and
|
||||
// it has to obey the same two gates a reply obeys (V-671).
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/kami/maven/internal/audio"
|
||||
"github.com/kami/maven/internal/voice"
|
||||
)
|
||||
|
||||
func nudgeAudio() audio.Audio {
|
||||
return audio.Audio{Format: audio.PCM16kMono, Bytes: make([]byte, 8000)}
|
||||
}
|
||||
|
||||
// pushFrame builds the frame mavend's voicesink sends.
|
||||
func pushFrame(t *testing.T, a audio.Audio) voice.Push {
|
||||
t.Helper()
|
||||
body, err := json.Marshal(voice.AudioNudgePush{
|
||||
RuleName: "test-rule",
|
||||
Severity: 3,
|
||||
Audio: a,
|
||||
Text: "пора пить воду",
|
||||
Ts: time.Unix(0, 0),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("marshal push: %v", err)
|
||||
}
|
||||
return voice.Push{Kind: voice.PushKindAudioNudge, Params: body}
|
||||
}
|
||||
|
||||
// The defect itself: the push arrived and nothing came out of the speaker.
|
||||
func TestNudgeReachesThePlayer(t *testing.T) {
|
||||
sess, p, snd := newTestSession(bargeInConfig{})
|
||||
(&nudgeHandler{sess: sess}).OnPush(pushFrame(t, nudgeAudio()))
|
||||
|
||||
if p.plays != 0 {
|
||||
t.Fatal("nudge played from the push goroutine; it must wait for the capture loop")
|
||||
}
|
||||
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
if p.plays != 1 {
|
||||
t.Fatalf("plays = %d, want 1", p.plays)
|
||||
}
|
||||
if len(p.last.Bytes) != 8000 {
|
||||
t.Errorf("played %d bytes, want the nudge audio", len(p.last.Bytes))
|
||||
}
|
||||
if sess.nudges != 1 {
|
||||
t.Errorf("nudges = %d, want 1", sess.nudges)
|
||||
}
|
||||
if len(snd.sent) != 0 {
|
||||
t.Errorf("a nudge must not be shipped back to the daemon as an utterance")
|
||||
}
|
||||
}
|
||||
|
||||
// A push of some other kind, or one carrying no audio, must not reach the
|
||||
// player and must not wedge the one that follows.
|
||||
func TestNudgeIgnoresUnusablePushes(t *testing.T) {
|
||||
sess, p, _ := newTestSession(bargeInConfig{})
|
||||
h := &nudgeHandler{sess: sess}
|
||||
|
||||
h.OnPush(voice.Push{Kind: "something-else", Params: json.RawMessage(`{}`)})
|
||||
h.OnPush(voice.Push{Kind: voice.PushKindAudioNudge, Params: json.RawMessage(`not json`)})
|
||||
h.OnPush(pushFrame(t, audio.Audio{Format: audio.PCM16kMono}))
|
||||
|
||||
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
if p.plays != 0 {
|
||||
t.Fatalf("plays = %d, want 0", p.plays)
|
||||
}
|
||||
|
||||
h.OnPush(pushFrame(t, nudgeAudio()))
|
||||
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
if p.plays != 1 {
|
||||
t.Fatalf("plays after a usable nudge = %d, want 1", p.plays)
|
||||
}
|
||||
}
|
||||
|
||||
// The half-duplex gate covers a nudge exactly as it covers a reply: she does
|
||||
// not start one over herself, and the mic stays muted while it runs.
|
||||
func TestNudgeWaitsForTheReplyToFinish(t *testing.T) {
|
||||
sess, p, _ := newTestSession(bargeInConfig{})
|
||||
speakThenPause(t, sess)
|
||||
if !p.Playing() {
|
||||
t.Fatal("expected the reply to be playing")
|
||||
}
|
||||
plays := p.plays
|
||||
|
||||
(&nudgeHandler{sess: sess}).OnPush(pushFrame(t, nudgeAudio()))
|
||||
for i := 0; i < 20; i++ {
|
||||
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
}
|
||||
if p.plays != plays {
|
||||
t.Fatalf("nudge cut across the reply: plays = %d, want %d", p.plays, plays)
|
||||
}
|
||||
|
||||
p.playing = false
|
||||
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
if p.plays != plays+1 {
|
||||
t.Fatalf("nudge never played after the reply ended: plays = %d", p.plays)
|
||||
}
|
||||
}
|
||||
|
||||
// Speaking a nudge must not leave half a sentence in the VAD. The frames
|
||||
// captured before it are pre-nudge speech, and splicing them onto whatever he
|
||||
// says afterwards ships one utterance that is two.
|
||||
func TestNudgeResetsTheVAD(t *testing.T) {
|
||||
sess, p, snd := newTestSession(bargeInConfig{})
|
||||
loud := frameAt(0.35)
|
||||
speechFrames := (defaultSpeechMs + defaultFrameMs - 1) / defaultFrameMs
|
||||
for i := 0; i < speechFrames+5; i++ {
|
||||
if err := sess.feed(context.Background(), loud); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
(&nudgeHandler{sess: sess}).OnPush(pushFrame(t, nudgeAudio()))
|
||||
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
if p.plays != 1 {
|
||||
t.Fatalf("nudge did not play: plays = %d", p.plays)
|
||||
}
|
||||
|
||||
// Playback ends, silence follows. The half-formed utterance must be gone
|
||||
// rather than closing on the first quiet frame.
|
||||
p.playing = false
|
||||
silenceFrames := (defaultSilenceMs+defaultFrameMs-1)/defaultFrameMs + 2
|
||||
for i := 0; i < silenceFrames; i++ {
|
||||
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
}
|
||||
if len(snd.sent) != 0 {
|
||||
t.Fatalf("sent %d utterances after a nudge, want 0", len(snd.sent))
|
||||
}
|
||||
}
|
||||
|
||||
// Two nudges queued back to back: the newer one is what he hears. The
|
||||
// PushHandler contract in internal/voice says the next nudge replaces the
|
||||
// stale one rather than dogpiling on it.
|
||||
func TestNudgeReplacesAnUnspokenOne(t *testing.T) {
|
||||
sess, p, _ := newTestSession(bargeInConfig{})
|
||||
h := &nudgeHandler{sess: sess}
|
||||
|
||||
h.OnPush(pushFrame(t, audio.Audio{Format: audio.PCM16kMono, Bytes: make([]byte, 4000)}))
|
||||
h.OnPush(pushFrame(t, audio.Audio{Format: audio.PCM16kMono, Bytes: make([]byte, 12000)}))
|
||||
|
||||
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
if p.plays != 1 {
|
||||
t.Fatalf("plays = %d, want 1", p.plays)
|
||||
}
|
||||
if len(p.last.Bytes) != 12000 {
|
||||
t.Errorf("played %d bytes, want the newer nudge", len(p.last.Bytes))
|
||||
}
|
||||
}
|
||||
+136
-1
@@ -7,6 +7,7 @@ package main
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/kami/maven/internal/audio"
|
||||
@@ -19,6 +20,15 @@ type utteranceSender interface {
|
||||
Send(ctx context.Context, utt audio.Audio, lang string) (audio.Audio, error)
|
||||
}
|
||||
|
||||
// keywordGate answers whether the keyword has just been spoken. The
|
||||
// production one is wakeWord; tests substitute a recorder, because a gate that
|
||||
// can only be exercised with three ONNX files is a gate nobody tests.
|
||||
type keywordGate interface {
|
||||
Feed(frame []int16) bool
|
||||
Reset()
|
||||
Score() float64
|
||||
}
|
||||
|
||||
// bargeInConfig holds the two numbers barge-in needs. Zero Frames disables
|
||||
// barge-in entirely — the half-duplex gate still runs.
|
||||
type bargeInConfig struct {
|
||||
@@ -62,11 +72,29 @@ type session struct {
|
||||
// whenever playback ends.
|
||||
loudFrames int
|
||||
|
||||
// wake is the keyword gate, or nil when no model was loaded. wakeUntil is
|
||||
// how long a keyword stays good for: he says "Мэйвен" and then a sentence,
|
||||
// and the VAD does not close the utterance until he stops, so the window
|
||||
// has to outlive the word by the length of what follows it.
|
||||
wake keywordGate
|
||||
wakeWindow time.Duration
|
||||
wakeUntil time.Time
|
||||
|
||||
// pending holds a nudge the push receiver handed over, waiting for the
|
||||
// capture loop to speak it. It is the one field written from another
|
||||
// goroutine, hence the mutex; everything else in this struct belongs to
|
||||
// the capture loop alone.
|
||||
nudgeMu sync.Mutex
|
||||
pending *audio.Audio
|
||||
|
||||
// counters, read by tests and logged on the way out.
|
||||
suppressed int // frames dropped because she was speaking
|
||||
dropped int // frames dropped as round-trip backlog
|
||||
bargeIns int // times playback was cut because he spoke over her
|
||||
sent int // utterances shipped to the daemon
|
||||
nudges int // proactive pushes spoken through the speaker
|
||||
wakes int // times the keyword opened the gate
|
||||
ignored int // complete utterances dropped because the keyword was absent
|
||||
|
||||
// loudSum and loudSeen accumulate the energy of suppressed frames, so
|
||||
// the operator can read what the room actually measures and set
|
||||
@@ -79,6 +107,12 @@ func newSession(vad *VAD, p player, s utteranceSender, lang string, barge bargeI
|
||||
return &session{vad: vad, player: p, sender: s, lang: lang, barge: barge, now: time.Now}
|
||||
}
|
||||
|
||||
// UseWakeWord puts the keyword gate in front of dispatch. Without it every
|
||||
// utterance is shipped, which is what mavwaked did before V-487 stage two.
|
||||
func (s *session) UseWakeWord(w keywordGate, window time.Duration) {
|
||||
s.wake, s.wakeWindow = w, window
|
||||
}
|
||||
|
||||
// frameDuration is the wall time one captured frame represents.
|
||||
const frameDuration = defaultFrameMs * time.Millisecond
|
||||
|
||||
@@ -138,6 +172,7 @@ func (s *session) feed(ctx context.Context, frame []byte) error {
|
||||
s.bargeIns++
|
||||
s.loudFrames = 0
|
||||
s.vad.Reset()
|
||||
s.resetWake()
|
||||
log.Printf("mavwaked: barge-in — stopped playback")
|
||||
s.replayRecent()
|
||||
return nil
|
||||
@@ -148,15 +183,101 @@ func (s *session) feed(ctx context.Context, frame []byte) error {
|
||||
if s.loudFrames != 0 {
|
||||
s.loudFrames = 0
|
||||
s.vad.Reset()
|
||||
// The wake word saw nothing during playback, so what it holds is from
|
||||
// before she spoke. Judging what he says next on it would score a
|
||||
// sentence that ended a reply ago.
|
||||
s.resetWake()
|
||||
}
|
||||
|
||||
utt, state := s.vad.Feed(PCMToI16(frame))
|
||||
if s.startPendingNudge() {
|
||||
return nil
|
||||
}
|
||||
|
||||
// The keyword is scored on the same frames the VAD sees, and only on the
|
||||
// ones that reach here: every path above returns while she is speaking, so
|
||||
// her own voice saying "Мэйвен" cannot wake her.
|
||||
pcm := PCMToI16(frame)
|
||||
if s.wake != nil && s.wake.Feed(pcm) {
|
||||
s.wakes++
|
||||
s.wakeUntil = s.now().Add(s.wakeWindow)
|
||||
log.Printf("mavwaked: keyword heard (score %.3f), listening for %s",
|
||||
s.wake.Score(), s.wakeWindow)
|
||||
}
|
||||
|
||||
utt, state := s.vad.Feed(pcm)
|
||||
if state == StateSpeech || utt.Bytes == nil {
|
||||
return nil
|
||||
}
|
||||
return s.dispatch(ctx, utt)
|
||||
}
|
||||
|
||||
// Nudge hands proactive audio to the session, to be spoken as soon as the
|
||||
// capture loop finds a quiet moment. Safe to call from the push receiver
|
||||
// goroutine; nothing else here is.
|
||||
//
|
||||
// A nudge arriving while one is already waiting REPLACES it. That is the
|
||||
// contract internal/voice states for PushHandler: the next nudge replaces the
|
||||
// stale one in his attention rather than dogpiling on it.
|
||||
func (s *session) Nudge(a audio.Audio) {
|
||||
if len(a.Bytes) == 0 {
|
||||
return
|
||||
}
|
||||
s.nudgeMu.Lock()
|
||||
if s.pending != nil {
|
||||
log.Printf("mavwaked: nudge replaced one still waiting to be spoken")
|
||||
}
|
||||
s.pending = &a
|
||||
s.nudgeMu.Unlock()
|
||||
}
|
||||
|
||||
// takeNudge removes and returns the waiting nudge, or nil.
|
||||
func (s *session) takeNudge() *audio.Audio {
|
||||
s.nudgeMu.Lock()
|
||||
defer s.nudgeMu.Unlock()
|
||||
a := s.pending
|
||||
s.pending = nil
|
||||
return a
|
||||
}
|
||||
|
||||
// startPendingNudge speaks a waiting nudge and reports whether it started
|
||||
// one. It runs on the capture loop, past the half-duplex gate, so a nudge
|
||||
// never cuts across a reply and never plays into a backlog drain.
|
||||
//
|
||||
// The VAD is reset first. Playback is about to suppress every frame until it
|
||||
// ends, and a half-heard sentence left in the VAD would splice onto whatever
|
||||
// he says afterwards. Barge-in needs no special case: it reads the player,
|
||||
// and the player does not care which audio it is playing.
|
||||
func (s *session) startPendingNudge() bool {
|
||||
a := s.takeNudge()
|
||||
if a == nil {
|
||||
return false
|
||||
}
|
||||
s.vad.Reset()
|
||||
s.nudges++
|
||||
log.Printf("mavwaked: speaking nudge (%.2fs audio)", a.Duration())
|
||||
s.player.Play(*a)
|
||||
return true
|
||||
}
|
||||
|
||||
// awake reports whether an utterance ending now was addressed to her.
|
||||
//
|
||||
// With no wake word loaded every utterance is, which is exactly what mavwaked
|
||||
// did before this gate existed. An operator with no model file gets the old
|
||||
// daemon rather than a daemon that refuses to hear anything.
|
||||
func (s *session) awake() bool {
|
||||
if s.wake == nil {
|
||||
return true
|
||||
}
|
||||
return s.now().Before(s.wakeUntil)
|
||||
}
|
||||
|
||||
// resetWake drops the gate's streaming state when there is a gate.
|
||||
func (s *session) resetWake() {
|
||||
if s.wake != nil {
|
||||
s.wake.Reset()
|
||||
}
|
||||
}
|
||||
|
||||
// keepRecent stores a copy of one barge-in trigger frame, keeping at most
|
||||
// barge.Frames of them.
|
||||
func (s *session) keepRecent(frame []byte) {
|
||||
@@ -197,6 +318,19 @@ func (s *session) replayRecent() {
|
||||
// whole backlog straight into the VAD, and a Send error did the same on every
|
||||
// failed turn, so a dead socket drove a retry loop off nothing but backlog.
|
||||
func (s *session) dispatch(ctx context.Context, utt audio.Audio) error {
|
||||
if !s.awake() {
|
||||
s.ignored++
|
||||
log.Printf("mavwaked: utterance ignored, keyword not heard (%.2fs, %d ignored so far)",
|
||||
utt.Duration(), s.ignored)
|
||||
s.vad.Reset()
|
||||
s.resetWake()
|
||||
return nil
|
||||
}
|
||||
// One keyword, one turn. A window that renewed itself on every reply would
|
||||
// leave the microphone open for as long as he kept talking, which is the
|
||||
// state this gate exists to end.
|
||||
s.wakeUntil = time.Time{}
|
||||
|
||||
log.Printf("mavwaked: utterance complete (%.2fs, %d bytes), sending...", utt.Duration(), len(utt.Bytes))
|
||||
start := s.now()
|
||||
reply, err := s.sender.Send(ctx, utt, s.lang)
|
||||
@@ -225,6 +359,7 @@ func (s *session) dispatch(ctx context.Context, utt audio.Audio) error {
|
||||
// recorded before she started speaking.
|
||||
func (s *session) dropBacklog(start time.Time) {
|
||||
s.vad.Reset()
|
||||
s.resetWake()
|
||||
s.loudFrames = 0
|
||||
s.recent = s.recent[:0]
|
||||
if elapsed := s.now().Sub(start); elapsed > 0 {
|
||||
|
||||
@@ -0,0 +1,171 @@
|
||||
package main
|
||||
|
||||
// silero-vad, the speech detector that replaces the energy threshold (V-487).
|
||||
//
|
||||
// Why an energy threshold is not a voice activity detector. It answers "is
|
||||
// this frame loud", and a fan, a door and a television are all loud. mavwaked
|
||||
// sends every utterance it accepts to speech-to-text and then to the daemon,
|
||||
// so a false trigger is a turn Maven takes on something nobody said to her.
|
||||
// Silero answers "is this frame speech", which is the question.
|
||||
//
|
||||
// It is 2.3MB of ONNX and runs on one CPU core in real time. That is not an
|
||||
// aside: this is the one model in the system that may never be offloaded or
|
||||
// gated on GPU admission, because a wake path that waits on a card is not a
|
||||
// wake path.
|
||||
//
|
||||
// Nil is a working value. Without -vad-model the daemon runs the energy VAD
|
||||
// exactly as it did before this file existed.
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sync"
|
||||
|
||||
ort "github.com/yalue/onnxruntime_go"
|
||||
)
|
||||
|
||||
const (
|
||||
// sileroWindow — samples per inference at 16kHz. The model is fixed at
|
||||
// 512 and does not accept another size, which is why this file
|
||||
// re-chunks rather than reusing the 480-sample capture frame. main.go
|
||||
// used to claim the two matched; that was true of silero v4.
|
||||
sileroWindow = 512
|
||||
|
||||
// sileroContext — samples of the previous window prepended to each
|
||||
// inference, as the reference implementation does. Without it the first
|
||||
// milliseconds of every window are judged with no history and speech
|
||||
// onsets score low.
|
||||
sileroContext = 64
|
||||
|
||||
// sileroState — the LSTM state carried between windows, [2][1][128].
|
||||
sileroStateDim = 128
|
||||
|
||||
// defaultSileroThreshold — probability above which a window is speech.
|
||||
// 0.5 is the reference default. Raising it costs speech onsets, which
|
||||
// are the quietest part of an utterance.
|
||||
defaultSileroThreshold = 0.5
|
||||
)
|
||||
|
||||
// sileroVAD holds one ONNX session and the streaming state around it. It is
|
||||
// fed 30ms capture frames and answers per frame, buffering across calls
|
||||
// because 480 samples never line up with a 512-sample window.
|
||||
type sileroVAD struct {
|
||||
mu sync.Mutex
|
||||
session *ort.DynamicAdvancedSession
|
||||
|
||||
pending []float32 // samples not yet part of a full window
|
||||
context [sileroContext]float32 // tail of the previous window
|
||||
state []float32 // [2][1][128], carried between windows
|
||||
last float64 // most recent probability, held between windows
|
||||
sr []int64
|
||||
}
|
||||
|
||||
// newSileroVAD loads the graph. The ONNX environment is initialised here when
|
||||
// nothing else has done it, because mavwaked has no embedder to do it first.
|
||||
func newSileroVAD(modelPath, libPath string) (*sileroVAD, error) {
|
||||
if !ort.IsInitialized() {
|
||||
if libPath != "" {
|
||||
ort.SetSharedLibraryPath(libPath)
|
||||
}
|
||||
if err := ort.InitializeEnvironment(); err != nil {
|
||||
return nil, fmt.Errorf("silero: onnx runtime: %w", err)
|
||||
}
|
||||
}
|
||||
s, err := ort.NewDynamicAdvancedSession(modelPath,
|
||||
[]string{"input", "state", "sr"}, []string{"output", "stateN"}, nil)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("silero: load %s: %w", modelPath, err)
|
||||
}
|
||||
return &sileroVAD{
|
||||
session: s,
|
||||
state: make([]float32, 2*sileroStateDim),
|
||||
sr: []int64{16000},
|
||||
}, nil
|
||||
}
|
||||
|
||||
// Speech reports whether the frame carries speech, and the probability behind
|
||||
// that answer. A frame that completes no window inherits the previous
|
||||
// probability, so the caller sees one answer per frame either way.
|
||||
func (s *sileroVAD) Speech(frame []int16, threshold float64) (bool, float64) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
|
||||
for _, v := range frame {
|
||||
s.pending = append(s.pending, float32(v)/32768.0)
|
||||
}
|
||||
for len(s.pending) >= sileroWindow {
|
||||
p, err := s.infer(s.pending[:sileroWindow])
|
||||
if err != nil {
|
||||
// A failed inference must not silence the microphone. Hold the
|
||||
// last answer and let the next window try again.
|
||||
break
|
||||
}
|
||||
s.last = p
|
||||
s.pending = s.pending[sileroWindow:]
|
||||
}
|
||||
return s.last >= threshold, s.last
|
||||
}
|
||||
|
||||
// infer runs one window and rolls the state and the context forward.
|
||||
func (s *sileroVAD) infer(window []float32) (float64, error) {
|
||||
in := make([]float32, sileroContext+sileroWindow)
|
||||
copy(in, s.context[:])
|
||||
copy(in[sileroContext:], window)
|
||||
|
||||
inT, err := ort.NewTensor(ort.NewShape(1, int64(len(in))), in)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
defer inT.Destroy()
|
||||
stT, err := ort.NewTensor(ort.NewShape(2, 1, sileroStateDim), s.state)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
defer stT.Destroy()
|
||||
srT, err := ort.NewTensor(ort.NewShape(1), s.sr)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
defer srT.Destroy()
|
||||
|
||||
out, err := ort.NewEmptyTensor[float32](ort.NewShape(1, 1))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
defer out.Destroy()
|
||||
next, err := ort.NewEmptyTensor[float32](ort.NewShape(2, 1, sileroStateDim))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
defer next.Destroy()
|
||||
|
||||
if err := s.session.Run(
|
||||
[]ort.Value{inT, stT, srT},
|
||||
[]ort.Value{out, next},
|
||||
); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
copy(s.state, next.GetData())
|
||||
copy(s.context[:], in[len(in)-sileroContext:])
|
||||
return float64(out.GetData()[0]), nil
|
||||
}
|
||||
|
||||
// Reset drops the streaming state. Called at every utterance boundary and
|
||||
// after barge-in, so echo-era history never scores the next sentence.
|
||||
func (s *sileroVAD) Reset() {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
s.pending = s.pending[:0]
|
||||
s.context = [sileroContext]float32{}
|
||||
for i := range s.state {
|
||||
s.state[i] = 0
|
||||
}
|
||||
s.last = 0
|
||||
}
|
||||
|
||||
// Close releases the session.
|
||||
func (s *sileroVAD) Close() error {
|
||||
if s == nil || s.session == nil {
|
||||
return nil
|
||||
}
|
||||
return s.session.Destroy()
|
||||
}
|
||||
@@ -0,0 +1,149 @@
|
||||
package main
|
||||
|
||||
// What this measures. The energy threshold cannot tell a voice from a
|
||||
// television, and every utterance it accepts becomes a turn. So the test that
|
||||
// matters is not "does silero find speech" — it is "does it decline what the
|
||||
// energy threshold accepts".
|
||||
//
|
||||
// Speech is the four piper fixtures mavsttd already scores against. They are
|
||||
// synthesised, so nothing of the owner's voice is committed. Non-speech is
|
||||
// white noise at the same loudness, which is the cheapest thing that fools an
|
||||
// energy floor and the honest floor for this claim.
|
||||
//
|
||||
// Both halves skip without models/vad/silero_vad.onnx and MAVEN_ONNX_LIB,
|
||||
// like the TestONNX measurements in internal/router/eval.
|
||||
|
||||
import (
|
||||
"math"
|
||||
"math/rand"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
const wavHeader = 44 // 16kHz mono s16le, written by piper
|
||||
|
||||
func loadSilero(t *testing.T) *sileroVAD {
|
||||
t.Helper()
|
||||
model := filepath.Join("..", "..", "models", "vad", "silero_vad.onnx")
|
||||
lib := os.Getenv("MAVEN_ONNX_LIB")
|
||||
if _, err := os.Stat(model); err != nil {
|
||||
t.Skipf("missing %s: %v", model, err)
|
||||
}
|
||||
if lib == "" {
|
||||
t.Skip("MAVEN_ONNX_LIB unset")
|
||||
}
|
||||
s, err := newSileroVAD(model, lib)
|
||||
if err != nil {
|
||||
t.Skipf("silero unavailable: %v", err)
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// feedAll runs a whole clip through a VAD and reports how many utterances it
|
||||
// produced and how many frames it called speech.
|
||||
func feedAll(v *VAD, pcm []int16) (utterances, speechFrames int) {
|
||||
for i := 0; i+frameSamples <= len(pcm); i += frameSamples {
|
||||
frame := pcm[i : i+frameSamples]
|
||||
utt, state := v.Feed(frame)
|
||||
if state == StateSpeech {
|
||||
speechFrames++
|
||||
}
|
||||
if utt.Bytes != nil {
|
||||
utterances++
|
||||
}
|
||||
}
|
||||
return utterances, speechFrames
|
||||
}
|
||||
|
||||
func readFixture(t *testing.T, name string) []int16 {
|
||||
t.Helper()
|
||||
raw, err := os.ReadFile(filepath.Join("..", "mavsttd", "testdata", name))
|
||||
if err != nil {
|
||||
t.Skipf("missing fixture %s: %v", name, err)
|
||||
}
|
||||
if len(raw) <= wavHeader {
|
||||
t.Fatalf("%s: %d bytes, no audio", name, len(raw))
|
||||
}
|
||||
return PCMToI16(raw[wavHeader:])
|
||||
}
|
||||
|
||||
// noise returns white noise scaled to the same RMS as ref. Same loudness,
|
||||
// nothing said.
|
||||
func noise(ref []int16, seed int64) []int16 {
|
||||
target := frameRMS(ref)
|
||||
r := rand.New(rand.NewSource(seed))
|
||||
out := make([]int16, len(ref))
|
||||
for i := range out {
|
||||
out[i] = int16(r.NormFloat64() * target * 32768.0)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func TestSileroHearsSpeechAndDeclinesNoise(t *testing.T) {
|
||||
s := loadSilero(t)
|
||||
defer s.Close()
|
||||
|
||||
for _, name := range []string{"ru_fact.wav", "ru_query.wav", "ru_reminder.wav", "en_act.wav"} {
|
||||
pcm := readFixture(t, name)
|
||||
|
||||
v := NewVAD(0, 0, 0, 0)
|
||||
v.UseSilero(s, defaultSileroThreshold)
|
||||
_, spoke := feedAll(v, pcm)
|
||||
if spoke == 0 {
|
||||
t.Errorf("%s: silero heard no speech in a spoken clip", name)
|
||||
}
|
||||
|
||||
s.Reset()
|
||||
v2 := NewVAD(0, 0, 0, 0)
|
||||
v2.UseSilero(s, defaultSileroThreshold)
|
||||
_, heard := feedAll(v2, noise(pcm, 7))
|
||||
|
||||
energy := NewVAD(0, 0, 0, 0)
|
||||
_, energyHeard := feedAll(energy, noise(pcm, 7))
|
||||
|
||||
t.Logf("%s: speech frames — silero on speech %d, silero on noise %d, energy on noise %d",
|
||||
name, spoke, heard, energyHeard)
|
||||
if heard >= energyHeard {
|
||||
t.Errorf("%s: silero called %d noise frames speech, energy called %d — no improvement",
|
||||
name, heard, energyHeard)
|
||||
}
|
||||
s.Reset()
|
||||
}
|
||||
}
|
||||
|
||||
// BenchmarkSileroFrame answers the only performance question that matters
|
||||
// here: one 30ms frame must cost far less than 30ms on one core, or the
|
||||
// detector cannot run always-on beside everything else on that machine.
|
||||
func BenchmarkSileroFrame(b *testing.B) {
|
||||
s := loadSilero(&testing.T{})
|
||||
if s == nil {
|
||||
b.Skip("silero unavailable")
|
||||
}
|
||||
defer s.Close()
|
||||
frame := make([]int16, frameSamples)
|
||||
for i := range frame {
|
||||
frame[i] = int16(i%400 - 200)
|
||||
}
|
||||
for i := 0; i < b.N; i++ {
|
||||
s.Speech(frame, defaultSileroThreshold)
|
||||
}
|
||||
}
|
||||
|
||||
// TestSileroRechunksAcrossFrames pins the reason this file exists. The capture
|
||||
// frame is 480 samples and the model window is 512, so a detector that ran one
|
||||
// inference per frame would be feeding the model a shape it does not accept.
|
||||
func TestSileroRechunksAcrossFrames(t *testing.T) {
|
||||
s := loadSilero(t)
|
||||
defer s.Close()
|
||||
|
||||
silence := make([]int16, frameSamples)
|
||||
for i := 0; i < 20; i++ {
|
||||
if _, p := s.Speech(silence, defaultSileroThreshold); math.IsNaN(p) {
|
||||
t.Fatalf("frame %d: probability is NaN", i)
|
||||
}
|
||||
}
|
||||
if len(s.pending) >= sileroWindow {
|
||||
t.Errorf("pending grew to %d samples, so windows are not being consumed", len(s.pending))
|
||||
}
|
||||
}
|
||||
+35
-1
@@ -68,6 +68,37 @@ type VAD struct {
|
||||
// follows the room's ambient level. Initialised to minRMS; updated
|
||||
// on each silence frame.
|
||||
floorRMS float64
|
||||
|
||||
// speech is silero-vad, or nil. When it is set the energy floor decides
|
||||
// nothing: the question becomes "is this speech" rather than "is this
|
||||
// loud", and the noise floor is not even tracked. Everything after that
|
||||
// answer — the speech hold, the silence hold, the length cap, the
|
||||
// buffer — is the same state machine either way, which is why the
|
||||
// detector goes here and not around this type.
|
||||
speech *sileroVAD
|
||||
speechMin float64
|
||||
}
|
||||
|
||||
// UseSilero swaps the energy threshold for the model. Passing nil is a
|
||||
// no-op, so a caller that could not load the graph keeps a working VAD.
|
||||
func (v *VAD) UseSilero(s *sileroVAD, threshold float64) {
|
||||
if s == nil {
|
||||
return
|
||||
}
|
||||
if threshold <= 0 {
|
||||
threshold = defaultSileroThreshold
|
||||
}
|
||||
v.speech = s
|
||||
v.speechMin = threshold
|
||||
}
|
||||
|
||||
// isSpeech answers the one question the state machine asks of a frame.
|
||||
func (v *VAD) isSpeech(frame []int16, rms float64) bool {
|
||||
if v.speech != nil {
|
||||
ok, _ := v.speech.Speech(frame, v.speechMin)
|
||||
return ok
|
||||
}
|
||||
return rms >= v.floorRMS
|
||||
}
|
||||
|
||||
// NewVAD creates a VAD with the given thresholds. Zero values use defaults.
|
||||
@@ -110,7 +141,7 @@ func (v *VAD) State() SpeechState { return v.state }
|
||||
// should send the audio to the voice server before feeding more frames.
|
||||
func (v *VAD) Feed(frame []int16) (_ audio.Audio, state SpeechState) {
|
||||
rms := frameRMS(frame)
|
||||
isSpeech := rms >= v.floorRMS
|
||||
isSpeech := v.isSpeech(frame, rms)
|
||||
|
||||
switch v.state {
|
||||
case StateSilence:
|
||||
@@ -175,6 +206,9 @@ func (v *VAD) Feed(frame []int16) (_ audio.Audio, state SpeechState) {
|
||||
func (v *VAD) Reset() { v.reset() }
|
||||
|
||||
func (v *VAD) reset() {
|
||||
if v.speech != nil {
|
||||
v.speech.Reset()
|
||||
}
|
||||
v.state = StateSilence
|
||||
v.speechFrames = 0
|
||||
v.silenceFrames = 0
|
||||
|
||||
@@ -0,0 +1,202 @@
|
||||
package main
|
||||
|
||||
// The three models behind the wake word (V-487 stage two).
|
||||
//
|
||||
// openWakeWord's pipeline, run in a row:
|
||||
//
|
||||
// audio -> melspectrogram.onnx -> 32-bin mel frames, one per 10ms
|
||||
// 76 frames -> embedding_model.onnx -> one 96-dim embedding per 80ms
|
||||
// 16 embeds -> maven_wakeword.onnx -> one score
|
||||
//
|
||||
// The first two are frozen and pretrained. Only the last was trained here,
|
||||
// which is why it is 100KB and the other two are megabytes. The shapes are
|
||||
// not guesses: 2.0s of 16kHz audio measures 197 mel frames, and 76-frame
|
||||
// windows at stride 8 give exactly the 16 embeddings the head was fitted on.
|
||||
//
|
||||
// This file knows ONNX and nothing about the 80ms cadence. wakeword.go knows
|
||||
// the cadence and nothing about tensors.
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
ort "github.com/yalue/onnxruntime_go"
|
||||
)
|
||||
|
||||
const (
|
||||
// melHop — samples per mel frame. 10ms at 16kHz.
|
||||
melHop = 160
|
||||
// melBins — mel bins per frame, fixed by melspectrogram.onnx.
|
||||
melBins = 32
|
||||
// embedFrames — mel frames one embedding is computed over, 760ms.
|
||||
embedFrames = 76
|
||||
// embedStride — mel frames between embeddings, 80ms.
|
||||
embedStride = 8
|
||||
// embedDim — the embedding width.
|
||||
embedDim = 96
|
||||
// headWindow — embeddings the head scores at once, 1.28s of audio.
|
||||
headWindow = 16
|
||||
|
||||
// melContext — samples of history prepended to each incremental mel
|
||||
// call, chosen so the eight frames this call yields continue exactly
|
||||
// where the previous call's eight stopped.
|
||||
//
|
||||
// melspectrogram.onnx returns N/160-3 frames for N samples, and frame i
|
||||
// covers [i*160, i*160+400). With 480 samples of history the buffer is
|
||||
// 1760 samples, which is 8 frames, and the oldest of them starts one hop
|
||||
// after the newest of the previous call. Less history leaves a gap: the
|
||||
// first frames of a bare chunk would be computed against silence.
|
||||
melContext = 480
|
||||
|
||||
// chunkSamples — audio per embedding step, 80ms.
|
||||
chunkSamples = embedStride * melHop
|
||||
)
|
||||
|
||||
// wakeModels holds the three ONNX sessions. It runs on CPU threads beside
|
||||
// silero and never touches the GPU. That is a rule, not a result: a wake word
|
||||
// that waits on card admission is not a wake word.
|
||||
type wakeModels struct {
|
||||
mel *ort.DynamicAdvancedSession
|
||||
emb *ort.DynamicAdvancedSession
|
||||
head *ort.DynamicAdvancedSession
|
||||
}
|
||||
|
||||
// newWakeModels loads all three. melPath and embedPath are openWakeWord's
|
||||
// frozen feature models; headPath is the keyword head trained for "Мэйвен".
|
||||
func newWakeModels(melPath, embedPath, headPath, libPath string) (*wakeModels, error) {
|
||||
if !ort.IsInitialized() {
|
||||
if libPath != "" {
|
||||
ort.SetSharedLibraryPath(libPath)
|
||||
}
|
||||
if err := ort.InitializeEnvironment(); err != nil {
|
||||
return nil, fmt.Errorf("wake word: onnx runtime: %w", err)
|
||||
}
|
||||
}
|
||||
// One thread per session, not the default of every core. Measured on
|
||||
// workpc: the default took mavwaked from 68% of one core to 335% of
|
||||
// three, for three graphs that each run in well under 80ms single
|
||||
// threaded. An always-on gate that eats a quarter of the workstation is
|
||||
// not a gate he will leave running.
|
||||
opts, err := ort.NewSessionOptions()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("wake word: session options: %w", err)
|
||||
}
|
||||
defer opts.Destroy()
|
||||
if err := opts.SetIntraOpNumThreads(1); err != nil {
|
||||
return nil, fmt.Errorf("wake word: intra-op threads: %w", err)
|
||||
}
|
||||
if err := opts.SetInterOpNumThreads(1); err != nil {
|
||||
return nil, fmt.Errorf("wake word: inter-op threads: %w", err)
|
||||
}
|
||||
open := func(p string, in, out []string) (*ort.DynamicAdvancedSession, error) {
|
||||
s, err := ort.NewDynamicAdvancedSession(p, in, out, opts)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("wake word: load %s: %w", p, err)
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
m := &wakeModels{}
|
||||
if m.mel, err = open(melPath, []string{"input"}, []string{"output"}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if m.emb, err = open(embedPath, []string{"input_1"}, []string{"conv2d_19"}); err != nil {
|
||||
m.Close()
|
||||
return nil, err
|
||||
}
|
||||
if m.head, err = open(headPath, []string{"embeddings"}, []string{"score"}); err != nil {
|
||||
m.Close()
|
||||
return nil, err
|
||||
}
|
||||
return m, nil
|
||||
}
|
||||
|
||||
// Close releases the three sessions.
|
||||
func (m *wakeModels) Close() {
|
||||
if m == nil {
|
||||
return
|
||||
}
|
||||
for _, s := range []*ort.DynamicAdvancedSession{m.mel, m.emb, m.head} {
|
||||
if s != nil {
|
||||
s.Destroy()
|
||||
}
|
||||
}
|
||||
m.mel, m.emb, m.head = nil, nil, nil
|
||||
}
|
||||
|
||||
// melFrames runs one buffer of samples and returns the mel frames it yielded.
|
||||
func (m *wakeModels) melFrames(buf []float32) ([][melBins]float32, error) {
|
||||
in, err := ort.NewTensor(ort.NewShape(1, int64(len(buf))), buf)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer in.Destroy()
|
||||
|
||||
n := int64(len(buf)/melHop - 3)
|
||||
if n < 1 {
|
||||
return nil, fmt.Errorf("wake word: %d samples yield no mel frames", len(buf))
|
||||
}
|
||||
out, err := ort.NewEmptyTensor[float32](ort.NewShape(1, 1, n, melBins))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer out.Destroy()
|
||||
|
||||
if err := m.mel.Run([]ort.Value{in}, []ort.Value{out}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
data := out.GetData()
|
||||
frames := make([][melBins]float32, n)
|
||||
for i := range frames {
|
||||
for j := 0; j < melBins; j++ {
|
||||
// The scaling openWakeWord applies between the two feature
|
||||
// models, and the head was fitted on its output.
|
||||
frames[i][j] = data[i*melBins+j]/10.0 + 2.0
|
||||
}
|
||||
}
|
||||
return frames, nil
|
||||
}
|
||||
|
||||
// embedding runs embedFrames mel frames through the frozen embedder.
|
||||
func (m *wakeModels) embedding(mels [][melBins]float32) ([embedDim]float32, error) {
|
||||
var e [embedDim]float32
|
||||
flat := make([]float32, 0, embedFrames*melBins)
|
||||
for _, f := range mels {
|
||||
flat = append(flat, f[:]...)
|
||||
}
|
||||
in, err := ort.NewTensor(ort.NewShape(1, embedFrames, melBins, 1), flat)
|
||||
if err != nil {
|
||||
return e, err
|
||||
}
|
||||
defer in.Destroy()
|
||||
out, err := ort.NewEmptyTensor[float32](ort.NewShape(1, 1, 1, embedDim))
|
||||
if err != nil {
|
||||
return e, err
|
||||
}
|
||||
defer out.Destroy()
|
||||
if err := m.emb.Run([]ort.Value{in}, []ort.Value{out}); err != nil {
|
||||
return e, err
|
||||
}
|
||||
copy(e[:], out.GetData())
|
||||
return e, nil
|
||||
}
|
||||
|
||||
// score runs the trained head over headWindow embeddings.
|
||||
func (m *wakeModels) score(embeds [][embedDim]float32) (float64, error) {
|
||||
flat := make([]float32, 0, headWindow*embedDim)
|
||||
for _, e := range embeds {
|
||||
flat = append(flat, e[:]...)
|
||||
}
|
||||
in, err := ort.NewTensor(ort.NewShape(1, headWindow, embedDim), flat)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
defer in.Destroy()
|
||||
out, err := ort.NewEmptyTensor[float32](ort.NewShape(1, 1))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
defer out.Destroy()
|
||||
if err := m.head.Run([]ort.Value{in}, []ort.Value{out}); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return float64(out.GetData()[0]), nil
|
||||
}
|
||||
@@ -0,0 +1,195 @@
|
||||
package main
|
||||
|
||||
// The wake word, "Мэйвен" (V-487 stage two).
|
||||
//
|
||||
// Silero answers "is this frame speech". It does not answer "was this said to
|
||||
// her", and until this file existed nothing did: every utterance near the
|
||||
// microphone became a turn. What made that safe rather than expensive was
|
||||
// SurfaceVoice capping acts at L0, and L0 does not cap reading, so the room
|
||||
// could still hear his facts read back.
|
||||
//
|
||||
// This file owns the 80ms cadence and the three rings of state between the
|
||||
// models. wakefeatures.go owns the tensors.
|
||||
//
|
||||
// Nil is a working value, and it is the CLOSED gate rather than the open one.
|
||||
// Feed on a nil receiver reports no keyword; session.go asks separately
|
||||
// whether a gate exists at all. That split is deliberate: a nil that answers
|
||||
// "yes, keyword" reads as a working wake word in every log line it produces.
|
||||
|
||||
import (
|
||||
"log"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// defaultWakeThreshold — score above which the keyword was said.
|
||||
//
|
||||
// Picked from the false-accept rate on held-out Russian speech, not from
|
||||
// accuracy: a miss costs him a repeat, a false accept costs a turn nobody
|
||||
// asked for. Over 65 minutes of Common Voice, 0.99 woke her three times and
|
||||
// 0.999 once, and the difference in recall was one render out of 126. So the
|
||||
// default is the strict one. `docs/evals/2026-08-09-wake-word.md` has both
|
||||
// tables.
|
||||
const defaultWakeThreshold = 0.999
|
||||
|
||||
// wakeWord is the streaming state around wakeModels. It is fed the same
|
||||
// capture frames the VAD sees and answers whether the keyword has just been
|
||||
// spoken.
|
||||
type wakeWord struct {
|
||||
mu sync.Mutex
|
||||
m *wakeModels
|
||||
|
||||
threshold float64
|
||||
|
||||
// pending holds captured samples not yet part of a full 80ms chunk, and
|
||||
// history holds the melContext samples before them.
|
||||
pending []float32
|
||||
history []float32
|
||||
|
||||
// mels is the newest embedFrames mel frames, oldest first.
|
||||
mels [][melBins]float32
|
||||
// embeds is the newest headWindow embeddings, oldest first.
|
||||
embeds [][embedDim]float32
|
||||
|
||||
last float64 // most recent score, held between chunks
|
||||
}
|
||||
|
||||
// newWakeWord loads the models and wraps them in the streaming gate.
|
||||
func newWakeWord(melPath, embedPath, headPath, libPath string, threshold float64) (*wakeWord, error) {
|
||||
m, err := newWakeModels(melPath, embedPath, headPath, libPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if threshold <= 0 {
|
||||
threshold = defaultWakeThreshold
|
||||
}
|
||||
return &wakeWord{m: m, threshold: threshold}, nil
|
||||
}
|
||||
|
||||
// Close releases the models.
|
||||
func (w *wakeWord) Close() {
|
||||
if w == nil {
|
||||
return
|
||||
}
|
||||
w.mu.Lock()
|
||||
defer w.mu.Unlock()
|
||||
w.m.Close()
|
||||
w.m = nil
|
||||
}
|
||||
|
||||
// Feed takes one capture frame and reports whether the keyword was heard on
|
||||
// it. A nil wakeWord hears nothing.
|
||||
func (w *wakeWord) Feed(frame []int16) bool {
|
||||
if w == nil {
|
||||
return false
|
||||
}
|
||||
w.mu.Lock()
|
||||
defer w.mu.Unlock()
|
||||
|
||||
for _, v := range frame {
|
||||
w.pending = append(w.pending, float32(v)/32768.0)
|
||||
}
|
||||
fired := false
|
||||
for len(w.pending) >= chunkSamples {
|
||||
chunk := w.pending[:chunkSamples]
|
||||
if w.step(chunk) {
|
||||
fired = true
|
||||
}
|
||||
w.history = append(w.history[:0], tailFloat32(append(w.history, chunk...), melContext)...)
|
||||
// Slide the remainder to the front rather than reslicing. This runs
|
||||
// every 80ms for as long as the daemon lives.
|
||||
w.pending = append(w.pending[:0], w.pending[chunkSamples:]...)
|
||||
}
|
||||
return fired
|
||||
}
|
||||
|
||||
// Reset drops the streaming state, so a fresh utterance is not judged on audio
|
||||
// from before it. Called after every dispatch and after barge-in, for the same
|
||||
// reason silero is: echo-era history must not score the next sentence, and her
|
||||
// own voice saying the keyword must not wake her.
|
||||
func (w *wakeWord) Reset() {
|
||||
if w == nil {
|
||||
return
|
||||
}
|
||||
w.mu.Lock()
|
||||
defer w.mu.Unlock()
|
||||
w.pending, w.history = w.pending[:0], w.history[:0]
|
||||
w.mels, w.embeds = nil, nil
|
||||
w.last = 0
|
||||
}
|
||||
|
||||
// Score returns the most recent score, for the operator to read out of the
|
||||
// journal when picking a threshold for his room.
|
||||
func (w *wakeWord) Score() float64 {
|
||||
if w == nil {
|
||||
return 0
|
||||
}
|
||||
w.mu.Lock()
|
||||
defer w.mu.Unlock()
|
||||
return w.last
|
||||
}
|
||||
|
||||
// step runs one 80ms chunk through all three models. It returns true when the
|
||||
// score crosses the threshold on this chunk.
|
||||
func (w *wakeWord) step(chunk []float32) bool {
|
||||
buf := make([]float32, 0, melContext+len(chunk))
|
||||
if pad := melContext - len(w.history); pad > 0 {
|
||||
buf = append(buf, make([]float32, pad)...)
|
||||
}
|
||||
buf = append(buf, tailFloat32(w.history, melContext)...)
|
||||
buf = append(buf, chunk...)
|
||||
|
||||
frames, err := w.m.melFrames(buf)
|
||||
if err != nil {
|
||||
// A failed inference must not silence the microphone. Hold the last
|
||||
// score and let the next chunk try again.
|
||||
log.Printf("mavwaked: wake word: mel: %v", err)
|
||||
return false
|
||||
}
|
||||
w.mels = tailMel(append(w.mels, frames...), embedFrames)
|
||||
if len(w.mels) < embedFrames {
|
||||
return false
|
||||
}
|
||||
e, err := w.m.embedding(w.mels)
|
||||
if err != nil {
|
||||
log.Printf("mavwaked: wake word: embedding: %v", err)
|
||||
return false
|
||||
}
|
||||
w.embeds = tailEmbed(append(w.embeds, e), headWindow)
|
||||
if len(w.embeds) < headWindow {
|
||||
return false
|
||||
}
|
||||
score, err := w.m.score(w.embeds)
|
||||
if err != nil {
|
||||
log.Printf("mavwaked: wake word: head: %v", err)
|
||||
return false
|
||||
}
|
||||
// Report the crossing, not the state. A keyword held above the threshold
|
||||
// for a second is one wake, and firing on every chunk of it would make the
|
||||
// gate look open when it is merely slow to fall.
|
||||
crossed := score >= w.threshold && w.last < w.threshold
|
||||
w.last = score
|
||||
return crossed
|
||||
}
|
||||
|
||||
// The three rings. Each keeps the newest n entries and nothing older.
|
||||
|
||||
func tailFloat32(s []float32, n int) []float32 {
|
||||
if len(s) <= n {
|
||||
return s
|
||||
}
|
||||
return s[len(s)-n:]
|
||||
}
|
||||
|
||||
func tailMel(s [][melBins]float32, n int) [][melBins]float32 {
|
||||
if len(s) <= n {
|
||||
return s
|
||||
}
|
||||
return append(s[:0], s[len(s)-n:]...)
|
||||
}
|
||||
|
||||
func tailEmbed(s [][embedDim]float32, n int) [][embedDim]float32 {
|
||||
if len(s) <= n {
|
||||
return s
|
||||
}
|
||||
return append(s[:0], s[len(s)-n:]...)
|
||||
}
|
||||
@@ -0,0 +1,174 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// fakeGate fires on demand instead of running three ONNX models. The gate's
|
||||
// own arithmetic is measured on real audio in docs/evals; what these tests
|
||||
// cover is the thing that decides whether an utterance is shipped.
|
||||
type fakeGate struct {
|
||||
fireOn int // fire when this many frames have been fed, 0 never fires
|
||||
fed int
|
||||
resets int
|
||||
}
|
||||
|
||||
func (g *fakeGate) Feed(_ []int16) bool {
|
||||
g.fed++
|
||||
return g.fireOn > 0 && g.fed == g.fireOn
|
||||
}
|
||||
func (g *fakeGate) Reset() { g.resets++ }
|
||||
func (g *fakeGate) Score() float64 { return 1 }
|
||||
|
||||
// wakingSession wires a session whose gate fires on the first frame it sees.
|
||||
func wakingSession(fireOn int, window time.Duration) (*session, *fakePlayer, *fakeSender, *fakeGate) {
|
||||
sess, p, snd := newTestSession(bargeInConfig{})
|
||||
g := &fakeGate{fireOn: fireOn}
|
||||
sess.UseWakeWord(g, window)
|
||||
return sess, p, snd, g
|
||||
}
|
||||
|
||||
func TestKeywordlessSpeechNeverReachesSTT(t *testing.T) {
|
||||
sess, p, snd, g := wakingSession(0, 8*time.Second)
|
||||
speakThenPause(t, sess)
|
||||
|
||||
if len(snd.sent) != 0 {
|
||||
t.Fatalf("sent %d utterances, want 0 — this is the whole point of V-487", len(snd.sent))
|
||||
}
|
||||
if sess.ignored != 1 {
|
||||
t.Errorf("ignored = %d, want 1", sess.ignored)
|
||||
}
|
||||
if p.plays != 0 {
|
||||
t.Errorf("plays = %d, want 0", p.plays)
|
||||
}
|
||||
if g.fed == 0 {
|
||||
t.Error("the gate was never fed a frame")
|
||||
}
|
||||
}
|
||||
|
||||
func TestKeywordOpensTheGate(t *testing.T) {
|
||||
sess, p, snd, _ := wakingSession(1, 8*time.Second)
|
||||
speakThenPause(t, sess)
|
||||
|
||||
if len(snd.sent) != 1 {
|
||||
t.Fatalf("sent %d utterances, want 1", len(snd.sent))
|
||||
}
|
||||
if sess.wakes != 1 {
|
||||
t.Errorf("wakes = %d, want 1", sess.wakes)
|
||||
}
|
||||
if sess.ignored != 0 {
|
||||
t.Errorf("ignored = %d, want 0", sess.ignored)
|
||||
}
|
||||
if p.plays != 1 {
|
||||
t.Errorf("plays = %d, want 1", p.plays)
|
||||
}
|
||||
}
|
||||
|
||||
// One keyword buys one turn. Without this the microphone stays open for as
|
||||
// long as he keeps talking, which is the state the gate exists to end.
|
||||
func TestOneKeywordBuysOneTurn(t *testing.T) {
|
||||
sess, p, snd, _ := wakingSession(1, 8*time.Second)
|
||||
speakThenPause(t, sess)
|
||||
p.Stop() // she finished her reply
|
||||
sess.discard = 0 // the backlog drain is not what this measures
|
||||
speakThenPause(t, sess)
|
||||
|
||||
if len(snd.sent) != 1 {
|
||||
t.Fatalf("sent %d utterances, want 1: the second had no keyword", len(snd.sent))
|
||||
}
|
||||
if sess.ignored != 1 {
|
||||
t.Errorf("ignored = %d, want 1", sess.ignored)
|
||||
}
|
||||
}
|
||||
|
||||
// The keyword is heard, then he says nothing for longer than the window. What
|
||||
// he says after that is not addressed to her.
|
||||
func TestTheKeywordExpires(t *testing.T) {
|
||||
sess, _, snd, _ := wakingSession(1, 500*time.Millisecond)
|
||||
now := time.Unix(1750000000, 0)
|
||||
sess.now = func() time.Time { return now }
|
||||
|
||||
if err := sess.feed(context.Background(), silentBytes()); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
if sess.wakes != 1 {
|
||||
t.Fatalf("wakes = %d, want 1", sess.wakes)
|
||||
}
|
||||
now = now.Add(2 * time.Second)
|
||||
speakThenPause(t, sess)
|
||||
|
||||
if len(snd.sent) != 0 {
|
||||
t.Fatalf("sent %d utterances, want 0 — the keyword had expired", len(snd.sent))
|
||||
}
|
||||
}
|
||||
|
||||
// Barge-in cuts her off whether or not the keyword was heard. What he says
|
||||
// after cutting her off still has to carry it.
|
||||
func TestBargeInStillInterruptsHer(t *testing.T) {
|
||||
sess, p, _, g := wakingSession(0, 8*time.Second)
|
||||
sess.barge = bargeInConfig{RMS: 0.2, Frames: 3}
|
||||
p.playing = true
|
||||
loud := frameAt(0.35)
|
||||
for i := 0; i < 4; i++ {
|
||||
if err := sess.feed(context.Background(), loud); err != nil {
|
||||
t.Fatalf("feed %d: %v", i, err)
|
||||
}
|
||||
}
|
||||
if sess.bargeIns != 1 {
|
||||
t.Fatalf("bargeIns = %d, want 1", sess.bargeIns)
|
||||
}
|
||||
if p.stops != 1 {
|
||||
t.Errorf("stops = %d, want 1", p.stops)
|
||||
}
|
||||
if g.resets == 0 {
|
||||
t.Error("barge-in left pre-playback audio in the gate")
|
||||
}
|
||||
}
|
||||
|
||||
// Her own reply must not wake her. Frames captured while the player runs never
|
||||
// reach the gate, and the gate is cleared when playback ends.
|
||||
func TestHerOwnVoiceNeverReachesTheGate(t *testing.T) {
|
||||
sess, p, _, g := wakingSession(1, 8*time.Second)
|
||||
p.playing = true
|
||||
for i := 0; i < 10; i++ {
|
||||
if err := sess.feed(context.Background(), frameAt(0.35)); err != nil {
|
||||
t.Fatalf("feed: %v", err)
|
||||
}
|
||||
}
|
||||
if g.fed != 0 {
|
||||
t.Fatalf("gate was fed %d frames while she was speaking, want 0", g.fed)
|
||||
}
|
||||
if sess.wakes != 0 {
|
||||
t.Errorf("wakes = %d, want 0", sess.wakes)
|
||||
}
|
||||
}
|
||||
|
||||
// No model, no gate: the daemon behaves exactly as it did before V-487 stage
|
||||
// two. An operator with a missing file gets yesterday's mavwaked, not one that
|
||||
// refuses to hear anything.
|
||||
func TestNoGateShipsEveryUtterance(t *testing.T) {
|
||||
sess, _, snd := newTestSession(bargeInConfig{})
|
||||
speakThenPause(t, sess)
|
||||
|
||||
if len(snd.sent) != 1 {
|
||||
t.Fatalf("sent %d utterances, want 1", len(snd.sent))
|
||||
}
|
||||
if sess.ignored != 0 {
|
||||
t.Errorf("ignored = %d, want 0", sess.ignored)
|
||||
}
|
||||
}
|
||||
|
||||
// A nil *wakeWord is the closed gate, not a crash and not an open one.
|
||||
func TestNilWakeWordHearsNothing(t *testing.T) {
|
||||
var w *wakeWord
|
||||
if w.Feed([]int16{0, 0, 0}) {
|
||||
t.Error("a nil wake word reported the keyword")
|
||||
}
|
||||
if w.Score() != 0 {
|
||||
t.Error("a nil wake word reported a score")
|
||||
}
|
||||
w.Reset()
|
||||
w.Close()
|
||||
}
|
||||
@@ -0,0 +1,156 @@
|
||||
"""CrisperWhisper 2.0 turbo as an HTTP service, for Maven's stt.Pair.
|
||||
|
||||
Two endpoints and no framework.
|
||||
|
||||
GET /health 200 once the model is loaded, 503 while it is loading.
|
||||
POST /transcribe raw 16kHz mono PCM in, {"text","confidence"} out.
|
||||
|
||||
The body is the PCM itself rather than JSON. A minute of 16kHz mono is under
|
||||
2MB raw and about 2.6MB base64, and the format is fixed at the Maven seam, so
|
||||
headers carry it more cheaply than an envelope.
|
||||
|
||||
Why this exists at all: whisper.cpp cannot load CW2. It derives its language
|
||||
count from the vocabulary size, and CW2's 51897 tokens shift seven special
|
||||
token ids. So mavsttd stays whisper.cpp on homesrv and this runs beside the
|
||||
model on workpc, where it scores 10.4% WER in Russian against the floor's 27.5%
|
||||
(docs/evals/2026-08-09-crisperwhisper2-russian-wer.md in the Maven repo).
|
||||
|
||||
Intended mode, not verbatim. The owner asked for what he meant to say, not
|
||||
every stutter on the way there.
|
||||
"""
|
||||
|
||||
import hmac
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
|
||||
import numpy as np
|
||||
|
||||
HOST = os.environ.get("CW2_HOST", "0.0.0.0")
|
||||
PORT = int(os.environ.get("CW2_PORT", "8081"))
|
||||
SIZE = os.environ.get("CW2_SIZE", "turbo")
|
||||
MODE = os.environ.get("CW2_MODE", "intended")
|
||||
TOKEN = os.environ.get("CW2_TOKEN", "")
|
||||
# 25MB is about thirteen minutes of 16kHz mono. Longer than any utterance and
|
||||
# short enough that a wrong caller cannot exhaust memory.
|
||||
MAX_BODY = int(os.environ.get("CW2_MAX_BODY", str(25 * 1024 * 1024)))
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO, format="%(asctime)s cw2: %(message)s", stream=sys.stderr
|
||||
)
|
||||
log = logging.getLogger("cw2")
|
||||
|
||||
_model = None
|
||||
# The card holds one model and transcribes one utterance at a time. The lock is
|
||||
# what makes a second caller wait rather than corrupt the first.
|
||||
_lock = threading.Lock()
|
||||
|
||||
|
||||
def load_model():
|
||||
global _model
|
||||
from crisperwhisper import CrisperWhisperModel
|
||||
|
||||
t0 = time.perf_counter()
|
||||
# backend is forced. With ctranslate2 importable, "auto" picks ct2, which is
|
||||
# CUDA-only and this card is AMD.
|
||||
m = CrisperWhisperModel(
|
||||
SIZE, backend="transformers", compute_type="float16", device="cuda"
|
||||
)
|
||||
_model = m
|
||||
log.info("loaded %s in %.1fs, mode=%s", SIZE, time.perf_counter() - t0, MODE)
|
||||
|
||||
|
||||
def authorised(headers):
|
||||
if not TOKEN:
|
||||
return True
|
||||
got = headers.get("Authorization", "")
|
||||
return hmac.compare_digest(got, "Bearer " + TOKEN)
|
||||
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
protocol_version = "HTTP/1.1"
|
||||
|
||||
def log_message(self, fmt, *args):
|
||||
log.info(fmt, *args)
|
||||
|
||||
def _send(self, code, payload):
|
||||
body = json.dumps(payload, ensure_ascii=False).encode("utf-8")
|
||||
self.send_response(code)
|
||||
self.send_header("Content-Type", "application/json; charset=utf-8")
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def do_GET(self):
|
||||
if self.path.rstrip("/") != "/health":
|
||||
self._send(404, {"error": "not found"})
|
||||
return
|
||||
if _model is None:
|
||||
self._send(503, {"status": "loading"})
|
||||
return
|
||||
self._send(200, {"status": "ok", "model": SIZE, "mode": MODE})
|
||||
|
||||
def do_POST(self):
|
||||
if self.path.rstrip("/") != "/transcribe":
|
||||
self._send(404, {"error": "not found"})
|
||||
return
|
||||
if not authorised(self.headers):
|
||||
self._send(401, {"error": "unauthorised"})
|
||||
return
|
||||
if _model is None:
|
||||
self._send(503, {"error": "loading"})
|
||||
return
|
||||
|
||||
length = int(self.headers.get("Content-Length", "0"))
|
||||
if length <= 0 or length > MAX_BODY:
|
||||
self._send(413, {"error": "bad body length"})
|
||||
return
|
||||
raw = self.rfile.read(length)
|
||||
|
||||
rate = int(self.headers.get("X-Sample-Rate", "16000"))
|
||||
channels = int(self.headers.get("X-Channels", "1"))
|
||||
bits = int(self.headers.get("X-Sample-Bits", "16"))
|
||||
lang = self.headers.get("X-Language", "ru") or "ru"
|
||||
if channels != 1 or bits != 16:
|
||||
self._send(400, {"error": "want 16-bit mono pcm"})
|
||||
return
|
||||
|
||||
# int16 little-endian to the float32 the encoder wants.
|
||||
wav = np.frombuffer(raw, dtype="<i2").astype(np.float32) / 32768.0
|
||||
if wav.size == 0:
|
||||
self._send(200, {"text": "", "confidence": 0.0})
|
||||
return
|
||||
|
||||
t0 = time.perf_counter()
|
||||
try:
|
||||
with _lock:
|
||||
res = _model.transcribe(wav, sr=rate, language=lang, mode=MODE)
|
||||
except Exception as exc: # noqa: BLE001 - the caller falls back to mavsttd
|
||||
log.exception("transcribe failed")
|
||||
self._send(500, {"error": str(exc)})
|
||||
return
|
||||
elapsed = time.perf_counter() - t0
|
||||
text = (res.text or "").strip()
|
||||
log.info("%.2fs audio in %.2fs: %r", wav.size / rate, elapsed, text[:60])
|
||||
# The model reports no calibrated score. 1.0 would be a claim, and the
|
||||
# Maven side reads confidence only to log it.
|
||||
self._send(200, {"text": text, "confidence": 0.0})
|
||||
|
||||
|
||||
def main():
|
||||
if not TOKEN:
|
||||
log.warning("no CW2_TOKEN set: anything on the LAN can post audio here")
|
||||
# Bind before loading, so a restart answers 503 rather than refusing the
|
||||
# connection. Both make Maven fall back, but only one of them says why.
|
||||
srv = ThreadingHTTPServer((HOST, PORT), Handler)
|
||||
threading.Thread(target=load_model, daemon=True).start()
|
||||
log.info("listening on %s:%d", HOST, PORT)
|
||||
srv.serve_forever()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,40 @@
|
||||
# maven-voice-tunnel — the ssh leg that carries the voice wire to homesrv.
|
||||
#
|
||||
# Runs on workpc, as a user unit (`systemctl --user`), beside mavgpud.service.
|
||||
#
|
||||
# WHY THIS EXISTS AT ALL. internal/voice is plaintext and unauthenticated.
|
||||
# Its own server doc says production binds inside the wg tunnel, because "the
|
||||
# wg layer IS the L0 floor". workpc is not a wg peer, it sits on wlan0. So ssh
|
||||
# is the substitute floor: it authenticates with his key and encrypts the leg,
|
||||
# and mavend's published port stays on homesrv loopback (127.0.0.1:9110).
|
||||
# Nothing about this puts a Maven port on the LAN.
|
||||
#
|
||||
# Do not replace this with a LAN bind. SurfaceVoice caps acts at L0, so an
|
||||
# unauthorized speaker could not run a destructive tool. It would still hear
|
||||
# his facts, his notes and his calendar read back, and L0 does not cap reading.
|
||||
#
|
||||
# install: cp to ~/.config/systemd/user/ on workpc
|
||||
# systemctl --user enable --now maven-voice-tunnel.service
|
||||
|
||||
[Unit]
|
||||
Description=SSH tunnel to mavend's voice wire on homesrv
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
# -N: no remote command, forwarding only.
|
||||
# ExitOnForwardFailure: fail loudly rather than sit up with a dead forward,
|
||||
# which is what makes Restart meaningful.
|
||||
# ServerAlive*: a laptop that suspends drops the tunnel silently otherwise.
|
||||
ExecStart=/usr/bin/ssh -N \
|
||||
-o ExitOnForwardFailure=yes \
|
||||
-o ServerAliveInterval=30 \
|
||||
-o ServerAliveCountMax=3 \
|
||||
-o BatchMode=yes \
|
||||
-L 127.0.0.1:9100:127.0.0.1:9110 \
|
||||
kami@192.168.1.104
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
+23
-2
@@ -78,10 +78,30 @@
|
||||
"Addressed by LAN address, not container name: mavgpud runs on another",
|
||||
"machine and there is no shared docker network to name it on."
|
||||
],
|
||||
"//workstation.stt": [
|
||||
"CrisperWhisper 2.0 turbo on the same machine, a second service on port",
|
||||
"8081 and not a second endpoint on mavgpud. whisper.cpp cannot load CW2 at",
|
||||
"all: it derives its language count from the vocabulary size, and CW2's",
|
||||
"51897 tokens shift seven special token ids. So it runs under transformers",
|
||||
"there and mavsttd stays whisper.cpp here.",
|
||||
"Worth the second service: CW2 turbo scores 10.4% WER in Russian against",
|
||||
"27.5% for the ggml-small.bin mavsttd loads, measured on 200 Golos clips",
|
||||
"in docs/evals/2026-08-09-crisperwhisper2-russian-wer.md.",
|
||||
"Deleting this block sends every utterance to mavsttd, which is what the",
|
||||
"box did before it existed. A worse transcript is still a turn, so the",
|
||||
"fallback is silent and Kami is never told which machine heard him.",
|
||||
"The token is what stops anything on the LAN posting audio to that port."
|
||||
],
|
||||
"workstation": {
|
||||
"url": "http://192.168.1.105:8080",
|
||||
"probe": "15s",
|
||||
"timeout": "90s"
|
||||
"timeout": "90s",
|
||||
"stt": {
|
||||
"url": "http://192.168.1.105:8081/transcribe",
|
||||
"token": "${MAVEN_STT_TOKEN}",
|
||||
"probe": "15s",
|
||||
"timeout": "10s"
|
||||
}
|
||||
},
|
||||
|
||||
"//search": [
|
||||
@@ -231,7 +251,8 @@
|
||||
"embedder": {
|
||||
"model_path": "/opt/maven/models/embedder/multilingual-e5-small/model_quantized.onnx",
|
||||
"tokenizer_path": "/opt/maven/models/embedder/multilingual-e5-small/tokenizer.json",
|
||||
"lib_path": "/opt/maven/lib/libonnxruntime.so"
|
||||
"lib_path": "/opt/maven/lib/libonnxruntime.so",
|
||||
"heads_path": "/opt/maven/models/embedder/router-heads/router_heads.onnx"
|
||||
},
|
||||
"llm_router": true,
|
||||
"query_min_score": 0.55,
|
||||
|
||||
+21
-5
@@ -2,9 +2,14 @@
|
||||
"listen": ":8080",
|
||||
"llama_addr": "127.0.0.1:10000",
|
||||
"llama_bin": "llama-server",
|
||||
"//llama_args": [
|
||||
"E4B carries no MTP tensors, so the speculative flags are gone with the 12B.",
|
||||
"MTP on this box is a separate gguf of architecture gemma4-assistant with",
|
||||
"nextn_predict_layers=4, and mtp-gemma-4-12B-it-BF16 is the only one there is.",
|
||||
"Its head is trained against the 12B's hidden states, so it cannot drive E4B."
|
||||
],
|
||||
"llama_args": [
|
||||
"-m", "/mnt/D/AI/gemma4/gemma-4-12B-it-qat-UD-Q4_K_XL.gguf",
|
||||
"-md", "/mnt/D/AI/gemma4/mtp-gemma-4-12B-it-BF16.gguf",
|
||||
"-m", "/mnt/D/AI/gemma4/gemma-4-E4B-it-qat-UD-Q4_K_XL.gguf",
|
||||
"-ngl", "99",
|
||||
"-fa", "on",
|
||||
"-np", "1",
|
||||
@@ -15,11 +20,22 @@
|
||||
"--batch-size", "2048",
|
||||
"--ubatch-size", "512",
|
||||
"--jinja",
|
||||
"--chat-template-kwargs", "{\"enable_thinking\":false}",
|
||||
"--spec-type", "draft-mtp",
|
||||
"--spec-draft-n-max", "2"
|
||||
"--chat-template-kwargs", "{\"enable_thinking\":false}"
|
||||
],
|
||||
|
||||
"//stt": [
|
||||
"CrisperWhisper 2.0 turbo, which Maven reaches directly on port 8081.",
|
||||
"mavgpud runs it because it is a ROCm process on this card: under its own",
|
||||
"systemd unit it registered on the KFD and the supervisor evicted",
|
||||
"llama-server every few seconds. CW2_TOKEN comes from the unit's",
|
||||
"EnvironmentFile and is never a flag value."
|
||||
],
|
||||
"stt": {
|
||||
"addr": "127.0.0.1:8081",
|
||||
"bin": "/home/kami/Programs/cw2-eval/.venv/bin/python",
|
||||
"args": ["/home/kami/Programs/cw2-service/serve.py"]
|
||||
},
|
||||
|
||||
"kfd_root": "/sys/class/kfd/kfd/proc",
|
||||
"drm_device": "/sys/class/drm/card1/device",
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
[Unit]
|
||||
# Runs on the workstation (bugmachine), not on homesrv. Install as a systemd
|
||||
# Runs on the workstation (workpc), not on homesrv. Install as a systemd
|
||||
# user unit and turn on lingering, so the card is supervised after a reboot
|
||||
# with nobody logged in:
|
||||
#
|
||||
@@ -8,10 +8,15 @@
|
||||
# scp deploy/mavgpud.service workpc:~/.config/systemd/user/mavgpud.service
|
||||
# ssh workpc 'systemctl --user daemon-reload && systemctl --user enable --now mavgpud'
|
||||
# sudo loginctl enable-linger kami
|
||||
Description=Maven GPU supervisor (holds llama-server while the card is free)
|
||||
Description=Maven GPU supervisor (holds llama-server and CW2 while the card is free)
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
# CW2_TOKEN for the transcriber child, which inherits this environment. The
|
||||
# token is read from a file and never appears as a flag value, the rule
|
||||
# mavpoll and mavmaild follow. Missing file, no transcriber auth, so keep the
|
||||
# dash off: a mavgpud that cannot read it must fail loudly.
|
||||
EnvironmentFile=%h/Programs/cw2-service/cw2.env
|
||||
ExecStart=%h/.local/bin/mavgpud -config %h/.config/mavgpud.json
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
# mavwaked — always-on listening, on workpc where the microphone is.
|
||||
#
|
||||
# User unit, beside mavgpud.service and maven-voice-tunnel.service. It is a
|
||||
# user unit because it needs his ALSA session and his ssh agent, and because
|
||||
# it should stop when he logs out.
|
||||
#
|
||||
# The keyword is "Мэйвен" and the three -wake- flags are what require it
|
||||
# (V-487 stage two). Without them anything spoken near the fifine becomes a
|
||||
# turn, which voiceSender makes safe rather than expensive: it sends
|
||||
# Surface=SurfaceVoice, capping every command at L0. That does not stop her
|
||||
# answering out loud, which is the whole reason the keyword exists.
|
||||
#
|
||||
# The threshold is 0.999 and it is the binary's default, so it is not passed.
|
||||
# It came from 65 minutes of held-out Russian speech through this same binary:
|
||||
# 0.9 false wakes an hour against 2.8 at 0.99, for one lost render out of 126
|
||||
# (docs/evals/2026-08-09-wake-word.md). If the room proves noisier than the
|
||||
# corpus, read the scores out of this unit's journal and pass -wake-threshold.
|
||||
# Do not lower it by guessing.
|
||||
#
|
||||
# A keyword shorter than 1.32s can be heard too late to be used, because the
|
||||
# head scores 1.28s of audio and the VAD has closed the utterance by then.
|
||||
# "Мэйвен, <request>" is unaffected. A bare "Мэйвен" is the case that fails.
|
||||
#
|
||||
# -vad-model is passed on purpose. Silero answers "is this frame speech" where
|
||||
# the energy floor answers "is this frame loud". It declines white noise at
|
||||
# the same RMS 0 frames to 68-99, and still hears all four spoken fixtures
|
||||
# (docs/evals/2026-08-09-silero-vad.md). It costs 509us a frame, 1.7% of one
|
||||
# core, and never touches the GPU. Drop the flag and the energy floor is back.
|
||||
#
|
||||
# -barge-in is NOT passed. The threshold is room-specific and this room has no
|
||||
# number yet. Turn it on only after reading the "suppressed while speaking"
|
||||
# means out of this unit's own journal, never by guessing.
|
||||
#
|
||||
# install: cp to ~/.config/systemd/user/ on workpc
|
||||
# systemctl --user enable --now mavwaked.service
|
||||
|
||||
[Unit]
|
||||
Description=Maven always-on listening (silero VAD, "Мэйвен" keyword)
|
||||
# The tunnel is the only path to mavend and the only thing authenticating it.
|
||||
Requires=maven-voice-tunnel.service
|
||||
After=maven-voice-tunnel.service
|
||||
|
||||
[Service]
|
||||
# The Scarlett Solo 4th Gen, and not the fifine. The fifine was the device
|
||||
# here for three days and mavwaked never logged one utterance in them, because
|
||||
# it returns RMS 0.00004 with its capture switch on and its ALSA volume at the
|
||||
# full 496 of 496. That silence is in the hardware, so no flag reaches it.
|
||||
#
|
||||
# Named CARD=Gen and not card 4, because a USB card number moves when
|
||||
# something else is replugged and this daemon must not change ears quietly.
|
||||
# Not "default" either: that follows whatever pipewire last decided.
|
||||
#
|
||||
# plughw and not hw. mavwaked asks arecord for 16kHz mono, which is what the
|
||||
# whole pipeline is canonical in. Neither microphone offers it, so bare hw
|
||||
# dies on "Channels count non available" before a frame is read. plughw puts
|
||||
# ALSA's downmix and resampler in front. Any replacement wants the same.
|
||||
#
|
||||
# The Scarlett measured RMS 0.003 against 0.14 on the onboard input, so its
|
||||
# front-panel gain is the thing to raise if she mishears. That is a knob, not
|
||||
# a control ALSA exposes. The two loud devices, the onboard ALC897 and the
|
||||
# camera, both clip at peak 1.0 and are worse candidates, not better ones.
|
||||
Environment=LD_LIBRARY_PATH=%h/.local/lib
|
||||
ExecStart=%h/.local/bin/mavwaked \
|
||||
-device plughw:CARD=Gen,DEV=0 \
|
||||
-addr 127.0.0.1:9100 \
|
||||
-lang ru \
|
||||
-vad-model %h/.local/share/maven/models/silero_vad.onnx \
|
||||
-wake-model %h/.local/share/maven/models/maven_wakeword.onnx \
|
||||
-wake-mel %h/.local/share/maven/models/melspectrogram.onnx \
|
||||
-wake-embed %h/.local/share/maven/models/embedding_model.onnx \
|
||||
-onnx-lib %h/.local/lib/libonnxruntime.so
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -53,6 +53,19 @@ services:
|
||||
# the decrypted working copy lives in RAM (see db_tmpfs in mavend.json).
|
||||
tmpfs:
|
||||
- /dev/shm
|
||||
# the voice wire, for mavwaked and mavenclient on workpc (V-515).
|
||||
#
|
||||
# LOOPBACK ONLY, and that is the whole security argument. internal/voice
|
||||
# is plaintext with no auth: its own server doc says production binds
|
||||
# inside the wg tunnel, "the wg layer IS the L0 floor". workpc is not a wg
|
||||
# peer, it is on wlan0. So the tunnel is ssh instead, terminated on this
|
||||
# loopback address, and nothing new is on the LAN. Anyone who could reach
|
||||
# a LAN-bound port here could push audio and hear his facts read back.
|
||||
# SurfaceVoice caps acts at L0; it does not cap reading.
|
||||
#
|
||||
# Host 9100 is Vikunja's MCP, hence 9110. The container side stays 9100
|
||||
# so mavweb keeps reaching mavend:9100 by name.
|
||||
ports: ["127.0.0.1:9110:9100"]
|
||||
|
||||
mavsttd:
|
||||
<<: *image
|
||||
|
||||
@@ -0,0 +1,220 @@
|
||||
# Deployment: the boxes, the models, the daemons
|
||||
|
||||
*Last verified: 2026-08-09 @ a9b480a*
|
||||
|
||||
What runs where, and why each choice was made. `CLAUDE.md` carries only the
|
||||
rules. This file carries the reasoning.
|
||||
|
||||
## The two boxes
|
||||
|
||||
**homesrv** is a Ryzen 5 5600U laptop and the deploy target. It offloads to the
|
||||
Vega iGPU over Vulkan (`n_gpu_layers: 99`). Compose passes `/dev/dri` and the
|
||||
render gid (993), and without both Vulkan enumerates zero devices and
|
||||
llama-server falls back to CPU silently.
|
||||
|
||||
**workpc** is the workstation, 16GB of VRAM, reached as `kami@workpc` at
|
||||
192.168.1.105. Model work moved there on 2026-08-02 by the owner's call, because
|
||||
homesrv cannot grow a GPU.
|
||||
|
||||
Three rules govern the seam:
|
||||
|
||||
- **The workstation is never assumed up.**
|
||||
- **Fall back silently** when it would only do the job better.
|
||||
- **Name the gap** when the resident model cannot do the job at all.
|
||||
|
||||
A world question goes through `LLMPhraser.PhraseWorld` and returns `worldGap`
|
||||
(`cmd/mavend/worldmodel.go`) rather than an invented answer. A box with no
|
||||
`workstation` block behaves exactly as it did before the seam. `docs/offload.md`
|
||||
says which caller is which.
|
||||
|
||||
## The resident model
|
||||
|
||||
**Qwen3-1.7B** (`UD-Q4_K_XL`), stock, not yet the CPT'd one. It is a Thinking
|
||||
variant, so `n_ctx` is 4096. Reasoning tokens need the room, and 4096 is what
|
||||
every score was measured at.
|
||||
|
||||
The target is the locally CPT'd Qwen3-1.7B (V-122, training in flight). Stock
|
||||
already speaks good Russian. What it gets wrong is the persona. It writes `я рад`
|
||||
where Maven needs `рада`.
|
||||
|
||||
**Do not bother with sub-500M models.** LFM2.5-230M and 350M were measured on
|
||||
2026-07-31 and both are unusable in Russian
|
||||
(`docs/evals/2026-07-31-model-bakeoff.md`). Their published IFEval and BFCL
|
||||
numbers are English-only.
|
||||
|
||||
Model files live in `/mnt/hdd1/llms`, bind-mounted to `/opt/maven/models/llm`.
|
||||
That **shadows** the repo's `models/llm/`, so a gguf sitting there is not loaded
|
||||
by anything. Swapping the resident model is a one-line change to
|
||||
`phraser.model_path` in `deploy/mavend.json`.
|
||||
|
||||
The workstation model is gemma-4-E4B as of 2026-08-09, replacing the 12B by the
|
||||
owner's call. Keep the 12B gguf. It is the better teacher for label runs, at
|
||||
72.7% destination against E4B's 57.6%.
|
||||
|
||||
## The embedder
|
||||
|
||||
**It stays on homesrv permanently**, because it backs the floor. It is
|
||||
multilingual-e5-small, quantized and asymmetric. `EmbedQuery` and `EmbedPassage`
|
||||
apply the `query:` and `passage:` prefixes it was trained with. Calling plain
|
||||
`Embed` on a note is a bug. See `docs/evals/2026-08-04-recall-e5-small.md`.
|
||||
|
||||
The vendored onnxruntime under `deps/` has two copies, and the stale one is
|
||||
1.17.1. The live runtime is 1.26.0, and the Go binding asks for API 26. Anything
|
||||
shipped to another box needs `deps/onnxruntime-linux-x64-1.26.0`.
|
||||
|
||||
## Speech-to-text
|
||||
|
||||
`sttSeam` in `cmd/mavend/voicewire.go` builds an `stt.Pair` beside `modelSeam`.
|
||||
It prefers CrisperWhisper 2.0 turbo on workpc with mavsttd as the floor. It takes
|
||||
only the silent half of the rule, because a worse transcript is still a turn. So
|
||||
`stt.Pair` has no `TranscribeRemote` and the fallback is never spoken.
|
||||
|
||||
CW2 turbo scores 10.4% WER in Russian against 27.5% for the `ggml-small.bin`
|
||||
mavsttd loads, over 200 Golos clips
|
||||
(`docs/evals/2026-08-09-crisperwhisper2-russian-wer.md`).
|
||||
|
||||
**whisper.cpp cannot load CW2 at all.** It reads its language count off the
|
||||
vocabulary size. CW2's 51897 tokens shift seven special token ids. So CW2 is its
|
||||
own transformers service on port 8081 (`deploy/cw2/serve.py`).
|
||||
`stt.HTTPTranscriber` posts raw PCM to it with a bearer token, because audio is
|
||||
the most sensitive thing that crosses this seam. The switch is `workstation.stt`
|
||||
in `deploy/mavend.json`, and deleting the block sends every utterance to mavsttd.
|
||||
|
||||
**mavgpud runs that service as a second child.** This is not an optimisation.
|
||||
CW2 is a ROCm process on the same card, so it registers on the KFD like any
|
||||
contender. Under its own systemd unit it made mavgpud evict llama-server every
|
||||
few seconds. That took the model arm down for eight minutes on 2026-08-09. The
|
||||
card needs one owner. CW2 is on the yield clock and not the idle one. At 1.6GB
|
||||
it denies the card to nobody.
|
||||
|
||||
Text-to-speech has not moved. piper on homesrv is the only synthesizer.
|
||||
|
||||
## The daemons
|
||||
|
||||
| Binary | Role |
|
||||
|---|---|
|
||||
| `mavend` | **Core.** Router, phraser, memory, reminders, digestion tick. Owns the DB and IPC socket. |
|
||||
| `mavweb` | HTTP UI and PWA (`/dash`, `/history`, `/trace`, `/notifications`, `/tools`), WebAuthn auth. |
|
||||
| `mavsttd` | Speech-to-text (whisper.cpp, CGO). |
|
||||
| `mavttsd` | Text-to-speech (piper subprocess). |
|
||||
| `mavwaked` | Wake-word and VAD gate. Runs on workpc. |
|
||||
| `mavenclient` | Voice loop client (mic, stt, core, tts). Not deployed. |
|
||||
| `mavpoll` | Environment poller: netdata alarms, uptime-kuma, zenmoney, wireguard presence. Writes facts, sends nothing. Telegram is `internal/delivery/telegramsink`. |
|
||||
| `mavcaldav` | CalDAV calendar sync. |
|
||||
| `mavmaild` | Mail reader (IMAP, read-only). Holds the IMAP password, core never sees it. |
|
||||
| `mavgpud` | GPU supervisor. **Runs on workpc**, own unit `deploy/mavgpud.service`. Keeps llama-server loaded while the card is free (V-488). Maven never asks it for anything and reads `/health` through `llm.Pair`. |
|
||||
| `mavupdate` | Not a daemon. Operator CLI a human runs on the box to deploy a new build. |
|
||||
|
||||
Two binaries have no Makefile target and neither is deployed. `mavseal` encrypts
|
||||
a live tmpfs working copy back to the ciphertext file when mavend was killed
|
||||
before `defer st.Close()` sealed it. `labelgen` runs the stage 0 grammars over
|
||||
utterances and prints JSONL, the training data for the routing heads.
|
||||
|
||||
Daemons are wired socket-to-socket, not linked. `internal/ipc` is the wire
|
||||
protocol. `deploy/mavend.json` sets socket paths, model paths and the phraser and
|
||||
embedder blocks, with `${VAR}` expansion from gitignored `deploy/telegram.env`.
|
||||
|
||||
### Who is in compose, and who is not
|
||||
|
||||
**`docker-compose.yml` runs five**: `mavend`, `mavsttd`, `mavttsd`, `mavweb`,
|
||||
`mavpoll`. Count against compose, not against the table above.
|
||||
|
||||
`mavmaild` and `mavcaldav` are commented out, each with the reason beside it. The
|
||||
first needs a mail account and the second a CalDAV account, and this box has
|
||||
neither. Two things ride on the CalDAV absence (V-644). Agenda questions route to
|
||||
`IntentQuery` at stage 0, and the `calendar` query source then reads a table
|
||||
nobody writes. And `loop.State.CalendarBusy` is fed by the same facts, so the
|
||||
gate's "do not nag mid-meeting" is permanently false.
|
||||
|
||||
`mavenclient` is still absent. `mavwaked` moved to workpc on 2026-08-09 (V-515).
|
||||
|
||||
### The voice wire
|
||||
|
||||
`internal/voice` is plaintext with no auth. Its own server doc says production
|
||||
binds inside the wg tunnel, because the wg layer is the L0 floor. workpc is not
|
||||
a wg peer, it sits on wlan0. So the tunnel is ssh instead.
|
||||
|
||||
mavend publishes the voice port to homesrv loopback only, `127.0.0.1:9110`.
|
||||
Host 9100 is Vikunja's MCP, hence 9110. The container side stays 9100 so mavweb
|
||||
keeps reaching `mavend:9100` by name. `deploy/maven-voice-tunnel.service` on
|
||||
workpc forwards it over his key.
|
||||
|
||||
**Do not replace this with a LAN bind.** `SurfaceVoice` caps acts at L0, so an
|
||||
unauthorized speaker could not run a destructive tool. L0 does not cap reading,
|
||||
so they would still hear his facts, notes and calendar read back.
|
||||
|
||||
Both `mavwaked` and `mavenclient` speak `voice.Dial`, not `ipc.Dial`. The
|
||||
`netaddr` token guards the daemon-to-daemon IPC seam and never touches this one.
|
||||
`ipc.Dial` does take `tcp://host:port?token=...`, which is why V-515 was filed
|
||||
as a config change. That premise was wrong, and the ssh leg is the correction.
|
||||
|
||||
The voice loop belongs on a client machine where the owner is standing, and that
|
||||
machine is workpc (V-463, `docs/plans/17-where-the-voice-loop-runs.md`). homesrv
|
||||
has a microphone, because it is a laptop, but it is in the wrong room.
|
||||
|
||||
### mavwaked on workpc
|
||||
|
||||
`deploy/mavwaked.service`, a user unit beside `mavgpud.service`. Two flags are
|
||||
deliberate.
|
||||
|
||||
`-vad-model` is passed. Silero answers "is this frame speech" where the energy
|
||||
floor answers "is this frame loud". It declines white noise at the same RMS, 0
|
||||
frames against 68 to 99, and still hears all four spoken fixtures
|
||||
(`docs/evals/2026-08-09-silero-vad.md`). It costs 509µs a frame and never touches
|
||||
the GPU. A model that will not load is logged and not fatal.
|
||||
|
||||
`-barge-in` is not passed. The threshold is room-specific and this room has no
|
||||
number yet. Read the "suppressed while speaking" means out of the journal first.
|
||||
|
||||
The device is `plughw:CARD=Gen,DEV=0`, the Scarlett Solo. It is `plughw` and
|
||||
not `hw` because mavwaked asks arecord for 16kHz mono. No microphone here
|
||||
offers that, so bare `hw` dies on "Channels count non available" before a
|
||||
frame is read. It is named `CARD=Gen` and not card 4 because a USB card number
|
||||
moves when something else is replugged.
|
||||
|
||||
It used to be the fifine on card 0, and that cost three days. mavwaked logged
|
||||
zero completed utterances across them, before the wake word existed and after.
|
||||
The fifine returns RMS 0.00004 with its capture switch on and its ALSA volume
|
||||
at the full 496 of 496. That silence is in the hardware and no flag reaches
|
||||
it. Over the same eight seconds of speech the onboard ALC897 read 0.142 and
|
||||
the camera 0.289, both clipping at peak 1.0. The Scarlett read 0.003 clean.
|
||||
|
||||
Check the level before blaming the gate. Stop the unit, run `arecord` against
|
||||
the device for five seconds, and measure. A live room floor reads near 0.001.
|
||||
|
||||
The three `-wake-` flags require the keyword "Мэйвен" (V-487 stage two). Drop
|
||||
them and the loop runs open, which is what it did before. The threshold is the
|
||||
binary's default of 0.999 and is not passed. Over 65 minutes of held-out
|
||||
Russian speech it woke her 0.9 times an hour against 2.8 at 0.99. That cost one
|
||||
lost render out of 126 (`docs/evals/2026-08-09-wake-word.md`).
|
||||
|
||||
The three sessions are pinned to one thread each. onnxruntime otherwise sizes
|
||||
its pool to every core and spins between runs, which took mavwaked from 68% of
|
||||
one core to 335%. With the cap it sits at 81%, so the gate costs about 13%.
|
||||
|
||||
A keyword shorter than 1.32s can be heard too late to be used. The head scores
|
||||
1.28s of audio, and the VAD has closed the utterance by then.
|
||||
"Мэйвен, <request>" is unaffected. A bare "Мэйвен" is the case that fails.
|
||||
|
||||
mavwaked connects at startup and holds the conn, so a nudge routed to voice
|
||||
reaches the speaker before he has said anything (V-671). It used to connect
|
||||
lazily, which made the failure silent rather than absent: after one utterance
|
||||
the session existed, `PushToMostRecent` succeeded, the dispatcher stopped
|
||||
rerouting to telegram and ntfy, and mavwaked discarded the audio.
|
||||
|
||||
**Passwords are read from files, never taken as flag values.** `mavcaldav` uses
|
||||
`-pass-file` and `-render-pass-file`. `mavpoll` and `mavmaild` follow the same
|
||||
rule.
|
||||
|
||||
## Web UI conventions
|
||||
|
||||
Server-rendered pages share `cmd/mavweb/static/ui.css` (served at `/ui.css`) and
|
||||
the shell partial in `cmd/mavweb/shell.html`. A page opens with
|
||||
`{{template "shellTop" "<page-key>"}}` and closes with `{{template "shellBottom"}}`,
|
||||
and the key marks the active sidebar link.
|
||||
|
||||
Every page is its own embedded `.html` file next to `main.go`. No page markup
|
||||
lives in Go, and the sidebar is data (`sidebarSections`, `pageIcon`) the template
|
||||
renders. No per-page `<style>` beyond true one-offs. Wrap every table in
|
||||
`<div class=scroll>` so wide data pans on a phone. Local preview and headless
|
||||
screenshot recipes are in `AGENTS.md`.
|
||||
@@ -308,6 +308,29 @@ The count is of CONSECUTIVE step-asides. It resets the moment he answers, in
|
||||
too. "Позвонить маме" against a question about the time is still him in the
|
||||
exchange. The retry it costs is bound enough on its own.
|
||||
|
||||
#### And it may ride four turns in all
|
||||
|
||||
Decided 2026-08-08 (V-663), because the bound above did not move the number it
|
||||
was written for. Twenty-six of 140 turns carried a tail before it landed and
|
||||
twenty-six carried one after.
|
||||
|
||||
Two bounds rearm each other. An aside spends no attempt, so `MaxAttempts` never
|
||||
reaches it. A turn that reads as a failed answer zeroes `Suspends`, so
|
||||
`MaxSuspends` never reaches the asides. Alternating them, each bound is restored
|
||||
by the other's traffic. Measured on 2026-08-08: one question about a reminder's
|
||||
day rode turns 7 to 13. It ended only because turn 14 was a new request.
|
||||
|
||||
`PendingQuestion.Rides` counts the same event as `Suspends` with the resets
|
||||
taken out. It is set once, incremented only in `noteSuspended`, carried across
|
||||
the re-park in `askRemainingGap`, and read by nothing that could lower it.
|
||||
`MaxRides` is 4, one looser than `MaxSuspends` so that the tighter statement
|
||||
about a run stays reachable.
|
||||
|
||||
This is a bound, not a cure. It ends the measured ride one turn early. Most of
|
||||
that ride's length is attempts, spent because `classifyTurnRole` reads "спасибо"
|
||||
and "привет" as failed answers to a question about a day. That is the next
|
||||
thing to fix and it is not a bound.
|
||||
|
||||
The re-ask is also two sentences rather than one. It used to be spliced onto the
|
||||
answer with a comma. On a real answer that buries the question in the tail of
|
||||
one run-on thought:
|
||||
|
||||
@@ -30,6 +30,12 @@ Maven is the user-facing control center, but not the source of truth for identit
|
||||
|
||||
Maven provides the human interface over the other systems.
|
||||
|
||||
| Service | Owns | Maven's client | Configured at |
|
||||
|---|---|---|---|
|
||||
| **Nexus** | Canonical entity ids, names, aliases, relationships. Projects, services, devices, people, pets, places. | `nexusClient` in `cmd/mavend/ecosystem.go`, `POST /api/v1/resolve` | `nexus.url` (`http://nexus:9740`) |
|
||||
| **Praxis** | Operational attention and item lifecycle. What needs looking at, what changed, what is unresolved. | `praxisClient`, the HTTP tools API under `/api/v1/tools/` | `praxis.url` (`http://praxis:8989`) |
|
||||
| **Hexis** | The capability registry and the only path to executing anything. | vendored `github.com/kami/hexis/pkg/client` | `hexis.url` (`http://hexis:9741`) |
|
||||
|
||||
It is responsible for:
|
||||
|
||||
- interpreting Russian and English utterances
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
# The parked clarify ride, bounded and re-measured
|
||||
|
||||
Date: 2026-08-08, V-663. Same 140 turns, same driver, third and fourth runs of
|
||||
the day. Before is `d6f3914`, after is that plus two changes.
|
||||
|
||||
## What was measured before
|
||||
|
||||
One question about a reminder's day rode turns 6 to 13. It ended only because
|
||||
turn 14 was a new request. Two of those turns are the worst replies in the
|
||||
corpus:
|
||||
|
||||
```text
|
||||
спасибо -> Сейчас 21:25. В какой день?
|
||||
привет -> Сейчас 21:25. В какой день?
|
||||
```
|
||||
|
||||
V-654 had already added `MaxSuspends` and the tail count had not moved.
|
||||
|
||||
## Why three bounds let it happen
|
||||
|
||||
The TTL, `MaxAttempts` and `MaxSuspends` all exist and all were rearmed.
|
||||
|
||||
An aside spends no attempt, so `MaxAttempts` never reaches it. A turn reading as
|
||||
a failed answer zeroes `Suspends`, so `MaxSuspends` never reaches the asides.
|
||||
`спасибо` read as a failed answer. So the pleasantry both produced the reply
|
||||
above and paid for it out of the budget meant to end the ride.
|
||||
|
||||
## The two changes
|
||||
|
||||
`PendingQuestion.Rides` counts the same event as `Suspends` with the resets
|
||||
taken out. Set once, incremented only in `noteSuspended`, carried across the
|
||||
re-park in `askRemainingGap`. `MaxRides` is 4, one looser than `MaxSuspends` so
|
||||
that the tighter statement about a run stays reachable.
|
||||
|
||||
A pleasantry is an aside, not a failed answer. The set is a new closed lexicon
|
||||
entry, matched as whole utterances rather than tokens. `вечер` answers `это утра
|
||||
или вечера?` and `нет` answers a confirm. Anything that could fill a slot stays
|
||||
out.
|
||||
|
||||
## Result
|
||||
|
||||
| | before | after |
|
||||
|---|---|---|
|
||||
| turns carrying a clarify tail | 21 | 17 |
|
||||
| turns carrying any failure string | 31 | 29 |
|
||||
| the longest ride | 8 turns | 4 turns |
|
||||
|
||||
The turns carrying a tail, by number:
|
||||
|
||||
```text
|
||||
before 6 7 8 9 10 11 12 13 53 54 55 56 57 99 102 103 116 136 137 138 139
|
||||
after 6 7 8 9 53 54 55 56 57 99 100 101 116 136 137 138 139
|
||||
```
|
||||
|
||||
Turn 10 is the change. It now reads:
|
||||
|
||||
```text
|
||||
спасибо -> Прошлую просьбу отпускаю. Пожалуйста, я всегда готова помочь тебе.
|
||||
```
|
||||
|
||||
She lets the request go, says so, and answers the man. Turns 11 to 13 are clean.
|
||||
|
||||
**`MaxRides` is not what fired.** The pleasantry is an aside now, so it no
|
||||
longer breaks the run. `MaxSuspends` reached three on turn 10 and ended it.
|
||||
`Rides` is the backstop for the shape where an answer really does break the run.
|
||||
No turn in this corpus reaches it.
|
||||
|
||||
## What did not move
|
||||
|
||||
Four rides are untouched. Turns 53 to 57 are five consecutive asides against a
|
||||
reminder missing its day. Turn 58 is a new request that drops it. Nothing
|
||||
pleasant appears in that run, so neither change applies. Turns 99 to 101 shifted
|
||||
by one, and 116 and 136 to 139 are unchanged.
|
||||
|
||||
So the fix is worth four turns of twenty-one. What is left is asides against a
|
||||
question the owner never answers. `MaxSuspends` was written for that shape and
|
||||
does bound it, at four turns each.
|
||||
|
||||
## Not attributable
|
||||
|
||||
Latency moved p50 1.1s to 1.5s and p95 2.8s to 3.0s, and the 34.3s outlier in
|
||||
the earlier run is gone. Both runs had the workstation up. Read none of it as
|
||||
caused by this change.
|
||||
|
||||
One unrelated defect appeared in the after run and is recorded here because it
|
||||
is visible in the transcript. Turn 4 answered `Я записала твою привычкуRegarding
|
||||
coffee without sugar.` That is English leaking into a Russian reply with no
|
||||
space in front of it. It is a phrasing defect and it has no task yet.
|
||||
@@ -0,0 +1,146 @@
|
||||
# The routing heads, running in Go
|
||||
|
||||
Date: 2026-08-08. Vikunja V-664.
|
||||
Weights: `router_heads.onnx`, fp32, exported from `heads.pt` on workpc.
|
||||
Fixture: `internal/router/eval/ru_routing_v1.json`, 96 cases, 33 carrying a destination.
|
||||
Runner: `make t PKG=./internal/router/eval/ RUN=TestONNXRoutingHeads`.
|
||||
|
||||
The four heads of V-661 ran nowhere. This is the number they score through the
|
||||
Go cascade. Same fixture and same grammars as `TestONNXBaseline`, and only the
|
||||
middle stage varies.
|
||||
|
||||
## Headline
|
||||
|
||||
| | classifier + ONNX | heads + classifier | gemma-4-12b cascade |
|
||||
|---|---|---|---|
|
||||
| intent | 75.0% (72/96) | **96.9% (93/96)** | 84.4% |
|
||||
| destination | 33.3% (11/33) | **75.8% (25/33)** | 72.7% |
|
||||
| false clarify | 0 | 1 | 2 |
|
||||
| missed clarify | 8 | 1 | 1 |
|
||||
| p50 | 24.5ms | 27.9ms | 329ms |
|
||||
|
||||
A 118M encoder beats the 12B teacher it was distilled from. It wins on both
|
||||
halves of the route, at a twelfth of the latency. The workstation stays the
|
||||
better phraser and is no longer the better router.
|
||||
|
||||
The p50 is not the heads. Most of it is the classifier's own embedder pass on
|
||||
the turns the heads decline, plus process warm-up on the first case. The heads'
|
||||
own forward pass measures 7.3ms on workpc.
|
||||
|
||||
## Two defects were in the way, and the first was not in the heads
|
||||
|
||||
**The tokenizer read every long word backwards.** `encodeWord` backtracks the
|
||||
Viterbi path from the end of a word and prepends each piece. That puts them back
|
||||
in reading order, and a second reverse after the loop undid it. So
|
||||
`query: вода` tokenized to `[0 12 1294 41 12489 2]` where the reference
|
||||
tokenizer gives `[0 41 1294 12 12489 2]`.
|
||||
|
||||
It was found here and only here. The heads were trained through transformers and
|
||||
are read through the hand-written tokenizer. So a mismatch shows up as a score
|
||||
far below what Python measured on the same weights. Nothing else in the suite
|
||||
compares the two.
|
||||
|
||||
Measured on the recall fixture, same 27 cases either way:
|
||||
|
||||
| | reversed | fixed |
|
||||
|---|---|---|
|
||||
| recall@1 | 70.4% (19/27) | **77.8% (21/27)** |
|
||||
| recall@3 | 85.2% (23/27) | **96.3% (26/27)** |
|
||||
| answered after gate | 63.0% | 66.7% |
|
||||
| wrong note on top | 8 | 6 |
|
||||
| false recall | 0/5 | 1/5 |
|
||||
|
||||
The classifier barely moved, 76.0% to 75.0%, and destination 36.4% to 33.3%.
|
||||
Both are one case on 96 and neither is a finding. Seeds and queries were mangled
|
||||
the same way, so cosine survived it. Recall is where it cost, because a stored
|
||||
passage and a live query are different lengths and break differently.
|
||||
|
||||
The one new false recall is the honest cost and it is not being hidden. A
|
||||
sharper embedder scores every candidate higher, including the ones that should
|
||||
have stayed under the gate. That is the same trade `2026-08-04-recall-e5-small.md`
|
||||
recorded when e5-small replaced MiniLM.
|
||||
|
||||
The embedder id now carries a tokenizer revision, `model_quantized@384/tok2`.
|
||||
Stored vectors were written under rev 1 and no longer sit in the same space as a
|
||||
query embedded now. The model file's name never moved, so nothing would have
|
||||
triggered `ReembedAll`. On the box the marker fired on start, and the re-embed
|
||||
rewrote 65 notes and 19 facts in 5 seconds.
|
||||
|
||||
**The clarify head was being thrown away.** It was read only when the intent head
|
||||
cleared its own threshold. That cost 6 of the 8 ambiguous cases. `вода` reads as intent
|
||||
`act` at 0.233 and clarify at 0.983. Burying that handed the turn to the
|
||||
classifier, which routed it confidently and never asked. The clarify head answers
|
||||
a different question, which is whether there is enough here to act on at all. So
|
||||
it decides on its own and decides first.
|
||||
|
||||
| | intent-gated | clarify decides first |
|
||||
|---|---|---|
|
||||
| intent | 90.6% | 96.9% |
|
||||
| missed clarify | 7 | 1 |
|
||||
| false clarify | 0 | 1 |
|
||||
|
||||
## The threshold is measured, not chosen
|
||||
|
||||
Max softmax over the intent head, on the 88 cases carrying an intent:
|
||||
|
||||
| threshold | kept | accuracy kept | wrong kept | right dropped |
|
||||
|---|---|---|---|---|
|
||||
| 0.5 | 84 | 96.4% | 3 | 2 |
|
||||
| **0.6** | **81** | **97.5%** | **2** | **4** |
|
||||
| 0.7 | 75 | 97.3% | 2 | 10 |
|
||||
| 0.8 | 64 | 96.9% | 2 | 21 |
|
||||
| 0.9 | 46 | 100.0% | 0 | 37 |
|
||||
|
||||
0.6 is the knee. Every value from 0.7 to 0.85 drops right answers and keeps the
|
||||
same two wrong ones. 0.9 is the only value that clears them, and it costs 37
|
||||
correct routes to do it.
|
||||
|
||||
## Quantization was measured and rejected
|
||||
|
||||
| build | size | intent | destination | p50 |
|
||||
|---|---|---|---|---|
|
||||
| fp32 | 470MB | 83/88 (94.3%) | 28/33 (84.8%) | 7.3ms |
|
||||
| int8 | 118MB | 79/88 (89.8%) | 26/33 (78.8%) | 4.0ms |
|
||||
| fp16 | 235MB | will not load | — | — |
|
||||
|
||||
Python numbers, on the heads alone rather than through the cascade. int8 costs
|
||||
4.5 points of intent and 6 of destination to save 3ms. The cascade around it has
|
||||
a p50 over a second when the resident model answers. The fp16 graph is broken:
|
||||
`convert_float_to_float16` leaves a Cast node emitting float16 where the graph
|
||||
expects float, and onnxruntime refuses the session. It was not worth fixing.
|
||||
|
||||
The exporter also had to be told to write one file. It splits weights into a
|
||||
`.onnx.data` sidecar by default. This onnxruntime resolves that path against the
|
||||
process working directory rather than the model. A split graph loads from one
|
||||
directory only.
|
||||
|
||||
## What is still wrong
|
||||
|
||||
**Four of the eight destination misses are calendar.** Training cannot move them.
|
||||
The possessive agenda rules claim those cases at stage 0 and name nothing on
|
||||
purpose. That caution was free while nothing downstream could name anything
|
||||
either. It has now cost four points in three separate measurements. The call is
|
||||
the owner's and it is still open.
|
||||
|
||||
**The slot head is exported and not read.** Slots come from the stage-2
|
||||
extractor. Mapping BIO tags back to text needs character offsets the unigram
|
||||
tokenizer does not keep, which is its own piece of work.
|
||||
|
||||
**`поужинал` is a false clarify**, which is the same defect `thinSingleToken`
|
||||
was narrowed for on 2026-08-01, arriving now from a different direction.
|
||||
|
||||
## On the box
|
||||
|
||||
Deployed to homesrv the same day. `voice: routing heads loaded` on start, and
|
||||
`/trace` shows `routing-heads` winning or thinning every turn. The resident model
|
||||
and the classifier are both marked never asked. Live probes:
|
||||
|
||||
```text
|
||||
что такое TCP? -> kiwix a real definition
|
||||
кто такой Линус Торвальдс? -> kiwix a real answer
|
||||
во сколько я лёг вчера -> personal не нашла у тебя такой записи
|
||||
вода -> thinned to clarify at 0.233 / 0.983
|
||||
```
|
||||
|
||||
A missing or broken weights file logs and leaves the heads nil, which is
|
||||
byte-for-byte the cascade that shipped before this.
|
||||
@@ -0,0 +1,631 @@
|
||||
# Raw transcript, two weeks of usage
|
||||
|
||||
## Day 1
|
||||
|
||||
```
|
||||
YOU: доброе утро
|
||||
MAVEN: Я не говорю так сейчас, ведь на улице уже вечер.
|
||||
[2.8s t=218]
|
||||
|
||||
YOU: какой сегодня день?
|
||||
MAVEN: сегодня суббота, 8 августа 2026 года
|
||||
[0.0s t=219]
|
||||
|
||||
YOU: сколько времени?
|
||||
MAVEN: сейчас 21 час 9 минут
|
||||
[0.0s t=220]
|
||||
|
||||
YOU: запиши что я пью кофе без сахара
|
||||
MAVEN: Я записала твою заметку про кофе。
|
||||
[1.5s t=221]
|
||||
|
||||
YOU: мой любимый язык программирования go
|
||||
MAVEN: Я записала твою заметку про Go.
|
||||
[1.5s t=222]
|
||||
|
||||
YOU: напомни в 11:00 позвонить маме
|
||||
MAVEN: Сейчас 21:10. В какой день?
|
||||
[0.4s t=223]
|
||||
|
||||
YOU: что у меня сегодня?
|
||||
MAVEN: на 08.08.2026 ничего нет. На какое время поставить напоминание?
|
||||
[2.5s t=224]
|
||||
|
||||
YOU: что такое TCP?
|
||||
MAVEN: вот что я нашла: TCP (Transmission Control Protocol) — это транспортный протокол для передачи данных. Он обеспечивает доставку всех пакетов и позволяет собрать их в правильном порядке. Также TCP является частью сетевой модели TCP/IP, которая описывает способ передачи цифровых данных от источника к получателю. На какое время поставить напоминание?
|
||||
[2.8s t=225]
|
||||
|
||||
YOU: сколько будет 17 на 23?
|
||||
MAVEN: вот что я нашла: в источниках нет результата умножения 17 на 23. На какое время поставить напоминание?
|
||||
[1.6s t=226]
|
||||
|
||||
YOU: спасибо
|
||||
MAVEN: Сейчас 21:10. В какой день?
|
||||
[1.1s t=227]
|
||||
|
||||
```
|
||||
|
||||
## Day 2
|
||||
|
||||
```
|
||||
YOU: привет
|
||||
MAVEN: Сейчас 21:10. В какой день?
|
||||
[0.9s t=228]
|
||||
|
||||
YOU: что нового?
|
||||
MAVEN: не знаю — не нашла у тебя такой записи. На какое время поставить напоминание?
|
||||
[1.3s t=229]
|
||||
|
||||
YOU: какая погода?
|
||||
MAVEN: для какого города? На какое время поставить напоминание?
|
||||
[0.7s t=230]
|
||||
|
||||
YOU: запиши что пароль от вайфая лежит в ящике стола
|
||||
MAVEN: Прошлую просьбу отпускаю. Я записала, что пароль от вай-фай лежит в ящике стола.
|
||||
[1.5s t=231]
|
||||
|
||||
YOU: где лежит вайфай пароль?
|
||||
MAVEN: не знаю — не нашла у тебя такой записи.
|
||||
[0.5s t=232]
|
||||
|
||||
YOU: добавь молоко в список покупок
|
||||
MAVEN: Такую команду я не знаю.
|
||||
[1.0s t=233]
|
||||
|
||||
YOU: что у меня в списке покупок?
|
||||
MAVEN: в списке покупок пусто.
|
||||
[0.0s t=234]
|
||||
|
||||
YOU: кто такой Линус Торвальдс?
|
||||
MAVEN: вот что я нашла: Линус Торвальдс — финско-американский программист из Хельсинки, создатель ядра Linux. Он родился 28 декабря 1969 года и окончил факультет компьютерных наук Хельсинкского университета.
|
||||
[2.6s t=235]
|
||||
|
||||
YOU: какой у меня любимый язык?
|
||||
MAVEN: не знаю — не нашла у тебя такой записи.
|
||||
[0.6s t=236]
|
||||
|
||||
YOU: сколько у меня задач?
|
||||
MAVEN: сначала: проверить интейк форму на боксе (завтра); купить хлеб; купить молоко; полить цветы; починить кран
|
||||
[0.0s t=237]
|
||||
|
||||
```
|
||||
|
||||
## Day 3
|
||||
|
||||
```
|
||||
YOU: как дела?
|
||||
MAVEN: У меня всё хорошо, я готова помогать тебе. Чем могу быть полезна?
|
||||
[1.2s t=238]
|
||||
|
||||
YOU: напомни завтра в 9 утра купить хлеб
|
||||
MAVEN: хорошо, напомню завтра в 09:00.
|
||||
[0.3s t=239]
|
||||
|
||||
YOU: что у меня завтра?
|
||||
MAVEN: на 09.08.2026 ничего нет.
|
||||
[0.0s t=240]
|
||||
|
||||
YOU: отмени напоминание про хлеб
|
||||
MAVEN: Такую команду я не знаю.
|
||||
[0.5s t=241]
|
||||
|
||||
YOU: какие у меня напоминания?
|
||||
MAVEN: вот что я нашла: в источниках нет информации о твоих напоминаниях.
|
||||
[0.7s t=242]
|
||||
|
||||
YOU: сохрани мне адрес гостиницы в Сочи
|
||||
MAVEN: Я записала твой адрес отеля в Сочи.
|
||||
[2.2s t=243]
|
||||
|
||||
YOU: что я сохранил про Сочи?
|
||||
MAVEN: вот что я нашла: в источниках нет информации о том, что ты сохранил про Сочи.
|
||||
[1.5s t=244]
|
||||
|
||||
YOU: почему сервер тормозит?
|
||||
MAVEN: вот что я нашла: сервер может тормозить из-за неудачной конфигурации или неоптимальных настроек, отсутствия кэширования, неоптимизированного программного обеспечения, которое нужно обновлять, а также из-за режима энергосбережения процессора, когда он не успевает повысить частоту при нагрузке.
|
||||
[3.7s t=245]
|
||||
|
||||
YOU: хватает ли места под новые бэкапы?
|
||||
MAVEN: нового вот что: [$] Changes in shadow-utils password-expiration features; The Software Stewardship Lab launches; LightDM lives: version 1.33.0 released
|
||||
[0.7s t=246]
|
||||
|
||||
YOU: выключи свет в спальне
|
||||
MAVEN: Такую команду я не знаю.
|
||||
[0.4s t=247]
|
||||
|
||||
```
|
||||
|
||||
## Day 4
|
||||
|
||||
```
|
||||
YOU: доброе утро
|
||||
MAVEN: Сейчас уже вечер, поэтому я не могу сказать тебе «доброе утро».
|
||||
[1.4s t=248]
|
||||
|
||||
YOU: что я пропустил?
|
||||
MAVEN: вот что я нашла: в источниках нет информации о том, что ты пропустил.
|
||||
[1.3s t=249]
|
||||
|
||||
YOU: о чём мы вчера говорили?
|
||||
MAVEN: не знаю — не нашла у тебя такой записи.
|
||||
[1.0s t=250]
|
||||
|
||||
YOU: запиши что я записался к врачу на четверг
|
||||
MAVEN: Я записала твою запись к врачу.
|
||||
[1.2s t=251]
|
||||
|
||||
YOU: когда я иду к врачу?
|
||||
MAVEN: вот что я нашла: в источниках нет информации о том, когда именно ты идешь к врачу.
|
||||
[2.1s t=252]
|
||||
|
||||
YOU: что такое ZFS?
|
||||
MAVEN: вот что я нашла: ZFS — это современная и мощная файловая система, в которой сочетаются безопасность, масштабируемость и удобство управления данными. Она была разработана корпорацией Sun Microsystems и объединяет в себе файловую систему с менеджером томов. Изначально она создавалась для того, чтобы преодолеть недостатки традиционных RAID-массивов, такие как проблема «дыры записи».
|
||||
[2.1s t=253]
|
||||
|
||||
YOU: столица Франции?
|
||||
MAVEN: вот что я нашла: столица Франции — Париж.
|
||||
[3.7s t=254]
|
||||
|
||||
YOU: переведи слово ремонт на английский
|
||||
MAVEN: вот что я нашла: слово «ремонт» на английский можно перевести как repair (например, ремонт дороги или крыши) или fix (общее слово для починки техники и мебели).
|
||||
[2.7s t=255]
|
||||
|
||||
YOU: сколько я потратил в этом месяце?
|
||||
MAVEN: я не отслеживаю траты — не подключено.
|
||||
[0.7s t=256]
|
||||
|
||||
YOU: спокойной ночи
|
||||
MAVEN: Спокойной ночи. Я уже подготовила всё к твоему отдыху.
|
||||
[1.2s t=257]
|
||||
|
||||
```
|
||||
|
||||
## Day 5
|
||||
|
||||
```
|
||||
YOU: привет
|
||||
MAVEN: Привет! Я готова помогать тебе сегодня.
|
||||
[1.0s t=258]
|
||||
|
||||
YOU: какая погода в Москве?
|
||||
MAVEN: погода не настроена.
|
||||
[0.9s t=259]
|
||||
|
||||
YOU: что там с бэкапами?
|
||||
MAVEN: вот что я нашла: бэкап — это резервная копия данных, созданная в определенный момент времени и сохраненная на локальном или облачном носителе. Его делают, чтобы быстро восстановить информацию в случае необходимости.
|
||||
[2.3s t=260]
|
||||
|
||||
YOU: покажи что требует внимания
|
||||
MAVEN: мне пока нечего смотреть — у Praxis нет источников.
|
||||
[0.5s t=261]
|
||||
|
||||
YOU: отметь это как сделанное
|
||||
MAVEN: команды «отметь» пока нет в списке — она уже предложена, включи через клиент.
|
||||
[0.0s t=262]
|
||||
|
||||
YOU: запиши что я купил новые наушники
|
||||
MAVEN: Я записала твою покупку новых наушников.
|
||||
[1.8s t=263]
|
||||
|
||||
YOU: какие у меня заметки за неделю?
|
||||
MAVEN: не знаю — не нашла у тебя такой записи.
|
||||
[0.0s t=264]
|
||||
|
||||
YOU: расскажи про Kubernetes
|
||||
MAVEN: вот что я нашла: Kubernetes — это портативная расширяемая платформа с открытым исходным кодом для управления сервисами и контейнеризованными рабочими нагрузками. Управлять им можно через интерфейс в командной строке kubectl. Я нашла эти данные в предоставленных источниках.
|
||||
[1.8s t=265]
|
||||
|
||||
YOU: кто я?
|
||||
MAVEN: вот что я нашла: в источниках нет информации о том, кто ты.
|
||||
[1.4s t=266]
|
||||
|
||||
YOU: пока
|
||||
MAVEN: До свидания, пока!
|
||||
[1.4s t=267]
|
||||
|
||||
```
|
||||
|
||||
## Day 6
|
||||
|
||||
```
|
||||
YOU: доброе утро
|
||||
MAVEN: Сейчас уже вечер, так что доброе утро я тебе сказать не могу. Я готова помочь тебе с чем-нибудь другим.
|
||||
[1.4s t=268]
|
||||
|
||||
YOU: сколько времени?
|
||||
MAVEN: сейчас 21 час 11 минут
|
||||
[0.0s t=269]
|
||||
|
||||
YOU: напомни в 18:30 позвонить в банк
|
||||
MAVEN: Сейчас 21:11. В какой день?
|
||||
[0.3s t=270]
|
||||
|
||||
YOU: поставь чайник
|
||||
MAVEN: Сейчас 21:11. В какой день?
|
||||
[1.1s t=271]
|
||||
|
||||
YOU: включи музыку
|
||||
MAVEN: Сейчас 21:11. В какой день?
|
||||
[0.9s t=272]
|
||||
|
||||
YOU: что у меня в календаре на пятницу?
|
||||
MAVEN: вот что я нашла: в источниках нет информации о твоих личных событиях в календаре на пятницу. В одном из текстов упоминается пятница 24 июля 2026 года, а в другом — что сегодня пятница 7 августа 2026 года. На какое время поставить напоминание?
|
||||
[2.6s t=273]
|
||||
|
||||
YOU: во сколько у меня встреча?
|
||||
MAVEN: не знаю — не нашла у тебя такой записи. На какое время поставить напоминание?
|
||||
[0.3s t=274]
|
||||
|
||||
YOU: запиши что дедлайн по проекту в понедельник
|
||||
MAVEN: Прошлую просьбу отпускаю. Я записала твой дедлайн на понедельник.
|
||||
[1.7s t=275]
|
||||
|
||||
YOU: успею ли я до дедлайна?
|
||||
MAVEN: вот что я нашла: в источниках нет информации о твоем дедлайне, поэтому я не могу ответить, успеешь ты или нет.
|
||||
[2.0s t=276]
|
||||
|
||||
YOU: спасибо
|
||||
MAVEN: Пожалуйста, я всегда готова помочь тебе.
|
||||
[1.4s t=277]
|
||||
|
||||
```
|
||||
|
||||
## Day 7
|
||||
|
||||
```
|
||||
YOU: привет
|
||||
MAVEN: Привет! Чем я могу тебе помочь?
|
||||
[1.3s t=278]
|
||||
|
||||
YOU: как ты?
|
||||
MAVEN: Я в порядке, спасибо, что спросил. Готова помогать тебе!
|
||||
[1.4s t=279]
|
||||
|
||||
YOU: расскажи анекдот
|
||||
MAVEN: вот что я нашла: в источниках есть ссылки на сборники и сайты с анекдотами, но самих текстов анекдотов там нет, кроме одного: ударилась лягушка о землю головой и стала девицей прекрасной, а второй раз — месивом кровавым.
|
||||
[3.9s t=280]
|
||||
|
||||
YOU: что ты умеешь?
|
||||
MAVEN: Я Мэйвен, твоя помощница. Я живу на твоём сервере и отвечаю на твои вопросы. Я запоминаю то, что ты мне говоришь, веду заметки, ставлю напоминания, а также читаю твой календарь и задачи. Я отвечаю на вопросы о мире, используя поиск или офлайн-энциклопедию. Если ты разрешишь, я могу управлять домом, локальной сетью, лентами, списком покупок и погодой, а также писать в телеграм. Я говорю по-русски и по-английски.
|
||||
[2.9s t=281]
|
||||
|
||||
YOU: запиши что я начал бегать по утрам
|
||||
MAVEN: Я записала твою новую привычку
|
||||
[1.5s t=282]
|
||||
|
||||
YOU: я бегаю по утрам уже неделю
|
||||
MAVEN: записала: я бегаю по утрам уже неделю
|
||||
[0.6s t=283]
|
||||
|
||||
YOU: как часто я бегаю?
|
||||
MAVEN: вот что я нашла: в источниках нет информации о том, как часто бегаешь ты. Там указано, что для похудения частоту тренировок можно доводить до 5–6 раз в неделю, а новичкам ежедневные тренировки без подготовки могут привести к травмам.
|
||||
[3.3s t=284]
|
||||
|
||||
YOU: сколько стоит биткоин?
|
||||
MAVEN: вот что я нашла: в источниках нет информации о стоимости биткоина.
|
||||
[1.2s t=285]
|
||||
|
||||
YOU: какие новости?
|
||||
MAVEN: вот что нового: [$] Changes in shadow-utils password-expiration features; The Software Stewardship Lab launches; LightDM lives: version 1.33.0 released
|
||||
[0.8s t=286]
|
||||
|
||||
YOU: хорошего дня
|
||||
MAVEN: Я не могу сказать «хорошего дня», так как сейчас уже вечер. Хорошего отдыха!
|
||||
[1.3s t=287]
|
||||
|
||||
```
|
||||
|
||||
## Day 8
|
||||
|
||||
```
|
||||
YOU: доброе утро
|
||||
MAVEN: Сейчас вечер, так что доброе утро я не скажу. Чем я могу тебе помочь?
|
||||
[1.2s t=288]
|
||||
|
||||
YOU: что у меня сегодня?
|
||||
MAVEN: на 08.08.2026 ничего нет.
|
||||
[0.0s t=289]
|
||||
|
||||
YOU: напомни через час выпить воды
|
||||
MAVEN: хорошо, напомню сегодня в 22:11.
|
||||
[0.3s t=290]
|
||||
|
||||
YOU: я выпил воды
|
||||
MAVEN: записала: я выпил воды
|
||||
[0.9s t=291]
|
||||
|
||||
YOU: запиши что кот ест только сухой корм
|
||||
MAVEN: Я записала твою заметку про кота.
|
||||
[1.2s t=292]
|
||||
|
||||
YOU: чем питается кот?
|
||||
MAVEN: вот что я нашла: в природе кошки едят мышей, птиц и ящериц. Дома их правильно кормить мясом, мясными субпродуктами, яйцами и нежирной морской рыбой. Из-за особенностей пищеварения им не подходят каши, макароны и мучные изделия.
|
||||
[3.4s t=293]
|
||||
|
||||
YOU: что такое DNS?
|
||||
MAVEN: вот что я нашла: DNS (Domain Name System) — это распределенная база данных, которая сопоставляет домены с IP-адресами. Это не единая база на одном сервере, а иерархическая система серверов, работающих совместно. В ней хранятся записи, которые обеспечивают маршрутизацию трафика и управление электронной почтой.
|
||||
[2.0s t=294]
|
||||
|
||||
YOU: проверь статус uptime kuma
|
||||
MAVEN: uptime kuma — Проверяю статус Uptime Kuma...
|
||||
[1.1s t=295]
|
||||
|
||||
YOU: всё ли в порядке с сервером?
|
||||
MAVEN: мне пока нечего смотреть — у Praxis нет источников.
|
||||
[0.5s t=296]
|
||||
|
||||
YOU: спасибо
|
||||
MAVEN: Пожалуйста, я всегда рада помочь тебе.
|
||||
[2.7s t=297]
|
||||
|
||||
```
|
||||
|
||||
## Day 9
|
||||
|
||||
```
|
||||
YOU: привет
|
||||
MAVEN: Привет! Чем я могу тебе помочь сегодня?
|
||||
[1.9s t=298]
|
||||
|
||||
YOU: какой сегодня день недели?
|
||||
MAVEN: сегодня суббота, 8 августа 2026 года
|
||||
[0.0s t=299]
|
||||
|
||||
YOU: добавь хлеб и сыр в список покупок
|
||||
MAVEN: Такую команду я не знаю.
|
||||
[0.6s t=300]
|
||||
|
||||
YOU: что в списке покупок?
|
||||
MAVEN: в списке покупок пусто.
|
||||
[0.0s t=301]
|
||||
|
||||
YOU: удали молоко из списка
|
||||
MAVEN: Такую команду я не знаю.
|
||||
[0.4s t=302]
|
||||
|
||||
YOU: напомни завтра утром вынести мусор
|
||||
MAVEN: Сейчас 21:11. Когда?
|
||||
[0.3s t=303]
|
||||
|
||||
YOU: запиши что я поменял масло в машине
|
||||
MAVEN: Прошлую просьбу отпускаю. записала: запиши что я поменял масло в машине
|
||||
[0.7s t=304]
|
||||
|
||||
YOU: когда я менял масло?
|
||||
MAVEN: масло в машине — поменял
|
||||
[0.4s t=305]
|
||||
|
||||
YOU: сколько будет 144 делить на 12?
|
||||
MAVEN: вот что я нашла: в источниках указано решение деления 144 на 12 в столбик, но итоговый результат деления не прописан.
|
||||
[2.3s t=306]
|
||||
|
||||
YOU: пока
|
||||
MAVEN: До свидания! Если понадобится что-то записать или проверить, я всегда на связи.
|
||||
[1.2s t=307]
|
||||
|
||||
```
|
||||
|
||||
## Day 10
|
||||
|
||||
```
|
||||
YOU: доброе утро
|
||||
MAVEN: Я не могу сказать «доброе утро», так как сейчас уже вечер. Я записала твои предыдущие вопросы.
|
||||
[1.8s t=308]
|
||||
|
||||
YOU: что нового за ночь?
|
||||
MAVEN: вот что я нашла: в ночь на 8 августа вооруженные силы РФ нанесли второй за последнее время массированный удар, при этом киевская ПВО пропустила все российские ракеты.
|
||||
[3.3s t=309]
|
||||
|
||||
YOU: почему интернет медленный?
|
||||
MAVEN: вот что я нашла: интернет может работать медленно из-за плохого сигнала из-за удаленности вышки, исчерпанного лимита трафика по тарифу, нахождения вне зоны покрытия 4G/LTE, устаревшей или неисправной SIM-карты, а также из-за сбоя в настройках роутера. Чтобы исправить ситуацию, можно попробовать перезагрузить роутер или перенастроить устройство.
|
||||
[3.0s t=310]
|
||||
|
||||
YOU: какая скорость у меня сейчас?
|
||||
MAVEN: для какого города?
|
||||
[0.6s t=311]
|
||||
|
||||
YOU: запиши что новый роутер стоит 8000 рублей
|
||||
MAVEN: Я записала стоимость нового роутера.
|
||||
[1.2s t=312]
|
||||
|
||||
YOU: сколько стоил роутер?
|
||||
MAVEN: вот что я нашла: цена зависит от модели. Роутер от Ростелеком стоит 5 900 рублей, на OZON есть варианты от 4 052 до 3 251 рублей, а в Билайне цены начинаются от 2 149 рублей.
|
||||
[2.6s t=313]
|
||||
|
||||
YOU: что такое NAT?
|
||||
MAVEN: вот что я нашла: NAT (Network Address Translation) — это технология в сетях TCP/IP, которая позволяет изменять IP-адрес в заголовке пакета при его прохождении через устройство. Она перенаправляет трафик между локальной и глобальной сетями, позволяет объединять устройства в локальной сети под одним публичным IP-адресом, а также скрывает сеть и экономит IP.
|
||||
[2.1s t=314]
|
||||
|
||||
YOU: напомни в субботу позвонить бабушке
|
||||
MAVEN: Сейчас 21:12. Когда?
|
||||
[0.3s t=315]
|
||||
|
||||
YOU: покажи мои напоминания
|
||||
MAVEN: вот что я нашла: в источниках нет информации о твоих конкретных напоминаниях, там только описано, как их создавать и где искать в приложениях. На какое время поставить напоминание?
|
||||
[3.0s t=316]
|
||||
|
||||
YOU: спасибо
|
||||
MAVEN: Сейчас 21:12. Во сколько напомнить?
|
||||
[1.0s t=317]
|
||||
|
||||
```
|
||||
|
||||
## Day 11
|
||||
|
||||
```
|
||||
YOU: привет
|
||||
MAVEN: Сейчас 21:12. Скажи время — например, «в семь вечера» или «через час».
|
||||
[0.8s t=318]
|
||||
|
||||
YOU: как погода на выходных?
|
||||
MAVEN: для какого города? На какое время поставить напоминание?
|
||||
[0.7s t=319]
|
||||
|
||||
YOU: что у меня на этой неделе?
|
||||
MAVEN: вот что я нашла: в источниках нет информации о том, что у тебя на этой неделе. На какое время поставить напоминание?
|
||||
[1.0s t=320]
|
||||
|
||||
YOU: запиши что я хочу прочитать книгу про Go
|
||||
MAVEN: Прошлую просьбу отпускаю. Я записала твою заметку о книге по Go.
|
||||
[1.6s t=321]
|
||||
|
||||
YOU: что я хотел прочитать?
|
||||
MAVEN: не знаю — не нашла у тебя такой записи.
|
||||
[0.9s t=322]
|
||||
|
||||
YOU: объясни что такое горутина
|
||||
MAVEN: вот что я нашла: я проверила источники и узнала, что горутины — это легковесные потоки выполнения или функции, которые позволяют параллельно выполнять код внутри программы. Они являются легкими нитями, которые создаются и планируются для независимого выполнения планировщиком Go.
|
||||
[2.5s t=323]
|
||||
|
||||
YOU: кто написал Войну и мир?
|
||||
MAVEN: вот что я нашла: роман «Война и мир» написал Лев Николаевич Толстой.
|
||||
[2.2s t=324]
|
||||
|
||||
YOU: включи свет на кухне
|
||||
MAVEN: Такую команду я не знаю.
|
||||
[0.5s t=325]
|
||||
|
||||
YOU: закрой шторы в комнате
|
||||
MAVEN: Такую команду я не знаю.
|
||||
[0.4s t=326]
|
||||
|
||||
YOU: спокойной ночи
|
||||
MAVEN: Спокойной ночи! Я уже подготовила всё к твоему отдыху.
|
||||
[1.1s t=327]
|
||||
|
||||
```
|
||||
|
||||
## Day 12
|
||||
|
||||
```
|
||||
YOU: доброе утро
|
||||
MAVEN: Сейчас уже вечер, поэтому я не могу сказать тебе «доброе утро».
|
||||
[1.1s t=328]
|
||||
|
||||
YOU: сколько сейчас времени?
|
||||
MAVEN: сейчас 21 час 12 минут
|
||||
[0.0s t=329]
|
||||
|
||||
YOU: я не то имел в виду
|
||||
MAVEN: Я поняла тебя, исправь, пожалуйста, свою мысль. Что именно ты хотел сказать?
|
||||
[1.8s t=330]
|
||||
|
||||
YOU: о чём мы говорили?
|
||||
MAVEN: не знаю — не нашла у тебя такой записи.
|
||||
[0.6s t=331]
|
||||
|
||||
YOU: напомни
|
||||
MAVEN: Сейчас 21:12. Когда?
|
||||
[0.5s t=332]
|
||||
|
||||
YOU: сделай это
|
||||
MAVEN: пока не умею отвечать на этот вопрос. На какое время поставить напоминание?
|
||||
[0.5s t=333]
|
||||
|
||||
YOU: запиши что я перешёл на новый тариф
|
||||
MAVEN: Прошлую просьбу отпускаю. Я записала твою новую информацию о тарифе.
|
||||
[1.5s t=334]
|
||||
|
||||
YOU: какой у меня тариф?
|
||||
MAVEN: вот что я нашла: в источниках нет информации о твоем тарифе.
|
||||
[1.2s t=335]
|
||||
|
||||
YOU: сколько я плачу за интернет?
|
||||
MAVEN: вот что я нашла: в источниках нет информации о том, сколько именно ты платишь за интернет. Я проверила документы, но точной суммы там нет.
|
||||
[2.6s t=336]
|
||||
|
||||
YOU: спасибо
|
||||
MAVEN: Я поняла тебя. Что именно ты хочешь, чтобы я сделала сейчас?
|
||||
[1.5s t=337]
|
||||
|
||||
```
|
||||
|
||||
## Day 13
|
||||
|
||||
```
|
||||
YOU: привет
|
||||
MAVEN: Привет! Я готова помогать тебе. Чем займемся сегодня?
|
||||
[1.3s t=338]
|
||||
|
||||
YOU: что там с задачами?
|
||||
MAVEN: вот что я нашла: я отметила задачу про бэкапы как сделанную.
|
||||
[1.5s t=339]
|
||||
|
||||
YOU: закрывай
|
||||
MAVEN: какой пункт отметить сделанным?
|
||||
[0.0s t=340]
|
||||
|
||||
YOU: отметь задачу про бэкапы как сделанную
|
||||
MAVEN: Такую команду я не знаю.
|
||||
[0.6s t=341]
|
||||
|
||||
YOU: что осталось нерешённым?
|
||||
MAVEN: вот что я нашла: в твоей жизни всё, что остается нерешенным или незавершенным, будет продолжать существовать и требовать внимания. Также в источниках упоминается пример с скобками, который может остаться нерешенным, и вопрос о том, что остается нерешенным в краснодарских перевозках.
|
||||
[4.2s t=342]
|
||||
|
||||
YOU: запиши что я договорился о встрече в среду
|
||||
MAVEN: Я записала твою договоренность о встрече.
|
||||
[2.3s t=343]
|
||||
|
||||
YOU: когда у меня встреча?
|
||||
MAVEN: мне пока нечего смотреть — у Praxis нет источников.
|
||||
[0.0s t=344]
|
||||
|
||||
YOU: какая температура на улице?
|
||||
MAVEN: для какого города?
|
||||
[0.9s t=345]
|
||||
|
||||
YOU: что такое RAID 5?
|
||||
MAVEN: вот что я нашла: RAID 5 — это массив из независимых дисков, где данные и контрольные суммы (четность) распределяются по кругу на все диски. Этот уровень обеспечивает отказоустойчивость и повышенную производительность, при этом для его работы требуется не менее трех дисков.
|
||||
[2.3s t=346]
|
||||
|
||||
YOU: пока
|
||||
MAVEN: До встречи!
|
||||
[1.4s t=347]
|
||||
|
||||
```
|
||||
|
||||
## Day 14
|
||||
|
||||
```
|
||||
YOU: доброе утро
|
||||
MAVEN: Я не могу сказать «доброе утро», потому что сейчас вечер. Я уже записала твой вопрос и готова помочь с чем-то другим.
|
||||
[1.7s t=348]
|
||||
|
||||
YOU: подведи итоги недели
|
||||
MAVEN: week summary — подведи итоги недели
|
||||
[0.6s t=349]
|
||||
|
||||
YOU: что я делал за последние две недели?
|
||||
MAVEN: вот что я нашла: в источниках нет информации о том, что ты делал за последние две недели.
|
||||
[1.3s t=350]
|
||||
|
||||
YOU: какие заметки я сохранил?
|
||||
MAVEN: вот что я нашла: в источниках нет информации о том, какие именно заметки ты сохранил.
|
||||
[1.3s t=351]
|
||||
|
||||
YOU: о чём я чаще всего спрашиваю?
|
||||
MAVEN: обычно ты пьёшь воду около 00:58, отмечаешь «name» около 23:56 и отмечаешь «week summary» около 23:59 — по записям за последние 7 дней.
|
||||
[0.9s t=352]
|
||||
|
||||
YOU: напомни в понедельник в 10 проверить бэкапы
|
||||
MAVEN: Сейчас 21:12. Это утра или вечера?
|
||||
[0.3s t=353]
|
||||
|
||||
YOU: что у меня в понедельник?
|
||||
MAVEN: не знаю — не нашла у тебя такой записи. На какое время поставить напоминание?
|
||||
[0.3s t=354]
|
||||
|
||||
YOU: ты меня понимаешь?
|
||||
MAVEN: Да, я тебя прекрасно понимаю. Чем я могу тебе помочь? На какое время поставить напоминание?
|
||||
[1.3s t=355]
|
||||
|
||||
YOU: спасибо тебе
|
||||
MAVEN: Сейчас 21:12. Это утра или вечера?
|
||||
[1.0s t=356]
|
||||
|
||||
YOU: спокойной ночи
|
||||
MAVEN: хорошо, напомню послезавтра в 10:00.
|
||||
[1.2s t=357]
|
||||
|
||||
```
|
||||
@@ -0,0 +1,132 @@
|
||||
# The same fortnight, after V-655 merged
|
||||
|
||||
Date: 2026-08-08, a few hours after `2026-08-08-two-weeks.md`.
|
||||
Build: `f8fa0d1` on master, the five compose services rebuilt and recreated.
|
||||
Same driver, same 140 turns, same reach. This is the diff that baseline was for.
|
||||
|
||||
Master now carries V-655, V-659, V-660 and V-661. The change under test is V-655. A query source that decides by seed similarity
|
||||
is marked `guesses: true`. It is dropped when the cascade names a different
|
||||
destination.
|
||||
|
||||
## Two things confound the comparison and one of them matters
|
||||
|
||||
**The workstation was up for the re-run.** `llama-server` on 192.168.1.105
|
||||
answered a health probe with 200. So routing completed through `llm.Pair`
|
||||
against gemma-4-12b, which is the arm that names a destination. Its state
|
||||
during the baseline was not recorded. So a difference here may be the merge, or
|
||||
may be the better router, and this run cannot separate them.
|
||||
|
||||
**The store carried over**, as the baseline said it would. Facts written by the
|
||||
first run were present from turn 1 of the second.
|
||||
|
||||
## Numbers
|
||||
|
||||
| | baseline `beb093a` | after `f8fa0d1` |
|
||||
|---|---|---|
|
||||
| turns | 140 | 140 |
|
||||
| p50 | 1.5s | 1.2s |
|
||||
| p95 | 7.1s | 3.0s |
|
||||
| max | 33.7s | 4.2s |
|
||||
| transport errors | 0 | 0 |
|
||||
| turns carrying a failure string | 41 | 38 |
|
||||
|
||||
| string in the reply | before | after |
|
||||
|---|---|---|
|
||||
| `на какое время поставить напоминание` | 13 | 13 |
|
||||
| `не нашла у тебя такой записи` | 8 | 9 |
|
||||
| `Такую команду я не знаю` | 8 | 8 |
|
||||
| `для какого города` | 6 | 4 |
|
||||
| `В какой день` | 6 | 6 |
|
||||
| `пока не умею` | 5 | 1 |
|
||||
| `Когда?` | 3 | 3 |
|
||||
|
||||
Read the latency as unattributed. The workstation confound covers all of it.
|
||||
|
||||
## Defect 2 is the one this was for: four of six fixed
|
||||
|
||||
| utterance | before | after |
|
||||
|---|---|---|
|
||||
| `что такое TCP?` | `для какого города?` | a real definition |
|
||||
| `сколько будет 17 на 23?` | `для какого города?` | search, which has no answer |
|
||||
| `какой у меня любимый язык?` | kernel headlines | `не нашла у тебя такой записи` |
|
||||
| `что я сохранил про Сочи?` | `Хорошо, сохраню.` | answered as a question |
|
||||
| `какая скорость у меня сейчас?` | `для какого города?` | `для какого города?` |
|
||||
| `хватает ли места под новые бэкапы?` | kernel headlines | kernel headlines |
|
||||
|
||||
`что такое TCP?` is the clean win. `WorldQueryGrammars` names `world` at stage
|
||||
0, weather is dropped, and search answers.
|
||||
|
||||
`сколько будет 17 на 23?` moved source and not outcome. Weather no longer claims it. Search
|
||||
cannot do arithmetic, so the reply says the sources have no product of 17 and
|
||||
23. That is an honest gap where it used to be a wrong
|
||||
question. Arithmetic has no destination in the enum.
|
||||
|
||||
`что я сохранил про Сочи?` was defect 3 and it is gone. The utterance is no
|
||||
longer read as a capture.
|
||||
|
||||
**The two that did not move are both homelab questions.** They are exactly the
|
||||
cluster the destination fixture flagged. `SourceRecall`, `SourceNetwork` and
|
||||
`SourceAttention` overlap on every question about the box. Five of the seven
|
||||
floor cases in that fixture are homelab operations for the same reason. So this
|
||||
is the enum, not the walk.
|
||||
|
||||
## Defect 1 did not move at all
|
||||
|
||||
Twenty-six turns still carry a parked clarify tail, the same count as the
|
||||
baseline. `спасибо тебе` answers `Сейчас 21:12. Это утра или вечера?` and
|
||||
`спокойной ночи` answers `хорошо, напомню послезавтра в 10:00.`
|
||||
|
||||
V-655 was never going to touch this. A parked clarify is dialogue state and not
|
||||
a query source. It remains the single worst thing about talking to her. The week test, the
|
||||
fortnight test and this re-run all report it unchanged.
|
||||
|
||||
## A gap in the harness, fixed and re-run the same day
|
||||
|
||||
`ipc.ChatReply.Source` came back empty on all 140 turns, in both runs. The
|
||||
driver read the redirect parameter `src` and `cmd/mavweb/chat.go` writes `s`.
|
||||
So every finding above is read off the reply text instead of off the badge.
|
||||
|
||||
Fixed in V-662 and the 140 turns were driven a third time. Sixty-eight of them
|
||||
name a source. The rest are not query turns and never reach `queryWalk`.
|
||||
|
||||
| source | turns |
|
||||
|---|---|
|
||||
| search | 27 |
|
||||
| memory | 13 |
|
||||
| personal | 9 |
|
||||
| weather | 5 |
|
||||
| calendar | 3 |
|
||||
| attention | 3 |
|
||||
| list | 2 |
|
||||
| feeds | 2 |
|
||||
| tasks, money, self, habits | 1 each |
|
||||
|
||||
## What the badge shows that the wording did not
|
||||
|
||||
The two unfixed homelab turns are now direct evidence.
|
||||
|
||||
```text
|
||||
какая скорость у меня сейчас? -> weather
|
||||
хватает ли места под новые бэкапы? -> feeds
|
||||
```
|
||||
|
||||
Both are guessing sources claiming a turn about the box, exactly as the
|
||||
destination fixture predicted.
|
||||
|
||||
The badge also names a defect the wording hid. **Agenda questions are being
|
||||
claimed by the personal boundary and by Praxis, not by the calendar.**
|
||||
|
||||
```text
|
||||
во сколько у меня встреча? -> personal не нашла у тебя такой записи
|
||||
когда у меня встреча? -> attention у Praxis нет источников
|
||||
что у меня в понедельник? -> personal не нашла у тебя такой записи
|
||||
```
|
||||
|
||||
Calendar claimed 3 turns of the 6 that asked about the calendar. That is the
|
||||
same 3/6 the destination fixture scores and the same 3/6 every seed of the
|
||||
routing head scores. Three measurements agree. The cause is the one V-660 named. The possessive
|
||||
agenda rules claim these at stage 0 and name no destination, so the walk
|
||||
reaches `personal` and `attention` first.
|
||||
|
||||
This is the third independent confirmation that the possessive agenda rules
|
||||
should name the calendar. That call is still the owner's.
|
||||
@@ -45,8 +45,8 @@ present when a re-run reaches day 1.
|
||||
**Zero transport errors is not zero wrong answers.** It counts turns that
|
||||
failed to return a reply, and none did. Every quality number is below.
|
||||
|
||||
Those seven strings appear in 49 of 140 turns. Some turns carry two, because a
|
||||
parked clarify appends to whatever else was said.
|
||||
Those seven strings appear 49 times across 41 of 140 turns. Some turns carry
|
||||
two, because a parked clarify appends to whatever else was said.
|
||||
|
||||
The 33.7s outlier is one note write on day 2. p95 improved against the week
|
||||
despite it.
|
||||
|
||||
@@ -0,0 +1,110 @@
|
||||
# CrisperWhisper 2.0 in Russian, measured
|
||||
|
||||
Date: 2026-08-09. Vikunja V-665.
|
||||
Corpus: `bond005/sberdevices_golos_10h_crowd`, test split, first 200 clips.
|
||||
Harness: `~/Programs/cw2-eval` on workpc, not in this repo.
|
||||
Runner: `./.venv/bin/python run_asr.py <arm>...` then `score.py`.
|
||||
|
||||
The model card benchmarks disfluency F1 in German and English. It never names
|
||||
Russian and publishes no per-language WER. So the measurement came before the
|
||||
wiring.
|
||||
|
||||
## The corpus
|
||||
|
||||
200 clips, 13.7 minutes, 1001 reference words. Median clip 3.91s, range 1.04s
|
||||
to 13.5s. Golos crowd is short crowd-sourced Russian spoken close to the
|
||||
microphone, which is the nearest public thing to someone talking to Maven. The
|
||||
alternatives are read speech, which flatters every model equally.
|
||||
|
||||
Two rows carry a null transcription and are skipped.
|
||||
|
||||
Scoring normalizes both sides: lowercase, `ё` to `е`, punctuation stripped, and
|
||||
digits expanded to Russian words through num2words. Without that last step a
|
||||
model is penalized for writing `60000` where the reference says
|
||||
`шестьдесят тысяч`. Thousands separators are joined before expansion, or
|
||||
`60 000` expands to `шестьдесят ноль`.
|
||||
|
||||
## Headline
|
||||
|
||||
| arm | WER | CER | exact | empty | RTF |
|
||||
|---|---|---|---|---|---|
|
||||
| cw2-turbo-intended | **10.4%** | 3.4% | 65.5% | 0 | 0.065 |
|
||||
| cw2-turbo-verbatim | 10.8% | **3.1%** | **66.5%** | 0 | 0.065 |
|
||||
| whisper-turbo | 11.8% | 4.1% | 64.0% | 0 | 0.031 |
|
||||
| cw2-large-intended | 12.3% | 3.8% | 63.5% | 0 | 0.107 |
|
||||
| whisper-small | 27.5% | 9.8% | 35.0% | 0 | 0.026 |
|
||||
|
||||
`whisper-small` is the floor, because `ggml-small.bin` is what mavsttd loads on
|
||||
homesrv today. CW2 turbo beats it by 17 points of WER and takes exact matches
|
||||
from 35.0% to 65.5%.
|
||||
|
||||
Two results are worth naming beyond the winner. CW2 turbo beats its own base
|
||||
model, whisper-large-v3-turbo, by 1.4 points. And it beats CW2 large by 1.9
|
||||
points, which inverts what the card implies by calling turbo a degraded draft.
|
||||
No arm returned an empty transcript.
|
||||
|
||||
## Intended and verbatim are closer than the mode names suggest
|
||||
|
||||
The two modes disagree on 70 of the 200 clips before normalization and on 29
|
||||
after it. So the raw difference is mostly casing and punctuation, which
|
||||
normalization removes and which Maven does not read either.
|
||||
|
||||
Verbatim scores worse on WER and better on CER and exact matches. The reason is
|
||||
script, not disfluency:
|
||||
|
||||
```text
|
||||
ref: футбольный матч челси брайтон
|
||||
int: Футбольный матч Chelsea-Брайтон.
|
||||
ver: Футбольный матч Челси Брайтон.
|
||||
```
|
||||
|
||||
Intended writes foreign entity names in Latin script and verbatim
|
||||
transliterates them. Golos references are Cyrillic throughout, so verbatim
|
||||
collects the exact matches. That is a property of this corpus rather than a
|
||||
quality difference.
|
||||
|
||||
**This corpus cannot settle the mode choice.** Golos crowd is clean short
|
||||
commands with almost no disfluency. The two modes have nothing to disagree
|
||||
about here. They separate on spontaneous speech with fillers, restarts and
|
||||
repairs, which is what the owner speaks. Intended stays the choice for the
|
||||
reason it was always the choice. Maven wants what was meant, not every stumble
|
||||
on the way there.
|
||||
|
||||
The Latin-script habit is the one finding here that touches routing. The
|
||||
routing heads were trained on Cyrillic utterances, so an entity name arriving
|
||||
in Latin script is out of distribution for them. Nothing measures that yet.
|
||||
|
||||
## The runtime is workpc, because whisper.cpp cannot load CW2
|
||||
|
||||
`num_languages()` in `deps/whisper.cpp/src/whisper.cpp` derives the language
|
||||
count from the vocabulary size:
|
||||
|
||||
```cpp
|
||||
return n_vocab - 51765 - (is_multilingual() ? 1 : 0);
|
||||
```
|
||||
|
||||
CW2 carries 31 extra tokens, so `n_vocab` is 51897 and this yields 131
|
||||
languages. The derived `dt` offset becomes 33 and shifts seven special token
|
||||
ids, including `token_beg` and `token_transcribe`. The architecture is
|
||||
otherwise byte-identical to whisper-large-v3-turbo, and the new tokens sit
|
||||
above every whisper special id.
|
||||
|
||||
So loading CW2 in whisper.cpp is a patch to a vendored dependency, not a port.
|
||||
It was not taken, because STT is moving to workpc anyway under V-486. CW2 turbo
|
||||
becomes the preferred remote and `ggml-small.bin` on homesrv stays the floor,
|
||||
which is the shape `modelSeam` already uses for routing and replies. The 27.5%
|
||||
floor is what a turn falls back to when the workstation is down, and this table
|
||||
is what that costs.
|
||||
|
||||
## License
|
||||
|
||||
Standard CW2 weights carry `nyra-health-non-commercial-research`. The Pro
|
||||
variants are commercial-license only. Maven is personal and self-hosted, so the
|
||||
standard weights are usable and the Pro ones are not free to take.
|
||||
|
||||
## What is not measured
|
||||
|
||||
Disfluent spontaneous speech, which is the whole reason to prefer Intended.
|
||||
Long-form audio beyond 13.5s. Far-field or noisy microphones. English, which
|
||||
Maven also speaks. The ONNX turbo export, which was never run, since the
|
||||
transformers path already meets the latency budget at RTF 0.065.
|
||||
@@ -0,0 +1,61 @@
|
||||
# gemma-4-E4B on the phrasing and talk fixtures
|
||||
|
||||
Date: 2026-08-09. Box: workpc up, E4B loaded on 8080.
|
||||
`MAVEN_LLM_URL=http://192.168.1.105:8080 make eval-phrasing`.
|
||||
|
||||
This was the one unmeasured risk of the 2026-08-09 model swap. Routing was
|
||||
measured the same day and E4B lost four destination cases to the 12B. Phrasing
|
||||
was not measured at all, and phrasing is the half the owner hears.
|
||||
|
||||
## Result
|
||||
|
||||
| fixture | E4B | resident Qwen3-1.7B, 2026-08-05 |
|
||||
|---|---|---|
|
||||
| nudges | 15/15 (100%) | 15/15 (100%) |
|
||||
| talk, passes every check | **29/36 (80.6%)** | 25/36 (69.4%) |
|
||||
| lang | 36/36 | — |
|
||||
| feminine | 36/36 | 36/36 |
|
||||
| address | **36/36** | 33/36 |
|
||||
| ontopic | 29/36 | 28/36 |
|
||||
| p50 latency | **516ms** | 2.97s |
|
||||
| p95 latency | 921ms | — |
|
||||
| failed generations | 0 | 0 |
|
||||
|
||||
E4B beats the homesrv floor by four cases and answers about six times faster.
|
||||
Persona is clean: `lang`, `feminine` and `address` are perfect, and `address`
|
||||
is where the resident model still loses three. The 2026-08-05 measurement of the
|
||||
resident model is the comparison, since both ran the same 36-case fixture.
|
||||
|
||||
Every failure is `ontopic`. Nothing failed on persona, nothing failed to parse.
|
||||
|
||||
## The score is at the ceiling, not below it
|
||||
|
||||
The 2026-08-05 temperature sweep found two cases that fail at every temperature
|
||||
in every run: `reply-note-router` and `reply-fact-weight`. It named a defect in
|
||||
the reply phrasing path rather than sampling noise. It put the fixture's ceiling
|
||||
at 30/36 before persona is scored. Both cases are in E4B's failure list.
|
||||
|
||||
So 29/36 is one case off a ceiling nothing about the model can move. The swap is
|
||||
safe on phrasing. Read this next to the routing result, not instead of it. There
|
||||
E4B costs four destination cases and buys 50ms. Here it costs nothing.
|
||||
|
||||
## Two findings no check caught
|
||||
|
||||
**She says she wrote something down when she did not.** Asked what to do this
|
||||
evening, E4B writes "Я записала несколько идей!". Asked for a joke, it writes
|
||||
"Я записала одну забавную ситуацию!". Nothing was stored. No check scores it,
|
||||
because `ontopic` reads the subject and `cringe` reads pet names. A claim to
|
||||
have saved something is a claim about state, and it is wrong.
|
||||
|
||||
**Two `ontopic` failures look like check defects.** `know-dont-know` wants
|
||||
"не зна" or "не мог". It got "Я не умею знать личную информацию о твоих
|
||||
соседях", which declines correctly in words the check does not list.
|
||||
`know-hiccups` is the same shape. Neither is a model failure and both count
|
||||
against the score.
|
||||
|
||||
## Not measured here
|
||||
|
||||
A 12B control on the same fixture, which would need the card reloaded and is the
|
||||
owner's call. The talk fixture through the daemon rather than through the
|
||||
phraser directly. The CPT'd Qwen3-1.7B, which does not exist yet and is the
|
||||
reason `address` is a check at all.
|
||||
@@ -0,0 +1,51 @@
|
||||
# gemma-4-E4B against gemma-4-12B on the routing fixture
|
||||
|
||||
*Measured 2026-08-09 on workpc. The owner asked for the swap. This is what it costs.*
|
||||
|
||||
Both arms ran the same 96-case fixture through `TestLLMRouterBaseline`, minutes
|
||||
apart, against the same llama-server build and the same mavgpud. The 12B arm is a
|
||||
control run and not the 2026-08-02 number. That one predates five fixture cases,
|
||||
the destination labels and a llama.cpp upgrade.
|
||||
|
||||
| | full | intent-only | destination | p50 | p95 |
|
||||
|---|---|---|---|---|---|
|
||||
| gemma-4-12B-it-qat-UD-Q4_K_XL, MTP draft | 81/96 (84.4%) | 91.7% | 23/33 (69.7%) | 344ms | 471ms |
|
||||
| gemma-4-E4B-it-qat-UD-Q4_K_XL | 80/96 (83.3%) | 89.6% | 19/33 (57.6%) | 294ms | 562ms |
|
||||
|
||||
E4B costs one case of full accuracy, two of intent and **four of destination**,
|
||||
and buys 50ms at p50. Read the destination column as the finding. One case is
|
||||
three points on 33. So 23 against 19 is outside the noise a single case makes,
|
||||
and the other two columns are not.
|
||||
|
||||
Both arms produce three false clarifies and one missed clarify, and neither
|
||||
errored on any case.
|
||||
|
||||
## What E4B loses
|
||||
|
||||
Four of the five destination regressions are the same shape: it names nothing
|
||||
where the 12B names `recall` or `calendar`. `ru-query-015` ("сколько я прошёл
|
||||
шагов") goes further and names `self`. Naming nothing is the safe direction,
|
||||
because `SourceUnknown` walks the whole chain, so these turns are still answered.
|
||||
They cost latency and they are what a fourth head is meant to fix (V-546).
|
||||
|
||||
Two Russian intent cases regress, both with the interrogative off the front.
|
||||
`ru-chat-003` ("расскажи анекдот про программистов") goes to `query`.
|
||||
`ru-fact-003` ("поужинал") goes to `chat`.
|
||||
|
||||
## MTP
|
||||
|
||||
E4B has none, and there is no way to give it any on this box. MTP on workpc is
|
||||
a separate gguf of architecture `gemma4-assistant` carrying
|
||||
`nextn_predict_layers=4`, and `mtp-gemma-4-12B-it-BF16.gguf` is the only one on
|
||||
disk. Its head is trained against the 12B's hidden states, so it cannot drive an
|
||||
E4B target. Scanning both target ggufs finds no `nextn` tensors in either, so
|
||||
neither model self-speculates.
|
||||
|
||||
So the 12B arm above ran with speculative decoding and E4B ran without, and E4B
|
||||
was still faster.
|
||||
|
||||
## Cost on the card
|
||||
|
||||
E4B is 4.2GB against 6.7GB plus a 0.86GB draft. With CW2 resident at 1.6GB that
|
||||
is 5.8GB of 16GB against 9.2GB. Nothing in Maven needs the difference, so this is
|
||||
headroom for the owner's own jobs rather than a capability.
|
||||
@@ -0,0 +1,113 @@
|
||||
# Kiwix answered the wrong question, and the fix was not a relevance gate
|
||||
|
||||
Date: 2026-08-09. Task: V-668. Box: homesrv, workstation off.
|
||||
Book: `wikipedia_ru_all_maxi_2026-02` on `127.0.0.1:8034`.
|
||||
|
||||
## What started it
|
||||
|
||||
Two turns on 2026-08-09 came back wrong from the offline encyclopedia.
|
||||
"почему небо голубое" was answered off the song "Город золотой". "что такое
|
||||
TCP?" was answered off "Перехват TCP-соединения". Both were phrased
|
||||
confidently, because `queryKiwix` claims a turn whenever the search returns
|
||||
anything and `len(hits) == 0` is its only gate.
|
||||
|
||||
The plan was a relevance gate. multilingual-e5-small is asymmetric and trained
|
||||
for exactly this, `query:` against `passage:`, and the query vector is already
|
||||
held on the turn. The 2026-08-05 measurement that killed a search-quality gate
|
||||
killed three lexical signals. It says in its own words that it never probed
|
||||
Kiwix.
|
||||
|
||||
## The gate does not exist
|
||||
|
||||
Fourteen Russian questions, eight the encyclopedia can answer and six it
|
||||
cannot. Each question was searched, the top article read, and the cosine of
|
||||
`EmbedQuery(question)` against `EmbedPassage(article)` recorded.
|
||||
|
||||
| set | n | min | mean | max |
|
||||
|---|---|---|---|---|
|
||||
| answerable | 8 | 0.7934 | 0.8400 | 0.9087 |
|
||||
| not answerable | 6 | 0.7480 | 0.7852 | 0.8367 |
|
||||
|
||||
Two of the six unanswerable score above the weakest answerable one. That alone
|
||||
would be a poor threshold. The log killed it outright: seven of the eight
|
||||
answerable questions got a **wrong** article back, and those wrong articles
|
||||
scored high. The TCP hijacking article scored 0.8653, above five of the six
|
||||
unanswerable rows.
|
||||
|
||||
The finding is that this cosine measures topic and not answerhood. A page about
|
||||
hijacking TCP sessions is about TCP. No threshold separates it from a page that
|
||||
defines TCP, and one that tried would take the definition with it.
|
||||
|
||||
## The defect is retrieval
|
||||
|
||||
`internal/kiwix/client.go` has said it since it was written: ranking is keyword
|
||||
based, "why is the sky blue" finds a TV episode. `queryKiwix` sends the whole
|
||||
sentence. The English path has a rewriter that reduces a question to keywords
|
||||
with a model call. The Russian path reads the book verbatim (V-508) and had
|
||||
nothing. So the question words compete with the one word that names the article.
|
||||
|
||||
Dropping the question words changes the answer:
|
||||
|
||||
| sent | first hit |
|
||||
|---|---|
|
||||
| `кто написал Войну и мир` | Радуйся, мир (Доктор Кто) |
|
||||
| `Война и мир` | Война и мир |
|
||||
| `что такое TCP` | Перехват TCP-соединения |
|
||||
| `TCP` | TCP |
|
||||
|
||||
A ZIM is also addressable by title, which nothing here used. `/A/Франция`,
|
||||
`/A/TCP` and `/A/Небо` are 200. `/A/Трюмбальная_нидроскопия` is 404. So an
|
||||
exact title is safe to try first: it either answers or costs one request that
|
||||
says nothing.
|
||||
|
||||
The title has to carry its capital. `/A/фотосинтез` is a 404 and
|
||||
`/A/Фотосинтез` is a 200. The spoken form is tried first anyway, so a title
|
||||
that begins lowercase on purpose keeps its chance.
|
||||
|
||||
## What shipped, measured
|
||||
|
||||
`kiwix.Topic` drops the narrative request, the interrogative and a verb sitting
|
||||
behind one. It keeps everything else, because a word it cannot classify is more
|
||||
likely the topic than noise. `kiwix.TitlePath` tries the exact article before
|
||||
any ranking runs. Both apply on the verbatim path only, since reducing twice
|
||||
would take the topic off the rewriter's input.
|
||||
|
||||
| question | before | after |
|
||||
|---|---|---|
|
||||
| что такое TCP? | Перехват TCP-соединения | **TCP** (by title) |
|
||||
| что такое фотосинтез | C4-фотосинтез | **Фотосинтез** (by title) |
|
||||
| кто такой Линус Торвальдс? | Tux | **Торвальдс, Линус** (by title) |
|
||||
| кто написал Войну и мир | Радуйся, мир (Доктор Кто) | **Война и мир** |
|
||||
| столица Франции | Список столиц Олимпийских игр | **Париж** (by title) |
|
||||
| что такое чёрная дыра | Чёрная дыра | Чёрная дыра (by title) |
|
||||
| почему небо голубое | Город золотой | Под небом голубым… (фильм) |
|
||||
| почему трава зелёная | Сено | Зелень |
|
||||
|
||||
Five questions reach the right article where they did not. One was already
|
||||
right and stays right. Nothing regressed.
|
||||
|
||||
"столица Франции" is the surprise. The 2026-08-05 measurement named it as the
|
||||
case a quality gate must not break, because the answer is Париж and that word
|
||||
is not in the question. The ZIM holds a title redirect, so asking for the
|
||||
article titled "Столица Франции" returns Париж. Retrieval by title reaches an
|
||||
answer that retrieval by keyword cannot.
|
||||
|
||||
## What is still wrong
|
||||
|
||||
Two of the eight are still not answered, and both are the same shape. The
|
||||
question names no article and no redirect covers it. "почему небо голубое" is
|
||||
answered by Rayleigh scattering, and nothing in the question says so. Keyword
|
||||
retrieval cannot bridge that and neither can a threshold. The candidates are a
|
||||
semantic index over titles, or asking the resident model for the article title
|
||||
rather than for keywords.
|
||||
|
||||
`Response.Empty()` is still the whole gate. A wrong article that the search
|
||||
does return is still spoken. What this change buys is that the article is
|
||||
usually right, not that a wrong one is caught.
|
||||
|
||||
## Not measured here
|
||||
|
||||
The English path, which still goes through the rewriter and was not touched.
|
||||
SearXNG, where the same question about answerhood is open and the 2026-08-05
|
||||
result stands. The cascade end to end, since the workstation is off and the
|
||||
phrasing arm is the resident model.
|
||||
@@ -0,0 +1,63 @@
|
||||
# silero-vad against the energy threshold in mavwaked
|
||||
|
||||
*Measured 2026-08-09 on homesrv. V-487, stage one of two.*
|
||||
|
||||
mavwaked decided an utterance had started by comparing frame energy to an
|
||||
adaptive floor. That answers "is this frame loud". A fan, a door and a
|
||||
television are all loud, and every utterance mavwaked accepts becomes a turn.
|
||||
|
||||
silero-vad answers "is this frame speech". It is 2.3MB of ONNX and it replaces
|
||||
the comparison and nothing else. The speech hold, the silence hold, the length
|
||||
cap and the utterance buffer are the same state machine either way.
|
||||
|
||||
## What it declines
|
||||
|
||||
Speech is the four piper fixtures `mavsttd` already scores against, so nothing
|
||||
of the owner's voice is committed. Non-speech is white noise at the same RMS as the clip beside it. That is the
|
||||
cheapest thing that fools an energy floor.
|
||||
|
||||
| clip | silero, speech frames on speech | silero on noise | energy on noise |
|
||||
|---|---|---|---|
|
||||
| ru_fact.wav | 59 | 0 | 68 |
|
||||
| ru_query.wav | 69 | 0 | 79 |
|
||||
| ru_reminder.wav | 80 | 0 | 89 |
|
||||
| en_act.wav | 90 | 0 | 99 |
|
||||
|
||||
The energy threshold accepts every noise clip as a complete utterance. Silero
|
||||
calls not one frame of any of them speech, and still hears all four spoken
|
||||
clips. `TestSileroHearsSpeechAndDeclinesNoise` is that table.
|
||||
|
||||
White noise is a floor, not a proof. It says nothing about a television, which
|
||||
is speech, or about a fan, which is narrowband. Those need room recordings and
|
||||
this box has none.
|
||||
|
||||
## What it costs
|
||||
|
||||
`BenchmarkSileroFrame` on the homesrv laptop (Ryzen 5 5600U), one 30ms frame
|
||||
through the model including the re-chunking:
|
||||
|
||||
509µs per frame
|
||||
|
||||
That is 1.7% of one core, on the slower of the two machines. The detector runs
|
||||
on the workstation beside the microphone, never on the GPU. This number is what
|
||||
says it does not need one.
|
||||
|
||||
## The window is 512 samples, not 480
|
||||
|
||||
`cmd/mavwaked/main.go` claimed the frame contract matched silero's input
|
||||
exactly. That was true of silero v4. Version 5 takes exactly 512 samples at 16kHz, plus 64 samples of context from
|
||||
the previous window. So `sileroVAD` buffers across capture frames, and a frame
|
||||
completing no window inherits the previous probability. `TestSileroRechunksAcrossFrames` pins it.
|
||||
|
||||
## Still an energy gate by default
|
||||
|
||||
`-vad-model` is empty in the code default, so a deployment that does not pass
|
||||
it runs exactly what shipped before. Barge-in is untouched and deliberately so. It reads frame energy while she is
|
||||
speaking, which is a different question from whether the frame is speech.
|
||||
|
||||
## Not done here
|
||||
|
||||
The wake word. This is stage one of the two V-487 asks for. The second needs a
|
||||
keyword model that does not exist yet. The pretrained openWakeWord keywords are
|
||||
English, and a Russian one has to be trained. Until then anything spoken near
|
||||
the microphone still becomes a turn. It is now merely required to be speech.
|
||||
@@ -0,0 +1,112 @@
|
||||
# The "Мэйвен" wake word: what it hears and what it invents
|
||||
|
||||
*Measured 2026-08-09 on workpc and homesrv. V-487, stage two of two.*
|
||||
|
||||
Stage one gave mavwaked silero-vad, which answers "is this frame speech".
|
||||
Nothing answered "was this said to her", so every utterance near the
|
||||
microphone became a turn. SurfaceVoice caps acts at L0, which made that safe
|
||||
rather than expensive. L0 does not cap reading, so the room could still hear
|
||||
his facts read back.
|
||||
|
||||
The keyword is "Мэйвен". openWakeWord's two frozen feature models do the
|
||||
hearing and a 100KB head trained here draws the boundary. It runs on CPU
|
||||
beside silero and never touches the GPU.
|
||||
|
||||
## Why a per-window accuracy is not a number anyone can act on
|
||||
|
||||
The gate scores every 80ms. A 1.7% false-accept rate per window sounds small
|
||||
and means a wake every few seconds. The useful question is how many times an hour
|
||||
it wakes on speech that was not the keyword. So every table below counts
|
||||
threshold crossings over whole clips and divides by the audio duration.
|
||||
|
||||
A crossing, not a window above the threshold. A keyword held high for half a
|
||||
second is one wake, not six.
|
||||
|
||||
## The data
|
||||
|
||||
Positives are 600 silero TTS renders of three stressings of the keyword, six
|
||||
speakers, ten trailing phrases, augmented eight ways each. Hard negatives are
|
||||
560 renders of confusable Russian words. Real speech is Common Voice ru and Golos.
|
||||
The 74257 Common Voice clips were already on workpc from the CrisperWhisper
|
||||
work. The 200 Golos clips came from the CW2 WER eval.
|
||||
|
||||
Splits are by source file. Augmented copies of one render on both sides of a
|
||||
split would measure memorisation.
|
||||
|
||||
Golos was never trained on at any stage, so it answers the harder question:
|
||||
does this survive a change of speakers and rooms.
|
||||
|
||||
## Three heads
|
||||
|
||||
Each row is a full retrain. The false-accept column is 8.89 hours of Common
|
||||
Voice that no stage of training had seen.
|
||||
|
||||
| trained on | recall (window) | false wakes/hour @0.99 |
|
||||
|---|---|---|
|
||||
| TTS + 13.7 min of Golos | 0.869 | not measurable |
|
||||
| + 4000 Common Voice clips | 0.836 | 21.9 |
|
||||
| + 3837 mined hard negatives | 0.784 | 4.2 |
|
||||
| + 753 more mined | 0.810 | 3.4 |
|
||||
|
||||
The first row is why the second exists. Thirteen minutes of held-out speech
|
||||
cannot measure a rate for a gate that scores twelve times a second. A head
|
||||
trained only against TTS learns to tell TTS from not-TTS.
|
||||
|
||||
Mining is the whole story after that. Random negatives teach the head what
|
||||
most speech sounds like. They do not teach it the few syllable sequences that
|
||||
score high, because 4000 clips barely contain them. So the current head was
|
||||
run over 20000 fresh clips, keeping every window it scored above 0.05. That
|
||||
found 3837 windows in 855512. Repeating those ten times in the next training
|
||||
run cut the rate five-fold.
|
||||
|
||||
The second round found 753 in 852240, a fifth of the yield, and bought a
|
||||
further 20%. It also recovered recall, which the first round had cost. Whether
|
||||
a third round is worth 25 minutes of workpc is untested.
|
||||
|
||||
## Where the threshold came from
|
||||
|
||||
Both columns are held out. Positives are the 126 renders in the test split.
|
||||
Speech is 65.1 minutes of Common Voice, disjoint from every training and
|
||||
mining pool. Both were run through the built `mavwaked` binary reading PCM from a
|
||||
file, not through the python that trained the head.
|
||||
|
||||
| threshold | renders shipped | false wakes/hour |
|
||||
|---|---|---|
|
||||
| 0.99 | 116 / 126 | 2.8 |
|
||||
| 0.999 | 115 / 126 | 0.9 |
|
||||
|
||||
One render against a third of the false wakes. `defaultWakeThreshold` is
|
||||
0.999.
|
||||
|
||||
Golos disagrees. It gave 2 wakes in 14 minutes at every threshold, which is
|
||||
8.7 per hour. Two events is not a rate. What it does say is that a handful of real utterances score above 0.999
|
||||
and no threshold will move them.
|
||||
|
||||
## What it costs him
|
||||
|
||||
Ten of the 126 held-out renders were heard and still dropped, and every one
|
||||
was an utterance shorter than 1.32s. The head scores 16 embeddings, or 1.28s of
|
||||
audio. The score therefore peaks up to a second after a short keyword ends.
|
||||
By then the VAD has closed the utterance and dispatch has already asked.
|
||||
|
||||
Real commands are "Мэйвен, <request>" and run past two seconds, which gives
|
||||
the head the whole request to peak during. A bare "Мэйвен" with nothing after
|
||||
it is the case that fails. One fix would hold an ignored utterance for a grace
|
||||
period and ship it if the keyword lands late. It is not built.
|
||||
|
||||
## What it costs the workstation
|
||||
|
||||
Under systemd on workpc, mavwaked sat at 335% of a core with the gate on and
|
||||
68% with only silero. onnxruntime sizes its thread pool to every core and spins
|
||||
between runs, and this gate runs three graphs twelve times a second. Pinning
|
||||
all three sessions to one thread brought it to 81%, so the keyword costs about
|
||||
13% of one core. The three graphs each finish in well under 80ms that way.
|
||||
|
||||
## What was not measured
|
||||
|
||||
No room recordings. Every negative above is a clean corpus clip. This gate
|
||||
will live among a television, a fan and the far side of a kitchen. None of
|
||||
those are in these numbers.
|
||||
|
||||
No measurement of him. Training on his voice means copying his transcripts off
|
||||
homesrv, which is his call and has not been asked.
|
||||
@@ -0,0 +1,74 @@
|
||||
# Language: what the model emits, and how Russian is matched
|
||||
|
||||
*Last verified: 2026-08-09 @ a9b480a*
|
||||
|
||||
Two contracts live here. What a model call is allowed to return, and which
|
||||
mechanism is allowed to recognise a Russian word.
|
||||
|
||||
## The LLM output contract
|
||||
|
||||
All phrasing paths emit `{"response":"...","mood":"..."}`. They fall back to
|
||||
plain text when the model skips the JSON.
|
||||
|
||||
**One parser, `parseResponseMood` in `internal/phraser/parse.go`.** Every path
|
||||
reaches it: the six `LLMPhraser` methods, `PhraseWorld`, and
|
||||
`Replier.PhraseReply`. `cmd/mavend/replier_llm.go` wraps the last of those,
|
||||
holds the stub fallback, and parses nothing itself. Mood is a fixed enum.
|
||||
|
||||
The router prompt is a separate contract:
|
||||
|
||||
```text
|
||||
[{"intent":<enum>, key?, value?, text?, verb?}, ...]
|
||||
```
|
||||
|
||||
over 7 intents: `fact, reminder, note, query, act, chat, system`.
|
||||
`llm/check_prompt_parity.py` in the training workspace enforces that the Go
|
||||
prompt and the relabelling prompt stay identical. They diverged once, and the
|
||||
relabelled set then taught a head the Go router never asks for.
|
||||
|
||||
## Russian patterns: three mechanisms, no fourth
|
||||
|
||||
Hand-written Russian stem patterns were swept out on 2026-08-04 by the owner's
|
||||
call. A regex whose output is a fact or a route is the defect. A regex over
|
||||
structured input, such as HTML, MIME, JSON, a URL or an argv list, is not.
|
||||
|
||||
Before writing a Russian word list, pick one of these.
|
||||
|
||||
### `internal/lexicon`, for closed classes
|
||||
|
||||
`lexicon_ru_v1.json` holds interrogatives, capture verbs, reminder verbs,
|
||||
cardinals, day offsets, parts of day, weekdays, months and spoken hours.
|
||||
Editing a word is a data change and there is exactly one copy. Months used to
|
||||
live in three files and drifted between them.
|
||||
|
||||
Cardinals carry the oblique forms, because a spoken time declines. `в семь` and
|
||||
`к семи` are one hour.
|
||||
|
||||
### `internal/morph`, for grammar
|
||||
|
||||
From the vendored golem Russian dictionary. `IsVerbForm` and `SameWord`.
|
||||
|
||||
Lemma matching is BROADER than stem-plus-one-ending. A verb slot that means the
|
||||
imperative must be matched exactly. `говори` and `говорил` share a
|
||||
lemma and only one of them is a command
|
||||
(`cmd/mavend/quiet_toggle.go`).
|
||||
|
||||
### `cmd/mavend/topics.go` and the embedder, for open sets
|
||||
|
||||
Use these when the question is what a turn is ABOUT. Frozen seeds per subject
|
||||
plus a real `other` class, scored against the turn's own query vector.
|
||||
|
||||
Same shape as the personal boundary in `personalboundary.go`, with one
|
||||
difference. A topic must clear the runner-up by `topicMargin`, because a false
|
||||
claim here spends a network scan rather than one honest "не знаю". The old
|
||||
keyword tests stay as the offline floor.
|
||||
|
||||
### The ecosystem trio
|
||||
|
||||
Use it when the answer is not in the utterance at all. Identity is Nexus's,
|
||||
never a local pattern.
|
||||
|
||||
## Seeds are scoring data
|
||||
|
||||
Editing one moves a recogniser. Re-measure against the `TestONNX*` tests rather
|
||||
than eyeballing the change.
|
||||
+38
-3
@@ -1,6 +1,6 @@
|
||||
# Offloading model work to the workstation
|
||||
|
||||
*Last verified: 2026-08-05 @ b789676. Living doc: correct it in place, do not append.*
|
||||
*Last verified: 2026-08-09 @ 50c6637. Living doc: correct it in place, do not append.*
|
||||
|
||||
Owner's call, 2026-08-02. Vikunja #483 is the umbrella. Tasks #484 to #487 are the
|
||||
work, and this file holds the shape and the rules all four must obey.
|
||||
@@ -105,6 +105,17 @@ how we find out whether the blind spot is real.
|
||||
untouched. The model, the context size, the layer count and the MTP flags are the
|
||||
owner's business and not this daemon's schema.
|
||||
|
||||
**Every GPU service on that box belongs under this supervisor**, added to
|
||||
`cmd/mavgpud` rather than to systemd beside it. The rule was learned on
|
||||
2026-08-09. The CW2 transcriber ran as its own user unit and registered on the
|
||||
KFD like any ROCm job. So the supervisor read its own transcriber as a
|
||||
contender. It yielded the card every few seconds and the gemma-4-12b arm was
|
||||
down for eight minutes before anyone looked. So the supervisor takes a `stt`
|
||||
block and starts CW2 itself. Yielding is all or nothing, because a job that
|
||||
wants the card wants all of it. Idle unloading is not. It applies to
|
||||
llama-server, which holds 8GB. CW2 holds 1.6GB, and unloading it would cost the
|
||||
next voice turn its quality for nothing.
|
||||
|
||||
## What stays on homesrv, permanently
|
||||
|
||||
The **embedder** (multilingual-e5-small, ONNX, CPU). It backs the classifier, which
|
||||
@@ -149,6 +160,29 @@ flips. It is wired anyway: `PhraseReminder` is on the same transport and is on.
|
||||
Then the embedder above, **whisper.cpp** in `mavsttd`, and **piper** in `mavttsd`.
|
||||
`mavwaked` uses no model at all: an energy-threshold VAD over 30ms frames.
|
||||
|
||||
Speech-to-text is wired as of 09-08-2026, and it takes only the silent half of the
|
||||
rule. A worse transcript is still a turn, so there is nothing to name a gap about
|
||||
and `stt.Pair` has no `TranscribeRemote`. `sttSeam` in `cmd/mavend/voicewire.go`
|
||||
builds it, beside `modelSeam` and at the same place in `wireVoice`, so the voice
|
||||
path and the meeting recorder still share one transcriber.
|
||||
|
||||
The remote is not a second endpoint on mavgpud. whisper.cpp cannot load
|
||||
CrisperWhisper 2.0 at all. It reads its language count off the vocabulary
|
||||
size, and CW2's 51897 tokens shift seven special token ids. So CW2 runs under
|
||||
transformers as its own service on port 8081, and `stt.HTTPTranscriber` is the
|
||||
second transport for the same seam. It posts raw PCM with the format in headers.
|
||||
It carries a bearer token, because audio is the most sensitive thing that
|
||||
crosses here.
|
||||
|
||||
It is a second endpoint on nothing, but it is a second **child** of mavgpud, and
|
||||
that part is not optional. See the supervisor section above for why: a ROCm
|
||||
service the supervisor does not own is a contender it yields to.
|
||||
|
||||
The margin is the reason: CW2 turbo scores 10.4% WER in Russian against 27.5% for
|
||||
the `ggml-small.bin` mavsttd loads, over 200 Golos clips
|
||||
(`docs/evals/2026-08-09-crisperwhisper2-russian-wer.md`). Text-to-speech has not
|
||||
moved and piper on homesrv is still the only synthesizer.
|
||||
|
||||
Speech-to-text stays two stages when it moves. One call carrying both a clip and the router
|
||||
prompt was measured on 05-08-2026. It scores 54.2% intent-only against 84.7% for whisper on
|
||||
homesrv, on the same 72 cases. The model transcribes clips it then routes wrong, so a long
|
||||
@@ -173,8 +207,9 @@ cleaner transcripts, not accuracy. See `docs/evals/2026-08-05-audio-in-routing.m
|
||||
which fixes what the 1.7B gets wrong: world knowledge, and the persona the CPT
|
||||
targets. The degradation path is already written and measured, since the
|
||||
classifier scores 68.8% full accuracy at p50 16.6µs on its own.
|
||||
3. **Speech-to-text and text-to-speech** (#486). They gain a real margin, but on
|
||||
quality alone, and both already work.
|
||||
3. **Speech-to-text and text-to-speech** (#486). Speech-to-text is wired, see
|
||||
above. Text-to-speech is not, and piper is good enough that nothing argues
|
||||
for moving it yet.
|
||||
4. **The wake word** (#487). Independent of all of the above.
|
||||
|
||||
## Assumptions
|
||||
|
||||
+462
@@ -0,0 +1,462 @@
|
||||
# Routing
|
||||
|
||||
*Last verified: 2026-08-09 @ 31b5093*
|
||||
|
||||
How an utterance becomes a `Decision`, why each stage exists, and what every
|
||||
stage has measured. `CLAUDE.md` carries the rules an agent must not break. This
|
||||
file carries the reasoning and the history behind them.
|
||||
|
||||
Two decisions come out of a route. **Intent** is one of seven values. **Source**
|
||||
is where the answer lives, and it is read on `IntentQuery` alone. They are scored
|
||||
separately, because one number hides which one moved.
|
||||
|
||||
## The cascade
|
||||
|
||||
| Stage | What it is | Where |
|
||||
|---|---|---|
|
||||
| 0 | Deterministic grammars over the utterance | `stage0.go`, `praxis.go`, `worldquery.go` |
|
||||
| 0b | Four ONNX heads on one e5-small forward pass | `heads.go` |
|
||||
| 1 | The resident model, GBNF-constrained JSON | `llmrouter.go` |
|
||||
| 2 | Nearest neighbour over frozen seed phrases | `classifier.go`, `embedder.go` |
|
||||
|
||||
Every stage may decline, and the next one answers. Any error at stage 0b or 1
|
||||
falls through, so a turn never breaks on a model.
|
||||
|
||||
The resident model arm is wired at `voice.go:214` through
|
||||
`pickLLMRouter(cfg.Voice.UseLLMRouter(), llmClient)`. The flag is
|
||||
`voice.llm_router` in `config.go`, `DefaultLLMRouter` is on, and
|
||||
`deploy/mavend.json` sets it `true`. With no llama-server to talk to,
|
||||
`pickLLMRouter` logs that and degrades to the classifier.
|
||||
|
||||
The classifier is the floor and not dead code. It runs when the resident model
|
||||
is off, when there is no llama-server to talk to, and on any per-turn error.
|
||||
Routing by seed similarity is the known cause of weak Russian queries. Deleting
|
||||
it would make a model outage a broken turn.
|
||||
|
||||
### Why there are two engines at all
|
||||
|
||||
The original design was the classifier alone. `docs/rearchitecture.md` replaced
|
||||
it with a model that emits structured JSON. The same weights phrase the reply.
|
||||
That demoted the embedder from a routing gate to a hint for recall. The model
|
||||
became the default on 2026-07-31.
|
||||
|
||||
The gap it buys is smaller than the design assumed. Measured 2026-08-02 on the
|
||||
77-case Russian fixture, the classifier scores 68.8% full accuracy at p50 16.6µs.
|
||||
Qwen3-1.7B scores 72.7% through the cascade. Four points, not a doubling.
|
||||
|
||||
An older figure of 36.8% for the classifier stood in `CLAUDE.md` until then. It
|
||||
predates the stage 0 rules and the seed additions. Both now score inside the
|
||||
classifier baseline.
|
||||
|
||||
Latency was misreported the same way. A figure of 2.7 seconds stood for two
|
||||
days and was contention rather than the model.
|
||||
`docs/evals/2026-07-31-routing.md` line 61 measures the router at p50 825ms and
|
||||
the cascade at p50 0.80s to 1.04s.
|
||||
|
||||
## Numbers
|
||||
|
||||
Three arms answer, so three numbers are live. Judge a routing change against the
|
||||
classifier and the resident model, since those are what always answer.
|
||||
|
||||
| Arm | Intent | Destination | p50 | Measured |
|
||||
|---|---|---|---|---|
|
||||
| classifier + ONNX | 76.0% (73/96) | 36.4% (12/33) | 16.6µs | 2026-08-08 |
|
||||
| resident Qwen3-1.7B, cascade | 80.2% | not measured | 1.19s | 2026-08-05 |
|
||||
| routing heads, cascade | 96.9% | 75.8% | 27.9ms | 2026-08-08 |
|
||||
| gemma-4-12b, cascade | 84.4% | 72.7% | 329ms | 2026-08-02 |
|
||||
| gemma-4-E4B, cascade | 89.6% | 57.6% (19/33) | 294ms | 2026-08-09 |
|
||||
|
||||
The fixture grew from 77 cases to 91 to 96. So a number is comparable only to
|
||||
another number on the same fixture. Sources:
|
||||
`docs/evals/2026-08-05-routing-resident-model.md`,
|
||||
`docs/evals/2026-08-02-workstation-gemma4-12b.md`,
|
||||
`docs/evals/2026-08-09-e4b-vs-12b-routing.md`,
|
||||
`docs/evals/2026-08-08-routing-heads-in-go.md`,
|
||||
`docs/evals/2026-08-08-destination-fixture.md`.
|
||||
|
||||
The resident model alone scores 37.4% full against 61.5% intent-only. The gap is
|
||||
slots and not routing. It routes `reminder` and leaves the time to the daemon,
|
||||
which is what the contract asks.
|
||||
|
||||
To re-run the resident model as router, start a **second** llama-server on a
|
||||
fixed host port. The resident one binds `--port 0` inside the container and no
|
||||
host process can reach it.
|
||||
|
||||
### The workstation is not the better router any more
|
||||
|
||||
It was, from 2026-08-02 until the heads landed. gemma-4-12b beat everything on
|
||||
the box at 84.4% intent and 72.7% destination. The heads beat it on both at a
|
||||
twelfth of the latency. The workstation stays the better phraser.
|
||||
|
||||
E4B replaced the 12B on 2026-08-09 by the owner's call. It is a step down on
|
||||
routing. Against a same-session 12B control it costs four destination cases and
|
||||
buys 50ms. Read destination as the finding. It names nothing where the 12B names
|
||||
`recall` or `calendar`, which is safe but walks the whole chain. It has no MTP
|
||||
and cannot be given any here. The only `gemma4-assistant` draft on disk is
|
||||
trained against the 12B's hidden states.
|
||||
|
||||
## Stage 0: what the grammars claim, and why
|
||||
|
||||
A rule at this stage is a claim. Either the model gets this wrong, or it wastes a
|
||||
second getting it right. Every rule was added against a measurement.
|
||||
|
||||
- **Agenda questions** (`AgendaQueryGrammars`, 2026-08-01). "что у меня сегодня",
|
||||
"во сколько у меня встреча" and anything naming a calendar go to `IntentQuery`.
|
||||
They were going to `IntentSystem`, where `replySystem` has no agenda arm and
|
||||
answered "пока не умею". Worth 2.6 points of full accuracy and calendar 0/2 to
|
||||
2/2.
|
||||
- **Rest of day and narrative** (V-498, 2026-08-04). `rest-of-day-query` claims
|
||||
"что дальше?". `NarrativeQueryGrammar` claims "расскажи про X", "объясни X" and
|
||||
"опиши X". Neither carries a question mark or an interrogative, so the model
|
||||
called both `IntentFact`. `IsQuestionShaped` caught the write downstream, so
|
||||
this was a latency and fixture defect rather than a correctness one. The
|
||||
narrative rule declines `chatNarrativeTopics`, because the query chain has no
|
||||
source that answers a joke or a bedtime story.
|
||||
- **Praxis** (V-516, 2026-08-05). `PraxisGrammars()` fills `Slots.Fn` with a
|
||||
capability name. These grammars are the **only** path to Praxis and not a
|
||||
faster one. The model reaches Praxis 0/12 alone, the same as the classifier.
|
||||
Nothing in the router prompt names a Praxis capability, so there is no string
|
||||
for it to write. Through the cascade it is 11/12. Measured overall 16/30 to
|
||||
27/30, lifecycle 0/5 to 5/5
|
||||
(`docs/evals/2026-08-05-praxis-reach.md`,
|
||||
`docs/evals/2026-08-05-reach-llm-router.md`).
|
||||
`handlePraxisAct` compares `Slots.Fn` to a capability alias. Otherwise that
|
||||
slot is filled from the deployment's enabled tool names, and no Praxis alias
|
||||
is on that list.
|
||||
- **World questions** (`WorldQueryGrammars`, V-655, 2026-08-07). "что такое X"
|
||||
and "сколько будет 17 на 23". Wired after the agenda rules and **before** the
|
||||
feed and list rules. "что такое лента" is a definition question, and the feed
|
||||
rule would take it on the noun alone.
|
||||
|
||||
`calendar-query` and `event-time-query` name the calendar as the destination.
|
||||
The possessive agenda rules deliberately do not. "что у меня в списке покупок"
|
||||
matches `agenda-query`, and naming the calendar there would take the list source
|
||||
off the turn. That caution now costs four destination cases. See the model arm
|
||||
below.
|
||||
|
||||
Go's `\b` is ASCII-only and never fires after a Cyrillic letter. A pattern needs
|
||||
an explicit `(\s|[?!.]|$)`.
|
||||
|
||||
`baselineGrammars` in `eval_test.go` mirrors `buildRouter` and has drifted before.
|
||||
`WorldQueryGrammars` was wired into the daemon by V-655 and not into the mirror,
|
||||
so the fixture scored a grammar set nobody runs. Fixed by V-659, worth 3 points
|
||||
of destination.
|
||||
|
||||
### Praxis lifecycle rules
|
||||
|
||||
A **stative** lifecycle word ("готово", "принято") needs an item named beside it.
|
||||
A bare **imperative** ("закрывай") may ask which one. It also requires a sentence
|
||||
naming no object of its own. Otherwise "закрой шторы в комнате" goes to Praxis
|
||||
instead of the house. A demonstrative ("отметь это как сделанное") resolves
|
||||
against `h.surfacedItems` only when exactly one item was spoken. Otherwise the
|
||||
turn goes back to the cascade rather than transitioning the wrong item.
|
||||
|
||||
### Slots on a stage 0 decision
|
||||
|
||||
`fillMatchedSlots` runs the stage 2 extractor over whatever a grammar built
|
||||
(V-572, 2026-08-06). It fills only the slots the grammar left empty. A matched
|
||||
value always wins, because the rule read a literal pattern and the extractor
|
||||
guesses.
|
||||
|
||||
It did not run before. So `ReminderGrammar` handed the daemon `HasTime: false`
|
||||
for "напомни в 11:00 позвонить маме", and `missingFor` read the silence as
|
||||
absence and asked "Когда?". It is inert for every grammar but the reminder:
|
||||
`Extract` fills Time, Fn and Key and nothing else. A stage 0 query costs 3.7µs
|
||||
against 3.9µs before, benchmarked at 20000x.
|
||||
|
||||
`Slots.Text` is deliberately not filled. A grammar that left it empty meant it,
|
||||
and `agendaQueryBuild` hands the query chain the utterance itself.
|
||||
|
||||
## Stage 0b: the routing heads
|
||||
|
||||
Routing has a bounded output space, so it is classification rather than
|
||||
generation (owner's call, V-546,
|
||||
`docs/plans/18-routing-heads-on-e5-small.md`). The 118M multilingual-e5-small is
|
||||
already resident. A softmax cannot emit a value that does not exist, so no
|
||||
grammar is needed. Max softmax is a calibratable confidence, where
|
||||
`Confidence: 1.0` was a hardcode. Training costs roughly 5e15 FLOPs, so 10 to 30
|
||||
minutes on the workstation. A 100M decoder from scratch is 10 to 20 GPU hours.
|
||||
|
||||
**Fine-tune a copy of the weights.** The resident embedder backs memory recall.
|
||||
Training it in place couples routing accuracy to recall@1, with nothing in the
|
||||
suite to name the trade.
|
||||
|
||||
Four heads share one masked mean pool, trained over three days. The measurements
|
||||
are `docs/evals/2026-08-08-routing-heads-two-head.md`,
|
||||
`docs/evals/2026-08-08-slot-head-three-head.md`,
|
||||
`docs/evals/2026-08-08-clarify-head-four-head.md` and
|
||||
`docs/evals/2026-08-08-massive-warm-start.md`.
|
||||
|
||||
| Head | Score | Notes |
|
||||
|---|---|---|
|
||||
| intent | 92.8% mean over 3 seeds | fixture is the 88 cases carrying an intent |
|
||||
| destination | 80.8% mean, best 29/33 | beats the 12B teacher it was distilled from |
|
||||
| slot BIO tags | 72.4% span F1 | still climbing when epoch selection stops it |
|
||||
| clarify | catches 7.0 of 8, 2.3 false of 88 | parity with the cascade, no rules in front |
|
||||
|
||||
Read the best destination run as one seed and not a headline. One case is 3
|
||||
points on a fixture this small. Head intent accuracy is **not** comparable to the
|
||||
cascade's 76.0% and 84.4%. A softmax has no clarify class, so the head's fixture
|
||||
is 88 cases and not 96.
|
||||
|
||||
Recall is 15/15 and world is 5/5.
|
||||
|
||||
**Mood is cut, not deferred.** The enum describes her own reply state, not the
|
||||
speaker's emotion, and no dataset maps onto it.
|
||||
|
||||
### The clarify head
|
||||
|
||||
Clarify is not a value of intent, so a softmax cannot emit it. It is a second
|
||||
question over the same pooled vector: can Maven act on this at all. Accuracy is
|
||||
the wrong number here and a head that never asks scores 91.7%.
|
||||
|
||||
Confidence is the other half. Max softmax over the intent head reads 0.851 where
|
||||
it is right and 0.604 where it is wrong. It ranks right above wrong in 83.4% of
|
||||
pairs.
|
||||
|
||||
It is not free the way the slot head was. Intent, destination and slot F1 each
|
||||
move down one to four points, inside the seed spread. `поужинал` is a false
|
||||
clarify on every seed. That is the same defect `thinSingleToken` was narrowed for
|
||||
on 2026-08-01.
|
||||
|
||||
The corpus is generated, because every existing row is answerable by
|
||||
construction. The router-prompt agreement filter cannot work here. `routeGrammar`
|
||||
has no clarify value, and a generated line always agrees with itself. A gemma
|
||||
judge replaces it. The first judge called 24 of 40 answerable rows underspecified.
|
||||
It judged against a generic assistant rather than against Maven's contract.
|
||||
|
||||
### The slot head
|
||||
|
||||
BIO slot tags had no Maven-domain corpus. That was true of found corpora and
|
||||
false of made ones. `label_slots.py` distils spans out of gemma-4-12b under a
|
||||
GBNF closed over Maven's own five slots. A span survives only when it is a
|
||||
literal substring of the utterance, so the agreement filter costs no second call.
|
||||
2178 spans over 1702 rows, 37 dropped, nothing unparsed.
|
||||
|
||||
Epoch selection reads the intent dev slice alone. That costs the slot head about
|
||||
4 points.
|
||||
|
||||
### Warm start and the floor
|
||||
|
||||
The MASSIVE warm-start of step 2 is worth nothing here. Stock e5-small ties it on
|
||||
intent and leads by a third of a case on destination. Nothing argues for keeping
|
||||
that step.
|
||||
|
||||
The floor was a corpus defect and it is fixed. The first 120 floor rows carried
|
||||
one sentence shape, so the head named a destination where the fixture says walk
|
||||
the chain. Rotating six shapes took the floor 3/7 to 6/7 and destination 75.8% to
|
||||
80.8%.
|
||||
|
||||
What is left is calendar at 3/6 on every seed, which training cannot move. The
|
||||
possessive agenda rules claim those cases at stage 0 and name nothing, so no
|
||||
label reaches the head.
|
||||
|
||||
### Reading them in Go
|
||||
|
||||
`RouterHeads` in `internal/router/heads.go` loads `router_heads.onnx` (V-664,
|
||||
2026-08-08). It reads intent, destination and clarify off one forward pass.
|
||||
|
||||
Three rules around it, each measured:
|
||||
|
||||
- The **clarify head decides first**, before the intent threshold. It answers a
|
||||
different question. A thin utterance scores low intent by construction, so
|
||||
gating it cost 6 of 8 ambiguous cases.
|
||||
- The **destination head is read on `IntentQuery` only**, since no other intent
|
||||
reaches `queryWalk`.
|
||||
- `headsThreshold` is 0.6, the measured knee. Every value up to 0.85 drops right
|
||||
answers and keeps the same two wrong ones.
|
||||
|
||||
`voice.embedder.heads_path` is the whole switch. Empty, missing or unloadable
|
||||
means the heads are nil. The cascade is then byte-for-byte what shipped before
|
||||
them.
|
||||
|
||||
### The tokenizer bug the heads found
|
||||
|
||||
`encodeWord` in `onnxembedder.go` read every long word backwards until 2026-08-08.
|
||||
It cost recall@1 7.4 points and recall@3 11.1. Nothing caught it, because seeds
|
||||
and queries were mangled the same way and cosine survived. The heads found it.
|
||||
They are trained through transformers and read through this.
|
||||
|
||||
The embedder id now carries a tokenizer revision (`@384/tok2`). So fixing the
|
||||
tokenizer triggers `ReembedAll` the way swapping the model file does. Bump
|
||||
`tokenizerRev` on any change to what it emits.
|
||||
|
||||
## Clarify
|
||||
|
||||
`Confidence: 1.0` was hardcoded in `llmrouter.go`. So the model path could never
|
||||
ask for clarification, and it missed 6 of 6 refusal cases (V-359). The bug had a
|
||||
second half. The model branch never consulted `r.threshold` at all, so a correct
|
||||
low confidence would have been discarded anyway.
|
||||
|
||||
Fixed 2026-07-31 with structural signal feeding the same stage 3 gate the
|
||||
classifier path already had (`gateLLMDecision` in `router.go`). Three signals: a
|
||||
single-token utterance, a keyless fact, an act with no allowlisted fn.
|
||||
|
||||
Re-measured: missed clarify 6/6 to 1, at the cost of 3 false clarifies and 2.6
|
||||
points of full accuracy. Two of the three false clarifies are acts the model
|
||||
mis-routed and the gate caught. Asking beats wrongly executing, so the fixture
|
||||
and the daemon disagree about what is correct there.
|
||||
|
||||
The third, `поужинал`, was a real defect. The single-token rule was an English
|
||||
intuition. It does not transfer to Russian, where one word is routinely a whole
|
||||
sentence.
|
||||
|
||||
Narrowed 2026-08-01. `thinSingleToken` (`internal/router/singletoken.go`) still
|
||||
thins a bare one-word nominal. It spares two classes. One is a closed lexicon of
|
||||
social and control singles ("привет", "стоп", "yes"). The other is any token
|
||||
carrying a Russian verb ending, because a verb already contains its subject. Both
|
||||
tests are offline and cost nothing. False clarifies 3 to 2, intent-only 74.0% to
|
||||
75.3%.
|
||||
|
||||
The two remaining false clarifies are the act-with-no-allowlisted-fn arm of the
|
||||
gate, not this rule.
|
||||
|
||||
## The destination
|
||||
|
||||
`query` was a shrug. The cascade sorted an utterance into one of seven intents,
|
||||
then `IntentQuery` handed the turn to `querySources` in the daemon. That is
|
||||
twenty-two branches deciding by seed similarity in a fixed order. It had no
|
||||
fixture, no accuracy number, no model arm and no floor.
|
||||
|
||||
`Decision.Source` (`internal/router/source.go`) is the second half of the route
|
||||
(V-655, 2026-08-07). Twelve destinations, not twenty-two. The three recall passes
|
||||
plus `fact-by-key` are one destination from outside. So are search, Kiwix and the
|
||||
URL reader.
|
||||
|
||||
`queryWalk` in `cmd/mavend/actions_query.go` takes sources **out** and moves none.
|
||||
That is the safety argument. The table's order is load-bearing. Every comment on
|
||||
it argues a reason between two sources. Above all it carries "the owner's data
|
||||
first, then the world". Naming `SourceWorld` does not send the turn outside on its
|
||||
own.
|
||||
|
||||
What comes out is only the sources that **guess**. Those decide a turn is theirs
|
||||
by cosine against frozen seeds, then answer whatever they claimed. They hold no
|
||||
table that could come back empty. Weather is the pure case and has no local data
|
||||
at all. It was measured on the box 2026-08-07
|
||||
(`docs/evals/2026-08-07-week-of-usage.md` section 4). It answered both "что такое
|
||||
TCP?" and "сколько будет 17 на 23?" with "для какого города?". The feed answered
|
||||
"какой у меня любимый язык?" with kernel headlines.
|
||||
|
||||
### Who may drop the personal boundary
|
||||
|
||||
The personal boundary guesses, so naming `SourceWorld` drops it. That is what
|
||||
stops it answering "кто такой Линус Торвальдс?" with "не нашла у тебя такой
|
||||
записи", which it did on 2026-08-07.
|
||||
|
||||
Three deciders name a destination and two of them infer it: the heads and the
|
||||
resident model. An inferred `SourceWorld` on a question about him would reach
|
||||
SearXNG. That widens what is asked rather than costing a local answer. So only a
|
||||
stage 0 grammar may drop it (owner's call, V-666, 2026-08-09).
|
||||
|
||||
`Decision.SourceAnchored` carries the provenance. It is a field and not
|
||||
`Stage == 0`. Stage 0 also means confidence 1.0 and an anchored claim band, and
|
||||
one of those could stop implying the others. `definitionQueryPattern` claims "кто
|
||||
такой X", so the 2026-08-07 case is still anchored and still answered.
|
||||
|
||||
`queryWalk` reads `SourceAnchored` for the query source marked `boundary: true`
|
||||
and no other. Every other guesser still comes off the turn, whoever named the
|
||||
destination. `TestOnlyAGrammarMayDropTheBoundary` and
|
||||
`TestNamingRecallKeepsTheBoundary` pin both directions.
|
||||
|
||||
### The destination fixture
|
||||
|
||||
`want_source` on `eval.Case` is a pointer, because the destination has three
|
||||
states and a bare string has two. Absent is every intent but query. Present and
|
||||
empty is the `SourceUnknown` contract: name nothing and walk the chain. Present
|
||||
and named is a destination the route must produce. Thirty-three of ninety-six
|
||||
cases carry one.
|
||||
|
||||
A destination miss does **not** fail the case. It lands in `Outcome.SourceReason`
|
||||
and never in `Reasons`, so `Accuracy` and `IntentAccuracy` mean what they meant.
|
||||
`SourceAccuracy` is a second number over the labelled cases alone.
|
||||
A route that lost its intent scores no destination hit. Otherwise a clarify would
|
||||
satisfy an empty label for free.
|
||||
|
||||
Seven cases assert the floor and five of them are homelab operations. They cluster
|
||||
because `SourceRecall`, `SourceNetwork` and `SourceAttention` overlap on every
|
||||
question about the box. `mavpoll` writes its netdata and uptime-kuma observations
|
||||
into the fact store recall reads. That is a finding about the enum, not a gap in
|
||||
the labelling. The other two are `ru-query-005` and `ru-query-014`. No query
|
||||
source reads the reminder store, and a deadline could sit in tasks, the calendar
|
||||
or Praxis. The owner confirmed all seven floor labels on 2026-08-08.
|
||||
|
||||
### The model arm
|
||||
|
||||
`routeGrammar` carries a `source` rule closed over `router.Sources` plus the
|
||||
empty floor (V-660, 2026-08-08). So the model cannot emit a destination that does
|
||||
not exist. The prompt lists the twelve in Russian and says `""` is a normal answer
|
||||
to give often. `LLMRouter.Route` reads it back through `ValidSource` and on
|
||||
`IntentQuery` alone.
|
||||
|
||||
Against gemma-4-12b the cascade scores destination 24/33 with intent unmoved, and
|
||||
recall goes 0/15 to 14/15.
|
||||
|
||||
**Stage 0 now costs four destination points.** It did not before. The four cases
|
||||
the cascade loses and the model alone wins are all calendar. The possessive agenda
|
||||
rules claim them first and name nothing on purpose. That caution was free while
|
||||
nothing downstream could name anything either. It is not free now, and the fix is
|
||||
the owner's call (V-660 open).
|
||||
|
||||
## The decision trace
|
||||
|
||||
Arbitration between the claimants on the utterance stream is order. It is
|
||||
hardcoded in the pre-route resolver ladder, in `buildRouter` and in
|
||||
`querySources`. Nothing recorded who lost until V-564.
|
||||
|
||||
`internal/decision` records one `Record` per turn. It holds every claimant, what
|
||||
it would have made the turn, the score it reported, and how it ended. A claimant
|
||||
won, declined, lost on score, was thinned by a gate or was **never asked**.
|
||||
|
||||
The record rides the context, the same seam `querysource.go` uses. So a claim
|
||||
site cannot change a route, and a context with no record costs nothing. It is
|
||||
installed in `runTurn`, so the mic, telegram and the web leave the same trail.
|
||||
|
||||
Adding a rung to the ladder in `runTurn` means adding its name to `preRouteLadder`
|
||||
in `cmd/mavend/decisiontrace.go`. Otherwise that rung is silently missing from the
|
||||
record.
|
||||
|
||||
### Why it persists now
|
||||
|
||||
The original rule was that nothing persists, because a turn record is read minutes
|
||||
later or never. Storage was a 25-turn in-memory ring read over `ipc.TurnDecisions`
|
||||
and rendered on `/trace`.
|
||||
|
||||
The owner reversed it on 2026-08-06 (V-629,
|
||||
`docs/plans/21-persisting-the-routing-trace.md`). The routing heads cannot be
|
||||
fitted or calibrated without real utterances. And 9 of the 31 modes in
|
||||
`internal/modes` have no seed example at all.
|
||||
|
||||
The ring did not move. `cmd/mavend/routingtrace.go` is a second sink beside it,
|
||||
writing `routing_traces` (migration #23). The utterance is stored in clear. A
|
||||
384-dimension vector of a short sentence is substantially recoverable, so storing
|
||||
vectors instead would be a privacy claim we cannot support. What makes it safe is
|
||||
the same thing that makes the fact store safe. Retention is 14 days, enforced on
|
||||
write and again on start, so a box that goes quiet does not keep every row.
|
||||
Nothing reads it outward. `Store.Wipe` deletes it with everything else.
|
||||
|
||||
### Corrections
|
||||
|
||||
A correction is promoted out into a seed-shaped row in `routing_labels`
|
||||
(migration #24) and kept, because a label is not a transcript. The transcript
|
||||
still expires.
|
||||
|
||||
A turn marked wrong with no target is a usable negative, so naming the intent is
|
||||
never required. The target is one of the seven intents and never free text.
|
||||
|
||||
All three reaches offer it as of 2026-08-06:
|
||||
|
||||
- `/chat` offers two buttons beside the reply, over `ipc.CorrectTurn` and the
|
||||
trace id that rides back on `ipc.ChatReply`.
|
||||
- Voice offers the `repair` rung, which has read spoken corrections since V-455.
|
||||
It now writes the durable label beside the classifier seed it always wrote. A
|
||||
spoken negative with no target is its own rung, `repair-negative` (V-636,
|
||||
`docs/plans/22-correcting-a-turn.md`).
|
||||
- Telegram offers an inline keyboard under the reply. It needed the chat to become
|
||||
readable first (V-637, `docs/plans/23-inbound-telegram.md`). The poller is dark
|
||||
unless the `telegram` block says `intake`. It long-polls, because the box takes
|
||||
no inbound connections. It accepts `chat_id` and no other sender, and it drops
|
||||
whatever queued while the daemon was down. It reaches the daemon through
|
||||
`ipc.CoreAPI` alone.
|
||||
|
||||
The turn source is still `tap:text` for both telegram and the web. So provenance
|
||||
cannot tell a chat turn from a typed one.
|
||||
@@ -0,0 +1,80 @@
|
||||
# Session workflow: the five stores and the guards
|
||||
|
||||
*Last verified: 2026-08-09 @ a9b480a*
|
||||
|
||||
How a session starts, where each kind of writing belongs, and what the hooks
|
||||
refuse. `CLAUDE.md` carries the commands. This file carries the reasoning.
|
||||
|
||||
## Five stores
|
||||
|
||||
Each owns something the others must not hold.
|
||||
|
||||
| Store | Holds | Lifetime |
|
||||
|---|---|---|
|
||||
| Vikunja task | goal, constraints, assumption ledger, status | durable |
|
||||
| `CLAUDE.md`, `AGENTS.md` | what an agent must know before touching code | durable |
|
||||
| `docs/` | design, measurements, decisions | durable |
|
||||
| `TASK.md` | the brief for this branch, written by `task start`, immutable | one branch |
|
||||
| `HANDOFF.md` | only what the next agent needs to resume | one session |
|
||||
|
||||
`TASK.md` and `.task/` are excluded through `.git/info/exclude`. `HANDOFF.md` is
|
||||
gitignored and injected at session start. If a line in the handoff would still
|
||||
matter next week, it is in the wrong file.
|
||||
|
||||
## Doc tiers
|
||||
|
||||
Tiered by path, so staleness is visible from the filename.
|
||||
|
||||
- Files directly under `docs/` are living. They carry a
|
||||
`Last verified: <date> @ <sha>` line and are corrected in place.
|
||||
- Files under `docs/evals/` are dated measurements and are never edited after
|
||||
the day. A newer number is a new file, not an edit.
|
||||
- Files under `docs/archive/` are dead and read by nobody by default.
|
||||
|
||||
## Vikunja
|
||||
|
||||
This repo is project **Maven** (ID 2). MCP at `http://localhost:9100/mcp`, or
|
||||
`http://192.168.1.104:9100/mcp` from workpc. Feature, bug and deploy tasks go
|
||||
there.
|
||||
|
||||
A task holds the goal, the constraints and the assumption ledger. A session
|
||||
without a task id cannot be resumed by anyone, so a session with none asks for
|
||||
one first.
|
||||
|
||||
**Close a finished task with `done: true` and nothing else** (owner's call,
|
||||
2026-08-07). Do not write a completion summary into the description on the way
|
||||
out. It is lost anyway, and the durable record is the commit messages and the
|
||||
merged PR. `update_task` carrying a `description` resets `done` to false, which
|
||||
is why a write-up ever took two calls.
|
||||
|
||||
## The branch tool
|
||||
|
||||
`~/.local/bin/task` owns the branch, the commit identity and the PR. One task,
|
||||
one session, one PR.
|
||||
|
||||
```sh
|
||||
task start <vikunja-id> # branch off origin/master, write TASK.md, fetch review comments
|
||||
task pr # push, open or refresh the PR, label Vikunja, notify
|
||||
task comments # re-pull this branch's review comments into .task/
|
||||
```
|
||||
|
||||
`/pickup` opens a session and `/wrap` closes it. Wrap at roughly half context
|
||||
rather than letting the session compact.
|
||||
|
||||
## Guards
|
||||
|
||||
Two hooks in `.githooks/`, tracked, wired with `core.hooksPath`. A fresh clone
|
||||
needs `git config core.hooksPath .githooks`.
|
||||
|
||||
- `pre-commit` refuses master, and refuses more than 300 changed lines in
|
||||
non-markdown files. Markdown is exempt and may land as one batch.
|
||||
- `commit-msg` requires the subject to end with `(V-<id>)`. `V-` and not `#`,
|
||||
because Gitea autolinks `#123` to a Gitea issue, which is the wrong tracker.
|
||||
|
||||
Two more guards live outside the repo, in `~/.claude/hooks/`. `diff-budget.sh`
|
||||
blocks further edits past 600 changed lines on a `task/` branch.
|
||||
`prose_lint_hook.py` checks prose on every write. Both measure against
|
||||
`origin/master`, so a local master that is ahead of the remote makes the diff
|
||||
budget read high.
|
||||
|
||||
`--no-verify` exists. Using it means saying why in the commit body.
|
||||
@@ -0,0 +1,71 @@
|
||||
# The world chain
|
||||
|
||||
*Last verified: 2026-08-09 @ a9b480a*
|
||||
|
||||
What happens when the answer is not his. `CLAUDE.md` carries the boundary rule.
|
||||
This file carries the mechanism and the measurements behind it.
|
||||
|
||||
## What replaced "never phones home"
|
||||
|
||||
That promise was deprecated on 2026-07-31 by the owner's call. A 1.7B does not
|
||||
know enough to answer world questions, so she reads external sources. Four rules
|
||||
replaced it:
|
||||
|
||||
- **No telemetry, no cloud model, no third-party account.** That part never
|
||||
changes. Nothing about Maven is reported to anyone and inference stays on the
|
||||
box.
|
||||
- **The owner's data first, then the world.** Every source reading his facts,
|
||||
notes, calendar, tasks or house runs before anything outside. The personal
|
||||
boundary sits between them. Reading beats recalling for a small model.
|
||||
- **The owner's notes and facts are never search input.** Only the utterance
|
||||
goes out. Never the persona block, the history, or matched notes.
|
||||
- **External search is allowed and off unless configured**, like weather and
|
||||
telegram. The code default is off. `deploy/mavend.json` ships a `search`
|
||||
block, so it is on for this box and deleting the block turns it off again.
|
||||
|
||||
**Live search leads and the ZIMs are the fallback** (owner's call, 2026-08-02).
|
||||
A self-hosted SearXNG answers first. The Kiwix ZIMs on homesrv answer when the
|
||||
search is empty, unreachable, or the line is down.
|
||||
|
||||
## The gate is emptiness and nothing else
|
||||
|
||||
`Response.Empty()` is the whole gate. There is no quality threshold in front of
|
||||
it. Four signals were tried and none separates a real question from an invented
|
||||
one.
|
||||
|
||||
Token overlap was the closest and it still fails. "столица Франции" would lose
|
||||
its answer, because the answer is Париж and that word is not in the question
|
||||
(`docs/evals/2026-08-05-search-quality-signals.md`).
|
||||
|
||||
**The embedder is not a fifth signal.** Query-to-passage cosine measures topic
|
||||
and not whether the passage answers. The two sets overlap
|
||||
(`docs/evals/2026-08-09-kiwix-topic-retrieval.md`).
|
||||
|
||||
## Timeouts
|
||||
|
||||
The connect phase alone is capped at `dialTimeout`, 1.5s, because a blackholed
|
||||
host once cost the owner 8 seconds. A slow instance that did connect keeps the
|
||||
full 8 (`docs/evals/2026-08-05-kiwix-offline-fallback.md`).
|
||||
|
||||
## Kiwix
|
||||
|
||||
**A Russian question reads `wikipedia_ru_all_maxi_2026-02` verbatim** through
|
||||
`kiwix.book_ru`. The RU→EN rewriter is the workaround for an English book and is
|
||||
skipped there. Kiwix catalog names come from the filename, not the `<name>`
|
||||
field.
|
||||
|
||||
**Kiwix ranks by keyword overlap.** Never send it a whole sentence. `kiwix.Topic`
|
||||
drops the narrative request, the interrogative and a verb behind one.
|
||||
|
||||
`kiwix.TitlePath` tries the exact article first, since a ZIM is addressable by
|
||||
title and a wrong title is a 404. `TitleCandidates` tries the spoken form and
|
||||
then the capitalized one. Both apply on the verbatim path alone. The rewriter
|
||||
already reduces a question, and reducing twice takes the topic off its input
|
||||
(V-668).
|
||||
|
||||
## Which source answered
|
||||
|
||||
**The claiming query source is readable on `/chat`** as a badge beside the
|
||||
reply. It is carried on `ipc.ChatReply.Source` and noted by `noteQuerySource` in
|
||||
`cmd/mavend/querysource.go`. It rides the context, so `handleText` keeps the one
|
||||
string signature the mic, telegram and the web share.
|
||||
@@ -37,6 +37,16 @@ type EmbedderConfig struct {
|
||||
ModelPath string `json:"model_path,omitempty"`
|
||||
TokenizerPath string `json:"tokenizer_path,omitempty"`
|
||||
LibPath string `json:"lib_path,omitempty"`
|
||||
|
||||
// HeadsPath — the routing heads graph, which is a fine-tuned COPY of the
|
||||
// model above with four linear heads on its pooled output (V-664). Empty
|
||||
// means no heads, and the cascade runs exactly as it did before they
|
||||
// existed. It shares LibPath and TokenizerPath, and router_heads.json is
|
||||
// read from the same directory.
|
||||
//
|
||||
// It must never be pointed at ModelPath. Memory recall depends on the
|
||||
// resident copy scoring what it scored, and the fine-tuned one does not.
|
||||
HeadsPath string `json:"heads_path,omitempty"`
|
||||
}
|
||||
|
||||
// WeatherConfig configures the weather provider for voice queries.
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
@@ -35,12 +36,54 @@ type WorkstationConfig struct {
|
||||
// 0 ⇒ DefaultWorkstationTimeout. A big model on a LAN host is slower than
|
||||
// the resident one, and a request that overruns falls back to the floor.
|
||||
Timeout Duration `json:"timeout,omitempty"`
|
||||
|
||||
// Stt — CrisperWhisper 2.0 on the same machine, a separate service on its
|
||||
// own port. Absent ⇒ every utterance goes to mavsttd, which is today.
|
||||
Stt *WorkstationSttConfig `json:"stt,omitempty"`
|
||||
}
|
||||
|
||||
// WorkstationSttConfig — speech-to-text on the workstation.
|
||||
//
|
||||
// It is a second service and not a second endpoint on mavgpud: whisper.cpp
|
||||
// cannot load CrisperWhisper 2.0 at all, because it derives its language count
|
||||
// from the vocabulary size and CW2's 51897 tokens shift seven special token
|
||||
// ids. So CW2 runs under transformers, and this block addresses it.
|
||||
//
|
||||
// Worth the trouble: CW2 turbo scores 10.4% WER in Russian against 27.5% for
|
||||
// the ggml-small.bin homesrv loads
|
||||
// (docs/evals/2026-08-09-crisperwhisper2-russian-wer.md).
|
||||
type WorkstationSttConfig struct {
|
||||
// URL — the transcribe endpoint, e.g.
|
||||
// "http://192.168.1.105:8081/transcribe". Empty ⇒ the block is normalised
|
||||
// to nil and mavsttd takes every turn.
|
||||
URL string `json:"url,omitempty"`
|
||||
|
||||
// Health — the admission endpoint. Empty ⇒ the URL's origin + "/health".
|
||||
// It answers 503 while the card is held, and that is the signal.
|
||||
Health string `json:"health,omitempty"`
|
||||
|
||||
// Token — the bearer token the service checks. Audio is the most sensitive
|
||||
// thing that crosses this seam, so a LAN deployment should set one. Write
|
||||
// it as ${MAVEN_STT_TOKEN} and keep the value in deploy/telegram.env, the
|
||||
// way every other secret in this file is written.
|
||||
Token string `json:"token,omitempty"`
|
||||
|
||||
// Probe — how often admission is re-checked. 0 ⇒ DefaultWorkstationProbe.
|
||||
Probe Duration `json:"probe,omitempty"`
|
||||
|
||||
// Timeout — the per-request budget for one utterance. 0 ⇒
|
||||
// DefaultWorkstationSttTimeout. A request that overruns falls back to
|
||||
// mavsttd, which costs a worse transcript and not the turn.
|
||||
Timeout Duration `json:"timeout,omitempty"`
|
||||
}
|
||||
|
||||
// Workstation defaults, applied in normaliseWorkstation.
|
||||
const (
|
||||
DefaultWorkstationProbe = 15 * time.Second
|
||||
DefaultWorkstationTimeout = 90 * time.Second
|
||||
// One utterance, not one completion. A voice turn waits on this, so the
|
||||
// budget is a few seconds and not a minute and a half.
|
||||
DefaultWorkstationSttTimeout = 10 * time.Second
|
||||
)
|
||||
|
||||
// normaliseWorkstation applies the block's defaults. No address, no preferred
|
||||
@@ -63,4 +106,36 @@ func (c *Config) normaliseWorkstation() {
|
||||
if w.Timeout <= 0 {
|
||||
w.Timeout = Duration(DefaultWorkstationTimeout)
|
||||
}
|
||||
normaliseWorkstationStt(w)
|
||||
}
|
||||
|
||||
// normaliseWorkstationStt applies the speech-to-text block's defaults. No
|
||||
// address, no remote: mavsttd then takes every utterance, which is today.
|
||||
func normaliseWorkstationStt(w *WorkstationConfig) {
|
||||
if w.Stt != nil && strings.TrimSpace(w.Stt.URL) == "" {
|
||||
w.Stt = nil
|
||||
}
|
||||
if w.Stt == nil {
|
||||
return
|
||||
}
|
||||
s := w.Stt
|
||||
if strings.TrimSpace(s.Health) == "" {
|
||||
s.Health = healthOrigin(s.URL)
|
||||
}
|
||||
if s.Probe <= 0 {
|
||||
s.Probe = Duration(DefaultWorkstationProbe)
|
||||
}
|
||||
if s.Timeout <= 0 {
|
||||
s.Timeout = Duration(DefaultWorkstationSttTimeout)
|
||||
}
|
||||
}
|
||||
|
||||
// healthOrigin derives the admission endpoint from the transcribe endpoint.
|
||||
// The URL names a path, so appending to it would ask for /transcribe/health.
|
||||
func healthOrigin(raw string) string {
|
||||
u, err := url.Parse(raw)
|
||||
if err != nil || u.Host == "" {
|
||||
return strings.TrimRight(raw, "/") + "/health"
|
||||
}
|
||||
return u.Scheme + "://" + u.Host + "/health"
|
||||
}
|
||||
|
||||
@@ -47,6 +47,11 @@ type PendingQuestion struct {
|
||||
// charging it a retry is the V-554 shape. See CanResume for why it is
|
||||
// counted at all.
|
||||
Suspends int
|
||||
// Rides counts every turn this question has ridden out on the end of
|
||||
// someone else's reply, over the whole life of the request. Unlike Suspends
|
||||
// it is never reset and never re-based, which is the only property that
|
||||
// matters about it (V-663).
|
||||
Rides int
|
||||
}
|
||||
|
||||
// MaxSuspends — how many times one question may step aside and come back before
|
||||
@@ -63,10 +68,52 @@ type PendingQuestion struct {
|
||||
// is that he has moved on and has not said so.
|
||||
const MaxSuspends = 3
|
||||
|
||||
// MaxRides — how many turns one question may ride out on the end of an
|
||||
// unrelated reply, counted over its whole life (V-663).
|
||||
//
|
||||
// MaxSuspends did not move the measurement it was written for. Twenty-six of
|
||||
// 140 turns carried a tail before it landed and twenty-six carried one after.
|
||||
// Every bound on this question is rearmed by something ordinary:
|
||||
//
|
||||
// - The TTL is an inactivity timer, and both noteSuspended and reaskOrGiveUp
|
||||
// restart it, so it cannot arrive while he keeps talking.
|
||||
// - Suspends is zeroed by any turn that reads as an answer, which is where
|
||||
// "спасибо" and "привет" land. It resets before anything is known to have
|
||||
// been filled.
|
||||
// - askRemainingGap builds a fresh question for the second gap, so a reminder
|
||||
// with two gaps gets a new allowance halfway through.
|
||||
//
|
||||
// So Suspends only bites on four strictly consecutive side queries with nothing
|
||||
// chat-like between them, which is not the shape real conversation has. Rides is
|
||||
// the same idea with the resets taken out: set once, incremented, carried
|
||||
// across a re-park, and read by nothing that could lower it.
|
||||
//
|
||||
// The shape it is aimed at is measured, not imagined. In the 2026-08-08 run one
|
||||
// question about a reminder's day rode turns 7 to 13 and ended only because
|
||||
// turn 14 was a new request. Three asides, then two turns that read as failed
|
||||
// answers, then two more asides. The asides spend no attempt and the answers
|
||||
// reset Suspends, so the two bounds take turns being rearmed by the other's
|
||||
// traffic.
|
||||
//
|
||||
// Four, not three. It has to be looser than MaxSuspends or that bound is dead
|
||||
// code, because Rides is never lower than Suspends and would always fire first.
|
||||
//
|
||||
// Do not read this as a fix for the whole ride. It ends the measured one a turn
|
||||
// early and no more. Most of that ride's length is attempts, spent by turns
|
||||
// like "спасибо" and "привет" being read as failed answers to a question about
|
||||
// a day. That is a defect in classifyTurnRole and not in any bound here.
|
||||
const MaxRides = 4
|
||||
|
||||
// CanResume reports whether this question may step aside once more. False ⇒ the
|
||||
// caller lets the request go and says so; it must never simply stop resuming,
|
||||
// because a question dropped in silence reads as one that was answered.
|
||||
func (q *PendingQuestion) CanResume() bool { return q.Suspends < MaxSuspends }
|
||||
//
|
||||
// Two bounds, and they answer different questions. Suspends asks whether he has
|
||||
// walked away from this exchange in the last few turns. Rides asks whether this
|
||||
// question has been riding long enough that the answer is no regardless.
|
||||
func (q *PendingQuestion) CanResume() bool {
|
||||
return q.Suspends < MaxSuspends && q.Rides < MaxRides
|
||||
}
|
||||
|
||||
// Action reads the parked question as the typed action it is assembling
|
||||
// (pending.go). Derived rather than stored: the question's fields stay the one
|
||||
|
||||
@@ -127,6 +127,22 @@ func (c *Client) Article(ctx context.Context, path string, maxRunes int) (crawl.
|
||||
return crawl.Extract(u, body, maxRunes), nil
|
||||
}
|
||||
|
||||
// TitlePath is the article path for an exact title, for Article to fetch.
|
||||
//
|
||||
// It exists because a ZIM is addressable by title and the full-text index is
|
||||
// not the only way in. "Франция", "TCP" and "Небо" resolve; "Трюмбальная
|
||||
// нидроскопия" is a 404, which is the honest answer and the reason this is
|
||||
// safe to try first. Measured on 2026-08-09, keyword search on the same terms
|
||||
// returns "Список пэров Франции" and "Список портов TCP и UDP" instead.
|
||||
//
|
||||
// A miss is normal rather than a failure. An article whose title inverts a name
|
||||
// ("Торвальдс, Линус") is a 404 here and the first hit in search, so the caller
|
||||
// falls through and loses nothing.
|
||||
func TitlePath(book, title string) string {
|
||||
t := strings.ReplaceAll(strings.TrimSpace(title), " ", "_")
|
||||
return "/content/" + url.PathEscape(book) + "/A/" + url.PathEscape(t)
|
||||
}
|
||||
|
||||
// rss mirrors just the bits of the RSS 2.0 reply we use.
|
||||
type rss struct {
|
||||
Items []struct {
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
package kiwix
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// TestLiveTopicBeatsTheSentence — the measurement V-668 turned on, kept as a
|
||||
// test so the claim can be re-run rather than believed.
|
||||
//
|
||||
// It prints the article the old path returned and the article the new one
|
||||
// returns, for the same question. It asserts nothing about which is better,
|
||||
// because "is this the right article" is a human's call. It fails only if the
|
||||
// two paths agree on every case, which would mean the change does nothing.
|
||||
//
|
||||
// MAVEN_KIWIX_URL=http://127.0.0.1:8034 make t PKG=./internal/kiwix/ RUN=TestLive V=1
|
||||
func TestLiveTopicBeatsTheSentence(t *testing.T) {
|
||||
base := os.Getenv("MAVEN_KIWIX_URL")
|
||||
if base == "" {
|
||||
t.Skip("MAVEN_KIWIX_URL unset — point it at the kiwix-server host port")
|
||||
}
|
||||
const book = "wikipedia_ru_all_maxi_2026-02"
|
||||
c := New(base)
|
||||
questions := []string{
|
||||
"что такое TCP?",
|
||||
"что такое фотосинтез",
|
||||
"кто такой Линус Торвальдс?",
|
||||
"кто написал Войну и мир",
|
||||
"что такое чёрная дыра",
|
||||
"почему небо голубое",
|
||||
"почему трава зелёная",
|
||||
"столица Франции",
|
||||
}
|
||||
moved := 0
|
||||
for _, q := range questions {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
|
||||
before := firstTitle(ctx, c, q, book)
|
||||
topic := Topic(q)
|
||||
after := ""
|
||||
for _, cand := range TitleCandidates(topic) {
|
||||
if page, err := c.Article(ctx, TitlePath(book, cand), 400); err == nil && page.Text != "" {
|
||||
after = page.Title + " (by title)"
|
||||
break
|
||||
}
|
||||
}
|
||||
if after == "" {
|
||||
after = firstTitle(ctx, c, topic, book)
|
||||
}
|
||||
cancel()
|
||||
if before != after {
|
||||
moved++
|
||||
}
|
||||
t.Logf("%-30s before=%-34q after=%q", q, before, after)
|
||||
}
|
||||
t.Logf("%d of %d questions reach a different article", moved, len(questions))
|
||||
if moved == 0 {
|
||||
t.Error("the topic path returns exactly what the sentence path returned")
|
||||
}
|
||||
}
|
||||
|
||||
func firstTitle(ctx context.Context, c *Client, pattern, book string) string {
|
||||
hits, err := c.Search(ctx, pattern, book, 3)
|
||||
if err != nil || len(hits) == 0 {
|
||||
return "(nothing)"
|
||||
}
|
||||
return hits[0].Title
|
||||
}
|
||||
@@ -0,0 +1,118 @@
|
||||
package kiwix
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"unicode"
|
||||
|
||||
"github.com/kami/maven/internal/lexicon"
|
||||
"github.com/kami/maven/internal/morph"
|
||||
)
|
||||
|
||||
// Topic reduces a question to the thing it is about, because Kiwix ranks by
|
||||
// keyword overlap and a whole sentence buries the keyword that matters.
|
||||
//
|
||||
// This package's own doc says it: "why is the sky blue" finds a TV episode.
|
||||
// Measured against the Russian ZIM on 2026-08-09, the sentence and the topic
|
||||
// return different articles for the same question. "кто написал Войну и мир"
|
||||
// returns "Радуйся, мир (Доктор Кто)"; "Войну и мир" returns the novel first.
|
||||
// "что такое TCP" returns "Перехват TCP-соединения"; "TCP" returns TCP. The
|
||||
// English path had a rewriter doing this with a model call. The Russian path
|
||||
// reads the book verbatim (V-508) and had nothing.
|
||||
//
|
||||
// It drops three things off the front and stops: the narrative request, the
|
||||
// interrogative, and a verb sitting between them and the noun. Everything else
|
||||
// is kept, because a word this cannot classify is more likely the topic than
|
||||
// noise. An empty return means the utterance was question words alone, and the
|
||||
// caller searches the sentence as before.
|
||||
func Topic(utterance string) string {
|
||||
words := strings.Fields(strings.TrimSpace(utterance))
|
||||
cut := 0
|
||||
for cut < len(words) {
|
||||
w := strings.Trim(strings.ToLower(words[cut]), ".,!?…:;\"'«»")
|
||||
if w == "" {
|
||||
cut++
|
||||
continue
|
||||
}
|
||||
switch {
|
||||
case inList(lexicon.NarrativeRequests(), w),
|
||||
inList(lexicon.Interrogatives(), w),
|
||||
inList(lexicon.FirstPerson(), w),
|
||||
// "что ТАКОЕ x", "кто ТАКОЙ x" — the copula that only ever follows
|
||||
// an interrogative, and never a topic on its own.
|
||||
cut > 0 && isCopula(w),
|
||||
// "расскажи ПРО x", "о x". One-letter and two-letter prepositions
|
||||
// are not a closed class worth a lexicon set of their own.
|
||||
cut > 0 && isLeadingPreposition(w),
|
||||
// "кто НАПИСАЛ Войну и мир". A verb here is the question's own
|
||||
// verb, not part of the title. Only after something was already
|
||||
// dropped, so "написал отчёт" as a topic survives intact.
|
||||
cut > 0 && morph.IsVerbForm(w):
|
||||
cut++
|
||||
default:
|
||||
// The question mark is the sentence's, not the title's, and Kiwix
|
||||
// carries it into the keyword match.
|
||||
topic := strings.TrimRight(strings.Join(words[cut:], " "), " .,!?…:;\"'«»")
|
||||
if !hasLetter(topic) {
|
||||
return ""
|
||||
}
|
||||
return topic
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// TitleCandidates is the topic as it might be titled, best first.
|
||||
//
|
||||
// A ZIM title is capitalized and the utterance is not: measured on 2026-08-09,
|
||||
// `/A/фотосинтез` is a 404 and `/A/Фотосинтез` is a 200. The spoken form is
|
||||
// tried first anyway, because a title that begins lowercase on purpose
|
||||
// ("iPhone") would not survive capitalizing it. Both are one request each
|
||||
// against a server on the same box, and a miss is a 404 rather than a wrong
|
||||
// article.
|
||||
func TitleCandidates(topic string) []string {
|
||||
if topic == "" {
|
||||
return nil
|
||||
}
|
||||
r := []rune(topic)
|
||||
up := unicode.ToUpper(r[0])
|
||||
if up == r[0] {
|
||||
return []string{topic}
|
||||
}
|
||||
return []string{topic, string(up) + string(r[1:])}
|
||||
}
|
||||
|
||||
func isCopula(w string) bool {
|
||||
switch w {
|
||||
case "такое", "такой", "такая", "такие", "is", "are", "was", "were":
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func isLeadingPreposition(w string) bool {
|
||||
switch w {
|
||||
case "про", "о", "об", "обо", "по", "about", "of", "on":
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func inList(list []string, w string) bool {
|
||||
for _, x := range list {
|
||||
if x == w {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// hasLetter is the guard against a topic that reduced to punctuation or digits
|
||||
// alone, which no ZIM title matches.
|
||||
func hasLetter(s string) bool {
|
||||
for _, r := range s {
|
||||
if unicode.IsLetter(r) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,67 @@
|
||||
package kiwix
|
||||
|
||||
import "testing"
|
||||
|
||||
// The cases the 2026-08-09 measurement turned on, plus the ones a topic must
|
||||
// not damage. Each left column returned a wrong article when it was sent whole.
|
||||
func TestTopicKeepsTheThingTheQuestionIsAbout(t *testing.T) {
|
||||
cases := []struct{ utterance, want string }{
|
||||
{"что такое TCP?", "TCP"},
|
||||
{"что такое фотосинтез", "фотосинтез"},
|
||||
{"кто такой Линус Торвальдс?", "Линус Торвальдс"},
|
||||
{"кто написал Войну и мир", "Войну и мир"},
|
||||
{"расскажи про битву при Ватерлоо", "битву при Ватерлоо"},
|
||||
{"what is photosynthesis", "photosynthesis"},
|
||||
// No question word, so there is nothing to drop. The topic is the
|
||||
// whole utterance and the search is what it was before.
|
||||
{"столица Франции", "столица Франции"},
|
||||
{"почему небо голубое", "небо голубое"},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := Topic(c.utterance); got != c.want {
|
||||
t.Errorf("Topic(%q) = %q, want %q", c.utterance, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A verb only goes when a question word already went. Otherwise "написал
|
||||
// отчёт" loses the verb that names what he means.
|
||||
func TestTopicDropsAVerbOnlyBehindAQuestionWord(t *testing.T) {
|
||||
if got := Topic("написал отчёт"); got != "написал отчёт" {
|
||||
t.Errorf("Topic dropped a leading verb with no question word: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// Question words alone reduce to nothing, and the caller reads that as "no
|
||||
// topic" and searches the sentence rather than searching the empty string.
|
||||
func TestTopicIsEmptyWhenNothingIsLeft(t *testing.T) {
|
||||
for _, q := range []string{"что такое?", "кто?", "почему", "???"} {
|
||||
if got := Topic(q); got != "" {
|
||||
t.Errorf("Topic(%q) = %q, want empty", q, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTitlePathEscapesAndUnderscores(t *testing.T) {
|
||||
got := TitlePath("wikipedia_ru_all_maxi_2026-02", "Чёрная дыра")
|
||||
want := "/content/wikipedia_ru_all_maxi_2026-02/A/%D0%A7%D1%91%D1%80%D0%BD%D0%B0%D1%8F_%D0%B4%D1%8B%D1%80%D0%B0"
|
||||
if got != want {
|
||||
t.Errorf("TitlePath = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// A ZIM title carries a leading capital and the utterance does not. The spoken
|
||||
// form is still tried first, so a title that begins lowercase on purpose keeps
|
||||
// its chance.
|
||||
func TestTitleCandidatesTryTheSpokenFormFirst(t *testing.T) {
|
||||
got := TitleCandidates("фотосинтез")
|
||||
if len(got) != 2 || got[0] != "фотосинтез" || got[1] != "Фотосинтез" {
|
||||
t.Errorf("TitleCandidates = %q", got)
|
||||
}
|
||||
if got := TitleCandidates("TCP"); len(got) != 1 || got[0] != "TCP" {
|
||||
t.Errorf("an already-capital topic was tried twice: %q", got)
|
||||
}
|
||||
if got := TitleCandidates(""); got != nil {
|
||||
t.Errorf("TitleCandidates(\"\") = %q, want nil", got)
|
||||
}
|
||||
}
|
||||
@@ -119,6 +119,11 @@ func PartsOfDay() []string { return words("parts_of_day") }
|
||||
// ReminderVerbs returns the imperatives that open a reminder.
|
||||
func ReminderVerbs() []string { return words("reminder_verbs") }
|
||||
|
||||
// Pleasantries returns the whole utterances that greet, thank or say goodbye.
|
||||
// Whole utterances and not tokens: see the set's own note for why the tokens
|
||||
// are unsafe alone.
|
||||
func Pleasantries() []string { return words("pleasantries") }
|
||||
|
||||
// TaskDoneWords returns the words that finish a task, and TaskDropWords the
|
||||
// words that abandon one. Two sets rather than one with a value, because the
|
||||
// store records which of the two happened and the caller has to say so.
|
||||
|
||||
@@ -176,6 +176,19 @@
|
||||
"morning", "afternoon", "evening", "night"
|
||||
]
|
||||
},
|
||||
"pleasantries": {
|
||||
"note": "Whole utterances that greet, thank or say goodbye. They ask for nothing and answer nothing, so a parked question must neither consume them as a failed answer nor be dropped by them (V-663). Matched as WHOLE utterances and never as tokens, because the tokens are not safe alone: \"вечер\" answers \"это утра или вечера?\" and \"нет\" answers a confirm. Anything that could fill a slot stays out. The control words (\"стоп\", \"отмена\") stay out too, because isCancel already owns them and they mean something stronger.",
|
||||
"words": [
|
||||
"привет", "приветик", "здравствуй", "здравствуйте",
|
||||
"доброе утро", "добрый день", "добрый вечер",
|
||||
"пока", "прощай", "до свидания", "спокойной ночи",
|
||||
"спасибо", "спасибо тебе", "большое спасибо", "благодарю",
|
||||
"извини", "извините", "прости", "простите",
|
||||
"hi", "hello", "hey", "bye", "goodbye",
|
||||
"good morning", "good evening", "good night",
|
||||
"thanks", "thank you", "thanks a lot", "sorry"
|
||||
]
|
||||
},
|
||||
"reminder_verbs": {
|
||||
"note": "The imperatives that mean \"remind me\", in the forms he speaks. The same kind of set as capture_verbs and decided the same way: it is her vocabulary, not a discovery about Russian (Vikunja #530). The alarm verbs joined them in V-627. \"разбуди меня в 6:30\" is a reminder that fires at the hour he gets up, and the set knew no form of it, so an alarm reached IntentReminder only by resembling one to the embedder.",
|
||||
"words": [
|
||||
|
||||
@@ -14,12 +14,15 @@ import (
|
||||
"github.com/kami/maven/internal/decision"
|
||||
)
|
||||
|
||||
// The two routing engines, named as claimants. They are one stage and not two,
|
||||
// because only one of them ever runs: the classifier is reached when the model
|
||||
// is absent or errored, never alongside it.
|
||||
// The three routing engines, named as claimants. The model and the classifier
|
||||
// are one stage and not two, because only one of them ever runs: the classifier
|
||||
// is reached when the model is absent or errored, never alongside it. The heads
|
||||
// run before both and decline on low confidence, so they can appear beside
|
||||
// either one in a record.
|
||||
const (
|
||||
claimantLLM = "llm-router"
|
||||
claimantClassifier = "classifier"
|
||||
claimantHeads = "routing-heads"
|
||||
)
|
||||
|
||||
// thinReason names which arm of gateLLMDecision cut the confidence. The gate
|
||||
|
||||
@@ -1,10 +1,13 @@
|
||||
package router
|
||||
|
||||
import "testing"
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestEmbedderIDFromModelPath(t *testing.T) {
|
||||
got := modelIDFromPath("/opt/maven/models/embedder/multilingual-e5-small.onnx")
|
||||
if got != "multilingual-e5-small@384" {
|
||||
if got != "multilingual-e5-small@384/tok2" {
|
||||
t.Fatalf("modelIDFromPath = %q", got)
|
||||
}
|
||||
// A different model file must produce a different id, even at 384 dim.
|
||||
@@ -12,6 +15,13 @@ func TestEmbedderIDFromModelPath(t *testing.T) {
|
||||
if old == got {
|
||||
t.Fatal("two different models share one id")
|
||||
}
|
||||
// The tokenizer is half of what makes a vector, and it changes under a
|
||||
// model file whose name never moves (V-664). An id that ignored it would
|
||||
// leave stored passages in one space and every new query in another, with
|
||||
// nothing to trigger the re-embed.
|
||||
if !strings.Contains(got, "/tok") {
|
||||
t.Fatalf("id %q does not name the tokenizer revision", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestEmbedderIDIncludesDim(t *testing.T) {
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
package eval
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/kami/maven/internal/config"
|
||||
"github.com/kami/maven/internal/router"
|
||||
)
|
||||
|
||||
// TestONNXRoutingHeads — the cascade with the routing heads wired, which is
|
||||
// what V-664 deploys. Opt-in via MAVEN_ONNX_LIB, same as TestONNXBaseline, and
|
||||
// one TestONNX* per process.
|
||||
//
|
||||
// The comparison worth reading is against TestONNXBaseline, which is the same
|
||||
// cascade with the same grammars and the same classifier floor and no heads.
|
||||
// Only the middle arm varies.
|
||||
//
|
||||
// It also checks the Go unigram tokenizer against the Python one, because the
|
||||
// heads were trained through transformers and are read through a hand-written
|
||||
// tokenizer. A mismatch shows up here as a score below what Python measured on
|
||||
// the same weights, and nowhere else.
|
||||
func TestONNXRoutingHeads(t *testing.T) {
|
||||
lib := os.Getenv("MAVEN_ONNX_LIB")
|
||||
if lib == "" {
|
||||
t.Skip("MAVEN_ONNX_LIB unset — see AGENTS.md § Embedder model for intent routing")
|
||||
}
|
||||
// Absolute, because onnxruntime resolves a graph's external weights file
|
||||
// against the model path it was given, and a relative one lands in the
|
||||
// test's working directory.
|
||||
root, err := filepath.Abs("../../..")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
model := filepath.Join(root, "models/embedder/multilingual-e5-small/model_quantized.onnx")
|
||||
tok := filepath.Join(root, "models/embedder/multilingual-e5-small/tokenizer.json")
|
||||
heads := filepath.Join(root, "models/embedder/router-heads/router_heads.onnx")
|
||||
for _, p := range []string{lib, model, tok, heads} {
|
||||
if _, err := os.Stat(p); err != nil {
|
||||
t.Skipf("missing %s: %v", p, err)
|
||||
}
|
||||
}
|
||||
emb, err2 := router.NewONNXEmbedder(model, tok, lib)
|
||||
if err2 != nil {
|
||||
t.Skipf("onnx embedder unavailable: %v", err2)
|
||||
}
|
||||
err = nil
|
||||
defer emb.Close()
|
||||
|
||||
h, err := router.NewRouterHeads(heads, tok)
|
||||
if err != nil {
|
||||
t.Skipf("routing heads unavailable: %v", err)
|
||||
}
|
||||
defer h.Close()
|
||||
|
||||
f, err := Load()
|
||||
if err != nil {
|
||||
t.Fatalf("Load: %v", err)
|
||||
}
|
||||
rep, err := Score(context.Background(), "heads+classifier", withHeads(t, emb, h), f)
|
||||
if err != nil {
|
||||
t.Fatalf("Score: %v", err)
|
||||
}
|
||||
t.Log("\n" + rep.String() + rep.Failures())
|
||||
}
|
||||
|
||||
// withHeads mirrors newBaselineRouter and adds the one arm under test. It is a
|
||||
// separate function rather than a parameter so the baseline's signature stays
|
||||
// the shape every other test calls it with.
|
||||
func withHeads(t *testing.T, emb router.Embedder, h *router.RouterHeads) *router.Router {
|
||||
t.Helper()
|
||||
acts := router.DefaultActMatcher{Fns: actFns}
|
||||
return router.New(router.Config{
|
||||
Grammars: baselineGrammars(acts),
|
||||
Classifier: newBaselineClassifier(t, emb),
|
||||
Extractor: router.Extractor{
|
||||
Time: router.StubDateTimeParser{},
|
||||
Acts: acts,
|
||||
Facts: router.DefaultFactParser{},
|
||||
},
|
||||
Threshold: config.DefaultRouterThreshold,
|
||||
Heads: h,
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,243 @@
|
||||
package router
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"math"
|
||||
"os"
|
||||
"path/filepath"
|
||||
|
||||
ort "github.com/yalue/onnxruntime_go"
|
||||
)
|
||||
|
||||
// The routing heads (V-546, V-661, V-664). Four linear heads over one masked
|
||||
// mean pool of a fine-tuned copy of multilingual-e5-small: intent,
|
||||
// destination, BIO slot tags and clarify. Trained on workpc, exported to ONNX,
|
||||
// and read here.
|
||||
//
|
||||
// Why this is not the classifier. The classifier compares one utterance to
|
||||
// frozen seed phrases by cosine. A head is a softmax over the label set, so it
|
||||
// cannot name a value that does not exist, and its max is a calibratable
|
||||
// confidence where Confidence: 1.0 was a hardcode.
|
||||
//
|
||||
// Why it is not the resident model either. It answers in single-digit
|
||||
// milliseconds against the model's p50 of 1.19s, and it names a destination
|
||||
// the classifier arm never names at all.
|
||||
//
|
||||
// The body is a COPY of the embedder weights, fine-tuned. It must never
|
||||
// replace models/embedder/multilingual-e5-small — memory recall depends on
|
||||
// that file scoring what it scored.
|
||||
//
|
||||
// The slot head is exported and deliberately not read. Slots already come from
|
||||
// the stage-2 extractor, and mapping BIO tags back to text needs character
|
||||
// offsets the unigram tokenizer does not keep. Reading it is separate work.
|
||||
const (
|
||||
// headsSeq — the sequence length the heads were trained at. Padding is
|
||||
// masked out of both attention and the pool, so this changes nothing but
|
||||
// truncation, and truncation is what training did at 64.
|
||||
headsSeq = 64
|
||||
|
||||
// headsThreshold — max softmax over the intent head, below which the heads
|
||||
// decline and the cascade carries on to the resident model.
|
||||
//
|
||||
// 0.6 is the knee measured on the 88-case intent fixture
|
||||
// (docs/evals/2026-08-08-routing-heads-in-go.md). It keeps 81 of 88 cases
|
||||
// at 97.5% accuracy. Every higher value up to 0.9 drops right answers and
|
||||
// keeps the same two wrong ones, so it buys nothing.
|
||||
headsThreshold = 0.6
|
||||
)
|
||||
|
||||
// RouterHeads runs the exported graph. Nil is a working value everywhere: a
|
||||
// deployment with no weights file routes exactly as it did before this
|
||||
// existed.
|
||||
type RouterHeads struct {
|
||||
tokenizer *unigramTokenizer
|
||||
session *ort.DynamicSession[int64, float32]
|
||||
intents []Intent
|
||||
sources []Source
|
||||
threshold float64
|
||||
}
|
||||
|
||||
// headsMeta — router_heads.json, written beside the weights by the exporter.
|
||||
// The label order is the head's output order and cannot be inferred from Go.
|
||||
type headsMeta struct {
|
||||
Intents []string `json:"intents"`
|
||||
Sources []string `json:"sources"`
|
||||
Prefix string `json:"prefix"`
|
||||
}
|
||||
|
||||
// NewRouterHeads loads the graph and its label order. modelPath points at the
|
||||
// .onnx; the external weights and router_heads.json sit beside it.
|
||||
//
|
||||
// It assumes the ONNX environment is already initialised, because the embedder
|
||||
// does that at startup and the runtime allows it once.
|
||||
func NewRouterHeads(modelPath, tokenizerPath string) (*RouterHeads, error) {
|
||||
metaPath := filepath.Join(filepath.Dir(modelPath), "router_heads.json")
|
||||
raw, err := os.ReadFile(metaPath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("heads: read %s: %w", metaPath, err)
|
||||
}
|
||||
var meta headsMeta
|
||||
if err := json.Unmarshal(raw, &meta); err != nil {
|
||||
return nil, fmt.Errorf("heads: parse %s: %w", metaPath, err)
|
||||
}
|
||||
if meta.Prefix != queryPrefix {
|
||||
return nil, fmt.Errorf("heads: trained with prefix %q, this build uses %q",
|
||||
meta.Prefix, queryPrefix)
|
||||
}
|
||||
|
||||
intents := make([]Intent, len(meta.Intents))
|
||||
for i, s := range meta.Intents {
|
||||
intents[i] = Intent(s)
|
||||
}
|
||||
sources := make([]Source, len(meta.Sources))
|
||||
for i, s := range meta.Sources {
|
||||
// SourceUnknown is not in Sources, because it is the absence of a
|
||||
// choice. It is a class the head can emit, and the one it should emit
|
||||
// often, so it is allowed here and nowhere else.
|
||||
if s != string(SourceUnknown) && !ValidSource(Source(s)) {
|
||||
return nil, fmt.Errorf("heads: unknown destination %q in %s", s, metaPath)
|
||||
}
|
||||
sources[i] = Source(s)
|
||||
}
|
||||
|
||||
tok, err := newUnigramTokenizer(tokenizerPath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("heads: tokenizer: %w", err)
|
||||
}
|
||||
session, err := ort.NewDynamicSession[int64, float32](
|
||||
modelPath,
|
||||
[]string{"input_ids", "attention_mask"},
|
||||
[]string{"intent", "source", "slots", "clarify"},
|
||||
)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("heads: create session: %w", err)
|
||||
}
|
||||
|
||||
return &RouterHeads{
|
||||
tokenizer: tok,
|
||||
session: session,
|
||||
intents: intents,
|
||||
sources: sources,
|
||||
threshold: headsThreshold,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (h *RouterHeads) Close() error {
|
||||
if h == nil {
|
||||
return nil
|
||||
}
|
||||
h.session.Destroy()
|
||||
return nil
|
||||
}
|
||||
|
||||
// headsResult — one forward pass, read back.
|
||||
type headsResult struct {
|
||||
Intent Intent
|
||||
Source Source
|
||||
Confidence float64
|
||||
Clarify bool
|
||||
}
|
||||
|
||||
// Route runs the heads and reports whether they are confident enough to answer.
|
||||
// A false second return is a decline, not an error: the cascade goes on to the
|
||||
// resident model, which is what happens today.
|
||||
func (h *RouterHeads) Route(ctx context.Context, utterance string) (headsResult, bool, error) {
|
||||
if h == nil {
|
||||
return headsResult{}, false, nil
|
||||
}
|
||||
ids, mask, _ := h.tokenizer.Encode(queryPrefix + utterance)
|
||||
ids, mask = ids[:headsSeq], mask[:headsSeq]
|
||||
// The tokenizer pads and truncates to its own length, which is longer than
|
||||
// this one. Cutting the tail can cut the separator with it, so put it back.
|
||||
if mask[headsSeq-1] == 1 {
|
||||
ids[headsSeq-1] = sepTokenID
|
||||
}
|
||||
|
||||
shape := ort.NewShape(1, headsSeq)
|
||||
idsT, err := ort.NewTensor(shape, ids)
|
||||
if err != nil {
|
||||
return headsResult{}, false, fmt.Errorf("heads: ids tensor: %w", err)
|
||||
}
|
||||
defer idsT.Destroy()
|
||||
maskT, err := ort.NewTensor(shape, mask)
|
||||
if err != nil {
|
||||
return headsResult{}, false, fmt.Errorf("heads: mask tensor: %w", err)
|
||||
}
|
||||
defer maskT.Destroy()
|
||||
|
||||
intentT, err := ort.NewEmptyTensor[float32](ort.NewShape(1, int64(len(h.intents))))
|
||||
if err != nil {
|
||||
return headsResult{}, false, fmt.Errorf("heads: intent tensor: %w", err)
|
||||
}
|
||||
defer intentT.Destroy()
|
||||
sourceT, err := ort.NewEmptyTensor[float32](ort.NewShape(1, int64(len(h.sources))))
|
||||
if err != nil {
|
||||
return headsResult{}, false, fmt.Errorf("heads: source tensor: %w", err)
|
||||
}
|
||||
defer sourceT.Destroy()
|
||||
slotsT, err := ort.NewEmptyTensor[float32](ort.NewShape(1, headsSeq, int64(numBIOTags)))
|
||||
if err != nil {
|
||||
return headsResult{}, false, fmt.Errorf("heads: slots tensor: %w", err)
|
||||
}
|
||||
defer slotsT.Destroy()
|
||||
clarifyT, err := ort.NewEmptyTensor[float32](ort.NewShape(1, 2))
|
||||
if err != nil {
|
||||
return headsResult{}, false, fmt.Errorf("heads: clarify tensor: %w", err)
|
||||
}
|
||||
defer clarifyT.Destroy()
|
||||
|
||||
if err := h.session.Run(
|
||||
[]*ort.Tensor[int64]{idsT, maskT},
|
||||
[]*ort.Tensor[float32]{intentT, sourceT, slotsT, clarifyT},
|
||||
); err != nil {
|
||||
return headsResult{}, false, fmt.Errorf("heads: run: %w", err)
|
||||
}
|
||||
|
||||
// The graph applies its own softmax, so these are probabilities and the max
|
||||
// is the same number the eval calibrated the threshold against.
|
||||
i, conf := argmax(intentT.GetData())
|
||||
res := headsResult{
|
||||
Intent: h.intents[i],
|
||||
Confidence: conf,
|
||||
}
|
||||
cl := clarifyT.GetData()
|
||||
res.Clarify = len(cl) == 2 && cl[1] > cl[0]
|
||||
|
||||
// The destination head is trained on query rows and is meaningless on any
|
||||
// other intent, the same way queryWalk is never reached by one.
|
||||
if res.Intent == IntentQuery {
|
||||
s, _ := argmax(sourceT.GetData())
|
||||
res.Source = h.sources[s]
|
||||
}
|
||||
|
||||
// The clarify head decides on its own, and it decides first. It answers a
|
||||
// different question from the intent head — not which intent, but whether
|
||||
// there is enough here to act on at all — so a low intent confidence is no
|
||||
// reason to discard it. It is usually the same turns: "вода" reads as
|
||||
// intent act at 0.23 and clarify at 0.98, and letting the intent threshold
|
||||
// bury that hands the turn to the classifier, which routes it confidently
|
||||
// and never asks.
|
||||
if res.Clarify {
|
||||
return res, true, nil
|
||||
}
|
||||
if conf < h.threshold {
|
||||
return res, false, nil
|
||||
}
|
||||
return res, true, nil
|
||||
}
|
||||
|
||||
// numBIOTags — O plus B- and I- for each of Maven's five slots. The head is not
|
||||
// read, but the graph writes it and the output tensor has to be the right size.
|
||||
const numBIOTags = 11
|
||||
|
||||
func argmax(v []float32) (int, float64) {
|
||||
best, bestV := 0, math.Inf(-1)
|
||||
for i, x := range v {
|
||||
if float64(x) > bestV {
|
||||
best, bestV = i, float64(x)
|
||||
}
|
||||
}
|
||||
return best, bestV
|
||||
}
|
||||
@@ -111,6 +111,21 @@ type Decision struct {
|
||||
// before this field existed. See source.go for why it is twelve values.
|
||||
Source Source
|
||||
|
||||
// SourceAnchored — a stage 0 grammar named that destination, matching a
|
||||
// literal pattern to do it. Only the router sets this, and only there.
|
||||
//
|
||||
// It exists because one thing downstream is not reversible by evidence
|
||||
// (V-666). Naming a destination normally takes guessing sources off a turn,
|
||||
// and one of those is the personal boundary, which is what stops a question
|
||||
// about him from reaching the world. A grammar that read "что такое X" may
|
||||
// take it off. A model or a softmax may not, because a wrong destination
|
||||
// there widens what leaves the box rather than costing an answer.
|
||||
//
|
||||
// Read Stage instead and the two decisions get coupled: stage 0 also means
|
||||
// confidence 1.0 and an anchored claim band, and a later cascade change
|
||||
// could make one true where the other is not.
|
||||
SourceAnchored bool
|
||||
|
||||
// Continued — this decision was rebuilt from the previous turn rather
|
||||
// than routed, because the utterance was an ellipsis ("а завтра?").
|
||||
// Handlers use it to know that Slots.Text is the PREVIOUS turn's topic
|
||||
|
||||
@@ -66,12 +66,19 @@ func NewONNXEmbedder(modelPath, tokenizerPath, libPath string) (*onnxEmbedder, e
|
||||
func (e *onnxEmbedder) Dim() int { return embedDim }
|
||||
|
||||
// ID names the loaded model for the DB marker (Vikunja #378): the model file's
|
||||
// own name plus the dimension, so pointing the config at another model changes
|
||||
// the string on its own.
|
||||
// own name, the dimension, and the tokenizer revision, so pointing the config
|
||||
// at another model changes the string on its own.
|
||||
func (e *onnxEmbedder) ID() string { return e.id }
|
||||
|
||||
// tokenizerRev — bumped whenever the tokenizer changes what it emits for the
|
||||
// same text, because that changes every vector while the model file's name
|
||||
// stays put. Rev 2 is the fix for the reversed word pieces (V-664): stored
|
||||
// passages embedded under rev 1 no longer sit in the same space as a query
|
||||
// embedded now, and ReembedAll rewrites them because this string moved.
|
||||
const tokenizerRev = 2
|
||||
|
||||
// modelIDFromPath turns /opt/.../multilingual-e5-small.onnx into
|
||||
// "multilingual-e5-small@384".
|
||||
// "multilingual-e5-small@384/tok2".
|
||||
func modelIDFromPath(modelPath string) string {
|
||||
name := modelPath
|
||||
if i := strings.LastIndexAny(name, "/\\"); i >= 0 {
|
||||
@@ -81,7 +88,7 @@ func modelIDFromPath(modelPath string) string {
|
||||
if name == "" {
|
||||
name = "onnx"
|
||||
}
|
||||
return fmt.Sprintf("%s@%d", name, embedDim)
|
||||
return fmt.Sprintf("%s@%d/tok%d", name, embedDim, tokenizerRev)
|
||||
}
|
||||
|
||||
// Embed treats the text as a query. The classifier compares one short
|
||||
@@ -339,14 +346,17 @@ func (t *unigramTokenizer) encodeWord(word string) []int64 {
|
||||
}
|
||||
}
|
||||
|
||||
// Backtracking walks the word from its end, and prepending each piece puts
|
||||
// it back in reading order. There used to be a second reverse after this
|
||||
// loop, which undid it: every multi-piece word came out backwards, and
|
||||
// "query: вода" tokenized to [0 12 1294 41 12489 2] where the reference
|
||||
// tokenizer gives [0 41 1294 12 12489 2] (V-664). A transformer reads
|
||||
// position, so the pieces of a long Russian word were being read in the
|
||||
// wrong order on every turn.
|
||||
var result []int64
|
||||
for i := n; i > 0; i = prev[i] {
|
||||
result = append([]int64{bestID[i]}, result...)
|
||||
}
|
||||
// Reverse
|
||||
for l, r := 0, len(result)-1; l < r; l, r = l+1, r-1 {
|
||||
result[l], result[r] = result[r], result[l]
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
|
||||
@@ -31,6 +31,12 @@ type Config struct {
|
||||
// error/parse failure, falls through to the classifier (never fails the
|
||||
// turn on the model).
|
||||
LLM *LLMRouter
|
||||
// Heads — optional routing heads over the fine-tuned embedder copy. When
|
||||
// set, Route consults them after stage 0 and before the LLM router. They
|
||||
// decline below their own confidence threshold, so a low-confidence turn
|
||||
// reaches the model exactly as it does today. Nil is the shipped-before
|
||||
// behaviour and costs nothing.
|
||||
Heads *RouterHeads
|
||||
}
|
||||
|
||||
// Router — the deterministic cascade. Route never guesses: stage 0 wins
|
||||
@@ -42,6 +48,7 @@ type Router struct {
|
||||
extractor Extractor
|
||||
threshold float64
|
||||
llm *LLMRouter
|
||||
heads *RouterHeads
|
||||
}
|
||||
|
||||
func New(cfg Config) *Router {
|
||||
@@ -51,6 +58,7 @@ func New(cfg Config) *Router {
|
||||
extractor: cfg.Extractor,
|
||||
threshold: cfg.Threshold,
|
||||
llm: cfg.LLM,
|
||||
heads: cfg.Heads,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -90,6 +98,10 @@ func (r *Router) Route(ctx context.Context, utterance string, now time.Time) (De
|
||||
continue // grammar matched shape but not content → fall through
|
||||
}
|
||||
d.Utterance = utterance
|
||||
// A literal pattern named that destination, which is the one provenance
|
||||
// allowed to take the personal boundary off a turn (V-666). Set here and
|
||||
// nowhere else, so no other arm of the cascade can claim it.
|
||||
d.SourceAnchored = d.Source != SourceUnknown
|
||||
// The grammar decided the intent; the extractor fills the slots it did
|
||||
// not match (V-572). See fillMatchedSlots for why every grammar gets it.
|
||||
r.fillMatchedSlots(ctx, &d, now)
|
||||
@@ -98,6 +110,60 @@ func (r *Router) Route(ctx context.Context, utterance string, now time.Time) (De
|
||||
}
|
||||
r.noteGrammarOutcomes(ctx, len(r.grammars), declinedBuild, "", "")
|
||||
|
||||
// stage 0b — routing heads (when wired). A softmax over the label set, so
|
||||
// it cannot name an intent or a destination that does not exist, and its
|
||||
// max is a real confidence. It runs before the model because it is three
|
||||
// orders of magnitude faster and scores better on both halves of the route.
|
||||
//
|
||||
// It declines below its threshold rather than clarifying. A declined turn
|
||||
// carries on to the model and then the classifier, which is what a box with
|
||||
// no weights file does on every turn.
|
||||
if r.heads != nil {
|
||||
res, ok, err := r.heads.Route(ctx, utterance)
|
||||
switch {
|
||||
case err != nil:
|
||||
log.Printf("router: heads fell through to the rest of the cascade: %v", err)
|
||||
decision.Note(ctx, decision.Claim{
|
||||
Stage: decision.StageRoute, Claimant: claimantHeads,
|
||||
Outcome: decision.Declined, Reason: "error: " + err.Error(),
|
||||
})
|
||||
case !ok:
|
||||
decision.Note(ctx, decision.Scored(decision.StageRoute, claimantHeads,
|
||||
string(res.Intent), res.Confidence, decision.Declined,
|
||||
"below the heads confidence threshold"))
|
||||
default:
|
||||
d := Decision{
|
||||
Utterance: utterance,
|
||||
Stage: 2,
|
||||
Intent: res.Intent,
|
||||
Confidence: res.Confidence,
|
||||
Source: res.Source,
|
||||
Clarify: res.Clarify,
|
||||
}
|
||||
r.fillSlots(ctx, &d, now)
|
||||
decision.Note(ctx, decision.Claim{
|
||||
Stage: decision.StageRoute, Claimant: claimantLLM,
|
||||
Outcome: decision.NeverAsked, Reason: "the routing heads answered",
|
||||
})
|
||||
decision.Note(ctx, decision.Claim{
|
||||
Stage: decision.StageRoute, Claimant: claimantClassifier,
|
||||
Outcome: decision.NeverAsked, Reason: "the routing heads answered",
|
||||
})
|
||||
outcome, reason := decision.Won, ""
|
||||
if d.Clarify {
|
||||
outcome, reason = decision.Thinned, "the clarify head says there is too little here to act on"
|
||||
}
|
||||
decision.Note(ctx, decision.Scored(decision.StageRoute, claimantHeads,
|
||||
string(d.Intent), d.Confidence, outcome, reason))
|
||||
return d, nil
|
||||
}
|
||||
} else {
|
||||
decision.Note(ctx, decision.Claim{
|
||||
Stage: decision.StageRoute, Claimant: claimantHeads,
|
||||
Outcome: decision.NeverAsked, Reason: "no routing heads are wired",
|
||||
})
|
||||
}
|
||||
|
||||
// stage 1a — LLM router (when wired). It reasons over the utterance instead
|
||||
// of nearest-centroid guessing. On any error/parse-fail, fall through to the
|
||||
// classifier cascade (never fail the turn on the model).
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
package stt
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/kami/maven/internal/audio"
|
||||
)
|
||||
|
||||
// HTTPTranscriber — speech-to-text on another host, over HTTP.
|
||||
//
|
||||
// mavsttd is whisper.cpp linked into a Go daemon and reached over a unix
|
||||
// socket. CrisperWhisper 2.0 cannot be reached that way: whisper.cpp derives
|
||||
// its language count from the vocabulary size, and CW2's 51897 tokens shift
|
||||
// seven special token ids. It runs under transformers instead, as a service
|
||||
// beside the model on workpc. See docs/evals/2026-08-09-crisperwhisper2-russian-wer.md.
|
||||
//
|
||||
// So this is the second transport for the same seam, not a second seam. The
|
||||
// caller still sees stt.Transcriber and one method.
|
||||
type HTTPTranscriber struct {
|
||||
url string
|
||||
token string
|
||||
lang string
|
||||
http *http.Client
|
||||
}
|
||||
|
||||
// NewHTTPTranscriber builds the remote client. token may be empty for a
|
||||
// service on a trusted socket, but audio is the most sensitive thing that
|
||||
// crosses this seam, so a LAN deployment should always set one.
|
||||
func NewHTTPTranscriber(url, token, lang string, timeout time.Duration) *HTTPTranscriber {
|
||||
return &HTTPTranscriber{
|
||||
url: url,
|
||||
token: token,
|
||||
lang: lang,
|
||||
http: &http.Client{Timeout: timeout},
|
||||
}
|
||||
}
|
||||
|
||||
// ErrFormat — the audio is not the one canonical shape. Refused at the seam
|
||||
// rather than sent to a model that expects something else.
|
||||
var ErrFormat = errors.New("stt: audio is not 16kHz mono pcm_s16le")
|
||||
|
||||
type httpTranscript struct {
|
||||
Text string `json:"text"`
|
||||
Confidence float64 `json:"confidence"`
|
||||
}
|
||||
|
||||
// Transcribe posts the raw PCM and reads back the text.
|
||||
//
|
||||
// The body is the PCM bytes themselves rather than JSON. A minute of 16kHz
|
||||
// mono is under 2MB raw and about 2.6MB base64, and the format is fixed by
|
||||
// audio.PCM16kMono, so a header carries it more cheaply than an envelope.
|
||||
func (t *HTTPTranscriber) Transcribe(ctx context.Context, a audio.Audio) (string, float64, error) {
|
||||
if !a.Format.IsValid() {
|
||||
return "", 0, ErrFormat
|
||||
}
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, t.url, bytes.NewReader(a.Bytes))
|
||||
if err != nil {
|
||||
return "", 0, fmt.Errorf("stt: build request: %w", err)
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/octet-stream")
|
||||
req.Header.Set("X-Sample-Rate", strconv.Itoa(a.Format.SampleRate))
|
||||
req.Header.Set("X-Channels", strconv.Itoa(a.Format.Channels))
|
||||
req.Header.Set("X-Sample-Bits", strconv.Itoa(a.Format.SampleBits))
|
||||
req.Header.Set("X-Language", t.lang)
|
||||
if t.token != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+t.token)
|
||||
}
|
||||
|
||||
resp, err := t.http.Do(req)
|
||||
if err != nil {
|
||||
return "", 0, fmt.Errorf("stt: post audio: %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return "", 0, fmt.Errorf("stt: remote returned %d", resp.StatusCode)
|
||||
}
|
||||
|
||||
var out httpTranscript
|
||||
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
|
||||
return "", 0, fmt.Errorf("stt: decode transcript: %w", err)
|
||||
}
|
||||
return out.Text, out.Confidence, nil
|
||||
}
|
||||
|
||||
var _ Transcriber = (*HTTPTranscriber)(nil)
|
||||
@@ -0,0 +1,89 @@
|
||||
package stt
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strconv"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/kami/maven/internal/audio"
|
||||
)
|
||||
|
||||
func TestHTTPTranscriberSendsRawPCM(t *testing.T) {
|
||||
t.Parallel()
|
||||
var gotBody []byte
|
||||
var gotHeader http.Header
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
gotBody, _ = io.ReadAll(r.Body)
|
||||
gotHeader = r.Header.Clone()
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_, _ = io.WriteString(w, `{"text":"привет","confidence":0.82}`)
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
a := audio.Audio{Format: audio.PCM16kMono, Bytes: []byte("pcm-bytes")}
|
||||
tr := NewHTTPTranscriber(srv.URL, "s3cret", "ru", 2*time.Second)
|
||||
text, conf, err := tr.Transcribe(context.Background(), a)
|
||||
if err != nil {
|
||||
t.Fatalf("Transcribe: %v", err)
|
||||
}
|
||||
if text != "привет" || conf != 0.82 {
|
||||
t.Fatalf("got %q %v", text, conf)
|
||||
}
|
||||
if string(gotBody) != "pcm-bytes" {
|
||||
t.Fatalf("body should be the PCM itself, got %q", gotBody)
|
||||
}
|
||||
if got := gotHeader.Get("X-Sample-Rate"); got != strconv.Itoa(audio.PCM16kMono.SampleRate) {
|
||||
t.Fatalf("X-Sample-Rate = %q", got)
|
||||
}
|
||||
if got := gotHeader.Get("X-Language"); got != "ru" {
|
||||
t.Fatalf("X-Language = %q", got)
|
||||
}
|
||||
// Audio is the most sensitive thing crossing this seam.
|
||||
if got := gotHeader.Get("Authorization"); got != "Bearer s3cret" {
|
||||
t.Fatalf("Authorization = %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHTTPTranscriberOmitsEmptyToken(t *testing.T) {
|
||||
t.Parallel()
|
||||
var auth string
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
auth = r.Header.Get("Authorization")
|
||||
_, _ = io.WriteString(w, `{"text":"x"}`)
|
||||
}))
|
||||
defer srv.Close()
|
||||
a := audio.Audio{Format: audio.PCM16kMono, Bytes: []byte("x")}
|
||||
if _, _, err := NewHTTPTranscriber(srv.URL, "", "ru", time.Second).Transcribe(context.Background(), a); err != nil {
|
||||
t.Fatalf("Transcribe: %v", err)
|
||||
}
|
||||
if auth != "" {
|
||||
t.Fatalf("Authorization should be absent, got %q", auth)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHTTPTranscriberRefusesWrongFormat(t *testing.T) {
|
||||
t.Parallel()
|
||||
a := audio.Audio{Format: audio.Format{SampleRate: 44100, Channels: 2, SampleBits: 16, Encoding: "pcm_s16le"}}
|
||||
_, _, err := NewHTTPTranscriber("http://example.invalid", "", "ru", time.Second).Transcribe(context.Background(), a)
|
||||
if !errors.Is(err, ErrFormat) {
|
||||
t.Fatalf("want ErrFormat, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHTTPTranscriberErrorsOnBadStatus(t *testing.T) {
|
||||
t.Parallel()
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.WriteHeader(http.StatusUnauthorized)
|
||||
}))
|
||||
defer srv.Close()
|
||||
a := audio.Audio{Format: audio.PCM16kMono, Bytes: []byte("x")}
|
||||
_, _, err := NewHTTPTranscriber(srv.URL, "", "ru", time.Second).Transcribe(context.Background(), a)
|
||||
if err == nil {
|
||||
t.Fatal("a 401 must be an error, so the Pair falls back")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,157 @@
|
||||
package stt
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"log"
|
||||
"net/http"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/kami/maven/internal/audio"
|
||||
)
|
||||
|
||||
// Pair — a preferred transcriber on the workstation, with mavsttd as the floor.
|
||||
//
|
||||
// Same arrangement as llm.Pair and for the same reason. The microphone is at
|
||||
// workpc, the card there has 16GB, and CrisperWhisper 2.0 turbo scores 10.4%
|
||||
// WER in Russian against 27.5% for the ggml-small.bin homesrv loads
|
||||
// (docs/evals/2026-08-09-crisperwhisper2-russian-wer.md). The workstation is
|
||||
// never assumed up: it sleeps, and the card is often held by a training run.
|
||||
//
|
||||
// Speech-to-text has only the silent half of the degradation rule. A worse
|
||||
// transcript is still a turn, and there is nothing to name a gap about, so
|
||||
// Transcribe always falls back. That is the whole difference from llm.Pair,
|
||||
// which also carries CompleteRemote for callers that must refuse instead.
|
||||
type Pair struct {
|
||||
remote Transcriber
|
||||
floor Transcriber
|
||||
|
||||
// up — the cached admission answer, written only by the prober and read by
|
||||
// every turn. A voice turn must never wait on a machine that may be asleep.
|
||||
up atomic.Bool
|
||||
|
||||
health string
|
||||
interval time.Duration
|
||||
http *http.Client
|
||||
stop chan struct{}
|
||||
stopOnce sync.Once
|
||||
}
|
||||
|
||||
const (
|
||||
probeTimeout = 2 * time.Second
|
||||
defaultProbeInterval = 15 * time.Second
|
||||
)
|
||||
|
||||
// ErrNoFloor — a Pair was built with no local transcriber to fall back to. A
|
||||
// configuration mistake: the floor is what makes the remote optional.
|
||||
var ErrNoFloor = errors.New("stt: no floor transcriber")
|
||||
|
||||
// NewPair builds the two-transcriber arrangement. remote may be nil, which is
|
||||
// the unconfigured deploy: every turn goes to the floor and nothing probes.
|
||||
func NewPair(remote, floor Transcriber, health string, interval time.Duration) *Pair {
|
||||
if interval <= 0 {
|
||||
// The config normalises this, so a zero here is a caller that built the
|
||||
// Pair directly. Panicking in a ticker is the wrong way to say so.
|
||||
interval = defaultProbeInterval
|
||||
}
|
||||
return &Pair{
|
||||
remote: remote,
|
||||
floor: floor,
|
||||
health: health,
|
||||
interval: interval,
|
||||
http: &http.Client{Timeout: probeTimeout},
|
||||
stop: make(chan struct{}),
|
||||
}
|
||||
}
|
||||
|
||||
// Start begins probing. The first probe runs before the first tick, so a
|
||||
// workstation that is already up serves the first utterance rather than the
|
||||
// second. Safe with a nil remote.
|
||||
func (p *Pair) Start(ctx context.Context) {
|
||||
if p.remote == nil || p.health == "" {
|
||||
return
|
||||
}
|
||||
go func() {
|
||||
p.probe(ctx)
|
||||
t := time.NewTicker(p.interval)
|
||||
defer t.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-p.stop:
|
||||
return
|
||||
case <-t.C:
|
||||
p.probe(ctx)
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
// Stop ends the prober. Idempotent and safe from two goroutines.
|
||||
func (p *Pair) Stop() {
|
||||
p.stopOnce.Do(func() { close(p.stop) })
|
||||
}
|
||||
|
||||
// Available reports whether the workstation will transcribe right now.
|
||||
func (p *Pair) Available() bool {
|
||||
return p.remote != nil && p.up.Load()
|
||||
}
|
||||
|
||||
func (p *Pair) probe(ctx context.Context) {
|
||||
ctx, cancel := context.WithTimeout(ctx, probeTimeout)
|
||||
defer cancel()
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, p.health, nil)
|
||||
if err != nil {
|
||||
p.set(false)
|
||||
return
|
||||
}
|
||||
resp, err := p.http.Do(req)
|
||||
if err != nil {
|
||||
p.set(false)
|
||||
return
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
p.set(resp.StatusCode == http.StatusOK)
|
||||
}
|
||||
|
||||
// set records the admission answer and logs only transitions. A machine that
|
||||
// sleeps nightly would otherwise write one line per interval forever.
|
||||
func (p *Pair) set(up bool) {
|
||||
if p.up.Swap(up) == up {
|
||||
return
|
||||
}
|
||||
if up {
|
||||
log.Printf("stt: workstation transcriber available at %s", p.health)
|
||||
} else {
|
||||
log.Print("stt: workstation transcriber unavailable, falling back to mavsttd")
|
||||
}
|
||||
}
|
||||
|
||||
// Transcribe sends the audio to the workstation when it will take work, and to
|
||||
// mavsttd otherwise. A remote that fails mid-request falls back too, because
|
||||
// the admission answer is a cache and can be one interval out of date.
|
||||
//
|
||||
// Killing the remote mid-session must not drop the turn. That is the whole
|
||||
// point of the floor, and it is what TestPairFallsBackWhenRemoteFails pins.
|
||||
func (p *Pair) Transcribe(ctx context.Context, a audio.Audio) (string, float64, error) {
|
||||
if p.floor == nil {
|
||||
return "", 0, ErrNoFloor
|
||||
}
|
||||
if p.Available() {
|
||||
text, conf, err := p.remote.Transcribe(ctx, a)
|
||||
if err == nil {
|
||||
log.Print("stt: transcribed on the workstation")
|
||||
return text, conf, nil
|
||||
}
|
||||
// The cached answer was wrong. Correct it now rather than sending the
|
||||
// next utterance into the same hole, then fall back.
|
||||
p.set(false)
|
||||
log.Printf("stt: workstation failed mid-request, falling back: %v", err)
|
||||
}
|
||||
return p.floor.Transcribe(ctx, a)
|
||||
}
|
||||
|
||||
var _ Transcriber = (*Pair)(nil)
|
||||
@@ -0,0 +1,143 @@
|
||||
package stt
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/kami/maven/internal/audio"
|
||||
)
|
||||
|
||||
// scripted — a Transcriber that answers with a fixed text, or fails.
|
||||
type scripted struct {
|
||||
text string
|
||||
err error
|
||||
calls atomic.Int32
|
||||
}
|
||||
|
||||
func (s *scripted) Transcribe(_ context.Context, _ audio.Audio) (string, float64, error) {
|
||||
s.calls.Add(1)
|
||||
if s.err != nil {
|
||||
return "", 0, s.err
|
||||
}
|
||||
return s.text, 0.9, nil
|
||||
}
|
||||
|
||||
func sample() audio.Audio {
|
||||
return audio.Audio{Format: audio.PCM16kMono, Bytes: make([]byte, 3200)}
|
||||
}
|
||||
|
||||
// up builds a Pair whose admission answer is already true, without probing.
|
||||
func up(remote, floor Transcriber) *Pair {
|
||||
p := NewPair(remote, floor, "", time.Minute)
|
||||
p.up.Store(true)
|
||||
return p
|
||||
}
|
||||
|
||||
func TestPairPrefersTheWorkstation(t *testing.T) {
|
||||
t.Parallel()
|
||||
remote := &scripted{text: "с рабочей станции"}
|
||||
floor := &scripted{text: "с homesrv"}
|
||||
text, _, err := up(remote, floor).Transcribe(context.Background(), sample())
|
||||
if err != nil {
|
||||
t.Fatalf("Transcribe: %v", err)
|
||||
}
|
||||
if text != "с рабочей станции" {
|
||||
t.Fatalf("want the remote transcript, got %q", text)
|
||||
}
|
||||
if floor.calls.Load() != 0 {
|
||||
t.Fatalf("floor was called %d times, want 0", floor.calls.Load())
|
||||
}
|
||||
}
|
||||
|
||||
// The turn is what matters. A remote that dies mid-session must cost a worse
|
||||
// transcript and nothing else. This is the V-486 bar.
|
||||
func TestPairFallsBackWhenRemoteFails(t *testing.T) {
|
||||
t.Parallel()
|
||||
remote := &scripted{err: errors.New("connection refused")}
|
||||
floor := &scripted{text: "с homesrv"}
|
||||
p := up(remote, floor)
|
||||
|
||||
text, conf, err := p.Transcribe(context.Background(), sample())
|
||||
if err != nil {
|
||||
t.Fatalf("a failed remote must not fail the turn: %v", err)
|
||||
}
|
||||
if text != "с homesrv" {
|
||||
t.Fatalf("want the floor transcript, got %q", text)
|
||||
}
|
||||
if conf != 0.9 {
|
||||
t.Fatalf("want the floor confidence, got %v", conf)
|
||||
}
|
||||
if p.Available() {
|
||||
t.Fatal("a failed request must correct the cached admission answer")
|
||||
}
|
||||
|
||||
// The next utterance goes straight to the floor rather than into the
|
||||
// same hole.
|
||||
if _, _, err := p.Transcribe(context.Background(), sample()); err != nil {
|
||||
t.Fatalf("second turn: %v", err)
|
||||
}
|
||||
if remote.calls.Load() != 1 {
|
||||
t.Fatalf("remote called %d times, want 1", remote.calls.Load())
|
||||
}
|
||||
}
|
||||
|
||||
func TestPairWithNoRemoteIsTheFloor(t *testing.T) {
|
||||
t.Parallel()
|
||||
floor := &scripted{text: "с homesrv"}
|
||||
p := NewPair(nil, floor, "", time.Minute)
|
||||
p.Start(context.Background()) // no health url, so this is a no-op
|
||||
if p.Available() {
|
||||
t.Fatal("an unconfigured remote is never available")
|
||||
}
|
||||
text, _, err := p.Transcribe(context.Background(), sample())
|
||||
if err != nil {
|
||||
t.Fatalf("Transcribe: %v", err)
|
||||
}
|
||||
if text != "с homesrv" {
|
||||
t.Fatalf("want the floor transcript, got %q", text)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPairWithNoFloorRefuses(t *testing.T) {
|
||||
t.Parallel()
|
||||
_, _, err := NewPair(nil, nil, "", time.Minute).Transcribe(context.Background(), sample())
|
||||
if !errors.Is(err, ErrNoFloor) {
|
||||
t.Fatalf("want ErrNoFloor, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPairProbeReadsHealth(t *testing.T) {
|
||||
t.Parallel()
|
||||
var ok atomic.Bool
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||
if !ok.Load() {
|
||||
w.WriteHeader(http.StatusServiceUnavailable)
|
||||
return
|
||||
}
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
p := NewPair(&scripted{text: "remote"}, &scripted{text: "floor"}, srv.URL, time.Minute)
|
||||
p.probe(context.Background())
|
||||
if p.Available() {
|
||||
t.Fatal("a 503 means the card is busy, so the workstation is not available")
|
||||
}
|
||||
ok.Store(true)
|
||||
p.probe(context.Background())
|
||||
if !p.Available() {
|
||||
t.Fatal("a 200 means the workstation will take work")
|
||||
}
|
||||
}
|
||||
|
||||
func TestPairStopIsIdempotent(t *testing.T) {
|
||||
t.Parallel()
|
||||
p := NewPair(nil, &scripted{}, "", time.Minute)
|
||||
p.Stop()
|
||||
p.Stop()
|
||||
}
|
||||
+118
-65
@@ -9,15 +9,18 @@
|
||||
//
|
||||
// The wire is symmetric: a Request from the client is answered by a
|
||||
// Response with a matching ID, OR a server-initiated Push frame (no ID)
|
||||
// may arrive interleaved. SendRequest loops reading frames, drops Push
|
||||
// frames to the harness if a receiver is running (or silently if not),
|
||||
// and returns the first Response with the matching ID.
|
||||
// may arrive interleaved. One reader goroutine per connection owns the
|
||||
// socket. It hands each Response to whichever SendRequest is waiting on
|
||||
// that ID and each Push to the handler, so a client may send and listen
|
||||
// at the same time on one conn. mavwaked needs exactly that: it speaks
|
||||
// utterances and it must hear nudges, and the server routes a nudge to
|
||||
// the session that spoke most recently, so a second listening conn would
|
||||
// never be picked (V-671).
|
||||
package voice
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net"
|
||||
@@ -41,13 +44,22 @@ type PushHandler interface {
|
||||
// Client — one connection to the voice.Server.
|
||||
type Client struct {
|
||||
addr string
|
||||
mu sync.Mutex
|
||||
c net.Conn
|
||||
nextID atomic.Uint64
|
||||
|
||||
// pushCh fan-out: a reader goroutine (started by RunPushReceiver)
|
||||
// writes Push frames here; SendRequest also drains it when no reader
|
||||
// is running (drops the frame in that case).
|
||||
mu sync.Mutex
|
||||
c net.Conn
|
||||
// pending holds one channel per in-flight request, keyed by frame id.
|
||||
// The reader goroutine delivers the Response here and deletes the entry.
|
||||
pending map[uint64]chan *Response
|
||||
// dead is closed by the reader goroutine when this conn ends, so a
|
||||
// waiting SendRequest fails at once instead of at its own deadline.
|
||||
dead chan struct{}
|
||||
|
||||
// wmu serialises writes. Frames must not interleave on the wire.
|
||||
wmu sync.Mutex
|
||||
|
||||
// pushH is set by RunPushReceiver and survives a reconnect, because the
|
||||
// client that wants pushes wants them on whatever conn it ends up with.
|
||||
pushMu sync.Mutex
|
||||
pushH PushHandler
|
||||
}
|
||||
@@ -55,6 +67,15 @@ type Client struct {
|
||||
// Dial returns a Client that will connect to addr on first use.
|
||||
func Dial(addr string) *Client { return &Client{addr: addr} }
|
||||
|
||||
// Connect opens the conn now rather than on the first request. mavwaked calls
|
||||
// it at startup: the server registers a session on accept, and a client that
|
||||
// has never connected cannot be sent a nudge.
|
||||
func (c *Client) Connect(ctx context.Context) error {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
return c.ensureConnLocked(ctx)
|
||||
}
|
||||
|
||||
// Close releases the conn. Idempotent.
|
||||
func (c *Client) Close() error {
|
||||
c.mu.Lock()
|
||||
@@ -75,12 +96,14 @@ func (c *Client) PushToTalk(ctx context.Context, a audio.Audio, lang string) (Pu
|
||||
return out, err
|
||||
}
|
||||
|
||||
// requestTimeout bounds a round-trip with no deadline on its context. It is
|
||||
// generous because the far end runs speech-to-text, a router and a voice.
|
||||
const requestTimeout = 120 * time.Second
|
||||
|
||||
// SendRequest sends one Request frame and waits for the matching Response.
|
||||
// Push frames received while waiting are dropped on the floor UNLESS a
|
||||
// PushHandler has been wired via RunPushReceiver, in which case the handler
|
||||
// is invoked inline (still synchronous with the SendRequest caller's
|
||||
// read). For sanity, the reference client runs either one-shot (no
|
||||
// receiver) or interactive (RunPushReceiver, no concurrent SendRequest).
|
||||
// Push frames arriving meanwhile go to the handler on the reader goroutine,
|
||||
// so listening and sending on one Client is supported rather than merely
|
||||
// tolerated.
|
||||
func (c *Client) SendRequest(ctx context.Context, m Method, params any, out any) error {
|
||||
body, err := marshalParams(params)
|
||||
if err != nil {
|
||||
@@ -94,58 +117,69 @@ func (c *Client) SendRequest(ctx context.Context, m Method, params any, out any)
|
||||
c.mu.Unlock()
|
||||
return err
|
||||
}
|
||||
conn := c.c
|
||||
conn, dead := c.c, c.dead
|
||||
ch := make(chan *Response, 1)
|
||||
c.pending[id] = ch
|
||||
c.mu.Unlock()
|
||||
|
||||
if dl, ok := ctx.Deadline(); ok {
|
||||
_ = conn.SetDeadline(dl)
|
||||
} else {
|
||||
_ = conn.SetDeadline(time.Now().Add(120 * time.Second))
|
||||
}
|
||||
defer conn.SetDeadline(time.Time{})
|
||||
|
||||
if err := writeFrame(conn, &req); err != nil {
|
||||
c.teardown()
|
||||
c.wmu.Lock()
|
||||
err = writeFrame(conn, &req)
|
||||
c.wmu.Unlock()
|
||||
if err != nil {
|
||||
c.forget(id)
|
||||
c.teardownConn(conn)
|
||||
return err
|
||||
}
|
||||
for {
|
||||
resp, push, err := readOneFrame(conn)
|
||||
if err != nil {
|
||||
c.teardown()
|
||||
return err
|
||||
}
|
||||
if push != nil {
|
||||
c.deliverPush(*push)
|
||||
continue
|
||||
}
|
||||
if resp.ID != id {
|
||||
continue // not ours; ignore (singleplex ⇒ shouldn't happen)
|
||||
}
|
||||
if resp.Error != nil {
|
||||
return hydrate(resp.Error)
|
||||
}
|
||||
if out != nil {
|
||||
if err := json.Unmarshal(resp.Result, out); err != nil {
|
||||
return fmt.Errorf("voice: unmarshal result: %w", err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
|
||||
timer := time.NewTimer(requestTimeout)
|
||||
defer timer.Stop()
|
||||
|
||||
var resp *Response
|
||||
select {
|
||||
case resp = <-ch:
|
||||
case <-dead:
|
||||
c.forget(id)
|
||||
return fmt.Errorf("voice: connection closed before reply")
|
||||
case <-ctx.Done():
|
||||
c.forget(id)
|
||||
return ctx.Err()
|
||||
case <-timer.C:
|
||||
c.forget(id)
|
||||
c.teardownConn(conn)
|
||||
return fmt.Errorf("voice: no reply within %s", requestTimeout)
|
||||
}
|
||||
|
||||
if resp.Error != nil {
|
||||
return hydrate(resp.Error)
|
||||
}
|
||||
if out != nil {
|
||||
if err := json.Unmarshal(resp.Result, out); err != nil {
|
||||
return fmt.Errorf("voice: unmarshal result: %w", err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// RunPushReceiver spawns a reader goroutine that delivers Push frames to h
|
||||
// until the conn closes or Close is called. Today's reference client uses
|
||||
// this in -listen mode (proactive voice playback). SendRequest and
|
||||
// RunPushReceiver SHOULD NOT be used concurrently on the same Client — the
|
||||
// wire is singleplex at the reference client's scale; production picks one
|
||||
// mode per conn. Returns when the goroutine ends (ctx cancel or conn close).
|
||||
// forget drops an abandoned request so a late Response is discarded rather
|
||||
// than delivered to nobody.
|
||||
func (c *Client) forget(id uint64) {
|
||||
c.mu.Lock()
|
||||
delete(c.pending, id)
|
||||
c.mu.Unlock()
|
||||
}
|
||||
|
||||
// RunPushReceiver wires h and blocks until the conn ends or ctx is
|
||||
// cancelled. Frames are read by the per-conn reader goroutine, so a client
|
||||
// may call SendRequest on the same Client while this is running. Returns nil
|
||||
// when the conn ended, so a caller that wants to stay reachable reconnects
|
||||
// and calls it again.
|
||||
func (c *Client) RunPushReceiver(ctx context.Context, h PushHandler) error {
|
||||
c.mu.Lock()
|
||||
if err := c.ensureConnLocked(ctx); err != nil {
|
||||
c.mu.Unlock()
|
||||
return err
|
||||
}
|
||||
conn := c.c
|
||||
dead := c.dead
|
||||
c.mu.Unlock()
|
||||
|
||||
c.pushMu.Lock()
|
||||
@@ -158,21 +192,34 @@ func (c *Client) RunPushReceiver(ctx context.Context, h PushHandler) error {
|
||||
c.pushMu.Unlock()
|
||||
}()
|
||||
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case <-dead:
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// readLoop owns conn for its whole life. It ends on any read error, which is
|
||||
// how a closed conn, a killed server and a cancelled dial all arrive here.
|
||||
func (c *Client) readLoop(conn net.Conn, dead chan struct{}) {
|
||||
defer close(dead)
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
default:
|
||||
}
|
||||
_, push, err := readOneFrame(conn)
|
||||
resp, push, err := readOneFrame(conn)
|
||||
if err != nil {
|
||||
if errors.Is(err, io.EOF) || errors.Is(err, net.ErrClosed) {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
c.teardownConn(conn)
|
||||
return
|
||||
}
|
||||
if push != nil {
|
||||
c.deliverPush(*push)
|
||||
continue
|
||||
}
|
||||
c.mu.Lock()
|
||||
ch := c.pending[resp.ID]
|
||||
delete(c.pending, resp.ID)
|
||||
c.mu.Unlock()
|
||||
if ch != nil {
|
||||
ch <- resp
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -196,13 +243,19 @@ func (c *Client) ensureConnLocked(ctx context.Context) error {
|
||||
return fmt.Errorf("voice: dial %s: %w", c.addr, err)
|
||||
}
|
||||
c.c = conn
|
||||
c.pending = make(map[uint64]chan *Response)
|
||||
c.dead = make(chan struct{})
|
||||
go c.readLoop(conn, c.dead)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *Client) teardown() {
|
||||
// teardownConn closes conn and forgets it, but only if it is still the live
|
||||
// one. A reconnect may already have replaced it, and closing the new conn
|
||||
// because the old one died takes the client down on every hiccup.
|
||||
func (c *Client) teardownConn(conn net.Conn) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
if c.c != nil {
|
||||
if c.c != nil && c.c == conn {
|
||||
_ = c.c.Close()
|
||||
c.c = nil
|
||||
}
|
||||
|
||||
@@ -221,3 +221,113 @@ func TestClientListenModeReceivesPush(t *testing.T) {
|
||||
type pushHandlerFunc func(Push)
|
||||
|
||||
func (f pushHandlerFunc) OnPush(p Push) { f(p) }
|
||||
|
||||
// The whole point of the per-conn reader (V-671): mavwaked speaks utterances
|
||||
// and must hear nudges, and the server routes a nudge to the session that
|
||||
// spoke most recently. A second listening conn would never be picked, so both
|
||||
// directions have to share one conn.
|
||||
func TestClientSendsAndListensOnOneConn(t *testing.T) {
|
||||
l := newTestListener(t)
|
||||
sess := NewSessions()
|
||||
h := &stubHandler{}
|
||||
srv := NewServer(l.Addr().String(), h, sess)
|
||||
if err := srv.Listen(); err != nil {
|
||||
t.Fatalf("listen: %v", err)
|
||||
}
|
||||
defer func() {
|
||||
_ = srv.Close()
|
||||
waitPort()
|
||||
}()
|
||||
go func() { _ = srv.Serve() }()
|
||||
|
||||
c := Dial(l.Addr().String())
|
||||
defer c.Close()
|
||||
|
||||
got := make(chan audio.Audio, 4)
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
go func() {
|
||||
_ = c.RunPushReceiver(ctx, pushHandlerFunc(func(p Push) {
|
||||
var ap AudioNudgePush
|
||||
if err := json.Unmarshal(p.Params, &ap); err == nil {
|
||||
got <- ap.Audio
|
||||
}
|
||||
}))
|
||||
}()
|
||||
|
||||
for i := 0; i < 100 && sess.Active() < 1; i++ {
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
}
|
||||
if sess.Active() != 1 {
|
||||
t.Fatalf("active sessions = %d, want exactly 1", sess.Active())
|
||||
}
|
||||
|
||||
// A round-trip while the receiver is running. Before the reader owned the
|
||||
// conn, this and the receiver raced for every frame.
|
||||
resp, err := c.PushToTalk(context.Background(), audio.Audio{Format: audio.PCM16kMono, Bytes: []byte("hello")}, "ru")
|
||||
if err != nil {
|
||||
t.Fatalf("PushToTalk with a receiver running: %v", err)
|
||||
}
|
||||
if resp.ReplyText != "got it" {
|
||||
t.Fatalf("ReplyText = %q, want %q", resp.ReplyText, "got it")
|
||||
}
|
||||
|
||||
// And the nudge still arrives, on the session that just spoke.
|
||||
err = sess.PushToMostRecent(context.Background(), AudioNudgePush{
|
||||
RuleName: "after-speaking",
|
||||
Audio: audio.Audio{Format: audio.PCM16kMono, Bytes: []byte("proactive")},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("PushToMostRecent: %v", err)
|
||||
}
|
||||
select {
|
||||
case a := <-got:
|
||||
if string(a.Bytes) != "proactive" {
|
||||
t.Fatalf("received %q, want the nudge audio", string(a.Bytes))
|
||||
}
|
||||
case <-time.After(2 * time.Second):
|
||||
t.Fatal("nudge never reached the handler after the client had spoken")
|
||||
}
|
||||
}
|
||||
|
||||
// A request abandoned by its context must not leave its slot behind, or a
|
||||
// long-running client leaks one channel per timeout.
|
||||
func TestClientForgetsAbandonedRequests(t *testing.T) {
|
||||
l := newTestListener(t)
|
||||
sess := NewSessions()
|
||||
srv := NewServer(l.Addr().String(), &blockingHandler{}, sess)
|
||||
if err := srv.Listen(); err != nil {
|
||||
t.Fatalf("listen: %v", err)
|
||||
}
|
||||
defer func() {
|
||||
_ = srv.Close()
|
||||
waitPort()
|
||||
}()
|
||||
go func() { _ = srv.Serve() }()
|
||||
|
||||
c := Dial(l.Addr().String())
|
||||
defer c.Close()
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
|
||||
defer cancel()
|
||||
_, err := c.PushToTalk(ctx, audio.Audio{Format: audio.PCM16kMono, Bytes: []byte("x")}, "ru")
|
||||
if err == nil {
|
||||
t.Fatal("expected the round-trip to fail on its context")
|
||||
}
|
||||
|
||||
c.mu.Lock()
|
||||
n := len(c.pending)
|
||||
c.mu.Unlock()
|
||||
if n != 0 {
|
||||
t.Fatalf("pending = %d after an abandoned request, want 0", n)
|
||||
}
|
||||
}
|
||||
|
||||
// blockingHandler never answers, so the client's context is what ends the
|
||||
// round-trip.
|
||||
type blockingHandler struct{}
|
||||
|
||||
func (blockingHandler) HandlePushToTalk(ctx context.Context, _ PushToTalkReq, _ uint64) (PushToTalkResp, error) {
|
||||
<-ctx.Done()
|
||||
return PushToTalkResp{}, ctx.Err()
|
||||
}
|
||||
|
||||
@@ -54,7 +54,11 @@ def turn(text):
|
||||
q = urllib.parse.parse_qs(urllib.parse.urlparse(loc).query)
|
||||
return {
|
||||
"reply": q.get("r", [""])[0],
|
||||
"source": q.get("src", q.get("source", [""]))[0],
|
||||
# "s", not "src". cmd/mavweb/chat.go writes the badge under that
|
||||
# name, and reading the wrong one cost both fortnight runs their
|
||||
# source column: every finding in those docs is inferred from the
|
||||
# reply wording instead.
|
||||
"source": q.get("s", [""])[0],
|
||||
"trace": q.get("t", [""])[0],
|
||||
"secs": dt,
|
||||
}
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
# Raw transcript, two weeks of usage
|
||||
|
||||
YOU: что такое TCP?
|
||||
MAVEN: есть такое: Tcp/Ip — Википедия — TCP позволяет регулировать нагрузку на сеть, а также уменьшать время ожидания данных при передаче на большие расстояния. Более того, TCP гарантирует, что полученные ...
|
||||
[2.1s src=search t=666]
|
||||
|
||||
YOU: кто такой Линус Торвальдс?
|
||||
MAVEN: вот что я нашла: Торвальдс, Линус — Википедия — Ли́нус Бенедикт То́рвальдс (встречается написание Ту́рвальдс, швед. Linus Benedict Torvalds МФА: [ˈliːn.ɵs ˈtuːr.valds] о файле; род. 28 декабря 1969, Хельсинки) — финско-американский программист, создатель ядра Linux.
|
||||
[0.8s src=search t=667]
|
||||
|
||||
YOU: что такое фотосинтез
|
||||
MAVEN: нашла вот это: Фотосинтез — Фотосинтез
|
||||
Схематическое изображение процесса фотосинтеза, происходящего в растениях
|
||||
Упрощённое уравнение фотосинтеза, происходящего в растениях
|
||||
Упрощённая схема фотосинтеза
|
||||
Фотоси́нтез (от др.-греч. φῶς — « свет » и σύνθεσις — «соединённый», «складной», «связывание», « синтез ») — сл…
|
||||
[0.6s src=kiwix t=668]
|
||||
|
||||
YOU: кто написал Войну и мир
|
||||
MAVEN: вот что я нашла: Война и мир — Война и мир
|
||||
Война и мир
|
||||
Война и миръ
|
||||
Обложка издания 1873 года
|
||||
Жанр
|
||||
роман-эпопея
|
||||
Автор
|
||||
Лев Николаевич Толстой
|
||||
Язык оригинала
|
||||
русский, фрагменты на французском и немецком языках
|
||||
Дата написания
|
||||
1863—1869, 1873
|
||||
Дата первой публикации
|
||||
1865—1869
|
||||
Предыдущее
|
||||
Казаки
|
||||
Следующее
|
||||
Анна Каренина
|
||||
Тек…
|
||||
[12.4s src=kiwix t=669]
|
||||
|
||||
```
|
||||
@@ -0,0 +1,4 @@
|
||||
{"reply": "есть такое: Tcp/Ip — Википедия — TCP позволяет регулировать нагрузку на сеть, а также уменьшать время ожидания данных при передаче на большие расстояния. Более того, TCP гарантирует, что полученные ...", "source": "search", "trace": "666", "secs": 2.1235475060530007, "day": 0, "n": 1, "utterance": "что такое TCP?"}
|
||||
{"reply": "вот что я нашла: Торвальдс, Линус — Википедия — Ли́нус Бенедикт То́рвальдс (встречается написание Ту́рвальдс, швед. Linus Benedict Torvalds МФА: [ˈliːn.ɵs ˈtuːr.valds] о файле; род. 28 декабря 1969, Хельсинки) — финско-американский программист, создатель ядра Linux.", "source": "search", "trace": "667", "secs": 0.7689587379572913, "day": 0, "n": 2, "utterance": "кто такой Линус Торвальдс?"}
|
||||
{"reply": "нашла вот это: Фотосинтез — Фотосинтез\nСхематическое изображение процесса фотосинтеза, происходящего в растениях\nУпрощённое уравнение фотосинтеза, происходящего в растениях\nУпрощённая схема фотосинтеза\nФотоси́нтез (от др.-греч. φῶς — « свет » и σύνθεσις — «соединённый», «складной», «связывание», « синтез ») — сл…", "source": "kiwix", "trace": "668", "secs": 0.575586927938275, "day": 0, "n": 3, "utterance": "что такое фотосинтез"}
|
||||
{"reply": "вот что я нашла: Война и мир — Война и мир\nВойна и мир\nВойна и миръ\nОбложка издания 1873 года\nЖанр\nроман-эпопея\nАвтор\nЛев Николаевич Толстой\nЯзык оригинала\nрусский, фрагменты на французском и немецком языках\nДата написания\n1863—1869, 1873\nДата первой публикации\n1865—1869\nПредыдущее\nКазаки\nСледующее\nАнна Каренина\nТек…", "source": "kiwix", "trace": "669", "secs": 12.44599153404124, "day": 0, "n": 4, "utterance": "кто написал Войну и мир"}
|
||||
@@ -0,0 +1,30 @@
|
||||
# Raw transcript, two weeks of usage
|
||||
|
||||
YOU: столица Франции
|
||||
MAVEN: вот что я нашла: Франция — Википедия — Она отличается ... реками. На Сене в 145 км от пролива Ла-Манш расположена столица Франции Париж....
|
||||
[2.3s src=search t=670]
|
||||
|
||||
YOU: что такое фотосинтез
|
||||
MAVEN: есть такое: Фотосинтез: процесс, фазы, функции и значение фотосинтеза — Бестселлеры Фоксфорда: собрали всё, что полюбили тысячи школьников и родителей 🧡 · Фотосинтез происходит в тканях, содержащих хлоропласты, — преимущественно в листе, на который приходится большая часть процессов фотосинтеза. Такая ткань наз…
|
||||
[0.8s src=search t=671]
|
||||
|
||||
YOU: что такое TCP?
|
||||
MAVEN: нашла вот это: TCP — TCP
|
||||
TCP
|
||||
Название
|
||||
Transmission Control Protocol
|
||||
Уровень (по модели OSI )
|
||||
Транспортный
|
||||
Семейство
|
||||
TCP/IP
|
||||
Спецификация
|
||||
RFC 793 (сентябрь 1981 года) / STD 7
|
||||
Основные реализации
|
||||
UNIX , Linux , BSD , Windows
|
||||
Расширяемость
|
||||
Опции
|
||||
Медиафайлы на Викискладе
|
||||
TCP ( англ. Transmission Control Protocol — п…
|
||||
[0.3s src=kiwix t=672]
|
||||
|
||||
```
|
||||
@@ -0,0 +1,3 @@
|
||||
{"reply": "вот что я нашла: Франция — Википедия — Она отличается ... реками. На Сене в 145 км от пролива Ла-Манш расположена столица Франции Париж....", "source": "search", "trace": "670", "secs": 2.295847482047975, "day": 0, "n": 1, "utterance": "столица Франции"}
|
||||
{"reply": "есть такое: Фотосинтез: процесс, фазы, функции и значение фотосинтеза — Бестселлеры Фоксфорда: собрали всё, что полюбили тысячи школьников и родителей 🧡 · Фотосинтез происходит в тканях, содержащих хлоропласты, — преимущественно в листе, на который приходится большая часть процессов фотосинтеза. Такая ткань наз…", "source": "search", "trace": "671", "secs": 0.8353607719764113, "day": 0, "n": 2, "utterance": "что такое фотосинтез"}
|
||||
{"reply": "нашла вот это: TCP — TCP\nTCP\nНазвание\nTransmission Control Protocol\nУровень (по модели OSI )\nТранспортный\nСемейство\nTCP/IP\nСпецификация\nRFC 793 (сентябрь 1981 года) / STD 7\nОсновные реализации\nUNIX , Linux , BSD , Windows\nРасширяемость\nОпции\nМедиафайлы на Викискладе\nTCP ( англ. Transmission Control Protocol — п…", "source": "kiwix", "trace": "672", "secs": 0.29547085403464735, "day": 0, "n": 3, "utterance": "что такое TCP?"}
|
||||
@@ -0,0 +1,19 @@
|
||||
# Raw transcript, two weeks of usage
|
||||
|
||||
YOU: что такое TCP?
|
||||
MAVEN: Вот что я нашла: TCP, или Transmission Control Protocol, — это один из основных протоколов передачи данных в интернете. Он работает на транспортном уровне модели OSI и предоставляет поток данных с установкой соединения, гарантируя целостность данных, повторно запрашивая потерянные данные и устраняя дублирование.
|
||||
[2.3s src=kiwix t=673]
|
||||
|
||||
YOU: кто такой Линус Торвальдс?
|
||||
MAVEN: Вот что я нашла: Линус Торвальдс — финно-американский программист, который является создателем ядра Linux. Он швед по имени Linus Benedict Torvalds, родился 28 декабря 1969 года в Хельсинки, Финляндия, и его род деятельности включает программирование, разработку программного обеспечения, писательство, изобретательство, ведение блога, а также он инженер, бизнесмен и специалист в области информатики.
|
||||
[2.4s src=kiwix t=674]
|
||||
|
||||
YOU: столица Франции
|
||||
MAVEN: Вот что я нашла: Столица Франции — Париж.
|
||||
[2.0s src=kiwix t=675]
|
||||
|
||||
YOU: почему небо голубое
|
||||
MAVEN: Я не нашла в источниках ответа на твой вопрос, почему небо голубое.
|
||||
[1.3s src=kiwix t=676]
|
||||
|
||||
```
|
||||
@@ -0,0 +1,4 @@
|
||||
{"reply": "Вот что я нашла: TCP, или Transmission Control Protocol, — это один из основных протоколов передачи данных в интернете. Он работает на транспортном уровне модели OSI и предоставляет поток данных с установкой соединения, гарантируя целостность данных, повторно запрашивая потерянные данные и устраняя дублирование.", "source": "kiwix", "trace": "673", "secs": 2.3279738989658654, "day": 0, "n": 1, "utterance": "что такое TCP?"}
|
||||
{"reply": "Вот что я нашла: Линус Торвальдс — финно-американский программист, который является создателем ядра Linux. Он швед по имени Linus Benedict Torvalds, родился 28 декабря 1969 года в Хельсинки, Финляндия, и его род деятельности включает программирование, разработку программного обеспечения, писательство, изобретательство, ведение блога, а также он инженер, бизнесмен и специалист в области информатики.", "source": "kiwix", "trace": "674", "secs": 2.42521625792142, "day": 0, "n": 2, "utterance": "кто такой Линус Торвальдс?"}
|
||||
{"reply": "Вот что я нашла: Столица Франции — Париж.", "source": "kiwix", "trace": "675", "secs": 1.9803519028937444, "day": 0, "n": 3, "utterance": "столица Франции"}
|
||||
{"reply": "Я не нашла в источниках ответа на твой вопрос, почему небо голубое.", "source": "kiwix", "trace": "676", "secs": 1.2865088270045817, "day": 0, "n": 4, "utterance": "почему небо голубое"}
|
||||
Reference in New Issue
Block a user