Compare commits
33 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 229890abd7 | |||
| 0db9ca084c | |||
| 96d97e8964 | |||
| 02f6e8ad4a | |||
| 999a5ad562 | |||
| 2ea39a3d41 | |||
| 8fb6f2154d | |||
| c938148619 | |||
| 2512d686a1 | |||
| f44abcc526 | |||
| a99932b427 | |||
| 6d5801bb1f | |||
| 8aba4845bf | |||
| 2ec92ee8bf | |||
| 7c77a378c1 | |||
| 672eabc134 | |||
| a1a2fa3704 | |||
| 22a4978459 | |||
| 1456336652 | |||
| b975716759 | |||
| 4b1edb0617 | |||
| 944e553669 | |||
| cc32c2c4ab | |||
| a1e97c94ac | |||
| c7f59e48f4 | |||
| 4666057066 | |||
| 7138086c3f | |||
| 83e168f326 | |||
| a4abcdefa3 | |||
| 68a3c85186 | |||
| 88c086482e | |||
| feabf9f350 | |||
| 50c6637c1b |
@@ -70,3 +70,5 @@ coverage.out
|
|||||||
|
|
||||||
# root .env — MAVEN_AMBIENT_TOKEN and friends, same class as deploy/telegram.env
|
# root .env — MAVEN_AMBIENT_TOKEN and friends, same class as deploy/telegram.env
|
||||||
.env
|
.env
|
||||||
|
# silero-vad, downloaded (see AGENTS.md)
|
||||||
|
/models/vad/
|
||||||
|
|||||||
@@ -95,6 +95,22 @@ model: the code puts `query: ` in front of a question and `passage: ` in front
|
|||||||
of a stored note, which is how e5 was trained. The quantized file is the one
|
of a stored note, which is how e5 was trained. The quantized file is the one
|
||||||
that is downloaded, deployed and measured.
|
that is downloaded, deployed and measured.
|
||||||
|
|
||||||
|
## Voice activity model for mavwaked
|
||||||
|
|
||||||
|
`mavwaked` decides an utterance has started with silero-vad when `-vad-model`
|
||||||
|
points at it, and with an energy threshold when it does not. The model is 2.3MB
|
||||||
|
and is not committed:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
mkdir -p models/vad
|
||||||
|
curl -sL -o models/vad/silero_vad.onnx \
|
||||||
|
https://github.com/snakers4/silero-vad/raw/master/src/silero_vad/data/silero_vad.onnx
|
||||||
|
```
|
||||||
|
|
||||||
|
It needs the same `libonnxruntime.so` the embedder needs, passed as `-onnx-lib`
|
||||||
|
or read from `MAVEN_ONNX_LIB`. The measurement is
|
||||||
|
`docs/evals/2026-08-09-silero-vad.md`, and the tests skip without the file.
|
||||||
|
|
||||||
**Also need ONNX Runtime** (`libonnxruntime.so`):
|
**Also need ONNX Runtime** (`libonnxruntime.so`):
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
|
|||||||
@@ -47,6 +47,31 @@ free — `worldGap` in `cmd/mavend/worldmodel.go`, which the owner hears instead
|
|||||||
answer. A box with no `workstation` block behaves exactly as it did before the seam: naming
|
answer. A box with no `workstation` block behaves exactly as it did before the seam: naming
|
||||||
a gap requires a gap. The offload table in `docs/offload.md` says which caller is which.
|
a gap requires a gap. The offload table in `docs/offload.md` says which caller is which.
|
||||||
|
|
||||||
|
**Speech-to-text moved on 2026-08-09** (V-486). `sttSeam` in `cmd/mavend/voicewire.go`
|
||||||
|
builds an `stt.Pair` beside `modelSeam`, preferring CrisperWhisper 2.0 turbo on workpc
|
||||||
|
with mavsttd as the floor. It takes only the silent half of the rule. A worse
|
||||||
|
transcript is still a turn, so `stt.Pair` has no `TranscribeRemote`. The fallback is
|
||||||
|
never spoken. CW2 turbo scores **10.4% WER in Russian against 27.5%** for the `ggml-small.bin`
|
||||||
|
mavsttd loads, over 200 Golos clips
|
||||||
|
(`docs/evals/2026-08-09-crisperwhisper2-russian-wer.md`). It runs in Intended mode, not
|
||||||
|
Verbatim, though that corpus cannot separate the two.
|
||||||
|
**whisper.cpp cannot load CW2 at all.** It reads its language count off the vocabulary
|
||||||
|
size, and CW2's 51897 tokens shift seven special token ids. So it is not a second
|
||||||
|
endpoint on mavgpud. It is its own transformers service on port 8081
|
||||||
|
(`deploy/cw2/serve.py`), which Maven reaches directly. `stt.HTTPTranscriber`
|
||||||
|
posts raw PCM to it with a bearer token, because audio is the most sensitive thing that
|
||||||
|
crosses this seam. The switch is `workstation.stt` in
|
||||||
|
`deploy/mavend.json`, and deleting the block sends every utterance to mavsttd.
|
||||||
|
**mavgpud runs that service as a second child.** That is not an optimisation. CW2 is a
|
||||||
|
ROCm process on the same card, so it registers on the KFD like any contender. Under its own
|
||||||
|
systemd unit it made mavgpud evict llama-server every few seconds. That took the
|
||||||
|
gemma-4-12b arm down for eight minutes on 2026-08-09 before anyone noticed. The card needs
|
||||||
|
one owner. Any GPU service added beside this daemon has the same defect, so add it to
|
||||||
|
`cmd/mavgpud` and not to systemd. CW2 is on the yield clock and not the idle one. At 1.6GB
|
||||||
|
it denies the card to nobody, and unloading it would only send the next voice turn to the
|
||||||
|
homesrv floor.
|
||||||
|
Text-to-speech has not moved and piper on homesrv is still the only synthesizer.
|
||||||
|
|
||||||
## Build & test
|
## Build & test
|
||||||
|
|
||||||
CGO daemons (`mavend`, `mavsttd`, `mavttsd`, `mavenclient`) need the vendored toolchain
|
CGO daemons (`mavend`, `mavsttd`, `mavttsd`, `mavenclient`) need the vendored toolchain
|
||||||
@@ -230,11 +255,31 @@ re-run it, start a **second** llama-server on a fixed host port — the resident
|
|||||||
`--port 0` inside the container and no host process can reach it.
|
`--port 0` inside the container and no host process can reach it.
|
||||||
|
|
||||||
**The numbers above are the homesrv floor, not the ceiling.** With the workstation up, routing
|
**The numbers above are the homesrv floor, not the ceiling.** With the workstation up, routing
|
||||||
completes through `llm.Pair` against gemma-4-12b and scores **84.4% full / 93.5% intent-only at
|
completes through `llm.Pair` against the model mavgpud holds, which is better than the resident
|
||||||
p50 329ms** — better than the resident model and about 2.5× faster (`docs/evals/2026-08-02-workstation-gemma4-12b.md`,
|
model and about 2.5× faster. gemma-4-12b scored **84.4% full / 93.5% intent-only at p50 329ms**
|
||||||
Vikunja #485). The workstation is never assumed up, so both sets of numbers are live. Judge a
|
(`docs/evals/2026-08-02-workstation-gemma4-12b.md`, Vikunja #485). The workstation is never
|
||||||
|
assumed up, so both sets of numbers are live. Judge a
|
||||||
routing change against the classifier and the resident model, since those are what always answer.
|
routing change against the classifier and the resident model, since those are what always answer.
|
||||||
|
|
||||||
|
**The workstation runs gemma-4-E4B since 2026-08-09** (owner's call), and it is a
|
||||||
|
step down measured the same day (`docs/evals/2026-08-09-e4b-vs-12b-routing.md`).
|
||||||
|
Against a same-session 12B control it scores **83.3% full / 89.6% intent-only,
|
||||||
|
destination 19/33 against 23/33, at p50 294ms against 344ms**. So it costs four
|
||||||
|
destination cases and buys 50ms. Read destination as the finding: it names nothing
|
||||||
|
where the 12B names `recall` or `calendar`, which is safe but walks the whole chain.
|
||||||
|
It also has no MTP and cannot be given any here. The only `gemma4-assistant`
|
||||||
|
draft on disk is trained against the 12B's hidden states.
|
||||||
|
**Phrasing was the unmeasured half and it is measured now**
|
||||||
|
(`docs/evals/2026-08-09-e4b-phrasing.md`). E4B scores nudges 15/15 and the
|
||||||
|
36-case talk fixture **29/36 at p50 516ms**, against the resident model's 25/36
|
||||||
|
at p50 2.97s. Persona is clean: `lang`, `feminine` and `address` are all 36/36,
|
||||||
|
where the resident model loses three on `address`. Every failure is `ontopic`
|
||||||
|
and none is a parse error. The 2026-08-05 temperature sweep put this fixture's
|
||||||
|
ceiling at 30/36, because two reply cases fail at every temperature (V-537), and
|
||||||
|
both are in E4B's failure list. So the swap costs nothing here. One defect no
|
||||||
|
check catches: in chat E4B claims "Я записала несколько идей!" when nothing was
|
||||||
|
stored, which is a wrong claim about state.
|
||||||
|
|
||||||
**The intended third engine is not a generative model** (owner's call, 05-08-2026, V-546,
|
**The intended third engine is not a generative model** (owner's call, 05-08-2026, V-546,
|
||||||
`docs/plans/18-routing-heads-on-e5-small.md`). Routing has a bounded output space, so it is
|
`docs/plans/18-routing-heads-on-e5-small.md`). Routing has a bounded output space, so it is
|
||||||
classification, and the 118M multilingual-e5-small is already resident. Three heads on one
|
classification, and the 118M multilingual-e5-small is already resident. Three heads on one
|
||||||
@@ -308,9 +353,36 @@ the possessive agenda rules claim those cases at stage 0 and name nothing, so no
|
|||||||
label reaches the head. That is the same trade V-660 flagged and it wants the
|
label reaches the head. That is the same trade V-660 flagged and it wants the
|
||||||
owner's call.
|
owner's call.
|
||||||
|
|
||||||
**Nothing of this runs in Go.** The weights are `heads.pt` and `out/body_heads/`
|
**The heads run in Go and route every turn, since 08-08-2026** (V-664,
|
||||||
on workpc. Reaching the daemon needs an ONNX export and a caller. The resident
|
`docs/evals/2026-08-08-routing-heads-in-go.md`). This section used to say
|
||||||
e5-small must not be replaced by the copy, because recall depends on that file.
|
nothing of it ran. `RouterHeads` in `internal/router/heads.go` loads
|
||||||
|
`router_heads.onnx` and reads intent, destination and clarify off one forward
|
||||||
|
pass. It is stage 0b: after the grammars, **before** the resident model, and the
|
||||||
|
classifier is still behind both. Through the cascade it scores intent **96.9%**
|
||||||
|
and destination **75.8%** at p50 27.9ms. That beats the gemma-4-12b cascade,
|
||||||
|
84.4% and 72.7%, at a twelfth of its 329ms. The workstation stays the better
|
||||||
|
phraser and is no longer the better router.
|
||||||
|
|
||||||
|
Three rules around it. The **clarify head decides first**, before the intent
|
||||||
|
threshold. It answers a different question. A thin utterance scores low
|
||||||
|
intent by construction, so gating it cost 6 of 8 ambiguous cases. The
|
||||||
|
**destination head is read on `IntentQuery` only**, since no other intent
|
||||||
|
reaches `queryWalk`. And `headsThreshold` is 0.6, the measured knee: every value
|
||||||
|
to 0.85 drops right answers and keeps the same two wrong ones.
|
||||||
|
|
||||||
|
`voice.embedder.heads_path` is the whole switch. Empty, missing or unloadable
|
||||||
|
means the heads are nil and the cascade is byte-for-byte what shipped before
|
||||||
|
them. **It must never be pointed at `model_path`.** The resident e5-small must
|
||||||
|
not be replaced by the fine-tuned copy. Recall depends on that file scoring
|
||||||
|
what it scored.
|
||||||
|
|
||||||
|
**The hand-written tokenizer read every long word backwards** until this task
|
||||||
|
(`encodeWord`, `onnxembedder.go`). It cost recall@1 7.4 points and recall@3 11.1.
|
||||||
|
Nothing caught it because seeds and queries were mangled the same way, so cosine
|
||||||
|
survived. The heads found it. They are trained through transformers and read
|
||||||
|
through this. The embedder id now carries a tokenizer revision
|
||||||
|
(`@384/tok2`), so fixing the tokenizer triggers `ReembedAll` the way swapping the
|
||||||
|
model file does. Bump `tokenizerRev` on any change to what it emits.
|
||||||
|
|
||||||
`Confidence: 1.0` used to be hardcoded in `llmrouter.go`, so the LLM
|
`Confidence: 1.0` used to be hardcoded in `llmrouter.go`, so the LLM
|
||||||
path could never ask for clarification (6/6 refusal cases missed on the fixture) — Vikunja
|
path could never ask for clarification (6/6 refusal cases missed on the fixture) — Vikunja
|
||||||
@@ -448,13 +520,21 @@ they look rather than guess.
|
|||||||
**The personal boundary is the one exception and it is deliberate.** It guesses,
|
**The personal boundary is the one exception and it is deliberate.** It guesses,
|
||||||
so naming `SourceWorld` drops it. That is what stops it answering "кто такой
|
so naming `SourceWorld` drops it. That is what stops it answering "кто такой
|
||||||
Линус Торвальдс?" with "не нашла у тебя такой записи", which it did on
|
Линус Торвальдс?" with "не нашла у тебя такой записи", which it did on
|
||||||
2026-08-07. The cost is that a destination a model wrote can now take the
|
2026-08-07. `TestNamingRecallKeepsTheBoundary` pins the other half: naming
|
||||||
boundary off a turn. A question about him that the model calls `world` reaches
|
`SourceRecall` keeps the boundary in front of the world.
|
||||||
SearXNG, where today the boundary stops it. Only the utterance leaves the box,
|
|
||||||
never his notes or history, so this widens what is asked and not what is sent.
|
**Only a stage 0 grammar may drop it** (owner's call, 09-08-2026, V-666). The
|
||||||
`TestNamingRecallKeepsTheBoundary` pins the other half: naming `SourceRecall`
|
question of who is allowed to was open until then. Three deciders name a
|
||||||
keeps the boundary in front of the world. Whether a model may drop it at all is
|
destination and two of them infer it: the routing heads and the resident model.
|
||||||
the owner's call and has not been made.
|
An inferred `SourceWorld` on a question about him would reach SearXNG, and that
|
||||||
|
widens what is asked rather than costing a local answer. So `Decision.SourceAnchored`
|
||||||
|
carries the provenance. It is a field and not `Stage == 0`. Stage 0 also means
|
||||||
|
confidence 1.0 and an anchored claim band, and one of those could stop implying
|
||||||
|
the others. `queryWalk` reads it for the source marked `boundary: true` and for
|
||||||
|
no other. So every other guesser still comes off the turn, whoever named the
|
||||||
|
destination. `TestOnlyAGrammarMayDropTheBoundary` pins both directions.
|
||||||
|
`definitionQueryPattern` claims "кто такой X", so the 2026-08-07 case is still
|
||||||
|
anchored and still answered.
|
||||||
|
|
||||||
What comes out is only the sources that **guess**. Those decide a turn is theirs by
|
What comes out is only the sources that **guess**. Those decide a turn is theirs by
|
||||||
cosine against frozen seeds, then answer whatever they claimed. They hold no table
|
cosine against frozen seeds, then answer whatever they claimed. They hold no table
|
||||||
@@ -605,11 +685,28 @@ world questions, so she needs to read external sources. What replaces it:
|
|||||||
`wikipedia_ru_all_maxi_2026-02` verbatim** through `kiwix.book_ru`. The RU→EN rewriter
|
`wikipedia_ru_all_maxi_2026-02` verbatim** through `kiwix.book_ru`. The RU→EN rewriter
|
||||||
is the workaround for an English book and is skipped there. Kiwix catalog names come
|
is the workaround for an English book and is skipped there. Kiwix catalog names come
|
||||||
from the filename, not the `<name>` field.
|
from the filename, not the `<name>` field.
|
||||||
|
**That verbatim path sent the whole sentence to a keyword engine until 09-08-2026**
|
||||||
|
(V-668, `docs/evals/2026-08-09-kiwix-topic-retrieval.md`). Kiwix ranks by keyword
|
||||||
|
overlap, so the question words outrank the one word naming the article. "что такое TCP"
|
||||||
|
returned "Перехват TCP-соединения". "кто написал Войну и мир" returned an episode of
|
||||||
|
Doctor Who. `kiwix.Topic` drops the narrative request, the interrogative and a verb
|
||||||
|
behind one. `kiwix.TitlePath` tries the exact article first, since a ZIM is addressable
|
||||||
|
by title and a wrong title is a 404. Five of eight questions reach the right article
|
||||||
|
where they did not, one was already right, and nothing regressed. The title needs its
|
||||||
|
leading capital, so `TitleCandidates` tries the spoken form and then the capitalized
|
||||||
|
one. **"столица Франции" is answered by a title redirect to Париж**, which is the case
|
||||||
|
the 2026-08-05 measurement named as unreachable by any lexical signal. Both apply on the
|
||||||
|
verbatim path alone. The rewriter already reduces a question, and reducing twice takes
|
||||||
|
the topic off its input.
|
||||||
`Response.Empty()` is the whole gate and there is no quality threshold in front of it:
|
`Response.Empty()` is the whole gate and there is no quality threshold in front of it:
|
||||||
the three signals one could read were measured on 2026-08-05 and none of them separate a
|
the three signals one could read were measured on 2026-08-05 and none of them separate a
|
||||||
real question from an invented one. Token overlap would cost "столица Франции" its
|
real question from an invented one. Token overlap would cost "столица Франции" its
|
||||||
answer, because the answer is Париж and that word is not in the question. See
|
answer, because the answer is Париж and that word is not in the question. See
|
||||||
`docs/evals/2026-08-05-search-quality-signals.md` (V-539). **Which query source claimed
|
`docs/evals/2026-08-05-search-quality-signals.md` (V-539). **The embedder is not a
|
||||||
|
fourth signal**, measured 2026-08-09 (V-668). Query-to-passage cosine scores 0.79 to
|
||||||
|
0.91 on answerable questions and 0.75 to 0.84 on unanswerable ones, and the sets
|
||||||
|
overlap. The wrong TCP article scored 0.8653, above five of six unanswerable rows. It
|
||||||
|
measures topic and not whether the passage answers, so no threshold splits them. **Which query source claimed
|
||||||
a turn is readable on `/chat`** as a badge beside the reply, carried on
|
a turn is readable on `/chat`** as a badge beside the reply, carried on
|
||||||
`ipc.ChatReply.Source` and noted by `noteQuerySource` in `cmd/mavend/querysource.go`. It
|
`ipc.ChatReply.Source` and noted by `noteQuerySource` in `cmd/mavend/querysource.go`. It
|
||||||
rides the context, so `handleText` keeps the one string signature the mic, telegram and
|
rides the context, so `handleText` keeps the one string signature the mic, telegram and
|
||||||
|
|||||||
@@ -13,6 +13,7 @@ import (
|
|||||||
"github.com/kami/maven/internal/crawl"
|
"github.com/kami/maven/internal/crawl"
|
||||||
"github.com/kami/maven/internal/decision"
|
"github.com/kami/maven/internal/decision"
|
||||||
"github.com/kami/maven/internal/ipc"
|
"github.com/kami/maven/internal/ipc"
|
||||||
|
"github.com/kami/maven/internal/kiwix"
|
||||||
"github.com/kami/maven/internal/memory"
|
"github.com/kami/maven/internal/memory"
|
||||||
"github.com/kami/maven/internal/morning"
|
"github.com/kami/maven/internal/morning"
|
||||||
"github.com/kami/maven/internal/phraser"
|
"github.com/kami/maven/internal/phraser"
|
||||||
@@ -79,6 +80,15 @@ type querySource struct {
|
|||||||
// not named do not get to try. The lookups still run, because a named
|
// not named do not get to try. The lookups still run, because a named
|
||||||
// destination is evidence and not a promise.
|
// destination is evidence and not a promise.
|
||||||
guesses bool
|
guesses bool
|
||||||
|
|
||||||
|
// boundary — dropping this source widens what leaves the box, so only a
|
||||||
|
// literal pattern may do it (V-666, owner's call of 2026-08-09).
|
||||||
|
//
|
||||||
|
// Every other guesser costs an answer when it is wrongly taken off a turn.
|
||||||
|
// This one costs the rule that a question about him never reaches an
|
||||||
|
// upstream engine. A grammar read the words to name a destination. A model
|
||||||
|
// and a softmax both inferred one, and neither may spend that.
|
||||||
|
boundary bool
|
||||||
}
|
}
|
||||||
|
|
||||||
// querySources is the ordered chain actionQuery walks; first source to claim
|
// querySources is the ordered chain actionQuery walks; first source to claim
|
||||||
@@ -156,7 +166,7 @@ var querySources = []querySource{
|
|||||||
// below answers from the world's. A question about him that got this far
|
// below answers from the world's. A question about him that got this far
|
||||||
// has no answer in his data, and no outside source can supply one, so this
|
// has no answer in his data, and no outside source can supply one, so this
|
||||||
// stops the walk rather than let the encyclopedia and the model guess.
|
// stops the walk rather than let the encyclopedia and the model guess.
|
||||||
{name: "personal", answer: (*reactiveHandler).queryPersonal, dest: router.SourceRecall, guesses: true},
|
{name: "personal", answer: (*reactiveHandler).queryPersonal, dest: router.SourceRecall, guesses: true, boundary: true},
|
||||||
// The world, read live. Owner's ruling of 2026-08-02: a metasearch hit beats
|
// The world, read live. Owner's ruling of 2026-08-02: a metasearch hit beats
|
||||||
// a frozen ZIM, so SearXNG asks before Kiwix does. Nothing of his is at
|
// a frozen ZIM, so SearXNG asks before Kiwix does. Nothing of his is at
|
||||||
// stake by this point — the boundary above already stopped every question
|
// stake by this point — the boundary above already stopped every question
|
||||||
@@ -198,12 +208,17 @@ var querySources = []querySource{
|
|||||||
// No destination named ⇒ the table exactly as written, which is what shipped
|
// No destination named ⇒ the table exactly as written, which is what shipped
|
||||||
// before the field existed. That is the floor. The classifier arm names
|
// before the field existed. That is the floor. The classifier arm names
|
||||||
// nothing, so a box whose model is down routes queries the way it always did.
|
// nothing, so a box whose model is down routes queries the way it always did.
|
||||||
func queryWalk(dest router.Source) (walk, skipped []querySource) {
|
// The personal boundary is the one exception, and anchored is what buys it
|
||||||
|
// (V-666). A grammar matched a literal pattern to name the destination. The
|
||||||
|
// routing heads and the resident model inferred one, and an inferred SourceWorld
|
||||||
|
// takes the boundary off a question about him. That widens what is asked
|
||||||
|
// upstream rather than costing a local answer, so those two keep it.
|
||||||
|
func queryWalk(dest router.Source, anchored bool) (walk, skipped []querySource) {
|
||||||
if dest == router.SourceUnknown {
|
if dest == router.SourceUnknown {
|
||||||
return querySources, nil
|
return querySources, nil
|
||||||
}
|
}
|
||||||
for _, s := range querySources {
|
for _, s := range querySources {
|
||||||
if s.guesses && s.dest != dest {
|
if s.guesses && s.dest != dest && (anchored || !s.boundary) {
|
||||||
skipped = append(skipped, s)
|
skipped = append(skipped, s)
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
@@ -219,7 +234,7 @@ func (h *reactiveHandler) actionQuery(ctx context.Context, dec router.Decision)
|
|||||||
// (V-564). Finish names everyone below the winner.
|
// (V-564). Finish names everyone below the winner.
|
||||||
decision.Expect(ctx, decision.StageQuery, querySourceNames())
|
decision.Expect(ctx, decision.StageQuery, querySourceNames())
|
||||||
rec := decision.From(ctx)
|
rec := decision.From(ctx)
|
||||||
walk, skipped := queryWalk(dec.Source)
|
walk, skipped := queryWalk(dec.Source, dec.SourceAnchored)
|
||||||
for _, src := range skipped {
|
for _, src := range skipped {
|
||||||
rec.Note(decision.Claim{
|
rec.Note(decision.Claim{
|
||||||
Stage: decision.StageQuery, Claimant: src.name, Outcome: decision.NeverAsked,
|
Stage: decision.StageQuery, Claimant: src.name, Outcome: decision.NeverAsked,
|
||||||
@@ -912,6 +927,28 @@ func (h *reactiveHandler) queryKiwix(ctx context.Context, t *queryTurn) (string,
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The topic, not the sentence (V-668). Kiwix ranks by keyword overlap, so
|
||||||
|
// the question words outrank the one word that names the article: measured
|
||||||
|
// on 2026-08-09, "что такое TCP" returns "Перехват TCP-соединения" and
|
||||||
|
// "TCP" returns TCP. Only the verbatim path needs this. The rewriter
|
||||||
|
// already reduces a question to English keywords, and reducing twice would
|
||||||
|
// take the topic off the input it reads.
|
||||||
|
if verbatim {
|
||||||
|
if topic := kiwix.Topic(pattern); topic != "" {
|
||||||
|
// The article named exactly, before any ranking runs. A ZIM is
|
||||||
|
// addressable by title and a wrong title is a 404, so this either
|
||||||
|
// answers or costs one request that says nothing.
|
||||||
|
for _, cand := range kiwix.TitleCandidates(topic) {
|
||||||
|
page, err := h.kiwix.client.Article(ctxK, kiwix.TitlePath(book, cand), h.kiwix.runes)
|
||||||
|
if err == nil && page.Text != "" {
|
||||||
|
log.Printf("voice: kiwix: %q in %q → title hit %q", topic, book, page.Title)
|
||||||
|
return h.kiwixReply(ctx, t, page.Title, page.Text)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
pattern = topic
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
hits, err := h.kiwix.client.Search(ctxK, pattern, book, h.kiwix.max)
|
hits, err := h.kiwix.client.Search(ctxK, pattern, book, h.kiwix.max)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
log.Printf("voice: kiwix: search %q: %v", pattern, err)
|
log.Printf("voice: kiwix: search %q: %v", pattern, err)
|
||||||
@@ -944,14 +981,18 @@ func (h *reactiveHandler) queryKiwix(ctx context.Context, t *queryTurn) (string,
|
|||||||
}
|
}
|
||||||
page = crawl.Page{Title: top.Title, Text: top.Snippet}
|
page = crawl.Page{Title: top.Title, Text: top.Snippet}
|
||||||
}
|
}
|
||||||
// Handed over the same way a note or a page is: context for the question he
|
return h.kiwixReply(ctx, t, top.Title, page.Text)
|
||||||
// asked, not something to recite.
|
}
|
||||||
snippet := top.Title + "\n" + crawl.TrimRunes(page.Text, h.kiwix.runes)
|
|
||||||
|
// kiwixReply hands one article over the same way a note or a page is handed
|
||||||
|
// over: context for the question he asked, not something to recite.
|
||||||
|
func (h *reactiveHandler) kiwixReply(ctx context.Context, t *queryTurn, title, text string) (string, bool) {
|
||||||
|
snippet := title + "\n" + crawl.TrimRunes(text, h.kiwix.runes)
|
||||||
reply := h.phraseSource(ctx, "kiwix", t.dec.Utterance, []string{snippet})
|
reply := h.phraseSource(ctx, "kiwix", t.dec.Utterance, []string{snippet})
|
||||||
if reply == "" {
|
if reply == "" {
|
||||||
// No phraser, or it failed. Read back the best hit rather than pretend
|
// No phraser, or it failed. Read back the best hit rather than pretend
|
||||||
// the search did not happen.
|
// the search did not happen.
|
||||||
return readBack(top.Title + " — " + page.Text), true
|
return readBack(title + " — " + text), true
|
||||||
}
|
}
|
||||||
return reply, true
|
return reply, true
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ func TestChatAnswersWithNoLlamaServer(t *testing.T) {
|
|||||||
dead := llm.New("http://127.0.0.1:1", 500*time.Millisecond)
|
dead := llm.New("http://127.0.0.1:1", 500*time.Millisecond)
|
||||||
emb := router.NewHashEmbedder(1024)
|
emb := router.NewHashEmbedder(1024)
|
||||||
h.recall.embedder = emb
|
h.recall.embedder = emb
|
||||||
h.router = buildRouter(emb, h.matcher, 0.55, pickLLMRouter(true, dead))
|
h.router = buildRouter(emb, h.matcher, 0.55, pickLLMRouter(true, dead), nil)
|
||||||
h.replier = newLLMReplier(dead, nil)
|
h.replier = newLLMReplier(dead, nil)
|
||||||
|
|
||||||
ctx := withDialogueID(context.Background(), dialogueIDFor(sourceText, "web"))
|
ctx := withDialogueID(context.Background(), dialogueIDFor(sourceText, "web"))
|
||||||
|
|||||||
@@ -317,7 +317,7 @@ func TestClarifyExpiryIsAnnouncedAndWordsStillRoute(t *testing.T) {
|
|||||||
h, _, now := newClarifyHandler(t)
|
h, _, now := newClarifyHandler(t)
|
||||||
emb := router.NewHashEmbedder(1024)
|
emb := router.NewHashEmbedder(1024)
|
||||||
h.recall.embedder = emb
|
h.recall.embedder = emb
|
||||||
h.router = buildRouter(emb, h.matcher, 0.55, nil)
|
h.router = buildRouter(emb, h.matcher, 0.55, nil, nil)
|
||||||
|
|
||||||
if _, asked := h.askClarify(ctx, clarifyDec(router.IntentReminder, router.Slots{Text: "напомни"}, "напомни")); !asked {
|
if _, asked := h.askClarify(ctx, clarifyDec(router.IntentReminder, router.Slots{Text: "напомни"}, "напомни")); !asked {
|
||||||
t.Fatal("expected a question")
|
t.Fatal("expected a question")
|
||||||
@@ -671,7 +671,7 @@ func TestUnresolvedActSaysItDoesNotKnowTheCommand(t *testing.T) {
|
|||||||
func newRoutingClarifyHandler(t *testing.T) (*reactiveHandler, *store.Store) {
|
func newRoutingClarifyHandler(t *testing.T) (*reactiveHandler, *store.Store) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
h, st, _ := newClarifyHandler(t)
|
h, st, _ := newClarifyHandler(t)
|
||||||
h.router = buildRouter(router.NewHashEmbedder(1024), h.matcher, 0.55, nil)
|
h.router = buildRouter(router.NewHashEmbedder(1024), h.matcher, 0.55, nil, nil)
|
||||||
h.recall = recallWiring{embedder: router.NewHashEmbedder(1024), memStore: memory.NewInMemoryStore()}
|
h.recall = recallWiring{embedder: router.NewHashEmbedder(1024), memStore: memory.NewInMemoryStore()}
|
||||||
return h, st
|
return h, st
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -25,7 +25,7 @@ func traceHandler(t *testing.T, ring *decision.Ring) *reactiveHandler {
|
|||||||
return &reactiveHandler{
|
return &reactiveHandler{
|
||||||
api: api,
|
api: api,
|
||||||
recall: recallWiring{embedder: emb, memStore: memory.NewInMemoryStore()},
|
recall: recallWiring{embedder: emb, memStore: memory.NewInMemoryStore()},
|
||||||
router: buildRouter(emb, tool.NewMatcher(api), 0.55, nil),
|
router: buildRouter(emb, tool.NewMatcher(api), 0.55, nil, nil),
|
||||||
replier: voice.NewStubReplier(),
|
replier: voice.NewStubReplier(),
|
||||||
now: func() time.Time { return now },
|
now: func() time.Time { return now },
|
||||||
dataStore: st,
|
dataStore: st,
|
||||||
|
|||||||
@@ -171,7 +171,7 @@ func newDialogueHandler(t *testing.T) (*reactiveHandler, *store.Store, *time.Tim
|
|||||||
// and never a coincidence (V-577, V-579). checkEnd refuses any reminder
|
// and never a coincidence (V-577, V-579). checkEnd refuses any reminder
|
||||||
// landing on it, and at 09:00 the row that answers "на 9" would trip that.
|
// landing on it, and at 09:00 the row that answers "на 9" would trip that.
|
||||||
*now = time.Date(2026, 7, 31, 9, 17, 0, 0, time.UTC)
|
*now = time.Date(2026, 7, 31, 9, 17, 0, 0, time.UTC)
|
||||||
h.router = buildRouter(router.NewHashEmbedder(1024), h.matcher, 0.55, nil)
|
h.router = buildRouter(router.NewHashEmbedder(1024), h.matcher, 0.55, nil, nil)
|
||||||
h.recall = recallWiring{embedder: router.NewHashEmbedder(1024), memStore: memory.NewInMemoryStore()}
|
h.recall = recallWiring{embedder: router.NewHashEmbedder(1024), memStore: memory.NewInMemoryStore()}
|
||||||
return h, st, now
|
return h, st, now
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ func TestApplyAction_FactCapture_QueuesEntityResolution(t *testing.T) {
|
|||||||
|
|
||||||
emb := router.NewHashEmbedder(1024)
|
emb := router.NewHashEmbedder(1024)
|
||||||
matcher := tool.NewMatcher(api)
|
matcher := tool.NewMatcher(api)
|
||||||
rtr := buildRouter(emb, matcher, 0.55, nil)
|
rtr := buildRouter(emb, matcher, 0.55, nil, nil)
|
||||||
|
|
||||||
h := &reactiveHandler{
|
h := &reactiveHandler{
|
||||||
api: api,
|
api: api,
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ func newFactGateHandler(t *testing.T, now time.Time) (*reactiveHandler, ipc.Core
|
|||||||
h := &reactiveHandler{
|
h := &reactiveHandler{
|
||||||
api: api,
|
api: api,
|
||||||
recall: recallWiring{embedder: emb, memStore: memory.NewInMemoryStore()},
|
recall: recallWiring{embedder: emb, memStore: memory.NewInMemoryStore()},
|
||||||
router: buildRouter(emb, tool.NewMatcher(api), 0.55, nil),
|
router: buildRouter(emb, tool.NewMatcher(api), 0.55, nil, nil),
|
||||||
replier: voice.NewStubReplier(),
|
replier: voice.NewStubReplier(),
|
||||||
now: func() time.Time { return now },
|
now: func() time.Time { return now },
|
||||||
dataStore: st,
|
dataStore: st,
|
||||||
|
|||||||
@@ -45,7 +45,7 @@ func newNoteHandler(t *testing.T) (*reactiveHandler, *store.Store) {
|
|||||||
h := &reactiveHandler{
|
h := &reactiveHandler{
|
||||||
api: api,
|
api: api,
|
||||||
recall: recallWiring{embedder: emb, memStore: memory.NewInMemoryStore()},
|
recall: recallWiring{embedder: emb, memStore: memory.NewInMemoryStore()},
|
||||||
router: buildRouter(emb, tool.NewMatcher(api), 0.55, nil),
|
router: buildRouter(emb, tool.NewMatcher(api), 0.55, nil, nil),
|
||||||
replier: voice.NewStubReplier(),
|
replier: voice.NewStubReplier(),
|
||||||
now: func() time.Time { return now },
|
now: func() time.Time { return now },
|
||||||
dataStore: st,
|
dataStore: st,
|
||||||
|
|||||||
@@ -10,7 +10,7 @@ import (
|
|||||||
// whose model is down names nothing, and naming nothing has to walk the chain
|
// whose model is down names nothing, and naming nothing has to walk the chain
|
||||||
// the way it walked before the field existed.
|
// the way it walked before the field existed.
|
||||||
func TestNoDestinationWalksTheWholeChain(t *testing.T) {
|
func TestNoDestinationWalksTheWholeChain(t *testing.T) {
|
||||||
walk, skipped := queryWalk(router.SourceUnknown)
|
walk, skipped := queryWalk(router.SourceUnknown, false)
|
||||||
if len(skipped) != 0 {
|
if len(skipped) != 0 {
|
||||||
t.Errorf("skipped %d sources with no destination named, want none", len(skipped))
|
t.Errorf("skipped %d sources with no destination named, want none", len(skipped))
|
||||||
}
|
}
|
||||||
@@ -32,15 +32,16 @@ func TestANamedDestinationSilencesTheOtherGuessers(t *testing.T) {
|
|||||||
dest router.Source
|
dest router.Source
|
||||||
utterance string
|
utterance string
|
||||||
silenced string
|
silenced string
|
||||||
|
anchored bool // a stage 0 grammar named the destination
|
||||||
}{
|
}{
|
||||||
{router.SourceWorld, "что такое TCP?", "weather"},
|
{router.SourceWorld, "что такое TCP?", "weather", true},
|
||||||
{router.SourceWorld, "сколько будет 17 на 23?", "weather"},
|
{router.SourceWorld, "сколько будет 17 на 23?", "weather", true},
|
||||||
{router.SourceWorld, "кто такой Линус Торвальдс?", "personal"},
|
{router.SourceWorld, "кто такой Линус Торвальдс?", "personal", true},
|
||||||
{router.SourceRecall, "какой у меня любимый язык?", "feeds"},
|
{router.SourceRecall, "какой у меня любимый язык?", "feeds", false},
|
||||||
{router.SourceCalendar, "что в календаре на завтра?", "weather"},
|
{router.SourceCalendar, "что в календаре на завтра?", "weather", true},
|
||||||
}
|
}
|
||||||
for _, c := range cases {
|
for _, c := range cases {
|
||||||
walk, skipped := queryWalk(c.dest)
|
walk, skipped := queryWalk(c.dest, c.anchored)
|
||||||
if inWalk(walk, c.silenced) {
|
if inWalk(walk, c.silenced) {
|
||||||
t.Errorf("%q named %q: %q is still asked", c.utterance, c.dest, c.silenced)
|
t.Errorf("%q named %q: %q is still asked", c.utterance, c.dest, c.silenced)
|
||||||
}
|
}
|
||||||
@@ -56,7 +57,7 @@ func TestANamedDestinationSilencesTheOtherGuessers(t *testing.T) {
|
|||||||
// data first, then the world", and a destination a model wrote must not be able
|
// data first, then the world", and a destination a model wrote must not be able
|
||||||
// to reverse it.
|
// to reverse it.
|
||||||
func TestNamingTheWorldStillReadsHisDataFirst(t *testing.T) {
|
func TestNamingTheWorldStillReadsHisDataFirst(t *testing.T) {
|
||||||
walk, _ := queryWalk(router.SourceWorld)
|
walk, _ := queryWalk(router.SourceWorld, true)
|
||||||
for _, look := range []string{"fact-by-key", "embed", "memory", "notes"} {
|
for _, look := range []string{"fact-by-key", "embed", "memory", "notes"} {
|
||||||
if !inWalk(walk, look) {
|
if !inWalk(walk, look) {
|
||||||
t.Errorf("%q was dropped; only the sources that guess may be dropped", look)
|
t.Errorf("%q was dropped; only the sources that guess may be dropped", look)
|
||||||
@@ -74,7 +75,7 @@ func TestNamingTheWorldStillReadsHisDataFirst(t *testing.T) {
|
|||||||
// makes "какой у меня любимый язык?" answer "не нашла у тебя такой записи"
|
// makes "какой у меня любимый язык?" answer "не нашла у тебя такой записи"
|
||||||
// rather than reaching SearXNG once nothing local had it.
|
// rather than reaching SearXNG once nothing local had it.
|
||||||
func TestNamingRecallKeepsTheBoundary(t *testing.T) {
|
func TestNamingRecallKeepsTheBoundary(t *testing.T) {
|
||||||
walk, _ := queryWalk(router.SourceRecall)
|
walk, _ := queryWalk(router.SourceRecall, true)
|
||||||
if !inWalk(walk, "personal") {
|
if !inWalk(walk, "personal") {
|
||||||
t.Fatal("the personal boundary was skipped on a turn named for his own data")
|
t.Fatal("the personal boundary was skipped on a turn named for his own data")
|
||||||
}
|
}
|
||||||
@@ -83,12 +84,33 @@ func TestNamingRecallKeepsTheBoundary(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The owner's call of 2026-08-09 (V-666): only a stage 0 grammar may take the
|
||||||
|
// personal boundary off a turn. The routing heads and the resident model both
|
||||||
|
// name a destination by inference, and an inferred SourceWorld would send a
|
||||||
|
// question about him upstream. Every other guesser still goes.
|
||||||
|
func TestOnlyAGrammarMayDropTheBoundary(t *testing.T) {
|
||||||
|
walk, skipped := queryWalk(router.SourceWorld, false)
|
||||||
|
if !inWalk(walk, "personal") {
|
||||||
|
t.Error("an inferred destination took the boundary off the turn")
|
||||||
|
}
|
||||||
|
if !inWalk(skipped, "weather") {
|
||||||
|
t.Error("weather is still asked; the rule covers the boundary alone")
|
||||||
|
}
|
||||||
|
if posOf(walk, "personal") > posOf(walk, "search") {
|
||||||
|
t.Error("the boundary no longer sits in front of the world")
|
||||||
|
}
|
||||||
|
if anchored, _ := queryWalk(router.SourceWorld, true); inWalk(anchored, "personal") {
|
||||||
|
t.Error(`a grammar named the world and the boundary stayed: ` +
|
||||||
|
`"кто такой Линус Торвальдс?" is answered "не нашла у тебя такой записи" again`)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Whatever the destination, the walk is a subsequence of the table. Every
|
// Whatever the destination, the walk is a subsequence of the table. Every
|
||||||
// comment on that table argues an order between two sources, and none of those
|
// comment on that table argues an order between two sources, and none of those
|
||||||
// reasons is about this field.
|
// reasons is about this field.
|
||||||
func TestTheWalkNeverReordersTheTable(t *testing.T) {
|
func TestTheWalkNeverReordersTheTable(t *testing.T) {
|
||||||
for _, dest := range append([]router.Source{router.SourceUnknown}, router.Sources...) {
|
for _, dest := range append([]router.Source{router.SourceUnknown}, router.Sources...) {
|
||||||
walk, skipped := queryWalk(dest)
|
walk, skipped := queryWalk(dest, true)
|
||||||
if len(walk)+len(skipped) != len(querySources) {
|
if len(walk)+len(skipped) != len(querySources) {
|
||||||
t.Errorf("%q: %d walked + %d skipped, want %d", dest, len(walk), len(skipped), len(querySources))
|
t.Errorf("%q: %d walked + %d skipped, want %d", dest, len(walk), len(skipped), len(querySources))
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -22,7 +22,7 @@ func TestReactiveNotesReminders(t *testing.T) {
|
|||||||
|
|
||||||
emb := router.NewHashEmbedder(1024)
|
emb := router.NewHashEmbedder(1024)
|
||||||
matcher := tool.NewMatcher(api)
|
matcher := tool.NewMatcher(api)
|
||||||
rtr := buildRouter(emb, matcher, 0.55, nil)
|
rtr := buildRouter(emb, matcher, 0.55, nil, nil)
|
||||||
|
|
||||||
h := &reactiveHandler{
|
h := &reactiveHandler{
|
||||||
api: api,
|
api: api,
|
||||||
@@ -104,7 +104,7 @@ func TestSpokenTaskCaptureFilesATask(t *testing.T) {
|
|||||||
h := &reactiveHandler{
|
h := &reactiveHandler{
|
||||||
api: api,
|
api: api,
|
||||||
recall: recallWiring{embedder: emb, memStore: memory.NewInMemoryStore()},
|
recall: recallWiring{embedder: emb, memStore: memory.NewInMemoryStore()},
|
||||||
router: buildRouter(emb, matcher, 0.55, nil),
|
router: buildRouter(emb, matcher, 0.55, nil, nil),
|
||||||
replier: voice.NewStubReplier(),
|
replier: voice.NewStubReplier(),
|
||||||
now: func() time.Time { return now },
|
now: func() time.Time { return now },
|
||||||
dataStore: st,
|
dataStore: st,
|
||||||
|
|||||||
@@ -474,7 +474,7 @@ func newSimWorld(t *testing.T, sc scenario) *simWorld {
|
|||||||
// used to be built on a nil API, which meant any scenario that produced an
|
// used to be built on a nil API, which meant any scenario that produced an
|
||||||
// act panicked the moment the matcher was consulted.
|
// act panicked the moment the matcher was consulted.
|
||||||
matcher := tool.NewMatcher(api)
|
matcher := tool.NewMatcher(api)
|
||||||
rtr := buildRouter(emb, matcher, config.DefaultRouterThreshold, router.NewLLMRouter(scripted))
|
rtr := buildRouter(emb, matcher, config.DefaultRouterThreshold, router.NewLLMRouter(scripted), nil)
|
||||||
|
|
||||||
w.handler = &reactiveHandler{
|
w.handler = &reactiveHandler{
|
||||||
stt: simTranscriber{},
|
stt: simTranscriber{},
|
||||||
|
|||||||
@@ -0,0 +1,43 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/kami/maven/internal/config"
|
||||||
|
"github.com/kami/maven/internal/stt"
|
||||||
|
)
|
||||||
|
|
||||||
|
// A box with no workstation.stt block transcribes exactly as it did before the
|
||||||
|
// seam existed: the floor is handed back untouched, and nothing probes.
|
||||||
|
func TestSttSeamWithNoBlockIsTheFloor(t *testing.T) {
|
||||||
|
floor := stt.NewStub()
|
||||||
|
got, pair := sttSeam(&config.Config{}, floor)
|
||||||
|
if pair != nil {
|
||||||
|
t.Fatal("no block must build no pair")
|
||||||
|
}
|
||||||
|
if got != stt.Transcriber(floor) {
|
||||||
|
t.Fatal("no block must hand back the floor itself")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSttSeamPrefersTheWorkstation(t *testing.T) {
|
||||||
|
cfg := &config.Config{Workstation: &config.WorkstationConfig{
|
||||||
|
URL: "http://127.0.0.1:1",
|
||||||
|
Stt: &config.WorkstationSttConfig{
|
||||||
|
URL: "http://127.0.0.1:2/transcribe",
|
||||||
|
Health: "http://127.0.0.1:2/health",
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
got, pair := sttSeam(cfg, stt.NewStub())
|
||||||
|
if pair == nil {
|
||||||
|
t.Fatal("a configured block must build a pair")
|
||||||
|
}
|
||||||
|
defer pair.Stop()
|
||||||
|
if got != stt.Transcriber(pair) {
|
||||||
|
t.Fatal("the pair is what callers must transcribe through")
|
||||||
|
}
|
||||||
|
// Nothing answers on port 2, so the seam is the floor until it does.
|
||||||
|
if pair.Available() {
|
||||||
|
t.Fatal("an unreachable workstation must not be available")
|
||||||
|
}
|
||||||
|
}
|
||||||
+74
-2
@@ -36,6 +36,8 @@ type voiceWiring struct {
|
|||||||
sessions *voice.Sessions
|
sessions *voice.Sessions
|
||||||
voiceSink delivery.Sink
|
voiceSink delivery.Sink
|
||||||
embedder router.Embedder
|
embedder router.Embedder
|
||||||
|
// heads — the routing heads, nil unless embedder.heads_path is set.
|
||||||
|
heads *router.RouterHeads
|
||||||
handler *reactiveHandler // the reactive handler for IPC Chat
|
handler *reactiveHandler // the reactive handler for IPC Chat
|
||||||
// worker clients (set when configured as Remote): closed on shutdown so
|
// worker clients (set when configured as Remote): closed on shutdown so
|
||||||
// mavsttd / mavttsd don't keep a stale conn into a restarting daemon.
|
// mavsttd / mavttsd don't keep a stale conn into a restarting daemon.
|
||||||
@@ -53,6 +55,10 @@ type voiceWiring struct {
|
|||||||
// unless a `workstation` block names an address. Held here only so the
|
// unless a `workstation` block names an address. Held here only so the
|
||||||
// prober is stopped on shutdown; callers were handed it at build time.
|
// prober is stopped on shutdown; callers were handed it at build time.
|
||||||
pair *llm.Pair
|
pair *llm.Pair
|
||||||
|
// sttPair — CrisperWhisper 2.0 on the workstation with mavsttd as the
|
||||||
|
// floor, nil unless the `workstation.stt` block names an address. Held for
|
||||||
|
// the same reason as pair: to stop its prober on shutdown.
|
||||||
|
sttPair *stt.Pair
|
||||||
mcp *mcpWiring
|
mcp *mcpWiring
|
||||||
// home — the Home Assistant client, nil unless the `smarthome` block is
|
// home — the Home Assistant client, nil unless the `smarthome` block is
|
||||||
// enabled (Vikunja #256). Its devices land in the same allowlist as every
|
// enabled (Vikunja #256). Its devices land in the same allowlist as every
|
||||||
@@ -72,6 +78,9 @@ func (w *voiceWiring) close() {
|
|||||||
if w.embedder != nil {
|
if w.embedder != nil {
|
||||||
_ = w.embedder.Close()
|
_ = w.embedder.Close()
|
||||||
}
|
}
|
||||||
|
if w.heads != nil {
|
||||||
|
_ = w.heads.Close()
|
||||||
|
}
|
||||||
if w.server != nil {
|
if w.server != nil {
|
||||||
_ = w.server.Close()
|
_ = w.server.Close()
|
||||||
}
|
}
|
||||||
@@ -84,6 +93,9 @@ func (w *voiceWiring) close() {
|
|||||||
if w.pair != nil {
|
if w.pair != nil {
|
||||||
w.pair.Stop()
|
w.pair.Stop()
|
||||||
}
|
}
|
||||||
|
if w.sttPair != nil {
|
||||||
|
w.sttPair.Stop()
|
||||||
|
}
|
||||||
w.mcp.close()
|
w.mcp.close()
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -112,6 +124,7 @@ func wireVoice(cfg *config.Config, coreAPI ipc.CoreAPI, phr phraser.Phraser, mem
|
|||||||
} else {
|
} else {
|
||||||
transcriber = stt.NewStub()
|
transcriber = stt.NewStub()
|
||||||
}
|
}
|
||||||
|
transcriber, w.sttPair = sttSeam(cfg, transcriber)
|
||||||
w.transcriber = transcriber
|
w.transcriber = transcriber
|
||||||
|
|
||||||
// ----- tts (Stub in-process OR Remote) -----
|
// ----- tts (Stub in-process OR Remote) -----
|
||||||
@@ -147,6 +160,24 @@ func wireVoice(cfg *config.Config, coreAPI ipc.CoreAPI, phr phraser.Phraser, mem
|
|||||||
emb = router.NewHashEmbedder(1024)
|
emb = router.NewHashEmbedder(1024)
|
||||||
}
|
}
|
||||||
w.embedder = emb
|
w.embedder = emb
|
||||||
|
|
||||||
|
// ----- router: routing heads (only when configured, and never fatal) -----
|
||||||
|
// A missing or broken weights file logs and leaves w.heads nil, which is
|
||||||
|
// byte-for-byte the cascade that shipped before V-664. Refusing to start
|
||||||
|
// over a routing accelerator would trade a working box for a better one.
|
||||||
|
if cfg.Voice.Embedder != nil && cfg.Voice.Embedder.HeadsPath != "" {
|
||||||
|
h, err := router.NewRouterHeads(
|
||||||
|
cfg.Voice.Embedder.HeadsPath,
|
||||||
|
cfg.Voice.Embedder.TokenizerPath,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("voice: routing heads unavailable, cascade unchanged: %v", err)
|
||||||
|
} else {
|
||||||
|
log.Printf("voice: routing heads loaded from %s", cfg.Voice.Embedder.HeadsPath)
|
||||||
|
w.heads = h
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
repairFactVectors(dataStore, emb)
|
repairFactVectors(dataStore, emb)
|
||||||
checkStoredEmbedder(dataStore, emb)
|
checkStoredEmbedder(dataStore, emb)
|
||||||
// Retention is enforced on write, which is not enough on its own: a box that
|
// Retention is enforced on write, which is not enough on its own: a box that
|
||||||
@@ -223,7 +254,8 @@ func wireVoice(cfg *config.Config, coreAPI ipc.CoreAPI, phr phraser.Phraser, mem
|
|||||||
// against the classifier's 50.0%, at about 1s a turn instead of 30ms (see
|
// against the classifier's 50.0%, at about 1s a turn instead of 30ms (see
|
||||||
// config.VoiceConfig.LLMRouter). The classifier always stays wired as the
|
// config.VoiceConfig.LLMRouter). The classifier always stays wired as the
|
||||||
// fallback, so a model error never breaks a turn.
|
// fallback, so a model error never breaks a turn.
|
||||||
rtr := buildRouter(emb, matcher, threshold, pickLLMRouter(cfg.Voice.UseLLMRouter(), hot))
|
rtr := buildRouter(emb, matcher, threshold,
|
||||||
|
pickLLMRouter(cfg.Voice.UseLLMRouter(), hot), w.heads)
|
||||||
|
|
||||||
// ----- sessions registry (shared with voicesink) -----
|
// ----- sessions registry (shared with voicesink) -----
|
||||||
sessions := voice.NewSessions()
|
sessions := voice.NewSessions()
|
||||||
@@ -366,6 +398,44 @@ func modelSeam(cfg *config.Config, resident *llm.Client) (router.Completer, *llm
|
|||||||
return pair, pair
|
return pair, pair
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// sttSeam builds the transcription seam the voice path and the meeting
|
||||||
|
// recorder share. It is modelSeam for audio and follows the same rule.
|
||||||
|
//
|
||||||
|
// With no `workstation.stt` block it hands back the floor untouched, which is
|
||||||
|
// today's deploy exactly. With one, it is an stt.Pair preferring CrisperWhisper
|
||||||
|
// 2.0 on workpc, which scores 10.4% WER in Russian against the floor's 27.5%
|
||||||
|
// (docs/evals/2026-08-09-crisperwhisper2-russian-wer.md).
|
||||||
|
//
|
||||||
|
// Only the silent half of the degradation rule applies here. A worse transcript
|
||||||
|
// is still a turn, so there is nothing to name a gap about and the fallback is
|
||||||
|
// never spoken. That is why stt.Pair has no TranscribeRemote.
|
||||||
|
func sttSeam(cfg *config.Config, floor stt.Transcriber) (stt.Transcriber, *stt.Pair) {
|
||||||
|
if cfg.Workstation == nil || cfg.Workstation.Stt == nil {
|
||||||
|
return floor, nil
|
||||||
|
}
|
||||||
|
s := cfg.Workstation.Stt
|
||||||
|
lang := ""
|
||||||
|
if cfg.Voice != nil {
|
||||||
|
lang = cfg.Voice.Lang
|
||||||
|
if cfg.Voice.Stt != nil && cfg.Voice.Stt.Lang != "" {
|
||||||
|
lang = cfg.Voice.Stt.Lang
|
||||||
|
}
|
||||||
|
}
|
||||||
|
pair := stt.NewPair(
|
||||||
|
stt.NewHTTPTranscriber(s.URL, s.Token, lang, time.Duration(s.Timeout)),
|
||||||
|
floor,
|
||||||
|
s.Health,
|
||||||
|
time.Duration(s.Probe),
|
||||||
|
)
|
||||||
|
pair.Start(context.Background())
|
||||||
|
if s.Token == "" {
|
||||||
|
log.Print("voice: the workstation transcriber has no token, so anything on the LAN can post audio to it")
|
||||||
|
}
|
||||||
|
log.Printf("voice: workstation transcriber at %s, probed every %s, mavsttd as the floor",
|
||||||
|
s.URL, time.Duration(s.Probe))
|
||||||
|
return pair, pair
|
||||||
|
}
|
||||||
|
|
||||||
func pickLLMRouter(enabled bool, c router.Completer) *router.LLMRouter {
|
func pickLLMRouter(enabled bool, c router.Completer) *router.LLMRouter {
|
||||||
if !enabled {
|
if !enabled {
|
||||||
return nil
|
return nil
|
||||||
@@ -390,7 +460,8 @@ func pickLLMRouter(enabled bool, c router.Completer) *router.LLMRouter {
|
|||||||
// intent from seedDir (models/seeds/<intent>.txt) — see seedClassifier
|
// intent from seedDir (models/seeds/<intent>.txt) — see seedClassifier
|
||||||
// below for the current intent list and file names.
|
// below for the current intent list and file names.
|
||||||
// - Threshold is from voice.router_threshold config (default 0.55).
|
// - Threshold is from voice.router_threshold config (default 0.55).
|
||||||
func buildRouter(emb router.Embedder, acts router.ActMatcher, threshold float64, llmR *router.LLMRouter) *router.Router {
|
func buildRouter(emb router.Embedder, acts router.ActMatcher, threshold float64,
|
||||||
|
llmR *router.LLMRouter, heads *router.RouterHeads) *router.Router {
|
||||||
cls := router.NewClassifier(emb)
|
cls := router.NewClassifier(emb)
|
||||||
seedClassifier(cls)
|
seedClassifier(cls)
|
||||||
grammars := router.DefaultGrammars(acts)
|
grammars := router.DefaultGrammars(acts)
|
||||||
@@ -442,6 +513,7 @@ func buildRouter(emb router.Embedder, acts router.ActMatcher, threshold float64,
|
|||||||
},
|
},
|
||||||
Threshold: threshold,
|
Threshold: threshold,
|
||||||
LLM: llmR,
|
LLM: llmR,
|
||||||
|
Heads: heads,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+13
-4
@@ -34,22 +34,31 @@ type probe struct {
|
|||||||
drmDev string
|
drmDev string
|
||||||
}
|
}
|
||||||
|
|
||||||
// foreign lists every ROCm process that is not ours. selfPID is the supervisor's
|
// foreign lists every ROCm process that is not ours. self holds the pids of the
|
||||||
// llama-server child, or 0 when it is not running.
|
// supervisor's own children, and a child that is not running contributes 0.
|
||||||
|
//
|
||||||
|
// There is more than one child since 09-08-2026. CW2 registers on the KFD like
|
||||||
|
// any ROCm job, so a supervisor that excluded only llama-server would read its
|
||||||
|
// own transcriber as a contender, yield the card to it, and never keep a model
|
||||||
|
// loaded again.
|
||||||
//
|
//
|
||||||
// An unreadable kfd tree returns no processes and no error. That is deliberate
|
// An unreadable kfd tree returns no processes and no error. That is deliberate
|
||||||
// and it is the safe direction only because startVRAM also has to agree before
|
// and it is the safe direction only because startVRAM also has to agree before
|
||||||
// anything launches: a supervisor that cannot see the KFD never sees free VRAM
|
// anything launches: a supervisor that cannot see the KFD never sees free VRAM
|
||||||
// either, because the CPT run holding the card shows up in the drm totals.
|
// either, because the CPT run holding the card shows up in the drm totals.
|
||||||
func (p probe) foreign(selfPID int) []gpuProc {
|
func (p probe) foreign(self ...int) []gpuProc {
|
||||||
entries, err := os.ReadDir(p.kfdRoot)
|
entries, err := os.ReadDir(p.kfdRoot)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
mine := make(map[int]bool, len(self))
|
||||||
|
for _, pid := range self {
|
||||||
|
mine[pid] = true
|
||||||
|
}
|
||||||
var out []gpuProc
|
var out []gpuProc
|
||||||
for _, e := range entries {
|
for _, e := range entries {
|
||||||
pid, err := strconv.Atoi(e.Name())
|
pid, err := strconv.Atoi(e.Name())
|
||||||
if err != nil || pid == selfPID {
|
if err != nil || mine[pid] {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
out = append(out, gpuProc{
|
out = append(out, gpuProc{
|
||||||
|
|||||||
+46
-1
@@ -1,6 +1,7 @@
|
|||||||
package main
|
package main
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"net/url"
|
"net/url"
|
||||||
@@ -8,6 +9,7 @@ import (
|
|||||||
"path/filepath"
|
"path/filepath"
|
||||||
"strconv"
|
"strconv"
|
||||||
"testing"
|
"testing"
|
||||||
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
// fakeKFD builds the sysfs shape the workstation actually has: one directory
|
// fakeKFD builds the sysfs shape the workstation actually has: one directory
|
||||||
@@ -47,6 +49,24 @@ func TestForeignExcludesOurChild(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The transcriber is a ROCm process on the same card, so it registers on the
|
||||||
|
// KFD exactly like a contender does. Reading it as one is what happened on
|
||||||
|
// 2026-08-09 while CW2 ran under its own systemd unit: mavgpud yielded, waited
|
||||||
|
// five polls, loaded the model, yielded again, and never held it for a whole
|
||||||
|
// minute. Excluding every child is the fix and this is the test of it.
|
||||||
|
func TestForeignExcludesEveryChild(t *testing.T) {
|
||||||
|
p := probe{kfdRoot: fakeKFD(t, map[int]int64{478104: 12791693312, 999: 4096, 1001: 1717986918})}
|
||||||
|
|
||||||
|
ours := p.foreign(999, 1001)
|
||||||
|
if len(ours) != 1 || ours[0].PID != 478104 {
|
||||||
|
t.Fatalf("only the CPT run is a contender, got %+v", ours)
|
||||||
|
}
|
||||||
|
// A child that is not running reports pid 0, which must exclude nothing.
|
||||||
|
if got := p.foreign(999, 0); len(got) != 2 {
|
||||||
|
t.Errorf("a stopped child excludes nobody: got %d contenders, want 2", len(got))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// An empty KFD tree is the state that permits a start, so it must read as empty
|
// An empty KFD tree is the state that permits a start, so it must read as empty
|
||||||
// rather than as an error the caller has to interpret.
|
// rather than as an error the caller has to interpret.
|
||||||
func TestForeignEmptyAndMissing(t *testing.T) {
|
func TestForeignEmptyAndMissing(t *testing.T) {
|
||||||
@@ -81,7 +101,7 @@ func TestFreeVRAM(t *testing.T) {
|
|||||||
// rather than hanging or proxying into a closed port. Maven reads this endpoint
|
// rather than hanging or proxying into a closed port. Maven reads this endpoint
|
||||||
// on a timer forever, including while the workstation is busy.
|
// on a timer forever, including while the workstation is busy.
|
||||||
func TestHealthAndProxyRefuseWhenNotReady(t *testing.T) {
|
func TestHealthAndProxyRefuseWhenNotReady(t *testing.T) {
|
||||||
s := &supervisor{run: newRunner("/bin/true", nil, "")}
|
s := &supervisor{run: newRunner("fake", "/bin/true", nil, "")}
|
||||||
h := s.handler(mustURL(t, "http://127.0.0.1:1"))
|
h := s.handler(mustURL(t, "http://127.0.0.1:1"))
|
||||||
|
|
||||||
for _, path := range []string{"/health", "/v1/chat/completions"} {
|
for _, path := range []string{"/health", "/v1/chat/completions"} {
|
||||||
@@ -101,3 +121,28 @@ func mustURL(t *testing.T, s string) *url.URL {
|
|||||||
}
|
}
|
||||||
return u
|
return u
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Yielding is all or nothing. A CPT run wants the whole card, so handing back
|
||||||
|
// the language model while the transcriber keeps 1.6GB mapped would leave the
|
||||||
|
// other job failing its allocation, which is the outcome yielding exists to
|
||||||
|
// prevent.
|
||||||
|
func TestYieldStopsEveryChild(t *testing.T) {
|
||||||
|
idle := "while : ; do sleep 1 ; done"
|
||||||
|
s := &supervisor{
|
||||||
|
cfg: config{EvictAfter: 1, StopGrace: duration(2 * time.Second)},
|
||||||
|
probe: probe{kfdRoot: fakeKFD(t, map[int]int64{478104: 12791693312})},
|
||||||
|
run: newRunner("llama-server", fakeServer(t, idle), nil, ""),
|
||||||
|
stt: newRunner("cw2", fakeServer(t, idle), nil, ""),
|
||||||
|
}
|
||||||
|
for _, r := range s.children() {
|
||||||
|
if err := r.start(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
s.tick(context.Background())
|
||||||
|
for _, r := range s.children() {
|
||||||
|
if r.running() {
|
||||||
|
t.Errorf("%s outlived the yield", r.name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
+89
-16
@@ -10,6 +10,11 @@
|
|||||||
// the card. Not on demand, because a 7-14B takes tens of seconds to load and a
|
// the card. Not on demand, because a 7-14B takes tens of seconds to load and a
|
||||||
// world question would be answered by a gap every time the card had been quiet.
|
// world question would be answered by a gap every time the card had been quiet.
|
||||||
// Not always on, because that holds 16GB against the owner's own jobs.
|
// Not always on, because that holds 16GB against the owner's own jobs.
|
||||||
|
//
|
||||||
|
// It supervises a second child since 09-08-2026, the CW2 transcriber, and for
|
||||||
|
// one reason only: it is a ROCm process on the same card. Any GPU service the
|
||||||
|
// owner leaves running beside this daemon reads as a contender and evicts the
|
||||||
|
// model, so the card needs one owner rather than two neighbours.
|
||||||
package main
|
package main
|
||||||
|
|
||||||
import (
|
import (
|
||||||
@@ -36,6 +41,10 @@ type config struct {
|
|||||||
// owner's business and not this daemon's schema.
|
// owner's business and not this daemon's schema.
|
||||||
LlamaArgs []string `json:"llama_args"`
|
LlamaArgs []string `json:"llama_args"`
|
||||||
|
|
||||||
|
// Stt is optional. Without it mavgpud supervises llama-server alone, which
|
||||||
|
// is everything it did before 09-08-2026.
|
||||||
|
Stt *sttConfig `json:"stt,omitempty"`
|
||||||
|
|
||||||
KFDRoot string `json:"kfd_root"`
|
KFDRoot string `json:"kfd_root"`
|
||||||
DRMDevice string `json:"drm_device"`
|
DRMDevice string `json:"drm_device"`
|
||||||
|
|
||||||
@@ -51,6 +60,22 @@ type config struct {
|
|||||||
StartAfter int `json:"start_after_polls"`
|
StartAfter int `json:"start_after_polls"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// sttConfig is the CW2 transcriber, which mavgpud runs for one reason: it is a
|
||||||
|
// ROCm process on this card. Left to its own systemd unit it registers on the
|
||||||
|
// KFD, the supervisor reads it as a contender, and llama-server is evicted
|
||||||
|
// within two polls and restarted five polls later, forever. That thrash was
|
||||||
|
// observed on 2026-08-09 and it is what folded the service in here.
|
||||||
|
//
|
||||||
|
// Maven talks to it directly, not through this daemon. There is no proxy and no
|
||||||
|
// idle timer: at 1.6GB it denies the card to nobody, and unloading it would only
|
||||||
|
// send the next voice turn to the homesrv floor for no gain.
|
||||||
|
type sttConfig struct {
|
||||||
|
// Addr is where the service binds, and it is read only to probe /health.
|
||||||
|
Addr string `json:"addr"`
|
||||||
|
Bin string `json:"bin"`
|
||||||
|
Args []string `json:"args"`
|
||||||
|
}
|
||||||
|
|
||||||
func defaults() config {
|
func defaults() config {
|
||||||
return config{
|
return config{
|
||||||
Listen: ":8080",
|
Listen: ":8080",
|
||||||
@@ -99,12 +124,18 @@ func main() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
base := "http://" + cfg.LlamaAddr
|
base := "http://" + cfg.LlamaAddr
|
||||||
run := newRunner(cfg.LlamaBin, cfg.LlamaArgs, base+"/health")
|
run := newRunner("llama-server", cfg.LlamaBin, cfg.LlamaArgs, base+"/health")
|
||||||
sup := &supervisor{
|
sup := &supervisor{
|
||||||
cfg: cfg,
|
cfg: cfg,
|
||||||
probe: probe{kfdRoot: cfg.KFDRoot, drmDev: cfg.DRMDevice},
|
probe: probe{kfdRoot: cfg.KFDRoot, drmDev: cfg.DRMDevice},
|
||||||
run: run,
|
run: run,
|
||||||
}
|
}
|
||||||
|
if s := cfg.Stt; s != nil {
|
||||||
|
if s.Bin == "" || s.Addr == "" {
|
||||||
|
log.Fatal("mavgpud: stt needs both bin and addr")
|
||||||
|
}
|
||||||
|
sup.stt = newRunner("cw2", s.Bin, s.Args, "http://"+s.Addr+"/health")
|
||||||
|
}
|
||||||
sup.touch()
|
sup.touch()
|
||||||
|
|
||||||
ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
|
ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
|
||||||
@@ -129,13 +160,17 @@ func main() {
|
|||||||
shut, done := context.WithTimeout(context.Background(), 5*time.Second)
|
shut, done := context.WithTimeout(context.Background(), 5*time.Second)
|
||||||
defer done()
|
defer done()
|
||||||
_ = srv.Shutdown(shut)
|
_ = srv.Shutdown(shut)
|
||||||
run.stop(time.Duration(cfg.StopGrace))
|
for _, r := range sup.children() {
|
||||||
|
r.stop(time.Duration(cfg.StopGrace))
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
type supervisor struct {
|
type supervisor struct {
|
||||||
cfg config
|
cfg config
|
||||||
probe probe
|
probe probe
|
||||||
run *runner
|
run *runner
|
||||||
|
// stt is the CW2 transcriber, or nil when the config names none.
|
||||||
|
stt *runner
|
||||||
|
|
||||||
lastReq atomic.Int64 // unix nanos of the last request Maven sent
|
lastReq atomic.Int64 // unix nanos of the last request Maven sent
|
||||||
|
|
||||||
@@ -198,7 +233,11 @@ func (s *supervisor) loop(ctx context.Context) {
|
|||||||
// allocates, so we see a contender during its startup rather than after it has
|
// allocates, so we see a contender during its startup rather than after it has
|
||||||
// already failed to get the memory it wanted.
|
// already failed to get the memory it wanted.
|
||||||
func (s *supervisor) tick(ctx context.Context) {
|
func (s *supervisor) tick(ctx context.Context) {
|
||||||
others := s.probe.foreign(s.run.pid())
|
var pids []int
|
||||||
|
for _, r := range s.children() {
|
||||||
|
pids = append(pids, r.pid())
|
||||||
|
}
|
||||||
|
others := s.probe.foreign(pids...)
|
||||||
if len(others) > 0 {
|
if len(others) > 0 {
|
||||||
s.foreignStreak++
|
s.foreignStreak++
|
||||||
s.clearStreak = 0
|
s.clearStreak = 0
|
||||||
@@ -207,31 +246,65 @@ func (s *supervisor) tick(ctx context.Context) {
|
|||||||
s.clearStreak++
|
s.clearStreak++
|
||||||
}
|
}
|
||||||
|
|
||||||
if s.run.running() {
|
// Yielding is all or nothing. A CPT run wants the whole card, and handing
|
||||||
s.run.refreshReady(ctx)
|
// back 8GB while holding 1.6GB is the shape of a failed allocation.
|
||||||
switch {
|
if s.foreignStreak >= s.cfg.EvictAfter && s.anyRunning() {
|
||||||
case s.foreignStreak >= s.cfg.EvictAfter:
|
|
||||||
log.Printf("mavgpud: yielding the card to %s", describe(others))
|
log.Printf("mavgpud: yielding the card to %s", describe(others))
|
||||||
s.run.stop(time.Duration(s.cfg.StopGrace))
|
for _, r := range s.children() {
|
||||||
case s.idle() > time.Duration(s.cfg.IdleTimeout):
|
r.stop(time.Duration(s.cfg.StopGrace))
|
||||||
log.Printf("mavgpud: idle for %s, unloading", s.idle().Round(time.Second))
|
|
||||||
s.run.stop(time.Duration(s.cfg.StopGrace))
|
|
||||||
}
|
}
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
if s.clearStreak < s.cfg.StartAfter {
|
clear := s.clearStreak >= s.cfg.StartAfter
|
||||||
return
|
|
||||||
}
|
if s.run.running() {
|
||||||
if free := s.probe.freeVRAM(); free < s.cfg.MinFreeVRAM {
|
s.run.refreshReady(ctx)
|
||||||
return
|
if s.idle() > time.Duration(s.cfg.IdleTimeout) {
|
||||||
|
log.Printf("mavgpud: idle for %s, unloading", s.idle().Round(time.Second))
|
||||||
|
s.run.stop(time.Duration(s.cfg.StopGrace))
|
||||||
}
|
}
|
||||||
|
} else if clear && s.probe.freeVRAM() >= s.cfg.MinFreeVRAM {
|
||||||
s.touch() // the idle clock starts at load, not at the last request before it
|
s.touch() // the idle clock starts at load, not at the last request before it
|
||||||
if err := s.run.start(); err != nil {
|
if err := s.run.start(); err != nil {
|
||||||
log.Printf("mavgpud: start llama-server: %v", err)
|
log.Printf("mavgpud: start llama-server: %v", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if s.stt == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if s.stt.running() {
|
||||||
|
s.stt.refreshReady(ctx)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
// No VRAM precondition here, unlike llama-server. That check exists because
|
||||||
|
// a 12B refuses to load when the card is short, and 1.6GB fits wherever the
|
||||||
|
// KFD is clear. Reading free VRAM would also block the transcriber for good
|
||||||
|
// once the language model was resident, since it holds more than the floor.
|
||||||
|
if clear {
|
||||||
|
if err := s.stt.start(); err != nil {
|
||||||
|
log.Printf("mavgpud: start cw2: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *supervisor) children() []*runner {
|
||||||
|
if s.stt == nil {
|
||||||
|
return []*runner{s.run}
|
||||||
|
}
|
||||||
|
return []*runner{s.run, s.stt}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *supervisor) anyRunning() bool {
|
||||||
|
for _, r := range s.children() {
|
||||||
|
if r.running() {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
// describe names the contenders in the log. This log is the instrument for the
|
// describe names the contenders in the log. This log is the instrument for the
|
||||||
// open question in #488: whether polling the KFD misses a job that wants the
|
// open question in #488: whether polling the KFD misses a job that wants the
|
||||||
// card without registering there.
|
// card without registering there.
|
||||||
|
|||||||
+17
-13
@@ -10,14 +10,18 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
// runner owns one llama-server process. Owning it is the point of the daemon:
|
// runner owns one GPU process. Owning it is the point of the daemon: the
|
||||||
// the workstation cannot keep a 7-14B resident, because that holds 16GB against
|
// workstation cannot keep a 7-14B resident, because that holds 16GB against
|
||||||
// the owner's CPT runs, Correx and the manga-recap pipeline. So the thing that
|
// the owner's CPT runs, Correx and the manga-recap pipeline. So the thing that
|
||||||
// stays up is this, which costs no VRAM, and the model comes and goes under it.
|
// stays up is this, which costs no VRAM, and the model comes and goes under it.
|
||||||
|
//
|
||||||
|
// There are two of them since 09-08-2026: llama-server and the CW2 transcriber.
|
||||||
|
// name is what the log calls this one.
|
||||||
type runner struct {
|
type runner struct {
|
||||||
|
name string
|
||||||
bin string
|
bin string
|
||||||
args []string
|
args []string
|
||||||
// ready is llama-server's own /health, which answers "is a model loaded".
|
// ready is the child's own /health, which answers "is a model loaded".
|
||||||
// Loading a 7-14B takes tens of seconds, so started is not ready.
|
// Loading a 7-14B takes tens of seconds, so started is not ready.
|
||||||
readyURL string
|
readyURL string
|
||||||
|
|
||||||
@@ -32,9 +36,9 @@ type runner struct {
|
|||||||
http *http.Client
|
http *http.Client
|
||||||
}
|
}
|
||||||
|
|
||||||
func newRunner(bin string, args []string, readyURL string) *runner {
|
func newRunner(name, bin string, args []string, readyURL string) *runner {
|
||||||
return &runner{
|
return &runner{
|
||||||
bin: bin, args: args, readyURL: readyURL,
|
name: name, bin: bin, args: args, readyURL: readyURL,
|
||||||
http: &http.Client{Timeout: 2 * time.Second},
|
http: &http.Client{Timeout: 2 * time.Second},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -60,7 +64,7 @@ func (r *runner) isReady() bool {
|
|||||||
return r.ready
|
return r.ready
|
||||||
}
|
}
|
||||||
|
|
||||||
// start launches llama-server. It returns as soon as the process exists, not
|
// start launches the child. It returns as soon as the process exists, not
|
||||||
// when the model is loaded.
|
// when the model is loaded.
|
||||||
func (r *runner) start() error {
|
func (r *runner) start() error {
|
||||||
r.mu.Lock()
|
r.mu.Lock()
|
||||||
@@ -76,7 +80,7 @@ func (r *runner) start() error {
|
|||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
r.cmd, r.ready, r.yielding = cmd, false, false
|
r.cmd, r.ready, r.yielding = cmd, false, false
|
||||||
log.Printf("mavgpud: started llama-server pid=%d", cmd.Process.Pid)
|
log.Printf("mavgpud: started %s pid=%d", r.name, cmd.Process.Pid)
|
||||||
go func() {
|
go func() {
|
||||||
err := cmd.Wait()
|
err := cmd.Wait()
|
||||||
r.mu.Lock()
|
r.mu.Lock()
|
||||||
@@ -84,15 +88,15 @@ func (r *runner) start() error {
|
|||||||
r.cmd, r.ready, r.yielding = nil, false, false
|
r.cmd, r.ready, r.yielding = nil, false, false
|
||||||
r.mu.Unlock()
|
r.mu.Unlock()
|
||||||
if yielded {
|
if yielded {
|
||||||
log.Printf("mavgpud: llama-server stopped, card yielded (%v)", err)
|
log.Printf("mavgpud: %s stopped, card yielded (%v)", r.name, err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
log.Printf("mavgpud: llama-server exited: %v", err)
|
log.Printf("mavgpud: %s exited: %v", r.name, err)
|
||||||
}()
|
}()
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// stop ends llama-server and waits for the VRAM to come back. SIGTERM first so
|
// stop ends the child and waits for the VRAM to come back. SIGTERM first so
|
||||||
// it unmaps cleanly, SIGKILL after the grace window. Returning before the
|
// it unmaps cleanly, SIGKILL after the grace window. Returning before the
|
||||||
// process is gone would let the supervisor report a free card while 14GB is
|
// process is gone would let the supervisor report a free card while 14GB is
|
||||||
// still mapped, which is the one lie that would make yielding useless.
|
// still mapped, which is the one lie that would make yielding useless.
|
||||||
@@ -117,11 +121,11 @@ func (r *runner) stop(grace time.Duration) {
|
|||||||
}
|
}
|
||||||
time.Sleep(100 * time.Millisecond)
|
time.Sleep(100 * time.Millisecond)
|
||||||
}
|
}
|
||||||
log.Printf("mavgpud: llama-server did not exit in %s, killing", grace)
|
log.Printf("mavgpud: %s did not exit in %s, killing", r.name, grace)
|
||||||
_ = syscall.Kill(pgid, syscall.SIGKILL)
|
_ = syscall.Kill(pgid, syscall.SIGKILL)
|
||||||
}
|
}
|
||||||
|
|
||||||
// refreshReady asks llama-server whether the model is loaded. Called once per
|
// refreshReady asks the child whether the model is loaded. Called once per
|
||||||
// supervisor tick, never per request.
|
// supervisor tick, never per request.
|
||||||
func (r *runner) refreshReady(ctx context.Context) {
|
func (r *runner) refreshReady(ctx context.Context) {
|
||||||
if !r.running() {
|
if !r.running() {
|
||||||
@@ -141,6 +145,6 @@ func (r *runner) refreshReady(ctx context.Context) {
|
|||||||
r.ready = ok
|
r.ready = ok
|
||||||
r.mu.Unlock()
|
r.mu.Unlock()
|
||||||
if ok && !was {
|
if ok && !was {
|
||||||
log.Printf("mavgpud: model ready")
|
log.Printf("mavgpud: %s ready", r.name)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -25,7 +25,7 @@ func fakeServer(t *testing.T, body string) string {
|
|||||||
// status of a routine yield is identical to that of a real crash. Reading the
|
// status of a routine yield is identical to that of a real crash. Reading the
|
||||||
// mavgpud log, the two were indistinguishable (Vikunja #491).
|
// mavgpud log, the two were indistinguishable (Vikunja #491).
|
||||||
func TestStopMarksTheExitAsAYield(t *testing.T) {
|
func TestStopMarksTheExitAsAYield(t *testing.T) {
|
||||||
r := newRunner(fakeServer(t, "while : ; do sleep 1 ; done"), nil, "")
|
r := newRunner("fake", fakeServer(t, "while : ; do sleep 1 ; done"), nil, "")
|
||||||
if err := r.start(); err != nil {
|
if err := r.start(); err != nil {
|
||||||
t.Fatalf("start: %v", err)
|
t.Fatalf("start: %v", err)
|
||||||
}
|
}
|
||||||
@@ -49,7 +49,7 @@ func TestStopMarksTheExitAsAYield(t *testing.T) {
|
|||||||
// Stopping when nothing is running must not arm the flag for the next child.
|
// Stopping when nothing is running must not arm the flag for the next child.
|
||||||
// The next exit after that would be a real crash logged as a yield.
|
// The next exit after that would be a real crash logged as a yield.
|
||||||
func TestStopWithNoChildDoesNotArmTheFlag(t *testing.T) {
|
func TestStopWithNoChildDoesNotArmTheFlag(t *testing.T) {
|
||||||
r := newRunner("/nonexistent", nil, "")
|
r := newRunner("fake", "/nonexistent", nil, "")
|
||||||
r.stop(10 * time.Millisecond)
|
r.stop(10 * time.Millisecond)
|
||||||
r.mu.Lock()
|
r.mu.Lock()
|
||||||
defer r.mu.Unlock()
|
defer r.mu.Unlock()
|
||||||
|
|||||||
+27
-7
@@ -5,12 +5,17 @@
|
|||||||
// is detected sends it as a PushToTalk frame to the voice server. The reply
|
// is detected sends it as a PushToTalk frame to the voice server. The reply
|
||||||
// audio is played back through aplay(1).
|
// audio is played back through aplay(1).
|
||||||
//
|
//
|
||||||
// No wake-word model yet (MVP uses voice-activity-only trigger). The
|
// Voice activity is silero-vad when -vad-model points at the graph, and an
|
||||||
// SurfaceVoice auth layer caps all commands at L0 (no destructive acts),
|
// energy threshold when it does not. Silero declines noise the threshold
|
||||||
// making accidental triggers safe by design. A proper wake-word engine
|
// accepts: 0 frames against 68 to 99 on the four fixtures, measured in
|
||||||
// (openWakeWord / Silero VAD ONNX) is the planned upgrade — the VAD shape
|
// docs/evals/2026-08-09-silero-vad.md. Note that the model window is 512
|
||||||
// (30ms frames, 16kHz PCM) matches silero-vad's input interface exactly, so
|
// samples and the capture frame is 480, so silero.go re-chunks. This comment
|
||||||
// swapping energy-threshold for ONNX-inference is a local change in vad.go.
|
// used to say the two matched, which was true of silero v4.
|
||||||
|
//
|
||||||
|
// There is still no wake-word model, so anything spoken near the microphone
|
||||||
|
// becomes a turn (V-487 stage two). The SurfaceVoice auth layer caps all
|
||||||
|
// commands at L0 (no destructive acts), which is what makes an accidental
|
||||||
|
// trigger safe rather than expensive.
|
||||||
//
|
//
|
||||||
// While a reply is playing the capture side is muted (half-duplex): without
|
// While a reply is playing the capture side is muted (half-duplex): without
|
||||||
// it, Maven's own voice comes back in through the mic and she answers
|
// it, Maven's own voice comes back in through the mic and she answers
|
||||||
@@ -73,6 +78,9 @@ func run(args []string) error {
|
|||||||
bargeIn := flag.Bool("barge-in", false, "cut Maven off when he talks over her (needs a room-tuned -barge-in-rms)")
|
bargeIn := flag.Bool("barge-in", false, "cut Maven off when he talks over her (needs a room-tuned -barge-in-rms)")
|
||||||
bargeRMS := flag.Int("barge-in-rms", defaultBargeRMS, "RMS x10000 a frame must clear to count as barge-in")
|
bargeRMS := flag.Int("barge-in-rms", defaultBargeRMS, "RMS x10000 a frame must clear to count as barge-in")
|
||||||
bargeFrames := flag.Int("barge-in-frames", defaultBargeFrames, "consecutive frames over -barge-in-rms before playback is cut")
|
bargeFrames := flag.Int("barge-in-frames", defaultBargeFrames, "consecutive frames over -barge-in-rms before playback is cut")
|
||||||
|
vadModel := flag.String("vad-model", "", "silero-vad onnx file; empty runs the energy threshold instead")
|
||||||
|
vadThreshold := flag.Float64("vad-threshold", defaultSileroThreshold, "speech probability a frame must clear")
|
||||||
|
onnxLib := flag.String("onnx-lib", os.Getenv("MAVEN_ONNX_LIB"), "libonnxruntime.so, needed with -vad-model")
|
||||||
flag.CommandLine.Parse(args)
|
flag.CommandLine.Parse(args)
|
||||||
|
|
||||||
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM, syscall.SIGHUP)
|
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM, syscall.SIGHUP)
|
||||||
@@ -82,8 +90,20 @@ func run(args []string) error {
|
|||||||
vc := voice.Dial(*addr)
|
vc := voice.Dial(*addr)
|
||||||
defer vc.Close()
|
defer vc.Close()
|
||||||
|
|
||||||
// VAD engine.
|
// VAD engine. A model that will not load is logged and not fatal: the
|
||||||
|
// energy threshold is worse, and it is a great deal better than a
|
||||||
|
// listening client that refuses to start.
|
||||||
vad := NewVAD(*minRMS, *speechMs, *silenceMs, *maxMs)
|
vad := NewVAD(*minRMS, *speechMs, *silenceMs, *maxMs)
|
||||||
|
if *vadModel != "" {
|
||||||
|
s, err := newSileroVAD(*vadModel, *onnxLib)
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("mavwaked: silero unavailable, energy threshold unchanged: %v", err)
|
||||||
|
} else {
|
||||||
|
defer s.Close()
|
||||||
|
vad.UseSilero(s, *vadThreshold)
|
||||||
|
log.Printf("mavwaked: silero-vad from %s, threshold %.2f", *vadModel, *vadThreshold)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Audio source.
|
// Audio source.
|
||||||
var src io.ReadCloser
|
var src io.ReadCloser
|
||||||
|
|||||||
@@ -0,0 +1,171 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
// silero-vad, the speech detector that replaces the energy threshold (V-487).
|
||||||
|
//
|
||||||
|
// Why an energy threshold is not a voice activity detector. It answers "is
|
||||||
|
// this frame loud", and a fan, a door and a television are all loud. mavwaked
|
||||||
|
// sends every utterance it accepts to speech-to-text and then to the daemon,
|
||||||
|
// so a false trigger is a turn Maven takes on something nobody said to her.
|
||||||
|
// Silero answers "is this frame speech", which is the question.
|
||||||
|
//
|
||||||
|
// It is 2.3MB of ONNX and runs on one CPU core in real time. That is not an
|
||||||
|
// aside: this is the one model in the system that may never be offloaded or
|
||||||
|
// gated on GPU admission, because a wake path that waits on a card is not a
|
||||||
|
// wake path.
|
||||||
|
//
|
||||||
|
// Nil is a working value. Without -vad-model the daemon runs the energy VAD
|
||||||
|
// exactly as it did before this file existed.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"sync"
|
||||||
|
|
||||||
|
ort "github.com/yalue/onnxruntime_go"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
// sileroWindow — samples per inference at 16kHz. The model is fixed at
|
||||||
|
// 512 and does not accept another size, which is why this file
|
||||||
|
// re-chunks rather than reusing the 480-sample capture frame. main.go
|
||||||
|
// used to claim the two matched; that was true of silero v4.
|
||||||
|
sileroWindow = 512
|
||||||
|
|
||||||
|
// sileroContext — samples of the previous window prepended to each
|
||||||
|
// inference, as the reference implementation does. Without it the first
|
||||||
|
// milliseconds of every window are judged with no history and speech
|
||||||
|
// onsets score low.
|
||||||
|
sileroContext = 64
|
||||||
|
|
||||||
|
// sileroState — the LSTM state carried between windows, [2][1][128].
|
||||||
|
sileroStateDim = 128
|
||||||
|
|
||||||
|
// defaultSileroThreshold — probability above which a window is speech.
|
||||||
|
// 0.5 is the reference default. Raising it costs speech onsets, which
|
||||||
|
// are the quietest part of an utterance.
|
||||||
|
defaultSileroThreshold = 0.5
|
||||||
|
)
|
||||||
|
|
||||||
|
// sileroVAD holds one ONNX session and the streaming state around it. It is
|
||||||
|
// fed 30ms capture frames and answers per frame, buffering across calls
|
||||||
|
// because 480 samples never line up with a 512-sample window.
|
||||||
|
type sileroVAD struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
session *ort.DynamicAdvancedSession
|
||||||
|
|
||||||
|
pending []float32 // samples not yet part of a full window
|
||||||
|
context [sileroContext]float32 // tail of the previous window
|
||||||
|
state []float32 // [2][1][128], carried between windows
|
||||||
|
last float64 // most recent probability, held between windows
|
||||||
|
sr []int64
|
||||||
|
}
|
||||||
|
|
||||||
|
// newSileroVAD loads the graph. The ONNX environment is initialised here when
|
||||||
|
// nothing else has done it, because mavwaked has no embedder to do it first.
|
||||||
|
func newSileroVAD(modelPath, libPath string) (*sileroVAD, error) {
|
||||||
|
if !ort.IsInitialized() {
|
||||||
|
if libPath != "" {
|
||||||
|
ort.SetSharedLibraryPath(libPath)
|
||||||
|
}
|
||||||
|
if err := ort.InitializeEnvironment(); err != nil {
|
||||||
|
return nil, fmt.Errorf("silero: onnx runtime: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
s, err := ort.NewDynamicAdvancedSession(modelPath,
|
||||||
|
[]string{"input", "state", "sr"}, []string{"output", "stateN"}, nil)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("silero: load %s: %w", modelPath, err)
|
||||||
|
}
|
||||||
|
return &sileroVAD{
|
||||||
|
session: s,
|
||||||
|
state: make([]float32, 2*sileroStateDim),
|
||||||
|
sr: []int64{16000},
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Speech reports whether the frame carries speech, and the probability behind
|
||||||
|
// that answer. A frame that completes no window inherits the previous
|
||||||
|
// probability, so the caller sees one answer per frame either way.
|
||||||
|
func (s *sileroVAD) Speech(frame []int16, threshold float64) (bool, float64) {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
|
||||||
|
for _, v := range frame {
|
||||||
|
s.pending = append(s.pending, float32(v)/32768.0)
|
||||||
|
}
|
||||||
|
for len(s.pending) >= sileroWindow {
|
||||||
|
p, err := s.infer(s.pending[:sileroWindow])
|
||||||
|
if err != nil {
|
||||||
|
// A failed inference must not silence the microphone. Hold the
|
||||||
|
// last answer and let the next window try again.
|
||||||
|
break
|
||||||
|
}
|
||||||
|
s.last = p
|
||||||
|
s.pending = s.pending[sileroWindow:]
|
||||||
|
}
|
||||||
|
return s.last >= threshold, s.last
|
||||||
|
}
|
||||||
|
|
||||||
|
// infer runs one window and rolls the state and the context forward.
|
||||||
|
func (s *sileroVAD) infer(window []float32) (float64, error) {
|
||||||
|
in := make([]float32, sileroContext+sileroWindow)
|
||||||
|
copy(in, s.context[:])
|
||||||
|
copy(in[sileroContext:], window)
|
||||||
|
|
||||||
|
inT, err := ort.NewTensor(ort.NewShape(1, int64(len(in))), in)
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
defer inT.Destroy()
|
||||||
|
stT, err := ort.NewTensor(ort.NewShape(2, 1, sileroStateDim), s.state)
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
defer stT.Destroy()
|
||||||
|
srT, err := ort.NewTensor(ort.NewShape(1), s.sr)
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
defer srT.Destroy()
|
||||||
|
|
||||||
|
out, err := ort.NewEmptyTensor[float32](ort.NewShape(1, 1))
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
defer out.Destroy()
|
||||||
|
next, err := ort.NewEmptyTensor[float32](ort.NewShape(2, 1, sileroStateDim))
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
defer next.Destroy()
|
||||||
|
|
||||||
|
if err := s.session.Run(
|
||||||
|
[]ort.Value{inT, stT, srT},
|
||||||
|
[]ort.Value{out, next},
|
||||||
|
); err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
copy(s.state, next.GetData())
|
||||||
|
copy(s.context[:], in[len(in)-sileroContext:])
|
||||||
|
return float64(out.GetData()[0]), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Reset drops the streaming state. Called at every utterance boundary and
|
||||||
|
// after barge-in, so echo-era history never scores the next sentence.
|
||||||
|
func (s *sileroVAD) Reset() {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
s.pending = s.pending[:0]
|
||||||
|
s.context = [sileroContext]float32{}
|
||||||
|
for i := range s.state {
|
||||||
|
s.state[i] = 0
|
||||||
|
}
|
||||||
|
s.last = 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// Close releases the session.
|
||||||
|
func (s *sileroVAD) Close() error {
|
||||||
|
if s == nil || s.session == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return s.session.Destroy()
|
||||||
|
}
|
||||||
@@ -0,0 +1,149 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
// What this measures. The energy threshold cannot tell a voice from a
|
||||||
|
// television, and every utterance it accepts becomes a turn. So the test that
|
||||||
|
// matters is not "does silero find speech" — it is "does it decline what the
|
||||||
|
// energy threshold accepts".
|
||||||
|
//
|
||||||
|
// Speech is the four piper fixtures mavsttd already scores against. They are
|
||||||
|
// synthesised, so nothing of the owner's voice is committed. Non-speech is
|
||||||
|
// white noise at the same loudness, which is the cheapest thing that fools an
|
||||||
|
// energy floor and the honest floor for this claim.
|
||||||
|
//
|
||||||
|
// Both halves skip without models/vad/silero_vad.onnx and MAVEN_ONNX_LIB,
|
||||||
|
// like the TestONNX measurements in internal/router/eval.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"math"
|
||||||
|
"math/rand"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
const wavHeader = 44 // 16kHz mono s16le, written by piper
|
||||||
|
|
||||||
|
func loadSilero(t *testing.T) *sileroVAD {
|
||||||
|
t.Helper()
|
||||||
|
model := filepath.Join("..", "..", "models", "vad", "silero_vad.onnx")
|
||||||
|
lib := os.Getenv("MAVEN_ONNX_LIB")
|
||||||
|
if _, err := os.Stat(model); err != nil {
|
||||||
|
t.Skipf("missing %s: %v", model, err)
|
||||||
|
}
|
||||||
|
if lib == "" {
|
||||||
|
t.Skip("MAVEN_ONNX_LIB unset")
|
||||||
|
}
|
||||||
|
s, err := newSileroVAD(model, lib)
|
||||||
|
if err != nil {
|
||||||
|
t.Skipf("silero unavailable: %v", err)
|
||||||
|
}
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
|
// feedAll runs a whole clip through a VAD and reports how many utterances it
|
||||||
|
// produced and how many frames it called speech.
|
||||||
|
func feedAll(v *VAD, pcm []int16) (utterances, speechFrames int) {
|
||||||
|
for i := 0; i+frameSamples <= len(pcm); i += frameSamples {
|
||||||
|
frame := pcm[i : i+frameSamples]
|
||||||
|
utt, state := v.Feed(frame)
|
||||||
|
if state == StateSpeech {
|
||||||
|
speechFrames++
|
||||||
|
}
|
||||||
|
if utt.Bytes != nil {
|
||||||
|
utterances++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return utterances, speechFrames
|
||||||
|
}
|
||||||
|
|
||||||
|
func readFixture(t *testing.T, name string) []int16 {
|
||||||
|
t.Helper()
|
||||||
|
raw, err := os.ReadFile(filepath.Join("..", "mavsttd", "testdata", name))
|
||||||
|
if err != nil {
|
||||||
|
t.Skipf("missing fixture %s: %v", name, err)
|
||||||
|
}
|
||||||
|
if len(raw) <= wavHeader {
|
||||||
|
t.Fatalf("%s: %d bytes, no audio", name, len(raw))
|
||||||
|
}
|
||||||
|
return PCMToI16(raw[wavHeader:])
|
||||||
|
}
|
||||||
|
|
||||||
|
// noise returns white noise scaled to the same RMS as ref. Same loudness,
|
||||||
|
// nothing said.
|
||||||
|
func noise(ref []int16, seed int64) []int16 {
|
||||||
|
target := frameRMS(ref)
|
||||||
|
r := rand.New(rand.NewSource(seed))
|
||||||
|
out := make([]int16, len(ref))
|
||||||
|
for i := range out {
|
||||||
|
out[i] = int16(r.NormFloat64() * target * 32768.0)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSileroHearsSpeechAndDeclinesNoise(t *testing.T) {
|
||||||
|
s := loadSilero(t)
|
||||||
|
defer s.Close()
|
||||||
|
|
||||||
|
for _, name := range []string{"ru_fact.wav", "ru_query.wav", "ru_reminder.wav", "en_act.wav"} {
|
||||||
|
pcm := readFixture(t, name)
|
||||||
|
|
||||||
|
v := NewVAD(0, 0, 0, 0)
|
||||||
|
v.UseSilero(s, defaultSileroThreshold)
|
||||||
|
_, spoke := feedAll(v, pcm)
|
||||||
|
if spoke == 0 {
|
||||||
|
t.Errorf("%s: silero heard no speech in a spoken clip", name)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.Reset()
|
||||||
|
v2 := NewVAD(0, 0, 0, 0)
|
||||||
|
v2.UseSilero(s, defaultSileroThreshold)
|
||||||
|
_, heard := feedAll(v2, noise(pcm, 7))
|
||||||
|
|
||||||
|
energy := NewVAD(0, 0, 0, 0)
|
||||||
|
_, energyHeard := feedAll(energy, noise(pcm, 7))
|
||||||
|
|
||||||
|
t.Logf("%s: speech frames — silero on speech %d, silero on noise %d, energy on noise %d",
|
||||||
|
name, spoke, heard, energyHeard)
|
||||||
|
if heard >= energyHeard {
|
||||||
|
t.Errorf("%s: silero called %d noise frames speech, energy called %d — no improvement",
|
||||||
|
name, heard, energyHeard)
|
||||||
|
}
|
||||||
|
s.Reset()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// BenchmarkSileroFrame answers the only performance question that matters
|
||||||
|
// here: one 30ms frame must cost far less than 30ms on one core, or the
|
||||||
|
// detector cannot run always-on beside everything else on that machine.
|
||||||
|
func BenchmarkSileroFrame(b *testing.B) {
|
||||||
|
s := loadSilero(&testing.T{})
|
||||||
|
if s == nil {
|
||||||
|
b.Skip("silero unavailable")
|
||||||
|
}
|
||||||
|
defer s.Close()
|
||||||
|
frame := make([]int16, frameSamples)
|
||||||
|
for i := range frame {
|
||||||
|
frame[i] = int16(i%400 - 200)
|
||||||
|
}
|
||||||
|
for i := 0; i < b.N; i++ {
|
||||||
|
s.Speech(frame, defaultSileroThreshold)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestSileroRechunksAcrossFrames pins the reason this file exists. The capture
|
||||||
|
// frame is 480 samples and the model window is 512, so a detector that ran one
|
||||||
|
// inference per frame would be feeding the model a shape it does not accept.
|
||||||
|
func TestSileroRechunksAcrossFrames(t *testing.T) {
|
||||||
|
s := loadSilero(t)
|
||||||
|
defer s.Close()
|
||||||
|
|
||||||
|
silence := make([]int16, frameSamples)
|
||||||
|
for i := 0; i < 20; i++ {
|
||||||
|
if _, p := s.Speech(silence, defaultSileroThreshold); math.IsNaN(p) {
|
||||||
|
t.Fatalf("frame %d: probability is NaN", i)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(s.pending) >= sileroWindow {
|
||||||
|
t.Errorf("pending grew to %d samples, so windows are not being consumed", len(s.pending))
|
||||||
|
}
|
||||||
|
}
|
||||||
+35
-1
@@ -68,6 +68,37 @@ type VAD struct {
|
|||||||
// follows the room's ambient level. Initialised to minRMS; updated
|
// follows the room's ambient level. Initialised to minRMS; updated
|
||||||
// on each silence frame.
|
// on each silence frame.
|
||||||
floorRMS float64
|
floorRMS float64
|
||||||
|
|
||||||
|
// speech is silero-vad, or nil. When it is set the energy floor decides
|
||||||
|
// nothing: the question becomes "is this speech" rather than "is this
|
||||||
|
// loud", and the noise floor is not even tracked. Everything after that
|
||||||
|
// answer — the speech hold, the silence hold, the length cap, the
|
||||||
|
// buffer — is the same state machine either way, which is why the
|
||||||
|
// detector goes here and not around this type.
|
||||||
|
speech *sileroVAD
|
||||||
|
speechMin float64
|
||||||
|
}
|
||||||
|
|
||||||
|
// UseSilero swaps the energy threshold for the model. Passing nil is a
|
||||||
|
// no-op, so a caller that could not load the graph keeps a working VAD.
|
||||||
|
func (v *VAD) UseSilero(s *sileroVAD, threshold float64) {
|
||||||
|
if s == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if threshold <= 0 {
|
||||||
|
threshold = defaultSileroThreshold
|
||||||
|
}
|
||||||
|
v.speech = s
|
||||||
|
v.speechMin = threshold
|
||||||
|
}
|
||||||
|
|
||||||
|
// isSpeech answers the one question the state machine asks of a frame.
|
||||||
|
func (v *VAD) isSpeech(frame []int16, rms float64) bool {
|
||||||
|
if v.speech != nil {
|
||||||
|
ok, _ := v.speech.Speech(frame, v.speechMin)
|
||||||
|
return ok
|
||||||
|
}
|
||||||
|
return rms >= v.floorRMS
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewVAD creates a VAD with the given thresholds. Zero values use defaults.
|
// NewVAD creates a VAD with the given thresholds. Zero values use defaults.
|
||||||
@@ -110,7 +141,7 @@ func (v *VAD) State() SpeechState { return v.state }
|
|||||||
// should send the audio to the voice server before feeding more frames.
|
// should send the audio to the voice server before feeding more frames.
|
||||||
func (v *VAD) Feed(frame []int16) (_ audio.Audio, state SpeechState) {
|
func (v *VAD) Feed(frame []int16) (_ audio.Audio, state SpeechState) {
|
||||||
rms := frameRMS(frame)
|
rms := frameRMS(frame)
|
||||||
isSpeech := rms >= v.floorRMS
|
isSpeech := v.isSpeech(frame, rms)
|
||||||
|
|
||||||
switch v.state {
|
switch v.state {
|
||||||
case StateSilence:
|
case StateSilence:
|
||||||
@@ -175,6 +206,9 @@ func (v *VAD) Feed(frame []int16) (_ audio.Audio, state SpeechState) {
|
|||||||
func (v *VAD) Reset() { v.reset() }
|
func (v *VAD) Reset() { v.reset() }
|
||||||
|
|
||||||
func (v *VAD) reset() {
|
func (v *VAD) reset() {
|
||||||
|
if v.speech != nil {
|
||||||
|
v.speech.Reset()
|
||||||
|
}
|
||||||
v.state = StateSilence
|
v.state = StateSilence
|
||||||
v.speechFrames = 0
|
v.speechFrames = 0
|
||||||
v.silenceFrames = 0
|
v.silenceFrames = 0
|
||||||
|
|||||||
@@ -0,0 +1,156 @@
|
|||||||
|
"""CrisperWhisper 2.0 turbo as an HTTP service, for Maven's stt.Pair.
|
||||||
|
|
||||||
|
Two endpoints and no framework.
|
||||||
|
|
||||||
|
GET /health 200 once the model is loaded, 503 while it is loading.
|
||||||
|
POST /transcribe raw 16kHz mono PCM in, {"text","confidence"} out.
|
||||||
|
|
||||||
|
The body is the PCM itself rather than JSON. A minute of 16kHz mono is under
|
||||||
|
2MB raw and about 2.6MB base64, and the format is fixed at the Maven seam, so
|
||||||
|
headers carry it more cheaply than an envelope.
|
||||||
|
|
||||||
|
Why this exists at all: whisper.cpp cannot load CW2. It derives its language
|
||||||
|
count from the vocabulary size, and CW2's 51897 tokens shift seven special
|
||||||
|
token ids. So mavsttd stays whisper.cpp on homesrv and this runs beside the
|
||||||
|
model on workpc, where it scores 10.4% WER in Russian against the floor's 27.5%
|
||||||
|
(docs/evals/2026-08-09-crisperwhisper2-russian-wer.md in the Maven repo).
|
||||||
|
|
||||||
|
Intended mode, not verbatim. The owner asked for what he meant to say, not
|
||||||
|
every stutter on the way there.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import hmac
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
HOST = os.environ.get("CW2_HOST", "0.0.0.0")
|
||||||
|
PORT = int(os.environ.get("CW2_PORT", "8081"))
|
||||||
|
SIZE = os.environ.get("CW2_SIZE", "turbo")
|
||||||
|
MODE = os.environ.get("CW2_MODE", "intended")
|
||||||
|
TOKEN = os.environ.get("CW2_TOKEN", "")
|
||||||
|
# 25MB is about thirteen minutes of 16kHz mono. Longer than any utterance and
|
||||||
|
# short enough that a wrong caller cannot exhaust memory.
|
||||||
|
MAX_BODY = int(os.environ.get("CW2_MAX_BODY", str(25 * 1024 * 1024)))
|
||||||
|
|
||||||
|
logging.basicConfig(
|
||||||
|
level=logging.INFO, format="%(asctime)s cw2: %(message)s", stream=sys.stderr
|
||||||
|
)
|
||||||
|
log = logging.getLogger("cw2")
|
||||||
|
|
||||||
|
_model = None
|
||||||
|
# The card holds one model and transcribes one utterance at a time. The lock is
|
||||||
|
# what makes a second caller wait rather than corrupt the first.
|
||||||
|
_lock = threading.Lock()
|
||||||
|
|
||||||
|
|
||||||
|
def load_model():
|
||||||
|
global _model
|
||||||
|
from crisperwhisper import CrisperWhisperModel
|
||||||
|
|
||||||
|
t0 = time.perf_counter()
|
||||||
|
# backend is forced. With ctranslate2 importable, "auto" picks ct2, which is
|
||||||
|
# CUDA-only and this card is AMD.
|
||||||
|
m = CrisperWhisperModel(
|
||||||
|
SIZE, backend="transformers", compute_type="float16", device="cuda"
|
||||||
|
)
|
||||||
|
_model = m
|
||||||
|
log.info("loaded %s in %.1fs, mode=%s", SIZE, time.perf_counter() - t0, MODE)
|
||||||
|
|
||||||
|
|
||||||
|
def authorised(headers):
|
||||||
|
if not TOKEN:
|
||||||
|
return True
|
||||||
|
got = headers.get("Authorization", "")
|
||||||
|
return hmac.compare_digest(got, "Bearer " + TOKEN)
|
||||||
|
|
||||||
|
|
||||||
|
class Handler(BaseHTTPRequestHandler):
|
||||||
|
protocol_version = "HTTP/1.1"
|
||||||
|
|
||||||
|
def log_message(self, fmt, *args):
|
||||||
|
log.info(fmt, *args)
|
||||||
|
|
||||||
|
def _send(self, code, payload):
|
||||||
|
body = json.dumps(payload, ensure_ascii=False).encode("utf-8")
|
||||||
|
self.send_response(code)
|
||||||
|
self.send_header("Content-Type", "application/json; charset=utf-8")
|
||||||
|
self.send_header("Content-Length", str(len(body)))
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(body)
|
||||||
|
|
||||||
|
def do_GET(self):
|
||||||
|
if self.path.rstrip("/") != "/health":
|
||||||
|
self._send(404, {"error": "not found"})
|
||||||
|
return
|
||||||
|
if _model is None:
|
||||||
|
self._send(503, {"status": "loading"})
|
||||||
|
return
|
||||||
|
self._send(200, {"status": "ok", "model": SIZE, "mode": MODE})
|
||||||
|
|
||||||
|
def do_POST(self):
|
||||||
|
if self.path.rstrip("/") != "/transcribe":
|
||||||
|
self._send(404, {"error": "not found"})
|
||||||
|
return
|
||||||
|
if not authorised(self.headers):
|
||||||
|
self._send(401, {"error": "unauthorised"})
|
||||||
|
return
|
||||||
|
if _model is None:
|
||||||
|
self._send(503, {"error": "loading"})
|
||||||
|
return
|
||||||
|
|
||||||
|
length = int(self.headers.get("Content-Length", "0"))
|
||||||
|
if length <= 0 or length > MAX_BODY:
|
||||||
|
self._send(413, {"error": "bad body length"})
|
||||||
|
return
|
||||||
|
raw = self.rfile.read(length)
|
||||||
|
|
||||||
|
rate = int(self.headers.get("X-Sample-Rate", "16000"))
|
||||||
|
channels = int(self.headers.get("X-Channels", "1"))
|
||||||
|
bits = int(self.headers.get("X-Sample-Bits", "16"))
|
||||||
|
lang = self.headers.get("X-Language", "ru") or "ru"
|
||||||
|
if channels != 1 or bits != 16:
|
||||||
|
self._send(400, {"error": "want 16-bit mono pcm"})
|
||||||
|
return
|
||||||
|
|
||||||
|
# int16 little-endian to the float32 the encoder wants.
|
||||||
|
wav = np.frombuffer(raw, dtype="<i2").astype(np.float32) / 32768.0
|
||||||
|
if wav.size == 0:
|
||||||
|
self._send(200, {"text": "", "confidence": 0.0})
|
||||||
|
return
|
||||||
|
|
||||||
|
t0 = time.perf_counter()
|
||||||
|
try:
|
||||||
|
with _lock:
|
||||||
|
res = _model.transcribe(wav, sr=rate, language=lang, mode=MODE)
|
||||||
|
except Exception as exc: # noqa: BLE001 - the caller falls back to mavsttd
|
||||||
|
log.exception("transcribe failed")
|
||||||
|
self._send(500, {"error": str(exc)})
|
||||||
|
return
|
||||||
|
elapsed = time.perf_counter() - t0
|
||||||
|
text = (res.text or "").strip()
|
||||||
|
log.info("%.2fs audio in %.2fs: %r", wav.size / rate, elapsed, text[:60])
|
||||||
|
# The model reports no calibrated score. 1.0 would be a claim, and the
|
||||||
|
# Maven side reads confidence only to log it.
|
||||||
|
self._send(200, {"text": text, "confidence": 0.0})
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
if not TOKEN:
|
||||||
|
log.warning("no CW2_TOKEN set: anything on the LAN can post audio here")
|
||||||
|
# Bind before loading, so a restart answers 503 rather than refusing the
|
||||||
|
# connection. Both make Maven fall back, but only one of them says why.
|
||||||
|
srv = ThreadingHTTPServer((HOST, PORT), Handler)
|
||||||
|
threading.Thread(target=load_model, daemon=True).start()
|
||||||
|
log.info("listening on %s:%d", HOST, PORT)
|
||||||
|
srv.serve_forever()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
+23
-2
@@ -78,10 +78,30 @@
|
|||||||
"Addressed by LAN address, not container name: mavgpud runs on another",
|
"Addressed by LAN address, not container name: mavgpud runs on another",
|
||||||
"machine and there is no shared docker network to name it on."
|
"machine and there is no shared docker network to name it on."
|
||||||
],
|
],
|
||||||
|
"//workstation.stt": [
|
||||||
|
"CrisperWhisper 2.0 turbo on the same machine, a second service on port",
|
||||||
|
"8081 and not a second endpoint on mavgpud. whisper.cpp cannot load CW2 at",
|
||||||
|
"all: it derives its language count from the vocabulary size, and CW2's",
|
||||||
|
"51897 tokens shift seven special token ids. So it runs under transformers",
|
||||||
|
"there and mavsttd stays whisper.cpp here.",
|
||||||
|
"Worth the second service: CW2 turbo scores 10.4% WER in Russian against",
|
||||||
|
"27.5% for the ggml-small.bin mavsttd loads, measured on 200 Golos clips",
|
||||||
|
"in docs/evals/2026-08-09-crisperwhisper2-russian-wer.md.",
|
||||||
|
"Deleting this block sends every utterance to mavsttd, which is what the",
|
||||||
|
"box did before it existed. A worse transcript is still a turn, so the",
|
||||||
|
"fallback is silent and Kami is never told which machine heard him.",
|
||||||
|
"The token is what stops anything on the LAN posting audio to that port."
|
||||||
|
],
|
||||||
"workstation": {
|
"workstation": {
|
||||||
"url": "http://192.168.1.105:8080",
|
"url": "http://192.168.1.105:8080",
|
||||||
"probe": "15s",
|
"probe": "15s",
|
||||||
"timeout": "90s"
|
"timeout": "90s",
|
||||||
|
"stt": {
|
||||||
|
"url": "http://192.168.1.105:8081/transcribe",
|
||||||
|
"token": "${MAVEN_STT_TOKEN}",
|
||||||
|
"probe": "15s",
|
||||||
|
"timeout": "10s"
|
||||||
|
}
|
||||||
},
|
},
|
||||||
|
|
||||||
"//search": [
|
"//search": [
|
||||||
@@ -231,7 +251,8 @@
|
|||||||
"embedder": {
|
"embedder": {
|
||||||
"model_path": "/opt/maven/models/embedder/multilingual-e5-small/model_quantized.onnx",
|
"model_path": "/opt/maven/models/embedder/multilingual-e5-small/model_quantized.onnx",
|
||||||
"tokenizer_path": "/opt/maven/models/embedder/multilingual-e5-small/tokenizer.json",
|
"tokenizer_path": "/opt/maven/models/embedder/multilingual-e5-small/tokenizer.json",
|
||||||
"lib_path": "/opt/maven/lib/libonnxruntime.so"
|
"lib_path": "/opt/maven/lib/libonnxruntime.so",
|
||||||
|
"heads_path": "/opt/maven/models/embedder/router-heads/router_heads.onnx"
|
||||||
},
|
},
|
||||||
"llm_router": true,
|
"llm_router": true,
|
||||||
"query_min_score": 0.55,
|
"query_min_score": 0.55,
|
||||||
|
|||||||
+21
-5
@@ -2,9 +2,14 @@
|
|||||||
"listen": ":8080",
|
"listen": ":8080",
|
||||||
"llama_addr": "127.0.0.1:10000",
|
"llama_addr": "127.0.0.1:10000",
|
||||||
"llama_bin": "llama-server",
|
"llama_bin": "llama-server",
|
||||||
|
"//llama_args": [
|
||||||
|
"E4B carries no MTP tensors, so the speculative flags are gone with the 12B.",
|
||||||
|
"MTP on this box is a separate gguf of architecture gemma4-assistant with",
|
||||||
|
"nextn_predict_layers=4, and mtp-gemma-4-12B-it-BF16 is the only one there is.",
|
||||||
|
"Its head is trained against the 12B's hidden states, so it cannot drive E4B."
|
||||||
|
],
|
||||||
"llama_args": [
|
"llama_args": [
|
||||||
"-m", "/mnt/D/AI/gemma4/gemma-4-12B-it-qat-UD-Q4_K_XL.gguf",
|
"-m", "/mnt/D/AI/gemma4/gemma-4-E4B-it-qat-UD-Q4_K_XL.gguf",
|
||||||
"-md", "/mnt/D/AI/gemma4/mtp-gemma-4-12B-it-BF16.gguf",
|
|
||||||
"-ngl", "99",
|
"-ngl", "99",
|
||||||
"-fa", "on",
|
"-fa", "on",
|
||||||
"-np", "1",
|
"-np", "1",
|
||||||
@@ -15,11 +20,22 @@
|
|||||||
"--batch-size", "2048",
|
"--batch-size", "2048",
|
||||||
"--ubatch-size", "512",
|
"--ubatch-size", "512",
|
||||||
"--jinja",
|
"--jinja",
|
||||||
"--chat-template-kwargs", "{\"enable_thinking\":false}",
|
"--chat-template-kwargs", "{\"enable_thinking\":false}"
|
||||||
"--spec-type", "draft-mtp",
|
|
||||||
"--spec-draft-n-max", "2"
|
|
||||||
],
|
],
|
||||||
|
|
||||||
|
"//stt": [
|
||||||
|
"CrisperWhisper 2.0 turbo, which Maven reaches directly on port 8081.",
|
||||||
|
"mavgpud runs it because it is a ROCm process on this card: under its own",
|
||||||
|
"systemd unit it registered on the KFD and the supervisor evicted",
|
||||||
|
"llama-server every few seconds. CW2_TOKEN comes from the unit's",
|
||||||
|
"EnvironmentFile and is never a flag value."
|
||||||
|
],
|
||||||
|
"stt": {
|
||||||
|
"addr": "127.0.0.1:8081",
|
||||||
|
"bin": "/home/kami/Programs/cw2-eval/.venv/bin/python",
|
||||||
|
"args": ["/home/kami/Programs/cw2-service/serve.py"]
|
||||||
|
},
|
||||||
|
|
||||||
"kfd_root": "/sys/class/kfd/kfd/proc",
|
"kfd_root": "/sys/class/kfd/kfd/proc",
|
||||||
"drm_device": "/sys/class/drm/card1/device",
|
"drm_device": "/sys/class/drm/card1/device",
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
[Unit]
|
[Unit]
|
||||||
# Runs on the workstation (bugmachine), not on homesrv. Install as a systemd
|
# Runs on the workstation (workpc), not on homesrv. Install as a systemd
|
||||||
# user unit and turn on lingering, so the card is supervised after a reboot
|
# user unit and turn on lingering, so the card is supervised after a reboot
|
||||||
# with nobody logged in:
|
# with nobody logged in:
|
||||||
#
|
#
|
||||||
@@ -8,10 +8,15 @@
|
|||||||
# scp deploy/mavgpud.service workpc:~/.config/systemd/user/mavgpud.service
|
# scp deploy/mavgpud.service workpc:~/.config/systemd/user/mavgpud.service
|
||||||
# ssh workpc 'systemctl --user daemon-reload && systemctl --user enable --now mavgpud'
|
# ssh workpc 'systemctl --user daemon-reload && systemctl --user enable --now mavgpud'
|
||||||
# sudo loginctl enable-linger kami
|
# sudo loginctl enable-linger kami
|
||||||
Description=Maven GPU supervisor (holds llama-server while the card is free)
|
Description=Maven GPU supervisor (holds llama-server and CW2 while the card is free)
|
||||||
After=network.target
|
After=network.target
|
||||||
|
|
||||||
[Service]
|
[Service]
|
||||||
|
# CW2_TOKEN for the transcriber child, which inherits this environment. The
|
||||||
|
# token is read from a file and never appears as a flag value, the rule
|
||||||
|
# mavpoll and mavmaild follow. Missing file, no transcriber auth, so keep the
|
||||||
|
# dash off: a mavgpud that cannot read it must fail loudly.
|
||||||
|
EnvironmentFile=%h/Programs/cw2-service/cw2.env
|
||||||
ExecStart=%h/.local/bin/mavgpud -config %h/.config/mavgpud.json
|
ExecStart=%h/.local/bin/mavgpud -config %h/.config/mavgpud.json
|
||||||
Restart=always
|
Restart=always
|
||||||
RestartSec=5
|
RestartSec=5
|
||||||
|
|||||||
@@ -0,0 +1,146 @@
|
|||||||
|
# The routing heads, running in Go
|
||||||
|
|
||||||
|
Date: 2026-08-08. Vikunja V-664.
|
||||||
|
Weights: `router_heads.onnx`, fp32, exported from `heads.pt` on workpc.
|
||||||
|
Fixture: `internal/router/eval/ru_routing_v1.json`, 96 cases, 33 carrying a destination.
|
||||||
|
Runner: `make t PKG=./internal/router/eval/ RUN=TestONNXRoutingHeads`.
|
||||||
|
|
||||||
|
The four heads of V-661 ran nowhere. This is the number they score through the
|
||||||
|
Go cascade. Same fixture and same grammars as `TestONNXBaseline`, and only the
|
||||||
|
middle stage varies.
|
||||||
|
|
||||||
|
## Headline
|
||||||
|
|
||||||
|
| | classifier + ONNX | heads + classifier | gemma-4-12b cascade |
|
||||||
|
|---|---|---|---|
|
||||||
|
| intent | 75.0% (72/96) | **96.9% (93/96)** | 84.4% |
|
||||||
|
| destination | 33.3% (11/33) | **75.8% (25/33)** | 72.7% |
|
||||||
|
| false clarify | 0 | 1 | 2 |
|
||||||
|
| missed clarify | 8 | 1 | 1 |
|
||||||
|
| p50 | 24.5ms | 27.9ms | 329ms |
|
||||||
|
|
||||||
|
A 118M encoder beats the 12B teacher it was distilled from. It wins on both
|
||||||
|
halves of the route, at a twelfth of the latency. The workstation stays the
|
||||||
|
better phraser and is no longer the better router.
|
||||||
|
|
||||||
|
The p50 is not the heads. Most of it is the classifier's own embedder pass on
|
||||||
|
the turns the heads decline, plus process warm-up on the first case. The heads'
|
||||||
|
own forward pass measures 7.3ms on workpc.
|
||||||
|
|
||||||
|
## Two defects were in the way, and the first was not in the heads
|
||||||
|
|
||||||
|
**The tokenizer read every long word backwards.** `encodeWord` backtracks the
|
||||||
|
Viterbi path from the end of a word and prepends each piece. That puts them back
|
||||||
|
in reading order, and a second reverse after the loop undid it. So
|
||||||
|
`query: вода` tokenized to `[0 12 1294 41 12489 2]` where the reference
|
||||||
|
tokenizer gives `[0 41 1294 12 12489 2]`.
|
||||||
|
|
||||||
|
It was found here and only here. The heads were trained through transformers and
|
||||||
|
are read through the hand-written tokenizer. So a mismatch shows up as a score
|
||||||
|
far below what Python measured on the same weights. Nothing else in the suite
|
||||||
|
compares the two.
|
||||||
|
|
||||||
|
Measured on the recall fixture, same 27 cases either way:
|
||||||
|
|
||||||
|
| | reversed | fixed |
|
||||||
|
|---|---|---|
|
||||||
|
| recall@1 | 70.4% (19/27) | **77.8% (21/27)** |
|
||||||
|
| recall@3 | 85.2% (23/27) | **96.3% (26/27)** |
|
||||||
|
| answered after gate | 63.0% | 66.7% |
|
||||||
|
| wrong note on top | 8 | 6 |
|
||||||
|
| false recall | 0/5 | 1/5 |
|
||||||
|
|
||||||
|
The classifier barely moved, 76.0% to 75.0%, and destination 36.4% to 33.3%.
|
||||||
|
Both are one case on 96 and neither is a finding. Seeds and queries were mangled
|
||||||
|
the same way, so cosine survived it. Recall is where it cost, because a stored
|
||||||
|
passage and a live query are different lengths and break differently.
|
||||||
|
|
||||||
|
The one new false recall is the honest cost and it is not being hidden. A
|
||||||
|
sharper embedder scores every candidate higher, including the ones that should
|
||||||
|
have stayed under the gate. That is the same trade `2026-08-04-recall-e5-small.md`
|
||||||
|
recorded when e5-small replaced MiniLM.
|
||||||
|
|
||||||
|
The embedder id now carries a tokenizer revision, `model_quantized@384/tok2`.
|
||||||
|
Stored vectors were written under rev 1 and no longer sit in the same space as a
|
||||||
|
query embedded now. The model file's name never moved, so nothing would have
|
||||||
|
triggered `ReembedAll`. On the box the marker fired on start, and the re-embed
|
||||||
|
rewrote 65 notes and 19 facts in 5 seconds.
|
||||||
|
|
||||||
|
**The clarify head was being thrown away.** It was read only when the intent head
|
||||||
|
cleared its own threshold. That cost 6 of the 8 ambiguous cases. `вода` reads as intent
|
||||||
|
`act` at 0.233 and clarify at 0.983. Burying that handed the turn to the
|
||||||
|
classifier, which routed it confidently and never asked. The clarify head answers
|
||||||
|
a different question, which is whether there is enough here to act on at all. So
|
||||||
|
it decides on its own and decides first.
|
||||||
|
|
||||||
|
| | intent-gated | clarify decides first |
|
||||||
|
|---|---|---|
|
||||||
|
| intent | 90.6% | 96.9% |
|
||||||
|
| missed clarify | 7 | 1 |
|
||||||
|
| false clarify | 0 | 1 |
|
||||||
|
|
||||||
|
## The threshold is measured, not chosen
|
||||||
|
|
||||||
|
Max softmax over the intent head, on the 88 cases carrying an intent:
|
||||||
|
|
||||||
|
| threshold | kept | accuracy kept | wrong kept | right dropped |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| 0.5 | 84 | 96.4% | 3 | 2 |
|
||||||
|
| **0.6** | **81** | **97.5%** | **2** | **4** |
|
||||||
|
| 0.7 | 75 | 97.3% | 2 | 10 |
|
||||||
|
| 0.8 | 64 | 96.9% | 2 | 21 |
|
||||||
|
| 0.9 | 46 | 100.0% | 0 | 37 |
|
||||||
|
|
||||||
|
0.6 is the knee. Every value from 0.7 to 0.85 drops right answers and keeps the
|
||||||
|
same two wrong ones. 0.9 is the only value that clears them, and it costs 37
|
||||||
|
correct routes to do it.
|
||||||
|
|
||||||
|
## Quantization was measured and rejected
|
||||||
|
|
||||||
|
| build | size | intent | destination | p50 |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| fp32 | 470MB | 83/88 (94.3%) | 28/33 (84.8%) | 7.3ms |
|
||||||
|
| int8 | 118MB | 79/88 (89.8%) | 26/33 (78.8%) | 4.0ms |
|
||||||
|
| fp16 | 235MB | will not load | — | — |
|
||||||
|
|
||||||
|
Python numbers, on the heads alone rather than through the cascade. int8 costs
|
||||||
|
4.5 points of intent and 6 of destination to save 3ms. The cascade around it has
|
||||||
|
a p50 over a second when the resident model answers. The fp16 graph is broken:
|
||||||
|
`convert_float_to_float16` leaves a Cast node emitting float16 where the graph
|
||||||
|
expects float, and onnxruntime refuses the session. It was not worth fixing.
|
||||||
|
|
||||||
|
The exporter also had to be told to write one file. It splits weights into a
|
||||||
|
`.onnx.data` sidecar by default. This onnxruntime resolves that path against the
|
||||||
|
process working directory rather than the model. A split graph loads from one
|
||||||
|
directory only.
|
||||||
|
|
||||||
|
## What is still wrong
|
||||||
|
|
||||||
|
**Four of the eight destination misses are calendar.** Training cannot move them.
|
||||||
|
The possessive agenda rules claim those cases at stage 0 and name nothing on
|
||||||
|
purpose. That caution was free while nothing downstream could name anything
|
||||||
|
either. It has now cost four points in three separate measurements. The call is
|
||||||
|
the owner's and it is still open.
|
||||||
|
|
||||||
|
**The slot head is exported and not read.** Slots come from the stage-2
|
||||||
|
extractor. Mapping BIO tags back to text needs character offsets the unigram
|
||||||
|
tokenizer does not keep, which is its own piece of work.
|
||||||
|
|
||||||
|
**`поужинал` is a false clarify**, which is the same defect `thinSingleToken`
|
||||||
|
was narrowed for on 2026-08-01, arriving now from a different direction.
|
||||||
|
|
||||||
|
## On the box
|
||||||
|
|
||||||
|
Deployed to homesrv the same day. `voice: routing heads loaded` on start, and
|
||||||
|
`/trace` shows `routing-heads` winning or thinning every turn. The resident model
|
||||||
|
and the classifier are both marked never asked. Live probes:
|
||||||
|
|
||||||
|
```text
|
||||||
|
что такое TCP? -> kiwix a real definition
|
||||||
|
кто такой Линус Торвальдс? -> kiwix a real answer
|
||||||
|
во сколько я лёг вчера -> personal не нашла у тебя такой записи
|
||||||
|
вода -> thinned to clarify at 0.233 / 0.983
|
||||||
|
```
|
||||||
|
|
||||||
|
A missing or broken weights file logs and leaves the heads nil, which is
|
||||||
|
byte-for-byte the cascade that shipped before this.
|
||||||
@@ -0,0 +1,110 @@
|
|||||||
|
# CrisperWhisper 2.0 in Russian, measured
|
||||||
|
|
||||||
|
Date: 2026-08-09. Vikunja V-665.
|
||||||
|
Corpus: `bond005/sberdevices_golos_10h_crowd`, test split, first 200 clips.
|
||||||
|
Harness: `~/Programs/cw2-eval` on workpc, not in this repo.
|
||||||
|
Runner: `./.venv/bin/python run_asr.py <arm>...` then `score.py`.
|
||||||
|
|
||||||
|
The model card benchmarks disfluency F1 in German and English. It never names
|
||||||
|
Russian and publishes no per-language WER. So the measurement came before the
|
||||||
|
wiring.
|
||||||
|
|
||||||
|
## The corpus
|
||||||
|
|
||||||
|
200 clips, 13.7 minutes, 1001 reference words. Median clip 3.91s, range 1.04s
|
||||||
|
to 13.5s. Golos crowd is short crowd-sourced Russian spoken close to the
|
||||||
|
microphone, which is the nearest public thing to someone talking to Maven. The
|
||||||
|
alternatives are read speech, which flatters every model equally.
|
||||||
|
|
||||||
|
Two rows carry a null transcription and are skipped.
|
||||||
|
|
||||||
|
Scoring normalizes both sides: lowercase, `ё` to `е`, punctuation stripped, and
|
||||||
|
digits expanded to Russian words through num2words. Without that last step a
|
||||||
|
model is penalized for writing `60000` where the reference says
|
||||||
|
`шестьдесят тысяч`. Thousands separators are joined before expansion, or
|
||||||
|
`60 000` expands to `шестьдесят ноль`.
|
||||||
|
|
||||||
|
## Headline
|
||||||
|
|
||||||
|
| arm | WER | CER | exact | empty | RTF |
|
||||||
|
|---|---|---|---|---|---|
|
||||||
|
| cw2-turbo-intended | **10.4%** | 3.4% | 65.5% | 0 | 0.065 |
|
||||||
|
| cw2-turbo-verbatim | 10.8% | **3.1%** | **66.5%** | 0 | 0.065 |
|
||||||
|
| whisper-turbo | 11.8% | 4.1% | 64.0% | 0 | 0.031 |
|
||||||
|
| cw2-large-intended | 12.3% | 3.8% | 63.5% | 0 | 0.107 |
|
||||||
|
| whisper-small | 27.5% | 9.8% | 35.0% | 0 | 0.026 |
|
||||||
|
|
||||||
|
`whisper-small` is the floor, because `ggml-small.bin` is what mavsttd loads on
|
||||||
|
homesrv today. CW2 turbo beats it by 17 points of WER and takes exact matches
|
||||||
|
from 35.0% to 65.5%.
|
||||||
|
|
||||||
|
Two results are worth naming beyond the winner. CW2 turbo beats its own base
|
||||||
|
model, whisper-large-v3-turbo, by 1.4 points. And it beats CW2 large by 1.9
|
||||||
|
points, which inverts what the card implies by calling turbo a degraded draft.
|
||||||
|
No arm returned an empty transcript.
|
||||||
|
|
||||||
|
## Intended and verbatim are closer than the mode names suggest
|
||||||
|
|
||||||
|
The two modes disagree on 70 of the 200 clips before normalization and on 29
|
||||||
|
after it. So the raw difference is mostly casing and punctuation, which
|
||||||
|
normalization removes and which Maven does not read either.
|
||||||
|
|
||||||
|
Verbatim scores worse on WER and better on CER and exact matches. The reason is
|
||||||
|
script, not disfluency:
|
||||||
|
|
||||||
|
```text
|
||||||
|
ref: футбольный матч челси брайтон
|
||||||
|
int: Футбольный матч Chelsea-Брайтон.
|
||||||
|
ver: Футбольный матч Челси Брайтон.
|
||||||
|
```
|
||||||
|
|
||||||
|
Intended writes foreign entity names in Latin script and verbatim
|
||||||
|
transliterates them. Golos references are Cyrillic throughout, so verbatim
|
||||||
|
collects the exact matches. That is a property of this corpus rather than a
|
||||||
|
quality difference.
|
||||||
|
|
||||||
|
**This corpus cannot settle the mode choice.** Golos crowd is clean short
|
||||||
|
commands with almost no disfluency. The two modes have nothing to disagree
|
||||||
|
about here. They separate on spontaneous speech with fillers, restarts and
|
||||||
|
repairs, which is what the owner speaks. Intended stays the choice for the
|
||||||
|
reason it was always the choice. Maven wants what was meant, not every stumble
|
||||||
|
on the way there.
|
||||||
|
|
||||||
|
The Latin-script habit is the one finding here that touches routing. The
|
||||||
|
routing heads were trained on Cyrillic utterances, so an entity name arriving
|
||||||
|
in Latin script is out of distribution for them. Nothing measures that yet.
|
||||||
|
|
||||||
|
## The runtime is workpc, because whisper.cpp cannot load CW2
|
||||||
|
|
||||||
|
`num_languages()` in `deps/whisper.cpp/src/whisper.cpp` derives the language
|
||||||
|
count from the vocabulary size:
|
||||||
|
|
||||||
|
```cpp
|
||||||
|
return n_vocab - 51765 - (is_multilingual() ? 1 : 0);
|
||||||
|
```
|
||||||
|
|
||||||
|
CW2 carries 31 extra tokens, so `n_vocab` is 51897 and this yields 131
|
||||||
|
languages. The derived `dt` offset becomes 33 and shifts seven special token
|
||||||
|
ids, including `token_beg` and `token_transcribe`. The architecture is
|
||||||
|
otherwise byte-identical to whisper-large-v3-turbo, and the new tokens sit
|
||||||
|
above every whisper special id.
|
||||||
|
|
||||||
|
So loading CW2 in whisper.cpp is a patch to a vendored dependency, not a port.
|
||||||
|
It was not taken, because STT is moving to workpc anyway under V-486. CW2 turbo
|
||||||
|
becomes the preferred remote and `ggml-small.bin` on homesrv stays the floor,
|
||||||
|
which is the shape `modelSeam` already uses for routing and replies. The 27.5%
|
||||||
|
floor is what a turn falls back to when the workstation is down, and this table
|
||||||
|
is what that costs.
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
Standard CW2 weights carry `nyra-health-non-commercial-research`. The Pro
|
||||||
|
variants are commercial-license only. Maven is personal and self-hosted, so the
|
||||||
|
standard weights are usable and the Pro ones are not free to take.
|
||||||
|
|
||||||
|
## What is not measured
|
||||||
|
|
||||||
|
Disfluent spontaneous speech, which is the whole reason to prefer Intended.
|
||||||
|
Long-form audio beyond 13.5s. Far-field or noisy microphones. English, which
|
||||||
|
Maven also speaks. The ONNX turbo export, which was never run, since the
|
||||||
|
transformers path already meets the latency budget at RTF 0.065.
|
||||||
@@ -0,0 +1,61 @@
|
|||||||
|
# gemma-4-E4B on the phrasing and talk fixtures
|
||||||
|
|
||||||
|
Date: 2026-08-09. Box: workpc up, E4B loaded on 8080.
|
||||||
|
`MAVEN_LLM_URL=http://192.168.1.105:8080 make eval-phrasing`.
|
||||||
|
|
||||||
|
This was the one unmeasured risk of the 2026-08-09 model swap. Routing was
|
||||||
|
measured the same day and E4B lost four destination cases to the 12B. Phrasing
|
||||||
|
was not measured at all, and phrasing is the half the owner hears.
|
||||||
|
|
||||||
|
## Result
|
||||||
|
|
||||||
|
| fixture | E4B | resident Qwen3-1.7B, 2026-08-05 |
|
||||||
|
|---|---|---|
|
||||||
|
| nudges | 15/15 (100%) | 15/15 (100%) |
|
||||||
|
| talk, passes every check | **29/36 (80.6%)** | 25/36 (69.4%) |
|
||||||
|
| lang | 36/36 | — |
|
||||||
|
| feminine | 36/36 | 36/36 |
|
||||||
|
| address | **36/36** | 33/36 |
|
||||||
|
| ontopic | 29/36 | 28/36 |
|
||||||
|
| p50 latency | **516ms** | 2.97s |
|
||||||
|
| p95 latency | 921ms | — |
|
||||||
|
| failed generations | 0 | 0 |
|
||||||
|
|
||||||
|
E4B beats the homesrv floor by four cases and answers about six times faster.
|
||||||
|
Persona is clean: `lang`, `feminine` and `address` are perfect, and `address`
|
||||||
|
is where the resident model still loses three. The 2026-08-05 measurement of the
|
||||||
|
resident model is the comparison, since both ran the same 36-case fixture.
|
||||||
|
|
||||||
|
Every failure is `ontopic`. Nothing failed on persona, nothing failed to parse.
|
||||||
|
|
||||||
|
## The score is at the ceiling, not below it
|
||||||
|
|
||||||
|
The 2026-08-05 temperature sweep found two cases that fail at every temperature
|
||||||
|
in every run: `reply-note-router` and `reply-fact-weight`. It named a defect in
|
||||||
|
the reply phrasing path rather than sampling noise. It put the fixture's ceiling
|
||||||
|
at 30/36 before persona is scored. Both cases are in E4B's failure list.
|
||||||
|
|
||||||
|
So 29/36 is one case off a ceiling nothing about the model can move. The swap is
|
||||||
|
safe on phrasing. Read this next to the routing result, not instead of it. There
|
||||||
|
E4B costs four destination cases and buys 50ms. Here it costs nothing.
|
||||||
|
|
||||||
|
## Two findings no check caught
|
||||||
|
|
||||||
|
**She says she wrote something down when she did not.** Asked what to do this
|
||||||
|
evening, E4B writes "Я записала несколько идей!". Asked for a joke, it writes
|
||||||
|
"Я записала одну забавную ситуацию!". Nothing was stored. No check scores it,
|
||||||
|
because `ontopic` reads the subject and `cringe` reads pet names. A claim to
|
||||||
|
have saved something is a claim about state, and it is wrong.
|
||||||
|
|
||||||
|
**Two `ontopic` failures look like check defects.** `know-dont-know` wants
|
||||||
|
"не зна" or "не мог". It got "Я не умею знать личную информацию о твоих
|
||||||
|
соседях", which declines correctly in words the check does not list.
|
||||||
|
`know-hiccups` is the same shape. Neither is a model failure and both count
|
||||||
|
against the score.
|
||||||
|
|
||||||
|
## Not measured here
|
||||||
|
|
||||||
|
A 12B control on the same fixture, which would need the card reloaded and is the
|
||||||
|
owner's call. The talk fixture through the daemon rather than through the
|
||||||
|
phraser directly. The CPT'd Qwen3-1.7B, which does not exist yet and is the
|
||||||
|
reason `address` is a check at all.
|
||||||
@@ -0,0 +1,51 @@
|
|||||||
|
# gemma-4-E4B against gemma-4-12B on the routing fixture
|
||||||
|
|
||||||
|
*Measured 2026-08-09 on workpc. The owner asked for the swap. This is what it costs.*
|
||||||
|
|
||||||
|
Both arms ran the same 96-case fixture through `TestLLMRouterBaseline`, minutes
|
||||||
|
apart, against the same llama-server build and the same mavgpud. The 12B arm is a
|
||||||
|
control run and not the 2026-08-02 number. That one predates five fixture cases,
|
||||||
|
the destination labels and a llama.cpp upgrade.
|
||||||
|
|
||||||
|
| | full | intent-only | destination | p50 | p95 |
|
||||||
|
|---|---|---|---|---|---|
|
||||||
|
| gemma-4-12B-it-qat-UD-Q4_K_XL, MTP draft | 81/96 (84.4%) | 91.7% | 23/33 (69.7%) | 344ms | 471ms |
|
||||||
|
| gemma-4-E4B-it-qat-UD-Q4_K_XL | 80/96 (83.3%) | 89.6% | 19/33 (57.6%) | 294ms | 562ms |
|
||||||
|
|
||||||
|
E4B costs one case of full accuracy, two of intent and **four of destination**,
|
||||||
|
and buys 50ms at p50. Read the destination column as the finding. One case is
|
||||||
|
three points on 33. So 23 against 19 is outside the noise a single case makes,
|
||||||
|
and the other two columns are not.
|
||||||
|
|
||||||
|
Both arms produce three false clarifies and one missed clarify, and neither
|
||||||
|
errored on any case.
|
||||||
|
|
||||||
|
## What E4B loses
|
||||||
|
|
||||||
|
Four of the five destination regressions are the same shape: it names nothing
|
||||||
|
where the 12B names `recall` or `calendar`. `ru-query-015` ("сколько я прошёл
|
||||||
|
шагов") goes further and names `self`. Naming nothing is the safe direction,
|
||||||
|
because `SourceUnknown` walks the whole chain, so these turns are still answered.
|
||||||
|
They cost latency and they are what a fourth head is meant to fix (V-546).
|
||||||
|
|
||||||
|
Two Russian intent cases regress, both with the interrogative off the front.
|
||||||
|
`ru-chat-003` ("расскажи анекдот про программистов") goes to `query`.
|
||||||
|
`ru-fact-003` ("поужинал") goes to `chat`.
|
||||||
|
|
||||||
|
## MTP
|
||||||
|
|
||||||
|
E4B has none, and there is no way to give it any on this box. MTP on workpc is
|
||||||
|
a separate gguf of architecture `gemma4-assistant` carrying
|
||||||
|
`nextn_predict_layers=4`, and `mtp-gemma-4-12B-it-BF16.gguf` is the only one on
|
||||||
|
disk. Its head is trained against the 12B's hidden states, so it cannot drive an
|
||||||
|
E4B target. Scanning both target ggufs finds no `nextn` tensors in either, so
|
||||||
|
neither model self-speculates.
|
||||||
|
|
||||||
|
So the 12B arm above ran with speculative decoding and E4B ran without, and E4B
|
||||||
|
was still faster.
|
||||||
|
|
||||||
|
## Cost on the card
|
||||||
|
|
||||||
|
E4B is 4.2GB against 6.7GB plus a 0.86GB draft. With CW2 resident at 1.6GB that
|
||||||
|
is 5.8GB of 16GB against 9.2GB. Nothing in Maven needs the difference, so this is
|
||||||
|
headroom for the owner's own jobs rather than a capability.
|
||||||
@@ -0,0 +1,113 @@
|
|||||||
|
# Kiwix answered the wrong question, and the fix was not a relevance gate
|
||||||
|
|
||||||
|
Date: 2026-08-09. Task: V-668. Box: homesrv, workstation off.
|
||||||
|
Book: `wikipedia_ru_all_maxi_2026-02` on `127.0.0.1:8034`.
|
||||||
|
|
||||||
|
## What started it
|
||||||
|
|
||||||
|
Two turns on 2026-08-09 came back wrong from the offline encyclopedia.
|
||||||
|
"почему небо голубое" was answered off the song "Город золотой". "что такое
|
||||||
|
TCP?" was answered off "Перехват TCP-соединения". Both were phrased
|
||||||
|
confidently, because `queryKiwix` claims a turn whenever the search returns
|
||||||
|
anything and `len(hits) == 0` is its only gate.
|
||||||
|
|
||||||
|
The plan was a relevance gate. multilingual-e5-small is asymmetric and trained
|
||||||
|
for exactly this, `query:` against `passage:`, and the query vector is already
|
||||||
|
held on the turn. The 2026-08-05 measurement that killed a search-quality gate
|
||||||
|
killed three lexical signals. It says in its own words that it never probed
|
||||||
|
Kiwix.
|
||||||
|
|
||||||
|
## The gate does not exist
|
||||||
|
|
||||||
|
Fourteen Russian questions, eight the encyclopedia can answer and six it
|
||||||
|
cannot. Each question was searched, the top article read, and the cosine of
|
||||||
|
`EmbedQuery(question)` against `EmbedPassage(article)` recorded.
|
||||||
|
|
||||||
|
| set | n | min | mean | max |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| answerable | 8 | 0.7934 | 0.8400 | 0.9087 |
|
||||||
|
| not answerable | 6 | 0.7480 | 0.7852 | 0.8367 |
|
||||||
|
|
||||||
|
Two of the six unanswerable score above the weakest answerable one. That alone
|
||||||
|
would be a poor threshold. The log killed it outright: seven of the eight
|
||||||
|
answerable questions got a **wrong** article back, and those wrong articles
|
||||||
|
scored high. The TCP hijacking article scored 0.8653, above five of the six
|
||||||
|
unanswerable rows.
|
||||||
|
|
||||||
|
The finding is that this cosine measures topic and not answerhood. A page about
|
||||||
|
hijacking TCP sessions is about TCP. No threshold separates it from a page that
|
||||||
|
defines TCP, and one that tried would take the definition with it.
|
||||||
|
|
||||||
|
## The defect is retrieval
|
||||||
|
|
||||||
|
`internal/kiwix/client.go` has said it since it was written: ranking is keyword
|
||||||
|
based, "why is the sky blue" finds a TV episode. `queryKiwix` sends the whole
|
||||||
|
sentence. The English path has a rewriter that reduces a question to keywords
|
||||||
|
with a model call. The Russian path reads the book verbatim (V-508) and had
|
||||||
|
nothing. So the question words compete with the one word that names the article.
|
||||||
|
|
||||||
|
Dropping the question words changes the answer:
|
||||||
|
|
||||||
|
| sent | first hit |
|
||||||
|
|---|---|
|
||||||
|
| `кто написал Войну и мир` | Радуйся, мир (Доктор Кто) |
|
||||||
|
| `Война и мир` | Война и мир |
|
||||||
|
| `что такое TCP` | Перехват TCP-соединения |
|
||||||
|
| `TCP` | TCP |
|
||||||
|
|
||||||
|
A ZIM is also addressable by title, which nothing here used. `/A/Франция`,
|
||||||
|
`/A/TCP` and `/A/Небо` are 200. `/A/Трюмбальная_нидроскопия` is 404. So an
|
||||||
|
exact title is safe to try first: it either answers or costs one request that
|
||||||
|
says nothing.
|
||||||
|
|
||||||
|
The title has to carry its capital. `/A/фотосинтез` is a 404 and
|
||||||
|
`/A/Фотосинтез` is a 200. The spoken form is tried first anyway, so a title
|
||||||
|
that begins lowercase on purpose keeps its chance.
|
||||||
|
|
||||||
|
## What shipped, measured
|
||||||
|
|
||||||
|
`kiwix.Topic` drops the narrative request, the interrogative and a verb sitting
|
||||||
|
behind one. It keeps everything else, because a word it cannot classify is more
|
||||||
|
likely the topic than noise. `kiwix.TitlePath` tries the exact article before
|
||||||
|
any ranking runs. Both apply on the verbatim path only, since reducing twice
|
||||||
|
would take the topic off the rewriter's input.
|
||||||
|
|
||||||
|
| question | before | after |
|
||||||
|
|---|---|---|
|
||||||
|
| что такое TCP? | Перехват TCP-соединения | **TCP** (by title) |
|
||||||
|
| что такое фотосинтез | C4-фотосинтез | **Фотосинтез** (by title) |
|
||||||
|
| кто такой Линус Торвальдс? | Tux | **Торвальдс, Линус** (by title) |
|
||||||
|
| кто написал Войну и мир | Радуйся, мир (Доктор Кто) | **Война и мир** |
|
||||||
|
| столица Франции | Список столиц Олимпийских игр | **Париж** (by title) |
|
||||||
|
| что такое чёрная дыра | Чёрная дыра | Чёрная дыра (by title) |
|
||||||
|
| почему небо голубое | Город золотой | Под небом голубым… (фильм) |
|
||||||
|
| почему трава зелёная | Сено | Зелень |
|
||||||
|
|
||||||
|
Five questions reach the right article where they did not. One was already
|
||||||
|
right and stays right. Nothing regressed.
|
||||||
|
|
||||||
|
"столица Франции" is the surprise. The 2026-08-05 measurement named it as the
|
||||||
|
case a quality gate must not break, because the answer is Париж and that word
|
||||||
|
is not in the question. The ZIM holds a title redirect, so asking for the
|
||||||
|
article titled "Столица Франции" returns Париж. Retrieval by title reaches an
|
||||||
|
answer that retrieval by keyword cannot.
|
||||||
|
|
||||||
|
## What is still wrong
|
||||||
|
|
||||||
|
Two of the eight are still not answered, and both are the same shape. The
|
||||||
|
question names no article and no redirect covers it. "почему небо голубое" is
|
||||||
|
answered by Rayleigh scattering, and nothing in the question says so. Keyword
|
||||||
|
retrieval cannot bridge that and neither can a threshold. The candidates are a
|
||||||
|
semantic index over titles, or asking the resident model for the article title
|
||||||
|
rather than for keywords.
|
||||||
|
|
||||||
|
`Response.Empty()` is still the whole gate. A wrong article that the search
|
||||||
|
does return is still spoken. What this change buys is that the article is
|
||||||
|
usually right, not that a wrong one is caught.
|
||||||
|
|
||||||
|
## Not measured here
|
||||||
|
|
||||||
|
The English path, which still goes through the rewriter and was not touched.
|
||||||
|
SearXNG, where the same question about answerhood is open and the 2026-08-05
|
||||||
|
result stands. The cascade end to end, since the workstation is off and the
|
||||||
|
phrasing arm is the resident model.
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
# silero-vad against the energy threshold in mavwaked
|
||||||
|
|
||||||
|
*Measured 2026-08-09 on homesrv. V-487, stage one of two.*
|
||||||
|
|
||||||
|
mavwaked decided an utterance had started by comparing frame energy to an
|
||||||
|
adaptive floor. That answers "is this frame loud". A fan, a door and a
|
||||||
|
television are all loud, and every utterance mavwaked accepts becomes a turn.
|
||||||
|
|
||||||
|
silero-vad answers "is this frame speech". It is 2.3MB of ONNX and it replaces
|
||||||
|
the comparison and nothing else. The speech hold, the silence hold, the length
|
||||||
|
cap and the utterance buffer are the same state machine either way.
|
||||||
|
|
||||||
|
## What it declines
|
||||||
|
|
||||||
|
Speech is the four piper fixtures `mavsttd` already scores against, so nothing
|
||||||
|
of the owner's voice is committed. Non-speech is white noise at the same RMS as the clip beside it. That is the
|
||||||
|
cheapest thing that fools an energy floor.
|
||||||
|
|
||||||
|
| clip | silero, speech frames on speech | silero on noise | energy on noise |
|
||||||
|
|---|---|---|---|
|
||||||
|
| ru_fact.wav | 59 | 0 | 68 |
|
||||||
|
| ru_query.wav | 69 | 0 | 79 |
|
||||||
|
| ru_reminder.wav | 80 | 0 | 89 |
|
||||||
|
| en_act.wav | 90 | 0 | 99 |
|
||||||
|
|
||||||
|
The energy threshold accepts every noise clip as a complete utterance. Silero
|
||||||
|
calls not one frame of any of them speech, and still hears all four spoken
|
||||||
|
clips. `TestSileroHearsSpeechAndDeclinesNoise` is that table.
|
||||||
|
|
||||||
|
White noise is a floor, not a proof. It says nothing about a television, which
|
||||||
|
is speech, or about a fan, which is narrowband. Those need room recordings and
|
||||||
|
this box has none.
|
||||||
|
|
||||||
|
## What it costs
|
||||||
|
|
||||||
|
`BenchmarkSileroFrame` on the homesrv laptop (Ryzen 5 5600U), one 30ms frame
|
||||||
|
through the model including the re-chunking:
|
||||||
|
|
||||||
|
509µs per frame
|
||||||
|
|
||||||
|
That is 1.7% of one core, on the slower of the two machines. The detector runs
|
||||||
|
on the workstation beside the microphone, never on the GPU. This number is what
|
||||||
|
says it does not need one.
|
||||||
|
|
||||||
|
## The window is 512 samples, not 480
|
||||||
|
|
||||||
|
`cmd/mavwaked/main.go` claimed the frame contract matched silero's input
|
||||||
|
exactly. That was true of silero v4. Version 5 takes exactly 512 samples at 16kHz, plus 64 samples of context from
|
||||||
|
the previous window. So `sileroVAD` buffers across capture frames, and a frame
|
||||||
|
completing no window inherits the previous probability. `TestSileroRechunksAcrossFrames` pins it.
|
||||||
|
|
||||||
|
## Still an energy gate by default
|
||||||
|
|
||||||
|
`-vad-model` is empty in the code default, so a deployment that does not pass
|
||||||
|
it runs exactly what shipped before. Barge-in is untouched and deliberately so. It reads frame energy while she is
|
||||||
|
speaking, which is a different question from whether the frame is speech.
|
||||||
|
|
||||||
|
## Not done here
|
||||||
|
|
||||||
|
The wake word. This is stage one of the two V-487 asks for. The second needs a
|
||||||
|
keyword model that does not exist yet. The pretrained openWakeWord keywords are
|
||||||
|
English, and a Russian one has to be trained. Until then anything spoken near
|
||||||
|
the microphone still becomes a turn. It is now merely required to be speech.
|
||||||
+38
-3
@@ -1,6 +1,6 @@
|
|||||||
# Offloading model work to the workstation
|
# Offloading model work to the workstation
|
||||||
|
|
||||||
*Last verified: 2026-08-05 @ b789676. Living doc: correct it in place, do not append.*
|
*Last verified: 2026-08-09 @ 50c6637. Living doc: correct it in place, do not append.*
|
||||||
|
|
||||||
Owner's call, 2026-08-02. Vikunja #483 is the umbrella. Tasks #484 to #487 are the
|
Owner's call, 2026-08-02. Vikunja #483 is the umbrella. Tasks #484 to #487 are the
|
||||||
work, and this file holds the shape and the rules all four must obey.
|
work, and this file holds the shape and the rules all four must obey.
|
||||||
@@ -105,6 +105,17 @@ how we find out whether the blind spot is real.
|
|||||||
untouched. The model, the context size, the layer count and the MTP flags are the
|
untouched. The model, the context size, the layer count and the MTP flags are the
|
||||||
owner's business and not this daemon's schema.
|
owner's business and not this daemon's schema.
|
||||||
|
|
||||||
|
**Every GPU service on that box belongs under this supervisor**, added to
|
||||||
|
`cmd/mavgpud` rather than to systemd beside it. The rule was learned on
|
||||||
|
2026-08-09. The CW2 transcriber ran as its own user unit and registered on the
|
||||||
|
KFD like any ROCm job. So the supervisor read its own transcriber as a
|
||||||
|
contender. It yielded the card every few seconds and the gemma-4-12b arm was
|
||||||
|
down for eight minutes before anyone looked. So the supervisor takes a `stt`
|
||||||
|
block and starts CW2 itself. Yielding is all or nothing, because a job that
|
||||||
|
wants the card wants all of it. Idle unloading is not. It applies to
|
||||||
|
llama-server, which holds 8GB. CW2 holds 1.6GB, and unloading it would cost the
|
||||||
|
next voice turn its quality for nothing.
|
||||||
|
|
||||||
## What stays on homesrv, permanently
|
## What stays on homesrv, permanently
|
||||||
|
|
||||||
The **embedder** (multilingual-e5-small, ONNX, CPU). It backs the classifier, which
|
The **embedder** (multilingual-e5-small, ONNX, CPU). It backs the classifier, which
|
||||||
@@ -149,6 +160,29 @@ flips. It is wired anyway: `PhraseReminder` is on the same transport and is on.
|
|||||||
Then the embedder above, **whisper.cpp** in `mavsttd`, and **piper** in `mavttsd`.
|
Then the embedder above, **whisper.cpp** in `mavsttd`, and **piper** in `mavttsd`.
|
||||||
`mavwaked` uses no model at all: an energy-threshold VAD over 30ms frames.
|
`mavwaked` uses no model at all: an energy-threshold VAD over 30ms frames.
|
||||||
|
|
||||||
|
Speech-to-text is wired as of 09-08-2026, and it takes only the silent half of the
|
||||||
|
rule. A worse transcript is still a turn, so there is nothing to name a gap about
|
||||||
|
and `stt.Pair` has no `TranscribeRemote`. `sttSeam` in `cmd/mavend/voicewire.go`
|
||||||
|
builds it, beside `modelSeam` and at the same place in `wireVoice`, so the voice
|
||||||
|
path and the meeting recorder still share one transcriber.
|
||||||
|
|
||||||
|
The remote is not a second endpoint on mavgpud. whisper.cpp cannot load
|
||||||
|
CrisperWhisper 2.0 at all. It reads its language count off the vocabulary
|
||||||
|
size, and CW2's 51897 tokens shift seven special token ids. So CW2 runs under
|
||||||
|
transformers as its own service on port 8081, and `stt.HTTPTranscriber` is the
|
||||||
|
second transport for the same seam. It posts raw PCM with the format in headers.
|
||||||
|
It carries a bearer token, because audio is the most sensitive thing that
|
||||||
|
crosses here.
|
||||||
|
|
||||||
|
It is a second endpoint on nothing, but it is a second **child** of mavgpud, and
|
||||||
|
that part is not optional. See the supervisor section above for why: a ROCm
|
||||||
|
service the supervisor does not own is a contender it yields to.
|
||||||
|
|
||||||
|
The margin is the reason: CW2 turbo scores 10.4% WER in Russian against 27.5% for
|
||||||
|
the `ggml-small.bin` mavsttd loads, over 200 Golos clips
|
||||||
|
(`docs/evals/2026-08-09-crisperwhisper2-russian-wer.md`). Text-to-speech has not
|
||||||
|
moved and piper on homesrv is still the only synthesizer.
|
||||||
|
|
||||||
Speech-to-text stays two stages when it moves. One call carrying both a clip and the router
|
Speech-to-text stays two stages when it moves. One call carrying both a clip and the router
|
||||||
prompt was measured on 05-08-2026. It scores 54.2% intent-only against 84.7% for whisper on
|
prompt was measured on 05-08-2026. It scores 54.2% intent-only against 84.7% for whisper on
|
||||||
homesrv, on the same 72 cases. The model transcribes clips it then routes wrong, so a long
|
homesrv, on the same 72 cases. The model transcribes clips it then routes wrong, so a long
|
||||||
@@ -173,8 +207,9 @@ cleaner transcripts, not accuracy. See `docs/evals/2026-08-05-audio-in-routing.m
|
|||||||
which fixes what the 1.7B gets wrong: world knowledge, and the persona the CPT
|
which fixes what the 1.7B gets wrong: world knowledge, and the persona the CPT
|
||||||
targets. The degradation path is already written and measured, since the
|
targets. The degradation path is already written and measured, since the
|
||||||
classifier scores 68.8% full accuracy at p50 16.6µs on its own.
|
classifier scores 68.8% full accuracy at p50 16.6µs on its own.
|
||||||
3. **Speech-to-text and text-to-speech** (#486). They gain a real margin, but on
|
3. **Speech-to-text and text-to-speech** (#486). Speech-to-text is wired, see
|
||||||
quality alone, and both already work.
|
above. Text-to-speech is not, and piper is good enough that nothing argues
|
||||||
|
for moving it yet.
|
||||||
4. **The wake word** (#487). Independent of all of the above.
|
4. **The wake word** (#487). Independent of all of the above.
|
||||||
|
|
||||||
## Assumptions
|
## Assumptions
|
||||||
|
|||||||
@@ -37,6 +37,16 @@ type EmbedderConfig struct {
|
|||||||
ModelPath string `json:"model_path,omitempty"`
|
ModelPath string `json:"model_path,omitempty"`
|
||||||
TokenizerPath string `json:"tokenizer_path,omitempty"`
|
TokenizerPath string `json:"tokenizer_path,omitempty"`
|
||||||
LibPath string `json:"lib_path,omitempty"`
|
LibPath string `json:"lib_path,omitempty"`
|
||||||
|
|
||||||
|
// HeadsPath — the routing heads graph, which is a fine-tuned COPY of the
|
||||||
|
// model above with four linear heads on its pooled output (V-664). Empty
|
||||||
|
// means no heads, and the cascade runs exactly as it did before they
|
||||||
|
// existed. It shares LibPath and TokenizerPath, and router_heads.json is
|
||||||
|
// read from the same directory.
|
||||||
|
//
|
||||||
|
// It must never be pointed at ModelPath. Memory recall depends on the
|
||||||
|
// resident copy scoring what it scored, and the fine-tuned one does not.
|
||||||
|
HeadsPath string `json:"heads_path,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// WeatherConfig configures the weather provider for voice queries.
|
// WeatherConfig configures the weather provider for voice queries.
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
package config
|
package config
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"net/url"
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
@@ -35,12 +36,54 @@ type WorkstationConfig struct {
|
|||||||
// 0 ⇒ DefaultWorkstationTimeout. A big model on a LAN host is slower than
|
// 0 ⇒ DefaultWorkstationTimeout. A big model on a LAN host is slower than
|
||||||
// the resident one, and a request that overruns falls back to the floor.
|
// the resident one, and a request that overruns falls back to the floor.
|
||||||
Timeout Duration `json:"timeout,omitempty"`
|
Timeout Duration `json:"timeout,omitempty"`
|
||||||
|
|
||||||
|
// Stt — CrisperWhisper 2.0 on the same machine, a separate service on its
|
||||||
|
// own port. Absent ⇒ every utterance goes to mavsttd, which is today.
|
||||||
|
Stt *WorkstationSttConfig `json:"stt,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// WorkstationSttConfig — speech-to-text on the workstation.
|
||||||
|
//
|
||||||
|
// It is a second service and not a second endpoint on mavgpud: whisper.cpp
|
||||||
|
// cannot load CrisperWhisper 2.0 at all, because it derives its language count
|
||||||
|
// from the vocabulary size and CW2's 51897 tokens shift seven special token
|
||||||
|
// ids. So CW2 runs under transformers, and this block addresses it.
|
||||||
|
//
|
||||||
|
// Worth the trouble: CW2 turbo scores 10.4% WER in Russian against 27.5% for
|
||||||
|
// the ggml-small.bin homesrv loads
|
||||||
|
// (docs/evals/2026-08-09-crisperwhisper2-russian-wer.md).
|
||||||
|
type WorkstationSttConfig struct {
|
||||||
|
// URL — the transcribe endpoint, e.g.
|
||||||
|
// "http://192.168.1.105:8081/transcribe". Empty ⇒ the block is normalised
|
||||||
|
// to nil and mavsttd takes every turn.
|
||||||
|
URL string `json:"url,omitempty"`
|
||||||
|
|
||||||
|
// Health — the admission endpoint. Empty ⇒ the URL's origin + "/health".
|
||||||
|
// It answers 503 while the card is held, and that is the signal.
|
||||||
|
Health string `json:"health,omitempty"`
|
||||||
|
|
||||||
|
// Token — the bearer token the service checks. Audio is the most sensitive
|
||||||
|
// thing that crosses this seam, so a LAN deployment should set one. Write
|
||||||
|
// it as ${MAVEN_STT_TOKEN} and keep the value in deploy/telegram.env, the
|
||||||
|
// way every other secret in this file is written.
|
||||||
|
Token string `json:"token,omitempty"`
|
||||||
|
|
||||||
|
// Probe — how often admission is re-checked. 0 ⇒ DefaultWorkstationProbe.
|
||||||
|
Probe Duration `json:"probe,omitempty"`
|
||||||
|
|
||||||
|
// Timeout — the per-request budget for one utterance. 0 ⇒
|
||||||
|
// DefaultWorkstationSttTimeout. A request that overruns falls back to
|
||||||
|
// mavsttd, which costs a worse transcript and not the turn.
|
||||||
|
Timeout Duration `json:"timeout,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// Workstation defaults, applied in normaliseWorkstation.
|
// Workstation defaults, applied in normaliseWorkstation.
|
||||||
const (
|
const (
|
||||||
DefaultWorkstationProbe = 15 * time.Second
|
DefaultWorkstationProbe = 15 * time.Second
|
||||||
DefaultWorkstationTimeout = 90 * time.Second
|
DefaultWorkstationTimeout = 90 * time.Second
|
||||||
|
// One utterance, not one completion. A voice turn waits on this, so the
|
||||||
|
// budget is a few seconds and not a minute and a half.
|
||||||
|
DefaultWorkstationSttTimeout = 10 * time.Second
|
||||||
)
|
)
|
||||||
|
|
||||||
// normaliseWorkstation applies the block's defaults. No address, no preferred
|
// normaliseWorkstation applies the block's defaults. No address, no preferred
|
||||||
@@ -63,4 +106,36 @@ func (c *Config) normaliseWorkstation() {
|
|||||||
if w.Timeout <= 0 {
|
if w.Timeout <= 0 {
|
||||||
w.Timeout = Duration(DefaultWorkstationTimeout)
|
w.Timeout = Duration(DefaultWorkstationTimeout)
|
||||||
}
|
}
|
||||||
|
normaliseWorkstationStt(w)
|
||||||
|
}
|
||||||
|
|
||||||
|
// normaliseWorkstationStt applies the speech-to-text block's defaults. No
|
||||||
|
// address, no remote: mavsttd then takes every utterance, which is today.
|
||||||
|
func normaliseWorkstationStt(w *WorkstationConfig) {
|
||||||
|
if w.Stt != nil && strings.TrimSpace(w.Stt.URL) == "" {
|
||||||
|
w.Stt = nil
|
||||||
|
}
|
||||||
|
if w.Stt == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s := w.Stt
|
||||||
|
if strings.TrimSpace(s.Health) == "" {
|
||||||
|
s.Health = healthOrigin(s.URL)
|
||||||
|
}
|
||||||
|
if s.Probe <= 0 {
|
||||||
|
s.Probe = Duration(DefaultWorkstationProbe)
|
||||||
|
}
|
||||||
|
if s.Timeout <= 0 {
|
||||||
|
s.Timeout = Duration(DefaultWorkstationSttTimeout)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// healthOrigin derives the admission endpoint from the transcribe endpoint.
|
||||||
|
// The URL names a path, so appending to it would ask for /transcribe/health.
|
||||||
|
func healthOrigin(raw string) string {
|
||||||
|
u, err := url.Parse(raw)
|
||||||
|
if err != nil || u.Host == "" {
|
||||||
|
return strings.TrimRight(raw, "/") + "/health"
|
||||||
|
}
|
||||||
|
return u.Scheme + "://" + u.Host + "/health"
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -127,6 +127,22 @@ func (c *Client) Article(ctx context.Context, path string, maxRunes int) (crawl.
|
|||||||
return crawl.Extract(u, body, maxRunes), nil
|
return crawl.Extract(u, body, maxRunes), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TitlePath is the article path for an exact title, for Article to fetch.
|
||||||
|
//
|
||||||
|
// It exists because a ZIM is addressable by title and the full-text index is
|
||||||
|
// not the only way in. "Франция", "TCP" and "Небо" resolve; "Трюмбальная
|
||||||
|
// нидроскопия" is a 404, which is the honest answer and the reason this is
|
||||||
|
// safe to try first. Measured on 2026-08-09, keyword search on the same terms
|
||||||
|
// returns "Список пэров Франции" and "Список портов TCP и UDP" instead.
|
||||||
|
//
|
||||||
|
// A miss is normal rather than a failure. An article whose title inverts a name
|
||||||
|
// ("Торвальдс, Линус") is a 404 here and the first hit in search, so the caller
|
||||||
|
// falls through and loses nothing.
|
||||||
|
func TitlePath(book, title string) string {
|
||||||
|
t := strings.ReplaceAll(strings.TrimSpace(title), " ", "_")
|
||||||
|
return "/content/" + url.PathEscape(book) + "/A/" + url.PathEscape(t)
|
||||||
|
}
|
||||||
|
|
||||||
// rss mirrors just the bits of the RSS 2.0 reply we use.
|
// rss mirrors just the bits of the RSS 2.0 reply we use.
|
||||||
type rss struct {
|
type rss struct {
|
||||||
Items []struct {
|
Items []struct {
|
||||||
|
|||||||
@@ -0,0 +1,69 @@
|
|||||||
|
package kiwix
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"os"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TestLiveTopicBeatsTheSentence — the measurement V-668 turned on, kept as a
|
||||||
|
// test so the claim can be re-run rather than believed.
|
||||||
|
//
|
||||||
|
// It prints the article the old path returned and the article the new one
|
||||||
|
// returns, for the same question. It asserts nothing about which is better,
|
||||||
|
// because "is this the right article" is a human's call. It fails only if the
|
||||||
|
// two paths agree on every case, which would mean the change does nothing.
|
||||||
|
//
|
||||||
|
// MAVEN_KIWIX_URL=http://127.0.0.1:8034 make t PKG=./internal/kiwix/ RUN=TestLive V=1
|
||||||
|
func TestLiveTopicBeatsTheSentence(t *testing.T) {
|
||||||
|
base := os.Getenv("MAVEN_KIWIX_URL")
|
||||||
|
if base == "" {
|
||||||
|
t.Skip("MAVEN_KIWIX_URL unset — point it at the kiwix-server host port")
|
||||||
|
}
|
||||||
|
const book = "wikipedia_ru_all_maxi_2026-02"
|
||||||
|
c := New(base)
|
||||||
|
questions := []string{
|
||||||
|
"что такое TCP?",
|
||||||
|
"что такое фотосинтез",
|
||||||
|
"кто такой Линус Торвальдс?",
|
||||||
|
"кто написал Войну и мир",
|
||||||
|
"что такое чёрная дыра",
|
||||||
|
"почему небо голубое",
|
||||||
|
"почему трава зелёная",
|
||||||
|
"столица Франции",
|
||||||
|
}
|
||||||
|
moved := 0
|
||||||
|
for _, q := range questions {
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
|
||||||
|
before := firstTitle(ctx, c, q, book)
|
||||||
|
topic := Topic(q)
|
||||||
|
after := ""
|
||||||
|
for _, cand := range TitleCandidates(topic) {
|
||||||
|
if page, err := c.Article(ctx, TitlePath(book, cand), 400); err == nil && page.Text != "" {
|
||||||
|
after = page.Title + " (by title)"
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if after == "" {
|
||||||
|
after = firstTitle(ctx, c, topic, book)
|
||||||
|
}
|
||||||
|
cancel()
|
||||||
|
if before != after {
|
||||||
|
moved++
|
||||||
|
}
|
||||||
|
t.Logf("%-30s before=%-34q after=%q", q, before, after)
|
||||||
|
}
|
||||||
|
t.Logf("%d of %d questions reach a different article", moved, len(questions))
|
||||||
|
if moved == 0 {
|
||||||
|
t.Error("the topic path returns exactly what the sentence path returned")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func firstTitle(ctx context.Context, c *Client, pattern, book string) string {
|
||||||
|
hits, err := c.Search(ctx, pattern, book, 3)
|
||||||
|
if err != nil || len(hits) == 0 {
|
||||||
|
return "(nothing)"
|
||||||
|
}
|
||||||
|
return hits[0].Title
|
||||||
|
}
|
||||||
@@ -0,0 +1,118 @@
|
|||||||
|
package kiwix
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"unicode"
|
||||||
|
|
||||||
|
"github.com/kami/maven/internal/lexicon"
|
||||||
|
"github.com/kami/maven/internal/morph"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Topic reduces a question to the thing it is about, because Kiwix ranks by
|
||||||
|
// keyword overlap and a whole sentence buries the keyword that matters.
|
||||||
|
//
|
||||||
|
// This package's own doc says it: "why is the sky blue" finds a TV episode.
|
||||||
|
// Measured against the Russian ZIM on 2026-08-09, the sentence and the topic
|
||||||
|
// return different articles for the same question. "кто написал Войну и мир"
|
||||||
|
// returns "Радуйся, мир (Доктор Кто)"; "Войну и мир" returns the novel first.
|
||||||
|
// "что такое TCP" returns "Перехват TCP-соединения"; "TCP" returns TCP. The
|
||||||
|
// English path had a rewriter doing this with a model call. The Russian path
|
||||||
|
// reads the book verbatim (V-508) and had nothing.
|
||||||
|
//
|
||||||
|
// It drops three things off the front and stops: the narrative request, the
|
||||||
|
// interrogative, and a verb sitting between them and the noun. Everything else
|
||||||
|
// is kept, because a word this cannot classify is more likely the topic than
|
||||||
|
// noise. An empty return means the utterance was question words alone, and the
|
||||||
|
// caller searches the sentence as before.
|
||||||
|
func Topic(utterance string) string {
|
||||||
|
words := strings.Fields(strings.TrimSpace(utterance))
|
||||||
|
cut := 0
|
||||||
|
for cut < len(words) {
|
||||||
|
w := strings.Trim(strings.ToLower(words[cut]), ".,!?…:;\"'«»")
|
||||||
|
if w == "" {
|
||||||
|
cut++
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case inList(lexicon.NarrativeRequests(), w),
|
||||||
|
inList(lexicon.Interrogatives(), w),
|
||||||
|
inList(lexicon.FirstPerson(), w),
|
||||||
|
// "что ТАКОЕ x", "кто ТАКОЙ x" — the copula that only ever follows
|
||||||
|
// an interrogative, and never a topic on its own.
|
||||||
|
cut > 0 && isCopula(w),
|
||||||
|
// "расскажи ПРО x", "о x". One-letter and two-letter prepositions
|
||||||
|
// are not a closed class worth a lexicon set of their own.
|
||||||
|
cut > 0 && isLeadingPreposition(w),
|
||||||
|
// "кто НАПИСАЛ Войну и мир". A verb here is the question's own
|
||||||
|
// verb, not part of the title. Only after something was already
|
||||||
|
// dropped, so "написал отчёт" as a topic survives intact.
|
||||||
|
cut > 0 && morph.IsVerbForm(w):
|
||||||
|
cut++
|
||||||
|
default:
|
||||||
|
// The question mark is the sentence's, not the title's, and Kiwix
|
||||||
|
// carries it into the keyword match.
|
||||||
|
topic := strings.TrimRight(strings.Join(words[cut:], " "), " .,!?…:;\"'«»")
|
||||||
|
if !hasLetter(topic) {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return topic
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// TitleCandidates is the topic as it might be titled, best first.
|
||||||
|
//
|
||||||
|
// A ZIM title is capitalized and the utterance is not: measured on 2026-08-09,
|
||||||
|
// `/A/фотосинтез` is a 404 and `/A/Фотосинтез` is a 200. The spoken form is
|
||||||
|
// tried first anyway, because a title that begins lowercase on purpose
|
||||||
|
// ("iPhone") would not survive capitalizing it. Both are one request each
|
||||||
|
// against a server on the same box, and a miss is a 404 rather than a wrong
|
||||||
|
// article.
|
||||||
|
func TitleCandidates(topic string) []string {
|
||||||
|
if topic == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
r := []rune(topic)
|
||||||
|
up := unicode.ToUpper(r[0])
|
||||||
|
if up == r[0] {
|
||||||
|
return []string{topic}
|
||||||
|
}
|
||||||
|
return []string{topic, string(up) + string(r[1:])}
|
||||||
|
}
|
||||||
|
|
||||||
|
func isCopula(w string) bool {
|
||||||
|
switch w {
|
||||||
|
case "такое", "такой", "такая", "такие", "is", "are", "was", "were":
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
func isLeadingPreposition(w string) bool {
|
||||||
|
switch w {
|
||||||
|
case "про", "о", "об", "обо", "по", "about", "of", "on":
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
func inList(list []string, w string) bool {
|
||||||
|
for _, x := range list {
|
||||||
|
if x == w {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// hasLetter is the guard against a topic that reduced to punctuation or digits
|
||||||
|
// alone, which no ZIM title matches.
|
||||||
|
func hasLetter(s string) bool {
|
||||||
|
for _, r := range s {
|
||||||
|
if unicode.IsLetter(r) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
@@ -0,0 +1,67 @@
|
|||||||
|
package kiwix
|
||||||
|
|
||||||
|
import "testing"
|
||||||
|
|
||||||
|
// The cases the 2026-08-09 measurement turned on, plus the ones a topic must
|
||||||
|
// not damage. Each left column returned a wrong article when it was sent whole.
|
||||||
|
func TestTopicKeepsTheThingTheQuestionIsAbout(t *testing.T) {
|
||||||
|
cases := []struct{ utterance, want string }{
|
||||||
|
{"что такое TCP?", "TCP"},
|
||||||
|
{"что такое фотосинтез", "фотосинтез"},
|
||||||
|
{"кто такой Линус Торвальдс?", "Линус Торвальдс"},
|
||||||
|
{"кто написал Войну и мир", "Войну и мир"},
|
||||||
|
{"расскажи про битву при Ватерлоо", "битву при Ватерлоо"},
|
||||||
|
{"what is photosynthesis", "photosynthesis"},
|
||||||
|
// No question word, so there is nothing to drop. The topic is the
|
||||||
|
// whole utterance and the search is what it was before.
|
||||||
|
{"столица Франции", "столица Франции"},
|
||||||
|
{"почему небо голубое", "небо голубое"},
|
||||||
|
}
|
||||||
|
for _, c := range cases {
|
||||||
|
if got := Topic(c.utterance); got != c.want {
|
||||||
|
t.Errorf("Topic(%q) = %q, want %q", c.utterance, got, c.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A verb only goes when a question word already went. Otherwise "написал
|
||||||
|
// отчёт" loses the verb that names what he means.
|
||||||
|
func TestTopicDropsAVerbOnlyBehindAQuestionWord(t *testing.T) {
|
||||||
|
if got := Topic("написал отчёт"); got != "написал отчёт" {
|
||||||
|
t.Errorf("Topic dropped a leading verb with no question word: %q", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Question words alone reduce to nothing, and the caller reads that as "no
|
||||||
|
// topic" and searches the sentence rather than searching the empty string.
|
||||||
|
func TestTopicIsEmptyWhenNothingIsLeft(t *testing.T) {
|
||||||
|
for _, q := range []string{"что такое?", "кто?", "почему", "???"} {
|
||||||
|
if got := Topic(q); got != "" {
|
||||||
|
t.Errorf("Topic(%q) = %q, want empty", q, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTitlePathEscapesAndUnderscores(t *testing.T) {
|
||||||
|
got := TitlePath("wikipedia_ru_all_maxi_2026-02", "Чёрная дыра")
|
||||||
|
want := "/content/wikipedia_ru_all_maxi_2026-02/A/%D0%A7%D1%91%D1%80%D0%BD%D0%B0%D1%8F_%D0%B4%D1%8B%D1%80%D0%B0"
|
||||||
|
if got != want {
|
||||||
|
t.Errorf("TitlePath = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A ZIM title carries a leading capital and the utterance does not. The spoken
|
||||||
|
// form is still tried first, so a title that begins lowercase on purpose keeps
|
||||||
|
// its chance.
|
||||||
|
func TestTitleCandidatesTryTheSpokenFormFirst(t *testing.T) {
|
||||||
|
got := TitleCandidates("фотосинтез")
|
||||||
|
if len(got) != 2 || got[0] != "фотосинтез" || got[1] != "Фотосинтез" {
|
||||||
|
t.Errorf("TitleCandidates = %q", got)
|
||||||
|
}
|
||||||
|
if got := TitleCandidates("TCP"); len(got) != 1 || got[0] != "TCP" {
|
||||||
|
t.Errorf("an already-capital topic was tried twice: %q", got)
|
||||||
|
}
|
||||||
|
if got := TitleCandidates(""); got != nil {
|
||||||
|
t.Errorf("TitleCandidates(\"\") = %q, want nil", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -14,12 +14,15 @@ import (
|
|||||||
"github.com/kami/maven/internal/decision"
|
"github.com/kami/maven/internal/decision"
|
||||||
)
|
)
|
||||||
|
|
||||||
// The two routing engines, named as claimants. They are one stage and not two,
|
// The three routing engines, named as claimants. The model and the classifier
|
||||||
// because only one of them ever runs: the classifier is reached when the model
|
// are one stage and not two, because only one of them ever runs: the classifier
|
||||||
// is absent or errored, never alongside it.
|
// is reached when the model is absent or errored, never alongside it. The heads
|
||||||
|
// run before both and decline on low confidence, so they can appear beside
|
||||||
|
// either one in a record.
|
||||||
const (
|
const (
|
||||||
claimantLLM = "llm-router"
|
claimantLLM = "llm-router"
|
||||||
claimantClassifier = "classifier"
|
claimantClassifier = "classifier"
|
||||||
|
claimantHeads = "routing-heads"
|
||||||
)
|
)
|
||||||
|
|
||||||
// thinReason names which arm of gateLLMDecision cut the confidence. The gate
|
// thinReason names which arm of gateLLMDecision cut the confidence. The gate
|
||||||
|
|||||||
@@ -1,10 +1,13 @@
|
|||||||
package router
|
package router
|
||||||
|
|
||||||
import "testing"
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
func TestEmbedderIDFromModelPath(t *testing.T) {
|
func TestEmbedderIDFromModelPath(t *testing.T) {
|
||||||
got := modelIDFromPath("/opt/maven/models/embedder/multilingual-e5-small.onnx")
|
got := modelIDFromPath("/opt/maven/models/embedder/multilingual-e5-small.onnx")
|
||||||
if got != "multilingual-e5-small@384" {
|
if got != "multilingual-e5-small@384/tok2" {
|
||||||
t.Fatalf("modelIDFromPath = %q", got)
|
t.Fatalf("modelIDFromPath = %q", got)
|
||||||
}
|
}
|
||||||
// A different model file must produce a different id, even at 384 dim.
|
// A different model file must produce a different id, even at 384 dim.
|
||||||
@@ -12,6 +15,13 @@ func TestEmbedderIDFromModelPath(t *testing.T) {
|
|||||||
if old == got {
|
if old == got {
|
||||||
t.Fatal("two different models share one id")
|
t.Fatal("two different models share one id")
|
||||||
}
|
}
|
||||||
|
// The tokenizer is half of what makes a vector, and it changes under a
|
||||||
|
// model file whose name never moves (V-664). An id that ignored it would
|
||||||
|
// leave stored passages in one space and every new query in another, with
|
||||||
|
// nothing to trigger the re-embed.
|
||||||
|
if !strings.Contains(got, "/tok") {
|
||||||
|
t.Fatalf("id %q does not name the tokenizer revision", got)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestEmbedderIDIncludesDim(t *testing.T) {
|
func TestEmbedderIDIncludesDim(t *testing.T) {
|
||||||
|
|||||||
@@ -0,0 +1,86 @@
|
|||||||
|
package eval
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/kami/maven/internal/config"
|
||||||
|
"github.com/kami/maven/internal/router"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TestONNXRoutingHeads — the cascade with the routing heads wired, which is
|
||||||
|
// what V-664 deploys. Opt-in via MAVEN_ONNX_LIB, same as TestONNXBaseline, and
|
||||||
|
// one TestONNX* per process.
|
||||||
|
//
|
||||||
|
// The comparison worth reading is against TestONNXBaseline, which is the same
|
||||||
|
// cascade with the same grammars and the same classifier floor and no heads.
|
||||||
|
// Only the middle arm varies.
|
||||||
|
//
|
||||||
|
// It also checks the Go unigram tokenizer against the Python one, because the
|
||||||
|
// heads were trained through transformers and are read through a hand-written
|
||||||
|
// tokenizer. A mismatch shows up here as a score below what Python measured on
|
||||||
|
// the same weights, and nowhere else.
|
||||||
|
func TestONNXRoutingHeads(t *testing.T) {
|
||||||
|
lib := os.Getenv("MAVEN_ONNX_LIB")
|
||||||
|
if lib == "" {
|
||||||
|
t.Skip("MAVEN_ONNX_LIB unset — see AGENTS.md § Embedder model for intent routing")
|
||||||
|
}
|
||||||
|
// Absolute, because onnxruntime resolves a graph's external weights file
|
||||||
|
// against the model path it was given, and a relative one lands in the
|
||||||
|
// test's working directory.
|
||||||
|
root, err := filepath.Abs("../../..")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
model := filepath.Join(root, "models/embedder/multilingual-e5-small/model_quantized.onnx")
|
||||||
|
tok := filepath.Join(root, "models/embedder/multilingual-e5-small/tokenizer.json")
|
||||||
|
heads := filepath.Join(root, "models/embedder/router-heads/router_heads.onnx")
|
||||||
|
for _, p := range []string{lib, model, tok, heads} {
|
||||||
|
if _, err := os.Stat(p); err != nil {
|
||||||
|
t.Skipf("missing %s: %v", p, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
emb, err2 := router.NewONNXEmbedder(model, tok, lib)
|
||||||
|
if err2 != nil {
|
||||||
|
t.Skipf("onnx embedder unavailable: %v", err2)
|
||||||
|
}
|
||||||
|
err = nil
|
||||||
|
defer emb.Close()
|
||||||
|
|
||||||
|
h, err := router.NewRouterHeads(heads, tok)
|
||||||
|
if err != nil {
|
||||||
|
t.Skipf("routing heads unavailable: %v", err)
|
||||||
|
}
|
||||||
|
defer h.Close()
|
||||||
|
|
||||||
|
f, err := Load()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Load: %v", err)
|
||||||
|
}
|
||||||
|
rep, err := Score(context.Background(), "heads+classifier", withHeads(t, emb, h), f)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Score: %v", err)
|
||||||
|
}
|
||||||
|
t.Log("\n" + rep.String() + rep.Failures())
|
||||||
|
}
|
||||||
|
|
||||||
|
// withHeads mirrors newBaselineRouter and adds the one arm under test. It is a
|
||||||
|
// separate function rather than a parameter so the baseline's signature stays
|
||||||
|
// the shape every other test calls it with.
|
||||||
|
func withHeads(t *testing.T, emb router.Embedder, h *router.RouterHeads) *router.Router {
|
||||||
|
t.Helper()
|
||||||
|
acts := router.DefaultActMatcher{Fns: actFns}
|
||||||
|
return router.New(router.Config{
|
||||||
|
Grammars: baselineGrammars(acts),
|
||||||
|
Classifier: newBaselineClassifier(t, emb),
|
||||||
|
Extractor: router.Extractor{
|
||||||
|
Time: router.StubDateTimeParser{},
|
||||||
|
Acts: acts,
|
||||||
|
Facts: router.DefaultFactParser{},
|
||||||
|
},
|
||||||
|
Threshold: config.DefaultRouterThreshold,
|
||||||
|
Heads: h,
|
||||||
|
})
|
||||||
|
}
|
||||||
@@ -0,0 +1,243 @@
|
|||||||
|
package router
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"math"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
|
||||||
|
ort "github.com/yalue/onnxruntime_go"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The routing heads (V-546, V-661, V-664). Four linear heads over one masked
|
||||||
|
// mean pool of a fine-tuned copy of multilingual-e5-small: intent,
|
||||||
|
// destination, BIO slot tags and clarify. Trained on workpc, exported to ONNX,
|
||||||
|
// and read here.
|
||||||
|
//
|
||||||
|
// Why this is not the classifier. The classifier compares one utterance to
|
||||||
|
// frozen seed phrases by cosine. A head is a softmax over the label set, so it
|
||||||
|
// cannot name a value that does not exist, and its max is a calibratable
|
||||||
|
// confidence where Confidence: 1.0 was a hardcode.
|
||||||
|
//
|
||||||
|
// Why it is not the resident model either. It answers in single-digit
|
||||||
|
// milliseconds against the model's p50 of 1.19s, and it names a destination
|
||||||
|
// the classifier arm never names at all.
|
||||||
|
//
|
||||||
|
// The body is a COPY of the embedder weights, fine-tuned. It must never
|
||||||
|
// replace models/embedder/multilingual-e5-small — memory recall depends on
|
||||||
|
// that file scoring what it scored.
|
||||||
|
//
|
||||||
|
// The slot head is exported and deliberately not read. Slots already come from
|
||||||
|
// the stage-2 extractor, and mapping BIO tags back to text needs character
|
||||||
|
// offsets the unigram tokenizer does not keep. Reading it is separate work.
|
||||||
|
const (
|
||||||
|
// headsSeq — the sequence length the heads were trained at. Padding is
|
||||||
|
// masked out of both attention and the pool, so this changes nothing but
|
||||||
|
// truncation, and truncation is what training did at 64.
|
||||||
|
headsSeq = 64
|
||||||
|
|
||||||
|
// headsThreshold — max softmax over the intent head, below which the heads
|
||||||
|
// decline and the cascade carries on to the resident model.
|
||||||
|
//
|
||||||
|
// 0.6 is the knee measured on the 88-case intent fixture
|
||||||
|
// (docs/evals/2026-08-08-routing-heads-in-go.md). It keeps 81 of 88 cases
|
||||||
|
// at 97.5% accuracy. Every higher value up to 0.9 drops right answers and
|
||||||
|
// keeps the same two wrong ones, so it buys nothing.
|
||||||
|
headsThreshold = 0.6
|
||||||
|
)
|
||||||
|
|
||||||
|
// RouterHeads runs the exported graph. Nil is a working value everywhere: a
|
||||||
|
// deployment with no weights file routes exactly as it did before this
|
||||||
|
// existed.
|
||||||
|
type RouterHeads struct {
|
||||||
|
tokenizer *unigramTokenizer
|
||||||
|
session *ort.DynamicSession[int64, float32]
|
||||||
|
intents []Intent
|
||||||
|
sources []Source
|
||||||
|
threshold float64
|
||||||
|
}
|
||||||
|
|
||||||
|
// headsMeta — router_heads.json, written beside the weights by the exporter.
|
||||||
|
// The label order is the head's output order and cannot be inferred from Go.
|
||||||
|
type headsMeta struct {
|
||||||
|
Intents []string `json:"intents"`
|
||||||
|
Sources []string `json:"sources"`
|
||||||
|
Prefix string `json:"prefix"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewRouterHeads loads the graph and its label order. modelPath points at the
|
||||||
|
// .onnx; the external weights and router_heads.json sit beside it.
|
||||||
|
//
|
||||||
|
// It assumes the ONNX environment is already initialised, because the embedder
|
||||||
|
// does that at startup and the runtime allows it once.
|
||||||
|
func NewRouterHeads(modelPath, tokenizerPath string) (*RouterHeads, error) {
|
||||||
|
metaPath := filepath.Join(filepath.Dir(modelPath), "router_heads.json")
|
||||||
|
raw, err := os.ReadFile(metaPath)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("heads: read %s: %w", metaPath, err)
|
||||||
|
}
|
||||||
|
var meta headsMeta
|
||||||
|
if err := json.Unmarshal(raw, &meta); err != nil {
|
||||||
|
return nil, fmt.Errorf("heads: parse %s: %w", metaPath, err)
|
||||||
|
}
|
||||||
|
if meta.Prefix != queryPrefix {
|
||||||
|
return nil, fmt.Errorf("heads: trained with prefix %q, this build uses %q",
|
||||||
|
meta.Prefix, queryPrefix)
|
||||||
|
}
|
||||||
|
|
||||||
|
intents := make([]Intent, len(meta.Intents))
|
||||||
|
for i, s := range meta.Intents {
|
||||||
|
intents[i] = Intent(s)
|
||||||
|
}
|
||||||
|
sources := make([]Source, len(meta.Sources))
|
||||||
|
for i, s := range meta.Sources {
|
||||||
|
// SourceUnknown is not in Sources, because it is the absence of a
|
||||||
|
// choice. It is a class the head can emit, and the one it should emit
|
||||||
|
// often, so it is allowed here and nowhere else.
|
||||||
|
if s != string(SourceUnknown) && !ValidSource(Source(s)) {
|
||||||
|
return nil, fmt.Errorf("heads: unknown destination %q in %s", s, metaPath)
|
||||||
|
}
|
||||||
|
sources[i] = Source(s)
|
||||||
|
}
|
||||||
|
|
||||||
|
tok, err := newUnigramTokenizer(tokenizerPath)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("heads: tokenizer: %w", err)
|
||||||
|
}
|
||||||
|
session, err := ort.NewDynamicSession[int64, float32](
|
||||||
|
modelPath,
|
||||||
|
[]string{"input_ids", "attention_mask"},
|
||||||
|
[]string{"intent", "source", "slots", "clarify"},
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("heads: create session: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return &RouterHeads{
|
||||||
|
tokenizer: tok,
|
||||||
|
session: session,
|
||||||
|
intents: intents,
|
||||||
|
sources: sources,
|
||||||
|
threshold: headsThreshold,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (h *RouterHeads) Close() error {
|
||||||
|
if h == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
h.session.Destroy()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// headsResult — one forward pass, read back.
|
||||||
|
type headsResult struct {
|
||||||
|
Intent Intent
|
||||||
|
Source Source
|
||||||
|
Confidence float64
|
||||||
|
Clarify bool
|
||||||
|
}
|
||||||
|
|
||||||
|
// Route runs the heads and reports whether they are confident enough to answer.
|
||||||
|
// A false second return is a decline, not an error: the cascade goes on to the
|
||||||
|
// resident model, which is what happens today.
|
||||||
|
func (h *RouterHeads) Route(ctx context.Context, utterance string) (headsResult, bool, error) {
|
||||||
|
if h == nil {
|
||||||
|
return headsResult{}, false, nil
|
||||||
|
}
|
||||||
|
ids, mask, _ := h.tokenizer.Encode(queryPrefix + utterance)
|
||||||
|
ids, mask = ids[:headsSeq], mask[:headsSeq]
|
||||||
|
// The tokenizer pads and truncates to its own length, which is longer than
|
||||||
|
// this one. Cutting the tail can cut the separator with it, so put it back.
|
||||||
|
if mask[headsSeq-1] == 1 {
|
||||||
|
ids[headsSeq-1] = sepTokenID
|
||||||
|
}
|
||||||
|
|
||||||
|
shape := ort.NewShape(1, headsSeq)
|
||||||
|
idsT, err := ort.NewTensor(shape, ids)
|
||||||
|
if err != nil {
|
||||||
|
return headsResult{}, false, fmt.Errorf("heads: ids tensor: %w", err)
|
||||||
|
}
|
||||||
|
defer idsT.Destroy()
|
||||||
|
maskT, err := ort.NewTensor(shape, mask)
|
||||||
|
if err != nil {
|
||||||
|
return headsResult{}, false, fmt.Errorf("heads: mask tensor: %w", err)
|
||||||
|
}
|
||||||
|
defer maskT.Destroy()
|
||||||
|
|
||||||
|
intentT, err := ort.NewEmptyTensor[float32](ort.NewShape(1, int64(len(h.intents))))
|
||||||
|
if err != nil {
|
||||||
|
return headsResult{}, false, fmt.Errorf("heads: intent tensor: %w", err)
|
||||||
|
}
|
||||||
|
defer intentT.Destroy()
|
||||||
|
sourceT, err := ort.NewEmptyTensor[float32](ort.NewShape(1, int64(len(h.sources))))
|
||||||
|
if err != nil {
|
||||||
|
return headsResult{}, false, fmt.Errorf("heads: source tensor: %w", err)
|
||||||
|
}
|
||||||
|
defer sourceT.Destroy()
|
||||||
|
slotsT, err := ort.NewEmptyTensor[float32](ort.NewShape(1, headsSeq, int64(numBIOTags)))
|
||||||
|
if err != nil {
|
||||||
|
return headsResult{}, false, fmt.Errorf("heads: slots tensor: %w", err)
|
||||||
|
}
|
||||||
|
defer slotsT.Destroy()
|
||||||
|
clarifyT, err := ort.NewEmptyTensor[float32](ort.NewShape(1, 2))
|
||||||
|
if err != nil {
|
||||||
|
return headsResult{}, false, fmt.Errorf("heads: clarify tensor: %w", err)
|
||||||
|
}
|
||||||
|
defer clarifyT.Destroy()
|
||||||
|
|
||||||
|
if err := h.session.Run(
|
||||||
|
[]*ort.Tensor[int64]{idsT, maskT},
|
||||||
|
[]*ort.Tensor[float32]{intentT, sourceT, slotsT, clarifyT},
|
||||||
|
); err != nil {
|
||||||
|
return headsResult{}, false, fmt.Errorf("heads: run: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The graph applies its own softmax, so these are probabilities and the max
|
||||||
|
// is the same number the eval calibrated the threshold against.
|
||||||
|
i, conf := argmax(intentT.GetData())
|
||||||
|
res := headsResult{
|
||||||
|
Intent: h.intents[i],
|
||||||
|
Confidence: conf,
|
||||||
|
}
|
||||||
|
cl := clarifyT.GetData()
|
||||||
|
res.Clarify = len(cl) == 2 && cl[1] > cl[0]
|
||||||
|
|
||||||
|
// The destination head is trained on query rows and is meaningless on any
|
||||||
|
// other intent, the same way queryWalk is never reached by one.
|
||||||
|
if res.Intent == IntentQuery {
|
||||||
|
s, _ := argmax(sourceT.GetData())
|
||||||
|
res.Source = h.sources[s]
|
||||||
|
}
|
||||||
|
|
||||||
|
// The clarify head decides on its own, and it decides first. It answers a
|
||||||
|
// different question from the intent head — not which intent, but whether
|
||||||
|
// there is enough here to act on at all — so a low intent confidence is no
|
||||||
|
// reason to discard it. It is usually the same turns: "вода" reads as
|
||||||
|
// intent act at 0.23 and clarify at 0.98, and letting the intent threshold
|
||||||
|
// bury that hands the turn to the classifier, which routes it confidently
|
||||||
|
// and never asks.
|
||||||
|
if res.Clarify {
|
||||||
|
return res, true, nil
|
||||||
|
}
|
||||||
|
if conf < h.threshold {
|
||||||
|
return res, false, nil
|
||||||
|
}
|
||||||
|
return res, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// numBIOTags — O plus B- and I- for each of Maven's five slots. The head is not
|
||||||
|
// read, but the graph writes it and the output tensor has to be the right size.
|
||||||
|
const numBIOTags = 11
|
||||||
|
|
||||||
|
func argmax(v []float32) (int, float64) {
|
||||||
|
best, bestV := 0, math.Inf(-1)
|
||||||
|
for i, x := range v {
|
||||||
|
if float64(x) > bestV {
|
||||||
|
best, bestV = i, float64(x)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return best, bestV
|
||||||
|
}
|
||||||
@@ -111,6 +111,21 @@ type Decision struct {
|
|||||||
// before this field existed. See source.go for why it is twelve values.
|
// before this field existed. See source.go for why it is twelve values.
|
||||||
Source Source
|
Source Source
|
||||||
|
|
||||||
|
// SourceAnchored — a stage 0 grammar named that destination, matching a
|
||||||
|
// literal pattern to do it. Only the router sets this, and only there.
|
||||||
|
//
|
||||||
|
// It exists because one thing downstream is not reversible by evidence
|
||||||
|
// (V-666). Naming a destination normally takes guessing sources off a turn,
|
||||||
|
// and one of those is the personal boundary, which is what stops a question
|
||||||
|
// about him from reaching the world. A grammar that read "что такое X" may
|
||||||
|
// take it off. A model or a softmax may not, because a wrong destination
|
||||||
|
// there widens what leaves the box rather than costing an answer.
|
||||||
|
//
|
||||||
|
// Read Stage instead and the two decisions get coupled: stage 0 also means
|
||||||
|
// confidence 1.0 and an anchored claim band, and a later cascade change
|
||||||
|
// could make one true where the other is not.
|
||||||
|
SourceAnchored bool
|
||||||
|
|
||||||
// Continued — this decision was rebuilt from the previous turn rather
|
// Continued — this decision was rebuilt from the previous turn rather
|
||||||
// than routed, because the utterance was an ellipsis ("а завтра?").
|
// than routed, because the utterance was an ellipsis ("а завтра?").
|
||||||
// Handlers use it to know that Slots.Text is the PREVIOUS turn's topic
|
// Handlers use it to know that Slots.Text is the PREVIOUS turn's topic
|
||||||
|
|||||||
@@ -66,12 +66,19 @@ func NewONNXEmbedder(modelPath, tokenizerPath, libPath string) (*onnxEmbedder, e
|
|||||||
func (e *onnxEmbedder) Dim() int { return embedDim }
|
func (e *onnxEmbedder) Dim() int { return embedDim }
|
||||||
|
|
||||||
// ID names the loaded model for the DB marker (Vikunja #378): the model file's
|
// ID names the loaded model for the DB marker (Vikunja #378): the model file's
|
||||||
// own name plus the dimension, so pointing the config at another model changes
|
// own name, the dimension, and the tokenizer revision, so pointing the config
|
||||||
// the string on its own.
|
// at another model changes the string on its own.
|
||||||
func (e *onnxEmbedder) ID() string { return e.id }
|
func (e *onnxEmbedder) ID() string { return e.id }
|
||||||
|
|
||||||
|
// tokenizerRev — bumped whenever the tokenizer changes what it emits for the
|
||||||
|
// same text, because that changes every vector while the model file's name
|
||||||
|
// stays put. Rev 2 is the fix for the reversed word pieces (V-664): stored
|
||||||
|
// passages embedded under rev 1 no longer sit in the same space as a query
|
||||||
|
// embedded now, and ReembedAll rewrites them because this string moved.
|
||||||
|
const tokenizerRev = 2
|
||||||
|
|
||||||
// modelIDFromPath turns /opt/.../multilingual-e5-small.onnx into
|
// modelIDFromPath turns /opt/.../multilingual-e5-small.onnx into
|
||||||
// "multilingual-e5-small@384".
|
// "multilingual-e5-small@384/tok2".
|
||||||
func modelIDFromPath(modelPath string) string {
|
func modelIDFromPath(modelPath string) string {
|
||||||
name := modelPath
|
name := modelPath
|
||||||
if i := strings.LastIndexAny(name, "/\\"); i >= 0 {
|
if i := strings.LastIndexAny(name, "/\\"); i >= 0 {
|
||||||
@@ -81,7 +88,7 @@ func modelIDFromPath(modelPath string) string {
|
|||||||
if name == "" {
|
if name == "" {
|
||||||
name = "onnx"
|
name = "onnx"
|
||||||
}
|
}
|
||||||
return fmt.Sprintf("%s@%d", name, embedDim)
|
return fmt.Sprintf("%s@%d/tok%d", name, embedDim, tokenizerRev)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Embed treats the text as a query. The classifier compares one short
|
// Embed treats the text as a query. The classifier compares one short
|
||||||
@@ -339,14 +346,17 @@ func (t *unigramTokenizer) encodeWord(word string) []int64 {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Backtracking walks the word from its end, and prepending each piece puts
|
||||||
|
// it back in reading order. There used to be a second reverse after this
|
||||||
|
// loop, which undid it: every multi-piece word came out backwards, and
|
||||||
|
// "query: вода" tokenized to [0 12 1294 41 12489 2] where the reference
|
||||||
|
// tokenizer gives [0 41 1294 12 12489 2] (V-664). A transformer reads
|
||||||
|
// position, so the pieces of a long Russian word were being read in the
|
||||||
|
// wrong order on every turn.
|
||||||
var result []int64
|
var result []int64
|
||||||
for i := n; i > 0; i = prev[i] {
|
for i := n; i > 0; i = prev[i] {
|
||||||
result = append([]int64{bestID[i]}, result...)
|
result = append([]int64{bestID[i]}, result...)
|
||||||
}
|
}
|
||||||
// Reverse
|
|
||||||
for l, r := 0, len(result)-1; l < r; l, r = l+1, r-1 {
|
|
||||||
result[l], result[r] = result[r], result[l]
|
|
||||||
}
|
|
||||||
return result
|
return result
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -31,6 +31,12 @@ type Config struct {
|
|||||||
// error/parse failure, falls through to the classifier (never fails the
|
// error/parse failure, falls through to the classifier (never fails the
|
||||||
// turn on the model).
|
// turn on the model).
|
||||||
LLM *LLMRouter
|
LLM *LLMRouter
|
||||||
|
// Heads — optional routing heads over the fine-tuned embedder copy. When
|
||||||
|
// set, Route consults them after stage 0 and before the LLM router. They
|
||||||
|
// decline below their own confidence threshold, so a low-confidence turn
|
||||||
|
// reaches the model exactly as it does today. Nil is the shipped-before
|
||||||
|
// behaviour and costs nothing.
|
||||||
|
Heads *RouterHeads
|
||||||
}
|
}
|
||||||
|
|
||||||
// Router — the deterministic cascade. Route never guesses: stage 0 wins
|
// Router — the deterministic cascade. Route never guesses: stage 0 wins
|
||||||
@@ -42,6 +48,7 @@ type Router struct {
|
|||||||
extractor Extractor
|
extractor Extractor
|
||||||
threshold float64
|
threshold float64
|
||||||
llm *LLMRouter
|
llm *LLMRouter
|
||||||
|
heads *RouterHeads
|
||||||
}
|
}
|
||||||
|
|
||||||
func New(cfg Config) *Router {
|
func New(cfg Config) *Router {
|
||||||
@@ -51,6 +58,7 @@ func New(cfg Config) *Router {
|
|||||||
extractor: cfg.Extractor,
|
extractor: cfg.Extractor,
|
||||||
threshold: cfg.Threshold,
|
threshold: cfg.Threshold,
|
||||||
llm: cfg.LLM,
|
llm: cfg.LLM,
|
||||||
|
heads: cfg.Heads,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -90,6 +98,10 @@ func (r *Router) Route(ctx context.Context, utterance string, now time.Time) (De
|
|||||||
continue // grammar matched shape but not content → fall through
|
continue // grammar matched shape but not content → fall through
|
||||||
}
|
}
|
||||||
d.Utterance = utterance
|
d.Utterance = utterance
|
||||||
|
// A literal pattern named that destination, which is the one provenance
|
||||||
|
// allowed to take the personal boundary off a turn (V-666). Set here and
|
||||||
|
// nowhere else, so no other arm of the cascade can claim it.
|
||||||
|
d.SourceAnchored = d.Source != SourceUnknown
|
||||||
// The grammar decided the intent; the extractor fills the slots it did
|
// The grammar decided the intent; the extractor fills the slots it did
|
||||||
// not match (V-572). See fillMatchedSlots for why every grammar gets it.
|
// not match (V-572). See fillMatchedSlots for why every grammar gets it.
|
||||||
r.fillMatchedSlots(ctx, &d, now)
|
r.fillMatchedSlots(ctx, &d, now)
|
||||||
@@ -98,6 +110,60 @@ func (r *Router) Route(ctx context.Context, utterance string, now time.Time) (De
|
|||||||
}
|
}
|
||||||
r.noteGrammarOutcomes(ctx, len(r.grammars), declinedBuild, "", "")
|
r.noteGrammarOutcomes(ctx, len(r.grammars), declinedBuild, "", "")
|
||||||
|
|
||||||
|
// stage 0b — routing heads (when wired). A softmax over the label set, so
|
||||||
|
// it cannot name an intent or a destination that does not exist, and its
|
||||||
|
// max is a real confidence. It runs before the model because it is three
|
||||||
|
// orders of magnitude faster and scores better on both halves of the route.
|
||||||
|
//
|
||||||
|
// It declines below its threshold rather than clarifying. A declined turn
|
||||||
|
// carries on to the model and then the classifier, which is what a box with
|
||||||
|
// no weights file does on every turn.
|
||||||
|
if r.heads != nil {
|
||||||
|
res, ok, err := r.heads.Route(ctx, utterance)
|
||||||
|
switch {
|
||||||
|
case err != nil:
|
||||||
|
log.Printf("router: heads fell through to the rest of the cascade: %v", err)
|
||||||
|
decision.Note(ctx, decision.Claim{
|
||||||
|
Stage: decision.StageRoute, Claimant: claimantHeads,
|
||||||
|
Outcome: decision.Declined, Reason: "error: " + err.Error(),
|
||||||
|
})
|
||||||
|
case !ok:
|
||||||
|
decision.Note(ctx, decision.Scored(decision.StageRoute, claimantHeads,
|
||||||
|
string(res.Intent), res.Confidence, decision.Declined,
|
||||||
|
"below the heads confidence threshold"))
|
||||||
|
default:
|
||||||
|
d := Decision{
|
||||||
|
Utterance: utterance,
|
||||||
|
Stage: 2,
|
||||||
|
Intent: res.Intent,
|
||||||
|
Confidence: res.Confidence,
|
||||||
|
Source: res.Source,
|
||||||
|
Clarify: res.Clarify,
|
||||||
|
}
|
||||||
|
r.fillSlots(ctx, &d, now)
|
||||||
|
decision.Note(ctx, decision.Claim{
|
||||||
|
Stage: decision.StageRoute, Claimant: claimantLLM,
|
||||||
|
Outcome: decision.NeverAsked, Reason: "the routing heads answered",
|
||||||
|
})
|
||||||
|
decision.Note(ctx, decision.Claim{
|
||||||
|
Stage: decision.StageRoute, Claimant: claimantClassifier,
|
||||||
|
Outcome: decision.NeverAsked, Reason: "the routing heads answered",
|
||||||
|
})
|
||||||
|
outcome, reason := decision.Won, ""
|
||||||
|
if d.Clarify {
|
||||||
|
outcome, reason = decision.Thinned, "the clarify head says there is too little here to act on"
|
||||||
|
}
|
||||||
|
decision.Note(ctx, decision.Scored(decision.StageRoute, claimantHeads,
|
||||||
|
string(d.Intent), d.Confidence, outcome, reason))
|
||||||
|
return d, nil
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
decision.Note(ctx, decision.Claim{
|
||||||
|
Stage: decision.StageRoute, Claimant: claimantHeads,
|
||||||
|
Outcome: decision.NeverAsked, Reason: "no routing heads are wired",
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
// stage 1a — LLM router (when wired). It reasons over the utterance instead
|
// stage 1a — LLM router (when wired). It reasons over the utterance instead
|
||||||
// of nearest-centroid guessing. On any error/parse-fail, fall through to the
|
// of nearest-centroid guessing. On any error/parse-fail, fall through to the
|
||||||
// classifier cascade (never fail the turn on the model).
|
// classifier cascade (never fail the turn on the model).
|
||||||
|
|||||||
@@ -0,0 +1,92 @@
|
|||||||
|
package stt
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"strconv"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/kami/maven/internal/audio"
|
||||||
|
)
|
||||||
|
|
||||||
|
// HTTPTranscriber — speech-to-text on another host, over HTTP.
|
||||||
|
//
|
||||||
|
// mavsttd is whisper.cpp linked into a Go daemon and reached over a unix
|
||||||
|
// socket. CrisperWhisper 2.0 cannot be reached that way: whisper.cpp derives
|
||||||
|
// its language count from the vocabulary size, and CW2's 51897 tokens shift
|
||||||
|
// seven special token ids. It runs under transformers instead, as a service
|
||||||
|
// beside the model on workpc. See docs/evals/2026-08-09-crisperwhisper2-russian-wer.md.
|
||||||
|
//
|
||||||
|
// So this is the second transport for the same seam, not a second seam. The
|
||||||
|
// caller still sees stt.Transcriber and one method.
|
||||||
|
type HTTPTranscriber struct {
|
||||||
|
url string
|
||||||
|
token string
|
||||||
|
lang string
|
||||||
|
http *http.Client
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewHTTPTranscriber builds the remote client. token may be empty for a
|
||||||
|
// service on a trusted socket, but audio is the most sensitive thing that
|
||||||
|
// crosses this seam, so a LAN deployment should always set one.
|
||||||
|
func NewHTTPTranscriber(url, token, lang string, timeout time.Duration) *HTTPTranscriber {
|
||||||
|
return &HTTPTranscriber{
|
||||||
|
url: url,
|
||||||
|
token: token,
|
||||||
|
lang: lang,
|
||||||
|
http: &http.Client{Timeout: timeout},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ErrFormat — the audio is not the one canonical shape. Refused at the seam
|
||||||
|
// rather than sent to a model that expects something else.
|
||||||
|
var ErrFormat = errors.New("stt: audio is not 16kHz mono pcm_s16le")
|
||||||
|
|
||||||
|
type httpTranscript struct {
|
||||||
|
Text string `json:"text"`
|
||||||
|
Confidence float64 `json:"confidence"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Transcribe posts the raw PCM and reads back the text.
|
||||||
|
//
|
||||||
|
// The body is the PCM bytes themselves rather than JSON. A minute of 16kHz
|
||||||
|
// mono is under 2MB raw and about 2.6MB base64, and the format is fixed by
|
||||||
|
// audio.PCM16kMono, so a header carries it more cheaply than an envelope.
|
||||||
|
func (t *HTTPTranscriber) Transcribe(ctx context.Context, a audio.Audio) (string, float64, error) {
|
||||||
|
if !a.Format.IsValid() {
|
||||||
|
return "", 0, ErrFormat
|
||||||
|
}
|
||||||
|
req, err := http.NewRequestWithContext(ctx, http.MethodPost, t.url, bytes.NewReader(a.Bytes))
|
||||||
|
if err != nil {
|
||||||
|
return "", 0, fmt.Errorf("stt: build request: %w", err)
|
||||||
|
}
|
||||||
|
req.Header.Set("Content-Type", "application/octet-stream")
|
||||||
|
req.Header.Set("X-Sample-Rate", strconv.Itoa(a.Format.SampleRate))
|
||||||
|
req.Header.Set("X-Channels", strconv.Itoa(a.Format.Channels))
|
||||||
|
req.Header.Set("X-Sample-Bits", strconv.Itoa(a.Format.SampleBits))
|
||||||
|
req.Header.Set("X-Language", t.lang)
|
||||||
|
if t.token != "" {
|
||||||
|
req.Header.Set("Authorization", "Bearer "+t.token)
|
||||||
|
}
|
||||||
|
|
||||||
|
resp, err := t.http.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return "", 0, fmt.Errorf("stt: post audio: %w", err)
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
return "", 0, fmt.Errorf("stt: remote returned %d", resp.StatusCode)
|
||||||
|
}
|
||||||
|
|
||||||
|
var out httpTranscript
|
||||||
|
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
|
||||||
|
return "", 0, fmt.Errorf("stt: decode transcript: %w", err)
|
||||||
|
}
|
||||||
|
return out.Text, out.Confidence, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var _ Transcriber = (*HTTPTranscriber)(nil)
|
||||||
@@ -0,0 +1,89 @@
|
|||||||
|
package stt
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strconv"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/kami/maven/internal/audio"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestHTTPTranscriberSendsRawPCM(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
var gotBody []byte
|
||||||
|
var gotHeader http.Header
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
gotBody, _ = io.ReadAll(r.Body)
|
||||||
|
gotHeader = r.Header.Clone()
|
||||||
|
w.Header().Set("Content-Type", "application/json")
|
||||||
|
_, _ = io.WriteString(w, `{"text":"привет","confidence":0.82}`)
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
a := audio.Audio{Format: audio.PCM16kMono, Bytes: []byte("pcm-bytes")}
|
||||||
|
tr := NewHTTPTranscriber(srv.URL, "s3cret", "ru", 2*time.Second)
|
||||||
|
text, conf, err := tr.Transcribe(context.Background(), a)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Transcribe: %v", err)
|
||||||
|
}
|
||||||
|
if text != "привет" || conf != 0.82 {
|
||||||
|
t.Fatalf("got %q %v", text, conf)
|
||||||
|
}
|
||||||
|
if string(gotBody) != "pcm-bytes" {
|
||||||
|
t.Fatalf("body should be the PCM itself, got %q", gotBody)
|
||||||
|
}
|
||||||
|
if got := gotHeader.Get("X-Sample-Rate"); got != strconv.Itoa(audio.PCM16kMono.SampleRate) {
|
||||||
|
t.Fatalf("X-Sample-Rate = %q", got)
|
||||||
|
}
|
||||||
|
if got := gotHeader.Get("X-Language"); got != "ru" {
|
||||||
|
t.Fatalf("X-Language = %q", got)
|
||||||
|
}
|
||||||
|
// Audio is the most sensitive thing crossing this seam.
|
||||||
|
if got := gotHeader.Get("Authorization"); got != "Bearer s3cret" {
|
||||||
|
t.Fatalf("Authorization = %q", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestHTTPTranscriberOmitsEmptyToken(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
var auth string
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
auth = r.Header.Get("Authorization")
|
||||||
|
_, _ = io.WriteString(w, `{"text":"x"}`)
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
a := audio.Audio{Format: audio.PCM16kMono, Bytes: []byte("x")}
|
||||||
|
if _, _, err := NewHTTPTranscriber(srv.URL, "", "ru", time.Second).Transcribe(context.Background(), a); err != nil {
|
||||||
|
t.Fatalf("Transcribe: %v", err)
|
||||||
|
}
|
||||||
|
if auth != "" {
|
||||||
|
t.Fatalf("Authorization should be absent, got %q", auth)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestHTTPTranscriberRefusesWrongFormat(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
a := audio.Audio{Format: audio.Format{SampleRate: 44100, Channels: 2, SampleBits: 16, Encoding: "pcm_s16le"}}
|
||||||
|
_, _, err := NewHTTPTranscriber("http://example.invalid", "", "ru", time.Second).Transcribe(context.Background(), a)
|
||||||
|
if !errors.Is(err, ErrFormat) {
|
||||||
|
t.Fatalf("want ErrFormat, got %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestHTTPTranscriberErrorsOnBadStatus(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
w.WriteHeader(http.StatusUnauthorized)
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
a := audio.Audio{Format: audio.PCM16kMono, Bytes: []byte("x")}
|
||||||
|
_, _, err := NewHTTPTranscriber(srv.URL, "", "ru", time.Second).Transcribe(context.Background(), a)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("a 401 must be an error, so the Pair falls back")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,157 @@
|
|||||||
|
package stt
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"log"
|
||||||
|
"net/http"
|
||||||
|
"sync"
|
||||||
|
"sync/atomic"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/kami/maven/internal/audio"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Pair — a preferred transcriber on the workstation, with mavsttd as the floor.
|
||||||
|
//
|
||||||
|
// Same arrangement as llm.Pair and for the same reason. The microphone is at
|
||||||
|
// workpc, the card there has 16GB, and CrisperWhisper 2.0 turbo scores 10.4%
|
||||||
|
// WER in Russian against 27.5% for the ggml-small.bin homesrv loads
|
||||||
|
// (docs/evals/2026-08-09-crisperwhisper2-russian-wer.md). The workstation is
|
||||||
|
// never assumed up: it sleeps, and the card is often held by a training run.
|
||||||
|
//
|
||||||
|
// Speech-to-text has only the silent half of the degradation rule. A worse
|
||||||
|
// transcript is still a turn, and there is nothing to name a gap about, so
|
||||||
|
// Transcribe always falls back. That is the whole difference from llm.Pair,
|
||||||
|
// which also carries CompleteRemote for callers that must refuse instead.
|
||||||
|
type Pair struct {
|
||||||
|
remote Transcriber
|
||||||
|
floor Transcriber
|
||||||
|
|
||||||
|
// up — the cached admission answer, written only by the prober and read by
|
||||||
|
// every turn. A voice turn must never wait on a machine that may be asleep.
|
||||||
|
up atomic.Bool
|
||||||
|
|
||||||
|
health string
|
||||||
|
interval time.Duration
|
||||||
|
http *http.Client
|
||||||
|
stop chan struct{}
|
||||||
|
stopOnce sync.Once
|
||||||
|
}
|
||||||
|
|
||||||
|
const (
|
||||||
|
probeTimeout = 2 * time.Second
|
||||||
|
defaultProbeInterval = 15 * time.Second
|
||||||
|
)
|
||||||
|
|
||||||
|
// ErrNoFloor — a Pair was built with no local transcriber to fall back to. A
|
||||||
|
// configuration mistake: the floor is what makes the remote optional.
|
||||||
|
var ErrNoFloor = errors.New("stt: no floor transcriber")
|
||||||
|
|
||||||
|
// NewPair builds the two-transcriber arrangement. remote may be nil, which is
|
||||||
|
// the unconfigured deploy: every turn goes to the floor and nothing probes.
|
||||||
|
func NewPair(remote, floor Transcriber, health string, interval time.Duration) *Pair {
|
||||||
|
if interval <= 0 {
|
||||||
|
// The config normalises this, so a zero here is a caller that built the
|
||||||
|
// Pair directly. Panicking in a ticker is the wrong way to say so.
|
||||||
|
interval = defaultProbeInterval
|
||||||
|
}
|
||||||
|
return &Pair{
|
||||||
|
remote: remote,
|
||||||
|
floor: floor,
|
||||||
|
health: health,
|
||||||
|
interval: interval,
|
||||||
|
http: &http.Client{Timeout: probeTimeout},
|
||||||
|
stop: make(chan struct{}),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Start begins probing. The first probe runs before the first tick, so a
|
||||||
|
// workstation that is already up serves the first utterance rather than the
|
||||||
|
// second. Safe with a nil remote.
|
||||||
|
func (p *Pair) Start(ctx context.Context) {
|
||||||
|
if p.remote == nil || p.health == "" {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
go func() {
|
||||||
|
p.probe(ctx)
|
||||||
|
t := time.NewTicker(p.interval)
|
||||||
|
defer t.Stop()
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
case <-p.stop:
|
||||||
|
return
|
||||||
|
case <-t.C:
|
||||||
|
p.probe(ctx)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Stop ends the prober. Idempotent and safe from two goroutines.
|
||||||
|
func (p *Pair) Stop() {
|
||||||
|
p.stopOnce.Do(func() { close(p.stop) })
|
||||||
|
}
|
||||||
|
|
||||||
|
// Available reports whether the workstation will transcribe right now.
|
||||||
|
func (p *Pair) Available() bool {
|
||||||
|
return p.remote != nil && p.up.Load()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *Pair) probe(ctx context.Context) {
|
||||||
|
ctx, cancel := context.WithTimeout(ctx, probeTimeout)
|
||||||
|
defer cancel()
|
||||||
|
req, err := http.NewRequestWithContext(ctx, http.MethodGet, p.health, nil)
|
||||||
|
if err != nil {
|
||||||
|
p.set(false)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
resp, err := p.http.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
p.set(false)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
p.set(resp.StatusCode == http.StatusOK)
|
||||||
|
}
|
||||||
|
|
||||||
|
// set records the admission answer and logs only transitions. A machine that
|
||||||
|
// sleeps nightly would otherwise write one line per interval forever.
|
||||||
|
func (p *Pair) set(up bool) {
|
||||||
|
if p.up.Swap(up) == up {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if up {
|
||||||
|
log.Printf("stt: workstation transcriber available at %s", p.health)
|
||||||
|
} else {
|
||||||
|
log.Print("stt: workstation transcriber unavailable, falling back to mavsttd")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Transcribe sends the audio to the workstation when it will take work, and to
|
||||||
|
// mavsttd otherwise. A remote that fails mid-request falls back too, because
|
||||||
|
// the admission answer is a cache and can be one interval out of date.
|
||||||
|
//
|
||||||
|
// Killing the remote mid-session must not drop the turn. That is the whole
|
||||||
|
// point of the floor, and it is what TestPairFallsBackWhenRemoteFails pins.
|
||||||
|
func (p *Pair) Transcribe(ctx context.Context, a audio.Audio) (string, float64, error) {
|
||||||
|
if p.floor == nil {
|
||||||
|
return "", 0, ErrNoFloor
|
||||||
|
}
|
||||||
|
if p.Available() {
|
||||||
|
text, conf, err := p.remote.Transcribe(ctx, a)
|
||||||
|
if err == nil {
|
||||||
|
log.Print("stt: transcribed on the workstation")
|
||||||
|
return text, conf, nil
|
||||||
|
}
|
||||||
|
// The cached answer was wrong. Correct it now rather than sending the
|
||||||
|
// next utterance into the same hole, then fall back.
|
||||||
|
p.set(false)
|
||||||
|
log.Printf("stt: workstation failed mid-request, falling back: %v", err)
|
||||||
|
}
|
||||||
|
return p.floor.Transcribe(ctx, a)
|
||||||
|
}
|
||||||
|
|
||||||
|
var _ Transcriber = (*Pair)(nil)
|
||||||
@@ -0,0 +1,143 @@
|
|||||||
|
package stt
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"sync/atomic"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/kami/maven/internal/audio"
|
||||||
|
)
|
||||||
|
|
||||||
|
// scripted — a Transcriber that answers with a fixed text, or fails.
|
||||||
|
type scripted struct {
|
||||||
|
text string
|
||||||
|
err error
|
||||||
|
calls atomic.Int32
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *scripted) Transcribe(_ context.Context, _ audio.Audio) (string, float64, error) {
|
||||||
|
s.calls.Add(1)
|
||||||
|
if s.err != nil {
|
||||||
|
return "", 0, s.err
|
||||||
|
}
|
||||||
|
return s.text, 0.9, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func sample() audio.Audio {
|
||||||
|
return audio.Audio{Format: audio.PCM16kMono, Bytes: make([]byte, 3200)}
|
||||||
|
}
|
||||||
|
|
||||||
|
// up builds a Pair whose admission answer is already true, without probing.
|
||||||
|
func up(remote, floor Transcriber) *Pair {
|
||||||
|
p := NewPair(remote, floor, "", time.Minute)
|
||||||
|
p.up.Store(true)
|
||||||
|
return p
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPairPrefersTheWorkstation(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
remote := &scripted{text: "с рабочей станции"}
|
||||||
|
floor := &scripted{text: "с homesrv"}
|
||||||
|
text, _, err := up(remote, floor).Transcribe(context.Background(), sample())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Transcribe: %v", err)
|
||||||
|
}
|
||||||
|
if text != "с рабочей станции" {
|
||||||
|
t.Fatalf("want the remote transcript, got %q", text)
|
||||||
|
}
|
||||||
|
if floor.calls.Load() != 0 {
|
||||||
|
t.Fatalf("floor was called %d times, want 0", floor.calls.Load())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The turn is what matters. A remote that dies mid-session must cost a worse
|
||||||
|
// transcript and nothing else. This is the V-486 bar.
|
||||||
|
func TestPairFallsBackWhenRemoteFails(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
remote := &scripted{err: errors.New("connection refused")}
|
||||||
|
floor := &scripted{text: "с homesrv"}
|
||||||
|
p := up(remote, floor)
|
||||||
|
|
||||||
|
text, conf, err := p.Transcribe(context.Background(), sample())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("a failed remote must not fail the turn: %v", err)
|
||||||
|
}
|
||||||
|
if text != "с homesrv" {
|
||||||
|
t.Fatalf("want the floor transcript, got %q", text)
|
||||||
|
}
|
||||||
|
if conf != 0.9 {
|
||||||
|
t.Fatalf("want the floor confidence, got %v", conf)
|
||||||
|
}
|
||||||
|
if p.Available() {
|
||||||
|
t.Fatal("a failed request must correct the cached admission answer")
|
||||||
|
}
|
||||||
|
|
||||||
|
// The next utterance goes straight to the floor rather than into the
|
||||||
|
// same hole.
|
||||||
|
if _, _, err := p.Transcribe(context.Background(), sample()); err != nil {
|
||||||
|
t.Fatalf("second turn: %v", err)
|
||||||
|
}
|
||||||
|
if remote.calls.Load() != 1 {
|
||||||
|
t.Fatalf("remote called %d times, want 1", remote.calls.Load())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPairWithNoRemoteIsTheFloor(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
floor := &scripted{text: "с homesrv"}
|
||||||
|
p := NewPair(nil, floor, "", time.Minute)
|
||||||
|
p.Start(context.Background()) // no health url, so this is a no-op
|
||||||
|
if p.Available() {
|
||||||
|
t.Fatal("an unconfigured remote is never available")
|
||||||
|
}
|
||||||
|
text, _, err := p.Transcribe(context.Background(), sample())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Transcribe: %v", err)
|
||||||
|
}
|
||||||
|
if text != "с homesrv" {
|
||||||
|
t.Fatalf("want the floor transcript, got %q", text)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPairWithNoFloorRefuses(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
_, _, err := NewPair(nil, nil, "", time.Minute).Transcribe(context.Background(), sample())
|
||||||
|
if !errors.Is(err, ErrNoFloor) {
|
||||||
|
t.Fatalf("want ErrNoFloor, got %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPairProbeReadsHealth(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
var ok atomic.Bool
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
if !ok.Load() {
|
||||||
|
w.WriteHeader(http.StatusServiceUnavailable)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
p := NewPair(&scripted{text: "remote"}, &scripted{text: "floor"}, srv.URL, time.Minute)
|
||||||
|
p.probe(context.Background())
|
||||||
|
if p.Available() {
|
||||||
|
t.Fatal("a 503 means the card is busy, so the workstation is not available")
|
||||||
|
}
|
||||||
|
ok.Store(true)
|
||||||
|
p.probe(context.Background())
|
||||||
|
if !p.Available() {
|
||||||
|
t.Fatal("a 200 means the workstation will take work")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPairStopIsIdempotent(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
p := NewPair(nil, &scripted{}, "", time.Minute)
|
||||||
|
p.Stop()
|
||||||
|
p.Stop()
|
||||||
|
}
|
||||||
@@ -0,0 +1,41 @@
|
|||||||
|
# Raw transcript, two weeks of usage
|
||||||
|
|
||||||
|
YOU: что такое TCP?
|
||||||
|
MAVEN: есть такое: Tcp/Ip — Википедия — TCP позволяет регулировать нагрузку на сеть, а также уменьшать время ожидания данных при передаче на большие расстояния. Более того, TCP гарантирует, что полученные ...
|
||||||
|
[2.1s src=search t=666]
|
||||||
|
|
||||||
|
YOU: кто такой Линус Торвальдс?
|
||||||
|
MAVEN: вот что я нашла: Торвальдс, Линус — Википедия — Ли́нус Бенедикт То́рвальдс (встречается написание Ту́рвальдс, швед. Linus Benedict Torvalds МФА: [ˈliːn.ɵs ˈtuːr.valds] о файле; род. 28 декабря 1969, Хельсинки) — финско-американский программист, создатель ядра Linux.
|
||||||
|
[0.8s src=search t=667]
|
||||||
|
|
||||||
|
YOU: что такое фотосинтез
|
||||||
|
MAVEN: нашла вот это: Фотосинтез — Фотосинтез
|
||||||
|
Схематическое изображение процесса фотосинтеза, происходящего в растениях
|
||||||
|
Упрощённое уравнение фотосинтеза, происходящего в растениях
|
||||||
|
Упрощённая схема фотосинтеза
|
||||||
|
Фотоси́нтез (от др.-греч. φῶς — « свет » и σύνθεσις — «соединённый», «складной», «связывание», « синтез ») — сл…
|
||||||
|
[0.6s src=kiwix t=668]
|
||||||
|
|
||||||
|
YOU: кто написал Войну и мир
|
||||||
|
MAVEN: вот что я нашла: Война и мир — Война и мир
|
||||||
|
Война и мир
|
||||||
|
Война и миръ
|
||||||
|
Обложка издания 1873 года
|
||||||
|
Жанр
|
||||||
|
роман-эпопея
|
||||||
|
Автор
|
||||||
|
Лев Николаевич Толстой
|
||||||
|
Язык оригинала
|
||||||
|
русский, фрагменты на французском и немецком языках
|
||||||
|
Дата написания
|
||||||
|
1863—1869, 1873
|
||||||
|
Дата первой публикации
|
||||||
|
1865—1869
|
||||||
|
Предыдущее
|
||||||
|
Казаки
|
||||||
|
Следующее
|
||||||
|
Анна Каренина
|
||||||
|
Тек…
|
||||||
|
[12.4s src=kiwix t=669]
|
||||||
|
|
||||||
|
```
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
{"reply": "есть такое: Tcp/Ip — Википедия — TCP позволяет регулировать нагрузку на сеть, а также уменьшать время ожидания данных при передаче на большие расстояния. Более того, TCP гарантирует, что полученные ...", "source": "search", "trace": "666", "secs": 2.1235475060530007, "day": 0, "n": 1, "utterance": "что такое TCP?"}
|
||||||
|
{"reply": "вот что я нашла: Торвальдс, Линус — Википедия — Ли́нус Бенедикт То́рвальдс (встречается написание Ту́рвальдс, швед. Linus Benedict Torvalds МФА: [ˈliːn.ɵs ˈtuːr.valds] о файле; род. 28 декабря 1969, Хельсинки) — финско-американский программист, создатель ядра Linux.", "source": "search", "trace": "667", "secs": 0.7689587379572913, "day": 0, "n": 2, "utterance": "кто такой Линус Торвальдс?"}
|
||||||
|
{"reply": "нашла вот это: Фотосинтез — Фотосинтез\nСхематическое изображение процесса фотосинтеза, происходящего в растениях\nУпрощённое уравнение фотосинтеза, происходящего в растениях\nУпрощённая схема фотосинтеза\nФотоси́нтез (от др.-греч. φῶς — « свет » и σύνθεσις — «соединённый», «складной», «связывание», « синтез ») — сл…", "source": "kiwix", "trace": "668", "secs": 0.575586927938275, "day": 0, "n": 3, "utterance": "что такое фотосинтез"}
|
||||||
|
{"reply": "вот что я нашла: Война и мир — Война и мир\nВойна и мир\nВойна и миръ\nОбложка издания 1873 года\nЖанр\nроман-эпопея\nАвтор\nЛев Николаевич Толстой\nЯзык оригинала\nрусский, фрагменты на французском и немецком языках\nДата написания\n1863—1869, 1873\nДата первой публикации\n1865—1869\nПредыдущее\nКазаки\nСледующее\nАнна Каренина\nТек…", "source": "kiwix", "trace": "669", "secs": 12.44599153404124, "day": 0, "n": 4, "utterance": "кто написал Войну и мир"}
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
# Raw transcript, two weeks of usage
|
||||||
|
|
||||||
|
YOU: столица Франции
|
||||||
|
MAVEN: вот что я нашла: Франция — Википедия — Она отличается ... реками. На Сене в 145 км от пролива Ла-Манш расположена столица Франции Париж....
|
||||||
|
[2.3s src=search t=670]
|
||||||
|
|
||||||
|
YOU: что такое фотосинтез
|
||||||
|
MAVEN: есть такое: Фотосинтез: процесс, фазы, функции и значение фотосинтеза — Бестселлеры Фоксфорда: собрали всё, что полюбили тысячи школьников и родителей 🧡 · Фотосинтез происходит в тканях, содержащих хлоропласты, — преимущественно в листе, на который приходится большая часть процессов фотосинтеза. Такая ткань наз…
|
||||||
|
[0.8s src=search t=671]
|
||||||
|
|
||||||
|
YOU: что такое TCP?
|
||||||
|
MAVEN: нашла вот это: TCP — TCP
|
||||||
|
TCP
|
||||||
|
Название
|
||||||
|
Transmission Control Protocol
|
||||||
|
Уровень (по модели OSI )
|
||||||
|
Транспортный
|
||||||
|
Семейство
|
||||||
|
TCP/IP
|
||||||
|
Спецификация
|
||||||
|
RFC 793 (сентябрь 1981 года) / STD 7
|
||||||
|
Основные реализации
|
||||||
|
UNIX , Linux , BSD , Windows
|
||||||
|
Расширяемость
|
||||||
|
Опции
|
||||||
|
Медиафайлы на Викискладе
|
||||||
|
TCP ( англ. Transmission Control Protocol — п…
|
||||||
|
[0.3s src=kiwix t=672]
|
||||||
|
|
||||||
|
```
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
{"reply": "вот что я нашла: Франция — Википедия — Она отличается ... реками. На Сене в 145 км от пролива Ла-Манш расположена столица Франции Париж....", "source": "search", "trace": "670", "secs": 2.295847482047975, "day": 0, "n": 1, "utterance": "столица Франции"}
|
||||||
|
{"reply": "есть такое: Фотосинтез: процесс, фазы, функции и значение фотосинтеза — Бестселлеры Фоксфорда: собрали всё, что полюбили тысячи школьников и родителей 🧡 · Фотосинтез происходит в тканях, содержащих хлоропласты, — преимущественно в листе, на который приходится большая часть процессов фотосинтеза. Такая ткань наз…", "source": "search", "trace": "671", "secs": 0.8353607719764113, "day": 0, "n": 2, "utterance": "что такое фотосинтез"}
|
||||||
|
{"reply": "нашла вот это: TCP — TCP\nTCP\nНазвание\nTransmission Control Protocol\nУровень (по модели OSI )\nТранспортный\nСемейство\nTCP/IP\nСпецификация\nRFC 793 (сентябрь 1981 года) / STD 7\nОсновные реализации\nUNIX , Linux , BSD , Windows\nРасширяемость\nОпции\nМедиафайлы на Викискладе\nTCP ( англ. Transmission Control Protocol — п…", "source": "kiwix", "trace": "672", "secs": 0.29547085403464735, "day": 0, "n": 3, "utterance": "что такое TCP?"}
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
# Raw transcript, two weeks of usage
|
||||||
|
|
||||||
|
YOU: что такое TCP?
|
||||||
|
MAVEN: Вот что я нашла: TCP, или Transmission Control Protocol, — это один из основных протоколов передачи данных в интернете. Он работает на транспортном уровне модели OSI и предоставляет поток данных с установкой соединения, гарантируя целостность данных, повторно запрашивая потерянные данные и устраняя дублирование.
|
||||||
|
[2.3s src=kiwix t=673]
|
||||||
|
|
||||||
|
YOU: кто такой Линус Торвальдс?
|
||||||
|
MAVEN: Вот что я нашла: Линус Торвальдс — финно-американский программист, который является создателем ядра Linux. Он швед по имени Linus Benedict Torvalds, родился 28 декабря 1969 года в Хельсинки, Финляндия, и его род деятельности включает программирование, разработку программного обеспечения, писательство, изобретательство, ведение блога, а также он инженер, бизнесмен и специалист в области информатики.
|
||||||
|
[2.4s src=kiwix t=674]
|
||||||
|
|
||||||
|
YOU: столица Франции
|
||||||
|
MAVEN: Вот что я нашла: Столица Франции — Париж.
|
||||||
|
[2.0s src=kiwix t=675]
|
||||||
|
|
||||||
|
YOU: почему небо голубое
|
||||||
|
MAVEN: Я не нашла в источниках ответа на твой вопрос, почему небо голубое.
|
||||||
|
[1.3s src=kiwix t=676]
|
||||||
|
|
||||||
|
```
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
{"reply": "Вот что я нашла: TCP, или Transmission Control Protocol, — это один из основных протоколов передачи данных в интернете. Он работает на транспортном уровне модели OSI и предоставляет поток данных с установкой соединения, гарантируя целостность данных, повторно запрашивая потерянные данные и устраняя дублирование.", "source": "kiwix", "trace": "673", "secs": 2.3279738989658654, "day": 0, "n": 1, "utterance": "что такое TCP?"}
|
||||||
|
{"reply": "Вот что я нашла: Линус Торвальдс — финно-американский программист, который является создателем ядра Linux. Он швед по имени Linus Benedict Torvalds, родился 28 декабря 1969 года в Хельсинки, Финляндия, и его род деятельности включает программирование, разработку программного обеспечения, писательство, изобретательство, ведение блога, а также он инженер, бизнесмен и специалист в области информатики.", "source": "kiwix", "trace": "674", "secs": 2.42521625792142, "day": 0, "n": 2, "utterance": "кто такой Линус Торвальдс?"}
|
||||||
|
{"reply": "Вот что я нашла: Столица Франции — Париж.", "source": "kiwix", "trace": "675", "secs": 1.9803519028937444, "day": 0, "n": 3, "utterance": "столица Франции"}
|
||||||
|
{"reply": "Я не нашла в источниках ответа на твой вопрос, почему небо голубое.", "source": "kiwix", "trace": "676", "secs": 1.2865088270045817, "day": 0, "n": 4, "utterance": "почему небо голубое"}
|
||||||
Reference in New Issue
Block a user