Maven/docs/routing.md

# Routing

*Last verified: 2026-08-11 @ 25ed201*

How an utterance becomes a `Decision`, why each stage exists, and what every
stage has measured. `CLAUDE.md` carries the rules an agent must not break. This
file carries the reasoning and the history behind them.

Two decisions come out of a route. **Intent** is one of seven values. **Source**
is where the answer lives, and it is read on `IntentQuery` alone. They are scored
separately, because one number hides which one moved.

## The cascade

| Stage | What it is | Where |
|---|---|---|
| 0 | Deterministic grammars over the utterance | `stage0.go`, `praxis.go`, `worldquery.go` |
| 0b | Four ONNX heads on one e5-small forward pass | `heads.go` |
| 1 | The resident model, GBNF-constrained JSON | `llmrouter.go` |
| 2 | Nearest neighbour over frozen seed phrases | `classifier.go`, `embedder.go` |

Every stage may decline, and the next one answers. Any error at stage 0b or 1
falls through, so a turn never breaks on a model.

The resident model arm is wired at `voice.go:214` through
`pickLLMRouter(cfg.Voice.UseLLMRouter(), llmClient)`. The flag is
`voice.llm_router` in `config.go`, `DefaultLLMRouter` is on, and
`deploy/mavend.json` sets it `true`. With no llama-server to talk to,
`pickLLMRouter` logs that and degrades to the classifier.

The classifier is the floor and not dead code. It runs when the resident model
is off, when there is no llama-server to talk to, and on any per-turn error.
Routing by seed similarity is the known cause of weak Russian queries. Deleting
it would make a model outage a broken turn.

### Why there are two engines at all

The original design was the classifier alone. `docs/rearchitecture.md` replaced
it with a model that emits structured JSON. The same weights phrase the reply.
That demoted the embedder from a routing gate to a hint for recall. The model
became the default on 2026-07-31.

The gap it buys is smaller than the design assumed. Measured 2026-08-02 on the
77-case Russian fixture, the classifier scores 68.8% full accuracy at p50 16.6µs.
Qwen3-1.7B scores 72.7% through the cascade. Four points, not a doubling.

An older figure of 36.8% for the classifier stood in `CLAUDE.md` until then. It
predates the stage 0 rules and the seed additions. Both now score inside the
classifier baseline.

Latency was misreported the same way. A figure of 2.7 seconds stood for two
days and was contention rather than the model.
`docs/evals/2026-07-31-routing.md` line 61 measures the router at p50 825ms and
the cascade at p50 0.80s to 1.04s.

## Numbers

Three arms answer, so three numbers are live. Judge a routing change against the
classifier and the resident model, since those are what always answer.

| Arm | Intent | Destination | p50 | Measured |
|---|---|---|---|---|
| classifier + ONNX | 76.0% (73/96) | 36.4% (12/33) | 16.6µs | 2026-08-08 |
| resident Qwen3-1.7B, cascade | 80.2% | not measured | 1.19s | 2026-08-05 |
| routing heads, cascade | 96.9% | 75.8% | 27.9ms | 2026-08-08 |
| gemma-4-12b, cascade | 84.4% | 72.7% | 329ms | 2026-08-02 |
| gemma-4-E4B, cascade | 89.6% | 57.6% (19/33) | 294ms | 2026-08-09 |

The fixture grew from 77 cases to 91 to 96. So a number is comparable only to
another number on the same fixture. Sources:
`docs/evals/2026-08-05-routing-resident-model.md`,
`docs/evals/2026-08-02-workstation-gemma4-12b.md`,
`docs/evals/2026-08-09-e4b-vs-12b-routing.md`,
`docs/evals/2026-08-08-routing-heads-in-go.md`,
`docs/evals/2026-08-08-destination-fixture.md`.

The resident model alone scores 37.4% full against 61.5% intent-only. The gap is
slots and not routing. It routes `reminder` and leaves the time to the daemon,
which is what the contract asks.

To re-run the resident model as router, start a **second** llama-server on a
fixed host port. The resident one binds `--port 0` inside the container and no
host process can reach it.

### The workstation is not the better router any more

It was, from 2026-08-02 until the heads landed. gemma-4-12b beat everything on
the box at 84.4% intent and 72.7% destination. The heads beat it on both at a
twelfth of the latency. The workstation stays the better phraser.

E4B replaced the 12B on 2026-08-09 by the owner's call. It is a step down on
routing. Against a same-session 12B control it costs four destination cases and
buys 50ms. Read destination as the finding. It names nothing where the 12B names
`recall` or `calendar`, which is safe but walks the whole chain. It has no MTP
and cannot be given any here. The only `gemma4-assistant` draft on disk is
trained against the 12B's hidden states.

## Stage 0: what the grammars claim, and why

A rule at this stage is a claim. Either the model gets this wrong, or it wastes a
second getting it right. Every rule was added against a measurement.

- **Agenda questions** (`AgendaQueryGrammars`, 2026-08-01). "что у меня сегодня",
  "во сколько у меня встреча" and anything naming a calendar go to `IntentQuery`.
  They were going to `IntentSystem`, where `replySystem` has no agenda arm and
  answered "пока не умею". Worth 2.6 points of full accuracy and calendar 0/2 to
  2/2.
- **Rest of day and narrative** (V-498, 2026-08-04). `rest-of-day-query` claims
  "что дальше?". `NarrativeQueryGrammar` claims "расскажи про X", "объясни X" and
  "опиши X". Neither carries a question mark or an interrogative, so the model
  called both `IntentFact`. `IsQuestionShaped` caught the write downstream, so
  this was a latency and fixture defect rather than a correctness one. The
  narrative rule declines `chatNarrativeTopics`, because the query chain has no
  source that answers a joke or a bedtime story.
- **Praxis** (V-516, 2026-08-05). `PraxisGrammars()` fills `Slots.Fn` with a
  capability name. These grammars are the **only** path to Praxis and not a
  faster one. The model reaches Praxis 0/12 alone, the same as the classifier.
  Nothing in the router prompt names a Praxis capability, so there is no string
  for it to write. Through the cascade it is 11/12. Measured overall 16/30 to
  27/30, lifecycle 0/5 to 5/5
  (`docs/evals/2026-08-05-praxis-reach.md`,
  `docs/evals/2026-08-05-reach-llm-router.md`).
  `handlePraxisAct` compares `Slots.Fn` to a capability alias. Otherwise that
  slot is filled from the deployment's enabled tool names, and no Praxis alias
  is on that list.
- **World questions** (`WorldQueryGrammars`, V-655, 2026-08-07). "что такое X"
  and "сколько будет 17 на 23". Wired after the agenda rules and **before** the
  feed and list rules. "что такое лента" is a definition question, and the feed
  rule would take it on the noun alone.

`calendar-query` and `event-time-query` name the calendar as the destination.
The possessive agenda rules deliberately do not. "что у меня в списке покупок"
matches `agenda-query`, and naming the calendar there would take the list source
off the turn. That caution now costs four destination cases. See the model arm
below.

Go's `\b` is ASCII-only and never fires after a Cyrillic letter. A pattern needs
an explicit `(\s|[?!.]|$)`.

The stage 0 set lives in `router.StageZeroGrammars` (`internal/router/stagezero.go`).
Both `buildRouter` and the eval fixture call it. The daemon and the measurement
cannot disagree about which rules exist, or in what order.

It was two lists until V-693 and it drifted twice. V-655 wired
`WorldQueryGrammars` into the daemon and not into the fixture. That cost 3 points
of destination and V-659 fixed it. `BareCaptureGrammar` then did the same thing,
from V-557 until V-693 found it. That one moved no number, which is the point:
the fixture had been scoring a set nobody ran and nothing said so.

### Praxis lifecycle rules

A **stative** lifecycle word ("готово", "принято") needs an item named beside it.
A bare **imperative** ("закрывай") may ask which one. It also requires a sentence
naming no object of its own. Otherwise "закрой шторы в комнате" goes to Praxis
instead of the house. A demonstrative ("отметь это как сделанное") resolves
against `h.surfacedItems` only when exactly one item was spoken. Otherwise the
turn goes back to the cascade rather than transitioning the wrong item.

### Slots on a stage 0 decision

`fillMatchedSlots` runs the stage 2 extractor over whatever a grammar built
(V-572, 2026-08-06). It fills only the slots the grammar left empty. A matched
value always wins, because the rule read a literal pattern and the extractor
guesses.

It did not run before. So `ReminderGrammar` handed the daemon `HasTime: false`
for "напомни в 11:00 позвонить маме", and `missingFor` read the silence as
absence and asked "Когда?". It is inert for every grammar but the reminder:
`Extract` fills Time, Fn and Key and nothing else. A stage 0 query costs 3.7µs
against 3.9µs before, benchmarked at 20000x.

`Slots.Text` is deliberately not filled. A grammar that left it empty meant it,
and `agendaQueryBuild` hands the query chain the utterance itself.

## Stage 0b: the routing heads

Routing has a bounded output space, so it is classification rather than
generation (owner's call, V-546,
`docs/plans/18-routing-heads-on-e5-small.md`). The 118M multilingual-e5-small is
already resident. A softmax cannot emit a value that does not exist, so no
grammar is needed. Max softmax is a calibratable confidence, where
`Confidence: 1.0` was a hardcode. Training costs roughly 5e15 FLOPs, so 10 to 30
minutes on the workstation. A 100M decoder from scratch is 10 to 20 GPU hours.

**Fine-tune a copy of the weights.** The resident embedder backs memory recall.
Training it in place couples routing accuracy to recall@1, with nothing in the
suite to name the trade.

Four heads share one masked mean pool, trained over three days. The measurements
are `docs/evals/2026-08-08-routing-heads-two-head.md`,
`docs/evals/2026-08-08-slot-head-three-head.md`,
`docs/evals/2026-08-08-clarify-head-four-head.md` and
`docs/evals/2026-08-08-massive-warm-start.md`.

| Head | Score | Notes |
|---|---|---|
| intent | 92.8% mean over 3 seeds | fixture is the 88 cases carrying an intent |
| destination | 80.8% mean, best 29/33 | beats the 12B teacher it was distilled from |
| slot BIO tags | 72.4% span F1 | still climbing when epoch selection stops it |
| clarify | catches 7.0 of 8, 2.3 false of 88 | parity with the cascade, no rules in front |

Read the best destination run as one seed and not a headline. One case is 3
points on a fixture this small. Head intent accuracy is **not** comparable to the
cascade's 76.0% and 84.4%. A softmax has no clarify class, so the head's fixture
is 88 cases and not 96.

Recall is 15/15 and world is 5/5.

**Mood is cut, not deferred.** The enum describes her own reply state, not the
speaker's emotion, and no dataset maps onto it.

### The clarify head

Clarify is not a value of intent, so a softmax cannot emit it. It is a second
question over the same pooled vector: can Maven act on this at all. Accuracy is
the wrong number here and a head that never asks scores 91.7%.

Confidence is the other half. Max softmax over the intent head reads 0.851 where
it is right and 0.604 where it is wrong. It ranks right above wrong in 83.4% of
pairs.

It is not free the way the slot head was. Intent, destination and slot F1 each
move down one to four points, inside the seed spread. `поужинал` is a false
clarify on every seed. That is the same defect `thinSingleToken` was narrowed for
on 2026-08-01.

The corpus is generated, because every existing row is answerable by
construction. The router-prompt agreement filter cannot work here. `routeGrammar`
has no clarify value, and a generated line always agrees with itself. A gemma
judge replaces it. The first judge called 24 of 40 answerable rows underspecified.
It judged against a generic assistant rather than against Maven's contract.

### The slot head

BIO slot tags had no Maven-domain corpus. That was true of found corpora and
false of made ones. `label_slots.py` distils spans out of gemma-4-12b under a
GBNF closed over Maven's own five slots. A span survives only when it is a
literal substring of the utterance, so the agreement filter costs no second call.
2178 spans over 1702 rows, 37 dropped, nothing unparsed.

Epoch selection reads the intent dev slice alone. That costs the slot head about
4 points.

### Warm start and the floor

The MASSIVE warm-start of step 2 is worth nothing here. Stock e5-small ties it on
intent and leads by a third of a case on destination. Nothing argues for keeping
that step.

The floor was a corpus defect and it is fixed. The first 120 floor rows carried
one sentence shape, so the head named a destination where the fixture says walk
the chain. Rotating six shapes took the floor 3/7 to 6/7 and destination 75.8% to
80.8%.

What is left is calendar at 3/6 on every seed, which training cannot move. The
possessive agenda rules claim those cases at stage 0 and name nothing, so no
label reaches the head.

### Reading them in Go

`RouterHeads` in `internal/router/heads.go` loads `router_heads.onnx` (V-664,
2026-08-08). It reads intent, destination and clarify off one forward pass.

Three rules around it, each measured:

- The **clarify head decides first**, before the intent threshold. It answers a
  different question. A thin utterance scores low intent by construction, so
  gating it cost 6 of 8 ambiguous cases.
- The **destination head is read on `IntentQuery` only**, since no other intent
  reaches `queryWalk`.
- `headsThreshold` is 0.6, the measured knee. Every value up to 0.85 drops right
  answers and keeps the same two wrong ones.

`voice.embedder.heads_path` is the whole switch. Empty, missing or unloadable
means the heads are nil. The cascade is then byte-for-byte what shipped before
them.

Pointing it at `model_path` is refused at config load (V-692). An unloadable
weights file is not fatal, because the heads are an accelerator. A working file
in the wrong role is a different thing. The heads then score with the graph the
resident embedder scored with, and recall degrades with no log line. The check
cleans and absolutises both paths, then compares them with `os.SameFile`, so a
symlinked copy is caught too.

### The tokenizer bug the heads found

`encodeWord` in `onnxembedder.go` read every long word backwards until 2026-08-08.
It cost recall@1 7.4 points and recall@3 11.1. Nothing caught it, because seeds
and queries were mangled the same way and cosine survived. The heads found it.
They are trained through transformers and read through this.

The embedder id now carries a tokenizer revision (`@384/tok2`). So fixing the
tokenizer triggers `ReembedAll` the way swapping the model file does. Bump
`tokenizerRev` on any change to what it emits.

## Clarify

`Confidence: 1.0` was hardcoded in `llmrouter.go`. So the model path could never
ask for clarification, and it missed 6 of 6 refusal cases (V-359). The bug had a
second half. The model branch never consulted `r.threshold` at all, so a correct
low confidence would have been discarded anyway.

Fixed 2026-07-31 with structural signal feeding the same stage 3 gate the
classifier path already had (`gateLLMDecision` in `router.go`). Three signals: a
single-token utterance, a keyless fact, an act with no allowlisted fn.

Re-measured: missed clarify 6/6 to 1, at the cost of 3 false clarifies and 2.6
points of full accuracy. Two of the three false clarifies are acts the model
mis-routed and the gate caught. Asking beats wrongly executing, so the fixture
and the daemon disagree about what is correct there.

The third, `поужинал`, was a real defect. The single-token rule was an English
intuition. It does not transfer to Russian, where one word is routinely a whole
sentence.

Narrowed 2026-08-01. `thinSingleToken` (`internal/router/singletoken.go`) still
thins a bare one-word nominal. It spares two classes. One is a closed lexicon of
social and control singles ("привет", "стоп", "yes"). The other is any token
carrying a Russian verb ending, because a verb already contains its subject. Both
tests are offline and cost nothing. False clarifies 3 to 2, intent-only 74.0% to
75.3%.

The two remaining false clarifies are the act-with-no-allowlisted-fn arm of the
gate, not this rule.

## The destination

`query` was a shrug. The cascade sorted an utterance into one of seven intents,
then `IntentQuery` handed the turn to `querySources` in the daemon. That is
twenty-two branches deciding by seed similarity in a fixed order. It had no
fixture, no accuracy number, no model arm and no floor.

`Decision.Source` (`internal/router/source.go`) is the second half of the route
(V-655, 2026-08-07). Twelve destinations, not twenty-two. The three recall passes
plus `fact-by-key` are one destination from outside. So are search, Kiwix and the
URL reader.

`queryWalk` in `cmd/mavend/actions_query.go` takes sources **out** and moves none.
That is the safety argument. The table's order is load-bearing. Every comment on
it argues a reason between two sources. Above all it carries "the owner's data
first, then the world". Naming `SourceWorld` does not send the turn outside on its
own.

What comes out is only the sources that **guess**. Those decide a turn is theirs
by cosine against frozen seeds, then answer whatever they claimed. They hold no
table that could come back empty. Weather is the pure case and has no local data
at all. It was measured on the box 2026-08-07
(`docs/evals/2026-08-07-week-of-usage.md` section 4). It answered both "что такое
TCP?" and "сколько будет 17 на 23?" with "для какого города?". The feed answered
"какой у меня любимый язык?" with kernel headlines.

### Who may drop the personal boundary

The personal boundary guesses, so naming `SourceWorld` drops it. That is what
stops it answering "кто такой Линус Торвальдс?" with "не нашла у тебя такой
записи", which it did on 2026-08-07.

Three deciders name a destination and two of them infer it: the heads and the
resident model. An inferred `SourceWorld` on a question about him would reach
SearXNG. That widens what is asked rather than costing a local answer. So only a
stage 0 grammar may drop it (owner's call, V-666, 2026-08-09).

`Decision.SourceAnchored` carries the provenance. It is a field and not
`Stage == 0`. Stage 0 also means confidence 1.0 and an anchored claim band, and
one of those could stop implying the others. `definitionQueryPattern` claims "кто
такой X", so the 2026-08-07 case is still anchored and still answered.

`queryWalk` reads `SourceAnchored` for the query source marked `boundary: true`
and no other. Every other guesser still comes off the turn, whoever named the
destination. `TestOnlyAGrammarMayDropTheBoundary` and
`TestNamingRecallKeepsTheBoundary` pin both directions.

### The destination fixture

`want_source` on `eval.Case` is a pointer, because the destination has three
states and a bare string has two. Absent is every intent but query. Present and
empty is the `SourceUnknown` contract: name nothing and walk the chain. Present
and named is a destination the route must produce. Thirty-three of ninety-six
cases carry one.

A destination miss does **not** fail the case. It lands in `Outcome.SourceReason`
and never in `Reasons`, so `Accuracy` and `IntentAccuracy` mean what they meant.
`SourceAccuracy` is a second number over the labelled cases alone.
A route that lost its intent scores no destination hit. Otherwise a clarify would
satisfy an empty label for free.

Seven cases assert the floor and five of them are homelab operations. They cluster
because `SourceRecall`, `SourceNetwork` and `SourceAttention` overlap on every
question about the box. `mavpoll` writes its netdata and uptime-kuma observations
into the fact store recall reads. That is a finding about the enum, not a gap in
the labelling. The other two are `ru-query-005` and `ru-query-014`. No query
source reads the reminder store, and a deadline could sit in tasks, the calendar
or Praxis. The owner confirmed all seven floor labels on 2026-08-08.

### The model arm

`routeGrammar` carries a `source` rule closed over `router.Sources` plus the
empty floor (V-660, 2026-08-08). So the model cannot emit a destination that does
not exist. The prompt lists the twelve in Russian and says `""` is a normal answer
to give often. `LLMRouter.Route` reads it back through `ValidSource` and on
`IntentQuery` alone.

Against gemma-4-12b the cascade scores destination 24/33 with intent unmoved, and
recall goes 0/15 to 14/15.

**Stage 0 now costs four destination points.** It did not before. The four cases
the cascade loses and the model alone wins are all calendar. The possessive agenda
rules claim them first and name nothing on purpose. That caution was free while
nothing downstream could name anything either. It is not free now, and the fix is
the owner's call (V-660 open).

## The decision trace

Arbitration between the claimants on the utterance stream is order. It is
hardcoded in the pre-route resolver ladder, in `buildRouter` and in
`querySources`. Nothing recorded who lost until V-564.

`internal/decision` records one `Record` per turn. It holds every claimant, what
it would have made the turn, the score it reported, and how it ended. A claimant
won, declined, lost on score, was thinned by a gate or was **never asked**.

The record rides the context, the same seam `querysource.go` uses. So a claim
site cannot change a route, and a context with no record costs nothing. It is
installed in `runTurn`, so the mic, telegram and the web leave the same trail.

Adding a rung to the ladder in `runTurn` means adding its name to `preRouteLadder`
in `cmd/mavend/decisiontrace.go`. Otherwise that rung is silently missing from the
record.

### Why it persists now

The original rule was that nothing persists, because a turn record is read minutes
later or never. Storage was a 25-turn in-memory ring read over `ipc.TurnDecisions`
and rendered on `/trace`.

The owner reversed it on 2026-08-06 (V-629,
`docs/plans/21-persisting-the-routing-trace.md`). The routing heads cannot be
fitted or calibrated without real utterances. And 9 of the 31 modes in
`internal/modes` have no seed example at all.

The ring did not move. `cmd/mavend/routingtrace.go` is a second sink beside it,
writing `routing_traces` (migration #23). The utterance is stored in clear. A
384-dimension vector of a short sentence is substantially recoverable, so storing
vectors instead would be a privacy claim we cannot support. What makes it safe is
the same thing that makes the fact store safe. Retention is 14 days, enforced on
write and again on start, so a box that goes quiet does not keep every row.
Nothing reads it outward. `Store.Wipe` deletes it with everything else.

### Corrections

A correction is promoted out into a seed-shaped row in `routing_labels`
(migration #24) and kept, because a label is not a transcript. The transcript
still expires.

A turn marked wrong with no target is a usable negative, so naming the intent is
never required. The target is one of the seven intents and never free text.

All three reaches offer it as of 2026-08-06:

- `/chat` offers two buttons beside the reply, over `ipc.CorrectTurn` and the
  trace id that rides back on `ipc.ChatReply`.
- Voice offers the `repair` rung, which has read spoken corrections since V-455.
  It now writes the durable label beside the classifier seed it always wrote. A
  spoken negative with no target is its own rung, `repair-negative` (V-636,
  `docs/plans/22-correcting-a-turn.md`).
- Telegram offers an inline keyboard under the reply. It needed the chat to become
  readable first (V-637, `docs/plans/23-inbound-telegram.md`). The poller is dark
  unless the `telegram` block says `intake`. It long-polls, because the box takes
  no inbound connections. It accepts `chat_id` and no other sender, and it drops
  whatever queued while the daemon was down. It reaches the daemon through
  `ipc.CoreAPI` alone.

The turn source is still `tap:text` for both telegram and the web. So provenance
cannot tell a chat turn from a typed one.