# Two heads on e5-small, and the first destination the router did not need a model for

Measured 2026-08-08 on workpc (Radeon RX 7900 GRE, ROCm). Covers V-661, step 3 of
`docs/plans/18-routing-heads-on-e5-small.md`. Workspace is `~/Programs/embed-training`,
scripts `gen_query_source.py`, `label_source.py`, `build_heads_corpus.py`,
`train_heads.py`, `score_confidence.py`.

## Two heads, not four

Intent is 7 classes and destination is 12 plus the `SourceUnknown` floor, sharing
one masked mean pool over one forward pass. The plan asked for four. Two of them
have no labels and neither is a GPU problem.

**Mood is cut, not deferred.** Maven's enum is `neutral, happy, thinking, tired,
confused` and it describes her own reply state, not the speaker's emotion.
`psytechlab/EmpatheticIntents-ru` was the only candidate and its 32 emotion
labels do not map onto it. There is nothing to train against.

**BIO slot tags stay in the MASSIVE body.** No Maven-domain span corpus exists.
`2026-08-08-massive-warm-start.md` records that no second Russian slot-filling
corpus is reachable at all.

The destination loss is masked with `ignore_index`. Only a query turn reaches
`queryWalk`, so a reminder contributes nothing to it.

## Where the destination labels came from

V-660 taught the router prompt to name a destination. That made gemma-4-12b a
teacher, and this distils it.

Labelling the 300 query rows already in `train_v5.jsonl` gave 277 destinations.
The shape was unusable: the floor 101, calendar 62, recall 47, and `feeds` and
`attention` at zero. A 13-way softmax cannot learn a class with no examples.

`gen_query_source.py` is the destination half of `gen_corpus.py` and runs the
same two passes. Gemma writes questions whose answer lives in one named place.
The daemon's own `routeSystem` prompt then routes each one back. A line survives
only when the intent is `query` **and** the source is the destination it was
generated for. The glosses are copied verbatim out of `route_system.txt`, so the
generator and the labeller work from one definition.

The prompt and the GBNF are dumped from `internal/router/llmrouter.go` by
`TestDumpPrompt`, never retyped. The workspace held its own copies and V-660
changed both.

1229 kept of 2373 generated, 51.8%. Merged corpus is 3664 rows carrying 1727
destinations:

| | rows | | rows |
|---|---|---|---|
| the floor | 220 | world | 132 |
| calendar | 180 | money | 126 |
| tasks | 169 | list | 124 |
| recall | 167 | self, feeds, attention | 120 each |
| weather | 136 | network | 103 |
| | | home | 50 |

`home` is thin because the agreement filter rejected most of what was generated
for it. A question about the house routes `act` more often than `query`. That is
the filter working, and 50 is the finding rather than a shortfall.

**The floor was regenerated once.** The first 120 rows carried one sentence
shape across eight topics. That shape was "что там с X" and its two synonyms.
Every named destination varied and only the floor collapsed. The reason is that
the generator varies a topic, and ambiguity is not a topic.

`gen_query_source.py` now rotates six floor shapes. A `почему` question, a yes
or no question, and a question carried by intonation alone. Then a
better-or-worse question, a status question, and an existence question. That is
a fix to degenerate generation. It is not fitting to the fixture, whose floor
cases are homelab operations and match none of the six.

The 1229 generated rows carry `intent: null`. Every one is a query by
construction. There are five times as many as the corpus has query rows, so
including them would make query half the intent corpus.

## Result

Three seeds, two bodies, epoch chosen on the intent dev slice and never on a
destination number.

| body | floor corpus | intent mean | destination mean |
|---|---|---|---|
| warm-started `out/body_massive` | one shape | 93.6% | 75.8% |
| stock `multilingual-e5-small` | one shape | 93.6% | 76.8% |
| warm-started `out/body_massive` | six shapes | 93.6% | **80.8%** |

Best single run is destination **29/33 (87.9%)**, seed 0 on the rotated floor.
Peak 1.68GB of 17.2GB, under four minutes end to end.

Against the two arms already measured on the same 33 labelled cases:

| | destination |
|---|---|
| classifier cascade (V-659) | 12/33 (36.4%) |
| cascade + gemma-4-12b (V-660) | 24/33 (72.7%) |
| two heads on e5-small | 29/33 (87.9%) |

A 118M encoder beats the 12B teacher it was distilled from, on the fixture. The
per-destination split is where it happens: **recall 15/15** and **world 5/5**.
Recall was 0/15 on the cascade and 14/15 through gemma.

Read 87.9% as one seed of a mean of 80.8%, not as a headline. Three seeds score
29, 25 and 26 of 33. One case is 3 points on a fixture this small.

Intent is **not** comparable to the 73/96 and 81/96 figures those two arms
scored. A softmax has no clarify class. The head's fixture is the 88 cases that
carry an intent, and the 8 `want_clarify` cases are scored separately below.

## The MASSIVE warm-start is worth nothing here either

Step 2 measured it at +0.4 points of intent accuracy and called that inside seed
noise. Destination was the open question, because MASSIVE has a
`definition_word` slot that looked like a `SourceWorld` signal sitting in a head
already trained.

It is not. The two bodies score the same intent mean to one decimal. Stock is
one point ahead on destination, which is a third of one case. Nothing here argues
for keeping the warm-start step. Dropping it removes a dependency on a corpus
pull that `datasets` 5.0 cannot do.

## The floor moved, calendar did not

Before the rotation, all seven misses at seed 0 were the floor and calendar. The
head named a destination where the fixture says walk the chain, and it was
confident doing it. `"почему сервер тормозит"` read `world` at 0.80.
`"хватает ли места под новые бэкапы"` read `network` at 0.82. Those are the five
homelab cases V-659 flagged, where `SourceRecall`, `SourceNetwork` and
`SourceAttention` all overlap. `mavpoll` writes its observations into the fact
store recall reads.

| | one shape | six shapes |
|---|---|---|
| the floor | 3/7 | 6/7, 6/7, 5/7 |
| calendar | 3/6 | 3/6, 3/6, 3/6 |
| recall | 15/15 | 15/15 at seed 0 |
| world | 5/5 | 5/5 |

The floor was a corpus defect and it cost 3 cases. Sentence variety carried it,
not homelab vocabulary, which the training rows still do not contain.

**Calendar is 3/6 at every seed and is a different problem.** It is the shape
V-660 named. The possessive agenda rules claim those cases at stage 0 and
deliberately name nothing, so no destination label reaches the head. Training
cannot move a case the head never sees. That one wants the owner's call.

## Max softmax separates, weakly, and the gate stays

The plan argues max softmax is a calibratable confidence where `Confidence: 1.0`
was a hardcode. Measured on the intent head:

| | n | mean confidence |
|---|---|---|
| correct | 80 | 0.897 |
| wrong | 8 | 0.705 |
| `want_clarify` | 8 | 0.685 |

The softest correct answer is 0.66 and 3 of 8 clarify cases sit below it. So a
single cut buys three clarifies at no false-clarify cost, and no more.

The other five explain themselves. `"напомни"` scores 0.94 as `reminder` and
`"сделай это"` scores 0.80 as `act`. Both are intent-certain and slot-empty, and
confidence was never the signal there. `gateLLMDecision` already catches exactly
that shape, an act with no allowlisted fn or a keyless fact, and it keeps doing
so. The head replaces the hardcode. It does not replace the gate.

## An incident worth recording

The first generation run produced zero rows for eight destinations. `mavgpud`
yields the card when another process wants it (V-488) and llama-server answers
503 until the model is back. Every generate call inside that window burned one of
the destination's batches. The run walked its own cap without a single successful
call. The log said `503` 260 times, and the summary line said 64.1% keep rate,
which read as success.

`call()` now retries a 503 with backoff. A generator that treats an unloaded
model as a bad generation is a silent-corpus bug, not a slow one.

## What this does not measure

**Nothing here runs in Go.** The heads are a `heads.pt` and an
`out/body_heads/` on workpc. Reaching the daemon needs an ONNX export and a
caller. The resident e5-small must not be replaced by this copy: recall depends
on that file, and `EmbedQuery`/`EmbedPassage` are its contract.

The destination fixture carries 4 of 13 classes: recall 15, floor 7, calendar 6,
world 5. `tasks`, `money`, `list`, `home`, `network`, `feeds`, `attention` and
`self` have no gold case. So 78.8% is silent on eight destinations that together
hold 800 training rows.

Latency was not measured. A forward pass of a 118M encoder should beat a 1.7B
decoder on a query turn. That is arithmetic, not a number from this box.