Train the destination head and beat the teacher (V-661)

Step 3 of the routing-heads plan. Intent and destination share one masked mean
pool on e5-small. Destination scores 26/33 against 12/33 for the classifier
cascade and 24/33 for the cascade with gemma-4-12b, which is the teacher these
labels were distilled from. Recall goes 0/15 to 15/15.

Two heads, not four, and both cuts are label problems rather than GPU time.
Mood describes her own reply state and no dataset maps onto it. BIO slot tags
have no Maven-domain corpus.

The MASSIVE warm-start from step 2 is worth nothing here either. Stock ties it
on intent and leads by a third of a case on destination.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
This commit is contained in:
2026-08-08 19:38:23 +04:00
parent e470435cf1
commit f55bedee2e
2 changed files with 201 additions and 0 deletions
+27
View File
@@ -246,6 +246,33 @@ was a hardcode. **Fine-tune a copy of the weights.** The resident embedder backs
recall. Training it in place couples routing accuracy to recall@1, with nothing in the
suite to name the trade.
**Two of those heads are trained as of 08-08-2026, and they are not the three
above** (V-661, `docs/evals/2026-08-08-routing-heads-two-head.md`). Intent and
destination share one masked mean pool. Destination scores **26/33 (78.8%)** on
the fixture. The classifier cascade scores 12/33 and the cascade with gemma-4-12b
scores 24/33, so a 118M encoder beats the 12B teacher it was distilled from.
Recall is 15/15 and world is 5/5. Intent is 93.6% mean over three seeds. That is
**not** comparable to the 76.0% and 84.4% those two arms scored: a softmax has no
clarify class, so the head's fixture is the 88 cases carrying an intent.
**Mood is cut, not deferred.** The enum describes her own reply state, not the
speaker's emotion, and no dataset maps onto it. **BIO slot tags have no
Maven-domain corpus**, so they stay in the MASSIVE body from step 2. Both are
label problems and neither is a GPU problem: the run is under four minutes.
The MASSIVE warm-start of step 2 is worth nothing here. Stock e5-small ties it on
intent and leads by a third of a case on destination. Nothing argues for keeping
that step.
What the head gets wrong is the floor. It names a destination where the fixture
says walk the chain, and it is confident doing it. `"почему сервер тормозит"`
reads `world` at 0.80. The training floor is generated ambiguous questions and the fixture floor
is homelab operations, which are not the same distribution.
**Nothing of this runs in Go.** The weights are `heads.pt` and `out/body_heads/`
on workpc. Reaching the daemon needs an ONNX export and a caller. The resident
e5-small must not be replaced by the copy, because recall depends on that file.
`Confidence: 1.0` used to be hardcoded in `llmrouter.go`, so the LLM
path could never ask for clarification (6/6 refusal cases missed on the fixture) — Vikunja
#359. Fixed 31-07-2026 with structural signal (single-token utterance, keyless fact, act with