Train the destination head and beat the teacher (V-661)
Step 3 of the routing-heads plan. Intent and destination share one masked mean pool on e5-small. Destination scores 26/33 against 12/33 for the classifier cascade and 24/33 for the cascade with gemma-4-12b, which is the teacher these labels were distilled from. Recall goes 0/15 to 15/15. Two heads, not four, and both cuts are label problems rather than GPU time. Mood describes her own reply state and no dataset maps onto it. BIO slot tags have no Maven-domain corpus. The MASSIVE warm-start from step 2 is worth nothing here either. Stock ties it on intent and leads by a third of a case on destination. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
This commit is contained in:
@@ -246,6 +246,33 @@ was a hardcode. **Fine-tune a copy of the weights.** The resident embedder backs
|
||||
recall. Training it in place couples routing accuracy to recall@1, with nothing in the
|
||||
suite to name the trade.
|
||||
|
||||
**Two of those heads are trained as of 08-08-2026, and they are not the three
|
||||
above** (V-661, `docs/evals/2026-08-08-routing-heads-two-head.md`). Intent and
|
||||
destination share one masked mean pool. Destination scores **26/33 (78.8%)** on
|
||||
the fixture. The classifier cascade scores 12/33 and the cascade with gemma-4-12b
|
||||
scores 24/33, so a 118M encoder beats the 12B teacher it was distilled from.
|
||||
Recall is 15/15 and world is 5/5. Intent is 93.6% mean over three seeds. That is
|
||||
**not** comparable to the 76.0% and 84.4% those two arms scored: a softmax has no
|
||||
clarify class, so the head's fixture is the 88 cases carrying an intent.
|
||||
|
||||
**Mood is cut, not deferred.** The enum describes her own reply state, not the
|
||||
speaker's emotion, and no dataset maps onto it. **BIO slot tags have no
|
||||
Maven-domain corpus**, so they stay in the MASSIVE body from step 2. Both are
|
||||
label problems and neither is a GPU problem: the run is under four minutes.
|
||||
|
||||
The MASSIVE warm-start of step 2 is worth nothing here. Stock e5-small ties it on
|
||||
intent and leads by a third of a case on destination. Nothing argues for keeping
|
||||
that step.
|
||||
|
||||
What the head gets wrong is the floor. It names a destination where the fixture
|
||||
says walk the chain, and it is confident doing it. `"почему сервер тормозит"`
|
||||
reads `world` at 0.80. The training floor is generated ambiguous questions and the fixture floor
|
||||
is homelab operations, which are not the same distribution.
|
||||
|
||||
**Nothing of this runs in Go.** The weights are `heads.pt` and `out/body_heads/`
|
||||
on workpc. Reaching the daemon needs an ONNX export and a caller. The resident
|
||||
e5-small must not be replaced by the copy, because recall depends on that file.
|
||||
|
||||
`Confidence: 1.0` used to be hardcoded in `llmrouter.go`, so the LLM
|
||||
path could never ask for clarification (6/6 refusal cases missed on the fixture) — Vikunja
|
||||
#359. Fixed 31-07-2026 with structural signal (single-token utterance, keyless fact, act with
|
||||
|
||||
Reference in New Issue
Block a user