Maven/docs/evals/2026-08-08-routing-heads-in-go.md

The routing heads, running in Go

Date: 2026-08-08. Vikunja V-664. Weights: router_heads.onnx, fp32, exported from heads.pt on workpc. Fixture: internal/router/eval/ru_routing_v1.json, 96 cases, 33 carrying a destination. Runner: make t PKG=./internal/router/eval/ RUN=TestONNXRoutingHeads.

The four heads of V-661 ran nowhere. This is the number they score through the Go cascade. Same fixture and same grammars as TestONNXBaseline, and only the middle stage varies.

Headline

	classifier + ONNX	heads + classifier	gemma-4-12b cascade
intent	75.0% (72/96)	96.9% (93/96)	84.4%
destination	33.3% (11/33)	75.8% (25/33)	72.7%
false clarify	0	1	2
missed clarify	8	1	1
p50	24.5ms	27.9ms	329ms

A 118M encoder beats the 12B teacher it was distilled from. It wins on both halves of the route, at a twelfth of the latency. The workstation stays the better phraser and is no longer the better router.

The p50 is not the heads. Most of it is the classifier's own embedder pass on the turns the heads decline, plus process warm-up on the first case. The heads' own forward pass measures 7.3ms on workpc.

Two defects were in the way, and the first was not in the heads

The tokenizer read every long word backwards. encodeWord backtracks the Viterbi path from the end of a word and prepends each piece. That puts them back in reading order, and a second reverse after the loop undid it. So query: вода tokenized to [0 12 1294 41 12489 2] where the reference tokenizer gives [0 41 1294 12 12489 2].

It was found here and only here. The heads were trained through transformers and are read through the hand-written tokenizer. So a mismatch shows up as a score far below what Python measured on the same weights. Nothing else in the suite compares the two.

Measured on the recall fixture, same 27 cases either way:

	reversed	fixed
recall@1	70.4% (19/27)	77.8% (21/27)
recall@3	85.2% (23/27)	96.3% (26/27)
answered after gate	63.0%	66.7%
wrong note on top	8	6
false recall	0/5	1/5

The classifier barely moved, 76.0% to 75.0%, and destination 36.4% to 33.3%. Both are one case on 96 and neither is a finding. Seeds and queries were mangled the same way, so cosine survived it. Recall is where it cost, because a stored passage and a live query are different lengths and break differently.

The one new false recall is the honest cost and it is not being hidden. A sharper embedder scores every candidate higher, including the ones that should have stayed under the gate. That is the same trade 2026-08-04-recall-e5-small.md recorded when e5-small replaced MiniLM.

The embedder id now carries a tokenizer revision, model_quantized@384/tok2. Stored vectors were written under rev 1 and no longer sit in the same space as a query embedded now. The model file's name never moved, so nothing would have triggered ReembedAll. On the box the marker fired on start, and the re-embed rewrote 65 notes and 19 facts in 5 seconds.

The clarify head was being thrown away. It was read only when the intent head cleared its own threshold. That cost 6 of the 8 ambiguous cases. вода reads as intent act at 0.233 and clarify at 0.983. Burying that handed the turn to the classifier, which routed it confidently and never asked. The clarify head answers a different question, which is whether there is enough here to act on at all. So it decides on its own and decides first.

	intent-gated	clarify decides first
intent	90.6%	96.9%
missed clarify	7	1
false clarify	0	1

The threshold is measured, not chosen

Max softmax over the intent head, on the 88 cases carrying an intent:

threshold	kept	accuracy kept	wrong kept	right dropped
0.5	84	96.4%	3	2
0.6	81	97.5%	2	4
0.7	75	97.3%	2	10
0.8	64	96.9%	2	21
0.9	46	100.0%	0	37

0.6 is the knee. Every value from 0.7 to 0.85 drops right answers and keeps the same two wrong ones. 0.9 is the only value that clears them, and it costs 37 correct routes to do it.

Quantization was measured and rejected

build	size	intent	destination	p50
fp32	470MB	83/88 (94.3%)	28/33 (84.8%)	7.3ms
int8	118MB	79/88 (89.8%)	26/33 (78.8%)	4.0ms
fp16	235MB	will not load	—	—

Python numbers, on the heads alone rather than through the cascade. int8 costs 4.5 points of intent and 6 of destination to save 3ms. The cascade around it has a p50 over a second when the resident model answers. The fp16 graph is broken: convert_float_to_float16 leaves a Cast node emitting float16 where the graph expects float, and onnxruntime refuses the session. It was not worth fixing.

The exporter also had to be told to write one file. It splits weights into a .onnx.data sidecar by default. This onnxruntime resolves that path against the process working directory rather than the model. A split graph loads from one directory only.

What is still wrong

Four of the eight destination misses are calendar. Training cannot move them. The possessive agenda rules claim those cases at stage 0 and name nothing on purpose. That caution was free while nothing downstream could name anything either. It has now cost four points in three separate measurements. The call is the owner's and it is still open.

The slot head is exported and not read. Slots come from the stage-2 extractor. Mapping BIO tags back to text needs character offsets the unigram tokenizer does not keep, which is its own piece of work.

поужинал is a false clarify, which is the same defect thinSingleToken was narrowed for on 2026-08-01, arriving now from a different direction.

On the box

Deployed to homesrv the same day. voice: routing heads loaded on start, and /trace shows routing-heads winning or thinning every turn. The resident model and the classifier are both marked never asked. Live probes:

что такое TCP?                 -> kiwix     a real definition
кто такой Линус Торвальдс?     -> kiwix     a real answer
во сколько я лёг вчера         -> personal  не нашла у тебя такой записи
вода                           -> thinned to clarify at 0.233 / 0.983

A missing or broken weights file logs and leaves the heads nil, which is byte-for-byte the cascade that shipped before this.