Fix the floor corpus and re-measure the destination head (V-661)
The first 120 floor rows carried one sentence shape, because the generator varies a topic and ambiguity is not a topic. Rotating six shapes takes the floor 3/7 to 6/7 and the destination mean 75.8% to 80.8%. Calendar stays 3/6 at every seed. The possessive agenda rules claim those cases at stage 0 and name nothing, so no label reaches the head. Also corrects the floor-case count in three files: five of the seven are homelab, not six.
This commit is contained in:
@@ -248,9 +248,11 @@ suite to name the trade.
|
||||
|
||||
**Two of those heads are trained as of 08-08-2026, and they are not the three
|
||||
above** (V-661, `docs/evals/2026-08-08-routing-heads-two-head.md`). Intent and
|
||||
destination share one masked mean pool. Destination scores **26/33 (78.8%)** on
|
||||
the fixture. The classifier cascade scores 12/33 and the cascade with gemma-4-12b
|
||||
scores 24/33, so a 118M encoder beats the 12B teacher it was distilled from.
|
||||
destination share one masked mean pool. Destination scores a mean **80.8%** over
|
||||
three seeds, best **29/33 (87.9%)**. The classifier cascade scores 12/33 and the
|
||||
cascade with gemma-4-12b scores 24/33, so a 118M encoder beats the 12B teacher it
|
||||
was distilled from. Read the best run as one seed and not a headline, because one
|
||||
case is 3 points on a fixture this small.
|
||||
Recall is 15/15 and world is 5/5. Intent is 93.6% mean over three seeds. That is
|
||||
**not** comparable to the 76.0% and 84.4% those two arms scored: a softmax has no
|
||||
clarify class, so the head's fixture is the 88 cases carrying an intent.
|
||||
@@ -264,10 +266,13 @@ The MASSIVE warm-start of step 2 is worth nothing here. Stock e5-small ties it o
|
||||
intent and leads by a third of a case on destination. Nothing argues for keeping
|
||||
that step.
|
||||
|
||||
What the head gets wrong is the floor. It names a destination where the fixture
|
||||
says walk the chain, and it is confident doing it. `"почему сервер тормозит"`
|
||||
reads `world` at 0.80. The training floor is generated ambiguous questions and the fixture floor
|
||||
is homelab operations, which are not the same distribution.
|
||||
The floor was a corpus defect and it is fixed. The first 120 floor rows carried
|
||||
one sentence shape, so the head named a destination where the fixture says walk
|
||||
the chain. Rotating six shapes took the floor 3/7 to 6/7 and destination 75.8% to
|
||||
80.8%. What is left is calendar at 3/6 on every seed, which training cannot move:
|
||||
the possessive agenda rules claim those cases at stage 0 and name nothing, so no
|
||||
label reaches the head. That is the same trade V-660 flagged and it wants the
|
||||
owner's call.
|
||||
|
||||
**Nothing of this runs in Go.** The weights are `heads.pt` and `out/body_heads/`
|
||||
on workpc. Reaching the daemon needs an ONNX export and a caller. The resident
|
||||
@@ -449,11 +454,14 @@ rules deliberately do not name it. And **recall is 0/15, because nothing
|
||||
anywhere names it**. Those turns are still answered, since the chain walks
|
||||
recall early. Recall is the number the fourth head has to move.
|
||||
|
||||
Seven cases assert the floor and six of them are homelab operations. They
|
||||
Seven cases assert the floor and five of them are homelab operations. They
|
||||
cluster because `SourceRecall`, `SourceNetwork` and `SourceAttention` overlap on
|
||||
every question about the box. `mavpoll` writes its netdata and uptime-kuma
|
||||
every question about the box. The other two are `ru-query-005` and
|
||||
`ru-query-014`. No query source reads the reminder store, and a deadline could
|
||||
sit in tasks, the calendar or Praxis. `mavpoll` writes its netdata and uptime-kuma
|
||||
observations into the fact store recall reads. That is a finding about the enum,
|
||||
not a gap in the labelling.
|
||||
not a gap in the labelling. The owner confirmed all seven floor labels on
|
||||
08-08-2026, so they are a decision rather than an agent's guess.
|
||||
|
||||
`baselineGrammars` in `eval_test.go` mirrors `buildRouter` and had drifted:
|
||||
`WorldQueryGrammars` was wired into the daemon by V-655 and not into the mirror,
|
||||
|
||||
Reference in New Issue
Block a user