router/semantic: slice 19 report — correct claim head and prose to the seed-stable fail-rate cut

2026-09-07 17:59:10 +04:00
parent 5397f09785
commit cf2af6ce1e
1 changed files with 3 additions and 2 deletions
@@ -1,4 +1,4 @@
-# Slice 19: from-scratch tiny sequence encoders halve the sparse gate's capability-question fail rate, and still land near chance on the leave-generator-out pragmatics split
+# Slice 19: from-scratch tiny sequence encoders beat sparse and e5 on the capability-question holdout (seed-stable FA 0.48-0.55 vs 0.667 / ~0.98) yet cap at ~50% accuracy on the leave-generator-out pragmatics split — a pretrained prior is the next step

 Date: 2026-09-08 · Task: V-726 (slice 19) · Box: homesrv, Ryzen 5 5600U, 13 GB, CPU-only (this is the production machine) · Build: `cmd/semantic-router-experiment/slice19_*.py`, torch 2.14.0+cpu in `/tmp/mvn-exp-venv`, no GPU.

@@ -152,7 +152,8 @@ positive — every row here must *not* trip the gate). Threshold 0.5, seed 17:
 | char_cnn_medium | 123 | 0.976 | 0.024 |

 Against the sparse gate (0.667 FA → 0.333 accuracy) the leading from-scratch
-configs halve the fail rate. Against e5 (1.000 / 0.976) the win is decisive —
+configs cut the fail rate by a quarter to a third on seed-stable seeds. Against
+e5 (1.000 / 0.976) the win is decisive —
 the frozen embedder genuinely cannot separate a held-out capability question
 from its executable sibling.