claude
dc73fc4e31
router/semantic: slice 20 pretrained-rubert-tiny pragmatics specialist tooling + tokenizer audit gate
...
cmd/semantic-router-experiment/slice20_pretrained.py (516 lines, over
300-line hook): tokenize/grouped/lfo/metrics/runtime/onnx/ceiling
subcommands, MAX_LEN=25, lr grid 1e-5/2e-5/5e-5 x seeds 42/17/7, 4-thread
fp32, early stop on val PR-AUC. slice20_audit.py: WordPiece audit gate
(UNK rate 0.0026, seq p99 19, no loss above 96). artifacts under /tmp/mvn-s20/.
2026-09-07 21:40:49 +04:00
claude
7d31de5a6b
router/semantic: slice 19 from-scratch sequence pragmatics specialist tooling + eval index
2026-09-07 17:58:09 +04:00
claude
f2b65cd5d2
router/semantic: slice 18 sparse lexical action-gate probe tooling + eval index
2026-09-07 15:35:41 +04:00
claude
e80d45f4b3
router/semantic: slice 17 nonlinear MLP probe tooling + eval index
2026-09-07 15:23:39 +04:00
claude
774c0660b0
router/semantic: slice 16 diagnostic — action/non-action boundary analysis
...
Experiments:
- Residual-only OOF safety metrics
- False-action decomposition by semantic family (7 families, 24 split groups)
- Paired e5 geometry (gap=0.010 between action and capability question)
- Binary action probe (PR-AUC=0.707, ROC-AUC=0.863)
- Cost-sensitive linear classification
- Expanded action-threshold sweep (7 thresholds)
- Voice-like punctuation stress evaluation
- e5 + tiny structural features
- Four-hypothesis comparison table
Verdict: e5 representation is the primary bottleneck. Action seeds and
capability questions are nearly indistinguishable in embedding space.
2026-09-07 14:02:00 +04:00
claude
9df223907b
router/semantic: sklearn grouped CV linear head experiment (slice 15)
...
Python experiment: 6-class logistic regression over frozen e5 embeddings,
grouped cross-validation, calibration, abstention curves, contrast-family
analysis, action-threshold sweep.
2026-09-07 13:52:12 +04:00
claude
f8ec77de8a
router/semantic: e5 embedding cache for linear head experiment (slice 15)
...
Go tool that computes e5-small embeddings for every corpus row,
assigns frozen/dev split and grouped CV folds, outputs JSON.
2026-09-07 13:51:54 +04:00
claude