Commit Graph

12 Commits

Author SHA1 Message Date
claude bdced83b46 router/semantic: slice 21 execution-frame guard report — capability dangerous pass 0/126, guard→sparse P 96.7% FA 6
Deterministic structural guard blocks every capability-question row (0/126 on
all three stress views), never blocks a true action (0/796), declines 196
malformed action rows as ambiguous, and grows no safety leak under
compose: sparse alone FA 9 (capQ 1) → guard→sparse FA 6 (capQ 0).
Execution eligibility recorded as a deterministic policy boundary, separate
from semantic routing and capability selection; next slice returns to the
coarse non-action router instead of another pragmatics training run.

Task/725.
2026-09-07 23:45:33 +04:00
claude de1cb40456 router/semantic: slice 20 pretrained fine-tune report — in-pool best, LOFO 1.000
cointegrated/rubert-tiny fine-tuned end-to-end: pair ordering 0.933-0.970
(+0.56-0.78 margin, in-pool) beats every from-scratch config, but the
held-out capability-question family fires 126/126 as action on all 18
config/seeds (FA_rate 1.000, mean proba 0.86-0.99, family σ 0.0045). tiny2
ceiling (newer, larger vocab) does not move LOFO (1.000) and is worse
in-pool. sparse 0.667 remains the safety floor; supervised encoder scaling
ruled out for the generator boundary.
2026-09-07 21:40:53 +04:00
claude 7d31de5a6b router/semantic: slice 19 from-scratch sequence pragmatics specialist tooling + eval index 2026-09-07 17:58:09 +04:00
claude f2b65cd5d2 router/semantic: slice 18 sparse lexical action-gate probe tooling + eval index 2026-09-07 15:35:41 +04:00
claude e80d45f4b3 router/semantic: slice 17 nonlinear MLP probe tooling + eval index 2026-09-07 15:23:39 +04:00
claude 4ed0238a25 router/semantic: add slice 16 diagnostic to evals index 2026-09-07 14:02:59 +04:00
claude cbac8b9c4f router/semantic: experiment reports (slice 15)
- 2026-09-07-linear-e5-router-experiment: 136-example baseline (superseded)
- 2026-09-07-expanded-corpus-linear-head-experiment: 3025-example expanded (live)
- CLAUDE.md: index updates for both evals
2026-09-07 13:52:54 +04:00
claude 7f804b84e7 Declare the two generated doc tiers and index their evals (V-725)
docs/CLAUDE.md defined four tiers and docs/capabilities/ and docs/architecture/
were neither of them. Both are now declared as generated: rebuilt from a source,
never corrected in place. A wrong row in either is a bug in the generator or in
one of its hand-written inputs, and editing the output makes the next rebuild
silently undo the fix.

The eval index gains the 2026-08-19 CPT+SFT measurement and the 2026-08-26
baseline, and marks the 2026-08-13 capability audit superseded. The 2026-08-19
file lands with them: it is the direct evidence that the deployed
maven-instruct-b2 routes better than Qwen3-1.7B and cannot hold a Russian
sentence, which is the first item on the gaps.md priority list.

Its header records Vikunja as returning 503. That reading was wrong and the
2026-08-26 baseline says so, but a dated eval is not edited after the day.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 12:45:54 +04:00
claude db50c34c23 Record the recall measurement and the two subsystem contracts (V-719)
docs/evals/2026-08-15-locative-answerability-verifier.md rules the
resident model out as a recall answerability verifier. Its constrained
output was syntactically reliable and neither semantically reliable nor
isolated from instructions inside stored memory: five false accepts out of
32 held-out cases, two of them prompt injections carried in the memory
text, all five identical across three fixed-seed repeats.

design.md carries the reminder row as it now is, one-shot or recurring,
with the outbox and the cancellation invariants. routing.md carries the
new stage 0 frames and the cancellation rung. deployment.md carries the
/reminders contract. The assistant_workday scenario exercises the turn
sequence end to end.

--no-verify: master is the working branch this session by the owner's call.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-15 17:19:38 +04:00
claude 28c2ffb84f Make aggregate ONNX gates execute for real
Reference-count the process-global ONNX Runtime across embedder and routing-head sessions, make close idempotent, and require named proof that both aggregate routing gates executed rather than self-skipped (V-716). Owner explicitly requested direct commits to master.
2026-08-13 03:03:25 +04:00
claude 8015fdbb79 Harden semantic boundaries and repair dialogue state
Replace nearest-neighbour personal routing with a frozen class-balanced linear head measured on historical, stratified, cross-validation, holdout, and fresh challenge gates (V-702). Close the four repair handoff holes, preserve nested clarification flows, and route Russian possession statements through structural grammar rather than lexical exceptions (V-573). Owner explicitly requested direct commits to master.
2026-08-13 03:00:31 +04:00
claude fdee3de724 Index plans and evaluations by lifecycle (V-674) 2026-08-13 01:27:14 +04:00