Commit Graph

6 Commits

Author SHA1 Message Date
claude cbac8b9c4f router/semantic: experiment reports (slice 15)
- 2026-09-07-linear-e5-router-experiment: 136-example baseline (superseded)
- 2026-09-07-expanded-corpus-linear-head-experiment: 3025-example expanded (live)
- CLAUDE.md: index updates for both evals
2026-09-07 13:52:54 +04:00
claude 7f804b84e7 Declare the two generated doc tiers and index their evals (V-725)
docs/CLAUDE.md defined four tiers and docs/capabilities/ and docs/architecture/
were neither of them. Both are now declared as generated: rebuilt from a source,
never corrected in place. A wrong row in either is a bug in the generator or in
one of its hand-written inputs, and editing the output makes the next rebuild
silently undo the fix.

The eval index gains the 2026-08-19 CPT+SFT measurement and the 2026-08-26
baseline, and marks the 2026-08-13 capability audit superseded. The 2026-08-19
file lands with them: it is the direct evidence that the deployed
maven-instruct-b2 routes better than Qwen3-1.7B and cannot hold a Russian
sentence, which is the first item on the gaps.md priority list.

Its header records Vikunja as returning 503. That reading was wrong and the
2026-08-26 baseline says so, but a dated eval is not edited after the day.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 12:45:54 +04:00
claude db50c34c23 Record the recall measurement and the two subsystem contracts (V-719)
docs/evals/2026-08-15-locative-answerability-verifier.md rules the
resident model out as a recall answerability verifier. Its constrained
output was syntactically reliable and neither semantically reliable nor
isolated from instructions inside stored memory: five false accepts out of
32 held-out cases, two of them prompt injections carried in the memory
text, all five identical across three fixed-seed repeats.

design.md carries the reminder row as it now is, one-shot or recurring,
with the outbox and the cancellation invariants. routing.md carries the
new stage 0 frames and the cancellation rung. deployment.md carries the
/reminders contract. The assistant_workday scenario exercises the turn
sequence end to end.

--no-verify: master is the working branch this session by the owner's call.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-15 17:19:38 +04:00
claude 28c2ffb84f Make aggregate ONNX gates execute for real
Reference-count the process-global ONNX Runtime across embedder and routing-head sessions, make close idempotent, and require named proof that both aggregate routing gates executed rather than self-skipped (V-716). Owner explicitly requested direct commits to master.
2026-08-13 03:03:25 +04:00
claude 8015fdbb79 Harden semantic boundaries and repair dialogue state
Replace nearest-neighbour personal routing with a frozen class-balanced linear head measured on historical, stratified, cross-validation, holdout, and fresh challenge gates (V-702). Close the four repair handoff holes, preserve nested clarification flows, and route Russian possession statements through structural grammar rather than lexical exceptions (V-573). Owner explicitly requested direct commits to master.
2026-08-13 03:00:31 +04:00
claude fdee3de724 Index plans and evaluations by lifecycle (V-674) 2026-08-13 01:27:14 +04:00