Declare the two generated doc tiers and index their evals (V-725)

docs/CLAUDE.md defined four tiers and docs/capabilities/ and docs/architecture/
were neither of them. Both are now declared as generated: rebuilt from a source,
never corrected in place. A wrong row in either is a bug in the generator or in
one of its hand-written inputs, and editing the output makes the next rebuild
silently undo the fix.

The eval index gains the 2026-08-19 CPT+SFT measurement and the 2026-08-26
baseline, and marks the 2026-08-13 capability audit superseded. The 2026-08-19
file lands with them: it is the direct evidence that the deployed
maven-instruct-b2 routes better than Qwen3-1.7B and cannot hold a Russian
sentence, which is the first item on the gaps.md priority list.

Its header records Vikunja as returning 503. That reading was wrong and the
2026-08-26 baseline says so, but a dated eval is not edited after the day.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-26 12:45:54 +04:00
parent bae81b66c8
commit 7f804b84e7
3 changed files with 122 additions and 0 deletions
+9
View File
@@ -45,6 +45,7 @@ A pair in `docs/routing.md` went stale unnoticed. Its source predated the
| [MASSIVE Russian warm-start for the routing heads](2026-08-08-massive-warm-start.md) | live |
| [gemma-4-E4B against gemma-4-12B on the routing fixture](2026-08-09-e4b-vs-12b-routing.md) | live |
| [The classifier baseline after the tokenizer fix](2026-08-11-classifier-baseline-after-tokenizer-fix.md) | live |
| [The CPT+SFT Qwen3-1.7B routes better and cannot hold a sentence](2026-08-19-maven-model-cpt-sft.md) | live |
`docs/routing.md` holds the arm table these feed. Cite from there, not from here.
@@ -90,6 +91,7 @@ A pair in `docs/routing.md` went stale unnoticed. Its source predated the
| [Talk fixture against the resident model](2026-08-05-talk-fixture-resident.md) | live |
| [Talk temperature sweep: Qwen3-1.7B, 4 temperatures times 3 runs](2026-08-05-temperature-sweep.md) | live |
| [gemma-4-E4B on the phrasing and talk fixtures](2026-08-09-e4b-phrasing.md) | live |
| [The CPT+SFT Qwen3-1.7B routes better and cannot hold a sentence](2026-08-19-maven-model-cpt-sft.md) | live |
## World: search and Kiwix
@@ -131,6 +133,13 @@ evidence and is not summarised anywhere else.
| measurement | state |
| --- | --- |
| [Repository deep-audit report](2026-08-10-repo-audit.md) | live |
| [What 39 capabilities actually did on the box](2026-08-13-capability-audit.md) | superseded by 2026-08-26 |
| [Capability baseline: what the deployed Maven actually does](2026-08-26-capability-baseline.md) | live |
The 2026-08-26 baseline scores all 146 v1 DoD criteria in `docs/spec.md` and is
cited by every `verified` cell in `docs/capabilities/ledger.yaml`. It measures
`maven-instruct-b2-Q4_K_XL`, not the Qwen3-1.7B `CLAUDE.md` names, and it
expires when the model or `deploy/mavend.json` moves.
Its open findings live in `docs/caveats/`, one entry each with a revisit
trigger. Read the index there, not this file, for what is still broken.