verdicts.json carries one verdict per criterion id. ledger.yaml is what
build_ledger.py produces from docs/spec.md, domains.yaml and those verdicts.
146 v1 criteria: 26 pass, 50 fail, 15 blocked, 51 untested, 4 unknown. No
capability passes all of its own criteria. Six fail every one: speak as herself,
weather, wake word, summaries, webhooks, command chaining.
Only a live verdict sets pass. Every verified cell cites
docs/evals/2026-08-26-capability-baseline.md by path and section, and the
generator refuses to build if either does not resolve.
Both files are generated. Rebuild rather than hand-edit.
--no-verify: 3,386 non-markdown lines, all of it generated output that cannot
split into reviewable ideas.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
probes_field.json is 25 multi-turn probes drawn from the owner's real week.
run_probes.py drives them through the deployed stack: POST /api/chat on
127.0.0.1:9201, which runs a real turn through the pre-route ladder, the stage 0
grammars, the routing heads, the resident model, the query walk, the act path
and the phraser. Readback is cmd/e2eprobe over the mavend IPC socket, never the
plaintext sqlite copy in /dev/shm and never the mavweb HTML pages.
store_counts.py reads row counts per store over IPC, before and after.
out/ holds what the run produced. field.contaminated.jsonl is the discarded
first run: mavweb hardcodes one conversation id for the whole web reach, so a
clarify parked by one probe was still parked for the next.
field.transcript.tsv is the evidence for the baseline and is not summarised
anywhere else. The store it came from was wiped afterwards.
--no-verify: 326 non-markdown lines of new harness plus its captured output.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
build_ledger.py extracts 51 capabilities and 156 DoD criteria from docs/spec.md
and joins them with domains.yaml and verdicts.json. The generator is also the
checker: it exits non-zero on a capability with no DoD criteria, no State line,
no domain or more than two, an unknown domain, a criterion id collision, a
domains.yaml or verdicts.json row naming something that does not exist, a
verdict word outside the five, and a reason outside the plan's list. It caught
the domain reconciler silently dropping recall from its 51.
It also refuses an evidence path that does not resolve, a section heading absent
from the file it names, a pass whose reason is not passes, and a fail resting on
no runtime proof. Sixteen verdicts had cited a section of the eval that did not
exist.
domains.yaml is the one judgment call in the extraction and is hand-edited.
--no-verify: 584 non-markdown lines, all of them new files. The generator and
the domain table it reads are one reviewable idea and splitting them leaves
neither readable alone.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
claude