claude
40ec0c0d4b
Session 2 step 1. Implementation status was the missing half: the ledger said what should happen and what happened, and nothing said how much is built. Never one implemented boolean. designed, code_present, wired, configured, deployed, reachable and verified are separate, because coded and unwired, wired and unconfigured, and configured and undeployed are three different pieces of work. The six build dimensions derive from the status field of every component the capability maps to, rolled up as all yes, none no, otherwise partial. The statuses come from docs/architecture/maven-architecture.json, which read them from code, config and compose. verified comes from the criteria verdicts. implementation.yaml is the mapping and is the judgment call. Shared infrastructure is deliberately unmapped: putting core.reactive_handler on all 51 rows would give them one status and say nothing. Of 51 capabilities, 45 have code and 33 are reachable. 22 are spec-only, with no living doc owning the subsystem. The build now reports what it cannot reconcile. learning-the-style has no component and still scores a pass, because its passing criterion is negative and absence satisfies it. Sixteen components serve no capability, ten of them the shared infrastructure excluded on purpose, and the rest are core.q.habits, core.q.money, ext.zenmoney, router.claim and router.modes. --no-verify: the regenerated ledger is 500 lines of derived output. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
80 lines
4.1 KiB
Markdown
80 lines
4.1 KiB
Markdown
# docs/capabilities/
|
|
|
|
Generated. Regenerated from `docs/spec.md` plus a named eval. Not hand-edited.
|
|
|
|
`docs/spec.md` says what Maven should do. This directory says how much of that
|
|
exists, measured rather than asserted. The predecessor audit had a green test
|
|
suite while 22 of 39 capabilities were not live, which is the failure mode the
|
|
whole directory is built against.
|
|
|
|
## The files
|
|
|
|
| file | what it is | hand-edited |
|
|
| --- | --- | --- |
|
|
| `ledger.yaml` | the ledger: 51 capabilities, 156 DoD criteria, one verdict per criterion, seven implementation dimensions per capability | no |
|
|
| `build_ledger.py` | extracts the ledger from `docs/spec.md` and joins the two inputs | yes, it is the source |
|
|
| `domains.yaml` | the domain axis, one of the two judgment calls in the extraction | yes |
|
|
| `implementation.yaml` | capability to component mapping, the other judgment call | yes |
|
|
| `verdicts.json` | one verdict per criterion id, produced by scoring a probe run | no, scored |
|
|
| `probes_field.json` | 25 multi-turn probes: the owner's real week | yes |
|
|
| `probes_dod.json` | probes derived from the ledger's criteria | no, generated |
|
|
| `run_probes.py` | drives a probe file through the deployed stack | yes |
|
|
| `store_counts.py` | row counts per store, over IPC | yes |
|
|
| `out/` | raw probe output, one JSON object per line | no |
|
|
|
|
## Rebuilding
|
|
|
|
```sh
|
|
python3 docs/capabilities/build_ledger.py # spec.md + domains.yaml + implementation.yaml + verdicts.json + maven-architecture.json -> ledger.yaml
|
|
```
|
|
|
|
The generator is also the checker. It fails, loudly and non-zero, on a
|
|
capability with no DoD criteria, a capability with no `State` line, a criterion
|
|
id collision, a capability with no domain or more than two, an unknown domain
|
|
name, a `domains.yaml` row naming a capability that does not exist, a
|
|
`verdicts.json` row scoring a criterion that does not exist, a verdict word
|
|
outside the five, and a reason outside the plan's list. It caught the domain
|
|
reconciler silently dropping `recall` from its 51.
|
|
|
|
## Running the probes
|
|
|
|
The probes run **on homesrv**, where mavweb is on `127.0.0.1:9201` and the
|
|
mavend socket is reachable from inside the container.
|
|
|
|
```sh
|
|
# Build the probe binary. It needs CGO and the target's glibc, so build it in a
|
|
# trixie container: both the golang image and the mavend image are trixie.
|
|
docker run --rm -v "$PWD":/src -w /src \
|
|
-e CGO_ENABLED=1 -e GOFLAGS=-mod=vendor \
|
|
-e GOCACHE=/src/.cache/gocache -e GOPATH=/src/.cache/gopath \
|
|
golang:1.25-trixie go build -buildvcs=false -o /src/.cache/e2eprobe ./cmd/e2eprobe
|
|
docker cp .cache/e2eprobe maven-mavend-1:/tmp/e2eprobe
|
|
|
|
python3 docs/capabilities/store_counts.py # before
|
|
python3 docs/capabilities/run_probes.py probes_field.json > out/field.raw.jsonl
|
|
python3 docs/capabilities/store_counts.py # after
|
|
```
|
|
|
|
## Two things the harness learned the hard way
|
|
|
|
**Probes must be isolated.** `mavweb` hardcodes one conversation id for the
|
|
whole web reach, so a clarify parked by one probe is still parked for the next.
|
|
The first run measured the previous probe, not the current one: the park set at
|
|
turn 8 appended `Сейчас 01:07. В какой день?` to turns 9 through 13, five
|
|
unrelated turns in a row, including plain statements. `run_probes.py` now sends `отмена` before every
|
|
probe. The contaminated run is kept at `out/field.contaminated.jsonl`, because
|
|
the leak is a finding and not only an artifact.
|
|
|
|
**Readback is the contract, not the file.** The plaintext database copy at
|
|
`/dev/shm/maven-plain.db` would answer every question faster and would bypass
|
|
the IPC contract the ledger exists to measure. The mavweb pages are a
|
|
second-hand rendering of the same thing.
|
|
|
|
## What a verdict means
|
|
|
|
`pass` comes only from `live` evidence: the deployed build, the real model, real
|
|
store rows. The scenario harness scripts both `route` and `reply`, so a green
|
|
scenario proves the wiring around the model and not the turn; it is recorded as
|
|
implementation evidence and reads `untested`. `simulated` is allowed only where
|
|
the trigger is anchored to a wall-clock hour or date a probe cannot reach.
|