Maven/docs/plans/26-capability-ledger-and-baseline.md

# 26. The capability ledger and the empirical baseline

V-725. Written 2026-08-26 against `5cae33a`, filed after the fact.

Frozen once session 1 starts. Changes of mind go to the handoff, not back here.

## The question this answers

How much of the Maven described by `docs/spec.md` exists today, and where does
the current architecture support or obstruct that target.

## What this is not

- Not a redesign and not another architecture pass.
- No fixes. This pass produces the map used to decide what to build next.
- `docs/spec.md` is not edited. It is the authoritative statement of intent.
- A package, type or function existing is not proof a capability works.

## The causal order

```text
spec says what should happen
        ↓
the empirical run says what actually happens
        ↓
code and architecture explain why
        ↓
priority says what to fix
```

The reverse order is what the capability spec was written to prevent. Its
predecessor audit had a green suite while 22 of 39 capabilities were not live.

## The target Maven

A persistent personal agent. Request and response is one input mode, not the
shape of the system. The target is drawn around capabilities, never packages.

Eight domains:

| domain | holds |
| --- | --- |
| perception/context | presence, device state, time, environment, current activity, conversation state |
| memory/model of you | facts, preferences, relationships, history, commitments, notes, routines, inferred context |
| attention | what matters now, unfinished things, reminders, deadlines, anomalies, things worth surfacing |
| deliberation | interpret requests, resolve ambiguity, connect current events to past context, decide to act, query, ask or wait |
| initiative | nudges, follow-ups, missed-task recovery, opportunistic suggestions, background checks |
| action | local tools, home automation, other services, information retrieval, document and email workflows |
| interaction | voice, web, notifications, telegram or matrix, with continuity across surfaces |
| governance | permissions, confidence, confirmation, privacy, reversibility, non-autonomous propose-then-enable |

A ninth bucket, `operations`, holds what the spec files under Operations. It is
infrastructure, not agent behavior, and a forced fit would hide that.

## The five vertical slices

The slices the field probes exercise. Each is end to end, never one turn.

1. **reminder**: create, clarify time, correct, persist, fire, acknowledge.
2. **fact/note**: record, correct or supersede, retrieve later, answer with current truth.
3. **query**: understand, choose source, preserve follow-up context, answer.
4. **action**: understand target, confirm only when needed, execute, report the actual result.
5. **proactive**: detect the condition, decide whether to interrupt, deliver on the right channel.

## Three sessions

The baseline is perishable. It measures one model on one config, so a config
change invalidates it. The viewer is presentation over data that does not exist
yet.

| session | produces |
| --- | --- |
| 1 | `docs/capabilities/ledger.yaml` target side, then the empirical baseline as a dated eval |
| 2 | implementation mapping, `docs/capabilities/invariants.md`, `docs/capabilities/gaps.md`, the ranked list |
| 3 | the `capabilities` mode in the architecture viewer |

## Session 1, step 1: the skeleton

Mechanical extraction from `docs/spec.md` into `docs/capabilities/ledger.yaml`.
No implementation judgment, no verification status, no ranking.

Preserved per capability: id, section, `v1` or `deferred` scope, state
references, every DoD criterion, scenario names, and the findings the spec
already carries.

Added at extraction time:

- `domain`, at most two, primary first. More than two means it is two capabilities.
- A stable criterion id, `<capability-slug>#<4 hex of the criterion text>`. A
  positional id shifts when the spec gains a criterion, and the frozen eval
  would then cite the wrong one.

The 51-row capability-to-domain table goes to the owner once, before any probe
runs.

The five deferred capabilities enter the skeleton and are not probed. Their
criteria read `untested`, reason `deferred past v1`.

## Session 1, step 2: the empirical baseline

### Where it runs

Against the live `maven-mavend-1`, then `mavend -wipe -confirm-wipe` after the
run. A dry run without the confirm flag comes first and its row counts are the
eval's first artifact.

The live resident model is `maven-instruct-b2-Q4_K_XL`, not the Qwen3-1.7B that
`CLAUDE.md` still names. `deploy/mavend.json` carries the switch uncommitted.
Every number in the baseline is attributable to b2.

### Transport

`POST /api/chat` on `127.0.0.1:9201` drives a real turn through the deployed
stack. The reply comes back in the redirect `Location` as `?q=…&r=…&t=<trace id>`.
No new binary and no audio needed, and it covers the web reach.

### Readback

`cmd/e2eprobe`, built in a one-off `golang:1.25-trixie` container and copied in
with `docker cp`. Both images are trixie, so glibc matches. It gives typed IPC
JSON for `decisions`, `facts`, `notes`, `reminders`, `nudges`, `tools` and
`delivery-attempts`.

Not the sqlite file. The plaintext copy at `/dev/shm/maven-plain.db` bypasses the
contract the ledger exists to measure. Not the mavweb pages either: HTML is a
second-hand rendering.

### Two probe origins

- `dod:<criterion id>`, derived from the ledger. Only these set a DoD verdict.
- `field:<slice>`, the owner's real week. 25 multi-turn probes, drafted from the
  five slices, the three ugly conversations verbatim, the five existing
  scenarios, and what the box is configured for.

A failing `field` probe becomes an unresolved product question in
`invariants.md` where the spec never defined the behavior. It becomes a
missing-criterion finding against `docs/spec.md` where the spec should have
covered it. Neither edits the spec in this pass.

### Three evidence kinds

| kind | what it is | may set `pass` |
| --- | --- | --- |
| `live` | `e2eprobe` and `/api/chat` against the deployed build, real model, real store rows | yes |
| `replay` | the scenario harness, model faked, clock controlled | no |
| `simulated` | faked clock, real code path, no deployed process | no |

A green `replay` with no live probe reads `verified: untested`, with the replay
recorded as implementation evidence. The scenario harness scripts both `route`
and `reply`, so a pass proves the wiring around the model and not the turn.

`simulated` is allowed only where the trigger is anchored to a wall-clock hour
or date the pass cannot reach. The morning digest hour, the 14-day trace
retention sweep, a weekly routine. Everything else uses a compressed live
horizon, because `TickInterval` defaults to 60s and no override is configured.

### Reaches

`reachable` is per reach. Inbound: web, ipc. Outbound: `ntfysink`,
`telegramsink`, `voicesink`.

One delivery per outbound reach, three notifications total, each carrying a fixed
marker so a probe is not read as a real nudge. `reachable: pass` needs the
`delivery-attempts` row plus the owner confirming arrival. `voicesink` needs a
connected client on workpc, and records `blocked, no listener` when there is
none.

### Per-criterion verdict

Every DoD criterion gets `pass`, `fail`, `blocked`, `untested` or `unknown`, and
a reason distinguishing why it is not passing: code missing, wiring missing,
configuration missing, deployment missing, external dependency unavailable,
scenario missing, scenario fails, or implementation exists with no runtime proof.

Where a scenario covers a criterion, the exact step or assertion is named. Where
none exists, the spec's `(to write)` state stands. No evidence is invented.

Every generated claim carries `path:line`, or runtime or scenario evidence.
`unknown` where evidence is insufficient.

### Stop condition

Session 1 ends when every v1 DoD criterion carries a verdict with evidence
attached, and the eval is written. No mapping, no gaps, no ranking.

A baseline that stops halfway leaves a ledger that looks measured and is not.

## Session 2: the explanation

Implementation mapping onto the ledger, as independent dimensions, never
collapsed into one `implemented` boolean: `designed`, `code_present`, `wired`,
`configured`, `deployed`, `reachable`, `verified`.

`docs/capabilities/invariants.md` extracts the cross-cutting rules the 51
capabilities imply and do not state. Continuity across turns and across
reaches, memory and correction semantics, current context and presence,
proactive attention, interruption policy, clarification and follow-up ownership,
degradation and honesty, authority and confirmation, privacy boundaries,
learning from outcomes, capability composition, persistence across restart.

Each invariant is marked `explicit`, `implied` or `unresolved`, with evidence.
Nothing desired is invented where the sources do not define it. An unresolved
invariant is a product question, not a defect.

`docs/capabilities/gaps.md` compares responsibilities, never package names.
Classes: capability missing, capability partial, capability exists but
unreachable, capability exists but unverified, duplicated mechanism, missing
shared mechanism, current architecture conflicts with target behavior, and
architecture concern with no current product impact.

Every architecture concern names the capability or invariant it affects. One
affecting none is marked non-blocking cleanup explicitly.

### Prioritization

One final list, ranked:

1. Prevents intended everyday use today.
2. Makes existing behavior incorrect or unreliable.
3. Blocks multiple capabilities.
4. Prevents verification.
5. Architectural cleanup with no present user impact.

An unwired or unreachable future defect never outranks a live user-visible
failure because its architecture is ugly.

## Session 3: the viewer

A `capabilities` mode in the architecture viewer. The primary view is a matrix of
capability against `designed`, `coded`, `wired`, `configured`, `deployed`,
`reachable`, `verified`.

Clicking a capability shows the target DoD, implementation evidence,
verification evidence, affected components, scenarios, blockers and unresolved
product questions.

An `invariants` view shows which components participate in each cross-cutting
rule. Target, implementation and runtime verification are distinguished visually.

## Where the artifacts live

`docs/capabilities/` becomes a declared tier in `docs/CLAUDE.md`: generated,
regenerated from `docs/spec.md` plus a named eval, never hand-edited. The same
row legitimises `docs/architecture/`, which is currently an undeclared
directory.

The empirical run lands as one dated eval, `docs/evals/2026-08-26-capability-baseline.md`,
frozen on the day. Every `verified` cell in the ledger cites it by path, so no
status is an opinion.

## Decided, do not re-ask

- Three sessions, split on the causal order above.
- Target side of the ledger first, with no implementation or verification status.
- Two verified columns plus `simulated`. Only `live` sets `pass`.
- Probes run against the live container, with the store wiped afterwards.
- Row counts are read with a `mavend -wipe` dry run before anything else.
- Readback is `e2eprobe`, not sqlite and not the web pages.
- Both probe origins, `dod:` and `field:`, tagged.
- Deferred capabilities get the skeleton only.
- All three outbound reaches get one real delivery.
- Domain is a second axis beside the spec section.
- No fixes land in this pass.

## Findings recorded while planning

Live and reproduced, not inferred. They belong to the baseline, not to this plan.

- `POST /api/chat` with `что ты помнишь обо мне?` answered
  `Я не знаю вас или как вы себя называете`. Formal `вас` and `вы` on the wire,
  where `CLAUDE.md` requires informal singular `ты`. The phrasing eval passes.
- Vikunja was never down. `vikunja-mcp` publishes `127.0.0.1:9100` only, so the
  LAN address never answers from workpc. Three sessions read a refused
  connection as an outage and filed nothing.
- The deployed resident model is `maven-instruct-b2-Q4_K_XL`. `CLAUDE.md` still
  names Qwen3-1.7B.