Maven/docs/plans/26-capability-ledger-and-baseline.md

26. The capability ledger and the empirical baseline

V-725. Written 2026-08-26 against 5cae33a, filed after the fact.

Frozen once session 1 starts. Changes of mind go to the handoff, not back here.

The question this answers

How much of the Maven described by docs/spec.md exists today, and where does the current architecture support or obstruct that target.

What this is not

• Not a redesign and not another architecture pass.
• No fixes. This pass produces the map used to decide what to build next.
• docs/spec.md is not edited. It is the authoritative statement of intent.
• A package, type or function existing is not proof a capability works.

The causal order

spec says what should happen
        ↓
the empirical run says what actually happens
        ↓
code and architecture explain why
        ↓
priority says what to fix

The reverse order is what the capability spec was written to prevent. Its predecessor audit had a green suite while 22 of 39 capabilities were not live.

The target Maven

A persistent personal agent. Request and response is one input mode, not the shape of the system. The target is drawn around capabilities, never packages.

Eight domains:

domain	holds
perception/context	presence, device state, time, environment, current activity, conversation state
memory/model of you	facts, preferences, relationships, history, commitments, notes, routines, inferred context
attention	what matters now, unfinished things, reminders, deadlines, anomalies, things worth surfacing
deliberation	interpret requests, resolve ambiguity, connect current events to past context, decide to act, query, ask or wait
initiative	nudges, follow-ups, missed-task recovery, opportunistic suggestions, background checks
action	local tools, home automation, other services, information retrieval, document and email workflows
interaction	voice, web, notifications, telegram or matrix, with continuity across surfaces
governance	permissions, confidence, confirmation, privacy, reversibility, non-autonomous propose-then-enable

A ninth bucket, operations, holds what the spec files under Operations. It is infrastructure, not agent behavior, and a forced fit would hide that.

The five vertical slices

The slices the field probes exercise. Each is end to end, never one turn.

1. reminder: create, clarify time, correct, persist, fire, acknowledge.
2. fact/note: record, correct or supersede, retrieve later, answer with current truth.
3. query: understand, choose source, preserve follow-up context, answer.
4. action: understand target, confirm only when needed, execute, report the actual result.
5. proactive: detect the condition, decide whether to interrupt, deliver on the right channel.

Three sessions

The baseline is perishable. It measures one model on one config, so a config change invalidates it. The viewer is presentation over data that does not exist yet.

session	produces
1	docs/capabilities/ledger.yaml target side, then the empirical baseline as a dated eval
2	implementation mapping, docs/capabilities/invariants.md, docs/capabilities/gaps.md, the ranked list
3	the capabilities mode in the architecture viewer

Session 1, step 1: the skeleton

Mechanical extraction from docs/spec.md into docs/capabilities/ledger.yaml. No implementation judgment, no verification status, no ranking.

Preserved per capability: id, section, v1 or deferred scope, state references, every DoD criterion, scenario names, and the findings the spec already carries.

Added at extraction time:

• domain, at most two, primary first. More than two means it is two capabilities.
• A stable criterion id, <capability-slug>#<4 hex of the criterion text>. A positional id shifts when the spec gains a criterion, and the frozen eval would then cite the wrong one.

The 51-row capability-to-domain table goes to the owner once, before any probe runs.

The five deferred capabilities enter the skeleton and are not probed. Their criteria read untested, reason deferred past v1.

Session 1, step 2: the empirical baseline

Where it runs

Against the live maven-mavend-1, then mavend -wipe -confirm-wipe after the run. A dry run without the confirm flag comes first and its row counts are the eval's first artifact.

The live resident model is maven-instruct-b2-Q4_K_XL, not the Qwen3-1.7B that CLAUDE.md still names. deploy/mavend.json carries the switch uncommitted. Every number in the baseline is attributable to b2.

Transport

POST /api/chat on 127.0.0.1:9201 drives a real turn through the deployed stack. The reply comes back in the redirect Location as ?q=…&r=…&t=<trace id>. No new binary and no audio needed, and it covers the web reach.

Readback

cmd/e2eprobe, built in a one-off golang:1.25-trixie container and copied in with docker cp. Both images are trixie, so glibc matches. It gives typed IPC JSON for decisions, facts, notes, reminders, nudges, tools and delivery-attempts.

Not the sqlite file. The plaintext copy at /dev/shm/maven-plain.db bypasses the contract the ledger exists to measure. Not the mavweb pages either: HTML is a second-hand rendering.

Two probe origins

• dod:<criterion id>, derived from the ledger. Only these set a DoD verdict.
• field:<slice>, the owner's real week. 25 multi-turn probes, drafted from the five slices, the three ugly conversations verbatim, the five existing scenarios, and what the box is configured for.

A failing field probe becomes an unresolved product question in invariants.md where the spec never defined the behavior. It becomes a missing-criterion finding against docs/spec.md where the spec should have covered it. Neither edits the spec in this pass.

Three evidence kinds

kind	what it is	may set pass
live	e2eprobe and /api/chat against the deployed build, real model, real store rows	yes
replay	the scenario harness, model faked, clock controlled	no
simulated	faked clock, real code path, no deployed process	no

A green replay with no live probe reads verified: untested, with the replay recorded as implementation evidence. The scenario harness scripts both route and reply, so a pass proves the wiring around the model and not the turn.

simulated is allowed only where the trigger is anchored to a wall-clock hour or date the pass cannot reach. The morning digest hour, the 14-day trace retention sweep, a weekly routine. Everything else uses a compressed live horizon, because TickInterval defaults to 60s and no override is configured.

Reaches

reachable is per reach. Inbound: web, ipc. Outbound: ntfysink, telegramsink, voicesink.

One delivery per outbound reach, three notifications total, each carrying a fixed marker so a probe is not read as a real nudge. reachable: pass needs the delivery-attempts row plus the owner confirming arrival. voicesink needs a connected client on workpc, and records blocked, no listener when there is none.

Per-criterion verdict

Every DoD criterion gets pass, fail, blocked, untested or unknown, and a reason distinguishing why it is not passing: code missing, wiring missing, configuration missing, deployment missing, external dependency unavailable, scenario missing, scenario fails, or implementation exists with no runtime proof.

Where a scenario covers a criterion, the exact step or assertion is named. Where none exists, the spec's (to write) state stands. No evidence is invented.

Every generated claim carries path:line, or runtime or scenario evidence. unknown where evidence is insufficient.

Stop condition

Session 1 ends when every v1 DoD criterion carries a verdict with evidence attached, and the eval is written. No mapping, no gaps, no ranking.

A baseline that stops halfway leaves a ledger that looks measured and is not.

Session 2: the explanation

Implementation mapping onto the ledger, as independent dimensions, never collapsed into one implemented boolean: designed, code_present, wired, configured, deployed, reachable, verified.

docs/capabilities/invariants.md extracts the cross-cutting rules the 51 capabilities imply and do not state. Continuity across turns and across reaches, memory and correction semantics, current context and presence, proactive attention, interruption policy, clarification and follow-up ownership, degradation and honesty, authority and confirmation, privacy boundaries, learning from outcomes, capability composition, persistence across restart.

Each invariant is marked explicit, implied or unresolved, with evidence. Nothing desired is invented where the sources do not define it. An unresolved invariant is a product question, not a defect.

docs/capabilities/gaps.md compares responsibilities, never package names. Classes: capability missing, capability partial, capability exists but unreachable, capability exists but unverified, duplicated mechanism, missing shared mechanism, current architecture conflicts with target behavior, and architecture concern with no current product impact.

Every architecture concern names the capability or invariant it affects. One affecting none is marked non-blocking cleanup explicitly.

Prioritization

One final list, ranked:

1. Prevents intended everyday use today.
2. Makes existing behavior incorrect or unreliable.
3. Blocks multiple capabilities.
4. Prevents verification.
5. Architectural cleanup with no present user impact.

An unwired or unreachable future defect never outranks a live user-visible failure because its architecture is ugly.

Session 3: the viewer

A capabilities mode in the architecture viewer. The primary view is a matrix of capability against designed, coded, wired, configured, deployed, reachable, verified.

Clicking a capability shows the target DoD, implementation evidence, verification evidence, affected components, scenarios, blockers and unresolved product questions.

An invariants view shows which components participate in each cross-cutting rule. Target, implementation and runtime verification are distinguished visually.

Where the artifacts live

docs/capabilities/ becomes a declared tier in docs/CLAUDE.md: generated, regenerated from docs/spec.md plus a named eval, never hand-edited. The same row legitimises docs/architecture/, which is currently an undeclared directory.

The empirical run lands as one dated eval, docs/evals/2026-08-26-capability-baseline.md, frozen on the day. Every verified cell in the ledger cites it by path, so no status is an opinion.

Decided, do not re-ask

• Three sessions, split on the causal order above.
• Target side of the ledger first, with no implementation or verification status.
• Two verified columns plus simulated. Only live sets pass.
• Probes run against the live container, with the store wiped afterwards.
• Row counts are read with a mavend -wipe dry run before anything else.
• Readback is e2eprobe, not sqlite and not the web pages.
• Both probe origins, dod: and field:, tagged.
• Deferred capabilities get the skeleton only.
• All three outbound reaches get one real delivery.
• Domain is a second axis beside the spec section.
• No fixes land in this pass.

Findings recorded while planning

Live and reproduced, not inferred. They belong to the baseline, not to this plan.

• POST /api/chat with что ты помнишь обо мне? answered Я не знаю вас или как вы себя называете. Formal вас and вы on the wire, where CLAUDE.md requires informal singular ты. The phrasing eval passes.
• Vikunja was never down. vikunja-mcp publishes 127.0.0.1:9100 only, so the LAN address never answers from workpc. Three sessions read a refused connection as an outage and filed nothing.
• The deployed resident model is maven-instruct-b2-Q4_K_XL. CLAUDE.md still names Qwen3-1.7B.