Maven/docs/evals/2026-08-13-capability-audit.md

Maven's gaps are configuration and data, not code: 17 of 39 capabilities live and all four broken ones need no Go written

• Date: 2026-08-13
• Task: unfiled at the time of the run. Recorded 2026-08-15 while writing docs/spec.md
• Box: homesrv, the running five-container stack (mavend, mavsttd, mavttsd, mavweb, mavpoll)
• Build: mavend and mavweb on 40bf556. mavsttd, mavttsd and mavpoll on a four-day-old image

Setup

Every capability was checked against the running stack, not against the code. Seven utterances went through POST /api/chat. The rest was read from the startup log, the container list, the config, the outbox and the ten web pages.

The four states mean:

• live: observed working on the box during the run.
• partial: works and has a named defect.
• broken: the path exists and cannot complete.
• off: switched off or absent on purpose.

A row whose evidence says "not probed" was read from configuration or logs and not exercised. Those rows are stated as such rather than scored optimistically.

Corrections applied after the run

The original run marked speech-to-text and text-to-speech partial and the wake word off, because the auditing box could not reach them. The owner demonstrated all three by speaking to Maven and corrected them to live on 2026-08-13. The tally below carries the correction. The per-row evidence names what the audit could and could not see.

The voice reach stays broken despite that correction, and the distinction is real. He reaches her by speaking, which is a pull. A proactive message needs a live voice session to push into. Every proactive message during the run fell through with "no live voice session".

Tally

state	count
live	17
partial	9
broken	4
off or absent	9
total	39

The turn

capability	state	evidence	named gap
Route an utterance	partial	Router, heads and 331 seeds loaded at start. Five of seven probes routed correctly.	Two probes about her own memory routed to remember instead of query.
Ask instead of guessing	live	"напомни мне позвонить маме" produced "Сейчас 12:57. Когда?"; "отмена" produced "Хорошо, отменила."	None. V-717 closed on this evidence.
Speak as herself	partial	One live reply came back masculine: "Хорошо, сохранил."	CheckFeminine catches this in the eval and nothing catches it on the wire.
Answer from your own data	live	"что у меня сегодня по плану?" returned the real 10:30 checklist and its three open items.	None. The strongest path she has.
Answer from the world	partial	"почему небо голубое?" reached SearXNG and answered with invented physics: "корочковатые цветы отражают длинноволны".	Retrieval works, the Russian summary does not. Response.Empty() is the only gate in front of it.
Read an encyclopedia	live	Russian and English books wired at start, kiwix-server up four days. Not probed with a question that reached it.	Confirm a Russian question lands on the Russian book.
Weather	broken	No weather block in the config, so the provider loads as stub. "какая сейчас погода?" answers "для какого города?" and the follow-up "Самара" dies with "Я тебя не разобрала."	Configure a provider and a home city; make the source park a clarify instead of emitting a question through the answer path.
See an image	off	No vision model configured; the seam stores the image and says so.	The gemma-4 mmproj is already on the box. V-667.

Memory

capability	state	evidence	named gap
Facts	live	Environment facts land every minute: desk_active from hyprland, netdata_alarm from the poller.	One probe answered "я записала информацию о тебе" and wrote nothing.
Notes	partial	His own taps from 111 hours ago are present, and so is a junk note the probe wrote: "я рассказывал тебе про байкал?" stored as a statement.	No surface deletes a note. /api/revert voids facts by key and nothing voids this. V-494.
Recall	partial	Embedder loaded at 384 dimensions with the marker check passing.	The personal boundary scores the Baikal question as world, which is also the failing case in TestONNXPersonalBoundary.
Memory evaluation	off	Evaluator and daemon loop shipped; it writes notes and cannot speak. Not probed.	Verify on the box before trusting anything it concluded. V-248.

Proactive

capability	state	evidence	named gap
Reminders	live	50 fired. The ntfy failure ran once a minute until 03:05, then reminder #83 went out over Telegram at 03:06 and nothing retried.	None. V-715 closed on this evidence.
Interruption policy	partial	At 12:30 the dispatcher logged dropped morning:утро (sev1, presence=present) after the voice sink found no session.	Nobody has verified the four outcomes against what they should be. V-281.
Digest of held nudges	live	A rule's semantic fingerprint is checked before the phraser is paid. Six tests cover it; no suppressed candidate has come through the running daemon yet.	Watch one real suppression cycle end to end. V-687.
Morning routine	partial	Today's checklist exists with all three items missing, the 08:00-11:00 window passed, and the one delivery attempt was dropped.	The routine builds the plan and never reaches him. V-280.
Routine proposals	live	Page renders, zero proposed.	Idle, not broken. Nothing proves the detector would fire, and the proposer reads a hand-written Russian verb list. V-606.
Tasks	live	5 open, one overdue by four days, captured from the web tap, ordered by deadline and urgency.	Mail-derived task candidates are the untested half. V-130.
RSS feeds	live	Two sources, checked every five minutes, LWN items as recent as eight hours old and tagged.	Read when asked, never announced. That is the intended shape.

Reach

capability	state	evidence	named gap
Telegram	live	Reminder #83 delivered at 03:06 through the socks relay; intake reading his chat since restart.	None.
ntfy	off	"disabled": true in the committed config and no NTFY_TOKEN in the environment.	Off on purpose after the 403 storm. Re-enabling needs a write-scoped token first.
Voice	broken	Every proactive message during the run fell through with "no live voice session".	The wire is up and nothing is listening on it. He reaches her by speaking; she cannot reach him.
Web UI	live	All ten pages answered 200, the slowest in 31ms.	None.

Speech and senses

capability	state	evidence	named gap
Speech to text	live	Corrected by the owner, who transcribed real speech on the day. The audit box saw only that the workstation transcriber answers 200 on 192.168.1.105:8081 and that mavend probes it every 15s.	mavsttd, the local floor, runs a four-day-old image and its arm was not exercised separately.
Text to speech	live	Corrected by the owner, who heard her speak on the day. The audit box saw only that the container was up.	Same four-day-old image.
Wake word	live	Corrected by the owner, who woke her by voice. Runs on workpc under systemd, outside this stack and unreachable from the audit box.	Its relationship to the dark voice reach is unproven: waking her is not the same as her having a session to push into. V-515.
Hearing	off	No capture client ships, so the path is unreachable.	Build the client or the capability stays theoretical. V-514.
Speaker recognition	off	Planned, not verified on the box.	V-255.

The ecosystem

capability	state	evidence	named gap
Nexus	broken	Reachable and empty: "no entities yet".	Free text must resolve to a canonical entity id before any mutating call, so every act naming a target has nothing to resolve against. Seeding is a Nexus-side job.
Praxis	live	Answers, reports nothing needing attention.	Nothing to do until it has something to surface.
Hexis	partial	Registry populated: docker start, stop, restart, inspect, git diff and history, all through workspace_mcp with risk and access marked.	Blocked behind the empty Nexus. The capabilities exist and the targets do not.
Smart home	off	"enabled": false in the config.	Blocked task, not a defect. V-256.
LAN scan	live	Enabled in config with subnets, ports and rate set. Not probed.	Bluetooth control stays blocked; no bluez on the box. V-257.

Operations

capability	state	evidence	named gap
The deployed stack	partial	Five containers up. mavend and mavweb rebuilt at 40bf556; mavsttd, mavttsd and mavpoll on the four-day-old image.	One docker compose up -d --force-recreate on the three closes the drift.
Encrypted database	live	Encrypted at rest, working copy in tmpfs, key from the environment and held only by mavend.	None.
Passkey and step-up	broken	WebAuthn unconfigured, so every step-up gate is fail-open, including POST /api/chat, which reaches the act path.	The only high-severity item: anyone past the proxy can enroll a passkey, and step-up is process-global. V-683.
Model swap	off	No phraser.swap_models allowlist, so the page has nothing to offer. A swap is never written down and does not survive a restart.	Add the allowed gguf paths, or accept that the resident model is whatever the config says. V-250.
Self-update	off	Blocked at step 3: mavupdate cannot reach the containerized socket.	V-477.
Tests and analyzers	live	Full race suite green across ./internal/... and ./cmd/.... Lint clean against 16 accepted findings, deadcode against 13.	One latency test fails only under the coverage run. V-718.

What this rules out

It rules out a roadmap ordered by code work. None of the four broken capabilities is a code defect. Weather has no configuration block. Nexus has no data. The voice reach has no listener. Step-up has no WebAuthn credential. Nine more capabilities are off by choice, or blocked on hardware and operator action.

It also rules out a definition of done that a test suite can score. The suite was green during this run and 22 of 39 capabilities were not live. Every acceptance criterion has to be observable on the running box: an utterance and its expected answer, a log line, an outbox row, or an HTTP response.

Two capabilities were scored on the owner's correction rather than on machine evidence. Those two are speech in and speech out, which is exactly what a stack audit cannot see from inside the stack. Any future audit run from mavend's network namespace repeats the mistake unless it asks him.