8153e5eaa5
Session 2 step 3, and the end of the explanation half. gaps.md compares responsibilities and never package names. Eight classes. The four capability classes are derived from the ledger's gap_class field and rebuild with build_ledger.py. The four architecture classes are read from findings.md and invariants.md, and every entry names the capability or invariant it affects. An entry naming neither is marked non-blocking cleanup in those words, which is the whole of class 8 and its eleven rows. Of 46 v1 capabilities: 5 missing, 21 partial and reachable, 9 built and unreachable, 11 reachable and unverified. The nine unreachable ones are seven config blocks and two compose entries. Not one is a code defect. The ranked list puts phrasing first: speak-as-herself fails all three criteria, and everything that asks the resident model to write a Russian sentence inherits that. His own name not being stored is second. Nine capabilities one config change from reachable is fourth, and it is the highest ratio of capability to work in the list. Items 10, 12 and 13 stall on unresolved invariants and are the owner's call, not work. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2691 lines
120 KiB
YAML
2691 lines
120 KiB
YAML
# Capability ledger, target side.
|
||
#
|
||
# GENERATED by docs/capabilities/build_ledger.py from docs/spec.md.
|
||
# Do not hand-edit. Domain assignment is the one human input and
|
||
# lives in docs/capabilities/domains.yaml.
|
||
#
|
||
# Verification is per criterion, from verdicts.json. Implementation is
|
||
# per capability, seven dimensions derived from the component statuses
|
||
# in docs/architecture/maven-architecture.json through the mapping in
|
||
# docs/capabilities/implementation.yaml. Never one boolean.
|
||
|
||
source: docs/spec.md
|
||
capability_count: 51
|
||
criterion_count: 156
|
||
|
||
section_findings:
|
||
- section: 'Memory'
|
||
finding: >-
|
||
The four memory rows below have no living doc. **Finding**: `docs/design.md` sketches the store and `docs/routing.md` covers recall's routing. Nothing owns the fact and note contracts, the supersede rule, or the digestion worker's consolidation pass. This is the largest documentation gap in the list.
|
||
|
||
capabilities:
|
||
- id: route-an-utterance
|
||
title: 'Route an utterance'
|
||
section: 'The turn'
|
||
scope: v1
|
||
domain: [deliberation]
|
||
state: >-
|
||
`docs/routing.md`.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'partial'
|
||
gap_class: 'capability partial'
|
||
components: [router.cascade, router.stage0, router.heads, router.llm, router.classifier, router.embedder, router.extractor, core.turn_route, core.topics, core.decision_trace, state.decision_ring, state.routing_traces, state.routing_labels]
|
||
scenarios:
|
||
- name: conversation_anaphora
|
||
exists: true
|
||
- name: assistant_workday
|
||
exists: true
|
||
scenario_note: >-
|
||
`conversation_anaphora`, `assistant_workday`.
|
||
criteria:
|
||
- id: 'route-an-utterance#b8fd'
|
||
text: >-
|
||
Seven intents score separately from the source, on every turn.
|
||
verified: fail
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
Of the 22 decision traces read back, only 15 carry a route stage. The other 7 were decided at pre-route or by a stage 0 grammar, which assigns an intent without scoring one. No trace anywhere shows seven intents scored separately, and only 11 traces reached the query stage where the source is scored. The two are scored separately where both run; 'on every turn' does not hold.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.raw.jsonl'
|
||
- id: 'route-an-utterance#4937'
|
||
text: >-
|
||
A question about what Maven remembers routes to query, not remember. The audit's two misrouted probes of seven pass.
|
||
verified: fail
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
"что ты помнишь обо мне?" routed to intent chat, not query, decided by the routing heads at 0.866. The audit had it going to remember; it is not fixed, it is different. "меня зовут Ками" routed to chat too, which is why no fact was written.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § Routing'
|
||
- id: 'route-an-utterance#3f22'
|
||
text: >-
|
||
Any resident-model error falls through to the classifier and the turn still answers.
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
Of the 22 traces read back, 15 carry a classifier claim and all 15 read never_asked, with the reason 'the routing heads answered' or 'the LLM router answered'. The other 7 never reached the route stage. No resident-model error occurred, so the fallthrough was not exercised.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.raw.jsonl'
|
||
- id: 'route-an-utterance#d5b4'
|
||
text: >-
|
||
Every turn writes a decision trace naming the stage that decided.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
All 34 probe turns returned a trace id and the 22 traces read back each name a winner and the deciding stage. The record is an in-memory ring of 25 and does not survive a restart, which this criterion does not ask about.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.raw.jsonl'
|
||
- 'internal/decision/ring.go:11'
|
||
|
||
- id: ask-instead-of-guessing
|
||
title: 'Ask instead of guessing'
|
||
section: 'The turn'
|
||
scope: v1
|
||
domain: [deliberation, interaction]
|
||
state: >-
|
||
`docs/routing.md`, the clarify head and the parked clarify ride.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'partial'
|
||
gap_class: 'capability partial'
|
||
components: [core.preroute, state.clarify_store, state.dialogue_sessions]
|
||
scenarios:
|
||
- name: conversation_anaphora
|
||
exists: true
|
||
scenario_note: >-
|
||
`conversation_anaphora`.
|
||
criteria:
|
||
- id: 'ask-instead-of-guessing#bf18'
|
||
text: >-
|
||
An incomplete reminder parks and asks for the missing slot, not for all of them.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
"напомни мне позвонить маме" asked only "Сейчас 01:09. Когда?", then "в шесть вечера" was taken as the answer and it asked only for the day.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.transcript.tsv field:reminder:clarify-the-missing-slot'
|
||
- id: 'ask-instead-of-guessing#99c0'
|
||
text: >-
|
||
"отмена" during a parked clarify drops the parked turn and says so.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
"отмена" during a parked clarify answered "Хорошо, отменила." and no reminder row was written for the kibble.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.transcript.tsv field:reminder:cancel-mid-clarify'
|
||
- id: 'ask-instead-of-guessing#fa88'
|
||
text: >-
|
||
A parked clarify survives an interleaved unrelated turn and resumes.
|
||
verified: fail
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
It survives and it does not resume. In the contaminated run the park set at turn 8 reached turns 9 through 13, appending 'В какой день?' or 'На какое время поставить напоминание?' to five consecutive unrelated turns, including 'меня зовут Ками' and two statements about where he works. It was then dropped at turn 14 by the 'Прошлую просьбу отпускаю' path, not by отмена and not by being answered. It never asks the original question again.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The first run measured itself'
|
||
- 'docs/capabilities/out/field.contaminated.jsonl'
|
||
|
||
- id: speak-as-herself
|
||
title: 'Speak as herself'
|
||
section: 'The turn'
|
||
scope: v1
|
||
domain: [interaction]
|
||
state: >-
|
||
`docs/language.md`.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'partial'
|
||
deployed: 'partial'
|
||
reachable: 'partial'
|
||
verified: 'no'
|
||
gap_class: 'capability exists but unreachable'
|
||
components: [core.phraser, core.replier, core.action_chat, core.model_seam, svc.llama_server, eval.phrasing]
|
||
scenarios:
|
||
- name: morning_missed
|
||
exists: true
|
||
- name: persona_wire
|
||
exists: false
|
||
scenario_note: >-
|
||
`morning_missed` pins the constant. `persona_wire` *(to write)* pins model output on the outbound path.
|
||
criteria:
|
||
- id: 'speak-as-herself#a4b7'
|
||
text: >-
|
||
No masculine self-reference reaches the wire. The audit caught "Хорошо, сохранил" live while `CheckFeminine` passed in the eval, so the check runs on the outbound path and not only in the eval.
|
||
verified: fail
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
Nothing checks the outbound path. CheckFeminine, CheckAddress and CheckCringe appear only under internal/phraser/eval, which scores the phrasing eval. No masculine self-reference appears in the 34 scored turns; every self-referring form there is feminine. The one masculine instance, 'Я ничего о тебе и вы никуда таки не упомянул', is from the contaminated run, which scores nothing.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The persona breaks on the wire'
|
||
- id: 'speak-as-herself#5347'
|
||
text: >-
|
||
She says "ты" throughout, and no pet name appears.
|
||
verified: fail
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
Two of the 34 scored turns used formal address on the wire: 'Давайте проверим.' and 'Похоже у вас есть интерес к математике! … а вы можете продолжить? Или вам нужны советы по образованию?'. A third instance is in the discarded contaminated run. No pet name appeared.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The persona breaks on the wire'
|
||
- id: 'speak-as-herself#aebc'
|
||
text: >-
|
||
A rejected reply is logged with what failed, not silently rephrased.
|
||
verified: fail
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
No rejection stage exists on the outbound path, so a rejected reply cannot be logged. The three checks live only under internal/phraser/eval. The criterion asks for a log of what failed and there is nothing to do the failing.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The persona breaks on the wire'
|
||
|
||
- id: answer-from-your-own-data
|
||
title: 'Answer from your own data'
|
||
section: 'The turn'
|
||
scope: v1
|
||
domain: [memory, governance]
|
||
state: >-
|
||
`docs/routing.md`, `queryWalk` in `cmd/mavend/actions_query.go`.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'partial'
|
||
gap_class: 'capability partial'
|
||
components: [core.query_chain, core.q.embed, core.q.memory, core.q.factbykey, core.q.notes, core.q.history, core.q.list, core.q.self, core.q.personal, state.list_items]
|
||
scenarios:
|
||
- name: assistant_workday
|
||
exists: true
|
||
- name: morning_missed
|
||
exists: true
|
||
scenario_note: >-
|
||
`assistant_workday`, `morning_missed`.
|
||
criteria:
|
||
- id: 'answer-from-your-own-data#bfbb'
|
||
text: >-
|
||
"что у меня сегодня по плану?" returns the real checklist and its open items.
|
||
verified: fail
|
||
reason: 'scenario fails'
|
||
detail: >-
|
||
'что у меня сегодня по плану?' answered 'на 26.08.2026 ничего не запланировано.' in 0.01s with an empty items list. Five live tasks existed at that turn; the sixth was created later by field:action:capture-a-task. The day plan and the task list disagree, and 'какие у меня задачи?' listed all of them.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.raw.jsonl field:query:own-data-first readback.plan'
|
||
- id: 'answer-from-your-own-data#dbeb'
|
||
text: >-
|
||
The owner's sources are asked before anything outside, every time.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
Eleven of the 22 traces reached queryWalk. On all eleven, every owner source that was asked was asked before any world source, and no world source won a turn an owner source had claimed. Seven guessing sources (attention, list, feeds, home, network, weather, self) were dropped where the cascade named a destination, which is queryWalk removing the sources marked guesses:true. The personal boundary was asked on every one.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.raw.jsonl readback.decisions'
|
||
- id: 'answer-from-your-own-data#1952'
|
||
text: >-
|
||
A source that looks rather than guesses is asked even when a destination is named.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
Settled on six live turns where the cascade named a destination ('world', 'recall', 'weather'). Only the seven sources marked guesses:true were removed; every looking source, including fact-by-key, day-plan, tasks, calendar, notes, memory and the personal boundary, was still asked and declined.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.raw.jsonl readback.decisions'
|
||
|
||
- id: answer-from-the-world
|
||
title: 'Answer from the world'
|
||
section: 'The turn'
|
||
scope: v1
|
||
domain: [action, governance]
|
||
state: >-
|
||
`docs/world.md`.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'no'
|
||
gap_class: 'capability partial'
|
||
components: [core.q.search, core.q.web, core.q.general, core.q.personal, ext.searxng]
|
||
scenarios:
|
||
- name: world_summary_quality
|
||
exists: false
|
||
scenario_note: >-
|
||
`world_summary_quality` *(to write)*.
|
||
criteria:
|
||
- id: 'answer-from-the-world#5319'
|
||
text: >-
|
||
A general-knowledge question returns a Russian summary that does not invent physics. The audit's "почему небо голубое?" answer is the failing case.
|
||
verified: fail
|
||
reason: 'scenario fails'
|
||
detail: >-
|
||
'почему небо голубое?' returned live from search in 6.29s: 'Почему небо голубое? - В статье разберемся, почему днем небо кажется голубым (см., например). Рассказываем про солнечный свет.' A scraped snippet carrying its own source's editorial voice, not a summary. It no longer invents physics, which the audit's answer did.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.transcript.tsv field:query:the-world-summary'
|
||
- id: 'answer-from-the-world#94a8'
|
||
text: >-
|
||
Only the utterance leaves the box. No persona block, no history, no matched notes.
|
||
verified: unknown
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
Not observable from the reply or from IPC readback. Settling it needs the outbound request body, which this transport does not expose.
|
||
evidence:
|
||
- 'docs/capabilities/README.md'
|
||
- id: 'answer-from-the-world#4587'
|
||
text: >-
|
||
Deleting the `search` block turns the capability off with a named gap, not an error.
|
||
verified: untested
|
||
reason: 'scenario missing'
|
||
detail: >-
|
||
The search block is present and complete in deploy/mavend.json. No probe removed it, because doing so would have changed the config the rest of the baseline is attributable to.
|
||
evidence:
|
||
- 'deploy/mavend.json'
|
||
|
||
- id: read-an-encyclopedia
|
||
title: 'Read an encyclopedia'
|
||
section: 'The turn'
|
||
scope: v1
|
||
domain: [action]
|
||
state: >-
|
||
`docs/world.md`, the Kiwix section.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'no'
|
||
gap_class: 'capability partial'
|
||
components: [core.q.kiwix, ext.kiwix]
|
||
scenarios:
|
||
- name: kiwix_language_pick
|
||
exists: false
|
||
scenario_note: >-
|
||
`kiwix_language_pick` *(to write)*.
|
||
criteria:
|
||
- id: 'read-an-encyclopedia#4546'
|
||
text: >-
|
||
A Russian question lands on the Russian book and an English one on the English book.
|
||
verified: fail
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
"расскажи про Байкал" was taken by the stage 0 narrative-query grammar and routed to query:memory. Kiwix was never_asked. The reply invented a URL to km.ru.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § Routing'
|
||
- id: 'read-an-encyclopedia#eeb8'
|
||
text: >-
|
||
Kiwix answers when the line is down.
|
||
verified: untested
|
||
reason: 'scenario missing'
|
||
detail: >-
|
||
The line stayed up throughout, so no probe exercised the offline fallback.
|
||
evidence:
|
||
- 'docs/capabilities/probes_field.json'
|
||
- id: 'read-an-encyclopedia#e846'
|
||
text: >-
|
||
The retrieved article is on the question's topic, not merely a lexical match.
|
||
verified: untested
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
No turn reached Kiwix, so no retrieved article exists to judge for topicality.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § Routing'
|
||
|
||
- id: weather
|
||
title: 'Weather'
|
||
section: 'The turn'
|
||
scope: v1
|
||
domain: [action, deliberation]
|
||
state: >-
|
||
`internal/weather`. No living doc covers it.
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'partial'
|
||
deployed: 'partial'
|
||
reachable: 'partial'
|
||
verified: 'no'
|
||
gap_class: 'capability exists but unreachable'
|
||
components: [core.q.weather, ext.openmeteo]
|
||
finding: >-
|
||
the provider seam, the home city and the clarify path have no written reasoning anywhere. The audit found the capability broken on configuration alone.
|
||
scenarios:
|
||
- name: weather_followup
|
||
exists: false
|
||
scenario_note: >-
|
||
`weather_followup` *(to write)*.
|
||
criteria:
|
||
- id: 'weather#f587'
|
||
text: >-
|
||
A `weather` block in `deploy/mavend.json` names a provider and a home city.
|
||
verified: fail
|
||
reason: 'configuration missing'
|
||
detail: >-
|
||
deploy/mavend.json has no weather key at all. Not a wrong value and not a disabled flag: the block is absent, so the provider loads as a stub.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § What is not configured at all'
|
||
- id: 'weather#24f6'
|
||
text: >-
|
||
"какая сейчас погода?" answers for the home city without asking.
|
||
verified: fail
|
||
reason: 'configuration missing'
|
||
detail: >-
|
||
"какая сейчас погода?" answered "для какого города?" There is no home city to answer for.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.transcript.tsv field:query:follow-up-context'
|
||
- id: 'weather#3de6'
|
||
text: >-
|
||
Naming another city answers for that city. The audit's follow-up "Самара" died with "Я тебя не разобрала", so the follow-up parks as a clarify instead of emitting a question through the answer path.
|
||
verified: fail
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
The weather source emits its question through the answer path and parks no clarify. The trace for 'Самара' shows pre-route clarify-answer declining with nothing pending, so the bare city name fell through to the router, scored intent act at 0.247, was thinned by the clarify head and went to action:action-handler. The reply was 'Я тебя не разобрала. Повтори, пожалуйста.' The spec names this exact shape: the follow-up must park as a clarify instead of emitting a question through the answer path.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § Routing'
|
||
- 'docs/capabilities/out/field.raw.jsonl field:query:follow-up-context readback.decisions'
|
||
- id: 'weather#ed1f'
|
||
text: >-
|
||
With no provider configured she names the gap and does not guess a forecast.
|
||
verified: fail
|
||
reason: 'configuration missing'
|
||
detail: >-
|
||
She does not guess a forecast, and she does not name the gap either. With no weather block the one live turn answered 'для какого города?', which hides the missing provider behind a question implying a city is all that is needed. The follow-up then died, so the two-turn dead end is the proof the gap is never named.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.transcript.tsv field:query:follow-up-context'
|
||
|
||
- id: see-an-image
|
||
title: 'See an image'
|
||
section: 'The turn'
|
||
scope: v1
|
||
domain: [action, perception]
|
||
state: >-
|
||
`internal/vision`, the seam that stores the image and says so. No living doc.
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'no'
|
||
deployed: 'no'
|
||
reachable: 'no'
|
||
verified: 'no'
|
||
gap_class: 'capability exists but unreachable'
|
||
components: [core.vision, state.media_blobs]
|
||
finding: >-
|
||
V-667 has the gemma-4 mmproj on the box and no written contract for what a vision call returns.
|
||
scenarios:
|
||
- name: vision_degraded
|
||
exists: false
|
||
scenario_note: >-
|
||
`vision_degraded` *(to write)*.
|
||
criteria:
|
||
- id: 'see-an-image#c6d5'
|
||
text: >-
|
||
An image sent through Telegram gets a Russian description.
|
||
verified: blocked
|
||
reason: 'configuration missing'
|
||
detail: >-
|
||
deploy/mavend.json has no vision block. The seam in internal/vision exists with four files and the gemma-4 mmproj is on the box, and nothing is wired to call it. V-667.
|
||
evidence:
|
||
- 'deploy/mavend.json'
|
||
- 'docs/spec.md § See an image'
|
||
- id: 'see-an-image#bd91'
|
||
text: >-
|
||
With no vision model configured she says she cannot look, and the image is stored.
|
||
verified: untested
|
||
reason: 'scenario missing'
|
||
detail: >-
|
||
The absent vision block is this criterion's premise, not its blocker: with no vision model configured she should say she cannot look and store the image. No image was sent through Telegram, so the degraded path was never exercised. vision_degraded does not exist.
|
||
evidence:
|
||
- 'deploy/mavend.json'
|
||
- id: 'see-an-image#0a4a'
|
||
text: >-
|
||
The vision call goes to the workstation and falls back silently when it is down.
|
||
verified: blocked
|
||
reason: 'configuration missing'
|
||
detail: >-
|
||
deploy/mavend.json has no vision block. The seam in internal/vision exists with four files and the gemma-4 mmproj is on the box, and nothing is wired to call it. V-667.
|
||
evidence:
|
||
- 'deploy/mavend.json'
|
||
- 'docs/spec.md § See an image'
|
||
|
||
- id: facts
|
||
title: 'Facts'
|
||
section: 'Memory'
|
||
scope: v1
|
||
domain: [memory]
|
||
state: >-
|
||
`internal/store/facts.go`, `cmd/mavend/factenrichment.go`. No living doc.
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'partial'
|
||
gap_class: 'capability partial'
|
||
components: [state.facts, core.action_fact, core.fact_enrichment, core.store_api]
|
||
scenarios:
|
||
- name: morning_missed
|
||
exists: true
|
||
scenario_note: >-
|
||
`morning_missed`.
|
||
criteria:
|
||
- id: 'facts#b404'
|
||
text: >-
|
||
A stated fact is written and confirmed in his own words, in the feminine.
|
||
verified: fail
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
"меня зовут Ками" routed to intent chat and answered "У тебя есть 35 сообщений." No facts row was written. A name fact does exist, id 506, written 2026-08-01 from tap:voice, so the path worked once.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The fact store is not a model of him'
|
||
- id: 'facts#c623'
|
||
text: >-
|
||
A confirmation that says she wrote something is never emitted without the row existing. The audit's "я записала информацию о тебе" wrote nothing.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
No false confirmation was emitted in 34 turns. The audit's "я записала информацию о тебе" over an empty write did not recur; this run fails earlier, by never routing the statement to remember at all.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.transcript.tsv'
|
||
- id: 'facts#e035'
|
||
text: >-
|
||
A superseding fact retires the old value and both are readable.
|
||
verified: fail
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
field:fact:supersede never reached a supersede. All three turns, including the two plain statements, were claimed by the personal query source and answered "не знаю — не нашла у тебя такой записи". Statements are routed as questions.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.transcript.tsv field:fact:supersede'
|
||
- id: 'facts#f917'
|
||
text: >-
|
||
`actionFact.Subject` resolves through Nexus, never through a local key.
|
||
verified: unknown
|
||
reason: 'external dependency unavailable'
|
||
detail: >-
|
||
Nexus holds no entities, so no subject resolution ran. Not decidable until Nexus is seeded.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The act path extracts a preposition'
|
||
|
||
- id: notes
|
||
title: 'Notes'
|
||
section: 'Memory'
|
||
scope: v1
|
||
domain: [memory]
|
||
state: >-
|
||
`internal/memory`. No living doc.
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'partial'
|
||
gap_class: 'capability partial'
|
||
components: [state.notes, core.action_note]
|
||
scenarios:
|
||
- name: note_delete
|
||
exists: false
|
||
scenario_note: >-
|
||
`note_delete` *(to write)*.
|
||
criteria:
|
||
- id: 'notes#0626'
|
||
text: >-
|
||
A note captured from any reach is recallable by question.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
"запиши: ключи от гаража лежат в верхнем ящике стола" was captured as note 183 and "где лежат ключи от гаража?" recalled it. The note text keeps the imperative prefix, so the stored text is the command rather than the content.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.raw.jsonl field:note:capture-then-recall'
|
||
- id: 'notes#5134'
|
||
text: >-
|
||
A note can be deleted by voice and from the web UI. `/api/revert` voids facts by key and nothing voids a note today (V-494).
|
||
verified: fail
|
||
reason: 'code missing'
|
||
detail: >-
|
||
cmd/mavweb/main.go registers no /notes handler and GET /notes returns 404. There is no page from which to delete one. V-494 unchanged.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The web UI has no notes page'
|
||
- id: 'notes#8167'
|
||
text: >-
|
||
A question is not stored as a statement. The audit's "я рассказывал тебе про байкал?" became a junk note.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
"я рассказывал тебе про байкал?" wrote no note row. The audit's junk note did not recur.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.raw.jsonl field:note:question-is-not-a-statement'
|
||
|
||
- id: recall
|
||
title: 'Recall'
|
||
section: 'Memory'
|
||
scope: v1
|
||
domain: [memory, governance]
|
||
state: >-
|
||
`docs/routing.md` for the query walk and the personal boundary.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'partial'
|
||
gap_class: 'capability partial'
|
||
components: [core.recall, state.memory_vectors, router.embedder, core.q.memory, core.q.notes]
|
||
scenarios:
|
||
- name: assistant_workday
|
||
exists: true
|
||
scenario_note: >-
|
||
`assistant_workday`.
|
||
criteria:
|
||
- id: 'recall#54fc'
|
||
text: >-
|
||
The embedder loads at 384 dimensions with the marker check passing, on every start.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
Startup log: "voice: onnx embedder loaded (384 dim)" then "voice: embedder marker ok (model_quantized@384/tok2)".
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § What is configured correctly'
|
||
- id: 'recall#91bf'
|
||
text: >-
|
||
`EmbedQuery` and `EmbedPassage` carry their prefixes. A plain `Embed` on a note fails the build or the test.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
internal/router/embedder.go exports EmbedQuery and EmbedPassage, which apply the query: and passage: prefixes multilingual-e5-small was trained with. The only plain Embed call outside a test is internal/memory/recalleval/recalleval.go:167, inside a wrapper.
|
||
evidence:
|
||
- 'internal/router/embedder.go:55'
|
||
- 'internal/router/embedder.go:65'
|
||
- id: 'recall#3e25'
|
||
text: >-
|
||
The personal boundary scores a question about him as personal. `TestONNXPersonalBoundary` is green.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
TestONNXPersonalBoundary passes in 5.68s with MAVEN_ONNX_LIB set, along with FourFold, SemanticGroupHoldout, Challenge, PostRetuneChallenge, Latency and FrozenHeadMatchesCorpusFit. TestONNXPersonalBoundaryStratified self-skips. The boundary also held on the wire: every recall probe about him was claimed by memory or personal and none reached a world source.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § Tests and analyzers'
|
||
- id: 'recall#8606'
|
||
text: >-
|
||
A recall miss says she does not remember, and does not answer from the world instead.
|
||
verified: fail
|
||
reason: 'scenario fails'
|
||
detail: >-
|
||
"что я говорил тебе про ремонт балкона?" answered "Уточни пожалуйста имя человека из которого ты будешь учиться или посмотреть информацию." That is neither a recall nor an admission of a miss. The turn was claimed by memory, so the boundary held and the phrasing did not.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.transcript.tsv field:note:recall-miss-is-honest'
|
||
|
||
- id: memory-evaluation
|
||
title: 'Memory evaluation'
|
||
section: 'Memory'
|
||
scope: v1
|
||
domain: [memory, deliberation]
|
||
state: >-
|
||
`internal/memeval`. No living doc.
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'no'
|
||
deployed: 'no'
|
||
reachable: 'no'
|
||
verified: 'no'
|
||
gap_class: 'capability exists but unreachable'
|
||
components: [core.memory_eval]
|
||
finding: >-
|
||
the evaluator ships, writes notes and cannot speak, and nothing records what its conclusions mean (V-248).
|
||
scenarios:
|
||
- name: memeval_run
|
||
exists: false
|
||
scenario_note: >-
|
||
`memeval_run` *(to write)*.
|
||
criteria:
|
||
- id: 'memory-evaluation#95a6'
|
||
text: >-
|
||
One evaluation run is observed on the box and its notes are read back.
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
The evaluator runs on its own loop and cannot speak. No run was observed and the notes it would have written are not distinguishable from RSS captures.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § What the run does not establish'
|
||
- id: 'memory-evaluation#8e4e'
|
||
text: >-
|
||
What it concluded is checkable against the notes it read.
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
The evaluator runs on its own loop and cannot speak. No run was observed and the notes it would have written are not distinguishable from RSS captures.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § What the run does not establish'
|
||
|
||
- id: reminders
|
||
title: 'Reminders'
|
||
section: 'Proactive'
|
||
scope: v1
|
||
domain: [attention, interaction]
|
||
state: >-
|
||
`internal/store`, `internal/delivery`. No living doc covers the reminder lifecycle.
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'partial'
|
||
gap_class: 'capability partial'
|
||
components: [state.reminders, core.action_reminder, core.dispatcher, state.delivery_attempts]
|
||
finding: >-
|
||
parking, firing, delivery, retry and cancellation are spread across three packages with no written contract.
|
||
scenarios:
|
||
- name: recurring_reminders
|
||
exists: false
|
||
scenario_note: >-
|
||
`recurring_reminders` *(to write)*. Cancellation is covered by the V-719 eval.
|
||
criteria:
|
||
- id: 'reminders#bdf8'
|
||
text: >-
|
||
A one-shot reminder set by voice fires at its time and is delivered.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
Reminder 98, set two minutes out through the web reach, fired at 21:13:46Z and delivery attempt 16344 records telegram sent at 01:14:05 local.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The reaches'
|
||
- id: 'reminders#9f9b'
|
||
text: >-
|
||
**Recurring works from speech**: meetings, pills, the dog, the vet and the kibble. `store.Reminder` carries `Cron` and `ipc.CreateReminder` takes a cron argument, and no caller in `cmd/mavend` passes one. Recurring is unbuilt with its storage and delivery already finished under it.
|
||
verified: fail
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
"напоминай мне каждый день в девять вечера дать собаке таблетку" produced a one-shot clarify asking for a day. Every reminder row in the store has cron: "". Storage and delivery carry Cron and no caller in cmd/mavend passes one, exactly as the spec states.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.raw.jsonl field:reminder:recurring-from-speech'
|
||
- id: 'reminders#217c'
|
||
text: >-
|
||
A recurring reminder states its schedule back when it is set, and again when asked.
|
||
verified: fail
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
No schedule was stated back, because none was set. "какие у меня повторяющиеся напоминания?" answered "Все напоминания: 0."
|
||
evidence:
|
||
- 'docs/capabilities/out/field.transcript.tsv field:reminder:recurring-from-speech'
|
||
- id: 'reminders#b47b'
|
||
text: >-
|
||
Cancellation works by voice and from `/reminders`, and a refusal is honoured.
|
||
verified: untested
|
||
reason: 'scenario missing'
|
||
detail: >-
|
||
One of three clauses was exercised. Cancellation by text through the web reach works: 'отмена' answered 'Хорошо, отменила.' and wrote no row. Cancellation from the /reminders page was not exercised, and no refusal was offered to be honoured. /reminders answers 200.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.transcript.tsv field:reminder:cancel-mid-clarify'
|
||
- id: 'reminders#8f69'
|
||
text: >-
|
||
A failed delivery retries into another reach rather than looping. The audit watched the ntfy failure run once a minute until 03:05.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
Delivery group reminder:ea93acef: ntfy failed and telegram sent in the same second. The reminder path retries into another reach rather than looping. The nudge path does not: 149 of the last 200 attempts are one rule, netdata_critical, retrying into the voice reach once a minute.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The reaches'
|
||
|
||
- id: interruption-policy
|
||
title: 'Interruption policy'
|
||
section: 'Proactive'
|
||
scope: v1
|
||
domain: [initiative, perception]
|
||
state: >-
|
||
`docs/handler-wiring.md` for the dispatch decision.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'no'
|
||
gap_class: 'capability exists but unverified'
|
||
components: [core.rules, core.dispatcher, core.gatherer, state.presence_state, state.nudges, state.tick_memo]
|
||
finding: >-
|
||
the four presence-and-severity outcomes have never been written down as intended behaviour, only as code (V-281).
|
||
scenarios:
|
||
- name: morning_missed
|
||
exists: true
|
||
- name: evening_degraded
|
||
exists: true
|
||
scenario_note: >-
|
||
`morning_missed`, `evening_degraded`.
|
||
criteria:
|
||
- id: 'interruption-policy#b7cc'
|
||
text: >-
|
||
The four outcomes are named in a doc before any of them changes.
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
The four presence-and-severity outcomes fire on the tick, not on a turn. 12 of the last 200 attempts are status dropped, which is the policy acting, and nothing in this run says whether it acted correctly.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § What the run does not establish'
|
||
- id: 'interruption-policy#5dfd'
|
||
text: >-
|
||
A severity-1 item with him present at the desk reaches him through some reach. The audit logged `dropped morning:утро (sev1, presence=present)`.
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
The four presence-and-severity outcomes fire on the tick, not on a turn. 12 of the last 200 attempts are status dropped, which is the policy acting, and nothing in this run says whether it acted correctly.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § What the run does not establish'
|
||
- id: 'interruption-policy#9f4c'
|
||
text: >-
|
||
Nothing unprompted arrives during a quiet tick.
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
The four presence-and-severity outcomes fire on the tick, not on a turn. 12 of the last 200 attempts are status dropped, which is the policy acting, and nothing in this run says whether it acted correctly.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § What the run does not establish'
|
||
|
||
- id: digest-of-held-nudges
|
||
title: 'Digest of held nudges'
|
||
section: 'Proactive'
|
||
scope: v1
|
||
domain: [initiative, attention]
|
||
state: >-
|
||
`internal/worker`, the digestion worker. No living doc.
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'no'
|
||
gap_class: 'capability exists but unverified'
|
||
components: [state.digest_entries, core.tick_loop, core.rules]
|
||
scenarios:
|
||
- name: evening_degraded
|
||
exists: true
|
||
scenario_note: >-
|
||
`evening_degraded`.
|
||
criteria:
|
||
- id: 'digest-of-held-nudges#7860'
|
||
text: >-
|
||
A suppressed nudge candidate is observed surfacing in a later digest, on the box.
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
Digestion runs on its own tick. No suppression cycle completed during the seven minutes the run took, and the nudge count did not move.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § What the run does not establish'
|
||
- id: 'digest-of-held-nudges#c933'
|
||
text: >-
|
||
The semantic fingerprint is checked before the phraser is paid.
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
Digestion runs on its own tick. No suppression cycle completed during the seven minutes the run took, and the nudge count did not move.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § What the run does not establish'
|
||
- id: 'digest-of-held-nudges#1480'
|
||
text: >-
|
||
Digestion never calls Hexis.
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
Digestion runs on its own tick. No suppression cycle completed during the seven minutes the run took, and the nudge count did not move.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § What the run does not establish'
|
||
|
||
- id: morning-routine
|
||
title: 'Morning routine'
|
||
section: 'Proactive'
|
||
scope: v1
|
||
domain: [initiative, interaction]
|
||
state: >-
|
||
`internal/morning`, `internal/routine`. No living doc.
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'no'
|
||
gap_class: 'capability exists but unverified'
|
||
components: [core.morning, core.q.dayplan]
|
||
scenarios:
|
||
- name: morning_missed
|
||
exists: true
|
||
scenario_note: >-
|
||
`morning_missed`.
|
||
criteria:
|
||
- id: 'morning-routine#3a4b'
|
||
text: >-
|
||
The morning plan reaches him inside its 08:00-11:00 window, `Europe/Samara`.
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
The morning plan is anchored to an 08:00-11:00 Europe/Samara window a probe cannot reach. Four of the last 200 delivery attempts are channel voice, rule morning:утро, status failed.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § What the run does not establish'
|
||
- id: 'morning-routine#d87d'
|
||
text: >-
|
||
When the voice reach has no session the plan falls back to the non-voice reaches rather than being dropped (owner's call; related V-281).
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
The morning plan is anchored to an 08:00-11:00 Europe/Samara window a probe cannot reach. Four of the last 200 delivery attempts are channel voice, rule morning:утро, status failed.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § What the run does not establish'
|
||
- id: 'morning-routine#4bf5'
|
||
text: >-
|
||
A missed window is stated as missed, not silently swallowed.
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
The morning plan is anchored to an 08:00-11:00 Europe/Samara window a probe cannot reach. Four of the last 200 delivery attempts are channel voice, rule morning:утро, status failed.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § What the run does not establish'
|
||
|
||
- id: routine-proposals
|
||
title: 'Routine proposals'
|
||
section: 'Proactive'
|
||
scope: v1
|
||
domain: [initiative, memory]
|
||
state: >-
|
||
`internal/routine`. No living doc.
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'no'
|
||
gap_class: 'capability exists but unverified'
|
||
components: [core.pattern, core.routines, state.proposed_routines, state.events]
|
||
finding: >-
|
||
the proposer reads a hand-written Russian verb list, which the language rules forbid as a route or fact source (V-606).
|
||
scenarios:
|
||
- name: routine_proposal
|
||
exists: false
|
||
scenario_note: >-
|
||
`routine_proposal` *(to write)*.
|
||
criteria:
|
||
- id: 'routine-proposals#a705'
|
||
text: >-
|
||
One proposal is observed on the box from real repeated behaviour.
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
The proposer reads repeated behaviour over days. Nothing it could propose from was created during the run, and zero proposals exist.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § What the run does not establish'
|
||
- id: 'routine-proposals#030a'
|
||
text: >-
|
||
The verb list is replaced by `internal/lexicon` or the embedder.
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
The proposer reads repeated behaviour over days. Nothing it could propose from was created during the run, and zero proposals exist.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § What the run does not establish'
|
||
- id: 'routine-proposals#d9d3'
|
||
text: >-
|
||
A proposal is a nudge he can decline, and declining it is stored.
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
The proposer reads repeated behaviour over days. Nothing it could propose from was created during the run, and zero proposals exist.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § What the run does not establish'
|
||
|
||
- id: tasks
|
||
title: 'Tasks'
|
||
section: 'Proactive'
|
||
scope: v1
|
||
domain: [attention, memory]
|
||
state: >-
|
||
`internal/tasks`. No living doc.
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'partial'
|
||
gap_class: 'capability exists but unverified'
|
||
components: [state.tasks, core.q.tasks]
|
||
scenarios:
|
||
- name: assistant_workday
|
||
exists: true
|
||
- name: morning_missed
|
||
exists: true
|
||
scenario_note: >-
|
||
`assistant_workday`, `morning_missed`.
|
||
criteria:
|
||
- id: 'tasks#0536'
|
||
text: >-
|
||
Open tasks are read back ordered by deadline and urgency.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
"какие у меня задачи?" returned the list ordered by deadline and urgency, naming the overdue one: "проверить интейк форму на боксе (просрочено на 17 дней)" then four aged items, then a count.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.transcript.tsv field:action:capture-a-task'
|
||
- id: 'tasks#58f2'
|
||
text: >-
|
||
A task captured by voice appears in the list and on the web UI.
|
||
verified: untested
|
||
reason: 'scenario missing'
|
||
detail: >-
|
||
Half the criterion is measured. 'добавь в задачи: заказать корм для собаки' landed as task 11, open, over the web reach, and appeared in the next turn's list read over IPC. That the row renders on the /tasks page was not observed: /tasks answers 200, which proves the route exists, not that task 11 is on it.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.raw.jsonl field:action:capture-a-task'
|
||
- id: 'tasks#6078'
|
||
text: >-
|
||
Mail-derived candidates are never spoken as tasks until he accepts one (V-130).
|
||
verified: blocked
|
||
reason: 'deployment missing'
|
||
detail: >-
|
||
mavmaild is built and is not in docker-compose.yml, so no mail-derived candidate exists to be spoken or withheld. V-130.
|
||
evidence:
|
||
- 'docs/spec.md § Email triage'
|
||
|
||
- id: rss-and-news
|
||
title: 'RSS and news'
|
||
section: 'Proactive'
|
||
scope: v1
|
||
domain: [action]
|
||
state: >-
|
||
`internal/rss`. No living doc.
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'partial'
|
||
gap_class: 'capability exists but unverified'
|
||
components: [core.feed_worker, core.q.feeds]
|
||
scenarios:
|
||
- name: morning_missed
|
||
exists: true
|
||
scenario_note: >-
|
||
`morning_missed`.
|
||
criteria:
|
||
- id: 'rss-and-news#4ec8'
|
||
text: >-
|
||
Feed items are read when asked and never announced unprompted. That is the intended shape, not a defect.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
"что нового в новостях?" returned three configured feed items in 1.67s, claimed by the feeds source. Nothing was announced unprompted during the run.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.transcript.tsv field:query:feeds-on-request'
|
||
- id: 'rss-and-news#b61d'
|
||
text: >-
|
||
A question about a topic finds the matching item across the configured feeds.
|
||
verified: untested
|
||
reason: 'scenario missing'
|
||
detail: >-
|
||
The probe asked for what is new, not for a topic, so cross-feed topic matching was not exercised.
|
||
evidence:
|
||
- 'docs/capabilities/probes_field.json'
|
||
- id: 'rss-and-news#2a27'
|
||
text: >-
|
||
A dead feed names itself as dead, and the others still answer.
|
||
verified: untested
|
||
reason: 'external dependency unavailable'
|
||
detail: >-
|
||
Both configured feeds answered. Killing one was out of scope.
|
||
evidence:
|
||
- 'docs/capabilities/probes_field.json'
|
||
|
||
- id: telegram
|
||
title: 'Telegram'
|
||
section: 'Reach'
|
||
scope: v1
|
||
domain: [interaction]
|
||
state: >-
|
||
`docs/deployment.md`, `internal/delivery/telegramsink`.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'partial'
|
||
gap_class: 'capability exists but unverified'
|
||
components: [core.sink_telegram, core.telegram_intake, ext.telegram, state.ack_sends]
|
||
scenarios:
|
||
- name: evening_degraded
|
||
exists: true
|
||
scenario_note: >-
|
||
`evening_degraded`.
|
||
criteria:
|
||
- id: 'telegram#b78d'
|
||
text: >-
|
||
An outbound message is delivered and the outbox row records the delivery.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
The outbox clause is live: attempt 16344, kind reminder, channel telegram, status sent at 2026-08-26T01:14:05.738+04:00, and 28 of the last 200 attempts are telegram with all 28 sent. 'sent' is what the sink writes when the Bot API accepts the payload. Arrival on his device was not confirmed and is listed as unestablished.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The reaches'
|
||
- id: 'telegram#c293'
|
||
text: >-
|
||
His chat is read continuously from restart.
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
telegram.intake is true in the config. No inbound message was sent during the baseline, so continuous reading from restart was not observed.
|
||
evidence:
|
||
- 'deploy/mavend.json'
|
||
- id: 'telegram#9eea'
|
||
text: >-
|
||
The socks relay being down names the gap and holds the message.
|
||
verified: untested
|
||
reason: 'scenario missing'
|
||
detail: >-
|
||
The socks relay was up throughout and no probe took it down. evening_degraded was not driven with the relay down.
|
||
evidence:
|
||
- 'docs/capabilities/probes_field.json'
|
||
|
||
- id: ntfy
|
||
title: 'ntfy'
|
||
section: 'Reach'
|
||
scope: v1
|
||
domain: [interaction]
|
||
state: >-
|
||
`internal/delivery/ntfysink`, disabled in the committed config.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'no'
|
||
deployed: 'no'
|
||
reachable: 'no'
|
||
verified: 'no'
|
||
gap_class: 'capability exists but unreachable'
|
||
components: [core.sink_ntfy, ext.ntfy]
|
||
scenarios:
|
||
- name: ntfy_403
|
||
exists: false
|
||
scenario_note: >-
|
||
`ntfy_403` *(to write)*.
|
||
criteria:
|
||
- id: 'ntfy#27b5'
|
||
text: >-
|
||
A write-scoped `NTFY_TOKEN` exists before re-enabling. It was switched off after a 403 storm.
|
||
verified: fail
|
||
reason: 'configuration missing'
|
||
detail: >-
|
||
No write-scoped NTFY_TOKEN exists. deploy/mavend.json carries "token": "${NTFY_TOKEN}" with "disabled": true, and deploy/telegram.env, the documented ${VAR} source, defines no NTFY_TOKEN. This is a measured negative, not an unreachable question. An ntfy attempt row was still written and still failed, so whatever disabled turns off, it is not the attempt.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The reaches'
|
||
- 'deploy/mavend.json'
|
||
- id: 'ntfy#bbb6'
|
||
text: >-
|
||
A 403 stops retrying instead of looping once a minute.
|
||
verified: untested
|
||
reason: 'scenario missing'
|
||
detail: >-
|
||
No 403 was provoked. The one ntfy attempt in this run failed with the token unset rather than with a 403.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The reaches'
|
||
|
||
- id: voice
|
||
title: 'Voice'
|
||
section: 'Reach'
|
||
scope: v1
|
||
domain: [interaction, governance]
|
||
state: >-
|
||
`docs/protocol.md` for the wire, `internal/delivery/voicesink` for the sink.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'partial'
|
||
gap_class: 'capability partial'
|
||
components: [core.voice_server, bnd.voice_tcp, core.sink_voice, proc.mavenclient]
|
||
finding: >-
|
||
the wire is documented and the listener is not. Nothing describes what holds a live voice session open.
|
||
scenarios:
|
||
- name: voice_push
|
||
exists: false
|
||
scenario_note: >-
|
||
`voice_push` *(to write)*.
|
||
criteria:
|
||
- id: 'voice#0529'
|
||
text: >-
|
||
A proactive message reaches him by speech without him speaking first. Every proactive message during the audit fell through with "no live voice session".
|
||
verified: fail
|
||
reason: 'deployment missing'
|
||
detail: >-
|
||
149 of the last 200 delivery attempts are the voice reach failing. 151 rows are one rule, netdata_critical, which retried once a minute for about three hours, 2026-08-16T22:17:20 to 2026-08-17T01:19:20, and then stopped. It is a bounded burst, not an ongoing storm. No listener has ever run on the reach; mavwaked, the client that would open a session, has been dead since 2026-08-25 18:21:35.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The reaches'
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The wake word has been deaf for seven hours'
|
||
- id: 'voice#819c'
|
||
text: >-
|
||
The port stays on homesrv loopback and reaches workpc over ssh.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
Narrowed to the loopback clause, which is measured: docker port maps 9100/tcp -> 127.0.0.1:9110, and voice.bind 0.0.0.0:9100 is the container's own namespace. The ssh leg was not observed. deploy/maven-voice-tunnel.service is a repo file whose header says to copy it to workpc, and its state on workpc was not queried.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § What is configured correctly'
|
||
- id: 'voice#492d'
|
||
text: >-
|
||
`SurfaceVoice` caps acts at L0, and reading is not capped.
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
Code reading only, which cannot carry a pass. internal/auth/tier.go:84 MaxLayer returns Layer0 for SurfaceVoice, Layer2 for telegram, Layer3 for pc_client, authed_page and core_process, and -1 for anything unrecognised, so it fails closed. No voice turn ran, so neither the cap nor the uncapped read was observed.
|
||
evidence:
|
||
- 'internal/auth/tier.go:84'
|
||
|
||
- id: web-ui
|
||
title: 'Web UI'
|
||
section: 'Reach'
|
||
scope: v1
|
||
domain: [interaction, governance]
|
||
state: >-
|
||
`docs/deployment.md`.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'partial'
|
||
gap_class: 'capability partial'
|
||
components: [proc.mavweb, bnd.http_web]
|
||
scenarios:
|
||
scenario_note: >-
|
||
covered by `cmd/mavweb` tests, not by a scenario.
|
||
criteria:
|
||
- id: 'web-ui#5a52'
|
||
text: >-
|
||
All pages answer 200.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
All fourteen registered pages answer 200, the slowest in 28ms.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The web UI has no notes page'
|
||
- id: 'web-ui#82c6'
|
||
text: >-
|
||
Every capability with a surface has a page: reminders, notes, tasks, facts.
|
||
verified: fail
|
||
reason: 'code missing'
|
||
detail: >-
|
||
The spec names four surfaces and two do not exist. GET /notes and GET /facts both 404, and cmd/mavweb/main.go registers no handler for either.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The web UI has no notes page'
|
||
- id: 'web-ui#e2e8'
|
||
text: >-
|
||
A destructive action on a page is gated by step-up.
|
||
verified: fail
|
||
reason: 'configuration missing'
|
||
detail: >-
|
||
Every step-up gate is fail-open. mavweb logs six unguarded surfaces at startup, POST /api/chat among them, and this baseline drove 58 turns through it without authenticating.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § Every step-up gate is fail-open'
|
||
|
||
- id: desk-notifications
|
||
title: 'Desk notifications'
|
||
section: 'Reach'
|
||
scope: v1
|
||
domain: [perception, interaction]
|
||
state: >-
|
||
`cmd/mavweb/ambient.go`, `internal/event`. No living doc.
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'no'
|
||
gap_class: 'capability partial'
|
||
components: [core.event_bus, proc.mavweb]
|
||
finding: >-
|
||
the inbound direction exists as the `ambient:notif` source and the outbound direction does not exist at all. Which one the owner means is an open product decision.
|
||
scenarios:
|
||
- name: morning_missed
|
||
exists: true
|
||
scenario_note: >-
|
||
`morning_missed` covers inbound. Outbound has none.
|
||
criteria:
|
||
- id: 'desk-notifications#7397'
|
||
text: >-
|
||
Inbound: a desktop notification becomes a fact at the ambient path's own confidence, filed low, and never spoken back unprompted.
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
The inbound path exists as the ambient:notif source and morning_missed covers it. No probe posted a desktop notification and morning_missed was not driven, so the confidence, the filing level and the never-spoken-unprompted clause were all unobserved.
|
||
evidence:
|
||
- 'docs/capabilities/probes_field.json'
|
||
- id: 'desk-notifications#f230'
|
||
text: >-
|
||
Outbound: a nudge can appear on the workpc desktop as a fourth reach, or the outbound half is explicitly dropped from v1.
|
||
verified: fail
|
||
reason: 'code missing'
|
||
detail: >-
|
||
Neither disjunct holds. internal/delivery/channel.go defines exactly four channels, voice, ntfy, telegram and drop, with no desktop sink, so a nudge cannot appear on the workpc desktop. And the outbound half has not been explicitly dropped from v1 either: docs/spec.md still records which direction the owner means as an open decision.
|
||
evidence:
|
||
- 'internal/delivery/channel.go'
|
||
- 'docs/spec.md'
|
||
|
||
- id: speech-to-text
|
||
title: 'Speech to text'
|
||
section: 'Speech and senses'
|
||
scope: v1
|
||
domain: [interaction, perception]
|
||
state: >-
|
||
`docs/offload.md`, `docs/deployment.md`.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'no'
|
||
gap_class: 'capability exists but unverified'
|
||
components: [core.stt_seam, proc.mavsttd, ext.whispercpp, ext.cw2_stt, bnd.worker]
|
||
scenarios:
|
||
scenario_note: >-
|
||
covered by `docs/evals/2026-08-09-crisperwhisper2-russian-wer.md`.
|
||
criteria:
|
||
- id: 'speech-to-text#f8ef'
|
||
text: >-
|
||
Russian speech transcribes accurately enough to route. `mavsttd` is the local floor and the workstation transcriber is the better path.
|
||
verified: untested
|
||
reason: 'scenario missing'
|
||
detail: >-
|
||
mavsttd is deployed and running on the 2026-08-08 image. POST /api/ptt is registered on the same mavweb mux the 58 chat turns used (cmd/mavweb/main.go:241) and proxies raw PCM onto mavend's voice port (cmd/mavweb/voiceproxy.go:47). The path was reachable from the measuring box throughout and no audio probe was written.
|
||
evidence:
|
||
- 'cmd/mavweb/main.go:241'
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The speech path was reachable and was not probed'
|
||
- id: 'speech-to-text#eafe'
|
||
text: >-
|
||
The workstation being down falls back to `mavsttd` silently.
|
||
verified: untested
|
||
reason: 'scenario missing'
|
||
detail: >-
|
||
The fallback is in effect right now and was not exercised. Both workstation ports refuse from workpc: 192.168.1.105:8080 and :8081 return no HTTP status. workstation.model_disabled is also true. So mavend's stt.Pair has already fallen to the mavsttd floor, and whether it did so silently is exactly what no probe asked. POST /api/ptt is registered on the same mavweb mux the 58 chat turns used (cmd/mavweb/main.go:241) and proxies raw PCM onto mavend's voice port (cmd/mavweb/voiceproxy.go:47). The path was reachable from the measuring box throughout and no audio probe was written.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The speech path was reachable and was not probed'
|
||
- id: 'speech-to-text#0959'
|
||
text: >-
|
||
The floor arm is exercised on its own, not only behind the workstation.
|
||
verified: untested
|
||
reason: 'scenario missing'
|
||
detail: >-
|
||
With the workstation transcriber refusing on both ports, mavsttd is the only arm in service, so the floor is trivially exercised on its own. Nothing posted audio to confirm it transcribes. POST /api/ptt is registered on the same mavweb mux the 58 chat turns used (cmd/mavweb/main.go:241) and proxies raw PCM onto mavend's voice port (cmd/mavweb/voiceproxy.go:47). The path was reachable from the measuring box throughout and no audio probe was written.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The speech path was reachable and was not probed'
|
||
|
||
- id: text-to-speech
|
||
title: 'Text to speech'
|
||
section: 'Speech and senses'
|
||
scope: v1
|
||
domain: [interaction]
|
||
state: >-
|
||
`docs/offload.md`, `docs/deployment.md`.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'no'
|
||
gap_class: 'capability exists but unverified'
|
||
components: [core.tts_seam, proc.mavttsd, ext.piper, bnd.worker]
|
||
scenarios:
|
||
- name: tts_normalisation
|
||
exists: false
|
||
scenario_note: >-
|
||
`tts_normalisation` *(to write)*.
|
||
criteria:
|
||
- id: 'text-to-speech#d477'
|
||
text: >-
|
||
A reply is spoken in Russian with correct number and abbreviation expansion.
|
||
verified: untested
|
||
reason: 'scenario missing'
|
||
detail: >-
|
||
mavttsd is deployed and running on the 2026-08-08 image. Nothing is missing from the deployment. POST /api/ptt is registered on the same mavweb mux the 58 chat turns used (cmd/mavweb/main.go:241) and proxies raw PCM onto mavend's voice port (cmd/mavweb/voiceproxy.go:47). The path was reachable from the measuring box throughout and no audio probe was written.
|
||
evidence:
|
||
- 'cmd/mavweb/main.go:241'
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The speech path was reachable and was not probed'
|
||
- id: 'text-to-speech#4dd3'
|
||
text: >-
|
||
`internal/ttsnorm` expands times and dates into `Europe/Samara` phrasing.
|
||
verified: untested
|
||
reason: 'scenario missing'
|
||
detail: >-
|
||
internal/ttsnorm is compiled into mavend, not mavttsd: ttsnorm.Speakable is called at cmd/mavend/voice.go on the reply text and in internal/delivery/voicesink on nudge text. mavend runs the 2026-08-19 image, the same process the baseline drove 58 turns through, so the code is deployed. The scenario tts_normalisation does not exist and no audio probe was written.
|
||
evidence:
|
||
- 'cmd/mavweb/main.go:241'
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The speech path was reachable and was not probed'
|
||
|
||
- id: wake-word
|
||
title: 'Wake word'
|
||
section: 'Speech and senses'
|
||
scope: v1
|
||
domain: [perception, interaction]
|
||
state: >-
|
||
`docs/deployment.md`, `mavwaked` under systemd on workpc.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'no'
|
||
gap_class: 'capability partial'
|
||
components: [proc.mavwaked, cfg.systemd, ext.alsa]
|
||
scenarios:
|
||
- name: voice_push
|
||
exists: false
|
||
scenario_note: >-
|
||
`voice_push` *(to write)*.
|
||
criteria:
|
||
- id: 'wake-word#10ca'
|
||
text: >-
|
||
"Мэйвен" wakes her and a near-miss does not.
|
||
verified: fail
|
||
reason: 'deployment missing'
|
||
detail: >-
|
||
mavwaked is enabled on workpc under a user unit and has been dead since 2026-08-25 18:21:35, seven hours before this baseline. It exited on 'arecord: pcm_read:2285: read error: No such device' after 2h45m, with status 0/SUCCESS, so systemd treated a lost microphone as a clean exit and did not restart it. Nothing on the box noticed. Neither the wake nor the near-miss half could be exercised.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The wake word has been deaf for seven hours'
|
||
- id: 'wake-word#16d8'
|
||
text: >-
|
||
Waking her opens a voice session the proactive path can push into. Waking is not the same as having a session (V-515).
|
||
verified: fail
|
||
reason: 'deployment missing'
|
||
detail: >-
|
||
The session clause is settled in the negative and does not need the wake word to settle it. 156 of the last 200 delivery attempts are the proactive path finding no voice session, 149 of them one rule retrying once a minute since 2026-08-16. Waking her is not the same as her having a session to push into (V-515), and with mavwaked dead since 18:21:35 neither half runs.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The reaches'
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The wake word has been deaf for seven hours'
|
||
|
||
- id: hearing
|
||
title: 'Hearing'
|
||
section: 'Speech and senses'
|
||
scope: v1
|
||
domain: [perception, interaction]
|
||
state: >-
|
||
`internal/capture`, `internal/audio`. No capture client ships (V-514).
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'no'
|
||
deployed: 'no'
|
||
reachable: 'no'
|
||
verified: 'no'
|
||
gap_class: 'capability exists but unreachable'
|
||
components: [core.capture, state.media_blobs]
|
||
scenarios:
|
||
- name: voice_push
|
||
exists: false
|
||
scenario_note: >-
|
||
`voice_push` *(to write)*.
|
||
criteria:
|
||
- id: 'hearing#1e54'
|
||
text: >-
|
||
A capture client runs on workpc and streams to `mavsttd` or the workstation.
|
||
verified: fail
|
||
reason: 'deployment missing'
|
||
detail: >-
|
||
A capture client does ship and it is cmd/mavwaked: it spawns arecord for 16kHz mono PCM, runs silero VAD and the wake head, and sends PushToTalk frames to mavend's voice port. It is installed and enabled on workpc and has been dead since 2026-08-25 18:21:35 on a lost microphone. The spec's 'no capture client ships (V-514)' is out of date; what is absent is mavenclient.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The wake word has been deaf for seven hours'
|
||
- id: 'hearing#b416'
|
||
text: >-
|
||
The path is reachable end to end from microphone to reply.
|
||
verified: untested
|
||
reason: 'scenario missing'
|
||
detail: >-
|
||
The path is reachable and was not exercised. POST /api/ptt is registered on the same mavweb mux the 58 chat turns used (cmd/mavweb/main.go:241) and proxies raw PCM onto mavend's voice port (cmd/mavweb/voiceproxy.go:47). The path was reachable from the measuring box throughout and no audio probe was written. The mavwaked half of the path is separately dead since 2026-08-25 18:21:35.
|
||
evidence:
|
||
- 'cmd/mavweb/voiceproxy.go:47'
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The speech path was reachable and was not probed'
|
||
|
||
- id: speaker-recognition
|
||
title: 'Speaker recognition'
|
||
section: 'Speech and senses'
|
||
scope: deferred
|
||
domain: [perception, governance]
|
||
state: >-
|
||
`internal/speaker`. No living doc (V-255).
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'no'
|
||
deployed: 'no'
|
||
reachable: 'no'
|
||
verified: 'no'
|
||
gap_class: 'capability exists but unreachable'
|
||
components: [core.speaker]
|
||
deferred_note: >-
|
||
**Deferred past v1**, owner's call 2026-08-15. See `docs/roadmap.md`.
|
||
scenarios:
|
||
- name: speaker_gate
|
||
exists: false
|
||
scenario_note: >-
|
||
`speaker_gate` *(to write)*.
|
||
criteria:
|
||
- id: 'speaker-recognition#3110'
|
||
text: >-
|
||
The owner's voice is distinguished from another voice.
|
||
verified: untested
|
||
reason: 'deferred past v1'
|
||
- id: 'speaker-recognition#1ff6'
|
||
text: >-
|
||
A voice that is not his cannot reach the act path.
|
||
verified: untested
|
||
reason: 'deferred past v1'
|
||
|
||
- id: nexus
|
||
title: 'Nexus'
|
||
section: 'The ecosystem'
|
||
scope: v1
|
||
domain: [deliberation, governance]
|
||
state: >-
|
||
`docs/ecosystem.md`.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'no'
|
||
gap_class: 'capability partial'
|
||
components: [ext.nexus, core.ecosystem, bnd.http_ecosystem, state.ecosystem_traces]
|
||
scenarios:
|
||
- name: act_degraded
|
||
exists: true
|
||
scenario_note: >-
|
||
`act_degraded`.
|
||
criteria:
|
||
- id: 'nexus#93be'
|
||
text: >-
|
||
Entities exist. Nexus answers "no entities yet" today, so every act naming a target has nothing to resolve against. Seeding is a Nexus-side job.
|
||
verified: fail
|
||
reason: 'external dependency unavailable'
|
||
detail: >-
|
||
Nexus still holds no entities. Four resolve calls ran, one per act turn, all status not_found: eco-traces 103, 105, 107 and 109. Seeding is a Nexus-side job and has not happened.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.raw.jsonl readback.eco-traces'
|
||
- id: 'nexus#4e31'
|
||
text: >-
|
||
Free text resolves to a canonical entity id before any mutating call.
|
||
verified: blocked
|
||
reason: 'external dependency unavailable'
|
||
detail: >-
|
||
Not observable. No resolve returned a canonical entity id and no mutating call ran, so the gate was never put to the test. Separately, the resolution is fed bad input: three unrelated utterances all produced the target «с», a one-letter preposition, including two that do not contain the word.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The act path extracts a preposition as the target'
|
||
- 'docs/capabilities/out/field.raw.jsonl readback.eco-traces'
|
||
- id: 'nexus#1ba8'
|
||
text: >-
|
||
Ambiguous resolution asks him and does not pick.
|
||
verified: untested
|
||
reason: 'external dependency unavailable'
|
||
detail: >-
|
||
With no entities, no resolution was ambiguous. Not decidable until Nexus is seeded.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.raw.jsonl readback.eco-traces'
|
||
- id: 'nexus#2309'
|
||
text: >-
|
||
Nexus down produces a named gap, not a broken turn.
|
||
verified: untested
|
||
reason: 'external dependency unavailable'
|
||
detail: >-
|
||
Nexus was up throughout. Taking it down was out of scope for a read-only baseline.
|
||
evidence:
|
||
- 'docs/capabilities/probes_field.json'
|
||
|
||
- id: praxis
|
||
title: 'Praxis'
|
||
section: 'The ecosystem'
|
||
scope: v1
|
||
domain: [attention, governance]
|
||
state: >-
|
||
`docs/ecosystem.md`.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'no'
|
||
gap_class: 'capability partial'
|
||
components: [ext.praxis, core.ecosystem, core.praxis_acts, core.q.attention, state.surfaced_items, state.ecosystem_traces]
|
||
scenarios:
|
||
- name: morning_missed
|
||
exists: true
|
||
- name: evening_degraded
|
||
exists: true
|
||
scenario_note: >-
|
||
`morning_missed`, `evening_degraded`.
|
||
criteria:
|
||
- id: 'praxis#8068'
|
||
text: >-
|
||
An item needing attention is read back on request.
|
||
verified: fail
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
The ecosystem wiring is present and correct: praxis.url is http://praxis:8989, the client is built at cmd/mavend/ecosystem.go, and the attention capability is registered. The gap is upstream of it. 'что требует моего внимания?' scored intent chat at 0.723 on the routing heads, so no query source ran, and the stage 0 praxis-attention grammar declined with 'pattern did not match'. PraxisGrammars() is the only path to Praxis. The file to open is the grammar, not the ecosystem client.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § Routing'
|
||
- id: 'praxis#ff0c'
|
||
text: >-
|
||
Reading an item aloud calls `Surface`, never `Acknowledge`.
|
||
verified: untested
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
No turn reached Praxis, so neither Surface nor Acknowledge was called. The eco-traces readback is a limit-20 tail of the whole run, 21:09:50 to 21:11:40, and every row is service nexus. The reason no turn reached Praxis is the routing miss recorded against praxis#8068.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.raw.jsonl readback.eco-traces'
|
||
- id: 'praxis#7eb1'
|
||
text: >-
|
||
Attention arrives over HTTP, never from its SQLite file.
|
||
verified: untested
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
No Praxis call ran during the baseline, so the transport was not observed.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.raw.jsonl readback.eco-traces'
|
||
- id: 'praxis#64fd'
|
||
text: >-
|
||
Digestion may summarise Praxis and may not call Hexis.
|
||
verified: untested
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
Corrected: internal/worker is the STT and TTS offload wire, not the digestion worker, so reading it for Hexis calls proved nothing. Digestion is cmd/mavend/tick_digest.go plus internal/loop. No digestion tick was observed during the run.
|
||
evidence:
|
||
- 'cmd/mavend/tick_digest.go'
|
||
- 'internal/loop'
|
||
|
||
- id: hexis
|
||
title: 'Hexis'
|
||
section: 'The ecosystem'
|
||
scope: v1
|
||
domain: [action, governance]
|
||
state: >-
|
||
`docs/ecosystem.md`.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'no'
|
||
gap_class: 'capability exists but unverified'
|
||
components: [ext.hexis, core.ecosystem, core.ecosystem_hexis_gate, core.action_act, core.risk_policy, state.tools, state.pending_act, state.ecosystem_traces]
|
||
scenarios:
|
||
- name: act_degraded
|
||
exists: true
|
||
scenario_note: >-
|
||
`act_degraded`.
|
||
criteria:
|
||
- id: 'hexis#c178'
|
||
text: >-
|
||
An act runs against a real target once Nexus has entities.
|
||
verified: blocked
|
||
reason: 'external dependency unavailable'
|
||
detail: >-
|
||
Nexus holds no entities, so no act had a target to run against. No Hexis call appears in any eco-trace from this run.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.raw.jsonl readback.eco-traces'
|
||
- id: 'hexis#cc3f'
|
||
text: >-
|
||
Confirmation binds capability id, target entity, arguments, requester and expiry.
|
||
verified: untested
|
||
reason: 'external dependency unavailable'
|
||
detail: >-
|
||
No act reached a confirm turn, because every one was refused at resolution.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The act path extracts a preposition as the target'
|
||
- id: 'hexis#752a'
|
||
text: >-
|
||
LLM output alone never authorizes.
|
||
verified: untested
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
No act was authorized during the run, so the negative could not be observed. The architecture pass's finding is the reason it stays open: internal/auth answers who may carry authority and internal/tool answers what proof a capability demands, and the two never meet. That is a missing join, not an unobserved mechanism.
|
||
evidence:
|
||
- 'docs/architecture/findings.md'
|
||
- id: 'hexis#7f56'
|
||
text: >-
|
||
Every call carries a correlation id, a contract version and `X-Requested-By: maven`.
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
cmd/mavend/ecosystem.go:76 sets the version header, X-Requested-By and X-Correlation-ID from the request context, minting none. Four correlation ids are observable, one per act turn: cor-8edceae547967f43, cor-a5d7ae8185a3f865, cor-515e897ddbbce659 and cor-5c5cbdfa2bec4efe. The header trio was not observed on the wire, because no Hexis call ran.
|
||
evidence:
|
||
- 'cmd/mavend/ecosystem.go:76'
|
||
- 'docs/capabilities/out/field.raw.jsonl readback.eco-traces'
|
||
|
||
- id: smart-home
|
||
title: 'Smart home'
|
||
section: 'The ecosystem'
|
||
scope: deferred
|
||
domain: [action, governance]
|
||
state: >-
|
||
`internal/smarthome`, disabled in config (V-256).
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'partial'
|
||
deployed: 'partial'
|
||
reachable: 'partial'
|
||
verified: 'no'
|
||
gap_class: 'capability exists but unreachable'
|
||
components: [ext.homeassistant, core.home_worker, core.q.home]
|
||
deferred_note: >-
|
||
**Deferred past v1**, owner's call 2026-08-15. See `docs/roadmap.md`.
|
||
scenarios:
|
||
- name: act_degraded
|
||
exists: true
|
||
scenario_note: >-
|
||
`act_degraded`.
|
||
criteria:
|
||
- id: 'smart-home#f85d'
|
||
text: >-
|
||
A device is controlled through Hexis, resolved through Nexus, never by free text.
|
||
verified: untested
|
||
reason: 'deferred past v1'
|
||
- id: 'smart-home#6a7a'
|
||
text: >-
|
||
Disabled means a named gap.
|
||
verified: untested
|
||
reason: 'deferred past v1'
|
||
|
||
- id: network-scans
|
||
title: 'Network scans'
|
||
section: 'The ecosystem'
|
||
scope: v1
|
||
domain: [action]
|
||
state: >-
|
||
`internal/netscan`, `internal/netaddr`. No living doc.
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'partial'
|
||
gap_class: 'capability exists but unverified'
|
||
components: [core.netscan, core.q.network]
|
||
scenarios:
|
||
- name: netscan_query
|
||
exists: false
|
||
scenario_note: >-
|
||
`netscan_query` *(to write)*.
|
||
criteria:
|
||
- id: 'network-scans#0b79'
|
||
text: >-
|
||
A scan of the configured subnets returns hosts and open ports on request.
|
||
verified: unknown
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
The network source answered live in 27.01s with 'нашла 3 устройства, из них 2 с вебом, 2 с ssh. список записала.' That is three counts. It names no address, no hostname and none of the four configured ports, so whether the scan covered the configured subnets and returned hosts and open ports is not decidable from the reply. The list it says it wrote was not read back.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.transcript.tsv field:query:network-scan-as-prose'
|
||
- id: 'network-scans#cb91'
|
||
text: >-
|
||
The result is read back as prose, not as a table dump.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
Read back as one prose sentence with counts, not as a table dump.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.transcript.tsv field:query:network-scan-as-prose'
|
||
- id: 'network-scans#1479'
|
||
text: >-
|
||
The rate limit in the config is honoured.
|
||
verified: unknown
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
The limiter was exercised: a real scan of the configured 192.168.1.0/24 across four ports ran end to end in 27.01s at a configured rate of 100. One wall-clock number cannot separate the limiter from ordinary latency, so the criterion is not decidable from this run.
|
||
evidence:
|
||
- 'docs/capabilities/out/field.transcript.tsv field:query:network-scan-as-prose'
|
||
|
||
- id: bluetooth-control
|
||
title: 'Bluetooth control'
|
||
section: 'The ecosystem'
|
||
scope: deferred
|
||
domain: [action]
|
||
state: >-
|
||
no package.
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'no'
|
||
wired: 'no'
|
||
configured: 'no'
|
||
deployed: 'no'
|
||
reachable: 'no'
|
||
verified: 'no'
|
||
gap_class: 'capability missing'
|
||
components: []
|
||
finding: >-
|
||
nothing exists, and the box has no bluez (V-257). This is the only v1 item blocked on the host rather than on code.
|
||
deferred_note: >-
|
||
**Deferred past v1**, owner's call 2026-08-15. See `docs/roadmap.md`.
|
||
scenarios:
|
||
- name: bluetooth_control
|
||
exists: false
|
||
scenario_note: >-
|
||
`bluetooth_control` *(to write)*.
|
||
criteria:
|
||
- id: 'bluetooth-control#05fe'
|
||
text: >-
|
||
bluez is present on the box that owns the radio.
|
||
verified: untested
|
||
reason: 'deferred past v1'
|
||
- id: 'bluetooth-control#be8d'
|
||
text: >-
|
||
A paired device is connected and disconnected by voice, through Hexis.
|
||
verified: untested
|
||
reason: 'deferred past v1'
|
||
|
||
- id: mcps
|
||
title: 'MCPs'
|
||
section: 'The ecosystem'
|
||
scope: v1
|
||
domain: [action, governance]
|
||
state: >-
|
||
`internal/mcp`. No living doc.
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'no'
|
||
deployed: 'no'
|
||
reachable: 'no'
|
||
verified: 'no'
|
||
gap_class: 'capability exists but unreachable'
|
||
components: [core.mcp_worker, ext.vikunja_mcp]
|
||
finding: >-
|
||
the allowlist, the stdio and http transports and the webfetch door all exist. Nothing records which servers may run, or why.
|
||
scenarios:
|
||
- name: mcp_tool_call
|
||
exists: false
|
||
scenario_note: >-
|
||
`mcp_tool_call` *(to write)*.
|
||
criteria:
|
||
- id: 'mcps#349c'
|
||
text: >-
|
||
A configured MCP server's tools are callable through the act path.
|
||
verified: blocked
|
||
reason: 'configuration missing'
|
||
detail: >-
|
||
One MCP server is configured, vikunja, with enabled: false and a four-tool allowlist. Nothing is callable through the act path while it is off.
|
||
evidence:
|
||
- 'deploy/mavend.json mcp.servers'
|
||
- id: 'mcps#79b8'
|
||
text: >-
|
||
The allowlist is the only path to a tool, and a tool outside it is refused.
|
||
verified: blocked
|
||
reason: 'configuration missing'
|
||
detail: >-
|
||
One MCP server is configured, vikunja, with enabled: false and a four-tool allowlist. Nothing is callable through the act path while it is off.
|
||
evidence:
|
||
- 'deploy/mavend.json mcp.servers'
|
||
- id: 'mcps#c89c'
|
||
text: >-
|
||
A server that dies is a named gap, and the rest of the registry still answers.
|
||
verified: blocked
|
||
reason: 'configuration missing'
|
||
detail: >-
|
||
Only one MCP server is configured, vikunja, and it is disabled. Even enabled, the clause about the rest of the registry still answering has nothing to measure against: a registry of one cannot demonstrate that the others survive a death.
|
||
evidence:
|
||
- 'deploy/mavend.json mcp.servers'
|
||
|
||
- id: the-deployed-stack
|
||
title: 'The deployed stack'
|
||
section: 'Operations'
|
||
scope: v1
|
||
domain: [operations]
|
||
state: >-
|
||
`docs/deployment.md`.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'no'
|
||
gap_class: 'capability partial'
|
||
components: [cfg.compose, cfg.mavend, proc.mavend, proc.mavweb, proc.mavsttd, proc.mavttsd, proc.mavpoll, proc.mavgpud, ext.netdata, ext.uptimekuma]
|
||
scenarios:
|
||
scenario_note: >-
|
||
none. This is checked by `docker compose ps` and the startup log.
|
||
criteria:
|
||
- id: 'the-deployed-stack#82b6'
|
||
text: >-
|
||
All five compose services run the current build. The audit found three on a four-day-old image.
|
||
verified: fail
|
||
reason: 'deployment missing'
|
||
detail: >-
|
||
Three of five run sha256:fc338a2f, built 2026-08-08. mavend and mavweb run sha256:016e80eb, built 2026-08-19. The audit found three on a four-day-old image; the gap is now eighteen days.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § The deployed stack is not one build'
|
||
- id: 'the-deployed-stack#b64a'
|
||
text: >-
|
||
`mavwaked` on workpc is on the current build too, or its drift is stated.
|
||
verified: untested
|
||
reason: 'external dependency unavailable'
|
||
detail: >-
|
||
mavwaked was not queried for its build. It runs on workpc under systemd, outside this stack, and the measuring box could not reach it. The criterion's second clause, 'or its drift is stated', is therefore also unmet: nothing states it.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § Tests and analyzers'
|
||
- id: 'the-deployed-stack#8454'
|
||
text: >-
|
||
A restart loses nothing.
|
||
verified: fail
|
||
reason: 'code missing'
|
||
detail: >-
|
||
A restart loses the decision ring. internal/decision/ring.go:5-11 says so by design: 'the whole store is memory: no migration, no insert on the answer'. Measured: 0 rows at 00:40 after a restart 49 minutes earlier, at trace id 977. The persistent tables do survive; the arbitration record has no persistence path to survive by.
|
||
evidence:
|
||
- 'internal/decision/ring.go:5'
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § Row counts before anything was written'
|
||
|
||
- id: encrypted-database
|
||
title: 'Encrypted database'
|
||
section: 'Operations'
|
||
scope: v1
|
||
domain: [operations, governance]
|
||
state: >-
|
||
`docs/deployment.md`, `docs/caveats/storage.md`.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'partial'
|
||
gap_class: 'capability partial'
|
||
components: [state.db_file, state.db, state.db_tmpfs, proc.mavseal]
|
||
scenarios:
|
||
scenario_note: >-
|
||
none. Checked by `mavseal` and the startup log.
|
||
criteria:
|
||
- id: 'encrypted-database#f1da'
|
||
text: >-
|
||
The database is encrypted at rest with the working copy in tmpfs.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
/var/lib/maven/maven.db.enc, 9.2 MB, mode 0600. The working copy is /dev/shm/maven-plain.db, which is tmpfs.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § What is configured correctly'
|
||
- id: 'encrypted-database#547b'
|
||
text: >-
|
||
The key comes from the environment and is held only by `mavend`.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
db_key_env is MAVEN_DB_KEY and deploy/db_key.env is gitignored. Only mavend mounts /var/lib/maven. Both copies are mode 0600 owned by maven: /var/lib/maven/maven.db.enc at rest and /dev/shm/maven-plain.db in tmpfs, both read with ls inside the container.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § What is configured correctly'
|
||
- 'deploy/mavend.json'
|
||
- id: 'encrypted-database#aa1c'
|
||
text: >-
|
||
Passwords are read from files, never taken as flag values.
|
||
verified: fail
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
The rule holds in the two daemons that are not deployed and breaks in one that is. cmd/mavcaldav/main.go:53 and cmd/mavmaild/main.go:60 both read from a file and say 'never passed as a flag value'; neither is in docker-compose.yml. cmd/mavpoll/main.go:63 takes -kuma-key as a flag value, and cmd/mavpoll/main.go:593 passes it to req.SetBasicAuth as the password. mavpoll is one of the five deployed services.
|
||
evidence:
|
||
- 'cmd/mavpoll/main.go:63'
|
||
- 'cmd/mavpoll/main.go:593'
|
||
- 'docker-compose.yml'
|
||
|
||
- id: passkey-and-step-up
|
||
title: 'Passkey and step-up'
|
||
section: 'Operations'
|
||
scope: v1
|
||
domain: [governance, operations]
|
||
state: >-
|
||
`docs/caveats/security.md`, `internal/webauthn`, `internal/auth`.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'no'
|
||
gap_class: 'capability partial'
|
||
components: [state.wrapped_key, state.passkey_file, core.daemon_lock, core.auth_gate, bnd.http_web]
|
||
scenarios:
|
||
- name: stepup_gate
|
||
exists: false
|
||
scenario_note: >-
|
||
`stepup_gate` *(to write)*.
|
||
criteria:
|
||
- id: 'passkey-and-step-up#98f0'
|
||
text: >-
|
||
WebAuthn is configured, so no step-up gate is fail-open. Today every one is, including `POST /api/chat`, which reaches the act path (V-683).
|
||
verified: fail
|
||
reason: 'configuration missing'
|
||
detail: >-
|
||
WebAuthn is unconfigured. mavweb logs SECURITY WARNING at startup naming six unguarded surfaces, POST /api/chat among them. /auth/passkey returns 404. V-683 unchanged and now measured.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § Every step-up gate is fail-open'
|
||
- id: 'passkey-and-step-up#61f6'
|
||
text: >-
|
||
Enrollment requires an existing credential once the first one exists.
|
||
verified: blocked
|
||
reason: 'configuration missing'
|
||
detail: >-
|
||
No credential exists, so the once-one-exists rule has nothing to trigger it.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § Every step-up gate is fail-open'
|
||
- id: 'passkey-and-step-up#0603'
|
||
text: >-
|
||
Step-up is per-request, not process-global.
|
||
verified: blocked
|
||
reason: 'configuration missing'
|
||
detail: >-
|
||
Not reachable while every gate is fail-open.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § Every step-up gate is fail-open'
|
||
|
||
- id: model-swap
|
||
title: 'Model swap'
|
||
section: 'Operations'
|
||
scope: deferred
|
||
domain: [operations]
|
||
state: >-
|
||
`docs/deployment.md` (V-250).
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'partial'
|
||
deployed: 'partial'
|
||
reachable: 'partial'
|
||
verified: 'no'
|
||
gap_class: 'capability exists but unreachable'
|
||
components: [core.modelswap, svc.llama_server]
|
||
deferred_note: >-
|
||
**Deferred past v1**, owner's call 2026-08-15. See `docs/roadmap.md`.
|
||
scenarios:
|
||
scenario_note: >-
|
||
none.
|
||
criteria:
|
||
- id: 'model-swap#1167'
|
||
text: >-
|
||
`phraser.swap_models` lists the allowed gguf paths, or the page is removed.
|
||
verified: untested
|
||
reason: 'deferred past v1'
|
||
- id: 'model-swap#9a0b'
|
||
text: >-
|
||
A swap survives a restart, or the page states that it will not.
|
||
verified: untested
|
||
reason: 'deferred past v1'
|
||
|
||
- id: self-update
|
||
title: 'Self-update'
|
||
section: 'Operations'
|
||
scope: deferred
|
||
domain: [operations]
|
||
state: >-
|
||
`cmd/mavupdate`, `internal/update`. Blocked at step 3: it cannot reach the containerized socket (V-477).
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'no'
|
||
gap_class: 'capability exists but unverified'
|
||
components: [proc.mavupdate]
|
||
deferred_note: >-
|
||
**Deferred past v1**, owner's call 2026-08-15. See `docs/roadmap.md`.
|
||
scenarios:
|
||
scenario_note: >-
|
||
none.
|
||
criteria:
|
||
- id: 'self-update#2494'
|
||
text: >-
|
||
An update runs to completion from inside the deployment.
|
||
verified: untested
|
||
reason: 'deferred past v1'
|
||
- id: 'self-update#2d51'
|
||
text: >-
|
||
A failed update rolls back and says so.
|
||
verified: untested
|
||
reason: 'deferred past v1'
|
||
|
||
- id: tests-and-analyzers
|
||
title: 'Tests and analyzers'
|
||
section: 'Operations'
|
||
scope: v1
|
||
domain: [operations]
|
||
state: >-
|
||
`docs/qa.md`, `docs/workflow.md`.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'partial'
|
||
gap_class: 'capability partial'
|
||
components: [eval.gates, eval.router, eval.phrasing]
|
||
scenarios:
|
||
scenario_note: >-
|
||
none. This gate is the suite itself.
|
||
criteria:
|
||
- id: 'tests-and-analyzers#f070'
|
||
text: >-
|
||
`make test` is green with `-race` and `MAVEN_ONNX_LIB` set, so the four `TestONNX*` measurements run instead of self-skipping.
|
||
verified: fail
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
make test is green with -race and -coverprofile and does NOT set MAVEN_ONNX_LIB. Makefile:196 is the whole recipe and the variable is absent; only make t and the four eval-* targets set it. Run under the test target's own environment, all eight TestONNXPersonalBoundary* self-skip in 0.00s and the package still prints ok. Grepping the make test output for 'skip' returns nothing because go test prints no SKIP line without -v.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § Tests and analyzers'
|
||
- 'Makefile:196'
|
||
- id: 'tests-and-analyzers#2c7f'
|
||
text: >-
|
||
`make analyze` passes against its baselines, and a fix deletes its entry.
|
||
verified: fail
|
||
reason: 'external dependency unavailable'
|
||
detail: >-
|
||
make analyze does not pass. staticcheck and deadcode produced nothing over their baselines. govulncheck reports 7 vulnerabilities, every one in the standard library, every one fixed in go1.25.13, and the vendored toolchain is go1.25.12: make: *** [Makefile:119: vuln] Error 3. The fix is a toolchain bump, not a code change, and the fixed toolchain is not vendored.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § Tests and analyzers'
|
||
- 'Makefile:119'
|
||
- id: 'tests-and-analyzers#edf0'
|
||
text: >-
|
||
The one latency test that fails only under coverage is fixed or filed (V-718).
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
The criterion is 'fixed or filed' and V-718 is filed. This run neither reproduces nor refutes it: TestONNXPersonalBoundaryLatency passed in 5.17s under make t, which sets MAVEN_ONNX_LIB and uses no coverage, and it self-skips under make test, which is the only invocation that does use coverage.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § Tests and analyzers'
|
||
|
||
- id: email-triage
|
||
title: 'Email triage'
|
||
section: 'Undesigned in v1'
|
||
scope: v1
|
||
domain: [action, governance]
|
||
state: >-
|
||
`internal/email`, `cmd/mavmaild`. Built and **not in `docker-compose.yml`**.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'partial'
|
||
deployed: 'no'
|
||
reachable: 'no'
|
||
verified: 'partial'
|
||
gap_class: 'capability exists but unreachable'
|
||
components: [proc.mavmaild, core.mail_intake, state.maildata]
|
||
finding: >-
|
||
the product decision comes first. What she does with his mail is undecided, and deploying the daemon before deciding writes the decision by accident.
|
||
scenarios:
|
||
- name: email_triage
|
||
exists: false
|
||
scenario_note: >-
|
||
`email_triage` *(to write)*.
|
||
criteria:
|
||
- id: 'email-triage#cce7'
|
||
text: >-
|
||
The owner has decided what triage means: read-only summary, task extraction, or reply drafting.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
The decision is recorded: task extraction into candidates he reviews on /tasks, and explicitly not the acting variant. docker-compose.yml:149 says core 'writes what the model extracts as task CANDIDATES he reviews on /tasks' and that 'nothing here can create a reminder, so a misread mail cannot fire'.
|
||
evidence:
|
||
- 'docker-compose.yml:149'
|
||
- id: 'email-triage#5e45'
|
||
text: >-
|
||
Mail-derived task candidates stay candidates until he accepts one (V-130).
|
||
verified: blocked
|
||
reason: 'deployment missing'
|
||
detail: >-
|
||
mavmaild is built and docker-compose.yml runs five services: mavend, mavsttd, mavttsd, mavweb, mavpoll. It is not among them. The spec makes the product decision the first criterion and that decision has not been recorded anywhere.
|
||
evidence:
|
||
- 'docker-compose.yml'
|
||
- 'docs/spec.md § Undesigned in v1'
|
||
- id: 'email-triage#9a78'
|
||
text: >-
|
||
No mail content leaves the box.
|
||
verified: blocked
|
||
reason: 'deployment missing'
|
||
detail: >-
|
||
mavmaild is built and docker-compose.yml runs five services: mavend, mavsttd, mavttsd, mavweb, mavpoll. It is not among them. The spec makes the product decision the first criterion and that decision has not been recorded anywhere.
|
||
evidence:
|
||
- 'docker-compose.yml'
|
||
- 'docs/spec.md § Undesigned in v1'
|
||
- id: 'email-triage#e37e'
|
||
text: >-
|
||
`mavmaild` is in compose, or its absence is deliberate and recorded.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
The second disjunct holds. docker-compose.yml:145 carries a ten-line commented block naming why mavmaild is off (no IMAP account on this box), the security shape (password from a file, core never sees it, nothing there can create a reminder), and the exact steps to enable it. The absence is deliberate and recorded, in the deployed artifact.
|
||
evidence:
|
||
- 'docker-compose.yml:145'
|
||
|
||
- id: calendar-management
|
||
title: 'Calendar management'
|
||
section: 'Undesigned in v1'
|
||
scope: v1
|
||
domain: [action, deliberation]
|
||
state: >-
|
||
`internal/calendar`, `cmd/mavcaldav`. Built and **not in `docker-compose.yml`**. Same product decision as email.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'partial'
|
||
reachable: 'partial'
|
||
verified: 'partial'
|
||
gap_class: 'capability exists but unreachable'
|
||
components: [proc.mavcaldav, core.q.calendar]
|
||
scenarios:
|
||
- name: calendar_create
|
||
exists: false
|
||
scenario_note: >-
|
||
`calendar_create` *(to write)*.
|
||
criteria:
|
||
- id: 'calendar-management#2ebc'
|
||
text: >-
|
||
Today's and tomorrow's events are read back, times in `Europe/Samara`.
|
||
verified: blocked
|
||
reason: 'external dependency unavailable'
|
||
detail: >-
|
||
Not an undecided product question. docker-compose.yml:175 names a concrete blocker: there is no CalDAV account on this box, so mavcaldav is commented out and the calendar query source reads facts nobody writes. Nothing about reading back, creating by voice or naming a conflict can be exercised until an account exists.
|
||
evidence:
|
||
- 'docker-compose.yml:175'
|
||
- id: 'calendar-management#4047'
|
||
text: >-
|
||
An event is created by voice, with the slot asked for rather than guessed.
|
||
verified: blocked
|
||
reason: 'external dependency unavailable'
|
||
detail: >-
|
||
Not an undecided product question. docker-compose.yml:175 names a concrete blocker: there is no CalDAV account on this box, so mavcaldav is commented out and the calendar query source reads facts nobody writes. Nothing about reading back, creating by voice or naming a conflict can be exercised until an account exists.
|
||
evidence:
|
||
- 'docker-compose.yml:175'
|
||
- id: 'calendar-management#5f7c'
|
||
text: >-
|
||
A conflicting event is stated as a conflict.
|
||
verified: blocked
|
||
reason: 'external dependency unavailable'
|
||
detail: >-
|
||
Not an undecided product question. docker-compose.yml:175 names a concrete blocker: there is no CalDAV account on this box, so mavcaldav is commented out and the calendar query source reads facts nobody writes. Nothing about reading back, creating by voice or naming a conflict can be exercised until an account exists.
|
||
evidence:
|
||
- 'docker-compose.yml:175'
|
||
- id: 'calendar-management#ff27'
|
||
text: >-
|
||
`mavcaldav` is in compose, or its absence is deliberate and recorded.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
The second disjunct holds and the block says so outright. docker-compose.yml:175 records that mavcaldav 'was built, listed in make build, and deployed nowhere, which is the worst of the three states, this block records the decision instead', then writes out what the absence costs: agenda questions route correctly and answer from an empty table, and loop.State.CalendarBusy stays false.
|
||
evidence:
|
||
- 'docker-compose.yml:175'
|
||
|
||
- id: web-crawling
|
||
title: 'Web crawling'
|
||
section: 'Undesigned in v1'
|
||
scope: v1
|
||
domain: [action, governance]
|
||
state: >-
|
||
`internal/crawl` with `robots.go` and `watch.go`, `internal/webfetch`. No living doc.
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'no'
|
||
gap_class: 'capability partial'
|
||
components: [core.crawl_worker, core.q.web]
|
||
finding: >-
|
||
politeness and robots are implemented. The scheduling policy is not written anywhere.
|
||
scenarios:
|
||
- name: crawl_watch
|
||
exists: false
|
||
scenario_note: >-
|
||
`crawl_watch` *(to write)*.
|
||
criteria:
|
||
- id: 'web-crawling#396b'
|
||
text: >-
|
||
A page is fetched on request and summarised in Russian.
|
||
verified: untested
|
||
reason: 'scenario missing'
|
||
detail: >-
|
||
crawl.on_demand is true with a 10s timeout and a 4000-rune cap, and internal/crawl carries robots.go and watch.go. No probe asked for a page, so nothing about politeness, the schedule or what leaves the box was observed.
|
||
evidence:
|
||
- 'deploy/mavend.json crawl'
|
||
- 'docs/capabilities/probes_field.json'
|
||
- id: 'web-crawling#cfa6'
|
||
text: >-
|
||
A watched page reports what changed, on a schedule he set by voice.
|
||
verified: fail
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
Watches are read only from cfg.Crawl.Watches in cmd/mavend/crawls.go. No grammar, intent or IPC method creates one, so a schedule cannot be set by voice, and the deployed crawl block configures no watches at all.
|
||
evidence:
|
||
- 'cmd/mavend/crawls.go'
|
||
- 'deploy/mavend.json'
|
||
- id: 'web-crawling#aece'
|
||
text: >-
|
||
robots and the politeness delay are honoured, observable in the log.
|
||
verified: untested
|
||
reason: 'scenario missing'
|
||
detail: >-
|
||
crawl.on_demand is true with a 10s timeout and a 4000-rune cap, and internal/crawl carries robots.go and watch.go. No probe asked for a page, so nothing about politeness, the schedule or what leaves the box was observed.
|
||
evidence:
|
||
- 'deploy/mavend.json crawl'
|
||
- 'docs/capabilities/probes_field.json'
|
||
- id: 'web-crawling#29e4'
|
||
text: >-
|
||
Only the URL and the utterance leave the box.
|
||
verified: untested
|
||
reason: 'scenario missing'
|
||
detail: >-
|
||
crawl.on_demand is true with a 10s timeout and a 4000-rune cap, and internal/crawl carries robots.go and watch.go. No probe asked for a page, so nothing about politeness, the schedule or what leaves the box was observed.
|
||
evidence:
|
||
- 'deploy/mavend.json crawl'
|
||
- 'docs/capabilities/probes_field.json'
|
||
|
||
- id: summaries
|
||
title: 'Summaries'
|
||
section: 'Undesigned in v1'
|
||
scope: v1
|
||
domain: [action, governance]
|
||
state: >-
|
||
no package.
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'no'
|
||
wired: 'no'
|
||
configured: 'no'
|
||
deployed: 'no'
|
||
reachable: 'no'
|
||
verified: 'no'
|
||
gap_class: 'capability missing'
|
||
components: []
|
||
finding: >-
|
||
summarisation exists inside the world chain and inside digestion, and nothing owns it as a capability he can ask for.
|
||
scenarios:
|
||
- name: summarise
|
||
exists: false
|
||
scenario_note: >-
|
||
`summarise` *(to write)*.
|
||
criteria:
|
||
- id: 'summaries#5846'
|
||
text: >-
|
||
"перескажи" over a note, a feed item, a page or a mail returns a Russian summary.
|
||
verified: fail
|
||
reason: 'code missing'
|
||
detail: >-
|
||
No package owns summarisation as a capability he can ask for. Summarising exists inside the world chain and inside digestion and nowhere as a thing to request. internal/ has no summar* package.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § Undesigned in v1, by inspection'
|
||
- id: 'summaries#546a'
|
||
text: >-
|
||
The summary names its source.
|
||
verified: fail
|
||
reason: 'code missing'
|
||
detail: >-
|
||
No package owns summarisation as a capability he can ask for. Summarising exists inside the world chain and inside digestion and nowhere as a thing to request. internal/ has no summar* package.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § Undesigned in v1, by inspection'
|
||
- id: 'summaries#bb6d'
|
||
text: >-
|
||
A summary that would invent content is refused, the way `Response.Empty()` gates a world answer.
|
||
verified: fail
|
||
reason: 'code missing'
|
||
detail: >-
|
||
No package owns summarisation as a capability he can ask for. Summarising exists inside the world chain and inside digestion and nowhere as a thing to request. internal/ has no summar* package.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § Undesigned in v1, by inspection'
|
||
|
||
- id: webhooks
|
||
title: 'Webhooks'
|
||
section: 'Undesigned in v1'
|
||
scope: v1
|
||
domain: [interaction, governance]
|
||
state: >-
|
||
only `internal/delivery/telegramsink/intake.go`, which is Telegram's own inbound webhook.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'no'
|
||
wired: 'no'
|
||
configured: 'no'
|
||
deployed: 'no'
|
||
reachable: 'no'
|
||
verified: 'no'
|
||
gap_class: 'capability missing'
|
||
components: []
|
||
finding: >-
|
||
there is no general webhook capability in either direction, and no doc says which direction is wanted.
|
||
scenarios:
|
||
- name: webhook_inbound
|
||
exists: false
|
||
scenario_note: >-
|
||
`webhook_inbound` *(to write)*.
|
||
criteria:
|
||
- id: 'webhooks#c2c3'
|
||
text: >-
|
||
Inbound: an authenticated external event becomes a fact or a nudge candidate.
|
||
verified: fail
|
||
reason: 'code missing'
|
||
detail: >-
|
||
The box has no inbound HTTP intake at all. internal/delivery/telegramsink/intake.go long-polls getUpdates outbound precisely because the box accepts no inbound connections and reaches api.telegram.org through a socks relay. There is no general webhook capability in either direction and internal/ has no webhook package.
|
||
evidence:
|
||
- 'internal/delivery/telegramsink/intake.go'
|
||
- id: 'webhooks#10d7'
|
||
text: >-
|
||
Outbound: a nudge can post to a configured URL as a reach.
|
||
verified: fail
|
||
reason: 'code missing'
|
||
detail: >-
|
||
The box has no inbound HTTP intake at all. internal/delivery/telegramsink/intake.go long-polls getUpdates outbound precisely because the box accepts no inbound connections and reaches api.telegram.org through a socks relay. There is no general webhook capability in either direction and internal/ has no webhook package.
|
||
evidence:
|
||
- 'internal/delivery/telegramsink/intake.go'
|
||
- id: 'webhooks#5134'
|
||
text: >-
|
||
Neither direction is reachable without authentication.
|
||
verified: fail
|
||
reason: 'code missing'
|
||
detail: >-
|
||
The box has no inbound HTTP intake at all. internal/delivery/telegramsink/intake.go long-polls getUpdates outbound precisely because the box accepts no inbound connections and reaches api.telegram.org through a socks relay. There is no general webhook capability in either direction and internal/ has no webhook package.
|
||
evidence:
|
||
- 'internal/delivery/telegramsink/intake.go'
|
||
|
||
- id: cron-jobs
|
||
title: 'Cron jobs'
|
||
section: 'Undesigned in v1'
|
||
scope: v1
|
||
domain: [action, governance]
|
||
state: >-
|
||
`internal/routine`, `cmd/mavend/tick_routines.go`.
|
||
implementation:
|
||
designed: 'yes'
|
||
code_present: 'yes'
|
||
wired: 'yes'
|
||
configured: 'yes'
|
||
deployed: 'yes'
|
||
reachable: 'yes'
|
||
verified: 'no'
|
||
gap_class: 'capability partial'
|
||
components: [core.routines, core.tick_loop]
|
||
finding: >-
|
||
routines carry a `Cron` and are a separate mechanism from reminders. Whether "cron jobs" means user-defined scheduled acts or the existing routines is undecided.
|
||
scenarios:
|
||
- name: cron_job
|
||
exists: false
|
||
scenario_note: >-
|
||
`cron_job` *(to write)*.
|
||
criteria:
|
||
- id: 'cron-jobs#d326'
|
||
text: >-
|
||
A scheduled job is created by voice, with its schedule stated back.
|
||
verified: untested
|
||
reason: 'scenario missing'
|
||
detail: >-
|
||
Three scheduling mechanisms exist side by side: routines carry a Cron, reminders carry a Cron no caller passes, and the digest runs on a tick. The spec's last criterion asks for that relationship to be written down so three schedulers do not exist; nothing writes it down.
|
||
evidence:
|
||
- 'docs/spec.md § Cron jobs'
|
||
- 'internal/routine'
|
||
- 'cmd/mavend/tick_routines.go'
|
||
- id: 'cron-jobs#5ec4'
|
||
text: >-
|
||
It runs on schedule in `Europe/Samara` and its run is recorded.
|
||
verified: untested
|
||
reason: 'scenario missing'
|
||
detail: >-
|
||
Three scheduling mechanisms exist side by side: routines carry a Cron, reminders carry a Cron no caller passes, and the digest runs on a tick. The spec's last criterion asks for that relationship to be written down so three schedulers do not exist; nothing writes it down.
|
||
evidence:
|
||
- 'docs/spec.md § Cron jobs'
|
||
- 'internal/routine'
|
||
- 'cmd/mavend/tick_routines.go'
|
||
- id: 'cron-jobs#d446'
|
||
text: >-
|
||
A job that runs an act is bound by the same confirmation rules as any act.
|
||
verified: untested
|
||
reason: 'scenario missing'
|
||
detail: >-
|
||
Three scheduling mechanisms exist side by side: routines carry a Cron, reminders carry a Cron no caller passes, and the digest runs on a tick. The spec's last criterion asks for that relationship to be written down so three schedulers do not exist; nothing writes it down.
|
||
evidence:
|
||
- 'docs/spec.md § Cron jobs'
|
||
- 'internal/routine'
|
||
- 'cmd/mavend/tick_routines.go'
|
||
- id: 'cron-jobs#acad'
|
||
text: >-
|
||
Its relationship to routines and to recurring reminders is written down, so three schedulers do not exist.
|
||
verified: fail
|
||
reason: 'code missing'
|
||
detail: >-
|
||
The document does not exist. Three scheduling mechanisms run side by side: routines carry a Cron read by cmd/mavend/tick_routines.go, reminders carry a Cron no caller in cmd/mavend passes, and the digest runs on its own tick. No doc under docs/ relates them, and the spec asks for exactly that so three schedulers do not exist.
|
||
evidence:
|
||
- 'internal/routine'
|
||
- 'cmd/mavend/tick_routines.go'
|
||
|
||
- id: learning-the-style
|
||
title: 'Learning the style'
|
||
section: 'Undesigned in v1'
|
||
scope: v1
|
||
domain: [memory, interaction]
|
||
state: >-
|
||
no package.
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'no'
|
||
wired: 'no'
|
||
configured: 'no'
|
||
deployed: 'no'
|
||
reachable: 'no'
|
||
verified: 'partial'
|
||
gap_class: 'capability missing'
|
||
components: []
|
||
finding: >-
|
||
nothing exists beyond `internal/phraser/eval/checks.go`, which scores style and does not learn it. Learning means behavioral, not weights: stored outcomes, no adapter, no training set.
|
||
scenarios:
|
||
- name: style_correction
|
||
exists: false
|
||
scenario_note: >-
|
||
`style_correction` *(to write)*.
|
||
criteria:
|
||
- id: 'learning-the-style#d4b0'
|
||
text: >-
|
||
A corrected phrasing is stored as an outcome and changes a later reply.
|
||
verified: fail
|
||
reason: 'code missing'
|
||
detail: >-
|
||
Nothing exists beyond internal/phraser/eval/checks.go, which scores style and does not learn it. internal/ has no learn* or outcome* package.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § Undesigned in v1, by inspection'
|
||
- id: 'learning-the-style#3a1c'
|
||
text: >-
|
||
No model weights change and no training set is built.
|
||
verified: pass
|
||
reason: 'passes'
|
||
detail: >-
|
||
A prohibition, and it holds vacuously. Nothing on the box learns style, so no model weights change and no style training set is built. internal/phraser/eval/checks.go scores style and does not learn it.
|
||
evidence:
|
||
- 'internal/phraser/eval/checks.go'
|
||
- id: 'learning-the-style#db0e'
|
||
text: >-
|
||
What was learned is readable on a page and can be deleted.
|
||
verified: fail
|
||
reason: 'code missing'
|
||
detail: >-
|
||
Nothing exists beyond internal/phraser/eval/checks.go, which scores style and does not learn it. internal/ has no learn* or outcome* package.
|
||
evidence:
|
||
- 'docs/evals/2026-08-26-capability-baseline.md § Undesigned in v1, by inspection'
|
||
|
||
- id: learning-from-mistakes
|
||
title: 'Learning from mistakes'
|
||
section: 'Undesigned in v1'
|
||
scope: v1
|
||
domain: [memory, deliberation]
|
||
state: >-
|
||
no package. Same behavioral rule as above.
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'no'
|
||
wired: 'no'
|
||
configured: 'no'
|
||
deployed: 'no'
|
||
reachable: 'no'
|
||
verified: 'no'
|
||
gap_class: 'capability missing'
|
||
components: []
|
||
scenarios:
|
||
- name: learn_from_dismissal
|
||
exists: false
|
||
scenario_note: >-
|
||
`learn_from_dismissal` *(to write)*.
|
||
criteria:
|
||
- id: 'learning-from-mistakes#858a'
|
||
text: >-
|
||
A dismissed nudge, a corrected phrasing and a repaired route are each stored as an outcome.
|
||
verified: fail
|
||
reason: 'code missing'
|
||
detail: >-
|
||
Two of three clauses already hold and one does not. A dismissed nudge is stored as an outcome in the nudges table, a dismissed routine proposal in proposed_routines, and a repaired route twice over, as a routing_labels row and as a classifier example. There is no store for a corrected phrasing: internal/ has no outcome package for style and nothing writes one.
|
||
evidence:
|
||
- 'cmd/mavend/repair.go'
|
||
- id: 'learning-from-mistakes#66f0'
|
||
text: >-
|
||
A repeated dismissal suppresses that nudge shape.
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
The suppression loop is built, wired and enabled. The nudges table stores an outcome per nudge, store.RecentOutcomes reads the last N non-pending outcomes per rule, and loop.TuneCooldown turns a high ignored rate into a longer cooldown. No probe produced repeated dismissals of one rule, so nothing about its behaviour was observed.
|
||
evidence:
|
||
- 'internal/loop'
|
||
- id: 'learning-from-mistakes#3203'
|
||
text: >-
|
||
A repaired route changes the next routing of the same utterance.
|
||
verified: untested
|
||
reason: 'implementation exists with no runtime proof'
|
||
detail: >-
|
||
Two correction paths exist, not one. POST /api/correct writes a routing_labels row, and cmd/mavend/repair.go (V-455) is a spoken correction taken pre-route that calls Router.CorrectMisroute and appends the utterance as a classifier example. routing_labels held 2 rows before the wipe. No probe corrected a route and then re-uttered it, so whether the next route changes was not observed.
|
||
evidence:
|
||
- 'cmd/mavend/repair.go'
|
||
- id: 'learning-from-mistakes#7c66'
|
||
text: >-
|
||
Every stored outcome is readable and deletable.
|
||
verified: fail
|
||
reason: 'code missing'
|
||
detail: >-
|
||
Same split as 858a. Nudge outcomes and routing labels are readable over IPC and through /api/revert-adjacent surfaces; a corrected phrasing has no store to be readable or deletable from.
|
||
evidence:
|
||
- 'cmd/mavend/repair.go'
|
||
|
||
- id: command-chaining
|
||
title: 'Command chaining'
|
||
section: 'Undesigned in v1'
|
||
scope: v1
|
||
domain: [deliberation, governance]
|
||
state: >-
|
||
no package. The `chain` in `internal/router` is the world chain and the source chain, not command chaining.
|
||
implementation:
|
||
designed: 'spec-only'
|
||
code_present: 'no'
|
||
wired: 'no'
|
||
configured: 'no'
|
||
deployed: 'no'
|
||
reachable: 'no'
|
||
verified: 'no'
|
||
gap_class: 'capability missing'
|
||
components: []
|
||
finding: >-
|
||
nothing exists.
|
||
scenarios:
|
||
- name: command_chain
|
||
exists: false
|
||
scenario_note: >-
|
||
`command_chain` *(to write)*.
|
||
criteria:
|
||
- id: 'command-chaining#da3d'
|
||
text: >-
|
||
"напомни мне и запиши это" performs both, or asks which one.
|
||
verified: fail
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
The compound parse exists and is live, and the dispatch does not. voice.llm_router is true on the deployed box and the router's grammar contract returns an array of actions for a compound utterance, which parseActions builds in full. Nothing dispatches actions beyond the first. The seam is internal/router/llmrouter.go, not an absent package.
|
||
evidence:
|
||
- 'internal/router/llmrouter.go'
|
||
- 'deploy/mavend.json'
|
||
- id: 'command-chaining#77b5'
|
||
text: >-
|
||
A chain containing an act confirms each act separately.
|
||
verified: fail
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
The compound parse exists and is live, and the dispatch does not. voice.llm_router is true on the deployed box and the router's grammar contract returns an array of actions for a compound utterance, which parseActions builds in full. Nothing dispatches actions beyond the first. The seam is internal/router/llmrouter.go, not an absent package.
|
||
evidence:
|
||
- 'internal/router/llmrouter.go'
|
||
- 'deploy/mavend.json'
|
||
- id: 'command-chaining#d4fa'
|
||
text: >-
|
||
A failed step stops the chain and names the step that failed.
|
||
verified: fail
|
||
reason: 'wiring missing'
|
||
detail: >-
|
||
The compound parse exists and is live, and the dispatch does not. voice.llm_router is true on the deployed box and the router's grammar contract returns an array of actions for a compound utterance, which parseActions builds in full. Nothing dispatches actions beyond the first. The seam is internal/router/llmrouter.go, not an absent package.
|
||
evidence:
|
||
- 'internal/router/llmrouter.go'
|
||
- 'deploy/mavend.json'
|
||
|