ShadowHarness records turns where both the legacy router and the
experiment model produce a decision. Reports agreement/disagreement
split by fast-path vs residual. No action, clarification, capability
selection, or reply depends on the shadow result.
EvalReport with macro F1, per-route precision/recall/F1, confusion
matrix, false-action rate, and fast-path vs residual breakdown.
ScoreEval runs a SemanticRouter against a frozen eval set and produces
all metrics needed for promotion decisions.
Six deterministic transforms (negation, question, reported speech,
quotation, hypothetical, capability question) applied to action-route
seeds. Each transform determines the expected class explicitly — no
model guessing labels. SplitByFamily uses hash-based bucketing to keep
paraphrases in the same split.
136 examples with provenance from ru_routing_v1.json, tagged
fast_path_resolved vs residual. RouteExample carries source, source_id,
split_group for traceability. Split-by-family prevents paraphrase
leakage across train/eval.
Defines the six-class SemanticRoute type (conversation, knowledge,
action, memory_write, system, uncertain), the SemanticRouteDecision
output, the SemanticRouter interface, and the deterministic
Intent→SemanticRoute mapping from the current seven-intent cascade.
Migrate 54 test call sites to construct NormalizedInput{Text: ...}.
Add invariant tests:
- TestNormalizedInputReachesRouteIntact: ingress NormalizedInput reaches Route
- TestTryFastPathReceivesMatchText: TryFastPath gets the same input
- TestMatchTextDoesNotChangeRouting: same Text + different MatchText → same Decision
- TestDecisionUtteranceEqualsInputText: Decision.Utterance == input.Text
Change Route from (ctx, utterance string, now) to (ctx, input NormalizedInput, now).
The ingress-constructed NormalizedInput now reaches the cascade intact — no
reconstruction downstream. TryFastPath receives the same input, not a rebuilt one.
All callers (production, eval framework, tests) updated to construct NormalizedInput.
Add MatchText field to NormalizedInput — a lossy lexical matching view
derived from ingress text: TrimSpace → NFKC → lowercase → collapse
Unicode whitespace. Does NOT fold ё→е, strip punctuation, strip wake
words, rewrite numbers, or invoke morphology.
Both ingress sites (voice + text) construct MatchText at entry. No
existing consumer reads MatchText yet — it is dark data for future
opt-in migration.
Vikunja: #725
Migrate the two remaining action-routing consumers from compatibility
Decision.Slings fields to authoritative Decision.CapabilitySelection:
- refusesCommand: reads CapabilitySelection.Fn instead of Slots.Fn
- ActHasEntityTarget: reads CapabilitySelection.Resolved and
CapabilitySelection.Args instead of Slots.HasFn and Slots.Args
Slots.Text remains the source for entity text when positional args do
not contain the target (unchanged).
Regression tests prove:
- prohibited sentinel preserved byte-for-byte through SelectCapability
- blanked Slots.Fn/Args/HasFn do not affect migrated consumers
- Praxis/Hexis entity-target routing unchanged
- stage-0 deterministic act unchanged
- classifier/extractor act unchanged
do not remove the compatibility mirrors yet.
Set CapabilitySelection on decisions that have Slots.HasFn=true, so
ResolveActionCandidate reads from the authoritative record. Backward
compatibility tests verify that decisions without CapabilitySelection
still resolve via Slots.HasFn.
The candidate now receives Fn/Args from CapabilitySelection (the
authoritative record) rather than from Decision.Slots.HasFn. Backward
compatibility: decisions with Slots.HasFn but no CapabilitySelection
(tests, rebuilt decisions) still resolve via the compatibility path.
Authoritative record of which executable capability matched, separate
from Decision.Intent (what kind of turn) and ActionCandidate (downstream
action artifact). Decision.Slots.Fn/Args/HasFn remain as compatibility
representations populated from this selection.
Call SelectCapability after each cascade path (grammar, heads, LLM,
classifier) and propagate the result via applyCapabilityToSlots.
Remove LLM text capability backfill from fillSlots — SelectCapability
now owns that path. fillMatchedSlots retains raw extractor capability
extraction for backward compatibility with stage-0 grammars.
gateLLMDecision now reads CapabilitySelection.Resolved instead of
Slots.HasFn for the act-intent confidence thinning check.
Add the explicit capability-selection boundary between route resolution
and action candidate production. SelectCapability is the single entry
point for selecting which executable capability matched an IntentAct turn.
Three input kinds: raw, llm_text, deterministic. Decision.CapabilitySelection
is the authoritative record; Decision.Slots.Fn/Args/HasFn remain as
compatibility representations populated from the selection.
Run routing and ecosystem fixtures through the baseline router and
report which component selected the exact function for every IntentAct
case. Shadow matcher comparison confirms zero disagreements.
Add ResolvedBy to dialogue.Slots and the toDialogueSlots/
applyDialogueSlots converters. Skip reflect-type check for this field
in parity test (dialogue cannot import router: import cycle).
Five disjoint values tracking which component selected the exact function:
grammar_fixed, grammar_matcher, extractor_raw, extractor_llm_text,
fallback_matcher. Slots.ResolvedBy carries provenance at the selection
point.
Introduce ActionValidationStatus enum (valid, unresolved, missing_argument,
invalid_argument, ambiguous_target) as the typed classification of validation
outcomes. ActionValidationResult now carries Status instead of boolean flags.
Backward-compatible: Unresolved() and Valid() methods preserved on the result.
Existing validation behavior unchanged: only blank Fn produces invalid_argument.
All downstream behavior (proposeGap, confirmation, task_status, praxis, hexis)
unchanged.
Tests added for all five status values, backward compatibility, and the full
validation → execution boundary.
Daemon half of the action-resolution boundary:
- Add resolveAction wrapper: delegates to ResolveActionCandidate,
records outcome in the decision trace (action-resolve:route/matcher)
- Refactor actionAct: remove matcher call, consume candidate, write
resolved values back into Slots for downstream branches
- Add 8 integration tests pinning all required scenarios:
route-sourced, matcher-sourced, matcher miss, destructive confirm,
task-status intercept, stage-0, learned-router, alias match
All existing tests pass. Execution/risk/confirmation unchanged.
Introduce the typed boundary between routing and action resolution:
- ActionCandidate: Fn, Args, Source (route|matcher), Producer, Confidence
- ResolveActionCandidate(dec, m): standalone function usable by both
the daemon and the eval harness
- Update eval harness Reach() to use ResolveActionCandidate instead of
duplicating the matcher fallback logic
This is the routing-side half of the action-resolution boundary.
The daemon integration follows in the next commit.
12 focused tests proving the first slice properties:
- text and voice enter equivalent typed turn input after stt
- stage-0 outputs remain identical with grammar producer
- classifier floor sets its producer
- clarification carries the classifier producer
- pre-route claims produce no route producer
- route producer appears on the decision record
- input source is preserved on the decision record
First behavior-preserving slice of the Maven redesign. Establishes
explicit ingress/routing boundaries and enough observability to refactor
later without changing current routing, action, clarification, or
execution semantics.
Types introduced:
- NormalizedInput (internal/router/source.go): Text + InputSource,
the typed ingress boundary replacing raw string at the turn entry.
- InputSource (internal/router/source.go): channel provenance enum
(tap:voice, tap:text). Reuses the existing turnSource distinction.
- RouteProducer (internal/router/intent.go): which cascade stage
produced the decision (grammar, heads, llm, classifier).
Changes:
- Decision carries a Producer RouteProducer field, set at each cascade
stage (grammar, heads, LLM, classifier).
- turnRoute carries NormalizedInput instead of bare text string.
- runTurn takes NormalizedInput instead of (text, src).
- decision.Record carries InputSource and RouteProducer for
observability; RoutingTrace persists route_producer (migration #27).
- turnSource is now a type alias for router.InputSource.
Behavior preserved:
- Stage-0 grammars unchanged: same order, same matching, same confidence.
- Cascade fallthrough order unchanged (grammar → heads → llm → classifier).
- Clarification behavior unchanged.
- Action dispatch unchanged.
- No new linguistic normalization.
Two defects found by screenshotting the built page under headless chromium,
which is the only way to see either.
The six relation filters and the component-type legend do nothing in views 6 and
7. Leaving them on screen reads as controls that are broken.
The ledger carries markdown inline code, because docs/spec.md does. The side
panel printed the backticks literally beside every path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Session 3, and the end of the plan.
View 6 is the matrix: 51 capabilities against designed, code_present, wired,
configured, deployed, reachable and verified, grouped by spec section or by
domain. Clicking a row opens the definition of done with every verdict, its
reason, its detail and its evidence paths, the components that carry the
capability, the blockers and the product questions it waits on.
View 7 is the twelve invariants. Each shows three things apart: target is
whether the rule is written down, implementation is the status of the
participating components, and runtime is what the probe run observed for the
capabilities it touches. Component and capability chips cross-link into the
other views.
invariants.yaml is the machine-readable half of invariants.md. The two exist
separately so the viewer can read one and a person can read the other, and
build_ledger.py refuses to build when they disagree: a missing heading, a count
mismatch, an unknown capability or component, or an unresolved invariant with no
product question.
build_viewer.py inlines ledger.yaml and invariants.yaml and derives nothing. The
ledger's build is the only thing allowed to decide a dimension.
check_viewer.js is the viewer's only check. A TypeError in a renderer shows as a
blank panel and not as an error, so it runs all seven views, all three flows,
all 51 capability panels and all 160 component panels against a DOM stub, and
fails on a panel that comes back thin. render.sh calls it and skips it with a
message when node is absent.
--no-verify: the template and the smoke test are 320 non-markdown lines.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
docs/CLAUDE.md defined four tiers and docs/capabilities/ and docs/architecture/
were neither of them. Both are now declared as generated: rebuilt from a source,
never corrected in place. A wrong row in either is a bug in the generator or in
one of its hand-written inputs, and editing the output makes the next rebuild
silently undo the fix.
The eval index gains the 2026-08-19 CPT+SFT measurement and the 2026-08-26
baseline, and marks the 2026-08-13 capability audit superseded. The 2026-08-19
file lands with them: it is the direct evidence that the deployed
maven-instruct-b2 routes better than Qwen3-1.7B and cannot hold a Russian
sentence, which is the first item on the gaps.md priority list.
Its header records Vikunja as returning 503. That reading was wrong and the
2026-08-26 baseline says so, but a dated eval is not edited after the day.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
docs/capabilities/build_ledger.py reads the component statuses out of
maven-architecture.json, so the whole implementation half of the ledger fails to
build on a clone that does not have it. It has to be tracked.
What lands: the five generator scripts, the viewer template, findings.md, the
README and the seven .mmd diagram sources, plus the inventory JSON itself.
verify_anchors.py resolves 681 of 692 claimed symbols to path:line and exits
non-zero on a miss, 11 skipped as config keys. That proves an identifier sits on
a line and nothing more. Writing the responsibility field caught 29 symbols
filed under the wrong component and 7 names invented outright, and a later
refutation pass caught 4 wrong readings on top of that.
What does not land, and is now gitignored: index.html at 836 KB of inlined JSON
and SVG, anchors.md, architecture-evidence.txt, tree.txt, the redacted compose
file, the rendered SVGs and maven-evidence.zip. All of them rebuild with
pack_evidence.sh.
render.sh is the only syntax check this repo has for a .mmd, and it found two
real parse errors on its first run.
--no-verify: 4,900 non-markdown lines. The inventory and its generator are one
artifact and neither is readable without the other.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Session 2 step 3, and the end of the explanation half.
gaps.md compares responsibilities and never package names. Eight classes. The
four capability classes are derived from the ledger's gap_class field and
rebuild with build_ledger.py. The four architecture classes are read from
findings.md and invariants.md, and every entry names the capability or invariant
it affects. An entry naming neither is marked non-blocking cleanup in those
words, which is the whole of class 8 and its eleven rows.
Of 46 v1 capabilities: 5 missing, 21 partial and reachable, 9 built and
unreachable, 11 reachable and unverified.
The nine unreachable ones are seven config blocks and two compose entries. Not
one is a code defect.
The ranked list puts phrasing first: speak-as-herself fails all three criteria,
and everything that asks the resident model to write a Russian sentence inherits
that. His own name not being stored is second. Nine capabilities one config
change from reachable is fourth, and it is the highest ratio of capability to
work in the list.
Items 10, 12 and 13 stall on unresolved invariants and are the owner's call, not
work.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Session 2 step 2. docs/spec.md states 51 capabilities one at a time. Twelve
rules run across all of them and no DoD states any of them, so breaking one
breaks many capabilities at once without producing a failing criterion.
Each is marked explicit, implied or unresolved, with evidence. Nothing wanted is
invented where the sources are silent.
Four are unresolved and belong to the owner rather than to a commit: authority
and confirmation, learning from outcomes, capability composition, and whether a
held nudge has a shelf life. Two of the three questions the freeze was called to
answer show up here as invariants 8 and 11.
Privacy boundaries and proactive attention are the two best-specified rules and
neither showed a live defect. Authority is the largest hole: internal/auth
answers who may carry what authority and does not bind the turn path,
internal/tool answers what effect an act has and is not keyed on the reach, and
praxisItemAction.handle has no gate at all.
The one file in docs/capabilities/ that is hand-written rather than generated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The rebuild section named the new inputs and nothing said what the columns
mean or how a partial arises.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Session 2 step 1. Implementation status was the missing half: the ledger said
what should happen and what happened, and nothing said how much is built.
Never one implemented boolean. designed, code_present, wired, configured,
deployed, reachable and verified are separate, because coded and unwired, wired
and unconfigured, and configured and undeployed are three different pieces of
work.
The six build dimensions derive from the status field of every component the
capability maps to, rolled up as all yes, none no, otherwise partial. The
statuses come from docs/architecture/maven-architecture.json, which read them
from code, config and compose. verified comes from the criteria verdicts.
implementation.yaml is the mapping and is the judgment call. Shared
infrastructure is deliberately unmapped: putting core.reactive_handler on all 51
rows would give them one status and say nothing.
Of 51 capabilities, 45 have code and 33 are reachable. 22 are spec-only, with no
living doc owning the subsystem.
The build now reports what it cannot reconcile. learning-the-style has no
component and still scores a pass, because its passing criterion is negative and
absence satisfies it. Sixteen components serve no capability, ten of them the
shared infrastructure excluded on purpose, and the rest are core.q.habits,
core.q.money, ext.zenmoney, router.claim and router.modes.
--no-verify: the regenerated ledger is 500 lines of derived output.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Filed after the fact, so the plan and the index both said it had none. The
Vikunja task carries the session-1 result and what sessions 2 and 3 still owe.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
What the deployed Maven did when it was asked, measured 2026-08-26 against
master 5cae33a plus the uncommitted deploy/mavend.json model switch. Frozen on
the day and not edited after it.
The number is attributable to the deployed resident model,
maven-instruct-b2-Q4_K_XL, not the Qwen3-1.7B that CLAUDE.md names. What comes
closest to working is what never asks that model to write a Russian sentence.
make test is green throughout. It is also green with the four TestONNX
measurements silently skipped, because the recipe does not set MAVEN_ONNX_LIB.
Every verdict was audited by an independent pass told to refute it, one auditor
per spec section, with pass attacked hardest. The corrections moved the tally by
nine: four passes withdrawn, five criteria filed untested turned out already
settled. The section on how the verdicts were checked names the mistakes, so the
next session does not have to trust that this one got it right.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
verdicts.json carries one verdict per criterion id. ledger.yaml is what
build_ledger.py produces from docs/spec.md, domains.yaml and those verdicts.
146 v1 criteria: 26 pass, 50 fail, 15 blocked, 51 untested, 4 unknown. No
capability passes all of its own criteria. Six fail every one: speak as herself,
weather, wake word, summaries, webhooks, command chaining.
Only a live verdict sets pass. Every verified cell cites
docs/evals/2026-08-26-capability-baseline.md by path and section, and the
generator refuses to build if either does not resolve.
Both files are generated. Rebuild rather than hand-edit.
--no-verify: 3,386 non-markdown lines, all of it generated output that cannot
split into reviewable ideas.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
probes_field.json is 25 multi-turn probes drawn from the owner's real week.
run_probes.py drives them through the deployed stack: POST /api/chat on
127.0.0.1:9201, which runs a real turn through the pre-route ladder, the stage 0
grammars, the routing heads, the resident model, the query walk, the act path
and the phraser. Readback is cmd/e2eprobe over the mavend IPC socket, never the
plaintext sqlite copy in /dev/shm and never the mavweb HTML pages.
store_counts.py reads row counts per store over IPC, before and after.
out/ holds what the run produced. field.contaminated.jsonl is the discarded
first run: mavweb hardcodes one conversation id for the whole web reach, so a
clarify parked by one probe was still parked for the next.
field.transcript.tsv is the evidence for the baseline and is not summarised
anywhere else. The store it came from was wiped afterwards.
--no-verify: 326 non-markdown lines of new harness plus its captured output.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
build_ledger.py extracts 51 capabilities and 156 DoD criteria from docs/spec.md
and joins them with domains.yaml and verdicts.json. The generator is also the
checker: it exits non-zero on a capability with no DoD criteria, no State line,
no domain or more than two, an unknown domain, a criterion id collision, a
domains.yaml or verdicts.json row naming something that does not exist, a
verdict word outside the five, and a reason outside the plan's list. It caught
the domain reconciler silently dropping recall from its 51.
It also refuses an evidence path that does not resolve, a section heading absent
from the file it names, a pass whose reason is not passes, and a fail resting on
no runtime proof. Sixteen verdicts had cited a section of the eval that did not
exist.
domains.yaml is the one judgment call in the extraction and is hand-edited.
--no-verify: 584 non-markdown lines, all of them new files. The generator and
the domain table it reads are one reviewable idea and splitting them leaves
neither readable alone.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three sessions on one causal order: the spec says what should happen, the
empirical run says what actually happens, code and architecture explain why,
priority says what to fix. The predecessor audit had a green suite while 22 of
39 capabilities were not live, which is the failure mode this order exists to
stop.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Owner's call. M1 is the turn path, M2 is memory he cannot correct, M3 is
step-up. They were one milestone and are three jobs: M1 and M2 touch
different code and owe different docs, and step-up is configuration, not
honesty. Nine milestones now.
Speaker recognition, smart home, bluetooth, model swap and self-update
move past v1. Bluetooth was on the v1 list and comes off it: no bluez on
the box. Their spec entries keep their DoD.
--no-verify: committing on master by the owner's call this session.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Ordered by what makes her untrustworthy if it ships late, not by code
work: the audit ruled that out, since none of the four broken
capabilities is a code defect. Honesty, then the config-and-data four,
then voice, then proactive delivery, then breadth, then email and
calendar behind their product decision, then the undesigned seven.
Doc gaps and missing scenarios bind every milestone rather than forming
one, so they cannot collect at the end.
--no-verify: committing on master by the owner's call this session.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
51 capabilities: the 39 rows from the 2026-08-13 audit plus 12 v1 items
that had no audit row. Each entry carries a state reference to the living
doc that owns it, a plain DoD list observable on the running box, and the
scenario file that scopes it.
Applying "state is a reference" found 17 capabilities with no living doc.
Only 5 of 51 entries cite a scenario that exists.
--no-verify: committing on master by the owner's call this session.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
39 capabilities read off the running five-container stack on 2026-08-13,
not off the code. 17 live, 9 partial, 4 broken, 9 off. The owner's
corrections are applied: speech in, speech out and wake word are live, and
he proved all three by speaking to her. The voice reach stays broken,
because reaching her by speaking is a pull and a proactive message needs a
session to push into.
The claim the spec has to be written against: none of the four broken
capabilities is a code defect. Weather has no config block, Nexus has no
data, the voice reach has no listener, step-up has no WebAuthn credential.
The race suite was green during a run where 22 of 39 capabilities were not
live, so no definition of done that a test suite can score is worth
writing. Every criterion has to be observable on the box.
The task id is unfiled: Vikunja answered 503 for the whole session.
--no-verify: master is the working branch this session by the owner's call.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
claude