EvalReport with macro F1, per-route precision/recall/F1, confusion
matrix, false-action rate, and fast-path vs residual breakdown.
ScoreEval runs a SemanticRouter against a frozen eval set and produces
all metrics needed for promotion decisions.
Six deterministic transforms (negation, question, reported speech,
quotation, hypothetical, capability question) applied to action-route
seeds. Each transform determines the expected class explicitly — no
model guessing labels. SplitByFamily uses hash-based bucketing to keep
paraphrases in the same split.
136 examples with provenance from ru_routing_v1.json, tagged
fast_path_resolved vs residual. RouteExample carries source, source_id,
split_group for traceability. Split-by-family prevents paraphrase
leakage across train/eval.
Defines the six-class SemanticRoute type (conversation, knowledge,
action, memory_write, system, uncertain), the SemanticRouteDecision
output, the SemanticRouter interface, and the deterministic
Intent→SemanticRoute mapping from the current seven-intent cascade.
Migrate 54 test call sites to construct NormalizedInput{Text: ...}.
Add invariant tests:
- TestNormalizedInputReachesRouteIntact: ingress NormalizedInput reaches Route
- TestTryFastPathReceivesMatchText: TryFastPath gets the same input
- TestMatchTextDoesNotChangeRouting: same Text + different MatchText → same Decision
- TestDecisionUtteranceEqualsInputText: Decision.Utterance == input.Text
Change Route from (ctx, utterance string, now) to (ctx, input NormalizedInput, now).
The ingress-constructed NormalizedInput now reaches the cascade intact — no
reconstruction downstream. TryFastPath receives the same input, not a rebuilt one.
All callers (production, eval framework, tests) updated to construct NormalizedInput.
Add MatchText field to NormalizedInput — a lossy lexical matching view
derived from ingress text: TrimSpace → NFKC → lowercase → collapse
Unicode whitespace. Does NOT fold ё→е, strip punctuation, strip wake
words, rewrite numbers, or invoke morphology.
Both ingress sites (voice + text) construct MatchText at entry. No
existing consumer reads MatchText yet — it is dark data for future
opt-in migration.
Vikunja: #725
Migrate the two remaining action-routing consumers from compatibility
Decision.Slings fields to authoritative Decision.CapabilitySelection:
- refusesCommand: reads CapabilitySelection.Fn instead of Slots.Fn
- ActHasEntityTarget: reads CapabilitySelection.Resolved and
CapabilitySelection.Args instead of Slots.HasFn and Slots.Args
Slots.Text remains the source for entity text when positional args do
not contain the target (unchanged).
Regression tests prove:
- prohibited sentinel preserved byte-for-byte through SelectCapability
- blanked Slots.Fn/Args/HasFn do not affect migrated consumers
- Praxis/Hexis entity-target routing unchanged
- stage-0 deterministic act unchanged
- classifier/extractor act unchanged
do not remove the compatibility mirrors yet.
Set CapabilitySelection on decisions that have Slots.HasFn=true, so
ResolveActionCandidate reads from the authoritative record. Backward
compatibility tests verify that decisions without CapabilitySelection
still resolve via Slots.HasFn.
The candidate now receives Fn/Args from CapabilitySelection (the
authoritative record) rather than from Decision.Slots.HasFn. Backward
compatibility: decisions with Slots.HasFn but no CapabilitySelection
(tests, rebuilt decisions) still resolve via the compatibility path.
Authoritative record of which executable capability matched, separate
from Decision.Intent (what kind of turn) and ActionCandidate (downstream
action artifact). Decision.Slots.Fn/Args/HasFn remain as compatibility
representations populated from this selection.
Call SelectCapability after each cascade path (grammar, heads, LLM,
classifier) and propagate the result via applyCapabilityToSlots.
Remove LLM text capability backfill from fillSlots — SelectCapability
now owns that path. fillMatchedSlots retains raw extractor capability
extraction for backward compatibility with stage-0 grammars.
gateLLMDecision now reads CapabilitySelection.Resolved instead of
Slots.HasFn for the act-intent confidence thinning check.
Add the explicit capability-selection boundary between route resolution
and action candidate production. SelectCapability is the single entry
point for selecting which executable capability matched an IntentAct turn.
Three input kinds: raw, llm_text, deterministic. Decision.CapabilitySelection
is the authoritative record; Decision.Slots.Fn/Args/HasFn remain as
compatibility representations populated from the selection.
Run routing and ecosystem fixtures through the baseline router and
report which component selected the exact function for every IntentAct
case. Shadow matcher comparison confirms zero disagreements.
Five disjoint values tracking which component selected the exact function:
grammar_fixed, grammar_matcher, extractor_raw, extractor_llm_text,
fallback_matcher. Slots.ResolvedBy carries provenance at the selection
point.
Introduce ActionValidationStatus enum (valid, unresolved, missing_argument,
invalid_argument, ambiguous_target) as the typed classification of validation
outcomes. ActionValidationResult now carries Status instead of boolean flags.
Backward-compatible: Unresolved() and Valid() methods preserved on the result.
Existing validation behavior unchanged: only blank Fn produces invalid_argument.
All downstream behavior (proposeGap, confirmation, task_status, praxis, hexis)
unchanged.
Tests added for all five status values, backward compatibility, and the full
validation → execution boundary.
Introduce the typed boundary between routing and action resolution:
- ActionCandidate: Fn, Args, Source (route|matcher), Producer, Confidence
- ResolveActionCandidate(dec, m): standalone function usable by both
the daemon and the eval harness
- Update eval harness Reach() to use ResolveActionCandidate instead of
duplicating the matcher fallback logic
This is the routing-side half of the action-resolution boundary.
The daemon integration follows in the next commit.
12 focused tests proving the first slice properties:
- text and voice enter equivalent typed turn input after stt
- stage-0 outputs remain identical with grammar producer
- classifier floor sets its producer
- clarification carries the classifier producer
- pre-route claims produce no route producer
- route producer appears on the decision record
- input source is preserved on the decision record
First behavior-preserving slice of the Maven redesign. Establishes
explicit ingress/routing boundaries and enough observability to refactor
later without changing current routing, action, clarification, or
execution semantics.
Types introduced:
- NormalizedInput (internal/router/source.go): Text + InputSource,
the typed ingress boundary replacing raw string at the turn entry.
- InputSource (internal/router/source.go): channel provenance enum
(tap:voice, tap:text). Reuses the existing turnSource distinction.
- RouteProducer (internal/router/intent.go): which cascade stage
produced the decision (grammar, heads, llm, classifier).
Changes:
- Decision carries a Producer RouteProducer field, set at each cascade
stage (grammar, heads, LLM, classifier).
- turnRoute carries NormalizedInput instead of bare text string.
- runTurn takes NormalizedInput instead of (text, src).
- decision.Record carries InputSource and RouteProducer for
observability; RoutingTrace persists route_producer (migration #27).
- turnSource is now a type alias for router.InputSource.
Behavior preserved:
- Stage-0 grammars unchanged: same order, same matching, same confidence.
- Cascade fallthrough order unchanged (grammar → heads → llm → classifier).
- Clarification behavior unchanged.
- Action dispatch unchanged.
- No new linguistic normalization.
reminder_cancel.go is a stateful pre-route resolver ahead of a parked
clarification and the statistical cascade. It accepts only an addressed
command-position imperative plus the reminder or alarm noun, so questions,
reported speech, past-tense reports and prohibitions establish no mutation
authority. Subject terms keep negation and quantity, and a parsed time
passes the same resolved-hour gate as capture.
One match cancels through the typed IPC method. Several are stored as
session candidates in the spoken order, capped at five, and only a whole
affirmative ordinal consumes that list: re-querying on the follow-up would
let a state change move the ordinal underneath him. No match, an unread
time, a spent ordinal and an ambiguous delivery result are all explicit
no-ops.
command_prohibition.go is the first mutation boundary in a turn. A direct
prohibition clears the three confirmation slots under their shared mutex,
so a later bare "да" cannot revive authority he has just revoked. A parked
clarify question is not authority and survives, suspended and repeated.
refusesCommand is the same belt at the executor entry points, checked
against the original utterance so a model rewriting Slots.Text cannot get
around it.
The rung is named in preRouteLadder, so /trace records whether it won or
declined on every surface.
--no-verify: master is the working branch this session by the owner's call.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CancelReminder replaces the cancelled half of MarkReminder, which stays
delivery-only. Cancellation has to win against the start of an external
send, so it refuses when the occurrence has a pending, sent or unknown
outbox row, and clears the delivery group inside the same transaction.
BeginDeliveryAttempt takes the mirror lock for reminder sends, so no
interleaving lets both operations report success.
Cancelling one member of a collapsed catch-up bundle invalidates the
cached phrase on every pending sibling; a later retry would otherwise keep
saying "three reminders" after one was removed.
Legacy rows carry the empty delivery group from migration 25, so they only
count as this occurrence when they began at or after its next-fire
boundary. Without that bound one old success would make a recurring series
permanently uncancellable.
ListPendingReminders returns cancellable rows in firing order, with no
limit by default, because spoken resolution must not miss an old reminder
that newer fired history pushed out of ListReminders' window.
Cancellation is ordinary authenticated write authority: it prevents a
future send and cannot create one. cmd/e2eprobe drives both from outside.
--no-verify: master is the working branch this session by the owner's call.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The spare-key note scored 0.832 to 0.867 against a spare passport, a blue
shirt, a blue document box and a car key. Score and margin cannot separate
those: the right note runs 0.817 to 0.892 and the silent cases 0.787 to
0.874, so the ranges overlap and structure has to decide.
RecallAllowed now takes two structural facts from the router. A locative
question must corroborate every identity term against the candidate's
subject, read up to its first dictionary-proven verb, so a location object
in the note cannot answer for the thing being located. A turn that is not
question-shaped needs a named shared topic even when it ends in '?', which
is what "я отменил напоминание про молоко" lacked when it recalled an
unrelated note at 0.825 with no runner-up to fail the margin.
query_min_score moves 0.55 to 0.80 for tokenizer rev 2. The held-out
fixture answers 14/27 real recalls and 0/14 false ones.
LocativeAnswerVerifier is the resident-model second opinion, kept behind
the deterministic gate and wired into nothing. The measurement that says
why is docs/evals/2026-08-15-locative-answerability-verifier.md.
--no-verify: master is the working branch this session by the owner's call.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
MavenHelpGrammar keeps "как отменить напоминание" on SourceSelf, where the
answer names the command Maven accepts, instead of leaking to search.
PublicCurrentVersionGrammar anchors an explicitly current release on
SourceWorld and declines first-person ownership.
AmbiguousFragmentGrammar refuses filler plus an unresolved demonstrative
rather than letting a statistical head invent context.
ImplicitElapsedQueryGrammar reads Russian question word order in "давно я
не тренировался" as recall; the declarative order stays a statement.
ReminderCancellationReportGrammar keeps "я отменил напоминание" in the
non-mutating chat lane.
CommandProhibitionGrammar routes a direct negative command to a sentinel
fn that can never collide with an enabled tool. ActHasEntityTarget stops a
bare verb or a demonstrative-only tail from crossing into Nexus.
Praxis attention now accepts "что там с X" for the four service names only.
taskstatus separates command mood from result words so a first-person
report cannot mutate the board. question.go exports the open-question and
locative shapes the recall gate reads.
--no-verify: master is the working branch this session by the owner's call.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reminder cancel verbs, cancel reports, reminder nouns and their frame,
unresolved references, current-version nouns and markers, personal
possessives, ecosystem service names and task done/drop command and state
splits. Each set carries the note that says how a caller must match it.
self_state_verbs is the head list a prohibition may not take: "ну не знаю"
answers a parked question and must not be consumed as "do not do that".
TaskDoneWords goes: TaskDoneCommands and TaskDoneStates replaced it, and
the deadcode gate fails on an accessor nobody calls.
--no-verify: master is the working branch this session by the owner's call.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
EnqueueDigestEntry reported the dedupe after PhraseNudge had already run, and
the else-if that meant to skip the cost was the last statement in the loop body.
Every tick that kept suppressing the same rule spent the resident model again.
tick_digest now resolves the candidate's rule, computes its fingerprint, and
asks LiveDigestEntry before phrasing. Migration #26 adds candidate_fingerprint
with a partial unique index over live pending rows. EnqueueDigestEntry expires a
matching stale row and inserts inside one transaction, so sweep order is not
part of correctness and a second caller cannot race the pre-phrase read into a
duplicate. Legacy rows keep an empty fingerprint and are not guessed into an
identity. Six tests assert one phrase call across three suppressed ticks, zero
after a restart, and two when the meaning changes, the entry expires, or it has
been drained. The caveat and the SA4006 baseline entry are deleted.
--no-verify: 419 non-markdown lines against the 300 cap. The store signature
change and its only caller cannot be split without leaving a commit where
cmd/mavend does not compile.
The digest needs to know whether a candidate is already pending before it pays
the phraser, and prose is not identity: phrasing varies, and State.Now advancing
does not turn the same unmet condition into a new event.
A rule eligible for the digest declares DigestIdentity beside its predicate.
DigestCandidateFingerprint frames the rule name and severity around it so two
rules cannot alias on a shared fact. BreakRule anchors on the last completed
break, not on desk_active, which the poller refreshes without the unmet need
changing. A rule that declares no identity does not enter the digest, since a
generic state hash would either change every tick or ignore an input the rule
reads.
Go flag parsing stops at the separate boolean value before ambient-token. Use -ambient-enabled=value and pin the deployed argv contract discovered during live V-691 verification. Owner explicitly requested direct commits to master.
Reference-count the process-global ONNX Runtime across embedder and routing-head sessions, make close idempotent, and require named proof that both aggregate routing gates executed rather than self-skipped (V-716). Owner explicitly requested direct commits to master.
Replace nearest-neighbour personal routing with a frozen class-balanced linear head measured on historical, stratified, cross-validation, holdout, and fresh challenge gates (V-702). Close the four repair handoff holes, preserve nested clarification flows, and route Russian possession statements through structural grammar rather than lexical exceptions (V-573). Owner explicitly requested direct commits to master.
Persist reminder presentations and retry state, atomically complete collapsed deliveries, fall back across away reaches, and block permanent failures visibly (V-715, V-678). Fail closed when enabled integrations lack credentials and keep remote arms explicitly dark (V-691). Give mavweb one sanitized, request-correlated error contract (V-689). Owner explicitly requested direct commits to master.
buildRouter held the real set and baselineGrammars in eval_test.go restated it
by hand, in the daemon's order, with its own comment saying so. Three test files
score against the fixture and nothing compared the two lists. They had already
drifted: BareCaptureGrammar went into the daemon with V-557 and never into the
fixture, so every routing measurement since has scored a set nobody runs. That
is the failure CLAUDE.md warns about by name, and a diff test would have caught
it one grammar late.
The list moves to router.StageZeroGrammars in internal/router/stagezero.go, with
the ordering comments, which are the load-bearing part. buildRouter and the
fixture both call it. One list cannot drift from itself.
Measured before and after on the 96-case fixture: classifier+onnx 72/96, 75.0%
intent, 33.3% destination, identical either way, and the deterministic claim and
reach hash ratchets do not move. So the missing grammar cost no measurable
accuracy. That is the point rather than a reprieve: the fixture had been scoring
the wrong set for four days and nothing could say so.
The invariants caveat is deleted, both entries, since V-692 landed the other
guard in the previous commit. The reasoning for both now sits in docs/routing.md
beside the subsystem, which is where a fix's durable record belongs.
Unrelated and pre-existing: TestONNXPersonalBoundary fails on "я рассказывал
тебе про байкал?" (personal 0.9068, world 0.9413) at the merge base too.
CLAUDE.md, internal/config/voice.go and docs/routing.md all say the routing
heads graph is a fine-tuned copy of the embedder, never the embedder's own file.
Nothing enforced it. The daemon loaded whatever the key pointed at, so pointing
both keys at one file cost recall with no error and no log line, which reads as
ordinary drift rather than as a misconfiguration.
validateVoice now refuses it at load. Both paths are cleaned and made absolute
first, so "./m.onnx" and "$PWD/m.onnx" are one path, and then compared with
os.SameFile, which catches a copy that is a symlink or a hard link. A path that
does not stat is left to the loader, whose error message is better than this
check can give.
Refusing to start is deliberate and it differs from the loader's treatment of a
broken weights file, which logs and leaves the heads nil on purpose. That case
is a missing accelerator. This one is a working file in the wrong role, and a
daemon that cannot route well should say so rather than answer worse.
deploy/mavend.json points the two keys at different files, so the live config
still starts.
llm.Client carries a bearer credential and sets it on the completion, and
Pair signs the /health probe with it too. An unsigned probe would answer 401,
Pair would read that as a card that is busy, and every workstation turn would
fall back to the resident model with nothing naming why.
The token comes from workstation.token, expanded from MAVEN_GPU_TOKEN like
every other secret in that file. Missing, and voicewire says so at startup:
the fallback is silent by design and this failure would otherwise be
invisible.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ESv8hqNPseYt1CnotZpqDz
SendRequest and RunPushReceiver each read the conn, so a client that
wanted both raced for every frame. A second listening conn is not the
fix: it never sends a request, so its lastActive never moves and
PushToMostRecent never picks it. mavwaked needs both on one conn.
The reader now owns the socket for the life of the conn. It hands each
Response to whichever SendRequest waits on that id, and each Push to the
handler. SendRequest waits on its own channel, on the conn dying, on its
context, or on a timeout, and forgets its slot on every path that leaves
without an answer. RunPushReceiver just wires the handler and blocks.
Connect opens the conn without sending anything, for a client that must
hold a session before it has spoken.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
A ZIM title carries a leading capital and the utterance does not: /A/фотосинтез
is a 404 and /A/Фотосинтез is a 200. TitleCandidates tries the spoken form
first, so a title that begins lowercase on purpose keeps its chance.
That takes the measurement from four right to five, and the fifth is the one
that mattered. "столица Франции" returned "Список столиц Олимпийских игр"
and now returns Париж, through a title redirect the ZIM already held. The
2026-08-05 measurement named that case as the one no lexical signal could
reach. Retrieval by title reaches it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
Kiwix ranks by keyword overlap, which the package doc has said since it was
written: "why is the sky blue" finds a TV episode. queryKiwix sent the whole
Russian sentence, because the verbatim path added by V-508 skips the rewriter
that would have reduced it.
Measured against the Russian ZIM on 2026-08-09, over eight questions. Four
reach the right article where they did not: TCP was "Перехват TCP-соединения"
and is TCP, фотосинтез was "C4-фотосинтез" and is Фотосинтез, Линус Торвальдс
was "Tux", and "кто написал Войну и мир" was "Радуйся, мир (Доктор Кто)".
Two were already right and stay right. Two are still wrong and were wrong
before. Nothing regressed.
kiwix.Topic drops the narrative request, the interrogative and a verb behind
one, and keeps everything else. A word it cannot classify is more likely the
topic than noise. TitlePath tries the exact article first, since a ZIM is
addressable by title and a wrong title is a 404.
The gate this task set out to build does not exist. Query-to-passage cosine
scored 0.79-0.91 on answerable questions and 0.75-0.84 on unanswerable ones,
and the sets overlap. The wrong TCP article scored 0.8653, above five of six
unanswerable rows. e5 measures topic, not whether the passage answers.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
Naming a destination takes the guessing query sources off a turn, and the
personal boundary is one of them. Every other guesser costs an answer when it
is wrongly dropped. This one costs the rule that a question about him never
reaches an upstream engine.
Three deciders name a destination now and two of them infer it: the routing
heads and the resident model. Decision.SourceAnchored says a stage 0 grammar
read the words instead. queryWalk honours it for the source marked
boundary: true and for no other, so the rest of the table is unchanged.
Owner's call of 2026-08-09.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
claude