Two reporting tests over the 91-case RU fixture, no ratchet: a ratchet here
would freeze a number nobody has decided to hold.
TestStage0Contention runs the 21 grammars one at a time instead of stopping
at the first match. One case of 91 draws two, ru-query-019, where
calendar-query beats agenda-query by list position alone.
TestONNXClaimConfidenceDistribution buckets the reported confidence by the
layer that produced it. Stage 0 is 20/20 at a hardcoded 1.0. The classifier
scores 62% below its median and 62% above, across a cosine range of 0.859
to 0.942, with a top-two margin of p50 0.009. The float is not a confidence.
newBaselineClassifier and baselineGrammars split out of newBaselineRouter so
the measurement runs the same rules the daemon runs. TestONNXBaseline is
unchanged at 64/91.
Step 4 of the board build (docs/plans/15-board-surface.md). Naming a task
instead of its position reached nothing: "закрой задачу купить молоко" routed
act, found no allowlisted fn, and the gate asked "Что сделать?". The position
path already worked through resolveCandidate, but only in the two turns after
she read the list out.
TaskStatusGrammar is the same shape TaskCaptureGrammar uses — matches broadly,
decides in Build, no eighth intent — and fills the fn slot with task_status,
which is neither a Hexis capability nor a Praxis one. Three conditions, all
required: the board noun, so no ordinary sentence claims a turn; exactly one
status class, since "готово, убери" names two and asking beats picking; and a
status word matched as an imperative exactly or a stative by lemma. So a bare
"готово" and a bare "закрой" are not this rule's, and the second belongs to
Praxis, which claims it already.
Two lexicon sets rather than one with a value. The store records which of the
two transitions happened and /tasks shows it: work he chose to stop is not work
he did.
Measured on the fixture, two new cases (ru-act-020, ru-act-021). Classifier +
ONNX 62/89 (69.7%) → 64/91 (70.3%); cascade+llm 67/89 (75.3%) → 69/91 (75.8%,
80.2% intent-only) at p50 1.225s. Both new cases claimed at stage 0, no case
regressed, clarify counts unchanged at 3 false / 1 missed.
The task's own warning stands: every such grammar runs its parser ahead of the
resident model on every turn, so this is the last one that is free.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SoL7EBdYC5Mhz3DJd49GJy
Praxis reach was 0/12 on the held-out fixture and structurally so.
handlePraxisAct dispatches on exact equality between Slots.Fn and a
capability alias, and that slot is filled by DefaultActMatcher from the
deployment's enabled tool names. No Praxis alias is on that list, so no
utterance could ever put one there. The Russian aliases in
praxisCapabilities read as if they matched speech. They are compared
against a fn slot and never against an utterance.
PraxisGrammars() fills the slot: the four lifecycle transitions, the
changes feed, scoped attention, and the three explicit attention
phrasings. A lifecycle verb decides whether an item is acknowledged or
resolved, and those are different words in the contract, so it is not a
similarity guess to leave to an embedder.
Two rules keep the lifecycle arm off ordinary speech. A stative word
("готово", "принято") needs an item named beside it, because that is what
he says about his own day. Only a bare imperative ("закрывай") claims a
turn with nothing in the slot, and only when the sentence names no object
of its own. Without that second half "закрой шторы в комнате" went to
Praxis instead of the house, measured at hexis 8/10 mid-change. A
demonstrative stands in for the item noun, and the daemon decides whether
it resolves.
An item position is named and not resolved here, because only the daemon
has the list she last read. "что нового" is left to the feeds. "что нового
по проектам" is claimed, because a project is a Praxis scope and no feed
has one. "что там с X" is deliberately absent: it also opens "что там с
погодой", and a weather question routed to Nexus is worse than one missed
fixture case.
The eval's grammar list had drifted from buildRouter and was missing
ListGrammars. Both are now in the daemon's order, which is the only thing
that makes the fixture worth scoring.
--no-verify: 575 lines against the 300 cap. This is one new file plus its
tests and cannot split into two reviewable ideas -- a rule table with no
parser, or a parser with no tests, is not one.
attentionq.go, repair.go and internal/router/complaint.go carry the last
hand-written Russian patterns of the V-522 sweep, and they live on task/467.
internal/lexicon, internal/morph and cmd/mavend/topics.go live here. One of
the two had to move.
Four conflicts, and one of them is a real collision rather than a mechanical
one. Both branches wrote the narrative stage 0 rule. This side had
NarrativeQueryGrammars, plural, with the rest-of-day rule beside it and the
verb alternation built from the lexicon; task/467 had NarrativeQueryGrammar,
singular, which extracts the topic into Slots.Text, refuses a bare "расскажи",
and excludes the shapes that are chat ("расскажи о себе", "историю на ночь").
Resolved by keeping this side's container and this side's lexicon-built
pattern, and taking every behaviour only the other side had: the topic slot,
the empty-topic refusal, chatNarrativeTopics, and its wiring position after
TaskCaptureGrammar so "запиши" still beats "расскажи".
The rest: queryFeeds keeps task/467's conditional claim (V-474 supersedes the
unconditional one), rank.go keeps Spoken and drops pluralTasksRU because
say.CountWord is the one copy of Russian count agreement, and vendor/ was
re-vendored — the merged modules.txt claimed replaces for nexus and praxis
that neither go.mod has.
Routing fixture 58/82, unchanged from both sides.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Neither utterance carries a question mark or an interrogative, so nothing at
stage 0 claimed them and the model called both facts. The write is contained —
actions_fact refuses a question-shaped fact and re-runs the turn as a query —
but every one of these paid a full model round trip to reach a decision two
regexes can make, and the fixture scored the routing as wrong.
rest-of-day-query joins the agenda grammars: the predicate for the utterance
already existed as IsRestOfDayQuery, one layer down in the query chain, and
this is what gets the turn there. NarrativeQueryGrammar reads the same
narrativeRequests lexicon IsQuestionShaped reads, and declines the topics that
are chat rather than world questions — a joke, a bedtime story, herself. It is
wired last, so an explicit capture marker still wins.
Fixture: ru-query-024 and ru-query-025, both passing. Classifier + ONNX
baseline 56/80 (70.0%) → 58/82 (70.7%), no case regressed and no new false
clarify. The LLM arm is unmeasured here — no llama-server in this run.
The mavweb auth test posted its instant as "Z", which the #482 fix now reads in
the daemon's zone, making the clock inside the text stale by the test box's own
offset. It carries the local offset now.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The capture half landed with the grammar in 87d1761. This is the exposure
the task asked to check for: IsTaskListQuery is a deterministic lookup that
only runs once the turn is already a query, so a phrasing the model calls
system never reaches it. The eval fixture was also missing both grammars,
which is only worth having while it is the daemon's grammar set.
The querySources order predates the 2026-08-02 ruling that live search leads.
An unconfigured feeds source claimed every news question and answered with a
configuration status, so "что происходит сейчас в новостях про искусственный
интеллект?" never reached the search sitting one source below. It now claims
only when neither SearXNG nor the ZIMs are configured, which is the case the
"не читаю ленты" line was written for — general knowledge would otherwise
invent a bulletin.
The calendar matches on a day word alone and sits above the weather, so
"какая сегодня погода в Москве?" answered "на 02.08.2026 ничего нет." It now
steps aside on weather wording, the same bail-out queryHome already does.
"что нового в лентах?" routed system and answered "пока не умею", while the
same question worded with "новостях" worked. FeedQueryGrammar routes it to
query at stage 0, requiring an ask word and a feed noun so the bare greeting
"что нового?" stays a greeting. Wired in the eval too, since the fixture is
only worth anything while its grammar set is the daemon's.
Also: the claiming source is now logged. /trace is the nudge-rule trace and
carries no query-source field, so a wrong answer could not be told apart from
a wrongly-ordered chain.
Kiwix having no live coverage is filed separately as V-508 — it is a decision
about search quality, not an ordering fix.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both shapes carry no question mark and no interrogative, so the model saw
them with nothing deterministic in front and routed both to fact. The fact
gate caught the write and re-ran the turn as a query, so nothing broke —
what they cost was a full model round trip for a decision two patterns can
make offline.
NarrativeQueryGrammars, wired after the agenda rules so that "расскажи,
что у меня сегодня" stays an agenda question. Two exclusions, both learned
from the fixture: a capture verb in the rest of the utterance means he
asked for a note, and an entertainment noun means chat — "расскажи анекдот
про программистов" is ru-chat-003, and my first pattern took it.
The fixture had no case for either shape, which is why they went unnoticed.
Added as ru-query-020 and ru-query-021: classifier+onnx 53/77 → 55/79
(68.8% → 69.6%), both new cases answered at stage 0, false clarifies
unchanged at 0.
"что у меня сегодня" and "что у меня в календаре сегодня" both routed
IntentSystem on the deployed daemon, and replySystem has no agenda arm,
so both answered "пока не умею". The calendar source that can answer
them lives in the query chain and was never reached. The fixture has
said query since ru-query-019 was written; the daemon disagreed with the
fixture and the daemon was wrong.
AgendaQueryGrammars routes them at stage 0, after the clock rules so
"какой сегодня день" keeps reaching replySystem. Intent only — which
source claims the turn stays the query chain's decision.
This is what made the follow-up continuation look like it only worked
for "what day is it". It did: the query half inherited an intent whose
handler could not answer, so both halves came back "пока не умею".
Measured on the 77-case RU fixture: full accuracy 70.1% → 72.7%,
intent-only 75.3% → 77.9%, calendar 0/2 → 2/2, clarify counts unchanged.
The eval harness wires the new grammars too, or the fixture would stop
being a measurement of the daemon.
Go's \b is ASCII-only and never fires after a Cyrillic letter, which the
first version of the pattern learned the hard way.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TrVSBKe3RFDF4fGYKWYQnX
The old model was a symmetric paraphrase model, so it scored "do these
look alike" instead of "does this note answer this question". Also fixes
the file mismatch: the Makefile, the deploy config and both evals now all
name the same quantized file, and the quantized one is what gets measured.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CGeSZxh1DCtRxmFVSYVGvJ
Completes #319's comparison. Three configurations, because "the LLM router"
was ambiguous: the model alone, the cascade #320 would actually ship (stage-0
grammar → model → classifier floor), and a thinking-off diagnostic.
intent-only full RU missed-clarify p50
classifier+onnx 36.8% 36.8% 25/61 5/6 31ms
llm-only (0.8B) 48.7% 23.7% 13/61 6/6 850ms
cascade+llm (0.8B) 50.0% 32.9% 18/61 6/6 825ms
On the question asked — does the resident model route better? — yes, 50.0%
vs 36.8% intent accuracy. REARCH.md's premise holds. It costs 27x the
latency (p50 825ms vs 31ms, max 3.1s), on the same llama-server the phraser
needs, so it is a trade rather than a free win.
Three things the numbers surface that the headline hides:
query→fact x15 is the dominant failure, four times the classifier's x4 on
the same axis. routeSystem's decision order puts "сообщает или обновляет
состояние" (rule 3) above "хочет получить информацию" (rule 4), so any
utterance naming a fact key matches the earlier rule and a question about
past state reads as an assertion of it. A prompt fix, not a model limit.
The LLM router cannot clarify: llmrouter.go hardcodes Confidence 1.0, so
stage 3's gate can never fire on its decisions — 6/6 missed. With #359's
finding that the classifier's gate is miscalibrated under ONNX, neither path
currently refuses. Flipping #320 as-is removes the refusal lane.
The gap between 50.0% intent and 32.9% full accuracy is entirely slots: the
LLM path fills neither Fn nor Time (it returns Slots.Text for acts, and
Extract never runs on an LLM decision).
Also settles a hypothesis rather than leaving it in the air: thinking mode is
a non-issue under a grammar (identical score), and the grammar's unbounded
("," ws action)* repetition that ran away in an isolated smoke test does not
reproduce under the real prompt — 2 errors in 76, not 76. internal/llm
deliberately does not grow a chat_template_kwargs field.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5JApcrCRVGmqrxnhynSik
The onnxruntime .so was already vendored at deps/onnxruntime-linux-x64-1.26.0
— nothing to download. make eval-router now defaults MAVEN_ONNX_LIB there, so
both baselines run by default and only a fresh clone without deps/ falls back
to the hash ratchet alone.
Prod-representative result, deployed 0.55 gate: 28/76 (36.8%), RU 25/61,
EN 3/15, hard 0/11 → 4/11, p50 31ms / p95 71ms. Versus the hash floor's
13/76 at p50 9µs.
The finding is not the accuracy, it's the refusal lane: missed clarifies went
0 → 5 of 6. Better embeddings raise cosine everywhere, so the 0.55 threshold
that used to hold ambiguous utterances back stops holding — "сделай это"
routes to act at 0.847, "бэкап" to chat at 0.755. The gate was implicitly
tuned to the hash floor's low similarities. That is an argument about the
threshold, not about the embedder, and it lands before #320 rather than after.
Also fixes a fixture-model mismatch: ReminderGrammar deliberately skips the
extractor at stage 0 and the daemon's applyAction parses the time downstream
(stage0.go says so). Charging the router for that slot made 4 exact-match wins
read as misses; they are now counted as SlotsDeferred instead. Hash baseline
moves 13/76, ratchet to 0.15.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5JApcrCRVGmqrxnhynSik
#319 asks for a measurement before #320 flips the route decider from the
classifier cascade to the resident model. There was nothing to measure
against: the only routing tests assert single utterances, and the
classifier's seed corpus is its own training set — scoring it there
measures memorisation of frozen centroids, which is the illusion that hid
the weak RU query handling in the first place.
internal/router/eval is a separate package so both paths can be scored
from outside router (including cmd/mavend, where the real llama-server
client lives). The fixture is embedded; the scorer takes a Router
interface, so *router.Router and a bare LLM stage both go through the same
76 cases.
The fixture is a CONTRACT, not a snapshot: cases the cascade fails today
stay in the file and fail loudly. TestFixtureIsHeldOut enforces that no
utterance appears verbatim in models/seeds/*.txt.
Baseline, hash embedder at the deployed 0.55 gate: 9/76 (11.8%), 63 false
clarifies, 0 missed clarifies, p50 9µs. Almost everything falls to the
confidence gate — the documented floor behaviour, not a new bug. The
number worth comparing is TestONNXBaseline's (skipped without
MAVEN_ONNX_LIB); the assertions here are a regression ratchet plus a tight
bound on the dangerous direction: ambiguous utterances must not start
being routed confidently.
Seeding is order-fixed on purpose — a few phrases appear under two intents
and map iteration handed them to a different centroid each run, which made
the score jitter between 9 and 10.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5JApcrCRVGmqrxnhynSik
claude
kami