Maven/docs/determinism.md

# Deterministic logic around a small model

*Last verified: 2026-08-02 @ 7079a24. Living doc: correct it in place, do not append.*

Written 2026-08-02. Branch `fix/integrated`.

## The question

Where does deterministic code attach, so that it helps the resident 1.7B now and
does not fight a larger model later.

## The mistake to avoid

Everything deterministic we have added so far sits in front of the model and
preempts it. Stage 0 matches, the model never sees the turn. That shape helps a
weak model and blocks a strong one, silently.

The fix is not to remove it. The fix is to know which rules are safe in that
position and to have a way to measure the rest.

## Four attachment points

**Bypass, before the model.** The only shape that saves the 2.7s p50. Safe when
the rule is a decision procedure over a closed set, not a guess over an open one.
Exact match and clock queries qualify.

**Evidence, beside the model.** Extractors emit candidate slots as a prior, not a
verdict. The prompt carries the prior and the validator reuses it. A small model
leans on it, a large one overrides it correctly.

**Grammar, around the model.** GBNF built from live state rather than hardcoded.
Costs nothing at runtime and prevents the error instead of catching it.

**Repair, after the model.** Validation failure re-asks with the specific error
rather than overriding. Self-retiring, because a better model trips it less.

## The constraint that ranks them

Latency must stay minimal. That demotes repair and promotes grammar.

- Grammar first. Zero runtime cost, immediate gain.
- Bypass keeps its place. It is the only thing that avoids a model call at all.
- Repair only where failure is rare, capped at one retry.
- Evidence is correct but costs a model call where a bypass costs none.
- Ecosystem calls belong in the snapshot path, in parallel, on strict deadlines.
  Never serial before routing.

## The line that never moves

Separate policy from capability compensation. They look alike and age oppositely.

Capability compensation exists because the model is weak. It should be measurable
and retirable.

Policy exists because we decided. The personal boundary, the feminine persona, the
never-search-his-notes rule, the Hexis allowlist and confirmation binding. None of
those yield to a smarter model. A larger model is more dangerous there, not less.

## What we keep

Every stage-0 rule stays exactly as it is. Retiring them was the wrong call and it
would throw away a day of measured gains.

- exact-match fast path
- clock rules
- `SystemTimeDateGrammars`
- `AgendaQueryGrammars`
- `thinSingleToken`, including the social lexicon and the verb-ending test

Three additions, none of which change behaviour:

1. Each rule gets an id and a fixture subset.
2. Each rule writes one trace line when it fires.
3. Each rule carries a comment saying whether its set is closed or open.

That preserves today's accuracy and buys the option to revisit later with numbers.

## What we build

**Dynamic grammars from live state.** The grammars today are static: `routeGrammar`
fixes the 7 intents, `responseGrammar` fixes the mood enum, kiwix `queryGrammar`
fixes word shape. Everything else is a free string.

Candidates in order of payoff:

- **Hexis capability ids.** After discovery the exact list is known. As an enum,
  the model cannot name a capability that does not exist.
- **Act fn allowlist.** Hits the two remaining false clarifies directly. They are
  the act-with-no-allowlisted-fn arm of `gateLLMDecision`, firing on invented verbs.
- **Calendar names.** Enumerate the real ones for agenda and query slots.
- **Known fact keys.** A read-back matches a stored key instead of inventing a
  synonym. This is the general form of the read side we scrapped on 2026-08-01.
- **Nexus display names as act targets.** Only while the list stays small.

Two rules or it backfires:

- **Always include an escape value.** A closed enum with no `other` forces a wrong
  pick instead of a decline. The escape is what feeds the clarify gate.
- **Cache the grammar string, keyed on the state that built it.** Rebuilding per
  turn is fine. Recompiling a large grammar per turn is not.

## Retrieval over regex

Resolve against stores that already exist rather than adding patterns.

Identity is the worked example. Nexus is authoritative, `actionFact` already sets
`Subject`, and `cmd/mavend/factenrichment.go` resolves it in the background. The
scrapped work invented a parallel key namespace with nothing reconciling the two.

A table that grows with real data beats patterns that grow with our patience.

Note a real gap: the personal boundary in `cmd/mavend/actions_query.go` guards
Maven's own store only. It does not know Nexus or Praxis exist.

## Prerequisites

Both are offline and need no deploy. Neither existed on 2026-08-01, and that is
why the day cost what it did.

**Failure taxonomy over the 77-case fixture.** Classify every miss as model
ignorance, contract loss, prompt ambiguity, or our own bug. Only model ignorance
deserves deterministic compensation. The other three get fixed once, for every
model size. Inference, not verified: much of what we patched was the last two.

**Per-assist ablation runner.** One command toggles each assist off and reports the
accuracy delta on its fixture subset. Then retiring an assist is a config flip and
a number, not an argument.

## Held

Not now, and nothing gets built for it.

- A 12B or 35B model. It may be cloud or the 16GB workstation, and cloud crosses
  the current no-third-party line.
- The escalation tier in `gateLLMDecision`.
- Converting the stage-0 heuristics to evidence.

One thing carries forward for free: deterministic assists emit confidence, never a
verdict. A verdict cannot escalate.

## Ruling on the idea list

Nineteen ideas, judged on whether they earn a place in Maven. Checked against the
tree on 2026-08-02, not from memory.

### Build. Absent, and worth it.

**SQLite FTS over embeddings.** No `fts5` anywhere in the tree. Lexical search is
faster than the ONNX embedder, deterministic, and strongest exactly where the
embedder is weakest, which is exact Russian names. Semantic search becomes the
fallback rather than the gate. This is the highest-value absent item.

**Cached TTS phrases.** No cache in `internal/tts`. Confirmations, clarifies and
refusals repeat constantly and their text is already fixed. Pre-rendering them is
cheap and pays straight into the minimal-latency constraint.

**Synthetic router dataset.** Generate Russian tool-calling examples from the same
schemas that will build the dynamic grammars. One source of truth for both, so the
model is trained on exactly the shapes it will be constrained to at inference.

**Command STT separate from dictation STT.** One path today. Commands want latency
and a small vocabulary, meeting capture wants accuracy and can take its time.
`internal/capture` already spools to disk, so the split follows the existing seam.
Medium priority, behind the four above.

### Polish. Present, incomplete.

**Strict JSON everywhere.** Done 02-08-2026. The replier sends
`phraser.ResponseGrammar`. It is exported once so its two copies cannot drift.
The meeting summariser is wrapped and unwrapped in the daemon's Completer, so
`internal/capture` stays text-in/text-out. Every model call now carries a grammar.

**Evidence-first prompting.** Done 02-08-2026, in the evidence branch of
`PhraseQuery`. Sources arrive numbered, one per line. The system prompt no longer
calls them all "заметки" and no longer lets the model add anything of its own.
Blank sources now take the knowledge branch instead of asking for an answer from
an empty list. Not yet measured against a live model. Run `make eval-phrasing`,
and watch the case `query-notes-do-not-answer`.

**Progressive inference.** What exists is a fallback cascade, not escalation. On
error it drops to something weaker. It never escalates on ambiguity. The upgrade is
held with the bigger-model question, and `gateLLMDecision` is the hook.

**Entity dictionaries.** Nexus is the canonical store, `behavior_ru.go` has
`KeyAliases`, `ecosystem_acts.go` has verb aliases. Morphology is the gap, and the
code says so in three places. Russian needs it and there is no stemmer in the repo.

**Assistant state machine.** `confirm.go` models pending confirmation and
`dialogue.Session` models the turn. There is no unified task state. Reuse the
Praxis vocabulary rather than inventing one, because surfaced, acknowledged and
resolved already mean something precise here.

**Session memory compiler.** The digestion tick, `internal/memory` and
`followUpMerge` do parts of this. Not a coherent compile step.

**Background memory maintenance.** Mostly present, and the absent part is small.
What exists: `internal/memeval` reads recent memory on a loop and writes
observations, deduped against its own prior output, unable to speak or act. Facts
supersede at write time through `voidsID`, `CorrectValue` and `VoidLatestFact`,
and `RecentActiveFactsByKind` reads only live rows. Digests, ecosystem traces and
media all prune.

Three gaps remain, all in the fact store:

- Superseding is turn-driven. Nothing reconciles two live facts that contradict
  unless a turn corrects one of them.
- Duplicates written by different sources or phrasings stay as separate live rows.
  There is no key-level merge pass.
- Nothing expires. The log is append-only and grows without bound, and no fact
  ever ages out on its own.

Worth doing, and smaller than it looked. Not urgent.

**Qwen CPT then narrow SFT.** In flight as #122. One disagreement with the idea as
written: do not drop persona from the SFT. Persona is the stated reason the CPT
exists, because stock writes `рад` where Maven needs `рада`. Keep the joint
router-plus-persona tune.

**Offline job queue.** The tick loop, `factEnrichmentWorker` with backoff, and the
media prune already defer work. A general queue is tidier, not more capable. Low
priority.

**Response templates.** Present in `clarify.go`, in the `replySystem` arms and in
`StubPhraser`. Worth extending to high-frequency confirmations, where latency and
persona correctness both matter. Do not extend further. Templating the
conversational reply removes the reason she is worth having.

### Reject.

**Grammar-first routing as a replacement for the LLM router.** Already measured.
The classifier scores 36.8% full accuracy against 72.7% through the cascade.
Replacing the model with rules halves the accuracy. Grammar-first as an ordering is
what stage 0 already is, and that stays.

**Hierarchical intent classification.** Seven intents is already the coarse layer.
The specialisation stage exists as per-intent slot filling in `Extractor.Extract`.
Adding a tier buys structure, not accuracy.

**Tool-specific micro-models.** Contradicts the one-resident-model constraint,
needs per-domain training data nobody has, and multiplies model loads on a single
Vega iGPU. Dynamic grammars give the same domain narrowing at zero runtime cost.

**Local knowledge graph.** Nexus owns entities and relationships. Building a second
graph in Maven breaks the ecosystem line and creates two answers to one question.
If graph traversal is wanted, it is a Nexus feature request.

**Preemptible training.** Training runs in a separate workspace, not on the serving
box. This only becomes real if CPT moves onto homesrv, and that is not the plan.

## Known open, carried over from the deleted handoff

Found live on 2026-08-01, not fixed. Everything else in that file was stale.

- **Kiwix ranks badly on a correct query.** The stop-word pass eats the "and". So
  "кто написал войну и мир" reaches Kiwix as "war peace author". The top hit is
  "List of peace activists" and she summarises that as the answer. The tag
  `scrapped/fact-and-kiwix-phrases` fixes the query text. The ranking is the ZIM
  search and is untouched either way.
- **Chat drags prior turns into an answer.** One live reply mixed the greeting, the
  height statement and a world question. It named Левитан as the author of Война и
  мир.
- **`safeKey` drops Cyrillic**, so Russian calendar events on one day collide.
  Vikunja #443 with three fix options. It is a migration, not a patch.

## Open after the 02-08-2026 deploy

SearXNG runs on homesrv at `http://searxng:9563`, on `maven_default`, and the
rebuilt `mavend` wires it. "кто написал войну и мир?" now routes to query, takes
4 results off the search, and answers Толстой with the lookup opener. Two things
that turn left unsettled.

- **The turn was slow, and nobody knows yet whether that is real.** Route 7s,
  search 1s, phrasing 15s. The p50 in `docs/evals/2026-07-31-routing.md` is 825ms. It
  was the first turn after a cold start with the model still warming, so it
  proves nothing either way. Re-run the same question warm before treating it as
  a regression. Do not plan latency work off this number.
- **The personal boundary has never run live.** `queryPersonal` in
  `cmd/mavend/actions_query.go` stops a question about him from reaching
  SearXNG. The question above is not one, so only tests cover it. Ask something
  about him on the deployed box and confirm from the log that no `voice: search`
  line appears for it.

## Sequence

1. Failure taxonomy over the fixture.
2. Ablation runner, plus ids and trace lines for the existing rules.
3. Dynamic grammar for the act fn allowlist.
4. Dynamic grammar for Hexis capability ids.
5. Dynamic grammar for calendar names and known fact keys.
6. Extend the personal boundary to the ecosystem stores.

Steps 1 and 2 come before anything is written in the router.