Compare commits

...

79 Commits

Author SHA1 Message Date
claude 1a309b283b router/semantic: slice 23 report — corrected population keeps every slice-22 conclusion, hash floor collapses to 0.066, pool 1509 non-action + 720 OOD 2026-09-08 04:02:03 +04:00
claude be71ac406b router/semantic: slice 23 corpus fast-path reconciliation — DeriveFastPath over the real router replaces the regex mirror, factory/merge validated, corpus rebuilt (dataset_hash unchanged), drift diagnostic and rerun tooling 2026-09-08 04:02:00 +04:00
claude bb6bd8efb9 router/semantic: slice 22 report — linear e5 at parity with the deployed cascade, cross-family transfer absent, 143 illegal actions, levers are deterministic 2026-09-08 01:27:21 +04:00
claude 29f74dd3cc router/semantic: slice 22 residual non-action router — emit step, Go harness (legacy + heads modes), Python experiment 2026-09-08 01:26:56 +04:00
claude bdced83b46 router/semantic: slice 21 execution-frame guard report — capability dangerous pass 0/126, guard→sparse P 96.7% FA 6
Deterministic structural guard blocks every capability-question row (0/126 on
all three stress views), never blocks a true action (0/796), declines 196
malformed action rows as ambiguous, and grows no safety leak under
compose: sparse alone FA 9 (capQ 1) → guard→sparse FA 6 (capQ 0).
Execution eligibility recorded as a deterministic policy boundary, separate
from semantic routing and capability selection; next slice returns to the
coarse non-action router instead of another pragmatics training run.

Task/725.
2026-09-07 23:45:33 +04:00
claude fa98e4722e router/semantic: slice 21 deterministic execution-frame guard engine, fixtures, runner and emit step
The guard answers one question — may this utterance become an executable
action — as a three-way policy gate (permissive / blocked / ambiguous) and
never decides what the utterance is. Rules are the encoding of the measured
slice-20 dev-pool discriminators: 126 capability-question rows are 42/42/42
addressed / bare-ability / bare-future; 127 bare можешь+пожалуйста rows are
100% action; can-you-please is 100% action. Reuses the shipped prohibition
parser, morph finiteness and lexicon fillers; reason vocabulary is closed.

Slice 21 (task/725, brief after the accepted slice 20).
2026-09-07 23:43:47 +04:00
claude de1cb40456 router/semantic: slice 20 pretrained fine-tune report — in-pool best, LOFO 1.000
cointegrated/rubert-tiny fine-tuned end-to-end: pair ordering 0.933-0.970
(+0.56-0.78 margin, in-pool) beats every from-scratch config, but the
held-out capability-question family fires 126/126 as action on all 18
config/seeds (FA_rate 1.000, mean proba 0.86-0.99, family σ 0.0045). tiny2
ceiling (newer, larger vocab) does not move LOFO (1.000) and is worse
in-pool. sparse 0.667 remains the safety floor; supervised encoder scaling
ruled out for the generator boundary.
2026-09-07 21:40:53 +04:00
claude dc73fc4e31 router/semantic: slice 20 pretrained-rubert-tiny pragmatics specialist tooling + tokenizer audit gate
cmd/semantic-router-experiment/slice20_pretrained.py (516 lines, over
300-line hook): tokenize/grouped/lfo/metrics/runtime/onnx/ceiling
subcommands, MAX_LEN=25, lr grid 1e-5/2e-5/5e-5 x seeds 42/17/7, 4-thread
fp32, early stop on val PR-AUC. slice20_audit.py: WordPiece audit gate
(UNK rate 0.0026, seq p99 19, no loss above 96). artifacts under /tmp/mvn-s20/.
2026-09-07 21:40:49 +04:00
claude cf2af6ce1e router/semantic: slice 19 report — correct claim head and prose to the seed-stable fail-rate cut 2026-09-07 17:59:10 +04:00
claude 5397f09785 router/semantic: slice 19 sequence pragmatics specialist report — sequence beats sparse on capability holdout, caps near chance on seed-stable seeds 2026-09-07 17:58:16 +04:00
claude 7d31de5a6b router/semantic: slice 19 from-scratch sequence pragmatics specialist tooling + eval index 2026-09-07 17:58:09 +04:00
claude 20215650f2 router/semantic: slice 18 sparse action-gate report — aggregate boundary solved, capability boundary collapsed 2026-09-07 15:35:48 +04:00
claude f2b65cd5d2 router/semantic: slice 18 sparse lexical action-gate probe tooling + eval index 2026-09-07 15:35:41 +04:00
claude 9a25945567 router/semantic: slice 17 nonlinear e5 MLP probe — action gate experiment report 2026-09-07 15:23:46 +04:00
claude e80d45f4b3 router/semantic: slice 17 nonlinear MLP probe tooling + eval index 2026-09-07 15:23:39 +04:00
claude 4ed0238a25 router/semantic: add slice 16 diagnostic to evals index 2026-09-07 14:02:59 +04:00
claude 774c0660b0 router/semantic: slice 16 diagnostic — action/non-action boundary analysis
Experiments:
- Residual-only OOF safety metrics
- False-action decomposition by semantic family (7 families, 24 split groups)
- Paired e5 geometry (gap=0.010 between action and capability question)
- Binary action probe (PR-AUC=0.707, ROC-AUC=0.863)
- Cost-sensitive linear classification
- Expanded action-threshold sweep (7 thresholds)
- Voice-like punctuation stress evaluation
- e5 + tiny structural features
- Four-hypothesis comparison table

Verdict: e5 representation is the primary bottleneck. Action seeds and
capability questions are nearly indistinguishable in embedding space.
2026-09-07 14:02:00 +04:00
claude cbac8b9c4f router/semantic: experiment reports (slice 15)
- 2026-09-07-linear-e5-router-experiment: 136-example baseline (superseded)
- 2026-09-07-expanded-corpus-linear-head-experiment: 3025-example expanded (live)
- CLAUDE.md: index updates for both evals
2026-09-07 13:52:54 +04:00
claude ad3f2d6e99 router/semantic: contract tests for expanded corpus (slice 15)
Minimum corpus size check (>=1000), source count validation,
split-group leakage tests for contrast families.
2026-09-07 13:52:30 +04:00
claude 87411e5fb1 router/semantic: expanded v2 corpus (3025 examples, 93 seed families)
Merged corpus: 535 frozen holdout + 2490 development pool.
Route distribution: action=990, knowledge=880, memory_write=705,
system=226, uncertain=121, conversation=103.
Dataset hash: b27fd48f478ca477cab1e59773bb353a
2026-09-07 13:52:29 +04:00
claude 4bb7555459 router/semantic: corpus-factory deterministic surface generation (slice 15)
93 semantic seed families across 6 routes. Surface generators produce:
direct, polite, modal, first-person, reordered, English, question forms.
Each seed specifies verb forms, subjects, objects, values, tags.
2026-09-07 13:52:12 +04:00
claude 9df223907b router/semantic: sklearn grouped CV linear head experiment (slice 15)
Python experiment: 6-class logistic regression over frozen e5 embeddings,
grouped cross-validation, calibration, abstention curves, contrast-family
analysis, action-threshold sweep.
2026-09-07 13:52:12 +04:00
claude f8ec77de8a router/semantic: e5 embedding cache for linear head experiment (slice 15)
Go tool that computes e5-small embeddings for every corpus row,
assigns frozen/dev split and grouped CV folds, outputs JSON.
2026-09-07 13:51:54 +04:00
claude 6397ea1cc9 router/semantic: merge-corpus tool for frozen holdout preservation (slice 15)
Merges frozen holdout from v1 with v2 development pool, deduplicates,
preserves frozen entries, validates, writes merged corpus.
2026-09-07 13:51:53 +04:00
claude 07bfcea8cc router/semantic: semantic seed type and surface generator (slice 15) 2026-09-07 13:51:33 +04:00
claude 59a0a08d32 router/semantic: legacy baseline eval test (slice 13)
TestLegacyBaseline builds a minimal but complete router (stage-0
grammars, hash-embedder classifier seeded from models/seeds, deployed
confidence threshold) and runs it against the full 136-example corpus.

Reports macro F1, false-action rate, fast-path/residual/router-residual
accuracy, per-route P/R/F1, confusion matrix, and contrast family
breakdown (negation, question, reported_speech, quotation, hypothetical,
capability_question). Pre-route consumed cases are tracked separately
from the fast-path bucket.

The hash embedder is deterministic, so this baseline is reproducible.
The ONNX embedder would score higher; measure both before drawing
conclusions.
2026-09-07 02:09:02 +04:00
claude 1b8ae3c3e5 router/semantic: baseline scoring and contrast family analysis (slice 13)
ScoreLegacy runs the actual router cascade against the 136-example
corpus and produces per-route precision/recall/F1, confusion matrix,
false-action breakdown, and fast-path/residual/router-residual/pre-route
consumption counts. Pre-route consumed cases (command-prohibition grammar
matches at stage 0) are tracked separately — these never reach the
general cascade and should not be scored by the learned router.

ContrastFamilies splits the baseline report by transform tag (negation,
question, reported_speech, quotation, hypothetical, capability_question)
and reports per-family accuracy and false-action rate.

LegacyReport.String() renders the full baseline report with confusion
matrix and false-action case listing.
2026-09-07 02:08:51 +04:00
claude 56051e58c0 router/semantic: baseline types, test helpers, leakage test (slice 13)
Baseline types: LegacyRouter interface (satisfied by *router.Router),
LegacyCase with PrerouteConsumed flag for command-prohibition detection,
LegacyReport with fast-path/residual/router-residual/pre-route breakdowns,
ContrastFamilyReport for per-transform-family scoring.

Test helpers: buildSeededClassifier (hash embedder seeded from
models/seeds/*.txt), actVerbList, seed file loading.

TestContrastFamiliesShareSplitGroup verifies that all contrastive
variants of one base seed share exactly one SplitGroup, preventing
train/eval leakage across the contrast family split.
2026-09-07 02:08:40 +04:00
claude d63619bd0e router/semantic: frozen holdout split and grouped CV (slice 13)
Add FrozenHoldoutSplit with deterministic 15% ratio using dedicated hash
seed. Produces frozen/dev partition with SplitGroup-aware leakage
prevention — all contrastive variants of one seed stay in the same split.

Add GroupedCVFolds for k-fold grouped cross-validation on the development
pool. Each fold preserves split_group boundaries; every example appears
in exactly one eval set across all folds.

Tests verify determinism (same split → same hash), no split_group leakage
across frozen/dev, route coverage in both pools, fold completeness, and
grouped CV coverage.
2026-09-07 02:08:30 +04:00
claude fb9b719f0b router/semantic: corpus validation infrastructure (slice 13)
Add ValidateCorpus and CorpusStatsFrom for structural integrity checks
on the semantic route corpus. Validates total counts, route/source sums,
fast-path+residual partition, empty SourceID/SplitGroup, invalid routes,
duplicate identity, and conflicting labels on identical text.

CorpusStats provides deterministic dataset hash (SHA-256 of sorted texts,
first 16 bytes). ReproducibilityMeta in CorpusEnvelope records source
fixture hashes, generator version, split algorithm, and dataset hash.

TestCorpusValidation exercises the full validation pipeline.
2026-09-07 02:06:58 +04:00
claude 2b38390c87 router/semantic: tests for eval scoring and shadow harness (slice 12) 2026-09-07 01:43:30 +04:00
claude 0ff4562e60 router/semantic: tests for contrastive safety transforms (slice 12) 2026-09-07 01:43:26 +04:00
claude bb7c460d12 router/semantic: tests for contract, mapping, corpus, and split (slice 12) 2026-09-07 01:43:23 +04:00
claude dd98da5c42 router/semantic: shadow harness for experiment observation (slice 12)
ShadowHarness records turns where both the legacy router and the
experiment model produce a decision. Reports agreement/disagreement
split by fast-path vs residual. No action, clarification, capability
selection, or reply depends on the shadow result.
2026-09-07 01:42:39 +04:00
claude 9e2abea16b router/semantic: eval metrics for frozen evaluation set (slice 12)
EvalReport with macro F1, per-route precision/recall/F1, confusion
matrix, false-action rate, and fast-path vs residual breakdown.
ScoreEval runs a SemanticRouter against a frozen eval set and produces
all metrics needed for promotion decisions.
2026-09-07 01:42:36 +04:00
claude b481792c02 router/semantic: contrastive safety transforms and split-by-family (slice 12)
Six deterministic transforms (negation, question, reported speech,
quotation, hypothetical, capability question) applied to action-route
seeds. Each transform determines the expected class explicitly — no
model guessing labels. SplitByFamily uses hash-based bucketing to keep
paraphrases in the same split.
2026-09-07 01:42:27 +04:00
claude 5865699a5a router/semantic: corpus format and v1 dataset (slice 12)
136 examples with provenance from ru_routing_v1.json, tagged
fast_path_resolved vs residual. RouteExample carries source, source_id,
split_group for traceability. Split-by-family prevents paraphrase
leakage across train/eval.
2026-09-07 01:42:22 +04:00
claude b6a21c17eb router/semantic: coarse-route contract, interface, and legacy mapping (slice 12)
Defines the six-class SemanticRoute type (conversation, knowledge,
action, memory_write, system, uncertain), the SemanticRouteDecision
output, the SemanticRouter interface, and the deterministic
Intent→SemanticRoute mapping from the current seven-intent cascade.
2026-09-07 01:42:17 +04:00
claude 494c719a5d router: update all Route callers for NormalizedInput + add invariant tests (slice 11)
Migrate 54 test call sites to construct NormalizedInput{Text: ...}.
Add invariant tests:
- TestNormalizedInputReachesRouteIntact: ingress NormalizedInput reaches Route
- TestTryFastPathReceivesMatchText: TryFastPath gets the same input
- TestMatchTextDoesNotChangeRouting: same Text + different MatchText → same Decision
- TestDecisionUtteranceEqualsInputText: Decision.Utterance == input.Text
2026-09-07 01:32:49 +04:00
claude dad3cd0738 router: Router.Route consumes NormalizedInput directly (slice 11)
Change Route from (ctx, utterance string, now) to (ctx, input NormalizedInput, now).
The ingress-constructed NormalizedInput now reaches the cascade intact — no
reconstruction downstream. TryFastPath receives the same input, not a rebuilt one.
All callers (production, eval framework, tests) updated to construct NormalizedInput.
2026-09-07 01:32:45 +04:00
claude a1e4743492 router: add TryFastPath tests and Route identity invariant (slice 10) 2026-09-07 01:24:28 +04:00
claude 2ef470c2ba router: add TryFastPath with stage-0 grammar evaluation (slice 10) 2026-09-07 01:24:25 +04:00
claude 80f807d2ad router: extract stage-0 grammars into TryFastPath (slice 10) 2026-09-07 01:24:19 +04:00
claude e9049ad27e router: add NormalizeMatchText tests and property invariants (slice 9b)
17 tests covering: whitespace collapse, case folding, NFKC normalization,
punctuation preservation, mixed-script identifiers, ё passthrough, and
property invariants (idempotent, deterministic, never removes punctuation/
wake words/numbers, never mutates original).

Test-only foldYo helper proves ё→е is lossy (всё → все) without
exposing an unused production function.

Vikunja: #725
2026-09-06 22:33:27 +04:00
claude 431e052e6f router: add MatchText to NormalizedInput and NormalizeMatchText function (slice 9a)
Add MatchText field to NormalizedInput — a lossy lexical matching view
derived from ingress text: TrimSpace → NFKC → lowercase → collapse
Unicode whitespace. Does NOT fold ё→е, strip punctuation, strip wake
words, rewrite numbers, or invoke morphology.

Both ingress sites (voice + text) construct MatchText at entry. No
existing consumer reads MatchText yet — it is dark data for future
opt-in migration.

Vikunja: #725
2026-09-06 22:32:51 +04:00
claude 0a2e194e76 router: migrate refusesCommand and ActHasEntityTarget to CapabilitySelection (slice 6c)
Migrate the two remaining action-routing consumers from compatibility
Decision.Slings fields to authoritative Decision.CapabilitySelection:

- refusesCommand: reads CapabilitySelection.Fn instead of Slots.Fn
- ActHasEntityTarget: reads CapabilitySelection.Resolved and
  CapabilitySelection.Args instead of Slots.HasFn and Slots.Args

Slots.Text remains the source for entity text when positional args do
not contain the target (unchanged).

Regression tests prove:
- prohibited sentinel preserved byte-for-byte through SelectCapability
- blanked Slots.Fn/Args/HasFn do not affect migrated consumers
- Praxis/Hexis entity-target routing unchanged
- stage-0 deterministic act unchanged
- classifier/extractor act unchanged

do not remove the compatibility mirrors yet.
2026-09-06 21:43:14 +04:00
claude 18900dd613 docs: add slice 6b capability selection boundary report 2026-09-06 21:02:21 +04:00
claude 2d42e98871 router: update ActionCandidate tests for CapabilitySelection (slice 6b)
Set CapabilitySelection on decisions that have Slots.HasFn=true, so
ResolveActionCandidate reads from the authoritative record. Backward
compatibility tests verify that decisions without CapabilitySelection
still resolve via Slots.HasFn.
2026-09-06 21:02:17 +04:00
claude 4a17821b06 router: ResolveActionCandidate reads CapabilitySelection first (slice 6b)
The candidate now receives Fn/Args from CapabilitySelection (the
authoritative record) rather than from Decision.Slots.HasFn. Backward
compatibility: decisions with Slots.HasFn but no CapabilitySelection
(tests, rebuilt decisions) still resolve via the compatibility path.
2026-09-06 21:02:13 +04:00
claude 4b24fbad98 router: add CapabilitySelection field to Decision (slice 6b)
Authoritative record of which executable capability matched, separate
from Decision.Intent (what kind of turn) and ActionCandidate (downstream
action artifact). Decision.Slots.Fn/Args/HasFn remain as compatibility
representations populated from this selection.
2026-09-06 21:02:09 +04:00
claude 8d83c66d11 router: wire SelectCapability into Router.Route pipeline (slice 6b)
Call SelectCapability after each cascade path (grammar, heads, LLM,
classifier) and propagate the result via applyCapabilityToSlots.

Remove LLM text capability backfill from fillSlots — SelectCapability
now owns that path. fillMatchedSlots retains raw extractor capability
extraction for backward compatibility with stage-0 grammars.

gateLLMDecision now reads CapabilitySelection.Resolved instead of
Slots.HasFn for the act-intent confidence thinning check.
2026-09-06 21:02:05 +04:00
claude a6a62a2d56 router: introduce CapabilitySelection type and SelectCapability stage (slice 6b)
Add the explicit capability-selection boundary between route resolution
and action candidate production. SelectCapability is the single entry
point for selecting which executable capability matched an IntentAct turn.

Three input kinds: raw, llm_text, deterministic. Decision.CapabilitySelection
is the authoritative record; Decision.Slots.Fn/Args/HasFn remain as
compatibility representations populated from the selection.
2026-09-06 21:01:59 +04:00
claude 05f791735d router: add diagnostic resolution method matrix tests
Run routing and ecosystem fixtures through the baseline router and
report which component selected the exact function for every IntentAct
case. Shadow matcher comparison confirms zero disagreements.
2026-09-06 13:51:37 +04:00
claude bdd79ad585 mavend: carry ResolvedBy through dialogue Slots bridge
Add ResolvedBy to dialogue.Slots and the toDialogueSlots/
applyDialogueSlots converters. Skip reflect-type check for this field
in parity test (dialogue cannot import router: import cycle).
2026-09-06 13:50:30 +04:00
claude 66f06796cb router: add provenance tests for ActionResolutionMethod
Pin each path: grammar_fixed, grammar_matcher, extractor_raw,
extractor_llm_text, fallback_matcher. Verify unresolved has empty
ResolvedBy. Verify fn/args remain byte-for-byte identical.
2026-09-06 13:50:22 +04:00
claude 06adc4702d router: set ResolvedBy at each function selection point
Assign provenance where the exact fn is produced:
- grammar_fixed: praxis/task-status grammars hardcode fn
- grammar_matcher: wakeword-act grammar invokes ActMatcher
- extractor_raw: Extractor.Extract matches over raw utterance
- extractor_llm_text: fillSlots LLM backfill matches cleaned text
- fallback_matcher: ResolveActionCandidate runs the fallback matcher

ResolveActionCandidate propagates Slots.ResolvedBy into
ActionCandidate.ResolvedBy. No selection behavior changes.
2026-09-06 13:50:03 +04:00
claude 1d02ba8936 router: add ActionResolutionMethod type and ResolvedBy field to Slots
Five disjoint values tracking which component selected the exact function:
grammar_fixed, grammar_matcher, extractor_raw, extractor_llm_text,
fallback_matcher. Slots.ResolvedBy carries provenance at the selection
point.
2026-09-06 13:49:50 +04:00
claude 66c578a6f4 router: introduce typed ActionValidationStatus boundary (slice 5)
Introduce ActionValidationStatus enum (valid, unresolved, missing_argument,
invalid_argument, ambiguous_target) as the typed classification of validation
outcomes. ActionValidationResult now carries Status instead of boolean flags.

Backward-compatible: Unresolved() and Valid() methods preserved on the result.
Existing validation behavior unchanged: only blank Fn produces invalid_argument.
All downstream behavior (proposeGap, confirmation, task_status, praxis, hexis)
unchanged.

Tests added for all five status values, backward compatibility, and the full
validation → execution boundary.
2026-09-06 13:07:08 +04:00
claude 356766bce1 mavend: centralize action validation boundary (slice 4) 2026-09-06 12:53:54 +04:00
claude 6a402bf556 docs: add action-resolution boundary slice report 2026-09-05 21:51:43 +04:00
claude f6d7b05161 mavend: add action-resolution regression tests
Eight integration tests pinning the action-resolution boundary:

1. TestActRouteSource_NoMatcherInvoke — HasFn=true, route-sourced
2. TestActMatcherSource_FallbackMatch — no Fn, matcher resolves
3. TestActMatcherMiss_ProposeGap — matcher miss → propose-gap
4. TestActDestructive_ConfirmationUnchanged — destructive → confirm
5. TestActTaskStatus_InterceptUnchanged — task-status intercepted
6. TestActStage0_SameResult — stage-0 act executes same tool
7. TestActLearnedRouter_NoFn_FallbackMatch — LLM no Fn → matcher
8. TestResolveAction_CandidateSource_Verified — verifies all paths

All 8 pass. All existing tests pass.
2026-09-05 21:50:50 +04:00
claude 025f81e961 mavend: wire resolveAction into actionAct
Daemon half of the action-resolution boundary:

- Add resolveAction wrapper: delegates to ResolveActionCandidate,
  records outcome in the decision trace (action-resolve:route/matcher)
- Refactor actionAct: remove matcher call, consume candidate, write
  resolved values back into Slots for downstream branches
- Add 8 integration tests pinning all required scenarios:
  route-sourced, matcher-sourced, matcher miss, destructive confirm,
  task-status intercept, stage-0, learned-router, alias match

All existing tests pass. Execution/risk/confirmation unchanged.
2026-09-05 21:50:33 +04:00
claude 064747f192 router: add ActionCandidate type and ResolveActionCandidate
Introduce the typed boundary between routing and action resolution:

- ActionCandidate: Fn, Args, Source (route|matcher), Producer, Confidence
- ResolveActionCandidate(dec, m): standalone function usable by both
  the daemon and the eval harness
- Update eval harness Reach() to use ResolveActionCandidate instead of
  duplicating the matcher fallback logic

This is the routing-side half of the action-resolution boundary.
The daemon integration follows in the next commit.
2026-09-05 21:49:25 +04:00
claude a55d90954a router: add boundary tests for typed ingress and route producer
12 focused tests proving the first slice properties:
- text and voice enter equivalent typed turn input after stt
- stage-0 outputs remain identical with grammar producer
- classifier floor sets its producer
- clarification carries the classifier producer
- pre-route claims produce no route producer
- route producer appears on the decision record
- input source is preserved on the decision record
2026-09-05 20:16:57 +04:00
claude 87a3b163e7 router: introduce typed ingress boundary and route producer observability
First behavior-preserving slice of the Maven redesign. Establishes
explicit ingress/routing boundaries and enough observability to refactor
later without changing current routing, action, clarification, or
execution semantics.

Types introduced:
- NormalizedInput (internal/router/source.go): Text + InputSource,
  the typed ingress boundary replacing raw string at the turn entry.
- InputSource (internal/router/source.go): channel provenance enum
  (tap:voice, tap:text). Reuses the existing turnSource distinction.
- RouteProducer (internal/router/intent.go): which cascade stage
  produced the decision (grammar, heads, llm, classifier).

Changes:
- Decision carries a Producer RouteProducer field, set at each cascade
  stage (grammar, heads, LLM, classifier).
- turnRoute carries NormalizedInput instead of bare text string.
- runTurn takes NormalizedInput instead of (text, src).
- decision.Record carries InputSource and RouteProducer for
  observability; RoutingTrace persists route_producer (migration #27).
- turnSource is now a type alias for router.InputSource.

Behavior preserved:
- Stage-0 grammars unchanged: same order, same matching, same confidence.
- Cascade fallthrough order unchanged (grammar → heads → llm → classifier).
- Clarification behavior unchanged.
- Action dispatch unchanged.
- No new linguistic normalization.
2026-09-05 20:16:40 +04:00
claude 2f338a1ab6 Hide the relation filters in the capability views and render inline code (V-725)
Two defects found by screenshotting the built page under headless chromium,
which is the only way to see either.

The six relation filters and the component-type legend do nothing in views 6 and
7. Leaving them on screen reads as controls that are broken.

The ledger carries markdown inline code, because docs/spec.md does. The side
panel printed the backticks literally beside every path.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 13:21:21 +04:00
claude be062b2d48 Add the capabilities and invariants views to the viewer (V-725)
Session 3, and the end of the plan.

View 6 is the matrix: 51 capabilities against designed, code_present, wired,
configured, deployed, reachable and verified, grouped by spec section or by
domain. Clicking a row opens the definition of done with every verdict, its
reason, its detail and its evidence paths, the components that carry the
capability, the blockers and the product questions it waits on.

View 7 is the twelve invariants. Each shows three things apart: target is
whether the rule is written down, implementation is the status of the
participating components, and runtime is what the probe run observed for the
capabilities it touches. Component and capability chips cross-link into the
other views.

invariants.yaml is the machine-readable half of invariants.md. The two exist
separately so the viewer can read one and a person can read the other, and
build_ledger.py refuses to build when they disagree: a missing heading, a count
mismatch, an unknown capability or component, or an unresolved invariant with no
product question.

build_viewer.py inlines ledger.yaml and invariants.yaml and derives nothing. The
ledger's build is the only thing allowed to decide a dimension.

check_viewer.js is the viewer's only check. A TypeError in a renderer shows as a
blank panel and not as an error, so it runs all seven views, all three flows,
all 51 capability panels and all 160 component panels against a DOM stub, and
fails on a panel that comes back thin. render.sh calls it and skips it with a
message when node is absent.

--no-verify: the template and the smoke test are 320 non-markdown lines.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 13:15:24 +04:00
claude 7f804b84e7 Declare the two generated doc tiers and index their evals (V-725)
docs/CLAUDE.md defined four tiers and docs/capabilities/ and docs/architecture/
were neither of them. Both are now declared as generated: rebuilt from a source,
never corrected in place. A wrong row in either is a bug in the generator or in
one of its hand-written inputs, and editing the output makes the next rebuild
silently undo the fix.

The eval index gains the 2026-08-19 CPT+SFT measurement and the 2026-08-26
baseline, and marks the 2026-08-13 capability audit superseded. The 2026-08-19
file lands with them: it is the direct evidence that the deployed
maven-instruct-b2 routes better than Qwen3-1.7B and cannot hold a Russian
sentence, which is the first item on the gaps.md priority list.

Its header records Vikunja as returning 503. That reading was wrong and the
2026-08-26 baseline says so, but a dated eval is not edited after the day.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 12:45:54 +04:00
claude bae81b66c8 Track the architecture observation and its inventory (V-725)
docs/capabilities/build_ledger.py reads the component statuses out of
maven-architecture.json, so the whole implementation half of the ledger fails to
build on a clone that does not have it. It has to be tracked.

What lands: the five generator scripts, the viewer template, findings.md, the
README and the seven .mmd diagram sources, plus the inventory JSON itself.
verify_anchors.py resolves 681 of 692 claimed symbols to path:line and exits
non-zero on a miss, 11 skipped as config keys. That proves an identifier sits on
a line and nothing more. Writing the responsibility field caught 29 symbols
filed under the wrong component and 7 names invented outright, and a later
refutation pass caught 4 wrong readings on top of that.

What does not land, and is now gitignored: index.html at 836 KB of inlined JSON
and SVG, anchors.md, architecture-evidence.txt, tree.txt, the redacted compose
file, the rendered SVGs and maven-evidence.zip. All of them rebuild with
pack_evidence.sh.

render.sh is the only syntax check this repo has for a .mmd, and it found two
real parse errors on its first run.

--no-verify: 4,900 non-markdown lines. The inventory and its generator are one
artifact and neither is readable without the other.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 12:45:20 +04:00
claude 8153e5eaa5 Classify every gap and rank the work (V-725)
Session 2 step 3, and the end of the explanation half.

gaps.md compares responsibilities and never package names. Eight classes. The
four capability classes are derived from the ledger's gap_class field and
rebuild with build_ledger.py. The four architecture classes are read from
findings.md and invariants.md, and every entry names the capability or invariant
it affects. An entry naming neither is marked non-blocking cleanup in those
words, which is the whole of class 8 and its eleven rows.

Of 46 v1 capabilities: 5 missing, 21 partial and reachable, 9 built and
unreachable, 11 reachable and unverified.

The nine unreachable ones are seven config blocks and two compose entries. Not
one is a code defect.

The ranked list puts phrasing first: speak-as-herself fails all three criteria,
and everything that asks the resident model to write a Russian sentence inherits
that. His own name not being stored is second. Nine capabilities one config
change from reachable is fourth, and it is the highest ratio of capability to
work in the list.

Items 10, 12 and 13 stall on unresolved invariants and are the owner's call, not
work.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 12:44:15 +04:00
claude 41c97bba8d Write the twelve cross-cutting invariants (V-725)
Session 2 step 2. docs/spec.md states 51 capabilities one at a time. Twelve
rules run across all of them and no DoD states any of them, so breaking one
breaks many capabilities at once without producing a failing criterion.

Each is marked explicit, implied or unresolved, with evidence. Nothing wanted is
invented where the sources are silent.

Four are unresolved and belong to the owner rather than to a commit: authority
and confirmation, learning from outcomes, capability composition, and whether a
held nudge has a shelf life. Two of the three questions the freeze was called to
answer show up here as invariants 8 and 11.

Privacy boundaries and proactive attention are the two best-specified rules and
neither showed a live defect. Authority is the largest hole: internal/auth
answers who may carry what authority and does not bind the turn path,
internal/tool answers what effect an act has and is not keyed on the reach, and
praxisItemAction.handle has no gate at all.

The one file in docs/capabilities/ that is hand-written rather than generated.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 12:40:52 +04:00
claude f9b0a96d9d Document the seven dimensions in the directory README (V-725)
The rebuild section named the new inputs and nothing said what the columns
mean or how a partial arises.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 12:38:18 +04:00
claude 40ec0c0d4b Map every capability to its components in seven dimensions (V-725)
Session 2 step 1. Implementation status was the missing half: the ledger said
what should happen and what happened, and nothing said how much is built.

Never one implemented boolean. designed, code_present, wired, configured,
deployed, reachable and verified are separate, because coded and unwired, wired
and unconfigured, and configured and undeployed are three different pieces of
work.

The six build dimensions derive from the status field of every component the
capability maps to, rolled up as all yes, none no, otherwise partial. The
statuses come from docs/architecture/maven-architecture.json, which read them
from code, config and compose. verified comes from the criteria verdicts.

implementation.yaml is the mapping and is the judgment call. Shared
infrastructure is deliberately unmapped: putting core.reactive_handler on all 51
rows would give them one status and say nothing.

Of 51 capabilities, 45 have code and 33 are reachable. 22 are spec-only, with no
living doc owning the subsystem.

The build now reports what it cannot reconcile. learning-the-style has no
component and still scores a pass, because its passing criterion is negative and
absence satisfies it. Sixteen components serve no capability, ten of them the
shared infrastructure excluded on purpose, and the rest are core.q.habits,
core.q.money, ext.zenmoney, router.claim and router.modes.

--no-verify: the regenerated ledger is 500 lines of derived output.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 12:37:37 +04:00
claude af6e6c9979 Give the plan its task id (V-725)
Filed after the fact, so the plan and the index both said it had none. The
Vikunja task carries the session-1 result and what sessions 2 and 3 still owe.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 12:27:54 +04:00
claude 3cced9a2e9 Freeze the empirical baseline as a dated eval (V-725)
What the deployed Maven did when it was asked, measured 2026-08-26 against
master 5cae33a plus the uncommitted deploy/mavend.json model switch. Frozen on
the day and not edited after it.

The number is attributable to the deployed resident model,
maven-instruct-b2-Q4_K_XL, not the Qwen3-1.7B that CLAUDE.md names. What comes
closest to working is what never asks that model to write a Russian sentence.

make test is green throughout. It is also green with the four TestONNX
measurements silently skipped, because the recipe does not set MAVEN_ONNX_LIB.

Every verdict was audited by an independent pass told to refute it, one auditor
per spec section, with pass attacked hardest. The corrections moved the tally by
nine: four passes withdrawn, five criteria filed untested turned out already
settled. The section on how the verdicts were checked names the mistakes, so the
next session does not have to trust that this one got it right.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 12:26:34 +04:00
claude bc1ef0f57f Score every criterion and build the ledger (V-725)
verdicts.json carries one verdict per criterion id. ledger.yaml is what
build_ledger.py produces from docs/spec.md, domains.yaml and those verdicts.

146 v1 criteria: 26 pass, 50 fail, 15 blocked, 51 untested, 4 unknown. No
capability passes all of its own criteria. Six fail every one: speak as herself,
weather, wake word, summaries, webhooks, command chaining.

Only a live verdict sets pass. Every verified cell cites
docs/evals/2026-08-26-capability-baseline.md by path and section, and the
generator refuses to build if either does not resolve.

Both files are generated. Rebuild rather than hand-edit.

--no-verify: 3,386 non-markdown lines, all of it generated output that cannot
split into reviewable ideas.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 12:26:33 +04:00
claude f002ce0e9c Add the probe harness and the raw output of the field run (V-725)
probes_field.json is 25 multi-turn probes drawn from the owner's real week.
run_probes.py drives them through the deployed stack: POST /api/chat on
127.0.0.1:9201, which runs a real turn through the pre-route ladder, the stage 0
grammars, the routing heads, the resident model, the query walk, the act path
and the phraser. Readback is cmd/e2eprobe over the mavend IPC socket, never the
plaintext sqlite copy in /dev/shm and never the mavweb HTML pages.

store_counts.py reads row counts per store over IPC, before and after.

out/ holds what the run produced. field.contaminated.jsonl is the discarded
first run: mavweb hardcodes one conversation id for the whole web reach, so a
clarify parked by one probe was still parked for the next.

field.transcript.tsv is the evidence for the baseline and is not summarised
anywhere else. The store it came from was wiped afterwards.

--no-verify: 326 non-markdown lines of new harness plus its captured output.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 12:26:32 +04:00
claude 3adfc3e0f9 Add the ledger generator, its domain axis and the directory README (V-725)
build_ledger.py extracts 51 capabilities and 156 DoD criteria from docs/spec.md
and joins them with domains.yaml and verdicts.json. The generator is also the
checker: it exits non-zero on a capability with no DoD criteria, no State line,
no domain or more than two, an unknown domain, a criterion id collision, a
domains.yaml or verdicts.json row naming something that does not exist, a
verdict word outside the five, and a reason outside the plan's list. It caught
the domain reconciler silently dropping recall from its 51.

It also refuses an evidence path that does not resolve, a section heading absent
from the file it names, a pass whose reason is not passes, and a fail resting on
no runtime proof. Sixteen verdicts had cited a section of the eval that did not
exist.

domains.yaml is the one judgment call in the extraction and is hand-edited.

--no-verify: 584 non-markdown lines, all of them new files. The generator and
the domain table it reads are one reviewable idea and splitting them leaves
neither readable alone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 12:26:30 +04:00
claude b6666196c1 Plan the capability ledger and the empirical baseline (V-725)
Three sessions on one causal order: the spec says what should happen, the
empirical run says what actually happens, code and architecture explain why,
priority says what to fix. The predecessor audit had a green suite while 22 of
39 capabilities were not live, which is the failure mode this order exists to
stop.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 12:26:28 +04:00
175 changed files with 84554 additions and 396 deletions
+20 -12
View File
@@ -11,14 +11,18 @@ started with an agent that inferred the goal instead of stating it back.
## 0. Get on the branch
```sh
task start <vikunja-id>
task start <vikunja-id> # with an id
git checkout -b task/<slug> # without one
```
`~/.local/bin/task` owns the branch, the identity and the PR. It cuts
`task/<id>-<slug>` off `origin/master` and sets the commit author to the `claude`
gitea user. It writes `TASK.md` from the Vikunja task, and pulls any waiting
An id is optional (owner's call, 2026-08-25). With one, `~/.local/bin/task` owns
the branch, the identity and the PR. It cuts `task/<id>-<slug>` off
`origin/master` and sets the commit author to the `claude` gitea user. It writes `TASK.md` from the Vikunja task, and pulls any waiting
review comments into `.task/review-comments.md`. Do not hand-roll any of that.
Without an id, branch by hand and skip `TASK.md`. The user's own brief is then
the goal, and step 4 restates it back to him instead.
`TASK.md` is the brief and it is immutable. If it says a PR already exists, this
is a review-fix session and not new work. Read the comments first.
@@ -34,9 +38,12 @@ If there is no handoff, that is normal. It means the last session closed clean.
In this order, and stop as soon as you have enough:
- The Vikunja task, by id. Project Maven is ID 2, MCP at `http://localhost:9100/mcp`.
The task description and its comments hold the goal, the constraints, and the
assumption ledger. This outranks the handoff on every conflict.
- The Vikunja task, if there is one. Project Maven is ID 2, MCP at
`http://localhost:9100/mcp`, reachable from workpc only through
`ssh -N -f -L 9100:127.0.0.1:9100 kami@192.168.1.104`. A refused connection is
the missing tunnel, not an outage. The task description and its comments hold
the goal, the constraints, and the assumption ledger. This outranks the handoff
on every conflict.
- `CLAUDE.md`, the section that covers the area you are about to touch.
- The one file under `docs/` that owns the area. Check its `Last verified` line.
If the sha is behind the code you are reading, say so in step 4 and trust the code.
@@ -44,8 +51,8 @@ In this order, and stop as soon as you have enough:
Do not read the dated files under `docs/evals/`. They are measurements from one day,
never updated. Read one only when you need the number it recorded.
If no task id is known, ask for one before doing anything else. Work without a task
is work nobody can resume.
With no task id, do not ask for one and do not stall. State it in step 4 as
`Task: unfiled` and carry on.
## 3. Look at the ground
@@ -58,7 +65,7 @@ Write at most five bullets and stop. Do not write code, do not open files to "ch
one thing first", do not start with a small safe change.
```
Task: V-359, one line.
Task: V-359, one line. `unfiled` when there is no id.
Done: what is already on the branch.
Next: the one thing this session does.
Constraints: what would make this wrong.
@@ -67,8 +74,9 @@ Assuming: the beliefs that, if false, waste the session.
Then ask: is this right? Wait for the answer.
A corrected assumption goes into the Vikunja task as a comment, not into the handoff.
The handoff dies tonight. The task does not.
A corrected assumption goes into the Vikunja task as a comment where there is a
task, because the handoff dies tonight and the task does not. Unfiled, it goes
into the handoff and nowhere else.
## 5. Then begin
+10 -4
View File
@@ -36,11 +36,13 @@ a commit message, not into a comment in the code.
Under 300 changed lines per commit in non-markdown files, enforced by `.githooks/pre-commit`.
Markdown is exempt and may land as one batch.
Each commit is one idea, subject in the repo's voice, lowercase area prefix, and it
ends with the Vikunja ref:
Each commit is one idea, subject in the repo's voice, lowercase area prefix. A
Vikunja ref is welcome where a task exists and is required nowhere: the
`commit-msg` hook that demanded it was deleted on 2026-08-25.
```
router: narrow the single-token rule (V-359)
router: narrow the single-token rule
```
If a change genuinely cannot split under 300 lines, say why in the commit body before
@@ -56,13 +58,17 @@ It refuses a dirty tree, pushes, opens or refreshes the PR against the repo defa
branch, labels the Vikunja task in-review, comments the PR url on it, and pushes an
ntfy. Do not push by hand and do not call `tea` yourself.
`task pr` needs an id. On a hand-cut branch with no task, push the branch and open
the PR by hand, and skip step 5.
## 5. Record what `task pr` cannot know
Comment on the Vikunja task: what you measured, what is still open. List every
assumption that turned out to be wrong. If the session found new work, create a task
for it now rather than describing it in prose.
This step is what makes the handoff disposable.
This step is what makes the handoff disposable. With no task, it cannot run, so the
handoff carries that content instead and stops being disposable. Say so in it.
## 6. Leave the handoff, or leave none
@@ -75,7 +81,7 @@ resume, and no history:
```markdown
# Handoff — <date>
Task: V-359 <one line>
Task: V-359 <one line>, or `unfiled`
Branch: task/359-<slug>, cut from master
## Where I stopped
-30
View File
@@ -1,30 +0,0 @@
#!/bin/sh
# Every commit names the Vikunja task it belongs to.
#
# router: narrow the single-token rule (V-359)
#
# V- and not #, because Gitea autolinks #359 to a Gitea issue, which is a
# different tracker and a wrong link.
#
# Exempt: merges, reverts, fixup/squash, and the initial commit.
msg_file=$1
subject=$(sed -n '1p' "$msg_file")
case "$subject" in
Merge\ *|Revert\ *|fixup!\ *|squash!\ *|amend!\ *) exit 0 ;;
esac
if [ -f "$(git rev-parse --git-dir)/MERGE_HEAD" ]; then
exit 0
fi
if printf '%s' "$subject" | grep -qE '\(V-[0-9]+\)$'; then
exit 0
fi
echo "commit-msg: subject must end with a Vikunja task ref." >&2
echo " got: $subject" >&2
echo " want: router: narrow the single-token rule (V-359)" >&2
echo " No task yet? Create one. Work without a task is work nobody can resume." >&2
exit 1
+15
View File
@@ -76,3 +76,18 @@ __pycache__/
.env
# silero-vad, downloaded (see AGENTS.md)
/models/vad/
# Go build cache and GOPATH from the containerised e2eprobe build. Created by
# the command in docs/capabilities/README.md, which runs as root in a container
# and so cannot share the host cache. Multi-GB, entirely reproducible.
/.cache/
# docs/architecture/ derived output. The sources, findings.md and the inventory
# JSON are tracked; these rebuild from them with pack_evidence.sh and are large.
/docs/architecture/index.html
/docs/architecture/anchors.md
/docs/architecture/architecture-evidence.txt
/docs/architecture/tree.txt
/docs/architecture/docker-compose.redacted.yml
/docs/architecture/diagrams/*.svg
/maven-evidence.zip
+5 -5
View File
@@ -201,14 +201,14 @@ ToolSearch("select:mcp__vikunja__list_tasks,mcp__vikunja__get_task_details,mcp__
```
- This repo is Vikunja project **Maven** (ID 2), MCP at
`http://localhost:9100/mcp`, or `http://192.168.1.104:9100/mcp` from workpc.
- **A session with no task id asks for one before it starts**, because work
without one is work nobody can resume.
`http://localhost:9100/mcp` on homesrv. **`vikunja-mcp` publishes to
`127.0.0.1:9100` only, so the LAN address never answers from workpc.** A
refused connection is that, not an outage: three sessions read it as "Vikunja
is down" and filed nothing. Tunnel first, then use `localhost`:
`ssh -N -f -L 9100:127.0.0.1:9100 kami@192.168.1.104`.
- **Close a finished task with `done: true` and nothing else** (owner's call,
2026-08-07). `update_task` carrying a `description` resets `done` to false.
- **`pre-commit` refuses master** and more than 300 changed lines in
non-markdown files. Markdown is exempt and may land as one batch.
- **`commit-msg` requires the subject to end with `(V-<id>)`.** `V-` and not
`#`, because Gitea autolinks `#123` to the wrong tracker.
- **`diff-budget.sh` blocks edits past 600 changed lines** on a `task/` branch.
- **`--no-verify` exists.** Using it means saying why in the commit body.
+93
View File
@@ -0,0 +1,93 @@
# Handoff
Master is at `5cae33a`, pushed, tree clean apart from this file. Working on master
raw by the owner's call: no branch, `--no-verify` on every commit with the reason
in the body.
## Landed this session
Twelve commits pushed. Nine were the previous session's tree, already described in
the commit log. Three are new:
- `78a9c61` `docs/spec.md`, 51 capabilities with a DoD each.
- `02e3d27` `docs/roadmap.md`, seven milestones, plus both pointer rows in `CLAUDE.md`.
- `5cae33a` honesty split into three milestones, five capabilities deferred past v1.
**Read `docs/spec.md` and `docs/roadmap.md` before anything else.** Every decision
from this session is in them. This file holds only what they do not.
## The two documents
`docs/spec.md` is the union of the 39 audited rows
(`docs/evals/2026-08-13-capability-audit.md`) and the owner's 18-item v1 list.
Twelve of his items had no audit row, so the file has 51 entries. Each entry
carries three parts. State is a reference to the living doc that owns it. DoD is
a plain list observable on the running box. Scenario names a file in
`cmd/mavend/testdata/scenarios/`.
`docs/roadmap.md` orders them into nine milestones. Honesty, then reach, then
breadth. Not ordered by code work, because none of the four broken capabilities is
a code defect.
## Decided, do not re-ask
- **v1 is a voice assistant, minimum viable.** Each DoD is written at
"voice-reachable and honest", not "feature-complete".
- **Honesty splits into three.** M1 the turn path, M2 memory he cannot correct,
M3 step-up. M1 and M2 touch different code and owe different docs. Step-up is
configuration, not honesty, and sits before M4 because M4 is what first makes
acts real.
- **Five capabilities deferred past v1** (owner's call, 2026-08-15): speaker
recognition, smart home, bluetooth control, model swap, self-update. Bluetooth
was on the v1 list and came off it. Their spec entries keep their DoD.
- **A milestone closes its own doc gaps and writes its own scenarios.** Neither
becomes a milestone of its own. Otherwise the 17 missing docs and 46 missing
scenario files collect at the end.
- **Learning means behavioral, not weights.** Stored outcomes only. No adapter, no
training set.
- **Email and calendar need the product decision before deploying.** Both are
built and neither is in `docker-compose.yml`. That is M8.
## Findings the spec pass produced
- **Recurring reminders do not exist on the spoken path.** `store.Reminder` carries
`Cron` and `ipc.CreateReminder` takes one. `grep "Cron:" --include='*.go'`
outside tests returns only `internal/ipc/client.go`, `internal/ipc/storeapi.go`
and `cmd/mavend/tick_routines.go`, and the last is routines, a separate
mechanism. Pills, the dog, the vet and the kibble are unbuilt on top of finished
storage and delivery. This is M6.
- **Seventeen capabilities have no living doc.** Memory is the worst cluster:
facts, notes and the digestion worker have no owning document at all.
- **Webhooks barely exist.** The only one in the tree is
`internal/delivery/telegramsink/intake.go`, Telegram's own inbound hook.
- **Command chaining does not exist.** The `chain` in `internal/router` is the
world chain and the source chain.
- **Desk notifications are inbound only.** `ambient:notif` reads his desktop.
There is no outbound desk reach, and which direction he meant is undecided.
- **Only 5 of 51 spec entries cite a scenario that exists.**
## Not filed, and this is the risk
Vikunja returned 503 across this session and the last, so **none of this has a
task id**. The three commits above are tagged `V-719`, which is the
reminder-cancellation task, not this work. Retag or file when Vikunja is back.
Unfiled, listed again at the end of `docs/roadmap.md`:
1. The capability audit itself, headed "unfiled".
2. Remember-versus-query misroute, two of seven audit probes.
3. The masculine reply on the wire, caught live while `CheckFeminine` passed.
4. Recurring reminders having no caller.
5. The seventeen capabilities with no living doc.
6. The 46 scenario files the spec names and does not have.
## Next
Open a session on M1, which is three gate items on the turn path and owes no new
doc. If Vikunja is up, file the six above first and give M1 a real id.
One command still outstanding from the last session, cheap and unrelated:
```sh
docker compose up -d --force-recreate mavsttd mavttsd mavpoll
```
File diff suppressed because it is too large Load Diff
+92
View File
@@ -0,0 +1,92 @@
package main
import (
"context"
"github.com/kami/maven/internal/decision"
"github.com/kami/maven/internal/router"
)
// resolveAction produces an ActionCandidate from a routing decision. It is the
// single boundary between routing and action execution: everything downstream
// (refusesCommand, task-status, Praxis, Hexis, proposeGap, tool.Executor.Exec)
// consumes the candidate rather than re-resolving the function.
//
// Delegates to router.ResolveActionCandidate for the resolution logic, then
// records the outcome in the decision trace.
func (h *reactiveHandler) resolveAction(ctx context.Context, dec router.Decision) router.ActionCandidate {
candidate := router.ResolveActionCandidate(dec, h.matcher)
// Record the resolution outcome in the decision trace.
if dec.Intent == router.IntentAct {
if candidate.ActionResolved() {
noteActionResolution(ctx, string(candidate.Source), candidate.Fn, true)
} else {
noteActionResolution(ctx, "matcher", "", false)
}
}
return candidate
}
// noteActionResolution records the action resolution outcome in the decision
// trace. A nil recorder is the normal case in tests.
func noteActionResolution(ctx context.Context, source, fn string, resolved bool) {
rec := decision.From(ctx)
if rec == nil {
return
}
outcome := decision.Declined
reason := "no match"
if resolved {
outcome = decision.Won
reason = "resolved via " + source
if fn != "" {
reason += ": " + fn
}
}
rec.Note(decision.Claim{
Stage: decision.StageAction,
Claimant: "action-resolve",
Outcome: outcome,
Reason: reason,
})
}
// noteActionValidation records the structural validation outcome in the
// decision trace. Five outcomes: unresolved (matcher miss), valid
// (structurally admissible), invalid_argument, missing_argument, or
// ambiguous_target (structurally malformed).
func noteActionValidation(ctx context.Context, v router.ActionValidationResult) {
rec := decision.From(ctx)
if rec == nil {
return
}
switch v.Status {
case router.ActionUnresolved:
rec.Note(decision.Claim{
Stage: decision.StageAction,
Claimant: "action-validation",
Outcome: decision.Declined,
Reason: "unresolved",
})
case router.ActionValid:
rec.Note(decision.Claim{
Stage: decision.StageAction,
Claimant: "action-validation",
Outcome: decision.Won,
Reason: "valid",
})
default:
reason := string(v.Status)
if len(v.Issues) > 0 {
reason = string(v.Status) + ":" + v.Issues[0].Reason
}
rec.Note(decision.Claim{
Stage: decision.StageAction,
Claimant: "action-validation",
Outcome: decision.Declined,
Reason: reason,
})
}
}
+556
View File
@@ -0,0 +1,556 @@
package main
import (
"context"
"strings"
"testing"
"time"
"github.com/kami/maven/internal/decision"
"github.com/kami/maven/internal/ipc"
"github.com/kami/maven/internal/router"
"github.com/kami/maven/internal/store"
"github.com/kami/maven/internal/tool"
)
// newActHandler builds a handler with the act path wired: a matcher over
// whatever tools the test enabled, no model, no ecosystem.
func newActHandler(t *testing.T) (*reactiveHandler, *store.Store) {
t.Helper()
st := newTestStore(t)
api := ipc.NewStoreAPI(st)
matcher := tool.NewMatcher(api)
h := &reactiveHandler{
api: api,
tools: tool.NewExecutor(api, 2*time.Second),
matcher: matcher,
now: func() time.Time { return time.Date(2026, 8, 1, 9, 0, 0, 0, time.UTC) },
}
return h, st
}
// TestActRouteSource_NoMatcherInvoke pins that an act with HasFn=true
// produces a candidate from the route and does not invoke the matcher.
func TestActRouteSource_NoMatcherInvoke(t *testing.T) {
h, st := newActHandler(t)
ctx := context.Background()
now := h.now()
// Enable a tool so the matcher has something to match against.
if err := st.EnableTool(ctx, "status", []string{"true"}, false, "test", now); err != nil {
t.Fatal(err)
}
// Act with HasFn=true: the candidate must come from the route.
reply := h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Utterance: "status",
Slots: router.Slots{Fn: "status", HasFn: true},
})
if !strings.Contains(reply, "готово") {
t.Errorf("route-sourced act replied %q; want it to have run", reply)
}
}
// TestActMatcherSource_FallbackMatch pins that an act without Fn invokes
// the matcher and produces a matcher-sourced candidate.
func TestActMatcherSource_FallbackMatch(t *testing.T) {
h, st := newActHandler(t)
ctx := context.Background()
now := h.now()
// Enable a tool so the matcher can find it.
if err := st.EnableTool(ctx, "status", []string{"true"}, false, "test", now); err != nil {
t.Fatal(err)
}
// Act without HasFn: the matcher must resolve "status" from the text.
reply := h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Utterance: "check status",
Slots: router.Slots{Text: "status"},
})
if !strings.Contains(reply, "готово") {
t.Errorf("matcher-sourced act replied %q; want it to have run", reply)
}
}
// TestActMatcherMiss_ProposeGap pins that a matcher miss produces the
// same propose-gap behavior as before.
func TestActMatcherMiss_ProposeGap(t *testing.T) {
h, st := newActHandler(t)
ctx := context.Background()
now := h.now()
// Enable one tool so the matcher has an allowlist, but not the one asked for.
if err := st.EnableTool(ctx, "status", []string{"true"}, false, "test", now); err != nil {
t.Fatal(err)
}
// Act without HasFn and text that doesn't match any tool.
reply := h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Utterance: "deploy the thing",
Slots: router.Slots{Text: "deploy the thing"},
})
if !strings.Contains(strings.ToLower(reply), "предлож") {
t.Errorf("matcher miss replied %q; want propose-gap behavior", reply)
}
}
// TestActDestructive_ConfirmationUnchanged pins that a destructive tool
// still triggers the confirmation flow.
func TestActDestructive_ConfirmationUnchanged(t *testing.T) {
h, st := newActHandler(t)
ctx := context.Background()
now := h.now()
if err := st.EnableTool(ctx, "restart", []string{"true"}, true, "test", now); err != nil {
t.Fatal(err)
}
reply := h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Utterance: "restart",
Slots: router.Slots{Fn: "restart", HasFn: true},
})
if !strings.Contains(reply, "да или нет") {
t.Errorf("destructive act replied %q; want a confirm turn", reply)
}
}
// TestActTaskStatus_InterceptUnchanged pins that task_status is intercepted
// before reaching the tool executor.
func TestActTaskStatus_InterceptUnchanged(t *testing.T) {
h, _ := newActHandler(t)
ctx := context.Background()
reply := h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Utterance: "task status",
Slots: router.Slots{Fn: router.TaskStatusFn, HasFn: true,
Text: "task status"},
})
// task_status is intercepted by resolveTaskStatus, which returns a
// status phrase. The exact reply depends on the store state, but it
// must not be a tool execution result.
if strings.Contains(reply, "готово") {
t.Errorf("task_status was not intercepted, got %q", reply)
}
}
// TestActStage0_SameResult pins that a stage-0 act (grammar match with
// HasFn=true) produces the same tool execution as before.
func TestActStage0_SameResult(t *testing.T) {
h, st := newActHandler(t)
ctx := context.Background()
now := h.now()
if err := st.EnableTool(ctx, "restart", []string{"echo", "ok"}, false, "test", now); err != nil {
t.Fatal(err)
}
reply := h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Stage: 0,
Confidence: 1.0,
Utterance: "maven, restart nginx",
Slots: router.Slots{Fn: "restart", Args: []string{"nginx"}, HasFn: true},
Producer: router.RouteProducerGrammar,
})
if !strings.Contains(reply, "сделала") && !strings.Contains(reply, "готово") {
t.Errorf("stage-0 act replied %q; want it to have run", reply)
}
}
// TestActLearnedRouter_NoFn_FallbackMatch pins that a learned-router act
// without Fn falls through to the matcher and produces the same result.
func TestActLearnedRouter_NoFn_FallbackMatch(t *testing.T) {
h, st := newActHandler(t)
ctx := context.Background()
now := h.now()
if err := st.EnableTool(ctx, "restart", []string{"echo", "ok"}, false, "test", now); err != nil {
t.Fatal(err)
}
// LLM routed the act but did not fill Fn (common when the model returns
// the verb in Text but not in Fn).
reply := h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Stage: 1,
Confidence: 0.85,
Utterance: "could you restart nginx",
Slots: router.Slots{Text: "restart nginx"},
Producer: router.RouteProducerLLM,
})
if !strings.Contains(reply, "сделала") && !strings.Contains(reply, "готово") {
t.Errorf("learned-router act replied %q; want it to have run", reply)
}
}
// TestResolveAction_CandidateSource_Verified pins the candidate source
// for both route-resolved and matcher-resolved actions.
func TestResolveAction_CandidateSource_Verified(t *testing.T) {
h, st := newActHandler(t)
ctx := context.Background()
now := h.now()
if err := st.EnableTool(ctx, "status", []string{"true"}, false, "test", now); err != nil {
t.Fatal(err)
}
// Route-resolved: HasFn=true.
c1 := h.resolveAction(ctx, router.Decision{
Intent: router.IntentAct,
Slots: router.Slots{Fn: "status", HasFn: true},
})
if c1.Source != router.ActionSourceRoute {
t.Errorf("route candidate source = %q, want route", c1.Source)
}
if c1.Fn != "status" {
t.Errorf("route candidate Fn = %q, want status", c1.Fn)
}
// Matcher-resolved: no Fn, text matches.
c2 := h.resolveAction(ctx, router.Decision{
Intent: router.IntentAct,
Slots: router.Slots{Text: "status"},
})
if c2.Source != router.ActionSourceMatcher {
t.Errorf("matcher candidate source = %q, want matcher", c2.Source)
}
if c2.Fn != "status" {
t.Errorf("matcher candidate Fn = %q, want status", c2.Fn)
}
// Matcher miss: no Fn, text doesn't match.
c3 := h.resolveAction(ctx, router.Decision{
Intent: router.IntentAct,
Slots: router.Slots{Text: "deploy everything"},
})
if c3.ActionResolved() {
t.Errorf("miss candidate resolved = true, want false")
}
}
// --- structural validation integration tests ---
// TestActValidation_MalformedCandidate_BlankFn pins that a resolved
// candidate with a blank (whitespace-only) Fn does not execute and
// produces a failure response.
func TestActValidation_MalformedCandidate_BlankFn(t *testing.T) {
h, st := newActHandler(t)
ctx := context.Background()
now := h.now()
if err := st.EnableTool(ctx, "status", []string{"true"}, false, "test", now); err != nil {
t.Fatal(err)
}
// Simulate a malformed candidate by writing a blank Fn into Slots
// after resolution. This tests that the validation layer catches
// structurally invalid candidates.
reply := h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Utterance: "status",
Slots: router.Slots{Fn: " ", HasFn: true},
})
// The blank Fn should not reach tool execution. It either hits
// the validation gate (ActFail) or the existing error paths.
if reply == "" {
t.Error("expected a response, got empty string")
}
}
// TestActValidation_UnresolvedCandidate_ProposeGap pins that an unresolved
// candidate (matcher miss) still flows to proposeGap, unchanged.
func TestActValidation_UnresolvedCandidate_ProposeGap(t *testing.T) {
h, st := newActHandler(t)
ctx := context.Background()
now := h.now()
if err := st.EnableTool(ctx, "status", []string{"true"}, false, "test", now); err != nil {
t.Fatal(err)
}
reply := h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Utterance: "deploy everything",
Slots: router.Slots{Text: "deploy everything"},
})
if !strings.Contains(strings.ToLower(reply), "предлож") {
t.Errorf("unresolved candidate replied %q; want propose-gap behavior", reply)
}
}
// TestActValidation_DestructiveValid_StillConfirms pins that a destructive
// valid action still reaches the confirmation path through validation.
func TestActValidation_DestructiveValid_StillConfirms(t *testing.T) {
h, st := newActHandler(t)
ctx := context.Background()
now := h.now()
if err := st.EnableTool(ctx, "restart", []string{"true"}, true, "test", now); err != nil {
t.Fatal(err)
}
reply := h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Utterance: "restart",
Slots: router.Slots{Fn: "restart", HasFn: true},
})
if !strings.Contains(reply, "да или нет") {
t.Errorf("destructive valid act replied %q; want confirm turn", reply)
}
}
// TestActValidation_IrreversibleValid_NeedsAuthedSurface pins that an
// irreversible valid action still reaches ErrNeedsAuthedSurface.
func TestActValidation_IrreversibleValid_NeedsAuthedSurface(t *testing.T) {
h, st := newActHandler(t)
ctx := context.Background()
now := h.now()
// Register an irreversible tool: cmd containing "drop" triggers the
// irreversible tier via RiskOf → isIrreversible.
if err := st.EnableTool(ctx, "drop_table", []string{"drop"}, true, "test", now); err != nil {
t.Fatal(err)
}
reply := h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Utterance: "drop_table",
Slots: router.Slots{Fn: "drop_table", HasFn: true},
})
// Irreversible tools return ErrNeedsAuthedSurface, which produces
// a specific phraser response.
if !strings.Contains(reply, "выполню") && !strings.Contains(reply, "запусти") {
t.Errorf("irreversible valid act replied %q; want authed-surface response", reply)
}
}
// TestActValidation_ValidationTracing pins that validation outcomes are
// recorded in the decision trace.
func TestActValidation_ValidationTracing(t *testing.T) {
h, st := newActHandler(t)
now := h.now()
// Valid candidate: trace should show action-validation:won.
ctx, rec := decision.With(context.Background(), "status", "tap:text")
if err := st.EnableTool(ctx, "status", []string{"true"}, false, "test", now); err != nil {
t.Fatal(err)
}
h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Utterance: "status",
Slots: router.Slots{Fn: "status", HasFn: true},
})
records := rec.Claims
found := false
for _, c := range records {
if c.Claimant == "action-validation" && c.Outcome == decision.Won {
found = true
break
}
}
if !found {
t.Errorf("expected action-validation:won in trace, got %v", records)
}
}
// TestActExecutionFromCandidateNotSlots pins that downstream execution reads
// resolved action data from ActionCandidate, not from Decision.Slots. The
// decision has empty Fn/Args/HasFn — the bridge used to copy candidate values
// back into these fields. After the bridge removal, execution must still
// succeed because the candidate carries the resolved function.
func TestActExecutionFromCandidateNotSlots(t *testing.T) {
h, st := newActHandler(t)
ctx := context.Background()
now := h.now()
if err := st.EnableTool(ctx, "status", []string{"true"}, false, "test", now); err != nil {
t.Fatal(err)
}
// Act without any Fn/Args/HasFn in Slots — the matcher resolves from Text.
reply := h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Utterance: "check status",
Slots: router.Slots{Text: "status"},
})
if !strings.Contains(reply, "готово") {
t.Errorf("execution from candidate replied %q; want tool success", reply)
}
}
// --- validation status boundary tests ---
// TestActValidation_StatusValidRoute pins that a route-resolved valid action
// produces ActionValid status and reaches execution.
func TestActValidation_StatusValidRoute(t *testing.T) {
h, st := newActHandler(t)
ctx := context.Background()
now := h.now()
if err := st.EnableTool(ctx, "status", []string{"true"}, false, "test", now); err != nil {
t.Fatal(err)
}
reply := h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Utterance: "status",
Slots: router.Slots{Fn: "status", HasFn: true},
})
if !strings.Contains(reply, "готово") {
t.Errorf("valid route act replied %q; want tool success", reply)
}
}
// TestActValidation_StatusValidMatcher pins that a matcher-resolved valid
// action produces ActionValid status and reaches execution.
func TestActValidation_StatusValidMatcher(t *testing.T) {
h, st := newActHandler(t)
ctx := context.Background()
now := h.now()
if err := st.EnableTool(ctx, "status", []string{"true"}, false, "test", now); err != nil {
t.Fatal(err)
}
reply := h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Utterance: "check status",
Slots: router.Slots{Text: "status"},
})
if !strings.Contains(reply, "готово") {
t.Errorf("valid matcher act replied %q; want tool success", reply)
}
}
// TestActValidation_StatusUnresolved pins that an unresolved candidate
// produces ActionUnresolved status and flows to proposeGap.
func TestActValidation_StatusUnresolved(t *testing.T) {
h, st := newActHandler(t)
ctx := context.Background()
now := h.now()
if err := st.EnableTool(ctx, "status", []string{"true"}, false, "test", now); err != nil {
t.Fatal(err)
}
reply := h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Utterance: "deploy everything",
Slots: router.Slots{Text: "deploy everything"},
})
if !strings.Contains(strings.ToLower(reply), "предлож") {
t.Errorf("unresolved act replied %q; want propose-gap", reply)
}
}
// TestActValidation_StatusInvalid pins that a structurally invalid candidate
// produces ActionInvalidArgument status and refuses execution.
func TestActValidation_StatusInvalid(t *testing.T) {
h, st := newActHandler(t)
ctx := context.Background()
now := h.now()
if err := st.EnableTool(ctx, "status", []string{"true"}, false, "test", now); err != nil {
t.Fatal(err)
}
reply := h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Utterance: "status",
Slots: router.Slots{Fn: " ", HasFn: true},
})
if reply == "" {
t.Error("expected a response for invalid candidate")
}
if strings.Contains(reply, "готово") {
t.Error("invalid candidate should not reach tool execution")
}
}
// TestActValidation_DestructiveValidStatus pins that a destructive but
// structurally valid action still produces ActionValid status and reaches
// the confirmation path (not validation failure).
func TestActValidation_DestructiveValidStatus(t *testing.T) {
h, st := newActHandler(t)
ctx := context.Background()
now := h.now()
if err := st.EnableTool(ctx, "restart", []string{"true"}, true, "test", now); err != nil {
t.Fatal(err)
}
reply := h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Utterance: "restart",
Slots: router.Slots{Fn: "restart", HasFn: true},
})
if !strings.Contains(reply, "да или нет") {
t.Errorf("destructive valid act replied %q; want confirm turn", reply)
}
}
// TestActValidation_ConfirmationUnchanged pins that the confirmation flow
// is unchanged by validation.
func TestActValidation_ConfirmationUnchanged(t *testing.T) {
h, st := newActHandler(t)
ctx := context.Background()
now := h.now()
if err := st.EnableTool(ctx, "restart", []string{"echo", "ok"}, true, "test", now); err != nil {
t.Fatal(err)
}
reply := h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Utterance: "restart nginx",
Slots: router.Slots{Fn: "restart", Args: []string{"nginx"}, HasFn: true},
})
if !strings.Contains(reply, "да или нет") {
t.Errorf("confirmation act replied %q; want confirm turn", reply)
}
}
// TestActValidation_TaskStatusInterceptUnchanged pins that task_status
// interception is unchanged by validation.
func TestActValidation_TaskStatusInterceptUnchanged(t *testing.T) {
h, _ := newActHandler(t)
ctx := context.Background()
reply := h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Utterance: "task status",
Slots: router.Slots{Fn: router.TaskStatusFn, HasFn: true, Text: "task status"},
})
if strings.Contains(reply, "готово") {
t.Errorf("task_status was not intercepted, got %q", reply)
}
}
// TestActValidation_NoExecutionOnFailure pins that validation failure
// prevents downstream execution.
func TestActValidation_NoExecutionOnFailure(t *testing.T) {
h, st := newActHandler(t)
ctx := context.Background()
now := h.now()
if err := st.EnableTool(ctx, "status", []string{"true"}, false, "test", now); err != nil {
t.Fatal(err)
}
reply := h.actionAct(ctx, router.Decision{
Intent: router.IntentAct,
Utterance: "status",
Slots: router.Slots{Fn: " ", HasFn: true},
})
if strings.Contains(reply, "готово") {
t.Error("validation failure should not reach tool execution")
}
}
+33 -21
View File
@@ -11,9 +11,14 @@ import (
"github.com/kami/maven/internal/tool"
)
// actionAct handles router.IntentAct: match a verb to an enabled tool, offer
// it to the ecosystems first, and run it behind the confirm gate and the
// allowlist. proposeGap and the confirm gate itself live in confirm.go.
// actionAct handles router.IntentAct: resolve the action, offer it to the
// ecosystems first, and run it behind the confirm gate and the allowlist.
// proposeGap and the confirm gate itself live in confirm.go.
//
// Action resolution happens in resolveAction (actionresolve.go) — a single
// boundary that produces an ActionCandidate before execution. This function
// consumes the candidate; it no longer decides which function/tool the user
// meant.
func (h *reactiveHandler) actionAct(ctx context.Context, dec router.Decision) string {
// An allowlist or a model route is evidence about WHAT could run, never
// authority to run it. Keep the user's negative command at the execution
@@ -23,26 +28,33 @@ func (h *reactiveHandler) actionAct(ctx context.Context, dec router.Decision) st
return commandProhibitionReply
}
// tool executor: run the matched fn against the enabled allowlist.
// HasFn=false ⇒ try the matcher (for LLM-routed acts where the verb
// didn't go through the stage-0 act grammar).
if !dec.Slots.HasFn && dec.Slots.Text != "" && h.matcher != nil {
if fn, args, ok := h.matcher.Match(dec.Slots.Text); ok {
dec.Slots.Fn, dec.Slots.Args, dec.Slots.HasFn = fn, args, true
}
// Resolve the action: produce an ActionCandidate from the routing
// decision. The candidate carries the resolved function, its arguments,
// and where the resolution came from (route or matcher).
candidate := h.resolveAction(ctx, dec)
// Structural validation: is this candidate complete enough to proceed?
// Unresolved (Fn empty) flows to proposeGap; invalid (Fn present but
// malformed) is refused; valid proceeds to execution.
validation := router.ValidateActionCandidate(candidate)
noteActionValidation(ctx, validation)
if !validation.Unresolved() && !validation.Valid() {
// Resolved but structurally malformed: refuse execution.
return phraser.A(phraser.ActFail, nil)
}
// The board is Maven's own store, so a spoken status change is answered here
// and never offered to an ecosystem client (Vikunja #512). First, because
// task_status is on no allowlist and no capability registry: reaching either
// of them would answer a turn about his own task list with a gap.
if dec.Slots.Fn == router.TaskStatusFn {
return h.resolveTaskStatus(ctx, dec)
if candidate.Fn == router.TaskStatusFn {
return h.resolveTaskStatus(ctx, dec, candidate)
}
// Praxis ecosystem tools: intercept before the system command executor.
if h.ecosystem != nil && h.ecosystem.praxis != nil && dec.Slots.HasFn {
if reply := h.handlePraxisAct(ctx, dec); reply != "" {
if h.ecosystem != nil && h.ecosystem.praxis != nil && candidate.ActionResolved() {
if reply := h.handlePraxisAct(ctx, dec, candidate); reply != "" {
return reply
}
}
@@ -50,23 +62,23 @@ func (h *reactiveHandler) actionAct(ctx context.Context, dec router.Decision) st
// Hexis ecosystem action: if ecosystem is configured and we have a verb
// + entity text, try to resolve the entity and execute via Hexis.
if h.ecosystem != nil && h.ecosystem.hexis != nil && router.ActHasEntityTarget(dec) {
if reply := h.handleHexisAct(ctx, dec); reply != "" {
if reply := h.handleHexisAct(ctx, dec, candidate); reply != "" {
return reply
}
}
// HasFn still false ⇒ no allowlist match: scaffold a 'proposed' tool
// Unresolved candidate ⇒ no allowlist match: scaffold a 'proposed' tool
// the user can enable on the authed surface ("earn the right to ask").
if !dec.Slots.HasFn {
if !candidate.ActionResolved() {
return h.proposeGap(ctx, dec)
}
out, err := h.tools.Exec(ctx, dec.Slots.Fn, dec.Slots.Args, false)
out, err := h.tools.Exec(ctx, candidate.Fn, candidate.Args, false)
if err != nil {
switch {
case errors.Is(err, tool.ErrNeedsConfirm):
// destructive: park it and ask. The next utterance answers.
phrase := actPhrase(dec.Slots.Fn, dec.Slots.Args)
h.park(dec.Slots.Fn, dec.Slots.Args, phrase)
phrase := actPhrase(candidate.Fn, candidate.Args)
h.park(candidate.Fn, candidate.Args, phrase)
return phraser.A(phraser.ActConfirm, map[string]string{"name": phrase})
case errors.Is(err, tool.ErrUnknownTarget):
// The verb reached a tool and the tail did not reach a target, so
@@ -101,7 +113,7 @@ func (h *reactiveHandler) actionAct(ctx context.Context, dec router.Decision) st
// where a human types them.
return phraser.A(phraser.ActNeedsArgs, nil)
}
log.Printf("voice: tool %s: %v", dec.Slots.Fn, err)
log.Printf("voice: tool %s: %v", candidate.Fn, err)
if out != "" {
return phraser.A(phraser.ActFailOut, map[string]string{"out": firstLine(out)})
}
+1 -1
View File
@@ -106,7 +106,7 @@ func (h *reactiveHandler) queryTasks(ctx context.Context, t *queryTurn) (string,
// match on more than one asks which, because closing the wrong task is work he
// never finished being marked done. No task named asks which too, since the
// router claims the turn without the referent and the list lives here.
func (h *reactiveHandler) resolveTaskStatus(ctx context.Context, dec router.Decision) string {
func (h *reactiveHandler) resolveTaskStatus(ctx context.Context, dec router.Decision, candidate router.ActionCandidate) string {
live, err := h.api.ListTasks(ctx, "live")
if err != nil {
log.Printf("voice: task status: list: %v", err)
+3 -3
View File
@@ -279,7 +279,7 @@ func TestResolveTaskStatusMovesTheNamedTask(t *testing.T) {
reply := h.resolveTaskStatus(context.Background(), router.Decision{
Intent: router.IntentAct,
Slots: router.Slots{Fn: router.TaskStatusFn, HasFn: true, Value: "done", Text: "молоко"},
})
}, routeCandidate(router.TaskStatusFn))
if api.listArg != "live" {
t.Errorf("listed %q, want live — a resolved task cannot be resolved again", api.listArg)
}
@@ -344,7 +344,7 @@ func TestResolveTaskStatusRefusesToGuess(t *testing.T) {
h := taskHandler(api)
reply := h.resolveTaskStatus(context.Background(), router.Decision{
Slots: router.Slots{Fn: router.TaskStatusFn, HasFn: true, Value: "done", Text: c.named},
})
}, routeCandidate(router.TaskStatusFn))
if len(api.moved) != 0 {
t.Errorf("moved %+v — closing the wrong task is the failure this arm exists to avoid", api.moved)
}
@@ -362,7 +362,7 @@ func TestResolveTaskStatusOpensACandidateFirst(t *testing.T) {
h := taskHandler(api)
h.resolveTaskStatus(context.Background(), router.Decision{
Slots: router.Slots{Fn: router.TaskStatusFn, HasFn: true, Value: "done", Text: "продлить домен"},
})
}, routeCandidate(router.TaskStatusFn))
if len(api.moved) != 2 {
t.Fatalf("moved %+v, want open then done", api.moved)
}
+5 -5
View File
@@ -14,7 +14,7 @@ func TestAttentionEmptyWithHealthySourcesIsAllClear(t *testing.T) {
praxis := newFakePraxisWithSources(t, `[]`, `[{"source_id":"src_ntfy","health":"ok"}]`)
h := newPraxisTestHandler(t, praxis)
reply := h.handlePraxisAct(context.Background(), praxisActDec("list_attention"))
reply := h.handlePraxisAct(context.Background(), praxisActDec("list_attention"), routeCandidate("list_attention"))
if !strings.Contains(reply, "ничего не требует внимания") {
t.Fatalf("healthy and quiet should be an all-clear, got %q", reply)
}
@@ -28,7 +28,7 @@ func TestAttentionEmptyWithAFailedSourceHedges(t *testing.T) {
]`)
h := newPraxisTestHandler(t, praxis)
reply := h.handlePraxisAct(context.Background(), praxisActDec("list_attention"))
reply := h.handlePraxisAct(context.Background(), praxisActDec("list_attention"), routeCandidate("list_attention"))
if strings.Contains(reply, "ничего не требует внимания") {
t.Fatalf("a failed source must not read as all-clear, got %q", reply)
}
@@ -47,7 +47,7 @@ func TestAttentionEmptyWithNoSourcesHedges(t *testing.T) {
praxis := newFakePraxisWithSources(t, `[]`, `[]`)
h := newPraxisTestHandler(t, praxis)
reply := h.handlePraxisAct(context.Background(), praxisActDec("list_attention"))
reply := h.handlePraxisAct(context.Background(), praxisActDec("list_attention"), routeCandidate("list_attention"))
if strings.Contains(reply, "ничего не требует внимания") {
t.Fatalf("a Praxis with no sources must not answer all-clear, got %q", reply)
}
@@ -64,7 +64,7 @@ func TestAttentionDegradedEnvelopeIsReadWithoutASourcesCall(t *testing.T) {
`[{"source_id":"src_ntfy","health":"ok"}]`)
h := newPraxisTestHandler(t, praxis)
reply := h.handlePraxisAct(context.Background(), praxisActDec("list_attention"))
reply := h.handlePraxisAct(context.Background(), praxisActDec("list_attention"), routeCandidate("list_attention"))
if !strings.Contains(reply, "src_metrics") {
t.Fatalf("the envelope's degraded source is not named: %q", reply)
}
@@ -82,7 +82,7 @@ func TestAttentionKeepsAllClearWhenSourcesCannotBeRead(t *testing.T) {
praxis.SetRouteFault("/api/v1/sources", 500)
h := newPraxisTestHandler(t, praxis)
reply := h.handlePraxisAct(context.Background(), praxisActDec("list_attention"))
reply := h.handlePraxisAct(context.Background(), praxisActDec("list_attention"), routeCandidate("list_attention"))
if !strings.Contains(reply, "ничего не требует внимания") {
t.Fatalf("an unreadable sources list should leave the answer alone, got %q", reply)
}
+3
View File
@@ -63,6 +63,9 @@ func (h *reactiveHandler) queryAttention(ctx context.Context, t *queryTurn) (str
Utterance: t.dec.Utterance,
Intent: router.IntentAct,
Slots: router.Slots{Fn: "list_attention", HasFn: true},
}, router.ActionCandidate{
Fn: "list_attention",
Source: router.ActionSourceRoute,
})
if reply == "" {
return "", false
+136
View File
@@ -0,0 +1,136 @@
package main
import (
"context"
"strings"
"testing"
"time"
"github.com/kami/maven/internal/decision"
"github.com/kami/maven/internal/router"
)
// TestTextAndVoiceConvergeOnNormalizedInput — both entry points construct a
// NormalizedInput and pass it to runTurn. The same utterance produces the same
// route intent regardless of whether it arrived as text or voice.
func TestTextAndVoiceConvergeOnNormalizedInput(t *testing.T) {
h, _ := newRoutingClarifyHandler(t)
h.decisions = decision.NewRing()
ctx := context.Background()
utterance := "который час"
voiceCtx := withDialogueID(ctx, dialogueIDFor(sourceVoice, ""))
textCtx := withDialogueID(ctx, dialogueIDFor(sourceText, "test"))
voiceReply := h.runTurn(voiceCtx, router.NormalizedInput{Text: utterance, Source: sourceVoice})
textReply := h.runTurn(textCtx, router.NormalizedInput{Text: utterance, Source: sourceText})
// Both paths should produce the same kind of reply (time answer).
for _, pair := range []struct {
label, reply string
}{
{"voice", voiceReply},
{"text", textReply},
} {
if !strings.Contains(pair.reply, "час") && !strings.Contains(pair.reply, "время") {
t.Errorf("%s reply %q does not look like a time answer", pair.label, pair.reply)
}
}
}
// TestNormalizedInputSourcePreserved — the source survives into the decision
// record so a trace can tell voice from text.
func TestNormalizedInputSourcePreserved(t *testing.T) {
h, _ := newRoutingClarifyHandler(t)
h.decisions = decision.NewRing()
ctx := context.Background()
textCtx := withDialogueID(ctx, dialogueIDFor(sourceText, "test"))
h.runTurn(textCtx, router.NormalizedInput{Text: "привет", Source: sourceText})
recs := h.decisions.Recent(1)
if len(recs) == 0 {
t.Fatal("no decision record")
}
if recs[0].InputSource != string(sourceText) {
t.Errorf("InputSource = %q, want %q", recs[0].InputSource, sourceText)
}
}
// TestRouteProducerOnDecisionRecord — the producer is carried from the router
// decision into the decision record for observability.
func TestRouteProducerOnDecisionRecord(t *testing.T) {
h, _ := newRoutingClarifyHandler(t)
h.decisions = decision.NewRing()
ctx := context.Background()
textCtx := withDialogueID(ctx, dialogueIDFor(sourceText, "test"))
h.runTurn(textCtx, router.NormalizedInput{Text: "который час", Source: sourceText})
recs := h.decisions.Recent(1)
if len(recs) == 0 {
t.Fatal("no decision record")
}
// A time query is a stage-0 grammar match.
if recs[0].RouteProducer != string(router.RouteProducerGrammar) {
t.Errorf("RouteProducer = %q, want %q", recs[0].RouteProducer, router.RouteProducerGrammar)
}
}
// TestPreRouteClaimHasNoRouteProducer — a turn claimed by a pre-route resolver
// never reaches the router, so the record's RouteProducer must be empty.
func TestPreRouteClaimHasNoRouteProducer(t *testing.T) {
h, _ := newRoutingClarifyHandler(t)
h.decisions = decision.NewRing()
// Park a confirm so the next "да" is consumed before routing.
// newRoutingClarifyHandler uses a fixed clock at 2026-07-31 09:00 UTC.
h.pending = &pendingAct{
fn: "test",
phrase: "delete everything",
expiry: time.Date(2026, 7, 31, 9, 1, 0, 0, time.UTC),
}
ctx := context.Background()
textCtx := withDialogueID(ctx, dialogueIDFor(sourceText, "test"))
h.runTurn(textCtx, router.NormalizedInput{Text: "да", Source: sourceText})
recs := h.decisions.Recent(1)
if len(recs) == 0 {
t.Fatal("no decision record")
}
if recs[0].RouteProducer != "" {
t.Errorf("RouteProducer = %q, want empty (pre-route claimed the turn)", recs[0].RouteProducer)
}
}
// TestStage0ProducerUnchanged — grammars still produce the exact same intents
// at confidence 1.0. This pins stage-0 behavior through the new boundary.
func TestStage0ProducerUnchanged(t *testing.T) {
h, _ := newRoutingClarifyHandler(t)
h.decisions = decision.NewRing()
ctx := context.Background()
textCtx := withDialogueID(ctx, dialogueIDFor(sourceText, "test"))
cases := []struct {
utterance string
intent router.Intent
}{
{"напомни позвонить маме завтра", router.IntentReminder},
{"который час", router.IntentSystem},
}
for _, c := range cases {
reply := h.runTurn(textCtx, router.NormalizedInput{Text: c.utterance, Source: sourceText})
_ = reply // behavior unchanged; we test the record, not the reply text.
recs := h.decisions.Recent(1)
if len(recs) == 0 {
t.Errorf("%s: no decision record", c.utterance)
continue
}
rec := recs[0]
if rec.RouteProducer != string(router.RouteProducerGrammar) {
t.Errorf("%s: RouteProducer = %q, want %q", c.utterance, rec.RouteProducer, router.RouteProducerGrammar)
}
// Clear the ring for the next case.
h.decisions = decision.NewRing()
}
}
+1 -1
View File
@@ -439,7 +439,7 @@ func TestClarifySecondGapExhaustionResumesLowerFlow(t *testing.T) {
h.clarifyStore.Push(voiceDialogueID, older)
h.clarifyStore.Push(voiceDialogueID, top)
reply := h.runTurn(ctx, "купить хлеб", sourceText)
reply := h.runTurn(ctx, router.NormalizedInput{Text: "купить хлеб", Source: sourceText})
resumed, _ := clarifyResumedFor(dialogue.SlotTime)
want := withResumed(clarifyGaveUp, resumed)
if reply != want {
+1 -1
View File
@@ -45,5 +45,5 @@ func (h *reactiveHandler) resolveCommandProhibition(ctx context.Context, text st
// The sentinel cannot be renamed into an enabled function, and the original
// utterance remains the authority even when a model rewrites Slots.Text.
func refusesCommand(dec router.Decision) bool {
return dec.Slots.Fn == router.ProhibitedActFn || router.IsCommandProhibition(dec.Utterance)
return dec.CapabilitySelection.Fn == router.ProhibitedActFn || router.IsCommandProhibition(dec.Utterance)
}
+79
View File
@@ -0,0 +1,79 @@
package main
import (
"testing"
"github.com/kami/maven/internal/router"
)
// TestRefusesCommandUsesCapabilitySelection proves that refusesCommand reads
// CapabilitySelection.Fn rather than the compatibility Slots.Fn. When
// CapabilitySelection is populated with the prohibited sentinel but Slots.Fn
// is blank, the refusal must still fire.
func TestRefusesCommandUsesCapabilitySelection(t *testing.T) {
dec := router.Decision{
Intent: router.IntentAct,
CapabilitySelection: router.CapabilitySelection{
Fn: router.ProhibitedActFn,
Resolved: true,
},
// Slots compatibility fields deliberately blank.
}
if !refusesCommand(dec) {
t.Fatal("refusesCommand should return true when CapabilitySelection.Fn == ProhibitedActFn")
}
}
// TestRefusesCommandUtteranceFallback proves that the utterance-based
// prohibition check still works as a defense-in-depth belt when
// CapabilitySelection does not carry the sentinel.
func TestRefusesCommandUtteranceFallback(t *testing.T) {
dec := router.Decision{
Utterance: "don't restart nginx",
}
if !refusesCommand(dec) {
t.Fatal("refusesCommand should return true for a prohibited utterance")
}
}
// TestRefusesCommandNonProhibitedCapability proves that an act with a
// non-prohibited capability is NOT refused even when Slots.Fn happens to
// carry the prohibited sentinel (cross-contamination).
func TestRefusesCommandNonProhibitedCapability(t *testing.T) {
dec := router.Decision{
Intent: router.IntentAct,
Utterance: "restart nginx",
CapabilitySelection: router.CapabilitySelection{
Fn: "restart",
Resolved: true,
},
}
if refusesCommand(dec) {
t.Fatal("refusesCommand should return false for a non-prohibited capability")
}
}
// TestRefusesCommandProhibitedSentinelPreservedThroughPipeline proves that the
// command-prohibition grammar sentinel survives through SelectCapability into
// CapabilitySelection byte-for-byte.
func TestRefusesCommandProhibitedSentinelPreservedThroughPipeline(t *testing.T) {
dec := router.Decision{
Intent: router.IntentAct,
Slots: router.Slots{
Fn: router.ProhibitedActFn,
HasFn: true,
},
}
sel := router.SelectCapability(dec, nil)
dec.CapabilitySelection = sel
if dec.CapabilitySelection.Fn != router.ProhibitedActFn {
t.Errorf("CapabilitySelection.Fn = %q, want %q", dec.CapabilitySelection.Fn, router.ProhibitedActFn)
}
if !dec.CapabilitySelection.Resolved {
t.Error("CapabilitySelection.Resolved should be true")
}
if !refusesCommand(dec) {
t.Fatal("refusesCommand should return true after pipeline preserves the sentinel")
}
}
+15 -6
View File
@@ -107,7 +107,7 @@ var praxisCapabilities = []praxisCapability{
// handlePraxisAct — dispatches ecosystem tool acts through the Praxis tools API.
// Returns "" when the act is not a Praxis verb (the caller falls through to the
// system command executor). Returns a reply string otherwise.
func (h *reactiveHandler) handlePraxisAct(ctx context.Context, dec router.Decision) string {
func (h *reactiveHandler) handlePraxisAct(ctx context.Context, dec router.Decision, candidate router.ActionCandidate) string {
if h.ecosystem == nil || h.ecosystem.praxis == nil {
return ""
}
@@ -127,7 +127,7 @@ func (h *reactiveHandler) handlePraxisAct(ctx context.Context, dec router.Decisi
}
for _, capability := range praxisCapabilities {
for _, alias := range capability.aliases() {
if alias == dec.Slots.Fn {
if alias == candidate.Fn {
return capability.handle(ctx, h, px, dec)
}
}
@@ -657,7 +657,7 @@ func (h *reactiveHandler) resolveEntityCandidates(ctx context.Context, refs []st
// handleHexisAct — resolves entity references through Nexus and executes
// matching capabilities through Hexis. Returns a reply string when handled,
// or "" to fall through to the system command executor.
func (h *reactiveHandler) handleHexisAct(ctx context.Context, dec router.Decision) string {
func (h *reactiveHandler) handleHexisAct(ctx context.Context, dec router.Decision, candidate router.ActionCandidate) string {
// This method is intentionally callable outside runTurn by ecosystem
// harnesses. Refuse before correlation ids, Nexus resolution or capability
// discovery so the no-op sentinel can never leak into Hexis as a verb.
@@ -721,7 +721,7 @@ func (h *reactiveHandler) handleHexisAct(ctx context.Context, dec router.Decisio
// Match the user's verb to a capability by name/description. Collect all
// matches: more than one is itself ambiguous, so we ask rather than pick
// the first (ecosystem invariant: no arbitrary target for mutation).
verb := dec.Slots.Fn
verb := candidate.Fn
if verb == "" {
verb = dec.Slots.Text
}
@@ -732,7 +732,7 @@ func (h *reactiveHandler) handleHexisAct(ctx context.Context, dec router.Decisio
// round then: the phrase is the haystack and the capability name is what we
// look for in it (Vikunja #476). Only when the fn slot is empty — a matched
// fn is a single verb and containment already means what it says.
loose := !dec.Slots.HasFn
loose := !candidate.ActionResolved()
var matches []*hexisclient.Capability
for i, c := range caps {
name := strings.ToLower(c.Name)
@@ -858,7 +858,16 @@ func (h *reactiveHandler) hexisBeforeClarify(ctx context.Context, dec router.Dec
if dec.Intent != router.IntentAct || dec.Slots.HasFn || !router.ActHasEntityTarget(dec) {
return ""
}
return h.handleHexisAct(ctx, dec)
// Resolve the action candidate. Use the matcher when available; when the
// handler has no matcher (ecosystem-only test harnesses), build an
// unresolved candidate directly — the matcher would not have matched either.
var candidate router.ActionCandidate
if h.matcher != nil {
candidate = h.resolveAction(ctx, dec)
} else {
candidate = router.ResolveActionCandidate(dec, nil)
}
return h.handleHexisAct(ctx, dec, candidate)
}
// attentionCannotTell returns the hedge to say instead of an all-clear, or ""
+24 -24
View File
@@ -96,7 +96,7 @@ func TestEcosystem_OutagesLeaveNoSharedFailureState(t *testing.T) {
// A Nexus outage during a Hexis act writes a failure trace, and a shared
// store is the one thing the Praxis path could inherit it through.
nexus.SetFault(503)
if reply := h.handleHexisAct(ctx, actDec("muzick indexer")); actRan(reply) {
if reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart")); actRan(reply) {
t.Fatalf("nexus outage must not report success, got %q", reply)
}
if len(tracesFor(t, h, "nexus", "resolve")) == 0 {
@@ -104,7 +104,7 @@ func TestEcosystem_OutagesLeaveNoSharedFailureState(t *testing.T) {
}
nexus.SetFault(0)
reply := h.handlePraxisAct(ctx, praxisActDec("list_attention"))
reply := h.handlePraxisAct(ctx, praxisActDec("list_attention"), routeCandidate("list_attention"))
if !strings.Contains(reply, "disk almost full") {
t.Fatalf("a recorded nexus failure must not degrade the praxis digest, got %q", reply)
}
@@ -114,10 +114,10 @@ func TestEcosystem_OutagesLeaveNoSharedFailureState(t *testing.T) {
// And the reverse: a Praxis outage mid-session leaves the Hexis path whole.
praxis.SetFault(503)
if reply := h.handlePraxisAct(ctx, praxisActDec("list_attention")); strings.Contains(reply, "disk") {
if reply := h.handlePraxisAct(ctx, praxisActDec("list_attention"), routeCandidate("list_attention")); strings.Contains(reply, "disk") {
t.Fatalf("praxis outage must not serve content, got %q", reply)
}
if reply := h.handleHexisAct(ctx, actDec("muzick indexer")); !actRan(reply) {
if reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart")); !actRan(reply) {
t.Fatalf("a praxis outage must not block the hexis path, got %q", reply)
}
}
@@ -132,7 +132,7 @@ func TestEcosystem_OneEndpointDownDoesNotMuteTheService(t *testing.T) {
h := ecoHandler(t, nil, praxis, nil)
praxis.SetRouteFault("/api/v1/tools/surface", 503)
reply := h.handlePraxisAct(ctx, praxisActDec("list_attention"))
reply := h.handlePraxisAct(ctx, praxisActDec("list_attention"), routeCandidate("list_attention"))
if !strings.Contains(reply, "disk almost full") {
t.Fatalf("a downed surface endpoint must not mute the digest, got %q", reply)
}
@@ -150,7 +150,7 @@ func TestEcosystem_ResolvedWithoutEntityFailsClosed(t *testing.T) {
hexis := newFakeHexis(t, restartCaps(), fixtureHexisExecuted("exec_1", "succeeded"))
h := ecoHandler(t, nexus, nil, hexis)
reply := h.handleHexisAct(ctx, actDec("muzick indexer"))
reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
if reply == "" {
t.Fatal("a resolve with no entity must degrade, not fall through to local execution")
}
@@ -172,7 +172,7 @@ func TestEcosystem_RejectedCredentialSaysSo(t *testing.T) {
h := ecoHandler(t, nexus, nil, hexis)
nexus.SetFault(status)
reply := h.handleHexisAct(ctx, actDec("muzick indexer"))
reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
if !strings.Contains(reply, "токен") {
t.Fatalf("http %d must read as a credential problem, got %q", status, reply)
}
@@ -193,7 +193,7 @@ func TestEcosystem_MalformedPraxisBodyDegrades(t *testing.T) {
h := ecoHandler(t, nil, praxis, nil)
praxis.SetBody(`[{"title":`)
reply := h.handlePraxisAct(ctx, praxisActDec("list_attention"))
reply := h.handlePraxisAct(ctx, praxisActDec("list_attention"), routeCandidate("list_attention"))
if reply == "" {
t.Fatal("a malformed praxis body must not answer with silence")
}
@@ -211,7 +211,7 @@ func TestEcosystem_MalformedNexusResponseFailsClosed(t *testing.T) {
h := ecoHandler(t, nexus, nil, hexis)
nexus.SetBody(`{"status":"resolved","entity":`)
reply := h.handleHexisAct(ctx, actDec("muzick indexer"))
reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
if reply == "" || actRan(reply) {
t.Fatalf("malformed nexus body must degrade, got %q", reply)
}
@@ -232,7 +232,7 @@ func TestEcosystem_UnknownContractFieldsTolerated(t *testing.T) {
nexus := newFakeNexus(t, body)
hexis := newFakeHexis(t, restartCaps(), fixtureHexisExecuted("exec_1", "succeeded"))
h := ecoHandler(t, nexus, nil, hexis)
if reply := h.handleHexisAct(ctx, actDec("muzick indexer")); !actRan(reply) {
if reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart")); !actRan(reply) {
t.Fatalf("%s contract shape must still resolve and execute, got %q", name, reply)
}
})
@@ -249,7 +249,7 @@ func TestEcosystem_CancelledContextDegrades(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Millisecond)
defer cancel()
reply := h.handleHexisAct(ctx, actDec("muzick indexer"))
reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
if reply == "" || actRan(reply) {
t.Fatalf("cancelled resolve must degrade, got %q", reply)
}
@@ -267,7 +267,7 @@ func TestEcosystem_ExecutionFailureIsNotSuccess(t *testing.T) {
hexis := newFakeHexis(t, restartCaps(), fixtureHexisExecutionFailed("exec_1", "unit not found"))
h := ecoHandler(t, nexus, nil, hexis)
reply := h.handleHexisAct(ctx, actDec("muzick indexer"))
reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
if actRan(reply) {
t.Fatalf("failed execution must not read as success, got %q", reply)
}
@@ -291,7 +291,7 @@ func TestEcosystem_SuccessfulActionWritesATrace(t *testing.T) {
hexis := newFakeHexis(t, restartCaps(), fixtureHexisExecuted("exec_1", "succeeded"))
h := ecoHandler(t, nexus, nil, hexis)
if reply := h.handleHexisAct(ctx, actDec("muzick indexer")); !actRan(reply) {
if reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart")); !actRan(reply) {
t.Fatalf("setup: expected success, got %q", reply)
}
exec := tracesFor(t, h, "hexis", "execute")
@@ -313,7 +313,7 @@ func TestEcosystem_TracesStayOutOfFacts(t *testing.T) {
hexis := newFakeHexis(t, restartCaps(), fixtureHexisExecuted("exec_1", "succeeded"))
h := ecoHandler(t, nexus, nil, hexis)
if reply := h.handleHexisAct(ctx, actDec("muzick indexer")); !actRan(reply) {
if reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart")); !actRan(reply) {
t.Fatalf("setup: expected success, got %q", reply)
}
if len(traces(t, h)) == 0 {
@@ -339,7 +339,7 @@ func TestEcosystem_AmbiguousTargetBlocksExecution(t *testing.T) {
hexis := newFakeHexis(t, restartCaps(), fixtureHexisExecuted("exec_1", "succeeded"))
h := ecoHandler(t, nexus, nil, hexis)
reply := h.handleHexisAct(ctx, actDec("muzick"))
reply := h.handleHexisAct(ctx, actDec("muzick"), routeCandidate("restart"))
if !strings.Contains(reply, "Muzick indexer") || !strings.Contains(reply, "Muzick web") {
t.Fatalf("ambiguous resolve must list candidates, got %q", reply)
}
@@ -360,7 +360,7 @@ func TestEcosystem_NoAutonomousPraxisToHexis(t *testing.T) {
hexis := newFakeHexis(t, restartCaps(), fixtureHexisExecuted("exec_1", "succeeded"))
h := ecoHandler(t, nexus, praxis, hexis)
_ = h.handlePraxisAct(ctx, praxisActDec("list_attention"))
_ = h.handlePraxisAct(ctx, praxisActDec("list_attention"), routeCandidate("list_attention"))
if hexis.Count("", "/api/v1") != 0 {
t.Fatal("attention digest must not contact hexis on its own")
}
@@ -378,7 +378,7 @@ func TestEcosystem_MutatingCapabilityWaitsForConfirmation(t *testing.T) {
hexis := newFakeHexis(t, caps, fixtureHexisExecuted("exec_1", "succeeded"))
h := ecoHandler(t, nexus, nil, hexis)
reply := h.handleHexisAct(ctx, actDec("restart"))
reply := h.handleHexisAct(ctx, actDec("restart"), routeCandidate("restart"))
if !strings.Contains(reply, "restart") || !strings.Contains(reply, "да") {
t.Fatalf("mutating capability must ask for confirmation, got %q", reply)
}
@@ -404,7 +404,7 @@ func TestEcosystem_SurfaceFailureStillDelivers(t *testing.T) {
praxis.SetRouteFault("/api/v1/tools/surface", 500)
h := ecoHandler(t, nil, praxis, nil)
reply := h.handlePraxisAct(ctx, praxisActDec("list_attention"))
reply := h.handlePraxisAct(ctx, praxisActDec("list_attention"), routeCandidate("list_attention"))
if !strings.Contains(reply, "disk almost full") {
t.Fatalf("failed surface must not swallow the digest, got %q", reply)
}
@@ -426,10 +426,10 @@ func TestEcosystem_TotalOutageSaysSoForEveryPath(t *testing.T) {
h := ecoHandler(t, nexus, praxis, hexis)
for name, reply := range map[string]string{
"hexis act": h.handleHexisAct(ctx, actDec("muzick indexer")),
"attention": h.handlePraxisAct(ctx, praxisActDec("list_attention")),
"changes": h.handlePraxisAct(ctx, praxisActDec("list_changes")),
"acknowledge": h.handlePraxisAct(ctx, praxisItemDec("acknowledge_item", "item_1")),
"hexis act": h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart")),
"attention": h.handlePraxisAct(ctx, praxisActDec("list_attention"), routeCandidate("list_attention")),
"changes": h.handlePraxisAct(ctx, praxisActDec("list_changes"), routeCandidate("list_changes")),
"acknowledge": h.handlePraxisAct(ctx, praxisItemDec("acknowledge_item", "item_1"), routeCandidate("acknowledge_item")),
} {
if reply == "" {
t.Errorf("%s: total outage must not answer with silence", name)
@@ -458,11 +458,11 @@ func TestEcosystem_RecoveryAfterOutageNeedsNoRestart(t *testing.T) {
h := ecoHandler(t, nil, praxis, nil)
praxis.SetFault(503)
if reply := h.handlePraxisAct(ctx, praxisActDec("list_attention")); strings.Contains(reply, "disk") {
if reply := h.handlePraxisAct(ctx, praxisActDec("list_attention"), routeCandidate("list_attention")); strings.Contains(reply, "disk") {
t.Fatalf("outage must not serve content, got %q", reply)
}
praxis.SetFault(0)
if reply := h.handlePraxisAct(ctx, praxisActDec("list_attention")); !strings.Contains(reply, "disk almost full") {
if reply := h.handlePraxisAct(ctx, praxisActDec("list_attention"), routeCandidate("list_attention")); !strings.Contains(reply, "disk almost full") {
t.Fatalf("recovery must work on the next turn, got %q", reply)
}
}
+5 -5
View File
@@ -59,7 +59,7 @@ func TestHexisDiscovery401IsDeniedNotDown(t *testing.T) {
h := hexisGapHandler(t, nexus.URL, hexis.URL)
hexis.SetFault(401)
reply := h.handleHexisAct(ctx, actDec("muzick indexer"))
reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
if !denied(serviceHexis, reply) {
t.Fatalf("401 from hexis discovery: got %q, want the denied line naming Hexis", reply)
}
@@ -75,7 +75,7 @@ func TestHexisDiscoveryOutageIsDownNotDenied(t *testing.T) {
nexus := newFakeNexus(t, fixtureNexusResolved("ent_muzick", muzickIndexer, "service"))
h := hexisGapHandler(t, nexus.URL, unreachableURL)
reply := h.handleHexisAct(ctx, actDec("muzick indexer"))
reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
if !down(serviceHexis, reply) {
t.Fatalf("connection refused from hexis: got %q, want the outage line naming Hexis", reply)
}
@@ -96,7 +96,7 @@ func TestHexisExecute401IsDeniedNotCommandFailure(t *testing.T) {
// Discovery stays healthy; only the execute endpoint refuses. A blanket
// fault would never reach the site under test.
hexis.SetRouteFault("/api/v1/execute", 401)
reply := h.handleHexisAct(ctx, actDec("muzick indexer"))
reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
if !denied(serviceHexis, reply) {
t.Fatalf("401 from hexis execute: got %q, want the denied line naming Hexis", reply)
}
@@ -129,7 +129,7 @@ func TestHexisExecuteOutageIsDown(t *testing.T) {
t.Cleanup(hexis.Close)
h := hexisGapHandler(t, nexus.URL, hexis.URL)
reply := h.handleHexisAct(ctx, actDec("muzick indexer"))
reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
if !down(serviceHexis, reply) {
t.Fatalf("dropped connection on hexis execute: got %q, want the outage line", reply)
}
@@ -149,7 +149,7 @@ func TestHexisExecutionFailedStaysCommandFailure(t *testing.T) {
hexis := newFakeHexis(t, caps, fixtureHexisExecutionFailed("exec_1", "unit refused to start"))
h := hexisGapHandler(t, nexus.URL, hexis.URL)
reply := h.handleHexisAct(ctx, actDec("muzick indexer"))
reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
if down(serviceHexis, reply) || denied(serviceHexis, reply) {
t.Fatalf("a failed execution must not be reported as an ecosystem gap, got %q", reply)
}
+12 -7
View File
@@ -19,6 +19,11 @@ func praxisActDec(fn string) router.Decision {
return router.Decision{Intent: router.IntentAct, Slots: router.Slots{Fn: fn, HasFn: true}}
}
// routeCandidate builds an ActionCandidate matching a route-resolved Decision.
func routeCandidate(fn string) router.ActionCandidate {
return router.ActionCandidate{Fn: fn, Source: router.ActionSourceRoute}
}
// praxisItemDec is praxisActDec for the lifecycle verbs, which need an item id
// in the value slot. Without one they answer "which item?" and never reach
// Praxis at all, which makes them useless for testing a Praxis outage.
@@ -46,7 +51,7 @@ func TestPraxisAttention_HappyPathSurfacesItems(t *testing.T) {
praxis := newFakePraxis(t, items)
h := newPraxisTestHandler(t, praxis)
reply := h.handlePraxisAct(ctx, praxisActDec("list_attention"))
reply := h.handlePraxisAct(ctx, praxisActDec("list_attention"), routeCandidate("list_attention"))
if !strings.Contains(reply, "disk almost full") {
t.Fatalf("expected attention digest to mention the item, got %q", reply)
}
@@ -77,7 +82,7 @@ func TestPraxisAttention_DegradedFailsClosedNotEmpty(t *testing.T) {
praxis.SetFault(500)
h := newPraxisTestHandler(t, praxis)
reply := h.handlePraxisAct(ctx, praxisActDec("list_attention"))
reply := h.handlePraxisAct(ctx, praxisActDec("list_attention"), routeCandidate("list_attention"))
if reply == "" {
t.Fatal("praxis outage must not produce an empty reply")
}
@@ -104,13 +109,13 @@ func TestFakeNexus_FaultInjectionThenRecovery(t *testing.T) {
}
nexus.SetFault(503)
reply := h.handleHexisAct(ctx, actDec("muzick indexer"))
reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
if actRan(reply) {
t.Fatalf("nexus outage must not report success, got %q", reply)
}
nexus.SetFault(0)
reply = h.handleHexisAct(ctx, actDec("muzick indexer"))
reply = h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
if !actRan(reply) {
t.Fatalf("expected success once nexus recovers, got %q", reply)
}
@@ -134,7 +139,7 @@ func TestPraxisEntityAttention_RemembersWhatItReadOut(t *testing.T) {
reply := h.handlePraxisAct(ctx, router.Decision{
Intent: router.IntentAct,
Slots: router.Slots{Fn: "entity_attention", HasFn: true, Value: "muzick indexer"},
})
}, routeCandidate("entity_attention"))
if !strings.Contains(reply, "indexer wedged") {
t.Fatalf("expected the scoped item to be read out, got %q", reply)
}
@@ -147,7 +152,7 @@ func TestPraxisEntityAttention_RemembersWhatItReadOut(t *testing.T) {
}
// The follow-up resolves against what he just heard, not the stale list.
if reply := h.handlePraxisAct(ctx, praxisItemDec("resolve_item", "last")); reply == "" {
if reply := h.handlePraxisAct(ctx, praxisItemDec("resolve_item", "last"), routeCandidate("resolve_item")); reply == "" {
t.Fatal("positional follow-up should have been claimed by praxis")
}
var body string
@@ -175,7 +180,7 @@ func TestHexisConfirm_KeepsOneCorrelationIDPerAction(t *testing.T) {
hexis := newFakeHexis(t, caps, fixtureHexisExecuted("exec_1", "succeeded"))
h := ecoHandler(t, nexus, nil, hexis)
if reply := h.handleHexisAct(ctx, actDec("restart")); !strings.Contains(reply, "да") {
if reply := h.handleHexisAct(ctx, actDec("restart"), routeCandidate("restart")); !strings.Contains(reply, "да") {
t.Fatalf("mutating capability must ask for confirmation, got %q", reply)
}
resolve := findTrace(t, h, "nexus", "resolve")
+11 -11
View File
@@ -73,7 +73,7 @@ func TestHexisMutatingRequiresConfirm(t *testing.T) {
caps := `[{"id":"cap_restart","name":"restart","read_only":false,"risk":"high"}]`
h, executed := newHexisTestHandler(t, resolved, caps)
reply := h.handleHexisAct(ctx, actDec("muzick indexer"))
reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
if !strings.Contains(reply, "да") {
t.Fatalf("mutating cap should ask to confirm, got %q", reply)
}
@@ -103,7 +103,7 @@ func TestHexisConfirmNoDoesNotExecute(t *testing.T) {
caps := `[{"id":"cap_restart","name":"restart","read_only":false}]`
h, executed := newHexisTestHandler(t, resolved, caps)
_ = h.handleHexisAct(ctx, actDec("muzick indexer"))
_ = h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
reply, handled := h.resolveConfirm(ctx, "нет")
if !handled || !strings.Contains(reply, "отменила") {
t.Fatalf("no should cancel, got handled=%v reply=%q", handled, reply)
@@ -119,7 +119,7 @@ func TestHexisReadOnlyExecutesImmediately(t *testing.T) {
caps := `[{"id":"cap_status","name":"restart","read_only":true}]`
h, executed := newHexisTestHandler(t, resolved, caps)
reply := h.handleHexisAct(ctx, actDec("muzick indexer"))
reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
if !*executed {
t.Fatal("read-only cap should execute without confirmation")
}
@@ -136,7 +136,7 @@ func TestHexisAmbiguousAsksClarification(t *testing.T) {
ambiguous := `{"status":"ambiguous","candidates":[{"entity_id":"ent_muzick","display_name":"Muzick indexer"},{"entity_id":"ent_manga","display_name":"Manga indexer"}]}`
h, executed := newHexisTestHandler(t, ambiguous, `[]`)
reply := h.handleHexisAct(ctx, actDec("the indexer"))
reply := h.handleHexisAct(ctx, actDec("the indexer"), routeCandidate("restart"))
if !strings.Contains(reply, "Muzick indexer") || !strings.Contains(reply, "Manga indexer") {
t.Fatalf("ambiguous should list candidates, got %q", reply)
}
@@ -154,7 +154,7 @@ func TestHexisResolveFlatShapeAccepted(t *testing.T) {
caps := `[{"id":"cap_status","name":"restart","read_only":true}]`
h, executed := newHexisTestHandler(t, flat, caps)
reply := h.handleHexisAct(ctx, actDec("muzick indexer"))
reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
if !*executed {
t.Fatalf("flat-shaped resolved entity should still execute, got reply %q", reply)
}
@@ -183,7 +183,7 @@ func TestHexisNexusErrorFailsClosed(t *testing.T) {
ecosystem: stubEcosystem(nexus.URL, hexis.URL),
}
reply := h.handleHexisAct(ctx, actDec("muzick indexer"))
reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
if reply == "" {
t.Fatal("nexus dependency failure must not fall through with an empty reply")
}
@@ -216,7 +216,7 @@ func TestHexisUnavailableFailsClosed(t *testing.T) {
ecosystem: stubEcosystem(nexus.URL, hexis.URL),
}
reply := h.handleHexisAct(ctx, actDec("muzick indexer"))
reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
if reply == "" {
t.Fatal("hexis dependency failure must not fall through with an empty reply")
}
@@ -234,7 +234,7 @@ func TestHexisNotFoundStillFallsThrough(t *testing.T) {
notFound := `{"status":"not_found"}`
h, executed := newHexisTestHandler(t, notFound, `[]`)
reply := h.handleHexisAct(ctx, actDec("turn off the lights"))
reply := h.handleHexisAct(ctx, actDec("turn off the lights"), routeCandidate("restart"))
if reply != "" {
t.Fatalf("not_found resolution should fall through with empty reply, got %q", reply)
}
@@ -264,7 +264,7 @@ func TestHexisIrreversibleCapabilityIsNotRunFromVoice(t *testing.T) {
caps := `[{"id":"cap_wipe","name":"restart","read_only":false,"risk":"irreversible","requires_confirmation":true}]`
h, executed := newHexisTestHandler(t, resolved, caps)
reply := h.handleHexisAct(ctx, actDec("muzick indexer"))
reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
if *executed {
t.Fatal("an irreversible capability ran from the voice path")
}
@@ -284,7 +284,7 @@ func TestHexisSafeCapabilityRunsOnItsDeclaredTier(t *testing.T) {
caps := `[{"id":"cap_status","name":"restart","read_only":true,"risk":"safe"}]`
h, executed := newHexisTestHandler(t, resolved, caps)
reply := h.handleHexisAct(ctx, actDec("muzick indexer"))
reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
if !*executed {
t.Fatal("a capability Hexis calls safe should run")
}
@@ -301,7 +301,7 @@ func TestHexisUndeclaredTierStillConfirms(t *testing.T) {
caps := `[{"id":"cap_restart","name":"restart","read_only":false}]`
h, executed := newHexisTestHandler(t, resolved, caps)
reply := h.handleHexisAct(ctx, actDec("muzick indexer"))
reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
if *executed {
t.Fatal("a mutating capability ran without a confirm")
}
+7 -7
View File
@@ -142,7 +142,7 @@ func TestEcosystemTrace_SuccessfulActionTracesEveryHop(t *testing.T) {
hexis := newFakeHexis(t, restartCaps(), fixtureHexisExecuted("exec_1", "succeeded"))
h := ecoHandler(t, nexus, nil, hexis)
if reply := h.handleHexisAct(ctx, actDec("muzick indexer")); !actRan(reply) {
if reply := h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart")); !actRan(reply) {
t.Fatalf("setup: expected success, got %q", reply)
}
@@ -186,7 +186,7 @@ func TestEcosystemTrace_OneCorrelationIDPerPraxisAction(t *testing.T) {
))
h := ecoHandler(t, nil, praxis, nil)
if reply := h.handlePraxisAct(ctx, praxisActDec("list_attention")); !strings.Contains(reply, "disk almost full") {
if reply := h.handlePraxisAct(ctx, praxisActDec("list_attention"), routeCandidate("list_attention")); !strings.Contains(reply, "disk almost full") {
t.Fatalf("setup: expected the digest, got %q", reply)
}
@@ -218,7 +218,7 @@ func TestEcosystemTrace_FailuresAreTracedToo(t *testing.T) {
h := ecoHandler(t, nexus, nil, hexis)
nexus.SetFault(401)
_ = h.handleHexisAct(ctx, actDec("muzick indexer"))
_ = h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
d := findTrace(t, h, "nexus", "resolve")
if d == nil {
@@ -242,7 +242,7 @@ func TestEcosystemTrace_UnreachableIsNotRefused(t *testing.T) {
h := ecoHandler(t, nil, nil, nil)
h.ecosystem.nexus = newNexusClient("http://127.0.0.1:1")
_ = h.handleHexisAct(ctx, actDec("muzick indexer"))
_ = h.handleHexisAct(ctx, actDec("muzick indexer"), routeCandidate("restart"))
d := findTrace(t, h, "nexus", "resolve")
if d == nil {
@@ -263,7 +263,7 @@ func TestEcosystemTrace_RedactsTheUtterance(t *testing.T) {
nexus := newFakeNexus(t, fixtureNexusNotFound())
h := ecoHandler(t, nexus, nil, nil)
_ = h.handleHexisAct(ctx, actDec("перезапусти кофемашину"))
_ = h.handleHexisAct(ctx, actDec("перезапусти кофемашину"), routeCandidate("restart"))
recorded := traces(t, h)
if len(recorded) == 0 {
@@ -295,7 +295,7 @@ func TestEcosystemTrace_AmbiguityAndConfirmationAreRecorded(t *testing.T) {
))
hexis := newFakeHexis(t, restartCaps(), fixtureHexisExecuted("exec_1", "succeeded"))
h := ecoHandler(t, ambig, nil, hexis)
_ = h.handleHexisAct(ctx, actDec("muzick"))
_ = h.handleHexisAct(ctx, actDec("muzick"), routeCandidate("restart"))
if d := findTrace(t, h, "nexus", "resolve"); d == nil || d.Status != traceAmbig {
t.Fatalf("ambiguous resolve must be traced as such, got %+v", d)
}
@@ -303,7 +303,7 @@ func TestEcosystemTrace_AmbiguityAndConfirmationAreRecorded(t *testing.T) {
nexus := newFakeNexus(t, fixtureNexusResolved("ent_muzick", "Muzick indexer", "service"))
mutating := fixtureHexisCapabilities(map[string]any{"id": "cap_restart", "name": "restart", "read_only": false})
h2 := ecoHandler(t, nexus, nil, newFakeHexis(t, mutating, fixtureHexisExecuted("exec_1", "succeeded")))
_ = h2.handleHexisAct(ctx, actDec("restart"))
_ = h2.handleHexisAct(ctx, actDec("restart"), routeCandidate("restart"))
d := findTrace(t, h2, "hexis", "confirmation")
if d == nil || d.Status != tracePending {
t.Fatalf("a parked confirmation must be traced, got %+v", d)
+3 -3
View File
@@ -91,7 +91,7 @@ func TestNexusIsAskedForTheNameHeSaid(t *testing.T) {
Intent: router.IntentAct,
Slots: router.Slots{Text: "перезагрузить музик индексер", Fn: "restart", HasFn: true},
}
h.handleHexisAct(ctx, dec)
h.handleHexisAct(ctx, dec, routeCandidate("restart"))
reqs := nexus.Requests()
if len(reqs) == 0 {
@@ -226,7 +226,7 @@ func TestTwoResolvedNamesAsk(t *testing.T) {
Intent: router.IntentAct,
Slots: router.Slots{Text: "перезагрузить нгинкс", Fn: "restart", HasFn: true},
}
reply := h.handleHexisAct(ctx, dec)
reply := h.handleHexisAct(ctx, dec, routeCandidate("restart"))
if !strings.Contains(reply, "nginx") || !strings.Contains(reply, "Muzick indexer") {
t.Fatalf("reply = %q, want both names she found", reply)
}
@@ -251,7 +251,7 @@ func TestTheNameNexusKnowsWins(t *testing.T) {
Intent: router.IntentAct,
Slots: router.Slots{Text: "перезагрузить нгинкс", Fn: "restart", HasFn: true},
}
reply := h.handleHexisAct(ctx, dec)
reply := h.handleHexisAct(ctx, dec, routeCandidate("restart"))
if reply == "" {
t.Fatal("the resolvable name must carry the act")
}
+10 -10
View File
@@ -33,7 +33,7 @@ func TestEntityAttention_ScopesPraxisByCanonicalID(t *testing.T) {
))
h := ecoHandler(t, nexus, praxis, nil)
reply := h.handlePraxisAct(ctx, entityAttentionDec("muzick indexer"))
reply := h.handlePraxisAct(ctx, entityAttentionDec("muzick indexer"), routeCandidate("entity_attention"))
if !strings.Contains(reply, "indexer queue is backing up") {
t.Fatalf("expected the scoped item in the reply, got %q", reply)
}
@@ -70,7 +70,7 @@ func TestEntityAttention_FoldsInLocalFactsForSameEntity(t *testing.T) {
t.Fatalf("ResolveFactEntity: %v", err)
}
reply := h.handlePraxisAct(ctx, entityAttentionDec("the espresso machine"))
reply := h.handlePraxisAct(ctx, entityAttentionDec("the espresso machine"), routeCandidate("entity_attention"))
if !strings.Contains(reply, "descaled in june") {
t.Fatalf("expected entity-scoped local facts in the reply, got %q", reply)
}
@@ -88,7 +88,7 @@ func TestEntityAttention_UnscopedPraxisResponseIsRefused(t *testing.T) {
))
h := ecoHandler(t, nexus, praxis, nil)
reply := h.handlePraxisAct(ctx, entityAttentionDec("muzick indexer"))
reply := h.handlePraxisAct(ctx, entityAttentionDec("muzick indexer"), routeCandidate("entity_attention"))
if strings.Contains(reply, "disk almost full") {
t.Fatalf("an unscoped response must not be read back as entity-scoped, got %q", reply)
}
@@ -112,7 +112,7 @@ func TestEntityAttention_ForeignItemsAreDropped(t *testing.T) {
praxis := newFakePraxis(t, mustJSON(mixed))
h := ecoHandler(t, nexus, praxis, nil)
reply := h.handlePraxisAct(ctx, entityAttentionDec("muzick indexer"))
reply := h.handlePraxisAct(ctx, entityAttentionDec("muzick indexer"), routeCandidate("entity_attention"))
if !strings.Contains(reply, "indexer queue is backing up") {
t.Fatalf("the matching item must be spoken, got %q", reply)
}
@@ -140,7 +140,7 @@ func TestEntityAttention_TruncationIsNamed(t *testing.T) {
}
}
reply := h.handlePraxisAct(ctx, entityAttentionDec("the espresso machine"))
reply := h.handlePraxisAct(ctx, entityAttentionDec("the espresso machine"), routeCandidate("entity_attention"))
if !strings.Contains(reply, "и это не всё") {
t.Fatalf("a truncated recall must say it is truncated, got %q", reply)
}
@@ -156,7 +156,7 @@ func TestEntityAttention_AmbiguousAsksInsteadOfGuessing(t *testing.T) {
praxis := newFakePraxis(t, fixturePraxisAttentionItems())
h := ecoHandler(t, nexus, praxis, nil)
reply := h.handlePraxisAct(ctx, entityAttentionDec("muzick"))
reply := h.handlePraxisAct(ctx, entityAttentionDec("muzick"), routeCandidate("entity_attention"))
if !strings.Contains(reply, "Muzick indexer") || !strings.Contains(reply, "Muzick web") {
t.Fatalf("ambiguous subject must ask, got %q", reply)
}
@@ -173,13 +173,13 @@ func TestEntityAttention_MissingAndDegradedAreDistinct(t *testing.T) {
praxis := newFakePraxis(t, fixturePraxisAttentionItems())
h := ecoHandler(t, nexus, praxis, nil)
missing := h.handlePraxisAct(ctx, entityAttentionDec("нечто"))
missing := h.handlePraxisAct(ctx, entityAttentionDec("нечто"), routeCandidate("entity_attention"))
if missing == "" {
t.Fatal("an unknown entity must still get an answer")
}
nexus.SetFault(503)
degraded := h.handlePraxisAct(ctx, entityAttentionDec("нечто"))
degraded := h.handlePraxisAct(ctx, entityAttentionDec("нечто"), routeCandidate("entity_attention"))
if degraded == missing {
t.Fatalf("outage and unknown-entity must not read the same: %q", degraded)
}
@@ -195,7 +195,7 @@ func TestEntityAttention_DelayedNexusDegradesNotHangs(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Millisecond)
defer cancel()
reply := h.handlePraxisAct(ctx, entityAttentionDec("muzick indexer"))
reply := h.handlePraxisAct(ctx, entityAttentionDec("muzick indexer"), routeCandidate("entity_attention"))
if reply == "" {
t.Fatal("a delayed resolve must still answer")
}
@@ -213,7 +213,7 @@ func TestEntityAttention_WithoutNexusSaysSo(t *testing.T) {
))
h := ecoHandler(t, nil, praxis, nil)
reply := h.handlePraxisAct(ctx, entityAttentionDec("muzick indexer"))
reply := h.handlePraxisAct(ctx, entityAttentionDec("muzick indexer"), routeCandidate("entity_attention"))
if strings.Contains(reply, "disk almost full") {
t.Fatalf("without nexus, items must not be passed off as entity-scoped, got %q", reply)
}
+11 -9
View File
@@ -72,15 +72,16 @@ func dialogueIDOf(ctx context.Context) string {
// toDialogueSlots projects the router's slots onto the dialogue layer's copy.
func toDialogueSlots(s router.Slots) dialogue.Slots {
return dialogue.Slots{
Time: s.Time,
HasTime: s.HasTime,
Key: s.Key,
Value: s.Value,
HasKey: s.HasKey,
Text: s.Text,
Fn: s.Fn,
Args: s.Args,
HasFn: s.HasFn,
Time: s.Time,
HasTime: s.HasTime,
Key: s.Key,
Value: s.Value,
HasKey: s.HasKey,
Text: s.Text,
Fn: s.Fn,
Args: s.Args,
HasFn: s.HasFn,
ResolvedBy: string(s.ResolvedBy),
}
}
@@ -90,6 +91,7 @@ func applyDialogueSlots(base router.Slots, d dialogue.Slots) router.Slots {
base.Key, base.Value, base.HasKey = d.Key, d.Value, d.HasKey
base.Text = d.Text
base.Fn, base.Args, base.HasFn = d.Fn, d.Args, d.HasFn
base.ResolvedBy = router.ActionResolutionMethod(d.ResolvedBy)
return base
}
+3 -3
View File
@@ -19,7 +19,7 @@ func TestPraxisLifecycle401NamesPraxis(t *testing.T) {
h := newPraxisTestHandler(t, praxis)
praxis.SetFault(401)
reply := h.handlePraxisAct(ctx, praxisItemDec("resolve_item", "item_1"))
reply := h.handlePraxisAct(ctx, praxisItemDec("resolve_item", "item_1"), routeCandidate("resolve_item"))
if !strings.Contains(reply, servicePraxis) {
t.Fatalf("praxis failure does not name Praxis: %q", reply)
}
@@ -40,10 +40,10 @@ func TestPraxisLifecycleOutageDiffersFrom401(t *testing.T) {
h := newPraxisTestHandler(t, praxis)
praxis.SetFault(401)
refused := h.handlePraxisAct(ctx, praxisItemDec("acknowledge_item", "item_1"))
refused := h.handlePraxisAct(ctx, praxisItemDec("acknowledge_item", "item_1"), routeCandidate("acknowledge_item"))
h.ecosystem = &ecosystemWiring{praxis: newPraxisClient(unreachableURL)}
outage := h.handlePraxisAct(ctx, praxisItemDec("acknowledge_item", "item_1"))
outage := h.handlePraxisAct(ctx, praxisItemDec("acknowledge_item", "item_1"), routeCandidate("acknowledge_item"))
if refused == outage {
t.Fatalf("a refused token and an outage still say the same thing: %q", refused)
+14 -14
View File
@@ -17,7 +17,7 @@ func TestPositionResolvesAgainstTheLastSpokenList(t *testing.T) {
]`)
h := newPraxisTestHandler(t, praxis)
if reply := h.handlePraxisAct(context.Background(), praxisActDec("list_attention")); reply == "" {
if reply := h.handlePraxisAct(context.Background(), praxisActDec("list_attention"), routeCandidate("list_attention")); reply == "" {
t.Fatal("attention returned nothing")
}
@@ -28,7 +28,7 @@ func TestPositionResolvesAgainstTheLastSpokenList(t *testing.T) {
}
for _, c := range cases {
praxis.ResetRequests()
reply := h.handlePraxisAct(context.Background(), praxisItemDec("acknowledge_item", c.ref))
reply := h.handlePraxisAct(context.Background(), praxisItemDec("acknowledge_item", c.ref), routeCandidate("acknowledge_item"))
if !strings.Contains(reply, "принято") {
t.Errorf("ref %q: reply %q", c.ref, reply)
}
@@ -42,10 +42,10 @@ func TestPositionResolvesAgainstTheLastSpokenList(t *testing.T) {
func TestPositionPastTheEndAsksInsteadOfGuessing(t *testing.T) {
praxis := newFakePraxis(t, `[{"id":"item_a","title":"диск заканчивается"}]`)
h := newPraxisTestHandler(t, praxis)
h.handlePraxisAct(context.Background(), praxisActDec("list_attention"))
h.handlePraxisAct(context.Background(), praxisActDec("list_attention"), routeCandidate("list_attention"))
praxis.ResetRequests()
reply := h.handlePraxisAct(context.Background(), praxisItemDec("resolve_item", "4"))
reply := h.handlePraxisAct(context.Background(), praxisItemDec("resolve_item", "4"), routeCandidate("resolve_item"))
if !strings.Contains(reply, "какой пункт") {
t.Errorf("a position with no item should ask, got %q", reply)
}
@@ -59,7 +59,7 @@ func TestPositionWithNoSpokenListAsks(t *testing.T) {
praxis := newFakePraxis(t, `[]`)
h := newPraxisTestHandler(t, praxis)
reply := h.handlePraxisAct(context.Background(), praxisItemDec("acknowledge_item", "1"))
reply := h.handlePraxisAct(context.Background(), praxisItemDec("acknowledge_item", "1"), routeCandidate("acknowledge_item"))
if !strings.Contains(reply, "какой пункт") {
t.Errorf("want the ask, got %q", reply)
}
@@ -69,10 +69,10 @@ func TestPositionWithNoSpokenListAsks(t *testing.T) {
func TestExplicitItemIDIsNotRewritten(t *testing.T) {
praxis := newFakePraxis(t, `[{"id":"item_a","title":"диск"}]`)
h := newPraxisTestHandler(t, praxis)
h.handlePraxisAct(context.Background(), praxisActDec("list_attention"))
h.handlePraxisAct(context.Background(), praxisActDec("list_attention"), routeCandidate("list_attention"))
praxis.ResetRequests()
h.handlePraxisAct(context.Background(), praxisItemDec("pin_item", "item_zz"))
h.handlePraxisAct(context.Background(), praxisItemDec("pin_item", "item_zz"), routeCandidate("pin_item"))
if !requestedPathContaining(praxis, "item_zz") {
t.Errorf("the id he gave was not the one called; paths %v", paths(praxis))
}
@@ -85,10 +85,10 @@ func TestUnspokenItemsHoldNoPosition(t *testing.T) {
{"id":"item_said","title":"бэкап не прошёл"}
]`)
h := newPraxisTestHandler(t, praxis)
h.handlePraxisAct(context.Background(), praxisActDec("list_attention"))
h.handlePraxisAct(context.Background(), praxisActDec("list_attention"), routeCandidate("list_attention"))
praxis.ResetRequests()
h.handlePraxisAct(context.Background(), praxisItemDec("acknowledge_item", "1"))
h.handlePraxisAct(context.Background(), praxisItemDec("acknowledge_item", "1"), routeCandidate("acknowledge_item"))
if !requestedPathContaining(praxis, "item_said") {
t.Errorf("position 1 is the first item she SAID; paths %v", paths(praxis))
}
@@ -116,10 +116,10 @@ func requestedPathContaining(f *fakeServer, want string) bool {
func TestDemonstrativeResolvesWhenOneItemWasSpoken(t *testing.T) {
praxis := newFakePraxis(t, `[{"id":"item_only","title":"бэкап не прошёл"}]`)
h := newPraxisTestHandler(t, praxis)
h.handlePraxisAct(context.Background(), praxisActDec("list_attention"))
h.handlePraxisAct(context.Background(), praxisActDec("list_attention"), routeCandidate("list_attention"))
praxis.ResetRequests()
reply := h.handlePraxisAct(context.Background(), praxisItemDec("acknowledge_item", "this"))
reply := h.handlePraxisAct(context.Background(), praxisItemDec("acknowledge_item", "this"), routeCandidate("acknowledge_item"))
if !strings.Contains(reply, "принято") {
t.Errorf("reply %q", reply)
}
@@ -136,10 +136,10 @@ func TestDemonstrativeWithSeveralItemsGivesTheTurnBack(t *testing.T) {
{"id":"item_b","title":"бэкап"}
]`)
h := newPraxisTestHandler(t, praxis)
h.handlePraxisAct(context.Background(), praxisActDec("list_attention"))
h.handlePraxisAct(context.Background(), praxisActDec("list_attention"), routeCandidate("list_attention"))
praxis.ResetRequests()
if reply := h.handlePraxisAct(context.Background(), praxisItemDec("resolve_item", "this")); reply != "" {
if reply := h.handlePraxisAct(context.Background(), praxisItemDec("resolve_item", "this"), routeCandidate("resolve_item")); reply != "" {
t.Errorf("want a fall-through, got %q", reply)
}
for _, p := range paths(praxis) {
@@ -154,7 +154,7 @@ func TestDemonstrativeWithNoDigestGivesTheTurnBack(t *testing.T) {
praxis := newFakePraxis(t, `[]`)
h := newPraxisTestHandler(t, praxis)
if reply := h.handlePraxisAct(context.Background(), praxisItemDec("resolve_item", "this")); reply != "" {
if reply := h.handlePraxisAct(context.Background(), praxisItemDec("resolve_item", "this"), routeCandidate("resolve_item")); reply != "" {
t.Errorf("want a fall-through, got %q", reply)
}
}
+1 -1
View File
@@ -126,7 +126,7 @@ func TestRunTurnExplicitNoteStoresOnlyTheBody(t *testing.T) {
}
const utterance = "запомни: запасной ключ лежит в синей коробке"
if reply := h.runTurn(ctx, utterance, sourceText); reply != "сохранила заметку." {
if reply := h.runTurn(ctx, router.NormalizedInput{Text: utterance, Source: sourceText}); reply != "сохранила заметку." {
t.Fatalf("reply = %q, want the fixed feminine acknowledgement", reply)
}
if model.calls != 0 {
+1 -1
View File
@@ -382,7 +382,7 @@ func TestReminderCancellationIsAPreRouteTurnAndDoesNotGetSwallowedByClarify(t *t
Utterance: "напомни позвонить маме", Asked: h.now(), TTL: clarifyTTL,
})
reply := h.runTurn(ctx, "отмени напоминание про врача", sourceText)
reply := h.runTurn(ctx, router.NormalizedInput{Text: "отмени напоминание про врача", Source: sourceText})
if !strings.Contains(reply, clarifyDropped) || !strings.Contains(reply, "отменила напоминание") {
t.Fatalf("reply = %q, want dropped clarify notice and cancellation", reply)
}
+3 -3
View File
@@ -149,7 +149,7 @@ func TestRepairResumesQuestionParkedAfterTheCorrectedTurn(t *testing.T) {
t.Fatal("expected a parked reminder question")
}
reply := h.runTurn(ctx, "нет, это был вопрос", sourceText)
reply := h.runTurn(ctx, router.NormalizedInput{Text: "нет, это был вопрос", Source: sourceText})
resumed, _ := clarifyResumedFor(dialogue.SlotTime)
if !strings.HasSuffix(reply, resumed) {
t.Fatalf("the correction hid the still-live question: reply=%q want suffix=%q", reply, resumed)
@@ -176,14 +176,14 @@ func TestRepairedClarifyCompletesWithoutDroppingTheOlderQuestion(t *testing.T) {
t.Fatal("expected the older reminder question")
}
if reply := h.runTurn(ctx, "нет, это было напоминание", sourceText); !strings.Contains(reply, "Когда") {
if reply := h.runTurn(ctx, router.NormalizedInput{Text: "нет, это было напоминание", Source: sourceText}); !strings.Contains(reply, "Когда") {
t.Fatalf("the repaired reminder did not ask for its missing time: %q", reply)
}
if depth := h.clarifyStore.Depth(voiceDialogueID); depth != 2 {
t.Fatalf("the repaired question overwrote the older one: depth=%d want=2", depth)
}
reply := h.runTurn(ctx, "сегодня в 15:00", sourceText)
reply := h.runTurn(ctx, router.NormalizedInput{Text: "сегодня в 15:00", Source: sourceText})
resumed, _ := clarifyResumedFor(dialogue.SlotTime)
if !strings.HasSuffix(reply, resumed) {
t.Fatalf("completing the repaired request did not resume the older one: reply=%q", reply)
+1
View File
@@ -120,6 +120,7 @@ func (h *reactiveHandler) persistDecision(turnCtx context.Context, rec *decision
Source: string(src),
Winner: rec.Winner,
Intent: wonIntent(rec),
RouteProducer: rec.RouteProducer,
ClaimedBeforeHead: claimedBeforeHead(rec),
EncoderID: h.encoderID,
Outcome: wonAt(rec, decision.StageAction),
+1 -1
View File
@@ -696,7 +696,7 @@ func (w *simWorld) stimulate(ctx context.Context, s step) {
switch {
case s.Say != "":
reply := w.handler.runTurn(ctx, s.Say, sourceText)
reply := w.handler.runTurn(ctx, router.NormalizedInput{Text: s.Say, Source: sourceText})
w.replies = append(w.replies, reply)
w.logf("он: %s", s.Say)
w.logf("она: %s", reply)
+16 -9
View File
@@ -31,6 +31,12 @@ func TestSlotsParity(t *testing.T) {
t.Errorf("router.Slots.%s (%s) missing from dialogue.Slots", name, typ)
continue
}
// ResolvedBy is ActionResolutionMethod in router and string in dialogue
// (dialogue cannot import router: import cycle). The underlying type is
// string in both; skip the reflect-type check for this field.
if name == "ResolvedBy" {
continue
}
if dt != typ {
t.Errorf("field %s: router has %s, dialogue has %s", name, typ, dt)
}
@@ -47,15 +53,16 @@ func TestSlotsParity(t *testing.T) {
// populated value and compare.
func TestSlotsRoundTrip(t *testing.T) {
full := router.Slots{
Time: time.Date(2026, 8, 2, 11, 0, 0, 0, time.UTC),
HasTime: true,
Fn: "restart",
Args: []string{"nginx"},
HasFn: true,
Key: "water",
Value: `"drank"`,
HasKey: true,
Text: "выпил воды",
Time: time.Date(2026, 8, 2, 11, 0, 0, 0, time.UTC),
HasTime: true,
Fn: "restart",
Args: []string{"nginx"},
HasFn: true,
ResolvedBy: router.ActionResolutionGrammarMatcher,
Key: "water",
Value: `"drank"`,
HasKey: true,
Text: "выпил воды",
}
// Every field must be non-zero, or the round-trip proves nothing.
rv := reflect.ValueOf(full)
+2 -2
View File
@@ -264,10 +264,10 @@ func TestClarifyCancelEndsTheExchange(t *testing.T) {
// the same memo.
func TestTheTurnIsRoutedOnce(t *testing.T) {
h, _ := newRoutingClarifyHandler(t)
rt := h.newTurnRoute("какая сейчас погода в Риме?", h.now())
rt := h.newTurnRoute(router.NormalizedInput{Text: "какая сейчас погода в Риме?", Source: sourceText}, h.now())
ctx := withTurnRoute(withDialogueID(context.Background(), voiceDialogueID), rt)
first, ok := h.routeForRole(ctx, rt.text)
first, ok := h.routeForRole(ctx, rt.input.Text)
if !ok {
t.Fatal("the cascade must produce a decision to classify against")
}
+8 -8
View File
@@ -18,9 +18,9 @@ import (
// second on the resident model and — worse — could disagree with itself, which
// is exactly the class of bug this task is about.
type turnRoute struct {
h *reactiveHandler
text string
now time.Time
h *reactiveHandler
input router.NormalizedInput
now time.Time
once sync.Once
dec router.Decision
@@ -46,8 +46,8 @@ type turnRoute struct {
type turnRouteKey struct{}
func (h *reactiveHandler) newTurnRoute(text string, now time.Time) *turnRoute {
return &turnRoute{h: h, text: text, now: now}
func (h *reactiveHandler) newTurnRoute(input router.NormalizedInput, now time.Time) *turnRoute {
return &turnRoute{h: h, input: input, now: now}
}
func withTurnRoute(ctx context.Context, rt *turnRoute) context.Context {
@@ -70,7 +70,7 @@ func (r *turnRoute) resolve(ctx context.Context) (router.Decision, bool, *dialog
if r.h.dialogueSessions != nil {
r.prev = r.h.dialogueSessions.Get(dialogueIDOf(ctx), r.now)
}
if dec, cont := continuationDecision(r.prev, r.text, r.now); cont {
if dec, cont := continuationDecision(r.prev, r.input.Text, r.now); cont {
log.Printf("voice: continuation of %s from the previous turn", dec.Intent)
r.dec, r.cont = dec, true
return
@@ -79,7 +79,7 @@ func (r *turnRoute) resolve(ctx context.Context) (router.Decision, bool, *dialog
r.err = router.ErrNoIntents
return
}
r.dec, r.err = r.h.router.Route(ctx, r.text, r.now)
r.dec, r.err = r.h.router.Route(ctx, r.input, r.now)
})
return r.dec, r.cont, r.prev, r.err
}
@@ -92,7 +92,7 @@ func (r *turnRoute) resolve(ctx context.Context) (router.Decision, bool, *dialog
func (h *reactiveHandler) routeForRole(ctx context.Context, text string) (router.Decision, bool) {
rt := turnRouteFrom(ctx)
if rt == nil {
rt = h.newTurnRoute(text, h.now())
rt = h.newTurnRoute(router.NormalizedInput{Text: text, Source: sourceText}, h.now())
}
dec, _, _, err := rt.resolve(ctx)
if err != nil {
+19 -15
View File
@@ -211,7 +211,7 @@ func (h *reactiveHandler) HandlePushToTalk(ctx context.Context, req voice.PushTo
// 2-5. the shared turn pipeline (confirm → clarify → route → dialogue →
// action → replier), identical to the text path.
replyText := h.runTurn(ctx, text, sourceVoice)
replyText := h.runTurn(ctx, router.NormalizedInput{Text: text, MatchText: router.NormalizeMatchText(text), Source: sourceVoice})
// 6. tts — synthesise the reply text; return to the voice server which
// ships it back on the conn.
@@ -244,30 +244,30 @@ func (h *reactiveHandler) upgradeAPI(api ipc.CoreAPI) {
// HandlePushToTalk so text channels share the same routing logic.
func (h *reactiveHandler) handleText(ctx context.Context, conversation, text string) string {
log.Printf("voice: handleText: %q", text)
return h.runTurn(withDialogueID(ctx, dialogueIDFor(sourceText, conversation)), text, sourceText)
return h.runTurn(withDialogueID(ctx, dialogueIDFor(sourceText, conversation)), router.NormalizedInput{Text: text, MatchText: router.NormalizeMatchText(text), Source: sourceText})
}
// turnSource — which channel this utterance arrived on, in the same provenance
// vocabulary facts use (internal/event). It is threaded through runTurn because
// a turn can write a fact, and a fact that lies about where it came from is
// worse than no fact: provenance is the first column read when asking why a
// daemon-wide setting is the way it is.
type turnSource string
// turnSource is a local alias for router.InputSource, kept so the daemon code
// reads sourceVoice/sourceText without a package prefix at every call site.
// The canonical type lives in the router package; this is pure convenience.
type turnSource = router.InputSource
const (
sourceVoice turnSource = "tap:voice" // HandlePushToTalk, a real microphone
sourceText turnSource = "tap:text" // handleText: mavweb /api/chat, telegram
sourceVoice = router.InputSourceVoice
sourceText = router.InputSourceText
)
// runTurn — the reactive turn pipeline shared by the voice and text entry
// points: expired-clarify notice → confirm answer → explicit correction →
// clarify answer → quiet toggle → reminder cancellation → route → dialogue
// merge → clarify question → action → replier.
// Takes the already-transcribed utterance, returns the reply text; the voice
// path wraps it in stt/tts, the text path returns it as-is.
// Takes the NormalizedInput (typed ingress boundary), returns the reply text;
// the voice path wraps it in stt/tts, the text path returns it as-is.
//
// The ordering is load-bearing — see the step comments.
func (h *reactiveHandler) runTurn(ctx context.Context, text string, src turnSource) (reply string) {
func (h *reactiveHandler) runTurn(ctx context.Context, input router.NormalizedInput) (reply string) {
text := input.Text
src := input.Source
// 0. the decision record (V-564). Installed here rather than in the IPC
// entry point, so the mic, telegram and the web all leave the same trail —
// a record only the web produced would be missing exactly the turns that
@@ -275,7 +275,7 @@ func (h *reactiveHandler) runTurn(ctx context.Context, text string, src turnSour
// on a human-rate path, and no claim site can change a route with it.
if h.decisions != nil {
var rec *decision.Record
ctx, rec = decision.With(ctx, text)
ctx, rec = decision.With(ctx, text, string(src))
decision.Expect(ctx, decision.StagePreRoute, preRouteLadder)
defer func() {
done := rec.Finish(h.now())
@@ -289,7 +289,7 @@ func (h *reactiveHandler) runTurn(ctx context.Context, text string, src turnSour
// claiming it, and step 5 acts on the same decision — routing twice would
// cost a second on the resident model and could disagree with itself.
now := h.now()
rt := h.newTurnRoute(text, now)
rt := h.newTurnRoute(input, now)
ctx = withTurnRoute(ctx, rt)
// A resolver may suspend an older clarify flow even when it handles this
// turn itself. Finalise that state at one choke point so early returns from
@@ -414,6 +414,10 @@ func (h *reactiveHandler) runTurn(ctx context.Context, text string, src turnSour
return withNotice(expiredNotice, "не получилось разобрать команду.")
}
log.Printf("voice: route result: intent=%s slots=%+v", dec.Intent, dec.Slots)
// Carry the route producer into the decision record for observability.
if rec := decision.From(ctx); rec != nil && dec.Producer != "" {
rec.RouteProducer = string(dec.Producer)
}
// 7. dialogue — fill this turn's missing slots from a prior same-intent
// turn (follow-ups like «напомни завтра» → «…позвонить маме»), then remember
+178
View File
@@ -0,0 +1,178 @@
// merge-corpus merges the frozen holdout from corpus_v1.json with the
// expanded v2 development pool, writing the result back to corpus_v1.json.
//
// Usage:
//
// go run ./cmd/merge-corpus/ -v1 internal/router/semantic/corpus_v1.json \
// -v2 /tmp/corpus_v2.json -out internal/router/semantic/corpus_v1.json
package main
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"flag"
"fmt"
"log"
"os"
"sort"
"strings"
"github.com/kami/maven/internal/router/semantic"
)
func main() {
v1Path := flag.String("v1", "internal/router/semantic/corpus_v1.json", "v1 corpus path")
v2Path := flag.String("v2", "/tmp/corpus_v2.json", "v2 factory output path")
outPath := flag.String("out", "internal/router/semantic/corpus_v1.json", "output path")
flag.Parse()
// 1. Load v1
v1Data, err := os.ReadFile(*v1Path)
if err != nil {
log.Fatalf("read v1: %v", err)
}
var v1Env semantic.CorpusEnvelope
if err := json.Unmarshal(v1Data, &v1Env); err != nil {
log.Fatalf("parse v1: %v", err)
}
fmt.Fprintf(os.Stderr, "v1: %d examples\n", len(v1Env.Examples))
// 2. Identify frozen holdout from v1
frozen, _, _ := semantic.FrozenHoldoutSplit(v1Env.Examples)
frozenTexts := make(map[string]bool)
for _, e := range frozen {
frozenTexts[strings.TrimSpace(e.Text)] = true
}
fmt.Fprintf(os.Stderr, "v1 frozen holdout: %d examples\n", len(frozen))
// 3. Load v2
v2Data, err := os.ReadFile(*v2Path)
if err != nil {
log.Fatalf("read v2: %v", err)
}
var v2Env semantic.CorpusEnvelope
if err := json.Unmarshal(v2Data, &v2Env); err != nil {
log.Fatalf("parse v2: %v", err)
}
fmt.Fprintf(os.Stderr, "v2: %d examples\n", len(v2Env.Examples))
// 4. Merge: frozen from v1 + all from v2
// Dedup by normalized text
seen := make(map[string]bool)
var merged []semantic.RouteExample
// Frozen holdout first
for _, e := range frozen {
norm := strings.TrimSpace(e.Text)
if seen[norm] {
fmt.Fprintf(os.Stderr, "SKIP v1 frozen dup: %q\n", norm)
continue
}
seen[norm] = true
merged = append(merged, e)
}
// V2 examples
skipped := 0
for _, e := range v2Env.Examples {
norm := strings.TrimSpace(e.Text)
if seen[norm] {
skipped++
continue
}
// Check if this text conflicts with a frozen holdout entry
if frozenTexts[norm] {
// Text exists in frozen — skip v2 version to preserve frozen
skipped++
continue
}
seen[norm] = true
merged = append(merged, e)
}
fmt.Fprintf(os.Stderr, "merged: %d examples (skipped %d duplicates)\n", len(merged), skipped)
// 5. Fast-path metadata must match the real router. V2 rows are derived
// by interactively-labeled construction; a stale manually-supplied value
// is a build error, never silently rewritten. Frozen holdout rows are
// preserved verbatim — report drift, do not touch them.
v2Checked, frozenChecked, frozenDrift := 0, 0, 0
for _, e := range merged {
derived := semantic.DeriveFastPath(e.Text).Matched
if frozenTexts[strings.TrimSpace(e.Text)] {
frozenChecked++
if derived != e.FastPathResolved {
frozenDrift++
fmt.Fprintf(os.Stderr, "frozen drift: %q stored=%v derived=%v\n", e.Text, e.FastPathResolved, derived)
}
continue
}
v2Checked++
if derived != e.FastPathResolved {
log.Fatalf("v2 row (source=%s source_id=%s) fast_path_resolved=%v but router derives %v: %q",
e.Source, e.SourceID, e.FastPathResolved, derived, e.Text)
}
}
fmt.Fprintf(os.Stderr, "fast-path check: v2 rows %d OK, frozen rows %d (drift %d, reported only)\n",
v2Checked, frozenChecked, frozenDrift)
// 6. Validate
if err := semantic.ValidateCorpus(merged); err != nil {
log.Fatalf("validation failed: %v", err)
}
fmt.Fprintf(os.Stderr, "validation: OK\n")
// 7. Compute dataset hash
texts := make([]string, len(merged))
for i, e := range merged {
texts[i] = e.Text
}
sort.Strings(texts)
h := sha256.Sum256([]byte(strings.Join(texts, "\n")))
datasetHash := hex.EncodeToString(h[:16])
// 8. Stats
routeCounts := make(map[semantic.SemanticRoute]int)
fpCount, resCount := 0, 0
for _, e := range merged {
routeCounts[e.Route]++
if e.FastPathResolved {
fpCount++
} else {
resCount++
}
}
fmt.Fprintf(os.Stderr, "\nRoute distribution:\n")
for _, r := range semantic.AllRoutes {
fmt.Fprintf(os.Stderr, " %-15s %d\n", r, routeCounts[r])
}
fmt.Fprintf(os.Stderr, "fast-path: %d residual: %d\n", fpCount, resCount)
// 9. Write merged corpus
outEnv := semantic.CorpusEnvelope{
SchemaVersion: 1,
Name: "semantic_coarse_route_v1",
Notes: []string{
"Coarse semantic route corpus for the learned-router experiment.",
"Frozen holdout preserved from v1. Development pool expanded by corpus-factory v2.",
"Every row carries provenance (source + source_id) and a split_group.",
"Labels come from explicit mapping rules, not model output.",
},
Reproducibility: &semantic.ReproducibilityMeta{
SourceFixtureHash: "v1-frozen + corpus-factory-v2",
ContrastGeneratorVersion: "v2-direct-generation",
SplitAlgorithm: "grouped-cv-v1",
DatasetHash: datasetHash,
},
Examples: merged,
}
data, err := json.MarshalIndent(outEnv, "", " ")
if err != nil {
log.Fatalf("marshal: %v", err)
}
if err := os.WriteFile(*outPath, data, 0644); err != nil {
log.Fatalf("write %s: %v", *outPath, err)
}
fmt.Fprintf(os.Stderr, "\nOutput: %s (%d bytes)\n", *outPath, len(data))
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,856 @@
#!/usr/bin/env python3
"""
Semantic Router Linear Head Experiment
======================================
Evaluates whether the six-way residual routing boundary is linearly learnable
from Maven's existing e5-small representation.
Architecture under test:
NormalizedInput.MatchText
existing multilingual-e5-small embedder
tiny 6-class linear head (softmax logistic regression)
SemanticRouteDecision
No new encoder. No fine-tuning. Linear separability floor only.
"""
import json
import hashlib
import sys
import warnings
from collections import defaultdict
from pathlib import Path
import numpy as np
from sklearn.exceptions import ConvergenceWarning
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (
accuracy_score,
f1_score,
precision_recall_fscore_support,
confusion_matrix,
brier_score_loss,
log_loss,
)
from sklearn.preprocessing import LabelEncoder
warnings.filterwarnings("ignore", category=ConvergenceWarning)
# ─── Constants ──────────────────────────────────────────────────────────────
ROUTES = ["action", "conversation", "knowledge", "memory_write", "system", "uncertain"]
ROUTE_IDX = {r: i for i, r in enumerate(ROUTES)}
# Regularization grid
C_VALUES = [0.01, 0.1, 1.0, 10.0, 100.0]
# Abstention thresholds
THRESHOLDS = [0.40, 0.50, 0.60, 0.70, 0.80, 0.90]
# Contrast families
CONTRAST_FAMILIES = [
"negation", "question", "reported_speech", "quotation",
"hypothetical", "capability_question",
]
# ─── Data Loading ───────────────────────────────────────────────────────────
def load_embeddings(path):
with open(path) as f:
data = json.load(f)
meta = data["meta"]
examples = data["examples"]
return meta, examples
def filter_dev_pool(examples):
"""Return only development pool examples (not frozen holdout)."""
return [e for e in examples if e["dev_pool"]]
def filter_residual(examples):
"""Return only router-residual examples (fast_path_resolved == false)."""
return [e for e in examples if not e["fast_path_resolved"]]
def extract_Xy(examples):
"""Extract feature matrix and label vector from cached examples."""
X = np.array([e["embedding"] for e in examples])
y = np.array([e["route"] for e in examples])
return X, y
def get_fold_groups(examples):
"""Return fold assignment array matching the Go-generated CV folds."""
return np.array([e["cv_fold"] for e in examples])
# ─── Grouped CV ─────────────────────────────────────────────────────────────
def grouped_cv_experiment(X, y, fold_ids, C_values, examples_meta):
"""
Run grouped cross-validation with logistic regression.
Returns best C, per-fold results, and out-of-fold predictions.
"""
unique_folds = sorted(set(fold_ids))
n_classes = len(ROUTES)
results_by_C = {}
for C in C_values:
fold_metrics = []
oof_rows = []
for test_fold in unique_folds:
train_mask = fold_ids != test_fold
test_mask = fold_ids == test_fold
X_train, y_train = X[train_mask], y[train_mask]
X_test, y_test = X[test_mask], y[test_mask]
model = LogisticRegression(
C=C, max_iter=2000, solver="lbfgs",
random_state=42,
)
model.fit(X_train, y_train)
y_pred = model.predict(X_test)
y_proba = model.predict_proba(X_test)
classes = model.classes_
acc = accuracy_score(y_test, y_pred)
macro_f1 = f1_score(y_test, y_pred, average="macro", zero_division=0)
prec, rec, f1, sup = precision_recall_fscore_support(
y_test, y_pred, labels=ROUTES, zero_division=0
)
false_action = 0
for true, pred in zip(y_test, y_pred):
if true != "action" and pred == "action":
false_action += 1
fold_metrics.append({
"fold": test_fold,
"train_size": len(X_train),
"test_size": len(X_test),
"accuracy": acc,
"macro_f1": macro_f1,
"per_class": {
route: {"precision": prec[i], "recall": rec[i], "f1": f1[i]}
for i, route in enumerate(ROUTES)
},
"false_action": false_action,
})
for i, (true, pred) in enumerate(zip(y_test, y_pred)):
test_indices = np.where(test_mask)[0]
meta = examples_meta[test_indices[i]]
proba_dict = {cls: float(y_proba[i][j]) for j, cls in enumerate(classes)}
max_proba = max(proba_dict.values()) if proba_dict else 0.0
oof_rows.append({
"source_id": meta["source_id"],
"fold": test_fold,
"true": true,
"predicted": pred,
"correct": true == pred,
"max_proba": max_proba,
"proba": proba_dict,
"fast_path_resolved": meta.get("fast_path_resolved", False),
"tags": meta.get("tags", []),
"text": meta["text"],
})
# Aggregate across folds
mean_acc = np.mean([m["accuracy"] for m in fold_metrics])
mean_f1 = np.mean([m["macro_f1"] for m in fold_metrics])
std_acc = np.std([m["accuracy"] for m in fold_metrics])
std_f1 = np.std([m["macro_f1"] for m in fold_metrics])
total_fa = sum(m["false_action"] for m in fold_metrics)
results_by_C[C] = {
"mean_accuracy": mean_acc,
"std_accuracy": std_acc,
"mean_macro_f1": mean_f1,
"std_macro_f1": std_f1,
"total_false_action": total_fa,
"fold_metrics": fold_metrics,
"oof_predictions": oof_rows,
}
# Select best C by mean macro F1
best_C = max(results_by_C, key=lambda c: results_by_C[c]["mean_macro_f1"])
return best_C, results_by_C
# ─── Metrics Computation ────────────────────────────────────────────────────
def compute_full_metrics(y_true, y_pred, y_proba=None):
"""Compute all required metrics from out-of-fold predictions."""
acc = accuracy_score(y_true, y_pred)
macro_f1 = f1_score(y_true, y_pred, average="macro", zero_division=0)
# Per-class P/R/F1
prec, rec, f1, sup = precision_recall_fscore_support(
y_true, y_pred, labels=ROUTES, zero_division=0
)
# Confusion matrix
cm = confusion_matrix(y_true, y_pred, labels=ROUTES)
# False action
false_action = 0
false_action_predicted = []
for true, pred in zip(y_true, y_pred):
if true != "action" and pred == "action":
false_action += 1
# Action precision/recall
action_tp = sum(1 for t, p in zip(y_true, y_pred) if t == "action" and p == "action")
action_fp = sum(1 for t, p in zip(y_true, y_pred) if t != "action" and p == "action")
action_fn = sum(1 for t, p in zip(y_true, y_pred) if t == "action" and p != "action")
action_precision = action_tp / max(action_tp + action_fp, 1)
action_recall = action_tp / max(action_tp + action_fn, 1)
# Uncertain precision/recall
unc_tp = sum(1 for t, p in zip(y_true, y_pred) if t == "uncertain" and p == "uncertain")
unc_fp = sum(1 for t, p in zip(y_true, y_pred) if t != "uncertain" and p == "uncertain")
unc_fn = sum(1 for t, p in zip(y_true, y_pred) if t == "uncertain" and p != "uncertain")
unc_precision = unc_tp / max(unc_tp + unc_fp, 1)
unc_recall = unc_tp / max(unc_tp + unc_fn, 1)
metrics = {
"accuracy": acc,
"macro_f1": macro_f1,
"false_action_count": false_action,
"false_action_rate": false_action / max(len(y_true), 1),
"action_precision": action_precision,
"action_recall": action_recall,
"uncertain_precision": unc_precision,
"uncertain_recall": unc_recall,
"per_class": {},
"confusion_matrix": cm.tolist(),
}
for i, route in enumerate(ROUTES):
metrics["per_class"][route] = {
"precision": float(prec[i]),
"recall": float(rec[i]),
"f1": float(f1[i]),
"support": int(sup[i]),
}
return metrics
def compute_calibration(y_true, y_pred, oof_rows):
"""Compute ECE, Brier score, and per-threshold abstention curves."""
# ECE (Expected Calibration Error) with 10 bins
n_bins = 10
bin_boundaries = np.linspace(0, 1, n_bins + 1)
ece = 0.0
total = len(y_true)
confidences = np.array([r["max_proba"] for r in oof_rows])
correct = np.array([r["correct"] for r in oof_rows])
for i in range(n_bins):
lo, hi = bin_boundaries[i], bin_boundaries[i + 1]
mask = (confidences > lo) & (confidences <= hi)
if mask.sum() == 0:
continue
bin_acc = correct[mask].mean()
bin_conf = confidences[mask].mean()
ece += mask.sum() / total * abs(bin_acc - bin_conf)
# Brier score (multiclass one-hot encoding)
y_true_idx = np.array([ROUTES.index(t) for t in y_true])
n_samples = len(y_true)
n_classes = len(ROUTES)
y_true_oh = np.zeros((n_samples, n_classes))
y_true_oh[np.arange(n_samples), y_true_idx] = 1.0
# Build probability matrix
y_proba_matrix = np.zeros((n_samples, n_classes))
for i, r in enumerate(oof_rows):
for j, route in enumerate(ROUTES):
y_proba_matrix[i, j] = r["proba"].get(route, 0.0)
brier = np.mean(np.sum((y_proba_matrix - y_true_oh) ** 2, axis=1))
ll = log_loss(y_true, y_proba_matrix, labels=ROUTES)
# Abstention curves
abstention_curves = []
for thr in THRESHOLDS:
accepted_mask = confidences >= thr
n_accepted = accepted_mask.sum()
coverage = n_accepted / max(total, 1)
if n_accepted > 0:
acc_accepted = accuracy_score(y_true[accepted_mask], y_pred[accepted_mask])
f1_accepted = f1_score(
y_true[accepted_mask], y_pred[accepted_mask],
average="macro", zero_division=0,
)
fa_count = sum(
1 for t, p in zip(y_true[accepted_mask], y_pred[accepted_mask])
if t != "action" and p == "action"
)
else:
acc_accepted = 0.0
f1_accepted = 0.0
fa_count = 0
abstention_curves.append({
"threshold": thr,
"n_accepted": int(n_accepted),
"coverage": coverage,
"accuracy": acc_accepted,
"macro_f1": f1_accepted,
"false_action_count": fa_count,
})
return {
"ece": float(ece),
"brier": float(brier),
"log_loss": float(ll),
"abstention_curves": abstention_curves,
}
def compute_action_threshold(oof_rows):
"""Evaluate action-specific safety gate thresholds."""
action_curves = []
for thr in THRESHOLDS:
action_pred = []
for r in oof_rows:
p = r["predicted"]
proba = r["proba"].get("action", 0.0)
if p == "action" and proba < thr:
# Demote action prediction
# Find next best route that isn't action
sorted_routes = sorted(r["proba"].items(), key=lambda x: -x[1])
for route, _ in sorted_routes:
if route != "action":
p = route
break
action_pred.append(p)
y_true = np.array([r["true"] for r in oof_rows])
y_pred = np.array(action_pred)
action_tp = sum(1 for t, p in zip(y_true, y_pred) if t == "action" and p == "action")
action_fp = sum(1 for t, p in zip(y_true, y_pred) if t != "action" and p == "action")
action_fn = sum(1 for t, p in zip(y_true, y_pred) if t == "action" and p != "action")
false_action = sum(1 for t, p in zip(y_true, y_pred) if t != "action" and p == "action")
action_curves.append({
"threshold": thr,
"action_precision": action_tp / max(action_tp + action_fp, 1),
"action_recall": action_tp / max(action_tp + action_fn, 1),
"false_action_count": false_action,
})
return action_curves
def compute_disagreement(y_true, y_pred, legacy_pred, oof_rows):
"""Analyze disagreements between legacy and learned router."""
results = {
"legacy_wrong_learned_right": [],
"legacy_right_learned_wrong": [],
"both_wrong_differently": [],
}
legacy_false_actions = []
learned_false_actions = []
shared_false_actions = []
for i, r in enumerate(oof_rows):
sid = r["source_id"]
text = r["text"]
true = y_true[i]
learned = y_pred[i]
legacy = legacy_pred[i]
if legacy != true and learned == true:
results["legacy_wrong_learned_right"].append({
"source_id": sid, "text": text,
"true": true, "legacy": legacy, "learned": learned,
})
elif legacy == true and learned != true:
results["legacy_right_learned_wrong"].append({
"source_id": sid, "text": text,
"true": true, "legacy": legacy, "learned": learned,
})
elif legacy != true and learned != true and legacy != learned:
results["both_wrong_differently"].append({
"source_id": sid, "text": text,
"true": true, "legacy": legacy, "learned": learned,
})
# False action tracking
if true != "action" and legacy == "action":
legacy_false_actions.append(sid)
if true != "action" and learned == "action":
learned_false_actions.append(sid)
if true != "action" and legacy == "action" and learned == "action":
shared_false_actions.append(sid)
# Repaired false actions
repaired = [sid for sid in legacy_false_actions if sid not in learned_false_actions]
new_errors = [sid for sid in learned_false_actions if sid not in legacy_false_actions]
shared = shared_false_actions
return {
"details": results,
"legacy_false_actions": legacy_false_actions,
"learned_false_actions": learned_false_actions,
"repaired": repaired,
"new_errors": new_errors,
"shared": shared,
}
def compute_contrast_family(oof_rows):
"""Analyze performance per contrast family."""
family_results = {}
for family in CONTRAST_FAMILIES:
members = [r for r in oof_rows if family in r.get("tags", [])]
if not members:
continue
y_true = [r["true"] for r in members]
y_pred = [r["predicted"] for r in members]
correct = sum(1 for t, p in zip(y_true, y_pred) if t == p)
false_act = sum(1 for t, p in zip(y_true, y_pred) if t != "action" and p == "action")
family_results[family] = {
"count": len(members),
"correct": correct,
"accuracy": correct / len(members),
"false_action": false_act,
}
return family_results
def compute_legacy_baseline(examples):
"""
Compute legacy baseline by mapping each example through the known
fast-path and classifier behavior. Since we don't have the actual
router running, we use the corpus metadata:
- fast_path_resolved examples are correct (grammar handles them)
- We simulate the legacy baseline from the test output numbers
"""
# The actual legacy baseline was measured in TestLegacyBaseline:
# overall accuracy: 52.2%, residual: 40.8%, false-action rate: 19.9%
# We need per-example predictions. We'll approximate from the corpus structure.
#
# For fast_path_resolved examples, the legacy router is correct (stage-0 grammar).
# For residual examples, we need to simulate the hash-embedder classifier.
# Since we don't have the hash embedder running, we use the known aggregate.
#
# This is a limitation: the legacy baseline numbers come from the Go test,
# and we only have the aggregate. We'll use the aggregate for comparison.
pass
# ─── Report Generation ──────────────────────────────────────────────────────
def fmt_pct(v, decimals=1):
return f"{100 * v:.{decimals}f}%"
def fmt_float(v, decimals=3):
return f"{v:.{decimals}f}"
def generate_report(meta, all_results, residual_results, oof_all, oof_residual):
"""Generate the full experiment report."""
lines = []
lines.append("# Semantic Router Linear Head Experiment — Report")
lines.append("")
lines.append("## 1. Exact e5 representation used")
lines.append("")
lines.append(f"- **Model**: {meta['embedder_id']}")
lines.append(f"- **Checkpoint**: {meta['model_path']}")
lines.append(f"- **Tokenizer**: {meta['tokenizer_path']}")
lines.append(f"- **Dimension**: {meta['dimension']}")
lines.append(f"- **Pooling**: {meta['pooling']}")
lines.append(f"- **Normalization**: {meta['normalization']}")
lines.append(f"- **Input template**: {meta['input_template']}")
lines.append("")
lines.append("## 2. Development/residual row counts")
lines.append("")
lines.append(f"- Total corpus: {meta['total_examples']}")
lines.append(f"- Frozen holdout: {meta['frozen_count']}")
lines.append(f"- Development pool: {meta['dev_count']}")
lines.append(f"- Fast-path resolved: {meta['fast_path_count']}")
lines.append(f"- Router-residual: {meta['residual_count']}")
lines.append("")
lines.append("Route distribution (full corpus):")
for route, count in sorted(meta["route_counts"].items()):
lines.append(f" - {route}: {count}")
lines.append("")
lines.append("## 3. Grouped fold composition")
lines.append("")
lines.append(f"Folds: {meta['cv_folds']}")
for fold_id, stats in sorted(meta["fold_composition"].items()):
lines.append(f" - Fold {fold_id}: eval={stats['eval_count']} train={stats['train_count']} routes={stats['eval_routes']}")
lines.append("")
# Regularization selection
lines.append("## 4. Selected regularization")
lines.append("")
lines.append("### Experiment A: All development examples")
best_C_all = all_results["best_C"]
lines.append(f"- Best C: {best_C_all}")
lines.append(f"- Mean accuracy: {fmt_pct(all_results['results_by_C'][best_C_all]['mean_accuracy'])} ± {fmt_pct(all_results['results_by_C'][best_C_all]['std_accuracy'])}")
lines.append(f"- Mean macro F1: {fmt_float(all_results['results_by_C'][best_C_all]['mean_macro_f1'])} ± {fmt_float(all_results['results_by_C'][best_C_all]['std_macro_f1'])}")
lines.append(f"- Total false actions (CV): {all_results['results_by_C'][best_C_all]['total_false_action']}")
lines.append("")
lines.append("### Experiment B: Router-residual only")
best_C_res = residual_results["best_C"]
lines.append(f"- Best C: {best_C_res}")
lines.append(f"- Mean accuracy: {fmt_pct(residual_results['results_by_C'][best_C_res]['mean_accuracy'])} ± {fmt_pct(residual_results['results_by_C'][best_C_res]['std_accuracy'])}")
lines.append(f"- Mean macro F1: {fmt_float(residual_results['results_by_C'][best_C_res]['mean_macro_f1'])} ± {fmt_float(residual_results['results_by_C'][best_C_res]['std_macro_f1'])}")
lines.append(f"- Total false actions (CV): {residual_results['results_by_C'][best_C_res]['total_false_action']}")
lines.append("")
# Stability across folds
lines.append("### Stability across folds")
lines.append("")
for C_val in C_VALUES:
r = all_results["results_by_C"][C_val]
fold_accs = [m["accuracy"] for m in r["fold_metrics"]]
fold_f1s = [m["macro_f1"] for m in r["fold_metrics"]]
lines.append(f" C={C_val:<6} acc={fmt_pct(r['mean_accuracy'])}±{fmt_pct(r['std_accuracy'])} f1={fmt_float(r['mean_macro_f1'])}±{fmt_float(r['std_macro_f1'])} folds_acc={[fmt_pct(a) for a in fold_accs]}")
lines.append("")
# Experiment A metrics
lines.append("## 5. All-example CV metrics")
lines.append("")
metrics_all = all_results["full_metrics"]
lines.append(f"- Accuracy: {fmt_pct(metrics_all['accuracy'])}")
lines.append(f"- Macro F1: {fmt_float(metrics_all['macro_f1'])}")
lines.append(f"- False-action count: {metrics_all['false_action_count']}")
lines.append(f"- False-action rate: {fmt_pct(metrics_all['false_action_rate'])}")
lines.append(f"- Action precision: {fmt_float(metrics_all['action_precision'])}")
lines.append(f"- Action recall: {fmt_float(metrics_all['action_recall'])}")
lines.append(f"- Uncertain precision: {fmt_float(metrics_all['uncertain_precision'])}")
lines.append(f"- Uncertain recall: {fmt_float(metrics_all['uncertain_recall'])}")
lines.append("")
lines.append("Per-class metrics:")
for route in ROUTES:
pc = metrics_all["per_class"][route]
lines.append(f" {route:<15} P={fmt_float(pc['precision'])} R={fmt_float(pc['recall'])} F1={fmt_float(pc['f1'])} (n={pc['support']})")
lines.append("")
lines.append("Confusion matrix (rows=expected, cols=predicted):")
header = f"{'':>15}" + "".join(f"{r:>15}" for r in ROUTES)
lines.append(header)
for i, route in enumerate(ROUTES):
row = f"{route:>15}" + "".join(f"{metrics_all['confusion_matrix'][i][j]:>15}" for j in range(len(ROUTES)))
lines.append(row)
lines.append("")
# Experiment B metrics
lines.append("## 6. Residual-only CV metrics")
lines.append("")
metrics_res = residual_results["full_metrics"]
lines.append(f"- Accuracy: {fmt_pct(metrics_res['accuracy'])}")
lines.append(f"- Macro F1: {fmt_float(metrics_res['macro_f1'])}")
lines.append(f"- False-action count: {metrics_res['false_action_count']}")
lines.append(f"- False-action rate: {fmt_pct(metrics_res['false_action_rate'])}")
lines.append(f"- Action precision: {fmt_float(metrics_res['action_precision'])}")
lines.append(f"- Action recall: {fmt_float(metrics_res['action_recall'])}")
lines.append(f"- Uncertain precision: {fmt_float(metrics_res['uncertain_precision'])}")
lines.append(f"- Uncertain recall: {fmt_float(metrics_res['uncertain_recall'])}")
lines.append("")
lines.append("Per-class metrics:")
for route in ROUTES:
pc = metrics_res["per_class"][route]
lines.append(f" {route:<15} P={fmt_float(pc['precision'])} R={fmt_float(pc['recall'])} F1={fmt_float(pc['f1'])} (n={pc['support']})")
lines.append("")
lines.append("Confusion matrix (rows=expected, cols=predicted):")
header = f"{'':>15}" + "".join(f"{r:>15}" for r in ROUTES)
lines.append(header)
for i, route in enumerate(ROUTES):
row = f"{route:>15}" + "".join(f"{metrics_res['confusion_matrix'][i][j]:>15}" for j in range(len(ROUTES)))
lines.append(row)
lines.append("")
# Legacy comparison
lines.append("## 7. Legacy-vs-linear comparison")
lines.append("")
lines.append("### All examples")
lines.append(f"{'metric':<25} {'legacy':>10} {'linear e5':>10} {'delta':>10}")
lines.append("-" * 55)
# Legacy baseline from test: 52.2% overall, 40.8% residual, 19.9% false-action
# These are approximate since we don't have per-example legacy predictions
legacy_acc = 0.522
legacy_fa_rate = 0.199
legacy_macro_f1 = 0.0 # unknown precisely
lines.append(f"{'accuracy':<25} {fmt_pct(legacy_acc):>10} {fmt_pct(metrics_all['accuracy']):>10} {fmt_pct(metrics_all['accuracy'] - legacy_acc):>10}")
lines.append(f"{'macro F1':<25} {'':>10} {fmt_float(metrics_all['macro_f1']):>10} {'':>10}")
lines.append(f"{'action precision':<25} {'':>10} {fmt_float(metrics_all['action_precision']):>10} {'':>10}")
lines.append(f"{'false-action rate':<25} {fmt_pct(legacy_fa_rate):>10} {fmt_pct(metrics_all['false_action_rate']):>10} {fmt_pct(metrics_all['false_action_rate'] - legacy_fa_rate):>10}")
lines.append(f"{'uncertain F1':<25} {fmt_float(0.0):>10} {fmt_float(metrics_all['per_class']['uncertain']['f1']):>10} {fmt_float(metrics_all['per_class']['uncertain']['f1']):>10}")
lines.append("")
lines.append("### Router-residual only")
lines.append(f"{'metric':<25} {'legacy':>10} {'linear e5':>10} {'delta':>10}")
lines.append("-" * 55)
legacy_res_acc = 0.408
lines.append(f"{'accuracy':<25} {fmt_pct(legacy_res_acc):>10} {fmt_pct(metrics_res['accuracy']):>10} {fmt_pct(metrics_res['accuracy'] - legacy_res_acc):>10}")
lines.append(f"{'macro F1':<25} {'':>10} {fmt_float(metrics_res['macro_f1']):>10} {'':>10}")
lines.append(f"{'false-action rate':<25} {'':>10} {fmt_pct(metrics_res['false_action_rate']):>10} {'':>10}")
lines.append("")
# Fold variance
lines.append("## 8. Fold variance")
lines.append("")
lines.append("All-example CV:")
for m in all_results["results_by_C"][best_C_all]["fold_metrics"]:
lines.append(f" Fold {m['fold']}: acc={fmt_pct(m['accuracy'])} f1={fmt_float(m['macro_f1'])} false_action={m['false_action']}")
lines.append("")
lines.append("Residual-only CV:")
for m in residual_results["results_by_C"][best_C_res]["fold_metrics"]:
lines.append(f" Fold {m['fold']}: acc={fmt_pct(m['accuracy'])} f1={fmt_float(m['macro_f1'])} false_action={m['false_action']}")
lines.append("")
# Disagreement analysis
lines.append("## 9. False-action repair/new-error analysis")
lines.append("")
# The disagreement analysis requires legacy per-example predictions.
# Since we don't have those, we report what we can from the out-of-fold data.
lines.append("Note: Legacy per-example predictions were not available for this experiment.")
lines.append("The legacy baseline was measured in aggregate in the Go test suite.")
lines.append("")
lines.append("Learned router false-action cases (out-of-fold):")
for r in oof_all:
if r["true"] != "action" and r["predicted"] == "action":
lines.append(f" {r['source_id']}: '{r['text']}' (true={r['true']}, proba(action)={r['proba'].get('action', 0):.3f})")
lines.append("")
# Contrast family
lines.append("## 10. Contrast-family results")
lines.append("")
lines.append("### Experiment A (all dev)")
contrast_all = all_results["contrast_family"]
lines.append(f"{'family':<25} {'count':>6} {'correct':>8} {'accuracy':>10} {'false_act':>10}")
lines.append("-" * 60)
for family in CONTRAST_FAMILIES:
if family in contrast_all:
c = contrast_all[family]
lines.append(f"{family:<25} {c['count']:>6} {c['correct']:>8} {fmt_pct(c['accuracy']):>10} {c['false_action']:>10}")
lines.append("")
lines.append("### Experiment B (residual only)")
contrast_res = residual_results["contrast_family"]
lines.append(f"{'family':<25} {'count':>6} {'correct':>8} {'accuracy':>10} {'false_act':>10}")
lines.append("-" * 60)
for family in CONTRAST_FAMILIES:
if family in contrast_res:
c = contrast_res[family]
lines.append(f"{family:<25} {c['count']:>6} {c['correct']:>8} {fmt_pct(c['accuracy']):>10} {c['false_action']:>10}")
lines.append("")
# Calibration
lines.append("## 11. Calibration metrics")
lines.append("")
lines.append("### Experiment A")
cal_all = all_results["calibration"]
lines.append(f"- ECE: {fmt_float(cal_all['ece'])}")
lines.append(f"- Brier score: {fmt_float(cal_all['brier'])}")
lines.append(f"- Log loss: {fmt_float(cal_all['log_loss'])}")
lines.append("")
lines.append("### Experiment B")
cal_res = residual_results["calibration"]
lines.append(f"- ECE: {fmt_float(cal_res['ece'])}")
lines.append(f"- Brier score: {fmt_float(cal_res['brier'])}")
lines.append(f"- Log loss: {fmt_float(cal_res['log_loss'])}")
lines.append("")
# Abstention curves
lines.append("## 12. Abstention curves")
lines.append("")
lines.append("### Experiment A (all dev)")
lines.append(f"{'threshold':>10} {'n_accepted':>11} {'coverage':>10} {'accuracy':>10} {'macro_f1':>10} {'false_act':>10}")
lines.append("-" * 62)
for curve in cal_all["abstention_curves"]:
lines.append(f"{curve['threshold']:>10.2f} {curve['n_accepted']:>11} {fmt_pct(curve['coverage']):>10} {fmt_pct(curve['accuracy']):>10} {fmt_float(curve['macro_f1']):>10} {curve['false_action_count']:>10}")
lines.append("")
lines.append("### Experiment B (residual only)")
lines.append(f"{'threshold':>10} {'n_accepted':>11} {'coverage':>10} {'accuracy':>10} {'macro_f1':>10} {'false_act':>10}")
lines.append("-" * 62)
for curve in cal_res["abstention_curves"]:
lines.append(f"{curve['threshold']:>10.2f} {curve['n_accepted']:>11} {fmt_pct(curve['coverage']):>10} {fmt_pct(curve['accuracy']):>10} {fmt_float(curve['macro_f1']):>10} {curve['false_action_count']:>10}")
lines.append("")
# Action threshold
lines.append("## 13. Action-threshold curve")
lines.append("")
lines.append("### Experiment A")
lines.append(f"{'threshold':>10} {'action_P':>10} {'action_R':>10} {'false_act':>10}")
lines.append("-" * 40)
for curve in all_results["action_threshold"]:
lines.append(f"{curve['threshold']:>10.2f} {fmt_float(curve['action_precision']):>10} {fmt_float(curve['action_recall']):>10} {curve['false_action_count']:>10}")
lines.append("")
lines.append("### Experiment B")
lines.append(f"{'threshold':>10} {'action_P':>10} {'action_R':>10} {'false_act':>10}")
lines.append("-" * 40)
for curve in residual_results["action_threshold"]:
lines.append(f"{curve['threshold']:>10.2f} {fmt_float(curve['action_precision']):>10} {fmt_float(curve['action_recall']):>10} {curve['false_action_count']:>10}")
lines.append("")
# Model artifact size
lines.append("## 14. Model artifact size and runtime cost")
lines.append("")
# Logistic regression: 6 classes × 384 features + 6 biases = 2310 parameters
n_params = len(ROUTES) * meta["dimension"] + len(ROUTES)
serialized_bytes = n_params * 4 # float32
lines.append(f"- Trainable parameters: {n_params}")
lines.append(f" - {len(ROUTES)} classes × {meta['dimension']} features = {len(ROUTES) * meta['dimension']} weights")
lines.append(f" - {len(ROUTES)} bias terms")
lines.append(f"- Serialized head size: {serialized_bytes} bytes ({serialized_bytes / 1024:.1f} KB)")
lines.append(f"- Additional inference FLOPs: {len(ROUTES) * meta['dimension']} multiply-accumulates")
lines.append(f"- Incremental cost (e5 already computed): ~{len(ROUTES) * meta['dimension']} FLOPs, <1µs")
lines.append(f"- Cost if semantic router must trigger its own e5: full ONNX inference (~{meta['dimension']} × 128 × 12 = ~590K FLOPs)")
lines.append("")
# Recommendation
lines.append("## 16. Recommendation")
lines.append("")
# Decision logic
all_f1 = metrics_all["macro_f1"]
res_f1 = metrics_res["macro_f1"]
res_acc = metrics_res["accuracy"]
if res_f1 > 0.5 and res_acc > 0.55:
verdict = "linear head sufficient"
detail = (f"Residual macro F1 of {fmt_float(res_f1)} and accuracy of {fmt_pct(res_acc)} "
f"exceed the legacy baseline (40.8% residual accuracy) by a meaningful margin. "
f"A linear head over frozen e5-small embeddings is a viable first production candidate.")
elif all_f1 > 0.5 and res_f1 < 0.5:
verdict = "need more data"
detail = (f"All-example F1 ({fmt_float(all_f1)}) is acceptable but residual-only F1 "
f"({fmt_float(res_f1)}) drops, suggesting the contrast-family examples are "
f"hard for a linear classifier. More contrastive training data may help.")
else:
verdict = "representation inadequate"
detail = (f"Neither all-example ({fmt_float(all_f1)}) nor residual-only ({fmt_float(res_f1)}) "
f"F1 reaches the minimum viable threshold. The e5-small linear separability floor "
f"is insufficient for this 6-way task. Consider a non-linear head or a different encoder.")
lines.append(f"**{verdict}**")
lines.append("")
lines.append(detail)
lines.append("")
# Commit hash
lines.append("## 17. Commit hash for experiment tooling")
lines.append("")
import subprocess
try:
result = subprocess.run(
["git", "rev-parse", "HEAD"],
capture_output=True, text=True, cwd="/home/kami/apps/Maven"
)
lines.append(f"`{result.stdout.strip()}`")
except Exception:
lines.append("(unable to determine)")
lines.append("")
return "\n".join(lines)
# ─── Main ───────────────────────────────────────────────────────────────────
def run_experiment(X, y, fold_ids, examples_meta, label):
"""Run the full experiment pipeline for one population."""
print(f"\n{'='*60}")
print(f" Experiment: {label}")
print(f" Samples: {len(y)} Features: {X.shape[1]} Folds: {len(set(fold_ids))}")
print(f"{'='*60}")
# 1. Grouped CV with regularization search
best_C, results_by_C = grouped_cv_experiment(X, y, fold_ids, C_VALUES, examples_meta)
# 2. Use best C to get full metrics from OOF predictions
oof_rows = results_by_C[best_C]["oof_predictions"]
oof_true = np.array([r["true"] for r in oof_rows])
oof_pred = np.array([r["predicted"] for r in oof_rows])
full_metrics = compute_full_metrics(oof_true, oof_pred)
# 3. Calibration
calibration = compute_calibration(oof_true, oof_pred, oof_rows)
# 4. Action threshold
action_threshold = compute_action_threshold(oof_rows)
# 5. Contrast family
contrast_family = compute_contrast_family(oof_rows)
return {
"best_C": best_C,
"results_by_C": results_by_C,
"full_metrics": full_metrics,
"calibration": calibration,
"action_threshold": action_threshold,
"contrast_family": contrast_family,
"oof_rows": oof_rows,
}
if __name__ == "__main__":
EMBEDDING_PATH = "/tmp/mvn-experiment/embeddings.json"
REPORT_PATH = "/tmp/mvn-experiment/report.md"
# Load data
meta, examples = load_embeddings(EMBEDDING_PATH)
print(f"Loaded {len(examples)} examples, embedder={meta['embedder_id']}, dim={meta['dimension']}")
# Development pool only
dev_examples = filter_dev_pool(examples)
print(f"Development pool: {len(dev_examples)} examples")
# Extract features
X_all, y_all = extract_Xy(dev_examples)
fold_ids_all = get_fold_groups(dev_examples)
# Experiment A: all dev examples
all_results = run_experiment(X_all, y_all, fold_ids_all, dev_examples, "All development examples")
# Experiment B: residual only
dev_residual = filter_residual(dev_examples)
X_res, y_res = extract_Xy(dev_residual)
fold_ids_res = get_fold_groups(dev_residual)
residual_results = run_experiment(X_res, y_res, fold_ids_res, dev_residual, "Router-residual only")
# Generate report
report = generate_report(
meta, all_results, residual_results,
all_results["oof_rows"], residual_results["oof_rows"],
)
with open(REPORT_PATH, "w") as f:
f.write(report)
print(f"\nReport written to {REPORT_PATH}")
# Also print summary
print("\n" + "="*60)
print(" SUMMARY")
print("="*60)
print(f" All-example: acc={fmt_pct(all_results['full_metrics']['accuracy'])} macro_f1={fmt_float(all_results['full_metrics']['macro_f1'])} false_action={all_results['full_metrics']['false_action_count']}")
print(f" Residual: acc={fmt_pct(residual_results['full_metrics']['accuracy'])} macro_f1={fmt_float(residual_results['full_metrics']['macro_f1'])} false_action={residual_results['full_metrics']['false_action_count']}")
print(f" Best C (all): {all_results['best_C']}")
print(f" Best C (res): {residual_results['best_C']}")
print(f" ECE (all): {fmt_float(all_results['calibration']['ece'])}")
print(f" ECE (res): {fmt_float(residual_results['calibration']['ece'])}")
print(f" Brier (all): {fmt_float(all_results['calibration']['brier'])}")
print(f" Brier (res): {fmt_float(residual_results['calibration']['brier'])}")
+273
View File
@@ -0,0 +1,273 @@
// semantic-router-experiment computes embeddings for the semantic coarse-route
// corpus using the deployed multilingual-e5-small ONNX model. It outputs a
// JSON file containing every corpus row with its embedding vector, split
// assignment, and fold membership for grouped cross-validation.
//
// Usage:
//
// MAVEN_ONNX_LIB=/path/to/libonnxruntime.so \
// go run ./cmd/semantic-router-experiment/ -out embeddings.json
package main
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"flag"
"fmt"
"log"
"os"
"sort"
"strings"
"github.com/kami/maven/internal/router"
"github.com/kami/maven/internal/router/semantic"
)
// CachedRow is one corpus row with its precomputed embedding and split metadata.
type CachedRow struct {
Text string `json:"text"`
Route string `json:"route"`
Source string `json:"source"`
SourceID string `json:"source_id"`
SplitGroup string `json:"split_group"`
Tags []string `json:"tags,omitempty"`
FastPathResolved bool `json:"fast_path_resolved"`
RouterResidual *bool `json:"router_residual,omitempty"`
TextHash string `json:"text_hash"`
Embedding []float32 `json:"embedding"`
EmbedderID string `json:"embedder_id"`
FrozenHoldout bool `json:"frozen_holdout"`
CVFold int `json:"cv_fold"`
DevPool bool `json:"dev_pool"`
FamilyID string `json:"family_id"`
}
// ExperimentMeta carries metadata about the experiment run.
type ExperimentMeta struct {
EmbedderID string `json:"embedder_id"`
ModelPath string `json:"model_path"`
TokenizerPath string `json:"tokenizer_path"`
Dimension int `json:"dimension"`
Pooling string `json:"pooling"`
Normalization string `json:"normalization"`
InputTemplate string `json:"input_template"`
TotalExamples int `json:"total_examples"`
FrozenCount int `json:"frozen_count"`
DevCount int `json:"dev_count"`
CVFolds int `json:"cv_folds"`
FoldComposition map[int]FoldStats `json:"fold_composition"`
RouteCounts map[string]int `json:"route_counts"`
SourceCounts map[string]int `json:"source_counts"`
FastPathCount int `json:"fast_path_count"`
ResidualCount int `json:"residual_count"`
HoldoutHash string `json:"holdout_hash"`
}
// FoldStats describes one CV fold.
type FoldStats struct {
EvalCount int `json:"eval_count"`
TrainCount int `json:"train_count"`
Routes map[string]int `json:"eval_routes"`
}
func main() {
outPath := flag.String("out", "embeddings.json", "output JSON path")
folds := flag.Int("folds", 5, "number of CV folds")
flag.Parse()
libPath := os.Getenv("MAVEN_ONNX_LIB")
if libPath == "" {
log.Fatal("MAVEN_ONNX_LIB must be set to the libonnxruntime.so path")
}
// Resolve model paths relative to the module root (cwd when running with go run).
modelPath := "models/embedder/multilingual-e5-small/model_quantized.onnx"
tokPath := "models/embedder/multilingual-e5-small/tokenizer.json"
for _, p := range []string{libPath, modelPath, tokPath} {
if _, err := os.Stat(p); err != nil {
log.Fatalf("missing %s: %v", p, err)
}
}
// Load corpus.
exs, err := semantic.LoadCorpus()
if err != nil {
log.Fatalf("load corpus: %v", err)
}
fmt.Fprintf(os.Stderr, "corpus: %d examples\n", len(exs))
// Initialize embedder.
emb, err := router.NewONNXEmbedder(modelPath, tokPath, libPath)
if err != nil {
log.Fatalf("init embedder: %v", err)
}
defer emb.Close()
fmt.Fprintf(os.Stderr, "embedder: %s (dim=%d)\n", emb.ID(), emb.Dim())
// Compute frozen holdout / dev pool split.
_, devPool, holdoutHash := semantic.FrozenHoldoutSplit(exs)
fmt.Fprintf(os.Stderr, "frozen holdout: hash=%s, dev pool: %d examples\n", holdoutHash, len(devPool))
// Compute grouped CV folds on dev pool only.
cvFolds := semantic.GroupedCVFolds(devPool, *folds)
fmt.Fprintf(os.Stderr, "cv folds: %d\n", len(cvFolds))
// Build a lookup: source_id → cv_fold (from dev pool only).
foldLookup := make(map[string]int)
for _, f := range cvFolds {
for _, e := range f.Eval {
foldLookup[e.SourceID] = f.Fold
}
}
// Build dev set membership lookup.
_, devSet, _ := semantic.FrozenHoldoutSplit(exs)
devIDs := make(map[string]bool)
for _, e := range devSet {
devIDs[e.SourceID] = true
}
// Embed all examples.
ctx := context.Background()
var cached []CachedRow
foldComp := make(map[int]*FoldStats)
for i := 0; i < *folds; i++ {
foldComp[i] = &FoldStats{Routes: make(map[string]int)}
}
routeCounts := make(map[string]int)
sourceCounts := make(map[string]int)
fpCount, resCount := 0, 0
for i, e := range exs {
textHash := sha256.Sum256([]byte(e.Text))
embedding, err := emb.EmbedQuery(ctx, e.Text)
if err != nil {
log.Fatalf("embed row %d (%s): %v", i, e.SourceID, err)
}
inDev := devIDs[e.SourceID]
fold := -1
if inDev {
if f, ok := foldLookup[e.SourceID]; ok {
fold = f
}
}
isFrozen := !inDev
cr := CachedRow{
Text: e.Text,
Route: string(e.Route),
Source: e.Source,
SourceID: e.SourceID,
SplitGroup: e.SplitGroup,
Tags: e.Tags,
FastPathResolved: e.FastPathResolved,
RouterResidual: e.RouterResidual,
TextHash: hex.EncodeToString(textHash[:]),
Embedding: embedding,
EmbedderID: emb.ID(),
FrozenHoldout: isFrozen,
CVFold: fold,
DevPool: inDev,
FamilyID: e.SplitGroup,
}
cached = append(cached, cr)
routeCounts[cr.Route]++
sourceCounts[cr.Source]++
if e.FastPathResolved {
fpCount++
} else {
resCount++
}
if inDev && fold >= 0 {
foldComp[fold].EvalCount++
foldComp[fold].Routes[cr.Route]++
}
}
// Compute train counts per fold.
for i := 0; i < *folds; i++ {
foldComp[i].TrainCount = len(devPool) - foldComp[i].EvalCount
}
// Sort route counts for deterministic output.
sortedRoutes := make([]string, 0, len(routeCounts))
for r := range routeCounts {
sortedRoutes = append(sortedRoutes, r)
}
sort.Strings(sortedRoutes)
sortedRouteCounts := make(map[string]int)
for _, r := range sortedRoutes {
sortedRouteCounts[r] = routeCounts[r]
}
// Build fold stats with sorted keys.
finalFoldComp := make(map[int]FoldStats)
for i := 0; i < *folds; i++ {
finalFoldComp[i] = *foldComp[i]
}
meta := ExperimentMeta{
EmbedderID: emb.ID(),
ModelPath: modelPath,
TokenizerPath: tokPath,
Dimension: emb.Dim(),
Pooling: "mean-pool + L2-normalize",
Normalization: "L2",
InputTemplate: "query: <text>",
TotalExamples: len(exs),
FrozenCount: len(exs) - len(devPool),
DevCount: len(devPool),
CVFolds: *folds,
FoldComposition: finalFoldComp,
RouteCounts: sortedRouteCounts,
SourceCounts: sourceCounts,
FastPathCount: fpCount,
ResidualCount: resCount,
HoldoutHash: holdoutHash,
}
// Output.
output := map[string]any{
"meta": meta,
"examples": cached,
}
data, err := json.MarshalIndent(output, "", " ")
if err != nil {
log.Fatalf("marshal: %v", err)
}
if err := os.WriteFile(*outPath, data, 0644); err != nil {
log.Fatalf("write %s: %v", *outPath, err)
}
// Print summary.
fmt.Fprintf(os.Stderr, "\n=== experiment metadata ===\n")
fmt.Fprintf(os.Stderr, "embedder: %s\n", meta.EmbedderID)
fmt.Fprintf(os.Stderr, "dimension: %d\n", meta.Dimension)
fmt.Fprintf(os.Stderr, "total: %d frozen: %d dev: %d\n", meta.TotalExamples, meta.FrozenCount, meta.DevCount)
fmt.Fprintf(os.Stderr, "fast-path: %d residual: %d\n", meta.FastPathCount, meta.ResidualCount)
fmt.Fprintf(os.Stderr, "routes: %s\n", formatMap(sortedRouteCounts))
fmt.Fprintf(os.Stderr, "fold composition:\n")
for i := 0; i < *folds; i++ {
fs := finalFoldComp[i]
fmt.Fprintf(os.Stderr, " fold %d: eval=%d train=%d routes=%s\n",
i, fs.EvalCount, fs.TrainCount, formatMap(fs.Routes))
}
fmt.Fprintf(os.Stderr, "output: %s (%d bytes)\n", *outPath, len(data))
}
func formatMap(m map[string]int) string {
var parts []string
for k, v := range m {
parts = append(parts, fmt.Sprintf("%s=%d", k, v))
}
sort.Strings(parts)
return "{" + strings.Join(parts, ", ") + "}"
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,665 @@
#!/usr/bin/env python3
"""
Slice 18: Sparse Lexical Action/Non-Action Gate
================================================
Answer: can Maven reliably distinguish executable requests from semantically
similar non-actions using lexical/local-order features alone?
Representations under test (all frozen-population, no e5):
A. word 1-2 grams, TF-IDF
B. character 3-5 grams, TF-IDF (Unicode, no transliteration)
C. [word ; char] combined TF-IDF
Population reused from slices 15-17: development corpus v2 (dev_pool), router
labels, SplitGroup, cv_fold, tags. The e5 embedding vectors are ignored.
"""
import io
import json
import re
import sys
import time
import unicodedata
import warnings
from collections import Counter, defaultdict
import numpy as np
from sklearn.exceptions import ConvergenceWarning
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (
accuracy_score,
average_precision_score,
confusion_matrix,
f1_score,
precision_recall_fscore_support,
roc_auc_score,
)
from sklearn.pipeline import make_pipeline
from scipy import sparse
warnings.filterwarnings("ignore", category=ConvergenceWarning)
warnings.filterwarnings("ignore", category=UserWarning)
EMBEDDING_PATH = "/tmp/mvn-experiment/embeddings.json"
# Present generator/contrast families in the v2 dev pool (for leave-family-out)
PRESENT_FAMILIES = [
"polite_request",
"modal_request",
"first_person_request",
"reordered_target",
"capability_question",
"question",
]
FAMILY_ALIASES = {
"direct_imperative": "direct_imperative",
"polite_request": "polite_request",
"modal_request": "modal_request",
"first_person_request": "first_person_request",
"reordered_target": "reordered_target",
"capability_question": "capability_question",
"question": "question",
}
# ─── NormalizeMatchText (replicated from internal/router/matchtext.go) ─────
def normalize_match_text(s: str) -> str:
"""NFKC → lowercase → collapse Unicode whitespace. Keeps punctuation, ё."""
out = unicodedata.normalize("NFKC", s).strip().lower()
out = re.sub(r"\s+", " ", out)
return out
# ─── Data Loading ───────────────────────────────────────────────────────────
def load_data():
with open(EMBEDDING_PATH) as f:
data = json.load(f)
return data["meta"], data["examples"]
def filter_dev_pool(examples):
return [e for e in examples if e["dev_pool"]]
def residual_only(examples):
return [e for e in examples if not e["fast_path_resolved"]]
def fmt_pct(v, d=1):
return f"{100*v:.{d}f}%"
def ff(v, d=3):
return f"{v:.{d}f}"
def strip_punct(text: str) -> str:
"""Remove all punctuation (shared with slice 16/17 apply_voice_stress)."""
t = re.sub(r"[?.!,;:]+$", "", text.strip())
t = re.sub(r"[^\w\s]", "", t)
t = t.lower()
t = re.sub(r"\s+", " ", t).strip()
return t
# ─── Feature Builders ───────────────────────────────────────────────────────
def build_features(texts, kind):
"""Build a TF-IDF matrix for the given representation kind.
kind in {'word','char','both'}. Returns (X_sparse, vectorizer)."""
if kind == "word":
vec = TfidfVectorizer(
ngram_range=(1, 2), analyzer="word",
strip_accents=None, lowercase=False,
min_df=2, sublinear_tf=True,
)
elif kind == "char":
# preserve case (already lowered) and identity of missing diacritics;
# token_pattern null => char analyzer
vec = TfidfVectorizer(
ngram_range=(3, 5), analyzer="char",
strip_accents=None, lowercase=False,
min_df=2, sublinear_tf=True,
)
elif kind == "both":
vec_word = TfidfVectorizer(
ngram_range=(1, 2), analyzer="word",
strip_accents=None, lowercase=False, min_df=2, sublinear_tf=True,
)
vec_char = TfidfVectorizer(
ngram_range=(3, 5), analyzer="char",
strip_accents=None, lowercase=False, min_df=2, sublinear_tf=True,
)
Xw = vec_word.fit_transform(texts)
Xc = vec_char.fit_transform(texts)
X = sparse.hstack([Xw, Xc]).tocsr()
return X, ("both", vec_word, vec_char)
X = vec.fit_transform(texts)
return X, vec
def vocab_size(vectorizer):
if isinstance(vectorizer, tuple):
_, vw, vc = vectorizer
return vw.get_feature_names_out().shape[0] + vc.get_feature_names_out().shape[0]
return vectorizer.get_feature_names_out().shape[0]
def transform_texts(texts, vectorizer):
"""Apply an already-fitted vectorizer (handles the 2-tuple 'both' case)."""
if isinstance(vectorizer, tuple):
_, vw, vc = vectorizer
Xw = vw.transform(texts)
Xc = vc.transform(texts)
return sparse.hstack([Xw, Xc]).tocsr()
return vectorizer.transform(texts)
# ─── Grouped CV ─────────────────────────────────────────────────────────────
def run_binary_grouped_cv(X, y, fold_ids, C=1.0):
"""Grouped CV for binary action vs not_action. Returns OOF rows + fold metrics."""
yb = np.array([1 if t == "action" else 0 for t in y])
fold_ids = np.asarray(fold_ids)
unique_folds = sorted(set(fold_ids.tolist()))
oof_rows = []
fold_metrics = []
for test_fold in unique_folds:
tr = fold_ids != test_fold
te = fold_ids == test_fold
clf = LogisticRegression(C=C, max_iter=2000, solver="lbfgs", random_state=42)
clf.fit(X[tr], yb[tr])
proba = clf.predict_proba(X[te])[:, 1]
pred = (proba >= 0.5).astype(int)
yt = yb[te]
fp = int(((yt == 0) & (pred == 1)).sum())
fn = int(((yt == 1) & (pred == 0)).sum())
tp = int(((yt == 1) & (pred == 1)).sum())
tn = int(((yt == 0) & (pred == 0)).sum())
roc = roc_auc_score(yt, proba) if len(np.unique(yt)) > 1 else 0.0
pr = average_precision_score(yt, proba) if len(np.unique(yt)) > 1 else 0.0
acc = accuracy_score(yt, pred)
prec = tp / max(tp + fp, 1)
rec = tp / max(tp + fn, 1)
fold_metrics.append({
"fold": int(test_fold), "n": int(len(yt)),
"roc_auc": roc, "pr_auc": pr,
"action_precision": prec, "action_recall": rec,
"fp": fp, "fn": fn, "tp": tp, "tn": tn,
"acc": acc,
})
te_idx = np.where(te)[0]
for i in range(len(yt)):
oof_rows.append({
"fold": int(test_fold),
"proba": float(proba[i]),
"pred": int(pred[i]),
"true": int(yt[i]),
})
return oof_rows, fold_metrics
# ─── Metrics from OOF ───────────────────────────────────────────────────────
def binary_metrics_from_oof(oof_rows):
yt = np.array([r["true"] for r in oof_rows])
yp = np.array([r["pred"] for r in oof_rows])
proba = np.array([r["proba"] for r in oof_rows])
n = len(yt)
tp = int(((yt == 1) & (yp == 1)).sum())
fp = int(((yt == 0) & (yp == 1)).sum())
fn = int(((yt == 1) & (yp == 0)).sum())
prec = tp / max(tp + fp, 1)
rec = tp / max(tp + fn, 1)
roc = roc_auc_score(yt, proba) if len(np.unique(yt)) > 1 else 0.0
pr = average_precision_score(yt, proba) if len(np.unique(yt)) > 1 else 0.0
return {
"n": n, "tp": tp, "fp": fp, "fn": fn,
"action_precision": prec, "action_recall": rec,
"fa_rate": fp / max(n, 1),
"roc_auc": roc, "pr_auc": pr,
}
def threshold_curve(oof_rows, thresholds):
yt = np.array([r["true"] for r in oof_rows])
proba = np.array([r["proba"] for r in oof_rows])
rows = []
for thr in thresholds:
yp = (proba >= thr).astype(int)
tp = int(((yt == 1) & (yp == 1)).sum())
fp = int(((yt == 0) & (yp == 1)).sum())
fn = int(((yt == 1) & (yp == 0)).sum())
rows.append({
"threshold": round(float(thr), 4),
"action_precision": round(tp / max(tp + fp, 1), 4),
"action_recall": round(tp / max(tp + fn, 1), 4),
"fa_count": fp,
"fa_rate": round(fp / max(len(yt), 1), 4),
})
return rows
# ─── Leave-Family-Out ───────────────────────────────────────────────────────
def run_leave_family_out(texts, y, family_per_row, family, kind):
"""Train without `family`, evaluate on `family` only."""
mask_members = family_per_row == family
if mask_members.sum() == 0:
return None
only_family = (mask_members).astype(bool)
train_idx = np.where(~only_family)[0]
test_idx = np.where(only_family)[0]
X, _ = build_features([texts[i] for i in train_idx], kind)
# map test rows onto the full vocabulary
tr_texts = [texts[i] for i in train_idx]
te_texts = [texts[i] for i in test_idx]
all_texts = tr_texts + te_texts
Xall, _ = build_features(all_texts, kind)
Xtr = Xall[:len(tr_texts)]
Xte = Xall[len(tr_texts):]
ytr = np.array([1 if y[i] == "action" else 0 for i in train_idx])
yte = np.array([1 if y[i] == "action" else 0 for i in test_idx])
clf = LogisticRegression(C=1.0, max_iter=2000, solver="lbfgs", random_state=42)
clf.fit(Xtr, ytr)
pred = clf.predict(Xte)
tp = int(((yte == 1) & (pred == 1)).sum())
fp = int(((yte == 0) & (pred == 1)).sum())
fn = int(((yte == 1) & (pred == 0)).sum())
prec = tp / max(tp + fp, 1)
rec = tp / max(tp + fn, 1)
acc = accuracy_score(yte, pred)
return {
"family": family, "rows": int(len(yte)),
"action_precision": prec, "action_recall": rec,
"fa_count": fp, "acc": acc,
"yticks": f"pos={int(yte.sum())} neg={int((yte==0).sum())}",
}
# ─── Paired Action/Capability Test ──────────────────────────────────────────
def paired_action_capability(texts, y, tags_per_row_by_idx, kind, C=1.0):
"""For the best sparse representation, evaluate pairwise ordering."""
idx = list(range(len(texts)))
emb = build_features(texts, kind)[0]
clf = LogisticRegression(C=C, max_iter=2000, solver="lbfgs", random_state=42)
yb = np.array([1 if t == "action" else 0 for t in y])
# use OOF-style: fit on full then derive? We report the pairwise score test;
# to avoid leakage we use grouped OOF proba via grouped cv.
return None
# ─── Main ──────────────────────────────────────────────────────────────────
def main():
meta, examples = load_data()
dev = filter_dev_pool(examples)
dev_res = residual_only(dev)
# normalized texts
for e in dev:
e["n_text"] = normalize_match_text(e["text"])
texts = [e["n_text"] for e in dev]
y = [e["route"] for e in dev]
folds = [e["cv_fold"] for e in dev]
tags = [set(e.get("tags", [])) for e in dev]
split_groups = [e["split_group"] for e in dev]
source_ids = [e["source_id"] for e in dev]
n_action = sum(1 for r in y if r == "action")
n_not = len(y) - n_action
print(f"Dev pool: {len(dev)} action={n_action} not_action={n_not}")
print()
results = {}
# ── 1. Build features and run grouped CV for each representation ──────
for kind in ["word", "char", "both"]:
print(f"\n=== {kind} TF-IDF ===")
t0 = time.time()
X, vec = build_features(texts, kind)
build_t = time.time() - t0
vs = vocab_size(vec)
print(f" vocab={vs} X.shape={X.shape} build={build_t:.2f}s")
oof, folds_m = run_binary_grouped_cv(X, y, folds, C=1.0)
m = binary_metrics_from_oof(oof)
results[kind] = {
"vs": vs, "build_t": build_t, "oof": oof, "fold_metrics": folds_m,
"metrics": m, "X": X, "vec": vec,
"texts": texts, "y": y, "folds": folds,
}
print(f" ROC={ff(m['roc_auc'])} PR={ff(m['pr_auc'])} P={ff(m['action_precision'])} "
f"R={ff(m['action_recall'])} FA={m['fp']} ({fmt_pct(m['fa_rate'])})")
# fold-level
for fm in folds_m:
print(f" fold {fm['fold']}: ROC={ff(fm['roc_auc'])} PR={ff(fm['pr_auc'])} "
f"P={ff(fm['action_precision'])} R={ff(fm['action_recall'])} "
f"FP={fm['fp']} FN={fm['fn']} n={fm['n']}")
# pick best by PR-AUC
best_kind = max(["word", "char", "both"], key=lambda k: results[k]["metrics"]["pr_auc"])
print(f"\nBest representation by PR-AUC: {best_kind}")
# ── 2. Leave-family-out for best kind ─────────────────────────────────
print(f"\n=== Leave-family-out ({best_kind}) ===")
fam_per_row = []
for tg in tags:
fam = None
# prefer the more specific contrast/request families first (a single
# utterance may carry several generator tags, e.g. capability_question
# plus direct_imperative). Check the informative ones before the
# generic direct_imperative fallback.
for f in ["capability_question", "question", "first_person_request",
"modal_request", "polite_request", "reordered_target",
"direct_imperative"]:
if f in tg:
fam = f
break
fam_per_row.append(fam)
fam_per_row = np.array(fam_per_row, dtype=object)
lfo = {}
for fam in PRESENT_FAMILIES:
r = run_leave_family_out(texts, y, fam_per_row, fam, best_kind)
if r is None:
print(f" {fam}: (no rows)")
continue
lfo[fam] = r
print(f" {fam}: rows={r['rows']} ({r['yticks']}) P={ff(r['action_precision'])} "
f"R={ff(r['action_recall'])} FA={r['fa_count']} acc={fmt_pct(r['acc'])}")
# ── 3. Action/capability paired test ─────────────────────────────────
print(f"\n=== Paired action/capability test ({best_kind}) ===")
# Use grouped-CV OOF proba for ordering (no leakage)
oof = results[best_kind]["oof"]
# map oof rows back by source_id order
# oof rows are appended per fold in dev order; reconstruct
# We'll re-embed and get proba via grouped CV with proba recorded per row.
# Re-run grouped CV capturing per-row proba aligned to dev indices.
X = results[best_kind]["X"]
yb = np.array([1 if r == "action" else 0 for r in y])
folds_arr = np.array(folds)
dev_proba = np.zeros(len(dev))
for te_fold in sorted(set(folds)):
tr = folds_arr != te_fold
te = folds_arr == te_fold
clf = LogisticRegression(C=1.0, max_iter=2000, solver="lbfgs", random_state=42)
clf.fit(X[tr], yb[tr])
dev_proba[te] = clf.predict_proba(X[te])[:, 1]
# group action seeds: pair each capability-question row with action rows
# sharing the same target object noun and the same act (turn_on/turn_off/…).
# Both were generated over a common entity+event inventory, so the object
# lexeme is the semantic link between a knowledge question and its
# executable sibling.
DEVICES = [
"свет", "люстру", "люстра", "жалюзи", "вытяжку", "вытяжка",
"вентилятор", "кондиционер", "телевизор", "лампу", "лампа",
"музыку", "музыка", "плеер", "колонку", "колонки", "чайник",
"бойлер", "обогреватель", "пылесос", "пылесосом", "пол",
"поливалки", "полив", "арка", "шторы", "штору", "штору",
"динамики", "дверь", "двери", "замок", "гараж", "ворота",
"кофе", "пасту", "зубы", "крючки", "лаймо", "куртку",
"будильник", "таймер", "напоминание", "расписание",
]
def object_nouns(t):
found = set()
tl = t.lower()
for d in DEVICES:
# match as standalone word (handle Russian case endings loosely via prefix)
if re.search(r"\b" + re.escape(d), tl):
found.add(d)
return found
# verb/act family per row from source_id (e.g. ha-light-off -> off)
def act_family(src):
# pull the 3rd token-ish: ha-light-off => 'off'; ha-light-on => 'on'
m = re.search(r"^(\w+)-([a-z_]+)-([a-zA-Z_]+)", src)
if m:
return f"{m.group(1)}-{m.group(2)}-{m.group(3)}"
# generic fallback
return src.split("-")[0]
cap_rows = [i for i in range(len(dev)) if "capability_question" in tags[i]]
act_idxs = [i for i in range(len(dev)) if y[i] == "action"]
# For each capability row, candidate sibling actions: same object noun
# AND same act (turn_on vs turn_off), i.e. same domain+object. We accept
# any action row sharing an object and matching the on/off sense if present.
pairs = []
for cidx in cap_rows:
c_obj = object_nouns(texts[cidx])
if not c_obj:
continue
# cap rows are kq-cap-<domain>-<n>; domain token after 'kq-cap-'
dom = re.search(r"kq-cap-([^-]+)", source_ids[cidx])
dom = dom.group(1) if dom else None
for aidx in act_idxs:
a_obj = object_nouns(texts[aidx])
if not (c_obj & a_obj):
continue
# require same home domain when both carry one
a_dom = re.search(r"^([a-z]+)-", source_ids[aidx])
a_dom = a_dom.group(1) if a_dom else None
if dom and a_dom and dom != a_dom:
continue
pairs.append((cidx, aidx))
# keep it bounded: cap row pairs with many actions (one per room); that's fine
order_ok = 0
margins = []
reversed_pairs = []
for cidx, aidx in pairs:
pc = dev_proba[cidx]
pa = dev_proba[aidx]
margins.append(pa - pc)
if pa > pc:
order_ok += 1
else:
reversed_pairs.append((texts[cidx][:40], pc, texts[aidx][:40], pa))
if pairs:
order_acc = order_ok / len(pairs)
margins_arr = np.array(margins)
print(f" pairs={len(pairs)} order_acc={ff(order_acc)} mean_margin={ff(margins_arr.mean())} "
f"median_margin={ff(np.median(margins_arr))} reversed={len(reversed_pairs)}")
for rev in reversed_pairs[:12]:
print(f" REV: cap '{rev[0]}' P={rev[1]:.3f} < act '{rev[2]}' P={rev[3]:.3f}")
else:
order_acc = None
print(" (no matched pairs)")
paired = {
"pairs": len(pairs), "order_acc": order_acc,
"mean_margin": float(np.mean(margins)) if margins else None,
"median_margin": float(np.median(margins)) if margins else None,
"reversed": len(reversed_pairs),
}
# ── 4. Voice stress scoring ───────────────────────────────────────────
print(f"\n=== Voice-like stress ({best_kind}) ===")
# Build stress variants, embed, score with a model trained on normal text
# Train one model on full dev (normal punctuation); score stressed variants.
clf_full = LogisticRegression(C=1.0, max_iter=2000, solver="lbfgs", random_state=42)
clf_full.fit(X, yb)
stress_texts = []
stress_meta = [] # (orig_index, mode)
for i in range(len(dev)):
orig = texts[i]
s_nofinal = re.sub(r"[?.!,;:]+$", "", orig)
s_all = strip_punct(dev[i]["text"])
if s_nofinal != orig:
stress_texts.append(s_nofinal); stress_meta.append((i, "no_final"))
if s_all != orig:
stress_texts.append(s_all); stress_meta.append((i, "all"))
if stress_texts:
# reuse the already-fitted best-kind vectorizer to keep the feature space
# identical to what the full model was trained on.
X_stress = transform_texts(stress_texts, results[best_kind]["vec"])
proba_stress = clf_full.predict_proba(X_stress)[:, 1]
else:
proba_stress = []
# classify ability: capability-question false-action rate and modal-action recall
stress_by_mode = defaultdict(list)
for (i, mode), p in zip(stress_meta, proba_stress):
stress_by_mode[mode].append((i, p, y[i], tags[i]))
voice = {}
for mode, rows in stress_by_mode.items():
cap_q = [p for (i, p, lbl, tg) in rows if "capability_question" in tg]
cap_fa = sum(1 for p in cap_q if p >= 0.5)
mod_act = [p for (i, p, lbl, tg) in rows if lbl == "action" and ("polite_request" in tg or "modal_request" in tg)]
mod_rec = sum(1 for p in mod_act if p >= 0.5) / max(len(mod_act), 1)
voice[mode] = {
"n": len(rows),
"cap_q_fa": cap_fa / max(len(cap_q), 1), "cap_q_n": len(cap_q),
"modal_action_recall": mod_rec, "modal_n": len(mod_act),
}
print(f" {mode}: n={len(rows)} cap_q_FA={fmt_pct(voice[mode]['cap_q_fa'])} ({voice[mode]['cap_q_n']}) "
f"modal_recall={ff(voice[mode]['modal_action_recall'])} ({voice[mode]['modal_n']})")
# ── 5. Punctuation ablation ───────────────────────────────────────────
print(f"\n=== Punctuation ablation ({best_kind}) ===")
texts_stripped = [strip_punct(dev[i]["text"]) for i in range(len(dev))]
X_stripped_train, vec_stripped = build_features(texts_stripped, best_kind)
clf_stripped = LogisticRegression(C=1.0, max_iter=2000, solver="lbfgs", random_state=42)
clf_stripped.fit(X_stripped_train, yb)
# eval on stripped (in-train) and punctuated (out-of-train) via same vectorizer
X_orig = transform_texts(texts, vec_stripped)
p_orig = clf_stripped.predict_proba(X_orig)[:, 1]
p_strip = clf_stripped.predict_proba(X_stripped_train)[:, 1]
def report_ablation(proba, name):
yp = (proba >= 0.5).astype(int)
tp = int(((yb == 1) & (yp == 1)).sum())
fp = int(((yb == 0) & (yp == 1)).sum())
fn = int(((yb == 1) & (yp == 0)).sum())
prec = tp / max(tp + fp, 1); rec = tp / max(tp + fn, 1)
print(f" trained-stripped, eval {name}: P={ff(prec)} R={ff(rec)} FA={fp} ({fmt_pct(fp/len(yb))})")
report_ablation(p_orig, "punctuated")
report_ablation(p_strip, "stripped")
# ── 6. Threshold curve for best kind ──────────────────────────────────
print(f"\n=== Threshold curve ({best_kind}) ===")
oof = results[best_kind]["oof"]
thresh = np.arange(0.10, 0.995, 0.015).tolist()
tcurve = threshold_curve(oof, thresh)
print(f"{'thr':>6} {'P':>6} {'R':>6} {'FA':>5} {'FArate':>8}")
for t in tcurve:
marker = ""
if t["action_precision"] >= 0.95:
marker = " ← P>=0.95"
print(f"{t['threshold']:>6.3f} {t['action_precision']:>6.3f} {t['action_recall']:>6.3f} "
f"{t['fa_count']:>5} {t['fa_rate']:>8.4f}{marker}")
p95 = [t for t in tcurve if t["action_precision"] >= 0.95 and t["action_recall"] > 0.01]
print(f"\nP>=0.95 region: {len(p95)} points; best recall there = "
f"{max((t['action_recall'] for t in p95), default=0.0):.4f}")
# threshold curve using the (no_leak) dev proba instead of .5-threshold OOF
# OOF pred used fixed 0.5; curve above re-derives from proba. Good.
# ── 7. False-action decomposition ────────────────────────────────────
print(f"\n=== False-action decomposition ({best_kind}) ===")
# recompute OOF predictions at 0.5 from stored 'pred'
fa_by_family = Counter()
fa_by_group = Counter()
# need oof aligned to source_ids — oof stored without source id; rebuild
# Re-do grouped cv capturing source_id + tag + split_group
fake_oof = []
for te_fold in sorted(set(folds_arr)):
tr = folds_arr != te_fold; te = folds_arr == te_fold
clf = LogisticRegression(C=1.0, max_iter=2000, solver="lbfgs", random_state=42)
clf.fit(X[tr], yb[tr])
p = clf.predict_proba(X[te])[:, 1]
pr = (p >= 0.5).astype(int)
te_idx = np.where(te)[0]
for k, i in enumerate(te_idx):
fake_oof.append({
"source_id": source_ids[i], "split_group": split_groups[i],
"tags": tags[i], "true": y[i], "proba": float(p[k]),
"pred": int(pr[k]), "fold": int(te_fold),
})
def classify_semantic_family(row):
tg = row["tags"]
if "capability_question" in tg: return "capability_question"
if "question" in tg: return "ordinary_question"
if "remember" in tg or "note" in tg or "idea" in tg or "free_form" in tg: return "memory_write"
if "version" in tg or "system" in tg or "health" in tg or "status" in tg: return "system"
if "recall" in tg or "world" in tg or "definition" in tg or "aggregate" in tg: return "knowledge_general"
if "greeting" in tg or "goodbye" in tg or "thanks" in tg: return "conversation"
if row["true"].startswith("system"): return "system"
return "other"
for r in fake_oof:
if r["pred"] == 1 and r["true"] != "action":
fam = classify_semantic_family(r)
fa_by_family[fam] += 1
fa_by_group[r["split_group"]] += 1
print("By family:")
for k, v in fa_by_family.most_common():
print(f" {k}: {v}")
print("By SplitGroup (top 15):")
for k, v in fa_by_group.most_common(15):
print(f" {k}: {v}")
# ── 8. Six-way probe on best representation ──────────────────────────
print(f"\n=== Six-way probe ({best_kind}) ===")
routes = ["action", "conversation", "knowledge", "memory_write", "system", "uncertain"]
y6 = np.array(y)
# fit grouped cv 6-way
oof6 = []
fold6 = []
for te_fold in sorted(set(folds_arr)):
tr = folds_arr != te_fold; te = folds_arr == te_fold
clf = LogisticRegression(C=1.0, max_iter=3000, solver="lbfgs", random_state=42)
clf.fit(X[tr], y6[tr])
pr = clf.predict(X[te])
p6 = clf.predict_proba(X[te])
classes = clf.classes_
te_idx = np.where(te)[0]
for k, i in enumerate(te_idx):
oof6.append({
"true": y6[i], "pred": pr[k], "proba": {c: float(p6[k][j]) for j, c in enumerate(classes)},
"tags": tags[i], "split_group": split_groups[i], "source_id": source_ids[i],
})
yt6 = [r["true"] for r in oof6]; yp6 = [r["pred"] for r in oof6]
acc6 = accuracy_score(yt6, yp6)
macro6 = f1_score(yt6, yp6, average="macro", zero_division=0)
prec6, rec6, f16, sup6 = precision_recall_fscore_support(yt6, yp6, labels=routes, zero_division=0)
fa6 = sum(1 for t, p in zip(yt6, yp6) if t != "action" and p == "action")
ap6 = sum(1 for t, p in zip(yt6, yp6) if t == "action" and p == "action") / max(sum(1 for p in yp6 if p == "action"), 1)
ar6 = sum(1 for t, p in zip(yt6, yp6) if t == "action" and p == "action") / max(sum(1 for t in yt6 if t == "action"), 1)
print(f" acc={fmt_pct(acc6)} macroF1={ff(macro6)} actionP={ff(ap6)} actionR={ff(ar6)} FA={fa6} ({fmt_pct(fa6/len(yt6))})")
for i, r in enumerate(routes):
print(f" {r:<12} P={ff(prec6[i])} R={ff(rec6[i])} F1={ff(f16[i])} n={int(sup6[i])}")
# ── 9. Artifact size / runtime ───────────────────────────────────────
print(f"\n=== Artifact size / runtime ===")
for kind in ["word", "char", "both"]:
rr = results[kind]
vec = rr["vec"]
ncoef = rr["metrics"]["n"]
# non-zero coefficients = vocab (TF-IDF), logistic has 1 weight per vocab
print(f" {kind}: vocab={rr['vs']} fp32 model bytes={rr['vs']*4} "
f"build={rr['build_t']:.3f}s")
# done
print("\nDone.")
if __name__ == "__main__":
main()
@@ -0,0 +1,81 @@
#!/usr/bin/env python3
"""
Slice 19 tokenizers, derived from the development corpus only.
A. CharVocab codepoint ids over the dev corpus (deterministic order)
B. BpeVocab2048 byte-level BPE, vocab ~2048, trained on dev corpus only
Records for §4 of the brief: vocab size, OOV behaviour, serialized tokenizer size.
"""
import re
import unicodedata
from tokenizers import Tokenizer
from tokenizers.decoders import ByteLevel as ByteLevelDecoder
from tokenizers.models import BPE
from tokenizers.pre_tokenizers import ByteLevel as ByteLevelPreTokenizer
from tokenizers.trainers import BpeTrainer
class CharVocab:
"""Codepoint ids from the dev corpus, sorted by codepoint value."""
def __init__(self, texts):
chars = set()
for t in texts:
chars.update(t)
self.id_to_char = sorted(chars)
self.char_to_id = {c: i + 1 for i, c in enumerate(self.id_to_char)} # 0 = PAD
self.pad = 0
@property
def size(self):
return len(self.id_to_char) + 1
def encode(self, text, max_len):
ids = [self.char_to_id.get(c, 0) for c in text] # 0 doubles as UNK/OOV
return ids[:max_len]
class BpeVocab:
"""Byte-level BPE, trained only on the strings it is given."""
def __init__(self, texts, vocab_size=2048, sep=""):
self.tok = Tokenizer(BPE())
self.tok.pre_tokenizer = ByteLevelPreTokenizer(trim_offsets=False)
self.tok.decoder = ByteLevelDecoder()
trainer = BpeTrainer(vocab_size=vocab_size, special_tokens=["[PAD]"],
show_progress=False)
# train on the corpus *strings*, byte-level BPE handles all codepoints
self.tok.train_from_iterator(texts, trainer=trainer)
self.pad_id = self.tok.token_to_id("[PAD]")
self._vocab = self.tok.get_vocab()
self._n = len(self._vocab)
@property
def size(self):
return self._n
def encode(self, text):
return self.tok.encode(text).ids
def serialized_bytes(self):
# measure the serialized tokenizer size on disk
import os
d = self.tok.to_str()
return len(d.encode("utf-8"))
def normalize_match_text(s: str) -> str:
"""NFKC → lowercase → collapse whitespace. Punctuation kept."""
out = unicodedata.normalize("NFKC", s).strip().lower()
out = re.sub(r"\s+", " ", out)
return out
def strip_punct(text: str) -> str:
"""Remove safe punctuation from an already-normalized text."""
t = re.sub(r"[^\w\s]", " ", text)
t = re.sub(r"\s+", " ", t).strip()
return t
@@ -0,0 +1,640 @@
#!/usr/bin/env python3
"""
Slice 19: tiny sequence-sensitive pragmatics specialists (binary action gate)
=============================================================================
A narrow binary specialist: executable request vs semantically similar
non-executable utterance, generalizing across surface-generator families.
Commands
grouped grouped semantic CV (existing cv_fold), all arch/sizes, saves
per-config per-fold model checkpoints + OOF proba per variant
metrics aggregate saved grouped-CV results into the report tables
(binary metrics, threshold curves, pair ordering, stress)
lfo leave-generator-out. All sizes on capability_question; the
other present families for the leading config only.
e5baseline frozen-e5 logistic + MLP baselines: grouped, cap-Q LOFO, pair
ordering (stress flagged NA no re-embed on this box)
runtime params / sizes / latency / tokenization for each candidate
Primary metrics (brief §2): cap-Q LOFO FA, pair ordering acc, pair margin,
grouped P/R, voice stress, fold variance. Aggregate accuracy is secondary.
"""
import argparse
import json
import os
import re
import resource
import subprocess
import sys
import time
import warnings
from collections import Counter
import numpy as np
warnings.filterwarnings("ignore")
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from slice19_bpe import BpeVocab, CharVocab, normalize_match_text, strip_punct
EMBEDDING_PATH = "/tmp/mvn-experiment/embeddings.json"
RESULTS_DIR = "/tmp/mvn-s19"
SEED = 42
MAX_CHAR = 64
MAX_BPE = 24
BPE_VOCAB = 2048
PRESENT_FAMILIES = [
"polite_request",
"modal_request",
"first_person_request",
"reordered_target",
"capability_question",
"question",
]
ABSENT_FAMILIES = ["negation", "reported_speech", "quotation", "hypothetical"]
FAMILY_PRIORITY = [
"capability_question", "question", "first_person_request",
"modal_request", "polite_request", "reordered_target", "direct_imperative",
]
DEVICES = [
"свет", "люстру", "люстра", "жалюзи", "вытяжку", "вытяжка",
"вентилятор", "кондиционер", "телевизор", "лампу", "лампа",
"музыку", "музыка", "плеер", "колонку", "колонки", "чайник",
"бойлер", "обогреватель", "пылесос", "пылесосом", "пол",
"поливалки", "полив", "арка", "шторы", "штору",
"динамики", "дверь", "двери", "замок", "гараж", "ворота",
"кофе", "пасту", "зубы", "крючки", "лаймо", "куртку",
"будильник", "таймер", "напоминание", "расписание",
]
ARCH_CONFIGS = {
"char_cnn": ["tiny", "medium"],
"bigru": ["tiny", "medium", "large"],
"tiny_transformer": ["small", "medium"],
}
TRAIN_HYPER = {
"char_cnn": dict(epochs=20, lr=1e-3, bs=64, clip=None),
"bigru": dict(epochs=15, lr=5e-4, bs=64, clip=1.0),
"tiny_transformer": dict(epochs=25, lr=5e-4, bs=64, clip=1.0),
}
VARIANTS = ["orig", "nofinal", "strip"]
# ─── data ───────────────────────────────────────────────────────────────────
def load_dev():
with open(EMBEDDING_PATH) as f:
data = json.load(f)
rows = [e for e in data["examples"] if e["dev_pool"]]
out = []
for e in rows:
n_text = normalize_match_text(e["text"])
out.append({
"text_orig": n_text,
"text_nofinal": re.sub(r"[?.!,;:]+$", "", n_text),
"text_strip": strip_punct(n_text),
"route": e["route"],
"y": 1 if e["route"] == "action" else 0,
"cv_fold": e["cv_fold"],
"split_group": e["split_group"],
"tags": set(e.get("tags", [])),
"source_id": e["source_id"],
"emb": np.asarray(e["embedding"], dtype=np.float32),
})
return out
def family_of(tags):
for f in FAMILY_PRIORITY:
if f in tags:
return f
return None
def build_pairs(rows, texts):
"""Capability-question vs action-by-shared-object pairs (eval only)."""
def object_nouns(t):
found = set()
tl = t.lower()
for d in DEVICES:
if re.search(r"\b" + re.escape(d), tl):
found.add(d)
return found
pairs = []
for cidx, r in enumerate(rows):
if "capability_question" not in r["tags"]:
continue
c_obj = object_nouns(texts[cidx])
if not c_obj:
continue
dom = re.search(r"kq-cap-([^-]+)", r["source_id"])
dom = dom.group(1) if dom else None
for aidx, ra in enumerate(rows):
if ra["y"] != 1:
continue
a_obj = object_nouns(texts[aidx])
if not (c_obj & a_obj):
continue
a_dom = re.search(r"^([a-z]+)-", ra["source_id"])
a_dom = a_dom.group(1) if a_dom else None
if dom and a_dom and dom != a_dom:
continue
pairs.append((cidx, aidx))
return pairs
def pair_metrics(pairs, proba):
if not pairs:
return {"pairs": 0}
margins = []
ties = 0
order = 0
rev = 0
for cidx, aidx in pairs:
pc, pa = proba[cidx], proba[aidx]
margins.append(pa - pc)
if pa > pc:
order += 1
elif pa == pc:
ties += 1
else:
rev += 1
m = np.array(margins)
return {
"pairs": len(pairs),
"ordering_acc": order / len(pairs),
"mean_margin": float(m.mean()),
"median_margin": float(np.median(m)),
"ties": ties,
"reversed": rev,
}
# ─── tokenizers ─────────────────────────────────────────────────────────────
def build_tokenizers(rows):
nat_texts = [r["text_orig"] for r in rows]
char_vocab = CharVocab(nat_texts)
bpe = BpeVocab(nat_texts, vocab_size=BPE_VOCAB)
return char_vocab, bpe
def encode_all(rows, tokenizer, kind):
"""Return dict variant -> (N, max_len) int64 array."""
max_len = MAX_CHAR if kind == "char" else MAX_BPE
out = {}
for v in VARIANTS:
arr = np.zeros((len(rows), max_len), dtype=np.int64)
for i, r in enumerate(rows):
if kind == "char":
ids = tokenizer.encode(r[f"text_{v}"], max_len)
arr[i, :len(ids)] = ids
else:
ids = tokenizer.encode(r[f"text_{v}"])[:max_len]
arr[i, :len(ids)] = ids
out[v] = arr
return out
# ─── training ───────────────────────────────────────────────────────────────
def build_model(arch, size, vocab_size, max_len):
import torch
from slice19_models import CharCNN, BiGRU, TinyTransformer
if arch == "char_cnn":
c = (dict(embed_dim=32, filters=64, widths=[3, 4, 5]) if size == "tiny"
else dict(embed_dim=64, filters=160, widths=[2, 3, 4, 5]))
return CharCNN(vocab_size, c["embed_dim"], c["filters"], c["widths"])
if arch == "bigru":
c = (dict(embed_dim=64, hidden=64) if size == "tiny" else
(dict(embed_dim=128, hidden=128) if size == "medium" else
dict(embed_dim=256, hidden=256)))
return BiGRU(vocab_size, c["embed_dim"], c["hidden"])
c = (dict(d_model=128, n_layers=2, n_heads=4) if size == "small" else
dict(d_model=192, n_layers=4, n_heads=4))
return TinyTransformer(vocab_size, c["d_model"], c["n_layers"], c["n_heads"],
max_len=max_len)
def train_binary(X, y, arch, size, vocab_size, seed_offset=0, log=False):
import torch
torch.manual_seed(SEED + seed_offset)
np.random.seed(SEED + seed_offset)
Xt = torch.from_numpy(X)
yt = torch.from_numpy(y.astype(np.float32))
model = build_model(arch, size, vocab_size, X.shape[1])
h = TRAIN_HYPER[arch]
opt = torch.optim.AdamW(model.parameters(), lr=h["lr"], weight_decay=1e-4)
lossf = torch.nn.BCEWithLogitsLoss()
n = X.shape[0]
model.train()
t0 = time.time()
for epoch in range(h["epochs"]):
perm = torch.randperm(n)
running = 0.0
n_b = 0
for start in range(0, n, h["bs"]):
idx = perm[start:start + h["bs"]]
xb = Xt[idx]
if xb.dim() == 1:
xb = xb.unsqueeze(0)
logits = model(xb)
loss = lossf(logits, yt[idx])
opt.zero_grad()
loss.backward()
if h["clip"]:
torch.nn.utils.clip_grad_norm_(model.parameters(), h["clip"])
opt.step()
running += float(loss)
n_b += 1
if log and (epoch + 1) % 5 == 0:
print(f" epoch {epoch+1}/{h['epochs']} loss {running/max(n_b,1):.4f}")
return model, time.time() - t0
def predict_proba(model, X, bs=256):
import torch
model.eval()
out = []
with torch.no_grad():
Xt = torch.from_numpy(X)
for start in range(0, X.shape[0], bs):
xb = Xt[start:start + bs]
if xb.dim() == 1:
xb = xb.unsqueeze(0)
logits = model(xb)
out.append(torch.sigmoid(logits).numpy())
return np.concatenate(out)
# ─── metrics helpers ────────────────────────────────────────────────────────
def binary_metrics(yt, proba, thr=0.5):
yp = (proba >= thr).astype(int)
tp = int(((yt == 1) & (yp == 1)).sum())
fp = int(((yt == 0) & (yp == 1)).sum())
fn = int(((yt == 1) & (yp == 0)).sum())
from sklearn.metrics import roc_auc_score, average_precision_score
roc = roc_auc_score(yt, proba) if len(np.unique(yt)) > 1 else 0.0
pr = average_precision_score(yt, proba) if len(np.unique(yt)) > 1 else 0.0
return {
"n": int(len(yt)), "tp": tp, "fp": fp, "fn": fn,
"P": tp / max(tp + fp, 1), "R": tp / max(tp + fn, 1),
"FA": fp, "FA_rate": fp / max(len(yt), 1),
"ROC_AUC": float(roc), "PR_AUC": float(pr),
}
def threshold_curve(yt, proba, thr_grid):
rows = []
for thr in thr_grid:
yp = (proba >= thr).astype(int)
tp = int(((yt == 1) & (yp == 1)).sum())
fp = int(((yt == 0) & (yp == 1)).sum())
fn = int(((yt == 1) & (yp == 0)).sum())
P = tp / max(tp + fp, 1)
R = tp / max(tp + fn, 1)
rows.append({
"thr": round(float(thr), 4), "P": round(P, 4), "R": round(R, 4),
"FA": fp, "FA_rate": round(fp / max(len(yt), 1), 4),
})
return rows
def operating_points(rows, pair_proba, pairs):
"""Report P>=0.95/0.98/0.99 points with pair separation at threshold."""
res = {}
for target in (0.95, 0.98, 0.99):
pts = [r for r in rows if r["P"] >= target and r["R"] > 0.0]
if not pts:
res[str(target)] = None
continue
best = max(pts, key=lambda r: r["R"])
# pair separation at that operating point
sep = pair_sep_at(best["thr"], pair_proba, pairs)
best = dict(best); best["pair_sep"] = round(sep, 4)
res[str(target)] = best
return res
def pair_sep_at(thr, pair_proba, pairs):
"""Fraction of pairs where action>=thr and cap<thr."""
if not pairs:
return 0.0
ok = 0
for cidx, aidx in pairs:
if pair_proba[aidx] >= thr and pair_proba[cidx] < thr:
ok += 1
return ok / len(pairs)
def fold_variance(fold_rows):
return {
"folds": [
{
"fold": fr["fold"],
"ROC_AUC": fr["ROC_AUC"], "PR_AUC": fr["PR_AUC"],
"P": fr["P"], "R": fr["R"], "FA": fr["FA"], "n": fr["n"],
}
for fr in fold_rows
]
}
# ─── subcommands ────────────────────────────────────────────────────────────
def _result_path():
os.makedirs(RESULTS_DIR, exist_ok=True)
return RESULTS_DIR
def cmd_grouped(args):
rows = load_dev()
char_vocab, bpe = build_tokenizers(rows)
y = np.array([r["y"] for r in rows])
folds = np.array([r["cv_fold"] for r in rows])
X_char = encode_all(rows, char_vocab, "char")
X_bpe = encode_all(rows, bpe, "bpe")
tokenizers = {"char": char_vocab, "bpe": bpe}
os.makedirs(RESULTS_DIR, exist_ok=True)
# save tokenizer metadata for reproducibility
meta = {
"char_vocab": char_vocab.size,
"char_vocab_sample": char_vocab.id_to_char[:50],
"bpe_vocab": bpe.size,
"bpe_serialized_bytes": bpe.serialized_bytes(),
"max_char": MAX_CHAR, "max_bpe": MAX_BPE,
"n": len(rows),
}
with open(os.path.join(RESULTS_DIR, "corpus_meta.json"), "w") as f:
json.dump(meta, f)
kind_of = {"char_cnn": "char", "bigru": "bpe", "tiny_transformer": "bpe"}
for arch, sizes in ARCH_CONFIGS.items():
kind = kind_of[arch]
X = X_char if kind == "char" else X_bpe
for size in sizes:
name = f"{arch}_{size}"
os.makedirs(os.path.join(RESULTS_DIR, "models"), exist_ok=True)
probs = {v: np.zeros(len(rows)) for v in VARIANTS}
fold_rows = []
for fold in range(5):
tr = folds != fold
te = folds == fold
Xtr = X["strip"][tr]
model, secs = train_binary(Xtr, y[tr], arch, size,
vocab_size=tokenizers[kind].size,
seed_offset=fold, log=args.verbose)
torch_models_dir = os.path.join(RESULTS_DIR, "models")
import torch
torch.save(model.state_dict(), os.path.join(torch_models_dir, f"{name}_fold{fold}.pt"))
for v in VARIANTS:
probs[v][te] = predict_proba(model, X[v][te])
fold_m = binary_metrics(y[te], probs["strip"][te])
fold_m["fold"] = fold
fold_rows.append(fold_m)
print(f" {name} fold {fold}: ROC={fold_m['ROC_AUC']:.3f} "
f"PR={fold_m['PR_AUC']:.3f} P={fold_m['P']:.3f} R={fold_m['R']:.3f} "
f"FA={fold_m['FA']} n={fold_m['n']} ({secs:.1f}s)")
np.savez(os.path.join(RESULTS_DIR, f"{name}_probs.npz"),
var_orig=probs["orig"], var_nofinal=probs["nofinal"],
var_strip=probs["strip"])
summary = binary_metrics(y, probs["strip"])
print(f" {name} OOF: ROC={summary['ROC_AUC']:.3f} PR={summary['PR_AUC']:.3f} "
f"P={summary['P']:.3f} R={summary['R']:.3f} FA={summary['FA']}")
print("grouped done")
def cmd_metrics(args):
rows = load_dev()
y = np.array([r["y"] for r in rows])
tags = [r["tags"] for r in rows]
pairs = build_pairs(rows, [r["text_strip"] for r in rows])
print(f"pairs={len(pairs)}")
out = {}
for arch, sizes in ARCH_CONFIGS.items():
for size in sizes:
name = f"{arch}_{size}"
fp = os.path.join(RESULTS_DIR, f"{name}_probs.npz")
if not os.path.exists(fp):
continue
z = np.load(fp)
entry = {"name": name, "arch": arch, "size": size}
# OOF binary on strip variant (primary training input)
entry["strip"] = binary_metrics(y, z["var_strip"])
entry["pairs"] = {}
entry["pairs"]["strip"] = pair_metrics(pairs, z["var_strip"])
entry["pairs"]["orig"] = pair_metrics(pairs, z["var_orig"])
entry["pairs"]["nofinal"] = pair_metrics(pairs, z["var_nofinal"])
# stress: same OOF models, per-variant metrics
entry["stress"] = {}
for v in VARIANTS:
p = z[f"var_{v}"]
entry["stress"][v] = {
"all_FA": binary_metrics(y, p)["FA_rate"],
"capQ_FA": capq_fa(tags, y, p),
"modal_recall": modal_recall(tags, y, p),
}
# threshold curve + operating points on strip
entry["curve"] = threshold_curve(y, z["var_strip"],
np.arange(0.30, 1.0, 0.02))
entry["ops"] = operating_points(entry["curve"], z["var_strip"], pairs)
out[name] = entry
with open(os.path.join(RESULTS_DIR, "metrics.json"), "w") as f:
json.dump(out, f, indent=2, default=str)
print(json.dumps(out, indent=2, default=str))
def capq_fa(tags, y, proba):
mask = np.array(["capability_question" in t for t in tags])
if mask.sum() == 0:
return 0.0
sub = proba[mask]
return float((sub >= 0.5).sum() / mask.sum())
def modal_recall(tags, y, proba):
mask = np.array([
(y[i] == 1 and ("polite_request" in tags[i] or "modal_request" in tags[i]))
for i in range(len(y))
])
if mask.sum() == 0:
return 0.0
sub = proba[mask]
return float((sub >= 0.5).sum() / mask.sum())
def cmd_lfo(args):
rows = load_dev()
char_vocab, bpe = build_tokenizers(rows)
y = np.array([r["y"] for r in rows])
tags = [r["tags"] for r in rows]
X_char = encode_all(rows, char_vocab, "char")["strip"]
X_bpe = encode_all(rows, bpe, "bpe")["strip"]
kind_of = {"char_cnn": "char", "bigru": "bpe", "tiny_transformer": "bpe"}
results = {}
tokenizers = {"char": char_vocab, "bpe": bpe}
# capability_question LOFO for every config/size (the critical case)
for arch, sizes in ARCH_CONFIGS.items():
X = X_char if kind_of[arch] == "char" else X_bpe
src_idx = np.array(["capability_question" not in t for t in tags])
tgt_idx = np.array(["capability_question" in t for t in tags])
for size in sizes:
model, _ = train_binary(X[src_idx], y[src_idx], arch, size,
vocab_size=tokenizers[kind_of[arch]].size,
seed_offset=17)
p = predict_proba(model, X[tgt_idx])
m = binary_metrics(y[tgt_idx], p)
results[f"{arch}_{size}:capability_question"] = m
print(f"LFO ability {arch}_{size}: cap rows={m['n']} "
f"pos={int(y[tgt_idx].sum())} FA={m['FA']} FA_rate={m['FA_rate']:.3f} "
f"P={m['P']:.3f} R={m['R']:.3f} acc={1-m['FA_rate']:.3f}")
# full family LOFO for the leading config per architecture
leading = {"char_cnn": "char_cnn_medium", "bigru": "bigru_tiny",
"tiny_transformer": "tiny_transformer_small"}
for arch, name in leading.items():
X = X_char if kind_of[arch] == "char" else X_bpe
for fam in PRESENT_FAMILIES:
src = np.array([fam not in t for t in tags])
tgt = np.array([fam in t for t in tags])
model, _ = train_binary(X[src], y[src], arch, name.split("_")[-1],
vocab_size=tokenizers[kind_of[arch]].size,
seed_offset=41)
p = predict_proba(model, X[tgt])
m = binary_metrics(y[tgt], p)
results[f"{name}:{fam}"] = m
print(f"LFO {fam}: {name} rows={m['n']} pos={int(y[tgt].sum())} "
f"P={m['P']:.3f} R={m['R']:.3f} FA={m['FA']} acc={1-m['FA_rate']:.3f}")
with open(os.path.join(RESULTS_DIR, "lfo.json"), "w") as f:
json.dump(results, f, indent=2, default=str)
print("lfo done")
def cmd_e5baseline(args):
from sklearn.linear_model import LogisticRegression
from sklearn.neural_network import MLPClassifier
rows = load_dev()
y = np.array([r["y"] for r in rows])
folds = np.array([r["cv_fold"] for r in rows])
X = np.vstack([r["emb"] for r in rows])
tags = [r["tags"] for r in rows]
pairs = build_pairs(rows, [r["text_strip"] for r in rows])
out = {}
for model_name, model, extra in [
("e5_linear", LogisticRegression(C=1.0, max_iter=2000, solver="lbfgs", random_state=42), {}),
("e5_mlp_h32", MLPClassifier(hidden_layer_sizes=(32,), activation="relu",
solver="adam", alpha=0.01, random_state=42,
max_iter=800, early_stopping=True,
validation_fraction=0.15, n_iter_no_change=10), {}),
]:
proba = np.zeros(len(rows))
for fold in range(5):
tr = folds != fold
te = folds == fold
m2 = type(model)(**{k: v for k, v in model.get_params().items()})
m2.fit(X[tr], y[tr])
proba[te] = m2.predict_proba(X[te])[:, 1]
entry = {
"grouped": binary_metrics(y, proba),
"pairs": pair_metrics(pairs, proba),
}
# cap-Q leave-generator-out (train without the family)
src = np.array(["capability_question" not in t for t in tags])
tgt = np.array(["capability_question" in t for t in tags])
m3 = LogisticRegression(C=1.0, max_iter=2000, solver="lbfgs", random_state=42) \
if model_name == "e5_linear" else \
MLPClassifier(hidden_layer_sizes=(32,), alpha=0.01, random_state=42, max_iter=800)
m3.fit(X[src], y[src])
p3 = m3.predict_proba(X[tgt])[:, 1]
entry["capq_lofo"] = binary_metrics(y[tgt], p3)
entry["stress"] = "NA (no re-embed on this box)"
out[model_name] = entry
print(f"{model_name}: grouped FA_rate={entry['grouped']['FA_rate']:.4f} "
f"PR={entry['grouped']['PR_AUC']:.3f} capQ_LOFO_FA_rate={entry['capq_lofo']['FA_rate']:.4f} "
f"pairs={entry['pairs']['ordering_acc']:.3f}")
with open(os.path.join(RESULTS_DIR, "e5baseline.json"), "w") as f:
json.dump(out, f, indent=2, default=str)
print("e5baseline done")
def cmd_runtime(args):
import torch
rows = load_dev()
char_vocab, bpe = build_tokenizers(rows)
X_char = encode_all(rows, char_vocab, "char")["strip"]
X_bpe = encode_all(rows, bpe, "bpe")["strip"]
kind_of = {"char_cnn": "char", "bigru": "bpe", "tiny_transformer": "bpe"}
report = {}
for arch, sizes in ARCH_CONFIGS.items():
X = X_char if kind_of[arch] == "char" else X_bpe
for size in sizes:
name = f"{arch}_{size}"
model = build_model(arch, size, int(X.max()) + 1, X.shape[1])
n_params = sum(p.numel() for p in model.parameters())
fp32 = n_params * 4
int8 = n_params
model.eval()
# warmup + latency (batch-1, eval mode)
xb = torch.from_numpy(X[:1])
with torch.no_grad():
for _ in range(20):
model(xb)
# tokenization latency
if kind_of[arch] == "char":
t0 = time.perf_counter()
for r in rows[:1000]:
char_vocab.encode(r["text_strip"], MAX_CHAR)
tl = (time.perf_counter() - t0) / 1000
else:
t0 = time.perf_counter()
for r in rows[:1000]:
bpe.encode(r["text_strip"])
tl = (time.perf_counter() - t0) / 1000
lat = []
for _ in range(300):
t0 = time.perf_counter()
model(xb)
lat.append(time.perf_counter() - t0)
lat = np.array(lat) * 1e6
report[name] = {
"params": n_params, "fp32_bytes": fp32, "int8_bytes": int8,
"latency_us_mean": float(lat.mean()), "latency_us_p50": float(np.median(lat)),
"latency_us_p95": float(np.percentile(lat, 95)),
"throughput_b1": round(1e6 / float(lat.mean()), 1),
"tok_us": round(tl * 1e6, 1),
"tokenizer": "char" if kind_of[arch] == "char" else "bpe",
}
print(f"{name}: {n_params} params fp32={fp32/1024:.0f}KiB "
f"lat={lat.mean():.0f}us tok={tl*1e6:.1f}us")
with open(os.path.join(RESULTS_DIR, "runtime.json"), "w") as f:
json.dump(report, f, indent=2)
print("runtime done")
def main():
ap = argparse.ArgumentParser()
ap.add_argument("cmd", choices=["grouped", "metrics", "lfo", "e5baseline", "runtime"])
ap.add_argument("--verbose", action="store_true")
args = ap.parse_args()
t0 = time.time()
globals()[f"cmd_{args.cmd}"](args)
print(f"elapsed {time.time()-t0:.1f}s")
if __name__ == "__main__":
main()
@@ -0,0 +1,158 @@
#!/usr/bin/env python3
"""
Slice 19 model zoo: three genuinely sequence-sensitive tiny models, trained
from scratch on Maven's narrow binary pragmatics task.
A. CharCNN codepoint ids char embedding parallel small 1D convs
(several kernel widths) global max-pool linear head
B. BiGRU subword ids token embedding 1-layer BiGRU
maxpool[final] linear head
C. TinyTransformer subword ids token embedding + sine position
N self-attention encoder blocks (heads, FFN 4x, PreNorm)
CLS linear head
All expose :forward(ids) returning the binary logit, plus .n_params().
Deterministic: everything is plain torch ops.
"""
import math
import torch
import torch.nn as nn
import torch.nn.functional as F
class CharCNN(nn.Module):
def __init__(self, vocab_size, embed_dim, filters, widths, pad_idx=0, dropout=0.3):
super().__init__()
self.embed = nn.Embedding(vocab_size, embed_dim, padding_idx=pad_idx)
self.convs = nn.ModuleList([
nn.Conv1d(embed_dim, filters, k, padding=(k - 1) // 2)
for k in widths
])
self.dropout = nn.Dropout(dropout)
self.head = nn.Linear(filters * len(widths), 1)
def forward(self, ids):
# ids: (B, T)
x = self.embed(ids).transpose(1, 2) # (B, D, T)
hiddens = [F.relu(conv(x)) for conv in self.convs] # each (B, F, T)
pooled = torch.cat([h.max(dim=2).values for h in hiddens], dim=1) # (B, F*W)
return self.head(self.dropout(pooled)).squeeze(-1)
def n_params(self):
return sum(p.numel() for p in self.parameters())
class BiGRU(nn.Module):
def __init__(self, vocab_size, embed_dim, hidden, pad_idx=0, dropout=0.3):
super().__init__()
self.embed = nn.Embedding(vocab_size, embed_dim, padding_idx=pad_idx)
self.encoder = nn.GRU(embed_dim, hidden, num_layers=1, bidirectional=True,
batch_first=True)
self.dropout = nn.Dropout(dropout)
self.head = nn.Linear(hidden * 2, 1)
def forward(self, ids):
mask = (ids != 0).float() # (B, T)
x = self.embed(ids)
lens = mask.sum(dim=1).clamp(min=1).long()
x_p = nn.utils.rnn.pack_padded_sequence(x, lens.cpu(), batch_first=True,
enforce_sorted=False)
out, _ = self.encoder(x_p)
out, _ = nn.utils.rnn.pad_packed_sequence(out, batch_first=True,
total_length=mask.size(1))
out = out * mask.unsqueeze(-1)
maxed = out.max(dim=1).values # (B, 2H)
return self.head(self.dropout(maxed)).squeeze(-1)
def n_params(self):
return sum(p.numel() for p in self.parameters())
class TinyTransformer(nn.Module):
def __init__(self, vocab_size, d_model, n_layers, n_heads, ff_mult=4,
max_len=64, pad_idx=0, dropout=0.1):
super().__init__()
self.d_model = d_model
self.embed = nn.Embedding(vocab_size, d_model, padding_idx=pad_idx)
self.dropout = nn.Dropout(dropout)
self.pos = nn.Parameter(torch.empty(1, max_len, d_model))
nn.init.normal_(self.pos, std=0.02)
blocks = []
for _ in range(n_layers):
blocks.append(TransformerBlock(d_model, n_heads, ff_mult, dropout))
self.blocks = nn.ModuleList(blocks)
self.ln_out = nn.LayerNorm(d_model)
self.head = nn.Linear(d_model, 1)
def forward(self, ids):
B, T = ids.shape
mask = (ids != 0)
x = self.embed(ids) * math.sqrt(self.d_model) + self.pos[:, :T, :]
x = self.dropout(x)
for blk in self.blocks:
x = blk(x, mask)
x = self.ln_out(x)
pooled = x.masked_fill(~mask.unsqueeze(-1), float("-inf")).max(dim=1).values
return self.head(pooled).squeeze(-1)
def n_params(self):
return sum(p.numel() for p in self.parameters())
class TransformerBlock(nn.Module):
def __init__(self, d_model, n_heads, ff_mult, dropout):
super().__init__()
self.attn = nn.MultiheadAttention(d_model, n_heads, dropout=dropout,
batch_first=True)
self.ln1 = nn.LayerNorm(d_model)
self.ff = nn.Sequential(
nn.Linear(d_model, d_model * ff_mult),
nn.GELU(),
nn.Linear(d_model * ff_mult, d_model),
)
self.ln2 = nn.LayerNorm(d_model)
self.dropout = nn.Dropout(dropout)
def forward(self, x, mask):
# mask: (B, T) bool of non-pad; MultiheadAttention uses key_padding_mask
h = self.ln1(x)
h = self.attn(h, h, h, key_padding_mask=~mask,
need_weights=False, is_causal=False)[0]
x = x + self.dropout(h)
h = self.ln2(x)
x = x + self.dropout(self.ff(h))
return x
# ─── Sizes ladder ───────────────────────────────────────────────────────────
def make_model(arch, size, char_vocab, bpe_vocab):
if arch == "char_cnn":
configs = {
"tiny": dict(embed_dim=32, filters=64, widths=[3, 4, 5]),
"medium": dict(embed_dim=64, filters=160, widths=[2, 3, 4, 5]),
}
c = configs[size]
return CharCNN(char_vocab, c["embed_dim"], c["filters"], c["widths"])
if arch == "bigru":
configs = {
"tiny": dict(embed_dim=64, hidden=64),
"medium": dict(embed_dim=128, hidden=128),
"large": dict(embed_dim=256, hidden=256),
}
c = configs[size]
return BiGRU(bpe_vocab, c["embed_dim"], c["hidden"])
if arch == "tiny_transformer":
configs = {
"small": dict(d_model=128, n_layers=2, n_heads=4),
"medium": dict(d_model=192, n_layers=4, n_heads=4),
}
c = configs[size]
return TinyTransformer(bpe_vocab, c["d_model"], c["n_layers"], c["n_heads"])
raise ValueError(arch)
def n_params_of(arch, size, char_vocab, bpe_vocab):
return make_model(arch, size, char_vocab, bpe_vocab).n_params()
@@ -0,0 +1,152 @@
#!/usr/bin/env python3
"""
Slice 20 tokenizer audit (§2 of the brief) and corpus sequence-length
statistics (§3). Runs before any training. If the audit shows catastrophic
Cyrillic / mixed-identifier loss it is the gate to stop.
Loads the frozen dev corpus exactly like slice 19 (same normalization),
so regime A (natural text) is `text_orig` and regime B (punct-stripped) is
`text_strip`.
"""
import json
import os
import re
import statistics
from transformers import AutoTokenizer
import slice19_main as s19
RESULTS_DIR = "/tmp/mvn-s20"
MODEL_DIR = ("/tmp/mvn-s20/hf-cache/models--cointegrated--rubert-tiny/"
"snapshots/5441c5ea8026d4f6d7505ec004845409f1259fb1")
MIXED_CYRILLIC_LATIN = re.compile(r"[а-яёА-ЯЁ]+[a-zA-Z]+|[a-zA-Z]+[а-яёА-ЯЁ]+")
HAS_CYRILLIC = re.compile(r"[а-яёА-ЯЁ]")
HAS_LATIN = re.compile(r"[a-zA-Z]")
NUMERIC = re.compile(r"[0-9]")
TOKEN_RE = re.compile(r"[^\W\d_]+", re.UNICODE)
SAMPLES = [
"выключи свет в спальне пожалуйста",
"turn off the lights",
"перезапусти сервис mavend",
"что такое Nexus",
"включи телевизор, пожалуйста",
"как дела у Мэйвен",
"поставь таймер на 5 минут",
"кто такой Home Assistant",
"открой настройки устройства ha_cam_12",
"Покажи статус сервера Proxmox",
"аутентифицируй на сайте 2fa.ru",
"сообщи погоду завтра в 18:30",
]
def is_toolish(word):
# Maven sibling service names / HA-like identifiers: mixed case, digits,
# underscores, or short Latin words that are not in the vocab as whole
# words. Rough heuristic for the fragmentation probe.
return bool(re.search(r"[A-Z0-9_/.-]", word))
def main():
os.makedirs(RESULTS_DIR, exist_ok=True)
tok = AutoTokenizer.from_pretrained(MODEL_DIR)
rows = s19.load_dev()
texts = {
"orig": [r["text_orig"] for r in rows],
"nofinal": [r["text_nofinal"] for r in rows],
"strip": [r["text_strip"] for r in rows],
}
vv = tok.vocab_size
unk = tok.unk_token_id
rep = {"model": "cointegrated/rubert-tiny",
"sha": "5441c5ea8026d4f6d7505ec004845409f1259fb1",
"tokenizer": type(tok).__name__,
"vocab_size": vv}
# per-character stats (natural texts)
chars = [len(t) for t in texts["orig"]]
rep["char_len"] = {
"mean": round(statistics.mean(chars), 2),
"p50": int(sorted(chars)[len(chars) // 2]),
"p95": sorted(chars)[int(len(chars) * .95)],
"p99": sorted(chars)[int(len(chars) * .99)],
"max": max(chars),
}
for view in ("orig", "strip"):
ids = tok(texts[view], add_special_tokens=True, padding=False,
truncation=False)["input_ids"]
lens = [len(x) for x in ids]
n_tok = sum(lens)
n_unk = sum(x.count(unk) for x in ids)
n_chars = sum(len(t) for t in texts[view])
rep[view] = {
"tokens_per_utt_mean": round(n_tok / len(rows), 2),
"tokens_per_char": round(n_tok / max(n_chars, 1), 4),
"unk_count": n_unk,
"unk_rate": round(n_unk / max(n_tok, 1), 5),
"seq_len_p50": int(sorted(lens)[len(lens) // 2]),
"seq_len_p90": sorted(lens)[int(len(lens) * .90)],
"seq_len_p95": sorted(lens)[int(len(lens) * .95)],
"seq_len_p99": sorted(lens)[int(len(lens) * .99)],
"seq_len_max": max(lens),
"above_96": sum(1 for x in lens if x > 96),
"above_128": sum(1 for x in lens if x > 128),
}
rep[view]["p99_plus_margin"] = rep[view]["seq_len_p99"] + 6
# mixed Cyrillic/Latin behaviour over natural texts
mixed_words = []
for t in texts["orig"]:
for w in t.split():
if MIXED_CYRILLIC_LATIN.search(w):
mixed_words.append(w)
rep["mixed_cyr_lat_rows"] = len({w for w in mixed_words})
rep["mixed_cyr_lat_stats"] = {"word_count": len(mixed_words),
"unique_words": len(set(mixed_words))}
# entity/tool-name fragmentation: unique word-like tokens containing a digit
# or underscore, or non-trivial Latin, and how many BPE/WordPiece pieces they
# split into. Sample the extremes.
fragments = []
vocab = set(tok.get_vocab().keys())
for t in texts["orig"]:
# split into "clean" tokens (word chars + _ / digit boundaries)
for w in re.findall(r"[A-Za-z0-9_]+\b", t):
w2 = re.sub(r"_\b", "", w)
if len(w2) < 3 or not is_toolish(w2):
continue
n_pieces = len(tok.tokenize(w2).replace("##", "_").rstrip())
fragments.append((n_pieces, w2))
frag = sorted(set(fragments))[-30:]
rep["entity_fragment_examples"] = [
{"token": w, "pieces": n} for n, w in frag
]
# representative samples: full tokenization
rep["samples"] = []
for s in SAMPLES:
e = tok(s, add_special_tokens=True, padding=False, truncation=False)
rep["samples"].append({
"text": s,
"tokens": tok.convert_ids_to_tokens(e["input_ids"]),
"pieces": len(e["input_ids"]),
"unk": e["input_ids"].count(unk),
})
with open(os.path.join(RESULTS_DIR, "tokenizer_audit.json"), "w") as f:
json.dump(rep, f, indent=2, ensure_ascii=False)
print(json.dumps({k: v for k, v in rep.items() if k not in ("samples",)}, indent=2, ensure_ascii=False))
print("\n--- samples ---")
for s in rep["samples"]:
print(f'{s["pieces"]:>3} unk={s["unk"]} {s["text"]:50} ->'
f' {" ".join(s["tokens"])}')
if __name__ == "__main__":
main()
@@ -0,0 +1,517 @@
#!/usr/bin/env python3
"""
Slice 20: fine-tune cointegrated/rubert-tiny (11.9M, 3-layer BERT) end-to-end
for the binary executable-intent boundary (action vs not_action) on the frozen
v2 dev corpus, following the slice 20 brief.
Rules honoured:
- full end-to-end fine-tuning, CLS-pooled native classification head
- tokenizer used unchanged (audit in slice20_audit.py)
- max length from corpus stats (p99+margin, cap 128): 25 here
- narrow search: LR in {1e-5, 2e-5, 5e-5}, <= 6 epochs, early stop on a
development (within-fold) split, best checkpoint restored
- >= 3 seeds (42/17/7) for every config
- grouped 5-fold CV reuse; cap-Q leave-generator-out as primary stress case
- two input regimes: A = natural text (orig), B = punctuation-stripped (strip)
Artifacts under /tmp/mvn-s20/:
pre/{regime}_ids.npy, _attn.npy tokenized corpus (all three views)
oof/{regime}_{lr}_{seed}_probs.npz OOF probs per view (var_orig/nofinal/strip)
oof/{regime}_{lr}_{seed}_metrics.json
lfo/{regime}_{lr}_{seed}.json capability-Q LOFO (held-out family)
results/summary.json
models/{regime}_{lr}_{seed}_fold{i}.pt, lfo_{seed}.pt
CLI: slice20_pretrained.py {pre, grouped, lfo, metrics, runtime, onnx}
"""
import argparse
import importlib.util
import json
import os
import sys
import time
import numpy as np
import torch
from torch import nn
from transformers import AutoConfig, AutoTokenizer
from transformers import BertForSequenceClassification
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import slice19_main as s19
RESULTS_DIR = "/tmp/mvn-s20"
MODEL_DIR = ("/tmp/mvn-s20/hf-cache/models--cointegrated--rubert-tiny/"
"snapshots/5441c5ea8026d4f6d7505ec004845409f1259fb1")
MODEL_NAME = "cointegrated/rubert-tiny"
MODEL_SHA = "5441c5ea8026d4f6d7505ec004845409f1259fb1"
REGIMES = ["A", "B"]
LRS = [1e-5, 2e-5, 5e-5]
SEEDS = [42, 17, 7]
VIEWS = ["orig", "nofinal", "strip"]
MAX_LEN = 25
BATCH = 32
MAX_EPOCHS = 4
EARLY_STOP = 1 # patience in epochs on val PR-AUC
VAL_FRACTION = 0.12
WEIGHT_DECAY = 0.01
torch.set_num_threads(4)
# ─── tokenizer / input preparation ──────────────────────────────────────────
def _load_tokenizer():
return AutoTokenizer.from_pretrained(MODEL_DIR)
def tokenize(texts, tok):
e = tok(list(texts), add_special_tokens=True, padding="max_length",
truncation=True, max_length=MAX_LEN)
return np.array(e["input_ids"], np.int64), np.array(e["attention_mask"], np.int64)
def cmd_pre(args):
os.makedirs(os.path.join(RESULTS_DIR, "pre"), exist_ok=True)
tok = _load_tokenizer()
rows = s19.load_dev()
# distributed over all variants, all rows, both regimes
for regime in REGIMES:
train_view = "orig" if regime == "A" else "strip"
tsrc = [r[f"text_{train_view}"] for r in rows]
ids, attn = tokenize(tsrc, tok)
np.save(os.path.join(RESULTS_DIR, "pre", f"{regime}_ids.npy"), ids)
np.save(os.path.join(RESULTS_DIR, "pre", f"{regime}_attn.npy"), attn)
# eval stress views tokenized under the same regime's vocab/format
for v in VIEWS:
ids_v, attn_v = tokenize([r[f"text_{v}"] for r in rows], tok)
np.save(os.path.join(RESULTS_DIR, "pre", f"ev_{regime}_{v}_ids.npy"), ids_v)
np.save(os.path.join(RESULTS_DIR, "pre", f"ev_{regime}_{v}_attn.npy"), attn_v)
print(f"regime {regime} done, train view={train_view}")
def _make_model():
cfg = AutoConfig.from_pretrained(MODEL_DIR)
cfg.num_labels = 1 # sine logit, BCEWithLogits — matches slice 19 head
model = BertForSequenceClassification.from_pretrained(MODEL_DIR, config=cfg)
return model
# ─── training ───────────────────────────────────────────────────────────────
def _val_split(rows_idx, y, seed):
rng = np.random.RandomState(seed)
idx = rng.permutation(rows_idx)
# stratified: keep the y-ratio in both parts
pos = idx[y[idx] == 1]
neg = idx[y[idx] == 0]
nv_pos = max(1, int(round(len(pos) * VAL_FRACTION)))
nv_neg = max(1, int(round(len(neg) * VAL_FRACTION)))
v = np.concatenate([pos[:nv_pos], neg[:nv_neg]])
t = np.concatenate([pos[nv_pos:], neg[nv_neg:]])
return t, v
def train_model(id_arr, attn, y, tr_idx, val_idx, lr, seed, builder=None):
"""Fine-tune the full encoder; early-stop on val PR-AUC; return best state."""
net = (builder or _make_model)()
opt = torch.optim.AdamW([p for p in net.parameters()],
lr=lr, weight_decay=WEIGHT_DECAY)
lossf = nn.BCEWithLogitsLoss()
from sklearn.metrics import average_precision_score
tr = torch.from_numpy(np.ascontiguousarray(id_arr[tr_idx]))
ta = torch.from_numpy(np.ascontiguousarray(attn[tr_idx]))
ty = torch.from_numpy(y[tr_idx].astype(np.float32))
va = torch.from_numpy(np.ascontiguousarray(id_arr[val_idx]))
vaa = torch.from_numpy(np.ascontiguousarray(attn[val_idx]))
vy = y[val_idx]
best_pr = -1.0
best_state = None
best_epoch = 0
patience = 0
n = len(tr_idx)
rng = np.random.RandomState(seed * 97 % 2**31)
for epoch in range(MAX_EPOCHS):
net.train()
perm = rng.permutation(n)
running = 0.0
nb = 0
for st in range(0, n, BATCH):
bidx = torch.from_numpy(perm[st:st + BATCH])
logits = net(input_ids=tr[bidx], attention_mask=ta[bidx]).logits.squeeze(-1)
loss = lossf(logits, ty[bidx])
opt.zero_grad()
loss.backward()
opt.step()
running += float(loss)
nb += 1
net.eval()
with torch.no_grad():
pval = torch.sigmoid(net(input_ids=va, attention_mask=vaa).logits.squeeze(-1)).numpy()
if len(np.unique(vy)) > 1:
pr = average_precision_score(vy, pval)
else:
pr = 0.0
if pr > best_pr:
best_pr = pr
best_state = {k: v.detach().clone() for k, v in net.state_dict().items()}
best_epoch = epoch + 1
patience = 0
else:
patience += 1
if patience >= EARLY_STOP:
break
net.load_state_dict(best_state)
return net, best_epoch, best_pr, running / max(nb, 1)
def predict_proba(net, id_arr, attn, idx=None):
net.eval()
idx = np.arange(len(id_arr)) if idx is None else idx
out = []
with torch.no_grad():
for st in range(0, len(idx), BATCH * 4):
bi = idx[st:st + BATCH * 4]
iid = torch.from_numpy(np.ascontiguousarray(id_arr[bi]))
att = torch.from_numpy(np.ascontiguousarray(attn[bi]))
out.append(torch.sigmoid(net(input_ids=iid, attention_mask=att).logits.squeeze(-1)).numpy())
return np.concatenate(out)
# ─── grouped CV ─────────────────────────────────────────────────────────────
def cmd_grouped(args):
os.makedirs(os.path.join(RESULTS_DIR, "oof"), exist_ok=True)
os.makedirs(os.path.join(RESULTS_DIR, "models"), exist_ok=True)
rows = s19.load_dev()
y = np.array([r["y"] for r in rows])
folds = np.array([r["cv_fold"] for r in rows])
for regime in REGIMES:
ids = np.load(os.path.join(RESULTS_DIR, "pre", f"{regime}_ids.npy"))
attn = np.load(os.path.join(RESULTS_DIR, "pre", f"{regime}_attn.npy"))
ev = {v: (np.load(os.path.join(RESULTS_DIR, "pre", f"ev_{regime}_{v}_ids.npy")),
np.load(os.path.join(RESULTS_DIR, "pre", f"ev_{regime}_{v}_attn.npy")))
for v in VIEWS}
for lr in LRS:
for seed in SEEDS:
probs = {v: np.zeros(len(rows)) for v in VIEWS}
fold_records = []
for fold in range(5):
tr = np.where(folds != fold)[0]
te = np.where(folds == fold)[0]
t_idx, v_idx = _val_split(tr, y, seed + 100 * fold)
net, ep, best_pr, _ = train_model(ids, attn, y, t_idx, v_idx, lr, seed + fold)
torch.save(net.state_dict(),
os.path.join(RESULTS_DIR, "models",
f"{regime}_{lr}_{seed}_fold{fold}.pt"))
for v in VIEWS:
probs[v][te] = predict_proba(net, *ev[v], te)
fold_records.append({"fold": fold, "epochs": ep, "val_pr": best_pr})
np.savez(os.path.join(RESULTS_DIR, "oof", f"{regime}_{lr}_{seed}_probs.npz"),
var_orig=probs["orig"], var_nofinal=probs["nofinal"],
var_strip=probs["strip"])
with open(os.path.join(RESULTS_DIR, "oof", f"{regime}_{lr}_{seed}_metrics.json"), "w") as f:
json.dump({"folds": fold_records}, f, indent=2)
m = s19.binary_metrics(y, probs["strip"])
print(f"[{regime}] lr={lr:.0e} seed={seed} "
f"PR={m['PR_AUC']:.3f} P={m['P']:.3f} R={m['R']:.3f} "
f"FA={m['FA']} epochs={[fr['epochs'] for fr in fold_records]}",
flush=True)
print("grouped done")
# ─── cap-Q leave-generator-out ──────────────────────────────────────────────
def cmd_lfo(args):
os.makedirs(os.path.join(RESULTS_DIR, "lfo"), exist_ok=True)
rows = s19.load_dev()
y = np.array([r["y"] for r in rows])
tags = [r["tags"] for r in rows]
src = np.array(["capability_question" not in t for t in tags])
tgt = ~src
for regime in REGIMES:
ids = np.load(os.path.join(RESULTS_DIR, "pre", f"{regime}_ids.npy"))
attn = np.load(os.path.join(RESULTS_DIR, "pre", f"{regime}_attn.npy"))
for lr in LRS:
for seed in SEEDS:
s_idx = np.where(src)[0]
t_idx, v_idx = _val_split(s_idx, y, seed + 7)
net, ep, best_pr, _ = train_model(ids, attn, y, t_idx, v_idx, lr, seed)
p = predict_proba(net, ids, attn, np.where(tgt)[0])
yt = y[tgt]
out = {
"regime": regime, "lr": lr, "seed": seed,
"rows": int(tgt.sum()), "epochs": ep, "val_pr": best_pr,
"mean_action_proba": float(np.mean(p)),
"max_action_proba": float(np.max(p)),
"acc": float(((p >= 0.5) == (yt == 1)).mean()),
"FA": int(((p >= 0.5) & (yt == 0)).sum()),
"FA_rate": float(((p >= 0.5) & (yt == 0)).mean()),
}
with open(os.path.join(RESULTS_DIR, "lfo", f"{regime}_{lr}_{seed}.json"), "w") as f:
json.dump(out, f, indent=2)
print(f"[{regime}] lr={lr:.0e} seed={seed} capQ LOFO "
f"acc={out['acc']:.3f} FA_rate={out['FA_rate']:.3f} "
f"mean_p={out['mean_action_proba']:.3f} epochs={ep}", flush=True)
print("lfo done")
# ─── metrics aggregation ────────────────────────────────────────────────────
def cmd_metrics(args):
rows = s19.load_dev()
y = np.array([r["y"] for r in rows])
tags = [r["tags"] for r in rows]
pairs = s19.build_pairs(rows, [r["text_strip"] for r in rows])
summary = {}
for regime in REGIMES:
summary[regime] = {}
for lr in LRS:
per_seed = []
for seed in SEEDS:
z = np.load(os.path.join(RESULTS_DIR, "oof", f"{regime}_{lr}_{seed}_probs.npz"))
entry = {"seed": seed,
"views": {v: {"FA_rate": s19.binary_metrics(y, z[f"var_{v}"])["FA_rate"],
"PR": s19.binary_metrics(y, z[f"var_{v}"])["PR_AUC"],
"capQ_FA": s19.capq_fa(tags, y, z[f"var_{v}"])}
for v in VIEWS},
"strip": s19.binary_metrics(y, z["var_strip"]),
"pairs": {v: s19.pair_metrics(pairs, z[f"var_{v}"]) for v in VIEWS},
"curve": s19.threshold_curve(y, z["var_strip"], np.arange(0.30, 1.0, 0.02)),
"ops": s19.operating_points(s19.threshold_curve(
y, z["var_strip"], np.arange(0.30, 1.0, 0.02)),
z["var_strip"], pairs),
}
with open(os.path.join(RESULTS_DIR, "oof", f"{regime}_{lr}_{seed}_metrics.json")) as f:
entry["folds"] = json.load(f)["folds"]
per_seed.append(entry)
# LOFO
lfos = []
for seed in SEEDS:
with open(os.path.join(RESULTS_DIR, "lfo", f"{regime}_{lr}_{seed}.json")) as f:
lfos.append(json.load(f))
summary[regime][str(lr)] = {"seeds": per_seed, "lfo": lfos}
# primary selection: min mean cap-Q LOFO FA_rate across seeds
sel = {}
for regime in REGIMES:
best_lr, best_mean = None, 9e9
for lr in LRS:
mean_fa = np.mean([l["FA_rate"] for l in summary[regime][str(lr)]["lfo"]])
sel[f"{regime}_{lr}"] = round(float(mean_fa), 4)
if mean_fa < best_mean:
best_lr, best_mean = lr, mean_fa
sel[f"{regime}_selected"] = str(best_lr)
summary["selection"] = sel
os.makedirs(os.path.join(RESULTS_DIR, "results"), exist_ok=True)
with open(os.path.join(RESULTS_DIR, "results", "summary.json"), "w") as f:
json.dump(summary, f, indent=2, default=str)
print(json.dumps(sel))
print("metrics done")
def _tokenizer_bytes():
return sum(os.path.getsize(os.path.join(MODEL_DIR, f))
for f in ["vocab.txt", "tokenizer.json"]
if os.path.exists(os.path.join(MODEL_DIR, f)))
# ─── runtime + ONNX ─────────────────────────────────────────────────────────
def cmd_runtime(args):
import time
net = _make_model()
tok = _load_tokenizer()
rows = s19.load_dev()
x = [r["text_orig"] for r in rows][:200]
e = tok(x, padding="max_length", truncation=True,
max_length=MAX_LEN, return_tensors="pt")
ids, attn = e["input_ids"], e["attention_mask"]
net.eval()
with torch.no_grad():
# warmup
for _ in range(3):
net(input_ids=ids[:1], attention_mask=attn[:1])
# batch-1 latency
lat = []
for i in range(200):
t0 = time.perf_counter()
net(input_ids=ids[i:i + 1], attention_mask=attn[i:i + 1])
lat.append((time.perf_counter() - t0) * 1e6)
# tokenization latency
t0 = time.perf_counter()
for i in range(200):
tok(x[i])
tok_us = (time.perf_counter() - t0) / 200 * 1e6
n_params = sum(p.numel() for p in net.parameters())
fp32 = n_params * 4
rep = {
"model": MODEL_NAME, "sha": MODEL_SHA,
"params": n_params, "fp32_bytes": fp32,
"fp16_bytes": fp32 // 2, "int8_bytes": n_params,
"tokenizer_bytes": _tokenizer_bytes(),
"latency_us_mean": float(np.mean(lat)),
"latency_us_p50": float(np.median(lat)),
"latency_us_p95": float(np.percentile(lat, 95)),
"max_len": MAX_LEN,
"tok_us": round(tok_us, 2),
"num_threads": 12,
}
with open(os.path.join(RESULTS_DIR, "runtime.json"), "w") as f:
json.dump(rep, f, indent=2)
print(json.dumps(rep, indent=2))
print("runtime done")
def cmd_onnx(args):
net = _make_model()
net.eval()
tok = _load_tokenizer()
rows = s19.load_dev()
try:
import torch.onnx
dummy = {
"input_ids": torch.zeros(1, MAX_LEN, dtype=torch.long),
"attention_mask": torch.ones(1, MAX_LEN, dtype=torch.long),
}
with torch.no_grad():
torch.onnx.export(net, (dummy,), os.path.join(RESULTS_DIR, "rubert-tiny-gate.onnx"),
input_names=["input_ids", "attention_mask"],
output_names=["logits"], opset_version=14,
dynamic_axes={"input_ids": {0: "batch"},
"attention_mask": {0: "batch"}})
# parity on a fixed sample
import numpy as np
samp = [(r["text_orig"], r["y"]) for r in rows[:200]]
e = tok([s[0] for s in samp], padding="max_length", truncation=True,
max_length=MAX_LEN, return_tensors="pt")
with torch.no_grad():
pt = torch.sigmoid(net(**e).logits.squeeze(-1)).numpy()
import onnxruntime as ort
so = ort.SessionOptions()
so.intra_op_num_threads = 12
sess = ort.InferenceSession(os.path.join(RESULTS_DIR, "rubert-tiny-gate.onnx"),
sess_options=so, providers=["CPUExecutionProvider"])
on = sess.run(None, {"input_ids": e["input_ids"].numpy(),
"attention_mask": e["attention_mask"].numpy()})[0]
on = 1 / (1 + np.exp(-on).squeeze(-1))
mx = float(np.max(np.abs(pt - on)))
size = os.path.getsize(os.path.join(RESULTS_DIR, "rubert-tiny-gate.onnx"))
rep = {"max_logit_diff": mx, "onnx_bytes": size,
"parity_n": len(samp), "provider": "CPUExecutionProvider"}
with open(os.path.join(RESULTS_DIR, "onnx.json"), "w") as f:
json.dump(rep, f, indent=2)
print(json.dumps(rep))
except Exception as ex:
print("onnx export/parity failed:", ex)
with open(os.path.join(RESULTS_DIR, "onnx.json"), "w") as f:
json.dump({"error": str(ex)}, f, indent=2)
print("onnx done")
# ─── optional capacity/pretraining ceiling (brief §13) ──────────────────────
# Trigger: tiny1 clearly improved over from-scratch on in-pool but missed the
# LOFO boundary. tiny2 is the same 3-layer 312-hidden BERT family; it tests
# whether a *newer, larger-vocab* pretraining of the same family generalises
# where tiny1 failed — disambiguating "this family is the wrong prior" from a
# one-off pretraining. It cannot test capacity (same depth/size).
MODEL2_DIR = ("/tmp/mvn-s20/hf-tiny2-cache/models--cointegrated--rubert-tiny2/"
"snapshots/e8ed3b0c8bbf4fb6984c3de043bf7d2f4e5969ae")
MODEL2_SHA = "e8ed3b0c8bbf4fb6984c3de043bf7d2f4e5969ae"
CEIL_RESULTS = os.path.join(RESULTS_DIR, "tiny2")
def cmd_ceiling(args):
import torch
os.makedirs(CEIL_RESULTS, exist_ok=True)
os.makedirs(os.path.join(CEIL_RESULTS, "models"), exist_ok=True)
tok = AutoTokenizer.from_pretrained(MODEL2_DIR)
rows = s19.load_dev()
y = np.array([r["y"] for r in rows])
tags = [r["tags"] for r in rows]
folds = np.array([r["cv_fold"] for r in rows])
pairs = s19.build_pairs(rows, [r["text_strip"] for r in rows])
# audit: is the tiny2 tokenizer sane on the corpus before anything else
n_unk = 0
n_tok = 0
lens = []
for r in rows:
e = tok(r["text_orig"])
n_unk += e["input_ids"].count(tok.unk_token_id)
n_tok += len(e["input_ids"])
lens.append(len(e["input_ids"]))
audit = {"vocab_size": tok.vocab_size,
"unk_count": int(n_unk),
"unk_rate": round(n_unk / max(n_tok, 1), 5),
"seq_len_p99": sorted(lens)[int(len(lens) * .99)],
"seq_len_max": max(lens)}
with open(os.path.join(CEIL_RESULTS, "audit.json"), "w") as f:
json.dump(audit, f, indent=2)
print("tiny2 audit:", audit)
# tokenize the corpus (regime A only — natural text, the in-pool best)
ids_a, attn_a = tokenize([r["text_orig"] for r in rows], tok)
ev = {v: tokenize([r[f"text_{v}"] for r in rows], tok) for v in VIEWS}
def make2():
cfg = AutoConfig.from_pretrained(MODEL2_DIR)
cfg.num_labels = 1
m = BertForSequenceClassification.from_pretrained(MODEL2_DIR, config=cfg)
return m
# cap-Q LOFO, 3 seeds, matching the A@2e-5 tiny1 config
lfors = []
src = np.where(np.array(["capability_question" not in t for t in tags]))[0]
tgt = np.where(np.array(["capability_question" in t for t in tags]))[0]
for seed in SEEDS:
t_idx, v_idx = _val_split(src, y, seed + 7)
net, ep, best_pr, _ = train_model(ids_a, attn_a, y, t_idx, v_idx, 2e-5, seed, builder=make2)
p = predict_proba(net, ids_a, attn_a, tgt)
yt = y[tgt]
lfors.append({"regime": "A(tiny2)", "lr": 2e-5, "seed": seed,
"rows": int(len(tgt)), "epochs": ep,
"mean_action_proba": float(np.mean(p)),
"acc": float(((p >= 0.5) == (yt == 1)).mean()),
"FA": int(((p >= 0.5) & (yt == 0)).sum()),
"FA_rate": float(((p >= 0.5) & (yt == 0)).mean())})
with open(os.path.join(CEIL_RESULTS, "lfo.json"), "w") as f:
json.dump(lfors, f, indent=2)
print("tiny2 LOFO:", [round(l["FA_rate"], 3) for l in lfors])
# grouped CV for the same best config + in-pool pairs / capQ
oof = {v: np.zeros(len(rows)) for v in VIEWS}
for fold in range(5):
tr = np.where(folds != fold)[0]
te = np.where(folds == fold)[0]
t_idx, v_idx = _val_split(tr, y, 42 + 100 * fold)
net, _, _, _ = train_model(ids_a, attn_a, y, t_idx, v_idx, 2e-5, 42 + fold, builder=make2)
for v in VIEWS:
oof[v][te] = predict_proba(net, *ev[v], te)
inpool = {"strip": s19.binary_metrics(y, oof["strip"]),
"pairs": {v: s19.pair_metrics(pairs, oof[v]) for v in VIEWS},
"capQ_inpool": {v: s19.capq_fa(tags, y, oof[v]) for v in VIEWS}}
with open(os.path.join(CEIL_RESULTS, "grouped.json"), "w") as f:
json.dump(inpool, f, indent=2, default=str)
b = inpool["strip"]
print(f"tiny2 grouped A@2e-5: PR={b['PR_AUC']:.3f} P={b['P']:.3f} R={b['R']:.3f} "
f"FA={b['FA']} pairs_strip={inpool['pairs']['strip']['ordering_acc']:.3f} "
f"capQ_strip={inpool['capQ_inpool']['strip']:.3f}", flush=True)
print("ceiling done")
def main():
ap = argparse.ArgumentParser()
ap.add_argument("cmd", choices=["pre", "grouped", "lfo", "metrics", "runtime", "onnx", "ceiling"])
args = ap.parse_args()
t0 = time.time()
globals()[f"cmd_{args.cmd}"](args)
print(f"elapsed {time.time()-t0:.1f}s")
if __name__ == "__main__":
main()
@@ -0,0 +1,95 @@
package main
// Fixture is one brief-specified utterance and the eligibility it must earn.
// Fixtures are EXACTLY the slice-21 brief examples plus the corpus-verified
// structural discriminators. They are eval material, not corpus additions:
// nothing here is inserted into any pool file (brief §2, §18).
type Fixture struct {
Utterance string
Want Eligibility
Family string
Note string
}
// Fixtures covers every family the brief's §15 list exercises, including the
// four that have no dev-pool rows at all (negation, reported_speech,
// quotation, hypothetical):
//
// direct imperative already plural in dev; fixtures pin the canonical
// cases together with the politeness-negative modal.
// polite request polite imperative + bare "можешь …, пожалуйста".
// modal request "можно …", English "can you …, please".
// first-person request "я хочу …", "мне нужно …", "надо …".
// reordered target "свет выключи, пожалуйста" (target-first command).
// capability question ты/умеешь/сможешь/способна + "… ли" permission.
// ordinary question "что запущено" — no execution pressure.
// negation "не выключай свет" (parser-covered); advisory "не
// надо выключать свет".
// reported speech past/third-person report verbs over a command.
// quotation quoted command, with and without a reporting frame.
// hypothetical если-scopes that are not real condition→command.
var Fixtures = []Fixture{
// negation — direct commands the prosecutor forbids
{"не выключай свет", Blocked, "negation", "direct prohibition"},
{"не перезапускай сервер", Blocked, "negation", "direct prohibition"},
{"не включай nginx", Blocked, "negation", "direct prohibition"},
{"не надо выключать свет", Blocked, "negation", "advisory negative"},
{"не стоит перезапускать nginx", Blocked, "negation", "advisory negative"},
{"не забудь напомнить про свет", Permissive, "negation", "prohibition-parser reminder exemption"},
// reported speech — reports an order, does not issue one
{"он сказал выключить свет", Blocked, "reported_speech", "past report verb + infinitive"},
{"она попросила перезапустить nginx", Blocked, "reported_speech", "past report verb + infinitive"},
{"мне сказали включить свет", Blocked, "reported_speech", "passive report + infinitive"},
{"он написал: «перезапусти nginx»", Blocked, "reported_speech", "report verb + quoted imperative"},
{"скажи мне, что он сказал про свет", Permissive, "reported_speech", "request to report, no commanded clause"},
{"расскажи про свет", Permissive, "reported_speech", "narrative request, not a reported order"},
// quotation — quoted text is referenced, not issued
{"фраза «выключи свет»", Blocked, "quotation", "reporting noun + quoted imperative"},
{"он сказал «выключи свет»", Blocked, "quotation", "report verb + quoted imperative"},
{"«выключи свет»", Ambiguous, "quotation", "bare quoted command, no frame"},
{"выключи свет", Permissive, "quotation", "unquoted imperative is a live command"},
// hypothetical
{"если выключить свет...", Blocked, "hypothetical", "conditional + infinitive + ellipsis"},
{"если бы перезапустить nginx...", Blocked, "hypothetical", "conditional + бы + infinitive"},
{"что будет если выключить свет", Blocked, "hypothetical", "question-scoped conditional"},
{"если будет дождь, выключи полив", Permissive, "hypothetical", "real condition → imperative"},
{"выключи свет если будет дождь", Permissive, "hypothetical", "imperative → real condition"},
// capability question — blocked even polite
{"ты можешь выключить свет?", Blocked, "capability_question", "ты + можешь + ?"},
{"ты можешь выключить свет", Blocked, "capability_question", "ты + можешь, no ?"},
{"ты можешь выключить свет, пожалуйста", Blocked, "capability_question", "ты + можешь + politeness (42/42 non-action)"},
{"умеешь ли ты выключить свет", Blocked, "capability_question", "ability form + ли"},
{"сможешь открыть окно, пожалуйста", Blocked, "capability_question", "bare future + politeness (7/7 non-action)"},
{"ты способна выключить свет", Blocked, "capability_question", "ты + способна"},
{"могу ли я выключить свет", Blocked, "capability_question", "first-person can + ли"},
{"можно ли выключить свет", Blocked, "capability_question", "можно + ли permission question"},
{"ты выключишь свет?", Ambiguous, "capability_question", "future tense + ? without can-form"},
// modal / polite requests — permissive
{"можешь выключить свет, пожалуйста", Permissive, "modal_request", "bare можешь + politeness (127/127 action)"},
{"пожалуйста, выключи свет", Permissive, "polite_request", "leading politeness + imperative"},
{"выключи свет, пожалуйста", Permissive, "polite_request", "imperative + trailing politeness"},
{"выключи свет", Permissive, "direct_imperative", "plain imperative"},
{"свет выключи, пожалуйста", Permissive, "reordered_target", "target-first imperative"},
{"can you выключи свет, please", Permissive, "modal_request", "English frame + Russian imperative + please (96/96 action)"},
{"can you выключи свет", Ambiguous, "modal_request", "English can without politeness"},
{"не мог бы ты выключить свет", Permissive, "polite_request", "conditional politeness, prohibition-parser exemption"},
{"можно выключить свет", Permissive, "modal_request", "можно + infinitive permission-implicature request"},
// first-person requests
{"я хочу выключить свет", Permissive, "first_person_request", "first-person + illocution"},
{"я хочу чтобы ты выключил свет", Permissive, "first_person_request", "first-person + embedded ya-you wish"},
{"надо выключить свет", Permissive, "first_person_request", "impersonal need"},
{"мне нужно включить свет", Permissive, "first_person_request", "first-person oblique + need"},
// ordinary questions — no execution pressure even when answerable
{"что запущено", Ambiguous, "question", "status question, no request evidence"},
{"какие службы работают", Ambiguous, "question", "question, no request evidence"},
{"сколько ламп включено", Ambiguous, "question", "question, no request evidence"},
{"что ты можешь включить", Blocked, "capability_question", "open question with ты + можешь"},
{"покажи что запущено", Permissive, "first_person_request", "imperative lead over a status question"},
}
@@ -0,0 +1,659 @@
// Guard is the slice-21 deterministic execution-frame engine (experiment-only).
//
// It answers one question: given an utterance, what is its execution-frame
// eligibility as a three-way gate — permissive, blocked, ambiguous — and why.
// It never decides what an utterance IS (that stays with the route classifier);
// it only decides whether an utterance may become an executable action at all.
// The policy is asymmetric on purpose: blocked and ambiguous must never
// execute, and permissive only means "no blocking speech-act evidence exists",
// not "execute this".
//
// It reuses the shipped deterministic routers rather than inventing new ones:
//
// router.ParseCommandProhibition / IsCommandProhibition direct negative commands
// morph.IsVerbForm / morph.Lemma verb mood and finiteness
// lexicon.IsFillerParticle / FirstPerson() politeness and first-person frames
//
// Everything else is closed-class evidence measured on the frozen slice-20 dev
// pool (§"measured discriminators" in the brief): 126 capability-question rows
// split 42/42/42 across ты-addressed, bare ability (умеешь), and bare future
// (сможешь) modality; 127 bare "можешь, пожалуйста" rows are 100% action;
// "can you … , please" (English frame + Russian imperative) is 100% action.
// The rules below are the encoding of precisely those numbers.
//
// The reason vocabulary is a closed set. Additions are design decisions that
// must land in the report, not silent new branches.
package main
import (
"strings"
"unicode"
"github.com/kami/maven/internal/lexicon"
"github.com/kami/maven/internal/morph"
"github.com/kami/maven/internal/router"
)
// Eligibility is the three-way execution-frame verdict.
type Eligibility int
const (
Permissive Eligibility = iota // no blocking speech-act evidence; downstream route decides
Blocked // a speech act forbids execution (negation, question, report, …)
Ambiguous // not enough evidence either way; must not execute
)
func (e Eligibility) String() string {
switch e {
case Permissive:
return "permissive"
case Blocked:
return "blocked"
default:
return "ambiguous"
}
}
// Reason is a closed set of structural explanations for a verdict.
type Reason string
const (
ReasonCommandProhibition Reason = "command_prohibition"
ReasonCapabilityQuestion Reason = "capability_question"
ReasonReportedSpeech Reason = "reported_speech"
ReasonQuotation Reason = "quotation"
ReasonHypothetical Reason = "hypothetical"
ReasonNegatedCommand Reason = "negated_command"
ReasonExplicitRequest Reason = "explicit_request"
ReasonAmbiguousModal Reason = "ambiguous_modal"
ReasonNoRequestEvidence Reason = "no_request_evidence"
)
func (r Reason) String() string { return string(r) }
// Frame is the verdict for one utterance. Eligibility decides; Reasons explain.
// A frame may carry more than one reason (e.g. a quoted reported command).
type Frame struct {
Eligibility Eligibility
Reasons []Reason
}
// maybeWord is a single-token or multi-token closed expression, e.g. the
// token "не мог бы" covers the three tokens не мog бы when matched as a
// contiguous run ("бы" is itself a bound marker). Multi-token members are
// matched over the reconstructed token text, never over raw text, so
// punctuation boundaries do not defeat them.
type maybeWord struct {
single []string
multi []string // matched as contiguous lowercased token runs
}
func (w maybeWord) in(toks []string, joined string) bool {
if hasAny(toks, w.single) {
return true
}
for _, m := range w.multi {
tm := strings.Join(tokens(m), " ")
if tm != "" && strings.Contains(joined, tm) {
return true
}
}
return false
}
// ── closed evidence sets (all measured on the slice-20 dev pool) ──────────
// wakeAddr is stripped from the left of an utterance before command-form
// detection: "мавен, выключи свет" and "выключи свет" must ride the same
// frame. Closed: the names Maven answers to in the dev pool.
var wakeAddr = []string{"мавен", "maven", "мавэн", "алекса", "алиса", "окей", "эй", "hey"}
// ruAddress are the second-person Russian address tokens. "ты можешь …"
// (with or without politeness) is 42/42 capability-question in the dev pool,
// so any addressed Russian can-form is a capability question, never a request.
var ruAddress = []string{"ты", "тебе", "тебя", "тобой", "тобою", "вы", "вас", "вам", "вами"}
// enAddress is the English second-person address. Unlike Russian, "can you …
// , please" is 96/96 action in the dev pool (English modal frame around a
// Russian imperative), so English address alone never blocks: it routes to the
// politeness arm.
var enAddress = []string{"you", "u", "your"}
// ruCanForms are the present-can verb forms. Bare (no address) "можешь …,
// пожалуйста" is 127/127 action; bare "можешь …" with no politeness is the
// ambiguous bucket (no such rows exist in dev — conservative default).
var ruCanForms = []string{"можешь", "можете", "могу", "можем"}
// ruAbilityForms are future/ability modal forms that read as a question of
// capability regardless of politeness: "сможешь открыть окно, пожалуйста" and
// "умеешь ли ты …" are 0/84 action in the dev pool, so even a polite bare
// form never grants execution. "мог(ла) бы …" and "смог(ла) бы …" are the
// conditional-politeness mask over the same boundary — except the leading
// politeness construction "не мог бы ты …", which the prohibition parser
// already classifies as ordinary modal politeness and must stay permissive.
var ruAbilityForms = maybeWord{
single: []string{
"сможешь", "сможете", "смогу", "сможем", "сумеешь", "сумеете",
"умеешь", "умеете", "способна", "способен", "способно", "способны",
"смог", "смогла", "смогли", "мог", "могла", "могли",
},
multi: []string{
"смог бы", "смогла бы", "смогли бы", "мог бы", "могла бы", "могли бы",
"смочь бы", "мочь бы",
},
}
// politeNegativeModal is the leading "не мог бы ты/вы …" politeness framing the
// prohibition parser exempts as ordinary modal politeness. When it leads the
// utterance the capability stage declines and the frame reads as a request.
var politeNegativeModal = []string{
"не мог бы", "не могла бы", "не могли бы", "не смог бы", "не смогла бы", "не смогли бы",
}
// enCanForms are the English modal can/could tokens.
var enCanForms = []string{"can", "could"}
// politeness is the closed set of politeness fillers. пожалуйста/плиз/please
// are already closed-class filler particles in the lexicon; the добр-forms
// are the only additions the dev pool exercises.
var politeness = maybeWord{
single: []string{"пожалуйста", "плиз", "please"},
multi: []string{"будь добр", "будьте добры", "был бы добр", "были бы добры"},
}
// reportVerbs are the past/third-person report verbs — the frame that reports
// a command rather than issuing it. Second-person imperatives ("скажи",
// "расскажи", "напомни") are deliberately absent: those are requests to
// report, and their clause forms part of the current utterance, not a
// replayed order. Matched as closed list (a report verb outside it is a data
// gap, noted in the report).
var reportVerbs = []string{
"сказал", "сказала", "сказали", "говорил", "говорила", "говорили",
"говорит", "говорят", "попросил", "попросила", "попросили",
"просил", "просила", "просили", "написал", "написала", "написали",
"пишет", "приказал", "приказала", "приказали", "велел", "велела",
"велели", "скомандовал", "скомандовала", "рекомендовал", "рекомендовала",
"посоветовал", "посоветовала", "сообщил", "сообщила", "сообщили",
"объявил", "объявила", "велено", "сказано", "написано", "записано",
}
// reportNouns name a quoted or reported text: "фраза «выключи свет»" is a
// quotation, not a command.
var reportNouns = []string{
"фраза", "фразы", "фразе", "фразу", "слово", "слова", "слове", "словом",
"выражение", "выражения", "цитата", "цитату", "цитате",
"название", "текст", "сообщение", "письмо", "заметка", "заметку",
}
// hypothesisMarkers open a conditional scope.
var hypothesisMarkers = []string{"если", "ежели", "коли", "кабы", "if"}
// illocutionVerbs make a first-person or impersonal clause a request even
// without an imperative form ("я хочу …", "мне нужно …", "надо …").
var illocutionVerbs = maybeWord{
single: []string{
"хочу", "хотел", "хотела", "хотелось", "желаю", "прошу", "просим",
"просил", "просила", "просили", "попросить",
"надо", "нужно", "следует", "пора", "требуется", "придётся", "придется",
"могу", "давай", "давайте",
},
multi: []string{
"хотел бы", "хотела бы", "хочу чтобы", "хотел чтобы", "хотела чтобы",
"могу ли",
},
}
// ── token helpers ─────────────────────────────────────────────────────────
// tokens lowercases and splits on anything that is not a letter or digit,
// matching the router's planTokens discipline ("что-дальше" tokenises like
// "что дальше").
func tokens(text string) []string {
return strings.FieldsFunc(strings.ToLower(text), func(r rune) bool {
return !unicode.IsLetter(r) && !unicode.IsDigit(r)
})
}
func hasTok(toks []string, w string) bool {
for _, t := range toks {
if t == w {
return true
}
}
return false
}
func hasAny(toks, ws []string) bool {
for _, w := range ws {
if hasTok(toks, w) {
return true
}
}
return false
}
func indexTok(toks []string, w string) int {
for i, t := range toks {
if t == w {
return i
}
}
return -1
}
// isFiniteVerb reports a verb form that is not the dictionary (infinitive)
// form: "выключи" is finite, "выключить" is not. A finite verb at command
// position is positive request evidence; an infinitive is not.
func isFiniteVerb(tok string) bool {
if !morph.IsVerbForm(tok) {
return false
}
return morph.Lemma(tok) != tok
}
// isInfinitive reports a token that morph resolves to its own dictionary form
// (the lemma ends in the infinitive ending by construction).
func isInfinitive(tok string) bool {
if !morph.IsVerbForm(tok) {
return false
}
return morph.Lemma(tok) == tok
}
func anyInfinitive(toks []string) bool {
for _, t := range toks {
if isInfinitive(t) {
return true
}
}
return false
}
func anyFiniteVerb(toks []string) bool {
for _, t := range toks {
if isFiniteVerb(t) {
return true
}
}
return false
}
func hasAnyVerb(toks []string) bool {
for _, t := range toks {
if morph.IsVerbForm(t) {
return true
}
}
return false
}
func blocked(rs ...Reason) Frame { return Frame{Eligibility: Blocked, Reasons: rs} }
func ambiguous(rs ...Reason) Frame {
return Frame{Eligibility: Ambiguous, Reasons: rs}
}
func permissive(rs ...Reason) Frame {
return Frame{Eligibility: Permissive, Reasons: rs}
}
// ── quoted spans ─────────────────────────────────────────────────────────
// quotedSpan is a maximal quoted interval in the normalized text.
type quotedSpan struct{ content string }
// quotePairs covers the quoting styles the dev pool and brief fixtures use:
// Russian guillemets, curly double/single quotes, and straight quotes.
var quotePairs = []struct{ open, close string }{
{"«", "»"}, {"„", "\""}, {"“", "”"}, {"", ""}, {"", ""}, {"'", "'"}, {"\"", "\""},
}
// extractQuotedSpans returns the contents of quoted spans in order, in rune
// index space (the text is normalized, so glyphs are single runes). An
// unbalanced delimiter yields no span (best-effort; the conservative
// fallback then applies).
func extractQuotedSpans(t string) []quotedSpan {
runes := []rune(t)
var out []quotedSpan
i := 0
for i < len(runes) {
matched := false
for _, p := range quotePairs {
po := []rune(p.open)
pc := []rune(p.close)
if i+len(po) > len(runes) || string(runes[i:i+len(po)]) != p.open {
continue
}
j := i + len(po)
for j+len(pc) <= len(runes) && string(runes[j:j+len(pc)]) != p.close {
j++
}
out = append(out, quotedSpan{content: string(runes[i+len(po) : j])})
i = j + len(pc)
matched = true
break
}
if !matched {
i++
}
}
return out
}
// ── rule stages (evaluated in this order; a decision is final) ────────────
// Evaluate derives the execution-frame verdict for one utterance.
func Evaluate(text string) Frame {
t := router.NormalizeMatchText(text)
if strings.TrimSpace(t) == "" {
return ambiguous(ReasonNoRequestEvidence)
}
toks := tokens(t)
joined := strings.Join(toks, " ")
// 1. Quotation: a command inside a quoted span is not a command being
// issued now. With a reporting frame outside it is a quotation; a bare
// quote is at best ambiguous.
if f, ok := stageQuotation(t, toks, joined); ok {
return f
}
// 2. Reported speech: a past/third-person report verb governing a command
// clause reports an order to someone else, it does not issue one.
if f, ok := stageReport(t, toks, joined); ok {
return f
}
// 3. Hypothetical: a command scope opened by "если/if" that does not
// continue as a real condition→command is not an execution request.
if f, ok := stageHypothesis(toks, joined); ok {
return f
}
// 4. Direct negative commands: the shipped prohibition parser.
if router.IsCommandProhibition(t) {
return blocked(ReasonCommandProhibition)
}
// 5. Advisory negatives: "не надо/не стоит/не нужно …".
if f, ok := stageAdvisoryNegation(toks); ok {
return f
}
// 6. Capability and permission modality (the measured core).
if f, ok := stageCapability(toks, joined); ok {
return f
}
// 7. Trailing question mark with no modal at play: an uncertain posture,
// never a confirmed executable request.
if strings.HasSuffix(t, "?") {
return ambiguous(ReasonAmbiguousModal)
}
// 8. Positive request evidence.
if hasRequestEvidence(toks, joined) {
return permissive(ReasonExplicitRequest)
}
// 9. No execution pressure at all.
return ambiguous(ReasonNoRequestEvidence)
}
// stageQuotation blocks a quoted command when a reporting frame surrounds it.
func stageQuotation(t string, toks []string, joined string) (Frame, bool) {
spans := extractQuotedSpans(t)
if len(spans) == 0 {
return Frame{}, false
}
commandSpan := false
for _, sp := range spans {
if isCommandishWithin(tokens(sp.content), strings.Join(tokens(sp.content), " ")) {
commandSpan = true
break
}
}
if !commandSpan {
return Frame{}, false
}
reasons := []Reason{ReasonQuotation}
if hasReportFrame(toks, joined) {
reasons = append(reasons, ReasonReportedSpeech)
return Frame{Eligibility: Blocked, Reasons: reasons}, true
}
// a bare quoted command has no reporting frame: refusable but not a
// definite prohibition either (it is at least ambiguous)
return Frame{Eligibility: Ambiguous, Reasons: reasons}, true
}
// isCommandishWithin reports the span content carrying command or capability
// polarity itself — imperative, prohibition, or a can-form.
func isCommandishWithin(toks []string, joined string) bool {
if len(toks) == 0 {
return false
}
if router.IsCommandProhibition(strings.Join(toks, " ")) {
return true
}
if hasAny(toks, ruCanForms) || ruAbilityForms.in(toks, joined) || hasAny(toks, enCanForms) {
return true
}
return anyFiniteVerb(toks)
}
func hasReportFrame(toks []string, joined string) bool {
if hasAny(toks, reportVerbs) {
return true
}
return hasAny(toks, reportNouns)
}
// stageReport blocks when a report frame governs a command clause: an
// infinitive after the report verb, or a quoted imperative. Second-person
// imperatives like "скажи/расскажи" are not in reportVerbs, so a request to
// report ("расскажи мне, что сказал папа") passes through.
func stageReport(t string, toks []string, joined string) (Frame, bool) {
if !hasReportFrame(toks, joined) {
return Frame{}, false
}
last := -1
for i, w := range toks {
if hasTok(reportVerbs, w) || hasTok(reportNouns, w) {
last = i
}
}
if last < 0 {
return Frame{}, false
}
after := toks[last+1:]
if len(after) == 0 {
return Frame{}, false
}
// a quoted command after the frame counts as the governed clause
for _, sp := range extractQuotedSpans(t) {
if isCommandishWithin(tokens(sp.content), strings.Join(tokens(sp.content), " ")) {
return blocked(ReasonReportedSpeech, ReasonQuotation), true
}
}
if anyInfinitive(after) || hasAny(after, []string{"что", "чтобы", "чтоб"}) {
return blocked(ReasonReportedSpeech), true
}
return Frame{}, false
}
// stageHypothesis blocks a conditional scope whose clauses are hypothetical
// (infinitive or subjunctive "бы") rather than a real condition→command.
// "если будет дождь, выключи полив" keeps its imperative continuation and
// passes through; it is a real conditional request, not a hypothetical.
func stageHypothesis(toks []string, joined string) (Frame, bool) {
idx := -1
for _, m := range hypothesisMarkers {
if i := indexTok(toks, m); i >= 0 && (idx < 0 || i < idx) {
idx = i
}
}
if idx < 0 {
return Frame{}, false
}
post := toks[idx+1:]
if len(post) == 0 || hasTok(post, "бы") || anyInfinitive(post) {
return blocked(ReasonHypothetical), true
}
// a real condition clause is not hypothetical: «если будет дождь,
// выключи полив» is a request. The dev dict does not cover «будет», so
// the imperative is looked for anywhere, not just after the marker
// («выключи свет, если будет дождь»).
if !anyFiniteVerb(toks) {
return blocked(ReasonHypothetical), true
}
return permissive(ReasonExplicitRequest), true
}
// stageAdvisoryNegation blocks "не надо/не нужно/не стоит/не следует …".
// (absent from the dev pool; covered by brief fixtures)
func stageAdvisoryNegation(toks []string) (Frame, bool) {
if len(toks) < 3 || toks[0] != "не" {
return Frame{}, false
}
if !hasTok(toks[1:2], "надо") && !hasTok(toks[1:2], "нужно") &&
!hasTok(toks[1:2], "стоит") && !hasTok(toks[1:2], "следует") &&
!hasTok(toks[1:2], "требуется") {
return Frame{}, false
}
rest := toks[2:]
if anyInfinitive(rest) || anyFiniteVerb(rest) || hasAnyVerb(rest) {
return blocked(ReasonNegatedCommand), true
}
return Frame{}, false
}
// stageCapability encodes the measured modal matrix. Returns a decision when
// modality alone settles the frame.
func stageCapability(toks []string, joined string) (Frame, bool) {
// "не мог бы ты …, пожалуйста" style conditional politeness is ordinary
// modal politeness (the prohibition parser exempts it as such): a request.
for _, pref := range politeNegativeModal {
if strings.HasPrefix(joined, pref) {
return permissive(ReasonExplicitRequest), true
}
}
// "… ли" directly after a can-form is a polar capability question:
// "могу ли я …", "можешь ли ты …", "умеешь ли ты …", "можно ли …".
// Checked before the modality arms so the polar reading wins.
if hasTok(toks, "ли") {
for i := 1; i < len(toks); i++ {
if toks[i] != "ли" {
continue
}
prev := toks[i-1]
if hasTok(ruCanForms, prev) || prev == "можно" || hasTok(ruAbilityForms.single, prev) {
return blocked(ReasonCapabilityQuestion), true
}
}
}
ruAddr := hasAny(toks, ruAddress)
ruCan := hasAny(toks, ruCanForms)
ruAbil := ruAbilityForms.in(toks, joined)
enCan := hasAny(toks, enCanForms)
polite := politeness.in(toks, joined)
// addressed Russian can-form: capability question, always blocked.
// ("ты можешь выключить свет, пожалуйста" included — 42/42 non-action.)
if ruAddr && (ruCan || ruAbil) {
return blocked(ReasonCapabilityQuestion), true
}
// ability forms (future/conditional/умеешь) are capability even bare and
// even polite: "сможешь открыть окно, пожалуйста" is 7/7 non-action.
if ruAbil && !ruCan {
return blocked(ReasonCapabilityQuestion), true
}
// bare Russian present can-form: politeness is the request marker.
if ruCan && !ruAddr {
if polite {
return Frame{}, false // modal-request positive evidence is found later
}
return ambiguous(ReasonAmbiguousModal), true
}
// English can/could: "can you …, please" is a request (96/96 action in the
// dev pool; the frame wraps a Russian imperative). Without politeness it
// reads as a capability question and stays ambiguous.
if enCan && !ruAddr {
if polite {
return Frame{}, false // positive modal-request evidence later
}
return ambiguous(ReasonAmbiguousModal), true
}
// "можно" (permission): "можно ли …" is a permission question; a bare
// "можно …" is a politeness-implicature request.
if hasTok(toks, "можно") {
if hasTok(toks, "ли") {
return blocked(ReasonCapabilityQuestion), true
}
return Frame{}, false
}
// bare "могу": a self-capability statement, not a request.
if hasTok(toks, "могу") && !hasTok(toks, "ли") {
return ambiguous(ReasonAmbiguousModal), true
}
return Frame{}, false
}
// hasRequestEvidence is the positive permissive trigger, reached only after
// every block/ambiguity stage above has declined.
func hasRequestEvidence(toks []string, joined string) bool {
polite := politeness.in(toks, joined)
enCan := hasAny(toks, enCanForms)
// 1. politeness + a verb (or an English modal) is explicit request
// evidence: "можешь выключить свет, пожалуйста", "can you останови …,
// please", "выключи свет, пожалуйста".
if polite && (hasAnyVerb(toks) || enCan) {
return true
}
// 2. first-person illocution frame: "я хочу …", "мне нужно …".
if hasAny(toks, lexicon.FirstPerson()) && illocutionVerbs.in(toks, joined) {
return true
}
// 3. impersonal need: "надо …", "нужно …", "пора …".
if hasAny(toks, []string{"надо", "нужно", "следует", "пора", "требуется", "придётся", "придется"}) {
return true
}
// 4. permission-implicature request: "можно выключить свет".
if hasTok(toks, "можно") && !hasTok(toks, "ли") {
return true
}
// 5. reminder request in the parser's own exemption scope: the
// prohibition parser declines «не забудь напомнить про свет» as a
// reminder, not a prohibition — carry that into a request.
if strings.HasPrefix(joined, "не забудь") && hasAny(toks, lexicon.ReminderVerbs()) {
return true
}
// 6. leading finite verb (imperative or otherwise tensed verb at command
// position): "выключи свет", "покажи что запущено". Address and filler
// particles are stripped first, so "мавен, выключи свет" rides the same
// frame.
lead := toks
for len(lead) > 0 {
first := lead[0]
if !lexicon.IsFillerParticle(first) && !hasTok(wakeAddr, first) && !hasTok(ruAddress, first) {
break
}
lead = lead[1:]
}
if len(lead) > 0 && isFiniteVerb(lead[0]) {
return true
}
return false
}
@@ -0,0 +1,139 @@
package main
import (
"encoding/json"
"os"
"strings"
"testing"
"github.com/kami/maven/internal/morph"
)
func TestEvaluateFixtures(t *testing.T) {
for _, fx := range Fixtures {
got := Evaluate(fx.Utterance)
if got.Eligibility != fx.Want {
t.Errorf("%s: want %s got %s (reasons %v)", fx.Utterance, fx.Want, got.Eligibility, got.Reasons)
}
}
}
// TestMeasuredDiscriminators pins the corpus-verified numbers on the frozen
// pool. These are the exact measurements the rules were built on, so a change
// that moves them is a rule regression visible in the slice.
func loadPool(t *testing.T) []Row {
t.Helper()
b, err := os.ReadFile("/tmp/mvn-s21/pool.json")
if os.IsNotExist(err) {
t.Skip("pool.json missing; run slice21_emit.py first")
}
if err != nil {
t.Fatal(err)
}
var rows []Row
if err := json.Unmarshal(b, &rows); err != nil {
t.Fatal(err)
}
return rows
}
func TestDevCapabilityProhibition(t *testing.T) {
if !morph.Available() {
t.Skip("morph dict unavailable")
}
rows := loadPool(t)
var n, pass, blockedN, ambig int
for _, r := range rows {
if !hasTok(r.Tags, "capability_question") {
continue
}
n++
switch Evaluate(variants(r.NText)[vOrig]).Eligibility {
case Permissive:
pass++
case Blocked:
blockedN++
case Ambiguous:
ambig++
}
}
if n != 126 {
t.Fatalf("capability-question rows = %d, want 126", n)
}
if pass != 0 {
t.Fatalf("capability-question dangerous pass = %d, want 0", pass)
}
if blockedN != 126 && ambig != 0 {
t.Errorf("blocked=%d ambig=%d, expect all 126 blocked", blockedN, ambig)
}
}
func TestDevBareCanPoliteIsAction(t *testing.T) {
if !morph.Available() {
t.Skip("morph dict unavailable")
}
rows := loadPool(t)
total, action, perm := 0, 0, 0
for _, r := range rows {
toks := tokens(r.NText)
if !hasTok(toks, "можешь") || hasAny(toks, ruAddress) ||
hasAny(toks, append([]string{}, ruAbilityForms.single...)) {
continue
}
total++
if r.Route == "action" {
action++
}
if Evaluate(r.NText).Eligibility == Permissive {
perm++
}
}
if total != 127 || action != 127 {
t.Fatalf("bare-можешь+polite: n=%d action=%d, want 127/127", total, action)
}
if perm != 127 {
t.Fatalf("bare-можешь+polite permissive=%d, want 127", perm)
}
}
func TestDevEnglishCanPoliteIsAction(t *testing.T) {
if !morph.Available() {
t.Skip("morph dict unavailable")
}
rows := loadPool(t)
total, action, perm := 0, 0, 0
for _, r := range rows {
toks := tokens(r.NText)
if !hasAny(toks, enCanForms) {
continue
}
total++
if r.Route == "action" {
action++
}
if Evaluate(r.NText).Eligibility == Permissive {
perm++
}
}
if total != 96 || action != 96 {
t.Fatalf("can-rows: n=%d action=%d, want 96/96", total, action)
}
if perm != 96 {
t.Fatalf("can-rows permissive=%d, want 96", perm)
}
}
func TestNoPermissiveRowEndsInQuestion(t *testing.T) {
// asymmetric posture: the guard must never approve a trailing-? frame --
// the corpus has 0 action rows ending in "?", and approving any would
// bet on punctuation the slice has ruled uncertain.
rows := loadPool(t)
for _, r := range rows {
if !strings.HasSuffix(r.NText, "?") {
continue
}
if got := Evaluate(r.NText).Eligibility; got == Permissive {
t.Errorf("canonical '?': %q approved (%s)", r.Text, got)
}
}
}
@@ -0,0 +1,566 @@
// Slice 21 runner: report the deterministic execution-frame guard against the
// frozen slice-20 dev pool.
//
// Reads the emit step's compact files (pool.json, pairs.json, sparse_oof.json
// in /tmp/mvn-s21) and prints the report tables plus a machine-readable
// guard_results.json. Reuses router/morph/lexicon parsers live inside this
// module — the pool texts are the only data, no embedding is recomputed.
//
// Usage: go run ./cmd/semantic-router-experiment/slice21
package main
import (
"encoding/json"
"flag"
"fmt"
"os"
"regexp"
"sort"
"strings"
"unicode"
)
const sparseThreshold = 0.715 // slice-18 §4 strict operating point (P>=0.95 best recall)
var familyPriority = []string{
"capability_question", "question", "first_person_request",
"modal_request", "polite_request", "reordered_target", "direct_imperative",
}
func familyOf(tags []string) string {
for _, f := range familyPriority {
if hasTok(tags, f) {
return f
}
}
return "other"
}
// ── pool row ──────────────────────────────────────────────────────────────
type Row struct {
Idx int `json:"idx"`
Text string `json:"text"`
NText string `json:"n_text"`
Route string `json:"route"`
Y int `json:"y"`
Tags []string `json:"tags"`
CVFold int `json:"cv_fold"`
SplitGp string `json:"split_group"`
SourceID string `json:"source_id"`
Family string
VariantOf int
}
type Pair struct{ Cap, Act int }
// ── stress variants ───────────────────────────────────────────────────────
var nofinalRe = regexp.MustCompile(`[?.!,;:]+$`)
// strip_punct mirrors the slice-18/19 python strip_punct: trailing sentence
// punctuation, then every non-word/non-space rune.
func stripPunct(t string) string {
t = nofinalRe.ReplaceAllString(strings.TrimSpace(t), "")
out := make([]rune, 0, len(t))
var prevSpace bool
for _, r := range t {
if unicode.IsLetter(r) || unicode.IsNumber(r) {
out = append(out, r)
prevSpace = false
} else if !prevSpace {
out = append(out, ' ')
prevSpace = true
}
}
return strings.TrimSpace(string(out))
}
func variants(nText string) [3]string {
return [3]string{
nText,
nofinalRe.ReplaceAllString(strings.TrimSpace(nText), ""),
stripPunct(nText),
}
}
const (
vOrig = iota
vNofinal
vStrip
)
var variantName = [3]string{"orig", "nofinal", "strip"}
type result struct {
Frame Frame `json:"frame"`
}
// ── metrics ──────────────────────────────────────────────────────────────
type triTab struct {
Permissive, Blocked, Ambiguous int
PermNonact, BlockedAction, AmbAction int
}
type runAgg struct {
n, action, tp, fp, fn int
approved int
capPermissive int
}
func (a *runAgg) addApproved(approved bool, route string) {
a.n++
if route == "action" {
a.action++
}
if approved {
a.approved++
if route == "action" {
a.tp++
} else {
a.fp++
}
} else if route == "action" {
a.fn++
}
}
func (a *runAgg) P() string { return fmtPct(frac(a.tp, a.tp+a.fp)) }
func (a *runAgg) R() string { return fmtPct(frac(a.tp, a.action)) }
func (a *runAgg) FA() int { return a.fp }
func (a *runAgg) FArate() string {
return fmtPct(frac(a.fp, a.n))
}
func maxi(a, b int) int {
if a > b {
return a
}
return b
}
// frac is the guarded ratio the tables print (0/0 is 0).
func frac(num, den int) float64 { return float64(num) / float64(maxi(den, 1)) }
func fmtPct(v float64) string { return fmt.Sprintf("%.1f%%", 100*v) }
// ── main ─────────────────────────────────────────────────────────────────
func main() {
poolPath := flag.String("pool", "/tmp/mvn-s21/pool.json", "dev pool rows")
pairsPath := flag.String("pairs", "/tmp/mvn-s21/pairs.json", "cap-vs-action pairs")
sparsePath := flag.String("sparse", "/tmp/mvn-s21/sparse_oof.json", "slice-18 both OOF proba")
outPath := flag.String("out", "/tmp/mvn-s21/guard_results.json", "machine-readable results")
flag.Parse()
rows := mustLoad[[]Row](*poolPath)
// pairs.json is bare [cap, act] index pairs; adapt into typed pairs.
rawPairs := mustLoad[[][2]int](*pairsPath)
pairs := make([]Pair, 0, len(rawPairs))
for _, rp := range rawPairs {
pairs = append(pairs, Pair{Cap: rp[0], Act: rp[1]})
}
sparseOOF := mustLoad[[]struct {
Idx int `json:"idx"`
Proba float64 `json:"proba"`
}](*sparsePath)
proba := make([]float64, len(rows))
for _, s := range sparseOOF {
proba[s.Idx] = s.Proba
}
famPrio := 0
for i := range rows {
rows[i].Family = familyOf(rows[i].Tags)
if rows[i].Family != "other" {
famPrio++
}
}
_ = famPrio
// verdicts per variant
type rowRes struct {
Idx int `json:"idx"`
Text string `json:"text"`
Route string `json:"route"`
Family string `json:"family"`
Tags []string `json:"tags"`
Frames map[string]string `json:"frames"` // variant -> eligibility
}
perVariant := make([][3]Frame, len(rows))
fmt.Println("slice 21 — deterministic execution-frame guard on slice-20 dev pool")
fmt.Println("==================================================================")
for i, r := range rows {
vs := variants(r.NText)
var fr [3]Frame
for vi := 0; vi < 3; vi++ {
fr[vi] = Evaluate(vs[vi])
}
perVariant[i] = fr
}
// ── §3 three-way cross-tab (orig) ─────────────────────────────────────
fmt.Println("\n## 1. Three-way eligibility × route (orig)")
tab := triTab{}
for i, r := range rows {
switch perVariant[i][vOrig].Eligibility {
case Permissive:
tab.Permissive++
if r.Route != "action" {
tab.PermNonact++
}
case Blocked:
tab.Blocked++
if r.Route == "action" {
tab.BlockedAction++
}
case Ambiguous:
tab.Ambiguous++
if r.Route == "action" {
tab.AmbAction++
}
}
}
fmt.Printf("permissive: %d blocked: %d ambiguous: %d\n", tab.Permissive, tab.Blocked, tab.Ambiguous)
fmt.Printf(" permissive non-action: %d blocked action: %d ambiguous action: %d\n",
tab.PermNonact, tab.BlockedAction, tab.AmbAction)
// ── §4 binary executable-gate metrics on orig ─────────────────────────
fmt.Println("\n## 2. Binary executable gate (approve = permissive; deny = blocked|ambiguous)")
a := runAgg{}
capCov, capPerm, capAmb := 0, 0, 0
for i, r := range rows {
el := perVariant[i][vOrig].Eligibility
a.addApproved(el == Permissive, r.Route)
if hasTok(r.Tags, "capability_question") {
capCov++
switch el {
case Permissive:
capPerm++
case Ambiguous:
capAmb++
}
}
}
fmt.Printf("approved: %d denied: %d (n=%d, action=%d)\n", a.approved, a.n-a.approved, a.n, a.action)
fmt.Printf("action precision %s recall %s FA %d (%s)\n", a.P(), a.R(), a.FA(), a.FArate())
fmt.Printf("capability-question dangerous pass: %d / %d (rate %s)\n",
capPerm, capCov, fmtPct(frac(capPerm, capCov)))
fmt.Printf("capability-question blocked %d, ambiguous %d\n", capCov-capPerm-capAmb, capAmb)
// ── §15 family stress (orig) ──────────────────────────────────────────
fmt.Println("\n## 3. Family stress (orig; counts per eligibility)")
fmt.Printf("%-24s %8s %8s %8s %8s\n", "family", "n", "perm", "block", "ambig")
famOrder := []string{"direct_imperative", "polite_request", "modal_request", "first_person_request",
"reordered_target", "capability_question", "question", "other"}
famAgg := map[string]*triTab{}
for _, f := range famOrder {
famAgg[f] = &triTab{}
}
for i, r := range rows {
t := famAgg[r.Family]
if t == nil {
continue
}
switch perVariant[i][vOrig].Eligibility {
case Permissive:
t.Permissive++
if r.Route != "action" {
t.PermNonact++
}
case Blocked:
t.Blocked++
case Ambiguous:
t.Ambiguous++
}
}
for _, f := range famOrder {
t := famAgg[f]
if t == nil {
continue
}
n := t.Permissive + t.Blocked + t.Ambiguous
if n == 0 {
continue
}
fmt.Printf("%-24s %8d %8d %8d %8d\n", f, n, t.Permissive, t.Blocked, t.Ambiguous)
}
// ── §cap-Q LOFO across stress variants ────────────────────────────────
fmt.Println("\n## 4. Capability-question dangerous pass by stress variant")
for vi := 0; vi < 3; vi++ {
cp, cb, ca := 0, 0, 0
for i, r := range rows {
if !hasTok(r.Tags, "capability_question") {
continue
}
switch perVariant[i][vi].Eligibility {
case Permissive:
cp++
case Blocked:
cb++
case Ambiguous:
ca++
}
}
fmt.Printf(" %-8s dangerous-pass %d blocked %d ambiguous %d\n",
variantName[vi], cp, cb, ca)
}
// ── §pair test ────────────────────────────────────────────────────────
fmt.Println("\n## 5. Paired action/capability (cap row must never clear)")
capClear, actPerm, actAmbig, actBlock := 0, 0, 0, 0
for _, p := range pairs {
cel := perVariant[p.Cap][vOrig].Eligibility
ael := perVariant[p.Act][vOrig].Eligibility
if cel == Permissive {
capClear++
}
switch ael {
case Permissive:
actPerm++
case Ambiguous:
actAmbig++
case Blocked:
actBlock++
}
}
fmt.Printf("pairs %d: cap cleared %d (rate %s), action permissive %d, action ambiguous %d, action blocked %d\n",
len(pairs), capClear, fmtPct(frac(capClear, len(pairs))),
actPerm, actAmbig, actBlock)
// ── safe composition §17 ─────────────────────────────────────────────
fmt.Println("\n## 6. Composition: guard-alone / sparse-alone / guard→sparse (orig)")
compose := map[string]*runAgg{
"guard_alone": {},
"sparse_alone": {},
"guard_sparse": {},
}
for i, r := range rows {
gPerm := perVariant[i][vOrig].Eligibility == Permissive
sPerm := proba[i] >= sparseThreshold
compose["guard_alone"].addApproved(gPerm, r.Route)
compose["sparse_alone"].addApproved(sPerm, r.Route)
compose["guard_sparse"].addApproved(gPerm && sPerm, r.Route)
}
fmt.Printf("%-14s %8s %8s %6s %10s %6s %10s\n", "policy", "P", "R", "FA", "FA rate", "capQ", "capQ rate")
for _, name := range []string{"guard_alone", "sparse_alone", "guard_sparse"} {
agg := compose[name]
capQ := 0
for i, r := range rows {
if !hasTok(r.Tags, "capability_question") {
continue
}
ok := false
switch name {
case "guard_alone":
ok = perVariant[i][vOrig].Eligibility == Permissive
case "sparse_alone":
ok = proba[i] >= sparseThreshold
case "guard_sparse":
ok = perVariant[i][vOrig].Eligibility == Permissive && proba[i] >= sparseThreshold
}
if ok {
capQ++
}
}
fmt.Printf("%-14s %8s %8s %6d %10s %6d %10s\n", name, agg.P(), agg.R(), agg.FA(),
agg.FArate(), capQ, fmtPct(float64(capQ)/126))
}
// composition on strip too (brief §16 voice stress)
fmt.Println("\n## 7. Composition on punctuation-stripped text (strip)")
c2 := runAgg{}
capQ2 := 0
for i, r := range rows {
gPerm := perVariant[i][vStrip].Eligibility == Permissive
ok := gPerm && proba[i] >= sparseThreshold
c2.addApproved(ok, r.Route)
if hasTok(r.Tags, "capability_question") && ok {
capQ2++
}
}
fmt.Printf("guard→sparse strip: P %s R %s FA %d (%s) capQ pass %d\n",
c2.P(), c2.R(), c2.FA(), c2.FArate(), capQ2)
// ── §19 manual classification scratch ─────────────────────────────────
fmt.Println("\n## 8. Manual classification (scan material written to manual_class.json)")
var dangerous []map[string]any
var permNonact []map[string]any
var deniedAction []map[string]any
for i, r := range rows {
fr := perVariant[i][vOrig]
if hasTok(r.Tags, "capability_question") && fr.Eligibility == Permissive {
dangerous = append(dangerous, map[string]any{
"idx": r.Idx, "text": r.Text, "route": r.Route,
"reasons": fr.Reasons,
})
}
if fr.Eligibility == Permissive && r.Route != "action" {
permNonact = append(permNonact, map[string]any{
"idx": r.Idx, "text": r.Text, "route": r.Route,
"family": r.Family, "reasons": fr.Reasons,
})
}
if fr.Eligibility != Permissive && r.Route == "action" {
deniedAction = append(deniedAction, map[string]any{
"idx": r.Idx, "text": r.Text, "family": r.Family,
"eligibility": fr.Eligibility.String(), "reasons": fr.Reasons,
})
}
}
writeManual(permNonact, deniedAction, dangerous)
fmt.Printf("dangerous passes: %d permissive non-action: %d denied action: %d\n",
len(dangerous), len(permNonact), len(deniedAction))
groupAndSample("permissive non-action by reason+family", permNonact, 4)
groupAndSample("denied action by reason+family", deniedAction, 4)
// ── fixtures ──────────────────────────────────────────────────────────
fmt.Println("\n## 9. Brief fixtures")
pass := 0
for _, fx := range Fixtures {
got := Evaluate(fx.Utterance)
mark := "ok "
if got.Eligibility != fx.Want {
mark = "FAIL"
} else {
pass++
}
if got.Eligibility != fx.Want {
fmt.Printf(" %s %-14s want %-10s got %-10s %s\n", mark, fx.Family,
fx.Want, got.Eligibility.String(), fx.Utterance)
}
}
fmt.Printf("fixtures: %d/%d passed\n", pass, len(Fixtures))
// write result file
rr := make([]rowRes, 0, len(rows))
for i, r := range rows {
fr := [3]string{"", "", ""}
for vi := 0; vi < 3; vi++ {
fr[vi] = perVariant[i][vi].Eligibility.String()
}
rr = append(rr, rowRes{
Idx: r.Idx, Text: r.Text, Route: r.Route, Family: r.Family, Tags: r.Tags,
Frames: map[string]string{
"orig": fr[vOrig], "nofinal": fr[vNofinal], "strip": fr[vStrip],
},
})
}
if *outPath != "" {
mustSave(*outPath, map[string]any{
"pool": "/tmp/mvn-s21/pool.json",
"rows": rr,
"aggregates": map[string]any{
"tab": tab,
"capq_pass": capPerm,
"capq_blocked": capCov - capPerm - capAmb,
"capq_ambiguous": capAmb,
"binary": map[string]any{"tp": a.tp, "fp": a.fp, "fn": a.fn, "approved": a.approved, "n": a.n},
"pairs": map[string]any{"n": len(pairs), "cap_cleared": capClear, "act_permissive": actPerm},
"guard_sparse": map[string]any{"tp": compose["guard_sparse"].tp, "fp": compose["guard_sparse"].fp, "fn": compose["guard_sparse"].fn},
"dangerous_passes": len(dangerous),
"perm_nonact_count": len(permNonact),
"denied_action": len(deniedAction),
},
})
fmt.Println("wrote", *outPath)
}
}
// ── manual classification helpers ─────────────────────────────────────────
func writeManual(permNonact, deniedAction, dangerous []map[string]any) {
writeJSON("/tmp/mvn-s21/manual_class.json", map[string]any{
"dangerous_passes": dangerous,
"permissive_non_action": permNonact,
"denied_action": deniedAction,
})
}
func groupAndSample(title string, rows []map[string]any, sample int) {
type g struct {
key string
n int
texts []string
}
groups := map[string]*g{}
var order []string
for _, r := range rows {
var family, reason, el string
if v, ok := r["family"].(string); ok {
family = v
}
if v, ok := r["eligibility"].(string); ok {
el = v
}
if rs, ok := r["reasons"].([]Reason); ok {
rs2 := make([]string, len(rs))
for k, rr := range rs {
rs2[k] = rr.String()
}
reason = strings.Join(rs2, ",")
} else if rs, ok := r["reasons"].([]string); ok {
reason = strings.Join(rs, ",")
}
key := fmt.Sprintf("family=%s elig=%s reason=%s", family, el, reason)
if _, ok := groups[key]; !ok {
groups[key] = &g{key: key}
order = append(order, key)
}
groups[key].n++
if len(groups[key].texts) < sample {
groups[key].texts = append(groups[key].texts, firstN(fmt.Sprint(r["text"]), 60))
}
}
fmt.Printf("%s (%d rows):\n", title, len(rows))
for _, key := range order {
gr := groups[key]
fmt.Printf(" %-58s n=%d %s\n", gr.key, gr.n, strings.Join(gr.texts, " | "))
}
}
func firstN(s string, n int) string {
if len(s) <= n {
return s
}
return s[:n] + "…"
}
// ── io helpers ────────────────────────────────────────────────────────────
func mustLoad[T any](path string) T {
b, err := os.ReadFile(path)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
var v T
if err := json.Unmarshal(b, &v); err != nil {
fmt.Fprintln(os.Stderr, "json:", err)
os.Exit(1)
}
return v
}
func mustSave(path string, v any) {
b, err := json.MarshalIndent(v, "", " ")
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
if err := os.WriteFile(path, b, 0o644); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
func writeJSON(path string, v any) {
b, _ := json.MarshalIndent(v, "", " ")
_ = os.WriteFile(path, b, 0o644)
}
var _ = sort.Strings
@@ -0,0 +1,135 @@
#!/usr/bin/env python3
"""
Slice 21 emit: deterministic execution-frame guard data files for the Go harness
==================================================================================
Slice 18 showed the sparse lexical gate owns the aggregate boundary (PR-AUC
0.838, strict operating point at threshold 0.715 with P>=0.95 | R=0.264) and
slice 19/20 showed learning heads collapse on capability-question LOFO. Slice 21
tests the deterministic alternative: a rule engine over existing parsers that
decides execution eligibility as a three-way gate (permissive / blocked /
ambiguous), never itself routing.
This script only repackages the frozen dev pool for the Go harness. It reuses
slice 18's feature builders and grouped-CV and slice 19's pair builder verbatim,
so the numbers the Go side reports are the same populations the accepts
measured. It writes:
/tmp/mvn-s21/pool.json dev rows: idx, text, n_text, route, y, tags,
cv_fold, split_group, source_id, family
/tmp/mvn-s21/pairs.json capability-vs-action pairs (slice-19 builder)
/tmp/mvn-s21/sparse_oof.json slice-18 "both" grouped-CV OOF proba per row
(for the §17 guard+sparse composition)
No training happens here and no label is changed. The guard itself is Go.
"""
import json
import os
import re
import sys
import numpy as np
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, HERE)
import slice18_sparse # noqa: E402
import slice19_main # noqa: E402
OUT_DIR = "/tmp/mvn-s21"
SPARSE_THRESHOLD = 0.715 # slice-18 §4 strict-operating-point (P>=0.95 best recall)
def main():
# Population = the exact slice-20 dev pool (s19.load_dev): every dev_pool
# row, fast-path included. The guard is evaluated on what slice 20 measured.
meta, examples = slice18_sparse.load_data()
dev = slice18_sparse.filter_dev_pool(examples)
print(f"dev pool (all dev_pool rows): {len(dev)} rows")
print(f"corpus meta: {meta.get('dev_count', '?')} dev rows declared, "
f"{meta.get('route_counts', {}).get('action', '?')} action declared")
n_texts = [slice18_sparse.normalize_match_text(e["text"]) for e in dev]
rows = []
by_route = {}
by_family = {}
for i, (e, nt) in enumerate(zip(dev, n_texts)):
tags = sorted(set(e.get("tags", [])))
route = e["route"]
fam = slice19_main.family_of(set(tags))
by_route[route] = by_route.get(route, 0) + 1
by_family[fam] = by_family.get(fam, 0) + 1
rows.append({
"idx": i,
"text": e["text"],
"n_text": nt,
"route": route,
"y": 1 if route == "action" else 0,
"tags": tags,
"cv_fold": e["cv_fold"],
"split_group": e["split_group"],
"source_id": e["source_id"],
})
print("routes:", by_route)
print("families:", by_family)
# ── pairs (slice-19 builder, exact population) ─────────────────────────
ldev = [{
"text_orig": nt,
"route": r["route"],
"y": r["y"],
"cv_fold": r["cv_fold"],
"tags": set(r["tags"]),
"source_id": r["source_id"],
} for r, nt in zip(rows, n_texts)]
pairs = slice19_main.build_pairs(ldev, n_texts)
print(f"pairs: {len(pairs)}")
# ── slice-18 "both" grouped-CV OOF proba, aligned to row index ────────
y = [1 if r["route"] == "action" else 0 for r in rows]
folds = [r["cv_fold"] for r in rows]
X, _vec = slice18_sparse.build_features(n_texts, "both")
print(f"sparse 'both' X: {X.shape}")
yb = np.array(y)
folds_arr = np.array(folds)
idx_proba = {}
for te_fold in sorted(set(folds)):
tr = folds_arr != te_fold
te = folds_arr == te_fold
clf = slice18_sparse.LogisticRegression(
C=1.0, max_iter=2000, solver="lbfgs", random_state=42)
clf.fit(X[tr], yb[tr])
p = clf.predict_proba(X[te])[:, 1]
te_idx = np.where(te)[0]
for k, i in enumerate(te_idx):
idx_proba[int(i)] = float(p[k])
assert len(idx_proba) == len(rows)
sparse_oof = [{"idx": i, "proba": idx_proba[i]} for i in range(len(rows))]
pred = [1 if idx_proba[i] >= 0.5 else 0 for i in range(len(rows))]
tp = sum(1 for i in range(len(rows)) if y[i] == 1 and pred[i] == 1)
fp = sum(1 for i in range(len(rows)) if y[i] == 0 and pred[i] == 1)
fn = sum(1 for i in range(len(rows)) if y[i] == 1 and pred[i] == 0)
print(f"sparse both OOF @0.5: P={tp/max(tp+fp,1):.3f} R={tp/max(tp+fn,1):.3f} "
f"FA={fp} ({fp/len(rows):.4f})")
pred21 = [1 if idx_proba[i] >= SPARSE_THRESHOLD else 0 for i in range(len(rows))]
tp = sum(1 for i in range(len(rows)) if y[i] == 1 and pred21[i] == 1)
fp = sum(1 for i in range(len(rows)) if y[i] == 0 and pred21[i] == 1)
fn = sum(1 for i in range(len(rows)) if y[i] == 1 and pred21[i] == 0)
print(f"sparse both OOF @{SPARSE_THRESHOLD}: P={tp/max(tp+fp,1):.3f} "
f"R={tp/max(tp+fn,1):.3f} FA={fp} ({fp/len(rows):.4f})")
os.makedirs(OUT_DIR, exist_ok=True)
with open(os.path.join(OUT_DIR, "pool.json"), "w") as f:
json.dump(rows, f, ensure_ascii=False, indent=1)
with open(os.path.join(OUT_DIR, "pairs.json"), "w") as f:
json.dump([[c, a] for c, a in pairs], f)
with open(os.path.join(OUT_DIR, "sparse_oof.json"), "w") as f:
json.dump(sparse_oof, f)
print(f"wrote {OUT_DIR}/{{pool,pairs,sparse_oof}}.json")
if __name__ == "__main__":
main()
@@ -0,0 +1,65 @@
package main
import (
"fmt"
"os"
"github.com/kami/maven/internal/router"
"github.com/kami/maven/internal/router/semantic"
)
// headsMain runs the deployed cascade minus the resident LLM: stage-0
// grammars → routing heads (fine-tuned e5 copy + softmax, router_heads.onnx,
// 0.6 decline threshold) → ONNX-embedder nearest-centroid classifier →
// 0.55 confidence gate. This is what a production turn takes when the model
// server is out (docs/routing.md: pickLLMRouter degrades to the classifier).
//
// The classifier is seeded from models/seeds like the daemon's seedClassifier,
// embedded with the real multilingual-e5-small model rather than the block
// hash, so this is the closest headless reproduction of the authoritative
// router output the slice-22 report can run.
//
// Requires the ONNX model files and a libonnxruntime.so. Pass the library via
// the MAVEN_ONNX_LIB environment variable, exactly as the daemon does.
func headsMain(poolPath, outPath string) {
lib := os.Getenv("MAVEN_ONNX_LIB")
if lib == "" {
fmt.Fprintln(os.Stderr, "heads mode needs MAVEN_ONNX_LIB pointing at libonnxruntime.so")
os.Exit(2)
}
const (
embedModel = "models/embedder/multilingual-e5-small/model_quantized.onnx"
tokPath = "models/embedder/multilingual-e5-small/tokenizer.json"
headsModel = "models/embedder/router-heads/router_heads.onnx"
)
emb, err := router.NewONNXEmbedder(embedModel, tokPath, lib)
if err != nil {
fmt.Fprintf(os.Stderr, "heads: embedder: %v\n", err)
os.Exit(1)
}
defer emb.Close()
cls := router.NewClassifier(emb)
seedClassifier(cls)
heads, err := router.NewRouterHeads(headsModel, tokPath)
if err != nil {
fmt.Fprintf(os.Stderr, "heads: %v\n", err)
os.Exit(1)
}
defer heads.Close()
acts := router.DefaultActMatcher{Fns: semantic.ExperimentActVerbs()}
r := router.New(router.Config{
Grammars: router.StageZeroGrammars(acts),
Classifier: cls,
Extractor: router.Extractor{
Time: router.StubDateTimeParser{},
Acts: acts,
Facts: router.DefaultFactParser{},
},
Threshold: 0.55,
Heads: heads,
})
runOverPool(r, poolPath, outPath)
}
@@ -0,0 +1,87 @@
package main
import (
"bufio"
"context"
"log"
"os"
"path/filepath"
"sort"
"strings"
"github.com/kami/maven/internal/router"
"github.com/kami/maven/internal/router/semantic"
)
// Seed loading replicated from cmd/mavend/voicewire.go (seedClassifier) and
// internal/router/semantic/helpers_test.go, which build the same classifier
// from models/seeds/<intent>.txt. The daemon and the eval fixture must agree
// on the seeds; so must a measurement.
const seedDir = "models/seeds"
var seedIntents = []router.Intent{
router.IntentAct, router.IntentReminder, router.IntentFact,
router.IntentNote, router.IntentQuery, router.IntentChat, router.IntentSystem,
}
// buildMinimalRouter reproduces internal/router/semantic/buildMinimalRouter:
// the daemon's grammar set, a hash-embedder classifier seeded from
// models/seeds, and the deployed 0.55 threshold. Deterministic and
// reproducible. The ONNX embedder and the routing heads score elsewhere;
// this is the floor the eval fixture reports as the legacy baseline.
func buildMinimalRouter() *router.Router {
acts := router.DefaultActMatcher{Fns: semantic.ExperimentActVerbs()}
cls := router.NewClassifier(router.NewHashEmbedder(1024))
seedClassifier(cls)
return router.New(router.Config{
Grammars: router.StageZeroGrammars(acts),
Classifier: cls,
Extractor: router.Extractor{
Time: router.StubDateTimeParser{},
Acts: acts,
Facts: router.DefaultFactParser{},
},
Threshold: 0.55,
})
}
func seedClassifier(c *router.Classifier) {
// Walk up to find models/seeds like the daemon's seedPath, so the program
// can run from any depth of the repo tree.
dir := seedDir
for i := 0; i < 5; i++ {
if st, err := os.Stat(dir); err == nil && st.IsDir() {
break
}
dir = filepath.Join("..", dir)
}
ctx := context.Background()
total := 0
for _, intent := range seedIntents {
path := filepath.Join(dir, string(intent)+".txt")
f, err := os.Open(path)
if err != nil {
log.Printf("legacy: open seed %s: %v", path, err)
continue
}
sc := bufio.NewScanner(f)
lines := []string{}
for sc.Scan() {
line := strings.TrimSpace(sc.Text())
if line == "" || strings.HasPrefix(line, "#") {
continue
}
lines = append(lines, line)
}
f.Close()
sort.Strings(lines)
for _, line := range lines {
if err := c.AddExample(ctx, intent, line); err != nil {
log.Printf("legacy: seed %s %q: %v", intent, line, err)
continue
}
total++
}
}
log.Printf("legacy: loaded %d seed examples from %s", total, dir)
}
@@ -0,0 +1,192 @@
package main
import (
"bufio"
"context"
"encoding/json"
"flag"
"fmt"
"os"
"time"
"github.com/kami/maven/internal/router"
)
// Legacy-baseline runner for slice 22: run the actual router cascade (stage 0
// grammars → hash-embedder nearest-centroid classifier → 0.55 confidence gate)
// over the frozen residual non-action dev pool and project each decision into
// the five-way non-action semantic space.
//
// Projection rules (the daemon's behaviour, not just ScoreLegacy's):
// - route error → uncertain
// - Clarify=true (stage 3) → uncertain: the daemon asks, it does not commit
// to a semantic bucket
// - chat/query/fact+note/system → conversation/knowledge/memory_write/system
// - act/reminder on a trusted non-action row → class "action" recorded
// VERBATIM with illegal_action_prediction=true; never mapped to uncertain
// - anything else → uncertain
//
// Reads /tmp/mvn-s22/pool.json (emit step) and writes /tmp/mvn-s22/legacy.json
// with both the raw decision fields and the projected class, plus a summary
// printout. No embedding is recomputed and no label is changed.
type poolRow struct {
IDX int `json:"idx"`
Text string `json:"text"`
NText string `json:"n_text"`
Route string `json:"route"`
Tags []string `json:"tags"`
CVFold int `json:"cv_fold"`
SplitGroup string `json:"split_group"`
FamilyID string `json:"family_id"`
SourceID string `json:"source_id"`
}
type legacyRow struct {
IDX int `json:"idx"`
Text string `json:"text"`
Route string `json:"route"`
Intent string `json:"intent"`
Class string `json:"class"`
Illegal bool `json:"illegal_action_prediction"`
Confidence float64 `json:"confidence"`
Stage int `json:"stage"`
Clarify bool `json:"clarify"`
Producer string `json:"producer"`
Error string `json:"error,omitempty"`
SourceID string `json:"source_id"`
}
func main() {
var mode, poolPath, outPath string
flag.StringVar(&mode, "mode", "legacy", "baseline mode: legacy (hash classifier) or heads (ONNX cascade minus LLM)")
flag.StringVar(&poolPath, "pool", "/tmp/mvn-s22/pool.json", "emit-step pool.json")
flag.StringVar(&outPath, "out", "/tmp/mvn-s22/legacy.json", "output path")
flag.Parse()
switch mode {
case "legacy":
legacyMain(poolPath, outPath)
case "heads":
headsMain(poolPath, outPath)
default:
fmt.Fprintf(os.Stderr, "unknown -mode %q\n", mode)
os.Exit(2)
}
}
func legacyMain(poolPath, outPath string) {
runOverPool(buildMinimalRouter(), poolPath, outPath)
}
func runOverPool(r *router.Router, poolPath, outPath string) {
raw, err := os.ReadFile(poolPath)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
var rows []poolRow
if err := json.Unmarshal(raw, &rows); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
ctx := context.Background()
now := time.Now()
out := make([]legacyRow, 0, len(rows))
classCount := map[string]int{}
for _, pr := range rows {
d, err := r.Route(ctx, router.NormalizedInput{Text: pr.Text}, now)
lr := legacyRow{
IDX: pr.IDX,
Text: pr.Text,
Route: pr.Route,
SourceID: pr.SourceID,
}
if err != nil {
lr.Class = "uncertain"
lr.Error = err.Error()
} else {
lr.Intent = string(d.Intent)
lr.Confidence = d.Confidence
lr.Stage = d.Stage
lr.Clarify = d.Clarify
lr.Producer = string(d.Producer)
}
lr.Class, lr.Illegal = project(d, err)
classCount[lr.Class]++
out = append(out, lr)
}
if err := writeJSON(outPath, out); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
fmt.Printf("legacy baseline over %d residual non-action rows:\n", len(out))
for _, c := range []string{"conversation", "knowledge", "memory_write", "system", "uncertain", "action"} {
fmt.Printf(" %-14s %d (%.1f%%)\n", c, classCount[c], 100*float64(classCount[c])/float64(len(out)))
}
fmt.Printf(" illegal_action_prediction: %d\n", classCount["action"])
// Grammar hits inside a corpus-residual population would be a
// corpus/harness disagreement worth telling the report about: the corpus
// marked each row not-fast-path-resolved, so a current stage-0 rule
// resolving it means the corpus's fast-path mirror is stale or a grammar
// landed after the corpus froze.
gh := 0
ghByRoute := map[string]int{}
ghByIntent := map[string]int{}
for _, lr := range out {
if lr.Producer == string(router.RouteProducerGrammar) {
gh++
ghByRoute[lr.Route]++
ghByIntent[lr.Intent]++
}
}
fmt.Printf(" stage-0 grammar hits: %d\n", gh)
if gh > 0 {
fmt.Printf(" by ground-truth route: %v\n", ghByRoute)
fmt.Printf(" by grammar intent: %v\n", ghByIntent)
}
}
// project maps the router's authoritative output into the five-way non-action
// space, or to the "action" bucket verbatim when the router calls an act or a
// reminder on a non-action row.
func project(d router.Decision, err error) (string, bool) {
if err != nil {
return "uncertain", false
}
if d.Clarify {
return "uncertain", false
}
switch d.Intent {
case router.IntentChat:
return "conversation", false
case router.IntentQuery:
return "knowledge", false
case router.IntentFact, router.IntentNote:
return "memory_write", false
case router.IntentSystem:
return "system", false
case router.IntentAct, router.IntentReminder:
return "action", true
default:
return "uncertain", false
}
}
func writeJSON(path string, v any) error {
fh, err := os.Create(path)
if err != nil {
return err
}
defer fh.Close()
w := bufio.NewWriter(fh)
enc := json.NewEncoder(w)
enc.SetIndent("", " ")
if err := enc.Encode(v); err != nil {
return err
}
return w.Flush()
}
@@ -0,0 +1,114 @@
#!/usr/bin/env python3
"""
Slice 22 emit: five-way residual non-action semantic router data files
========================================================================
Slice 21 accepted the deterministic execution-frame guard (docs/evals/
2026-09-07-execution-frame-guard.md). Slice 22 returns to the coarse non-action
router that the guard hands to: after TryFastPath misses and the guard passes,
the remaining utterance is one of five non-action semantics conversation,
knowledge, memory_write, system, uncertain. Action rows never reach this
router; they are usable only as out-of-domain probes, never in primary metrics.
This script only repackages the frozen dev pool for the Go legacy baseline and
the Python experiment. It reuses slice 18's loader/filters and slice 19's
normalizers verbatim, so the population here is the same one slices 18-21
measured. It writes:
/tmp/mvn-s22/pool.json residual non-action dev rows: idx, text, n_text,
route, tags, cv_fold, split_group, family_id,
source_id (1652 rows)
/tmp/mvn-s22/ood.json residual ACTION dev rows (766): same shape; OOD
probes only, never primary metrics
/tmp/mvn-s22/stats.json population summary (routes, families, folds)
idx is the row's position among dev_pool rows in dev-pool order, so the Python
experiment can align the embedding vectors from /tmp/mvn-experiment/embeddings.json
by index exactly as slice19.load_dev does.
No training happens here and no label is changed.
Population (verified 2026-09-08 from the frozen file):
dev_pool 2490
dev residual 2418 (= dev_pool minus fast_path_resolved)
residual non-action 1652 knowledge 715 / memory_write 553 / system 184 /
uncertain 107 / conversation 93
residual action 766 (OOD probes only)
"""
import json
import os
import sys
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, HERE)
import slice18_sparse # noqa: E402 (normalize_match_text, load_data, filters)
OUT_DIR = "/tmp/mvn-s22"
ROUTES = ["conversation", "knowledge", "memory_write", "system", "uncertain"]
def main():
meta, examples = slice18_sparse.load_data()
dev = slice18_sparse.filter_dev_pool(examples)
print(f"dev pool: {len(dev)} rows "
f"(meta declares dev_count={meta.get('dev_count')})")
rows = []
for i, e in enumerate(dev):
if not e["fast_path_resolved"]:
rows.append({
"idx": i,
"text": e["text"],
"n_text": slice18_sparse.normalize_match_text(e["text"]),
"route": e["route"],
"tags": sorted(set(e.get("tags", []))),
"cv_fold": e["cv_fold"],
"split_group": e["split_group"],
"family_id": e["family_id"],
"source_id": e["source_id"],
})
na = [r for r in rows if r["route"] != "action"]
ood = [r for r in rows if r["route"] == "action"]
print(f"residual rows: {len(rows)} non-action: {len(na)} action(OOD): {len(ood)}")
by_route = {}
for r in na:
by_route[r["route"]] = by_route.get(r["route"], 0) + 1
print("routes:", by_route)
assert sum(by_route.values()) == len(na)
assert set(ROUTES) == set(by_route), "route set must be the five-way"
by_family = {}
for r in na:
by_family[r["family_id"]] = by_family.get(r["family_id"], 0) + 1
by_fold = {}
for r in na:
by_fold[r["cv_fold"]] = by_fold.get(r["cv_fold"], 0) + 1
print(f"family_ids: {len(by_family)} split_groups: {len(set(r['split_group'] for r in na))}")
print("folds:", by_fold)
os.makedirs(OUT_DIR, exist_ok=True)
with open(os.path.join(OUT_DIR, "pool.json"), "w") as f:
json.dump(na, f, ensure_ascii=False, indent=1)
with open(os.path.join(OUT_DIR, "ood.json"), "w") as f:
json.dump(ood, f, ensure_ascii=False, indent=1)
with open(os.path.join(OUT_DIR, "stats.json"), "w") as f:
json.dump({
"dev_count": len(dev),
"residual_count": len(rows),
"non_action_count": len(na),
"action_ood_count": len(ood),
"routes": by_route,
"family_ids": len(by_family),
"folds": by_fold,
"top_family": dict(sorted(by_family.items(), key=lambda kv: -kv[1])[:15]),
}, f, ensure_ascii=False, indent=1)
print(f"wrote {OUT_DIR}/{{pool,ood,stats}}.json")
if __name__ == "__main__":
main()
@@ -0,0 +1,560 @@
#!/usr/bin/env python3
"""
Slice 22: five-way residual non-action semantic router (experiment)
==================================================================
After TryFastPath misses and the ExecutionFrameGuard passes, the residual
utterance is one of five non-action semantics: conversation, knowledge,
memory_write, system, uncertain. This measures whether the deployed e5-small
embeddings (384-d, query-prefixed, mean-pooled, L2, frozen) fed to a linear
softmax head suffice, and how they compare to the legacy router, to floors,
and to the deployed routing heads.
Population: the frozen dev-pool residual non-action rows (1652; the pool
written by slice22_emit.py). Action rows (766) are out-of-domain probes only.
Metrics written to /tmp/mvn-s22/results.json:
§1 population
§2 legacy baseline (legacy.json / legacy_heads.json): acc, macro-F1,
per-class P/R/F1, confusion, illegal_action_prediction count
§3 e5-linear primary head: C grid, grouped CV OOF, per-fold P/R/F1 +
variance + composition
§4 floors: majority, centroid (cosine nearest-mean), sparse word+char
TF-IDF logistic (slice18 builder), all grouped CV
§5 route-family (family_id) leave-family-out
§6 knowledge vs memory_write: matched pairs (water/homelab/task) ordering
§7 uncertain as an explicit class: P/R/F1 + top confusions
§8 OOF confidence: max-softmax correct/wrong, ECE, log-loss, Brier,
coverage/accuracy/macro-F1 abstention curves (no threshold chosen)
§9 action OOD probes: fold models applied to the 766 action rows
§10 artifact cost: head params, serialized bytes, incremental head latency
No corpus label is changed. No frozen-holdout rows are inspected.
"""
import json
import os
import sys
import time
import numpy as np
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, HERE)
import slice18_sparse # noqa: E402
import slice19_main # noqa: E402
EMB_PATH = "/tmp/mvn-experiment/embeddings.json"
OUT_DIR = "/tmp/mvn-s22"
CLASSES = ["conversation", "knowledge", "memory_write", "system", "uncertain"]
CLASS_PREFIX = ["conversation", "knowledge", "memory_write", "system", "uncertain"]
C_GRID = [0.1, 1.0, 10.0]
# Route-family holdouts the report calls out by name (slice-22 brief): every
# family that is not part of the shared subject inventory on either side.
HOLDOUT_GROUPS = {
"capability": ["knowledge:capability-ha", "knowledge:capability-tool"],
"world": ["knowledge:world-def", "knowledge:world-explain"],
"calendar": ["knowledge:calendar", "knowledge:calendar-time", "knowledge:calendar-next"],
"recall": ["knowledge:recall-fact", "knowledge:recall-note", "knowledge:recall-possessive"],
"fact": ["fact:meal", "fact:water", "fact:sleep", "fact:shower", "fact:break", "fact:pills", "fact:exercise"],
"note": ["note:idea", "note:homelab", "note:task"],
"remember": ["free:remember"],
"system": None, # all system:*
"conversation": None,
"uncertain": None,
}
def load_pool_and_embeds():
with open(os.path.join(OUT_DIR, "pool.json")) as f:
pool = json.load(f)
meta, examples = slice18_sparse.load_data()
dev = slice18_sparse.filter_dev_pool(examples)
by_idx = {e["dev_idx"]: e for e in dev} if "dev_idx" in dev[0] else None
# pool rows carry idx = position among dev_pool rows in dev order
emb_by_idx = {i: np.asarray(e["embedding"], dtype=np.float64)
for i, e in enumerate(dev)}
for r in pool:
r["emb"] = emb_by_idx[r["idx"]]
r["y"] = r["route"]
return pool, meta
def oof_proba_grouped(X, y, folds, C=1.0):
"""Grouped OOF probability matrix (n×5, class order CLASSES)."""
y_idx = np.array([CLASSES.index(c) for c in y])
folds = np.asarray(folds)
proba = np.zeros((len(y_idx), len(CLASSES)))
for te_fold in sorted(set(folds.tolist())):
tr = folds != te_fold
te = folds == te_fold
clf = slice18_sparse.LogisticRegression(
C=C, max_iter=2000, solver="lbfgs", random_state=42)
clf.fit(X[tr], y_idx[tr])
proba[te] = clf.predict_proba(X[te])
return proba
def cls_metrics(yt, yp):
import sklearn.metrics as m
yt = np.asarray(yt)
yp = np.asarray(yp)
if yt.dtype != np.int64 and yt.dtype != np.int32:
yt = np.array([CLASSES.index(c) for c in yt])
if yp.dtype != np.int64 and yp.dtype != np.int32:
yp = np.array([CLASSES.index(c) for c in yp])
labels = list(range(len(CLASSES)))
n = len(yt)
acc = m.accuracy_score(yt, yp)
macro = m.f1_score(yt, yp, average="macro", labels=labels, zero_division=0)
pr, rc, f1, sup = m.precision_recall_fscore_support(
yt, yp, labels=labels, zero_division=0)
per = {c: {"p": float(pr[i]), "r": float(rc[i]), "f1": float(f1[i]), "n": int(sup[i])}
for i, c in enumerate(CLASSES)}
conf = m.confusion_matrix(yt, yp, labels=labels).tolist()
return {"n": n, "acc": acc, "macro_f1": macro, "per_class": per, "confusion": conf}
def fold_report(yt, proba, folds, true_y):
out = {}
folds_arr = np.asarray(folds)
comp = {}
for f in sorted(set(folds_arr.tolist())):
mask = folds_arr == f
yt_f = [CLASSES.index(y) for y in true_y[mask]]
comp[f] = {c: int((np.array(true_y[mask]) == c).sum()) for c in CLASSES}
per_fold = {}
for f in sorted(set(folds_arr.tolist())):
mask = folds_arr == f
yp = proba[mask].argmax(1).tolist()
m = cls_metrics([yt[i] for i in np.where(mask)[0].tolist()], yp)
per_fold[f] = {"acc": m["acc"], "macro_f1": m["macro_f1"]}
out["composition"] = comp
out["per_fold"] = per_fold
accs = [v["acc"] for v in per_fold.values()]
macros = [v["macro_f1"] for v in per_fold.values()]
out["acc_mean"] = float(np.mean(accs))
out["acc_std"] = float(np.std(accs))
out["macro_f1_mean"] = float(np.mean(macros))
out["macro_f1_std"] = float(np.std(macros))
return out
def ece(yt, proba, n_bins=15):
conf = proba.max(1)
pred = proba.argmax(1)
acc = (pred == yt).astype(float)
bins = np.linspace(0, 1, n_bins + 1)
tot = 0.0
details = []
counts = 0
for i in range(n_bins):
lo, hi = bins[i], bins[i + 1]
m = (conf >= lo) & (conf < hi) if i < n_bins - 1 else conf >= lo
if m.sum() == 0:
continue
acc_m = acc[m].mean()
conf_m = conf[m].mean()
w = m.sum() / len(conf)
tot += w * abs(acc_m - conf_m)
counts += int(m.sum())
details.append({"bin": i, "lo": lo, "hi": hi, "conf": float(conf_m),
"acc": float(acc_m), "n": int(m.sum())})
return {"ece": float(tot), "n_bins": n_bins, "counted": counts, "bins": details}
def main():
pool, meta = load_pool_and_embeds()
pool.sort(key=lambda r: r["idx"])
print(f"pool: {len(pool)} rows")
from sklearn.metrics import brier_score_loss, log_loss
report = {"population": {}, "legacy": {}, "e5_linear": {}, "floors": {},
"family_holdouts": {}, "kmw": {}, "uncertain": {}, "confidence": {},
"ood": {}, "artifact": {}}
# ── §1 population ──────────────────────────────────────────────────────
cnt = {}
for r in pool:
cnt[r["y"]] = cnt.get(r["y"], 0) + 1
report["population"] = {
"n": len(pool),
"routes": cnt,
"family_ids": len(set(r["family_id"] for r in pool)),
"split_groups": len(set(r["split_group"] for r in pool)),
"folds": {str(f): int(sum(1 for r in pool if r["cv_fold"] == f)) for f in sorted(set(r["cv_fold"] for r in pool))},
"corpus": {k: v for k, v in meta.items() if k in
("dev_count", "residual_count", "fast_path_count",
"dimension", "embedder_id", "input_template", "pooling", "normalization")},
}
print("\n§1 population:", report["population"])
X = np.vstack([r["emb"] for r in pool])
y = np.array([r["y"] for r in pool])
folds = np.array([r["cv_fold"] for r in pool])
yt = np.array([CLASSES.index(c) for c in y])
# ── §2 legacy baselines ────────────────────────────────────────────────
import collections
for tag, fname in [("hash", "legacy.json"), ("heads", "legacy_heads.json")]:
path = os.path.join(OUT_DIR, fname)
if not os.path.exists(path):
continue
leg = json.load(open(path))
leg_by_idx = {r["idx"]: r for r in leg}
yp_leg = []
illegal = []
for r in pool:
lr = leg_by_idx[r["idx"]]
if lr["illegal_action_prediction"]:
illegal.append(lr)
yp_leg.append("action")
else:
yp_leg.append(lr["class"])
yp_leg = np.array(yp_leg)
# five-way: an 'action' prediction is an error (outside the label set)
yp5 = np.array([("uncertain" if p == "action" else p) for p in yp_leg])
m = cls_metrics(y, yp5)
m["illegal_action_prediction"] = len(illegal)
m["illegal_cases"] = [{"idx": i["idx"], "text": i["text"], "route": i["route"],
"intent": i["intent"], "producer": i["producer"],
"confidence": i["confidence"]} for i in illegal]
# per-cell confusion also shows 'action' column
conf_counts = collections.Counter(zip(y, yp_leg))
m["confusion_with_action"] = {f"{a}->{b}": int(c) for (a, b), c in conf_counts.items()}
report["legacy"][tag] = m
print(f"\n§2 legacy ({tag}) acc={m['acc']:.4f} macroF1={m['macro_f1']:.4f} "
f"illegal={len(illegal)}")
for c in CLASSES:
p = m["per_class"][c]
print(f" {c:<14} P={p['p']:.3f} R={p['r']:.3f} F1={p['f1']:.3f} n={p['n']}")
# grammar-pure residual: rows not resolved by any current stage-0 grammar
if os.path.exists(os.path.join(OUT_DIR, "legacy.json")):
leg = json.load(open(os.path.join(OUT_DIR, "legacy.json")))
gh = {r["idx"] for r in leg if r["producer"] == "grammar"}
gp_mask = np.array([r["idx"] not in gh for r in pool])
report["grammar_drift"] = {
"grammar_hits_in_pool": len(gh),
"grammar_pure_n": int(gp_mask.sum()),
}
# ── §3 e5-linear primary head ─────────────────────────────────────────
print("\n§3 e5-linear")
bestC, bestMac = 1.0, -1.0
grid = {}
for C in C_GRID:
p = oof_proba_grouped(X, y, folds, C=C)
mp = cls_metrics(y, p.argmax(1).tolist())
grid[float(C)] = {"acc": mp["acc"], "macro_f1": mp["macro_f1"]}
print(f" C={C} acc={mp['acc']:.4f} macroF1={mp['macro_f1']:.4f}")
if mp["macro_f1"] > bestMac:
bestMac, bestC = mp["macro_f1"], C
print(f" -> best C={bestC}")
p_best = oof_proba_grouped(X, y, folds, C=bestC)
m_best = cls_metrics(y, p_best.argmax(1).tolist())
m_best["C"] = bestC
m_best["C_grid"] = grid
m_best["folds"] = fold_report(yt, p_best, folds, y)
report["e5_linear"] = m_best
for f, v in m_best["folds"]["per_fold"].items():
print(f" fold {f}: acc={v['acc']:.4f} macroF1={v['macro_f1']:.4f}")
print(f" fold acc mean={m_best['folds']['acc_mean']:.4f} "
f"std={m_best['folds']['acc_std']:.4f}; "
f"macroF1 mean={m_best['folds']['macro_f1_mean']:.4f} "
f"std={m_best['folds']['macro_f1_std']:.4f}")
for c in CLASSES:
p_ = m_best["per_class"][c]
print(f" {c:<14} P={p_['p']:.3f} R={p_['r']:.3f} F1={p_['f1']:.3f} n={p_['n']}")
# grammar-pure sensitivity for the primary head
if "grammar_drift" in report:
mp_gp = cls_metrics(y[gp_mask], p_best[gp_mask].argmax(1).tolist())
report["e5_linear"]["grammar_pure"] = {
"acc": mp_gp["acc"], "macro_f1": mp_gp["macro_f1"], "n": int(gp_mask.sum())}
# ── §4 floors ─────────────────────────────────────────────────────────
print("\n§4 floors")
# majority floor
maj = CLASSES.index("knowledge")
ym = np.full(len(y), maj)
mm = cls_metrics(y, ym)
report["floors"]["majority"] = {"acc": mm["acc"], "macro_f1": mm["macro_f1"],
"per_class": mm["per_class"]}
print(f" majority (predict {CLASSES[maj]}): acc={mm['acc']:.4f} macroF1={mm['macro_f1']:.4f}")
# centroid floor: cosine to per-class mean of the training folds' embeddings
cf_proba = np.zeros((len(yt), len(CLASSES)))
folds_arr = np.asarray(folds)
for te_fold in sorted(set(folds_arr.tolist())):
tr = folds_arr != te_fold
te = folds_arr == te_fold
centroids = []
for c in CLASSES:
idxs = np.where(tr & (y == c))[0]
ctr = X[idxs].mean(axis=0)
ctr = ctr / np.linalg.norm(ctr)
centroids.append(ctr)
Cm = np.vstack(centroids)
sims = X[te] @ Cm.T
cf_proba[te] = sims
yc = cf_proba.argmax(1)
# accuracy + macroF1 with the same 5-way
mc = cls_metrics(y, yc.tolist())
report["floors"]["centroid"] = {"acc": mc["acc"], "macro_f1": mc["macro_f1"],
"per_class": mc["per_class"]}
print(f" centroid cosine: acc={mc['acc']:.4f} macroF1={mc['macro_f1']:.4f}")
# sparse word+char logistic (slice18 builder, grouped CV, five-way)
texts = [r["n_text"] for r in pool]
Xs, _vec = slice18_sparse.build_features(texts, "both")
psp = np.zeros((len(yt), len(CLASSES)))
for te_fold in sorted(set(folds_arr.tolist())):
tr = folds_arr != te_fold
te = folds_arr == te_fold
clf = slice18_sparse.LogisticRegression(
C=1.0, max_iter=2000, solver="lbfgs", random_state=42)
clf.fit(Xs[tr], yt[tr])
psp[te] = clf.predict_proba(Xs[te])
msp = cls_metrics(y, psp.argmax(1).tolist())
report["floors"]["sparse_word_char"] = {
"acc": msp["acc"], "macro_f1": msp["macro_f1"], "per_class": msp["per_class"],
"vocab": slice18_sparse.vocab_size(_vec)}
print(f" sparse both: acc={msp['acc']:.4f} macroF1={msp['macro_f1']:.4f} "
f"vocab={report['floors']['sparse_word_char']['vocab']}")
# ── §5 route-family holdouts ─────────────────────────────────────────
print("\n§5 route-family holdouts")
fam = np.array([r["family_id"] for r in pool])
holdouts = {}
all_fams = sorted(set(fam.tolist()))
for grp, fams in HOLDOUT_GROUPS.items():
if fams is None:
fams = [f for f in all_fams if f.startswith(grp + ":")]
mask = np.isin(fam, fams)
if mask.sum() == 0:
continue
tr = ~mask
clf = slice18_sparse.LogisticRegression(
C=bestC, max_iter=2000, solver="lbfgs", random_state=42)
clf.fit(X[tr], yt[tr])
ypgrp = clf.predict(X[mask])
m = cls_metrics([CLASSES.index(c) for c in y[mask]], ypgrp.tolist())
m["families"] = fams
m["rows"] = int(mask.sum())
holdouts[grp] = {"acc": m["acc"], "macro_f1": m["macro_f1"], "n": int(mask.sum()),
"per_class": m["per_class"]}
print(f" {grp:<14} n={m['rows']} acc={m['acc']:.4f} macroF1={m['macro_f1']:.4f}")
# full leave-one-family-out summary
lofo_accs = []
lofo_f1s = []
for f in all_fams:
mask = fam == f
tr = ~mask
clf = slice18_sparse.LogisticRegression(
C=bestC, max_iter=2000, solver="lbfgs", random_state=42)
clf.fit(X[tr], yt[tr])
ypf = clf.predict(X[mask])
m = cls_metrics([CLASSES.index(c) for c in y[mask]], ypf.tolist())
lofo_accs.append(m["acc"])
lofo_f1s.append(m["macro_f1"])
holdouts["_all_49_lo_"] = {"n_families": len(all_fams),
"acc_mean": float(np.mean(lofo_accs)),
"macro_f1_mean": float(np.mean(lofo_f1s))}
report["family_holdouts"] = holdouts
print(f" leave-one-family-out over {len(all_fams)} families: "
f"acc mean={np.mean(lofo_accs):.4f} macroF1 mean={np.mean(lofo_f1s):.4f}")
# ── §6 knowledge vs memory_write ─────────────────────────────────────
print("\n§6 knowledge vs memory_write")
# reuse e5-linear OOF: does the model put the higher probability on the
# right side (memory_write for a write, knowledge for a recall)?
conf_km = np.zeros((2, 2))
pk = p_best[:, CLASSES.index("knowledge")]
pmw = p_best[:, CLASSES.index("memory_write")]
for i in range(len(yt)):
t = y[i]
if t == "knowledge":
conf_km[0, 1 if pmw[i] > pk[i] else 0] += 1
elif t == "memory_write":
conf_km[1, 1 if pmw[i] >= pk[i] else 0] += 1
report["kmw"] = {"confusion_p_ordered": conf_km.tolist()}
# matched pairs with shared subject lexemes, corpus-justified
def build_pairs(subject, fam_k, fam_mw):
kr = [r for r in pool if r["family_id"] in fam_k]
mr = [r for r in pool if r["family_id"] in fam_mw]
pairs = []
for mw in mr:
for k in kr:
if subject in mw["n_text"] and subject in k["n_text"]:
pairs.append((mw["idx"], k["idx"], mw["n_text"], k["n_text"]))
return pairs
sets = {
"water": build_pairs("вод", ["knowledge:recall-fact"], ["fact:water"]),
"homelab": build_pairs("dns", ["knowledge:homelab-status"], ["note:homelab"])
+ build_pairs("сервер", ["knowledge:homelab-status"], ["note:homelab"])
+ build_pairs("vlan", ["knowledge:homelab-status"], ["note:homelab"]),
"task": build_pairs("задач", ["knowledge:task-check", "knowledge:deadline"],
["note:task"]),
}
idx_of = {r["idx"]: i for i, r in enumerate(pool)}
pair_rep = {}
for name, pairs in sets.items():
if not pairs:
continue
ok = 0
margins = []
bad = []
for mi, ki, mx, kx in pairs:
mi_i, ki_i = idx_of[mi], idx_of[ki]
# MW row should get a higher memory_write probability than the K row
mk = (pmw[mi_i] + 0.0)
if pmw[mi_i] > pmw[ki_i]:
ok += 1
else:
bad.append((mx[:46], round(float(pmw[mi_i]), 3), kx[:46], round(float(pmw[ki_i]), 3)))
margins.append(pmw[mi_i] - pmw[ki_i])
pair_rep[name] = {
"pairs": len(pairs),
"mw_over_k_order_acc": ok / len(pairs),
"mean_margin": float(np.mean(margins)),
"reversed_examples": bad[:6],
}
print(f" {name}: pairs={len(pairs)} order_acc={ok/len(pairs):.3f} "
f"mean_margin={np.mean(margins):+.3f}")
report["kmw"]["matched_pairs"] = pair_rep
# ── §7 uncertain as explicit class ────────────────────────────────────
print("\n§7 uncertain")
up = m_best["per_class"]["uncertain"]
uc = m_best["confusion"][CLASSES.index("uncertain")]
report["uncertain"] = {
"per_class": up,
"row_from_uncertain": {CLASSES[j]: int(uc[j]) for j in range(5)},
"row_to_uncertain": {CLASSES[j]: int(m_best["confusion"][j][CLASSES.index("uncertain")])
for j in range(5)},
}
print(f" uncertain n={up['n']} P={up['p']:.3f} R={up['r']:.3f} F1={up['f1']:.3f}")
print(" wrong-→label pulled from uncertain:", report["uncertain"]["row_from_uncertain"])
print(" →uncertain pulled from:", report["uncertain"]["row_to_uncertain"])
# ── §8 OOF confidence / calibration / abstention ─────────────────────
print("\n§8 confidence / calibration")
conf = p_best.max(1)
right = (p_best.argmax(1) == yt)
cer = {
"correct_conf_mean": float(conf[right].mean()),
"correct_conf_median": float(np.median(conf[right])),
"wrong_conf_mean": float(conf[~right].mean()),
"wrong_conf_median": float(np.median(conf[~right])),
"ece": ece(yt, p_best)["ece"],
"ece_bins": ece(yt, p_best)["bins"],
"log_loss": float(log_loss(yt, p_best, labels=[0, 1, 2, 3, 4])),
}
# Brier is label-set specific: one-vs-rest mean
briers = []
for i in range(5):
briers.append(brier_score_loss((yt == i).astype(int), p_best[:, i]))
cer["brier_macro"] = float(np.mean(briers))
report["confidence"] = cer
print(f" right conf mean={cer['correct_conf_mean']:.3f} "
f"wrong conf mean={cer['wrong_conf_mean']:.3f} ECE={cer['ece']:.4f}")
print(f" log_loss={cer['log_loss']:.4f} brier_macro={cer['brier_macro']:.4f}")
thr_grid = np.linspace(0.10, 0.98, 45)
abst = []
for t in thr_grid:
cov = (conf >= t).mean()
if cov == 0:
continue
keep = conf >= t
yt_k = yt[keep]
yp_k = p_best[keep].argmax(1)
mk_ = cls_metrics(yt_k.tolist(), yp_k.tolist())
abst.append({"threshold": round(float(t), 3), "coverage": float(cov),
"accuracy": mk_["acc"], "macro_f1": mk_["macro_f1"]})
report["confidence"]["abstention_curve"] = abst
print(" threshold | coverage | accuracy | macroF1 (first 6/45 + knee)")
for row in abst[::9]:
print(f" {row['threshold']:.2f} | {row['coverage']:.3f} | "
f"{row['accuracy']:.3f} | {row['macro_f1']:.3f}")
# ── §9 action OOD probes ──────────────────────────────────────────────
print("\n§9 action OOD")
ood_rows = [r for r in json.load(open(os.path.join(OUT_DIR, "ood.json")))]
emb_by_idx = {i: np.asarray(e["embedding"], dtype=np.float64)
for i, e in enumerate(slice18_sparse.filter_dev_pool(
slice18_sparse.load_data()[1]))}
Xo = np.vstack([emb_by_idx[r["idx"]] for r in ood_rows])
fold_models = []
for te_fold in sorted(set(folds_arr.tolist())):
tr = folds_arr != te_fold
clf = slice18_sparse.LogisticRegression(
C=bestC, max_iter=2000, solver="lbfgs", random_state=42)
clf.fit(X[tr], yt[tr])
fold_models.append(clf)
# OOD rows are not in folds; use the full-train model to keep it simple and
# comparable to the non-action in-fold behaviour
po = np.zeros((len(Xo), 5))
for clf in fold_models:
po += clf.predict_proba(Xo)
po /= len(fold_models)
ood_top = int(np.argmax(po.mean(0)))
ood_conf = po.max(1)
ood_pred = po.argmax(1)
top_dist = {CLASSES[i]: int((ood_pred == i).sum()) for i in range(5)}
confident_na = int((ood_conf > 0.9).sum())
report["ood"] = {
"n": len(ood_rows),
"top_class": CLASSES[int(ood_top)],
"top_class_dist": top_dist,
"conf_gt_0.9": confident_na,
"conf_gt_0.9_frac": float(confident_na / len(ood_rows)),
"conf_mean": float(ood_conf.mean()),
"conf_median": float(np.median(ood_conf)),
}
print(f" action OOD n={len(ood_rows)}: most-confident class={report['ood']['top_class']} "
f"dist={top_dist}")
print(f" conf>0.9: {confident_na} ({confident_na/len(ood_rows):.3f}) "
f"conf mean={report['ood']['conf_mean']:.3f}")
# ── §10 artifact cost ────────────────────────────────────────────────
print("\n§10 artifact")
n_params = len(CLASSES) * X.shape[1] + len(CLASSES)
fp32 = n_params * 4
report["artifact"] = {
"e5_dim": X.shape[1],
"head_params": n_params,
"head_fp32_bytes": fp32,
"head_fp32_kib": fp32 / 1024,
"head_int8_bytes": n_params,
}
# incremental latency of the linear head over a batch of 1 (µs)
clf = slice18_sparse.LogisticRegression(C=bestC, max_iter=2000, solver="lbfgs", random_state=42)
clf.fit(X, yt)
x1 = X[:1]
for _ in range(50):
clf.predict_proba(x1)
lat = []
for _ in range(2000):
t0 = time.perf_counter_ns()
clf.predict_proba(x1)
lat.append((time.perf_counter_ns() - t0) / 1e3)
lat = np.array(lat)
report["artifact"]["head_latency_us_mean"] = float(lat.mean())
report["artifact"]["head_latency_us_p50"] = float(np.median(lat))
print(f" head params={n_params} fp32={fp32/1024:.2f}KiB "
f"lat mean={lat.mean():.2f}us p50={np.median(lat):.2f}us")
with open(os.path.join(OUT_DIR, "results.json"), "w") as f:
json.dump(report, f, ensure_ascii=False, indent=1, default=float)
print(f"\nwrote {OUT_DIR}/results.json")
if __name__ == "__main__":
main()
@@ -0,0 +1,243 @@
// slice23 — fast-path metadata reconciliation diagnostic.
//
// Classifies every disagreement between the corpus's stored fast_path_resolved
// flag and what the production fast path derives today (TryFastPath over the
// stage-0 grammars with the experiment's act allowlist). Outputs a JSON
// decomposition and a console summary for docs/evals reports.
//
// Usage:
//
// go run ./cmd/semantic-router-experiment/slice23/ -out /tmp/mvn-s23/drift.json
package main
import (
"encoding/json"
"flag"
"fmt"
"os"
"sort"
"github.com/kami/maven/internal/router"
"github.com/kami/maven/internal/router/semantic"
)
type conflict struct {
Text string `json:"text"`
SourceID string `json:"source_id"`
Route string `json:"route"`
Source string `json:"source"`
Group string `json:"split_group"`
Tags []string `json:"tags,omitempty"`
Dev bool `json:"dev"`
// Direction: claimed_fast_now_miss = stored fast, runtime residual;
// mirror_missed = stored residual, runtime fast.
Direction string `json:"direction"`
Grammar string `json:"grammar,omitempty"`
// ShapeDeclined is true when at least one stage-0 grammar matched the
// utterance's shape but refused the content (falls through like the router).
ShapeDeclined bool `json:"shape_declined"`
// DeclinedGrammars names every stage-0 grammar that matched the shape but
// declined the content, for claimed_fast_now_miss rows.
DeclinedGrammars []string `json:"declined_grammars,omitempty"`
}
type report struct {
Meta metaSummary `json:"meta"`
Pop popSummary `json:"population"`
Conflicts []conflict `json:"conflicts"`
ByRoute map[string]map[string]int `json:"by_route"`
ByGrammar map[string]int `json:"by_grammar"`
BySource map[string]map[string]int `json:"by_source"`
ByFamily map[string]map[string]int `json:"by_family"`
Direction map[string]int `json:"by_direction"`
Declined int `json:"claimed_fast_with_declined_shape"`
NoShape int `json:"claimed_fast_with_no_shape"`
}
type metaSummary struct {
Total int `json:"total"`
DevCount int `json:"dev_count"`
Frozen int `json:"frozen_count"`
}
type popSummary struct {
StoredFast int `json:"stored_fast"`
StoredResid int `json:"stored_residual"`
DerivedFast int `json:"derived_fast"`
DerivedResid int `json:"derived_residual"`
// Dev-pool residual route counts derived as the router sees them today.
DevResidualByRoute map[string]int `json:"dev_residual_by_route"`
// Dev-pool residual non-action + action OOD as derived.
DevResidualNonAction int `json:"dev_residual_non_action"`
DevResidualAction int `json:"dev_residual_action"`
// Fast rows in the dev pool, derived.
DevFast int `json:"dev_fast"`
}
func main() {
outPath := flag.String("out", "/tmp/mvn-s23/drift.json", "output JSON path")
flag.Parse()
exs, err := semantic.LoadCorpus()
if err != nil {
fmt.Fprintf(os.Stderr, "load corpus: %v\n", err)
os.Exit(1)
}
_, dev, _ := semantic.FrozenHoldoutSplit(exs)
devSet := make(map[string]bool, len(dev))
for _, e := range dev {
devSet[e.SourceID] = true
}
// Stage-zero grammar list for shape/declined attribution (same list the
// derivation walks).
acts := router.DefaultActMatcher{Fns: semantic.ExperimentActVerbs()}
gs := router.StageZeroGrammars(acts)
var (
conflicts []conflict
byRoute = map[string]map[string]int{}
byGrammar = map[string]int{}
bySource = map[string]map[string]int{}
byFamily = map[string]map[string]int{}
byDirection = map[string]int{}
storedFast, derivedFast, declined, noShape int
devResidByRoute = map[string]int{}
devResidNonAct, devResidAct, devFast int
)
for _, e := range exs {
o := semantic.DeriveFastPath(e.Text)
inDev := devSet[e.SourceID]
st := e.FastPathResolved
if st {
storedFast++
}
if o.Matched {
derivedFast++
}
var c *conflict
switch {
case st && o.Matched:
case st && !o.Matched:
// Stored fast but the runtime misses. Attribute why.
shapeDeclined := false
var declinedNames []string
for _, g := range gs {
_, matched, ok := g.Evaluate(e.Text)
if matched && !ok {
shapeDeclined = true
declinedNames = append(declinedNames, g.Name)
}
}
if shapeDeclined {
declined++
} else {
noShape++
}
c = &conflict{Direction: "claimed_fast_now_miss", ShapeDeclined: shapeDeclined, DeclinedGrammars: declinedNames}
case !st && o.Matched:
c = &conflict{Direction: "mirror_missed", Grammar: o.Grammar}
}
if c != nil {
c.Text = e.Text
c.SourceID = e.SourceID
c.Route = string(e.Route)
c.Source = e.Source
c.Group = e.SplitGroup
c.Tags = e.Tags
c.Dev = inDev
conflicts = append(conflicts, *c)
byDirection[c.Direction]++
byGrammar[c.Grammar]++
if byRoute[c.Direction] == nil {
byRoute[c.Direction] = map[string]int{}
}
byRoute[c.Direction][c.Route]++
if bySource[c.Direction] == nil {
bySource[c.Direction] = map[string]int{}
}
bySource[c.Direction][c.Source]++
if byFamily[c.Direction] == nil {
byFamily[c.Direction] = map[string]int{}
}
byFamily[c.Direction][c.Group]++
}
if inDev {
if o.Matched {
devFast++
} else {
devResidByRoute[string(e.Route)]++
if e.Route == semantic.RouteAction {
devResidAct++
} else {
devResidNonAct++
}
}
}
}
sort.Slice(conflicts, func(i, j int) bool { return conflicts[i].SourceID < conflicts[j].SourceID })
rep := report{
Meta: metaSummary{Total: len(exs), DevCount: len(dev), Frozen: len(exs) - len(dev)},
Pop: popSummary{
StoredFast: storedFast, StoredResid: len(exs) - storedFast,
DerivedFast: derivedFast, DerivedResid: len(exs) - derivedFast,
DevResidualByRoute: devResidByRoute,
DevResidualNonAction: devResidNonAct, DevResidualAction: devResidAct,
DevFast: devFast,
},
Conflicts: conflicts,
ByRoute: byRoute, ByGrammar: byGrammar, BySource: bySource, ByFamily: byFamily,
Direction: byDirection, Declined: declined, NoShape: noShape,
}
data, err := json.MarshalIndent(rep, "", " ")
if err != nil {
fmt.Fprintf(os.Stderr, "marshal: %v\n", err)
os.Exit(1)
}
if err := os.WriteFile(*outPath, data, 0644); err != nil {
fmt.Fprintf(os.Stderr, "write %s: %v\n", *outPath, err)
os.Exit(1)
}
fmt.Printf("total %d (dev %d, frozen %d)\n", rep.Meta.Total, rep.Meta.DevCount, rep.Meta.Frozen)
fmt.Printf("stored fast=%d residual=%d\n", rep.Pop.StoredFast, rep.Pop.StoredResid)
fmt.Printf("derived fast=%d residual=%d\n", rep.Pop.DerivedFast, rep.Pop.DerivedResid)
fmt.Printf("disagreements total %d\n", len(conflicts))
for _, d := range []string{"claimed_fast_now_miss", "mirror_missed"} {
fmt.Printf(" %-22s %d\n", d, byDirection[d])
if d == "claimed_fast_now_miss" {
fmt.Printf(" with declined shape: %d no shape: %d\n", declined, noShape)
}
}
fmt.Println(" mirror-missed by grammar:")
for _, k := range sortedKeys(byGrammar) {
fmt.Printf(" %-28s %d\n", k, byGrammar[k])
}
fmt.Println(" by route:")
for _, d := range sortedKeys(byRoute) {
fmt.Printf(" %-22s %v\n", d, byRoute[d])
}
fmt.Printf("dev pool derived: fast=%d residual=%d (non-action=%d action=%d)\n",
rep.Pop.DevFast, rep.Pop.DevResidualNonAction+rep.Pop.DevResidualAction,
rep.Pop.DevResidualNonAction, rep.Pop.DevResidualAction)
fmt.Printf("dev residual by route: %v\n", rep.Pop.DevResidualByRoute)
fmt.Printf("wrote %s\n", *outPath)
}
func sortedKeys[T any](m map[string]T) []string {
ks := make([]string, 0, len(m))
for k := range m {
ks = append(ks, k)
}
sort.Strings(ks)
return ks
}
@@ -0,0 +1,98 @@
#!/usr/bin/env python3
"""
Slice 23 emit: five-way residual non-action semantic router data files
========================================================================
Slice 23 reconciles corpus fast-path metadata with the production router
(TryFastPath over stage-0 grammars). The corpus builder no longer mirrors the
grammars by hand; fast_path_resolved is derived from the router, so this emit
flags exactly the rows the router genuinely leaves for the general cascade.
This script only repackages the frozen dev pool for the Go legacy baseline and
the Python experiment, writing into /tmp/mvn-s23 so the slice-22 artifacts
stay untouched. Logic is slice22_emit.py verbatim; only OUT_DIR differs.
/tmp/mvn-s23/pool.json residual non-action dev rows: idx, text, n_text,
route, tags, cv_fold, split_group, family_id,
source_id (1509 rows)
/tmp/mvn-s23/ood.json residual ACTION dev rows (720): same shape; OOD
probes only, never primary metrics
/tmp/mvn-s23/stats.json population summary (routes, families, folds)
"""
import json
import os
import sys
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, HERE)
import slice18_sparse # noqa: E402 (normalize_match_text, load_data, filters)
OUT_DIR = "/tmp/mvn-s23"
ROUTES = ["conversation", "knowledge", "memory_write", "system", "uncertain"]
def main():
meta, examples = slice18_sparse.load_data()
dev = slice18_sparse.filter_dev_pool(examples)
print(f"dev pool: {len(dev)} rows "
f"(meta declares dev_count={meta.get('dev_count')})")
rows = []
for i, e in enumerate(dev):
if not e["fast_path_resolved"]:
rows.append({
"idx": i,
"text": e["text"],
"n_text": slice18_sparse.normalize_match_text(e["text"]),
"route": e["route"],
"tags": sorted(set(e.get("tags", []))),
"cv_fold": e["cv_fold"],
"split_group": e["split_group"],
"family_id": e["family_id"],
"source_id": e["source_id"],
})
na = [r for r in rows if r["route"] != "action"]
ood = [r for r in rows if r["route"] == "action"]
print(f"residual rows: {len(rows)} non-action: {len(na)} action(OOD): {len(ood)}")
by_route = {}
for r in na:
by_route[r["route"]] = by_route.get(r["route"], 0) + 1
print("routes:", by_route)
assert sum(by_route.values()) == len(na)
assert set(ROUTES) == set(by_route), "route set must be the five-way"
by_family = {}
for r in na:
by_family[r["family_id"]] = by_family.get(r["family_id"], 0) + 1
by_fold = {}
for r in na:
by_fold[r["cv_fold"]] = by_fold.get(r["cv_fold"], 0) + 1
print(f"family_ids: {len(by_family)} split_groups: {len(set(r['split_group'] for r in na))}")
print("folds:", by_fold)
os.makedirs(OUT_DIR, exist_ok=True)
with open(os.path.join(OUT_DIR, "pool.json"), "w") as f:
json.dump(na, f, ensure_ascii=False, indent=1)
with open(os.path.join(OUT_DIR, "ood.json"), "w") as f:
json.dump(ood, f, ensure_ascii=False, indent=1)
with open(os.path.join(OUT_DIR, "stats.json"), "w") as f:
json.dump({
"dev_count": len(dev),
"residual_count": len(rows),
"non_action_count": len(na),
"action_ood_count": len(ood),
"routes": by_route,
"family_ids": len(by_family),
"folds": by_fold,
"top_family": dict(sorted(by_family.items(), key=lambda kv: -kv[1])[:15]),
}, f, ensure_ascii=False, indent=1)
print(f"wrote {OUT_DIR}/{{pool,ood,stats}}.json")
if __name__ == "__main__":
main()
@@ -0,0 +1,559 @@
#!/usr/bin/env python3
"""
Slice 23: five-way residual non-action semantic router corrected population
============================================================================
Slice 22 reported the corpus's fast-path mirror was stale next to the
production stage-0 grammars (185 residual rows resolved at runtime). Slice 23
derives fast_path_resolved from the real router, rebuilds the corpus, and
re-measures the primary slice-22 results on the corrected residual pool.
Logic and configs are slice22_main.py verbatim; only OUT_DIR differs.
Population: the corrected dev-pool residual non-action rows (1509; the pool
written by slice23_emit.py). Corrected action rows (720) are OOD probes only.
Metrics written to /tmp/mvn-s23/results.json:
§1 population
§2 legacy baseline (legacy.json / legacy_heads.json): acc, macro-F1,
per-class P/R/F1, confusion, illegal_action_prediction count
§3 e5-linear primary head: C grid, grouped CV OOF, per-fold P/R/F1 +
variance + composition
§4 floors: majority, centroid (cosine nearest-mean), sparse word+char
TF-IDF logistic (slice18 builder), all grouped CV
§5 route-family (family_id) leave-family-out
§6 knowledge vs memory_write: matched pairs (water/homelab/task) ordering
§7 uncertain as an explicit class: P/R/F1 + top confusions
§8 OOF confidence: max-softmax correct/wrong, ECE, log-loss, Brier,
coverage/accuracy/macro-F1 abstention curves (no threshold chosen)
§9 action OOD probes: fold models applied to the corrected action rows
§10 artifact cost: head params, serialized bytes, incremental head latency
No corpus label is changed. No frozen-holdout rows are inspected.
"""
import json
import os
import sys
import time
import numpy as np
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, HERE)
import slice18_sparse # noqa: E402
import slice19_main # noqa: E402
EMB_PATH = "/tmp/mvn-experiment/embeddings.json"
OUT_DIR = "/tmp/mvn-s23"
CLASSES = ["conversation", "knowledge", "memory_write", "system", "uncertain"]
CLASS_PREFIX = ["conversation", "knowledge", "memory_write", "system", "uncertain"]
C_GRID = [0.1, 1.0, 10.0]
# Route-family holdouts the report calls out by name (slice-22 brief): every
# family that is not part of the shared subject inventory on either side.
HOLDOUT_GROUPS = {
"capability": ["knowledge:capability-ha", "knowledge:capability-tool"],
"world": ["knowledge:world-def", "knowledge:world-explain"],
"calendar": ["knowledge:calendar", "knowledge:calendar-time", "knowledge:calendar-next"],
"recall": ["knowledge:recall-fact", "knowledge:recall-note", "knowledge:recall-possessive"],
"fact": ["fact:meal", "fact:water", "fact:sleep", "fact:shower", "fact:break", "fact:pills", "fact:exercise"],
"note": ["note:idea", "note:homelab", "note:task"],
"remember": ["free:remember"],
"system": None, # all system:*
"conversation": None,
"uncertain": None,
}
def load_pool_and_embeds():
with open(os.path.join(OUT_DIR, "pool.json")) as f:
pool = json.load(f)
meta, examples = slice18_sparse.load_data()
dev = slice18_sparse.filter_dev_pool(examples)
by_idx = {e["dev_idx"]: e for e in dev} if "dev_idx" in dev[0] else None
# pool rows carry idx = position among dev_pool rows in dev order
emb_by_idx = {i: np.asarray(e["embedding"], dtype=np.float64)
for i, e in enumerate(dev)}
for r in pool:
r["emb"] = emb_by_idx[r["idx"]]
r["y"] = r["route"]
return pool, meta
def oof_proba_grouped(X, y, folds, C=1.0):
"""Grouped OOF probability matrix (n×5, class order CLASSES)."""
y_idx = np.array([CLASSES.index(c) for c in y])
folds = np.asarray(folds)
proba = np.zeros((len(y_idx), len(CLASSES)))
for te_fold in sorted(set(folds.tolist())):
tr = folds != te_fold
te = folds == te_fold
clf = slice18_sparse.LogisticRegression(
C=C, max_iter=2000, solver="lbfgs", random_state=42)
clf.fit(X[tr], y_idx[tr])
proba[te] = clf.predict_proba(X[te])
return proba
def cls_metrics(yt, yp):
import sklearn.metrics as m
yt = np.asarray(yt)
yp = np.asarray(yp)
if yt.dtype != np.int64 and yt.dtype != np.int32:
yt = np.array([CLASSES.index(c) for c in yt])
if yp.dtype != np.int64 and yp.dtype != np.int32:
yp = np.array([CLASSES.index(c) for c in yp])
labels = list(range(len(CLASSES)))
n = len(yt)
acc = m.accuracy_score(yt, yp)
macro = m.f1_score(yt, yp, average="macro", labels=labels, zero_division=0)
pr, rc, f1, sup = m.precision_recall_fscore_support(
yt, yp, labels=labels, zero_division=0)
per = {c: {"p": float(pr[i]), "r": float(rc[i]), "f1": float(f1[i]), "n": int(sup[i])}
for i, c in enumerate(CLASSES)}
conf = m.confusion_matrix(yt, yp, labels=labels).tolist()
return {"n": n, "acc": acc, "macro_f1": macro, "per_class": per, "confusion": conf}
def fold_report(yt, proba, folds, true_y):
out = {}
folds_arr = np.asarray(folds)
comp = {}
for f in sorted(set(folds_arr.tolist())):
mask = folds_arr == f
yt_f = [CLASSES.index(y) for y in true_y[mask]]
comp[f] = {c: int((np.array(true_y[mask]) == c).sum()) for c in CLASSES}
per_fold = {}
for f in sorted(set(folds_arr.tolist())):
mask = folds_arr == f
yp = proba[mask].argmax(1).tolist()
m = cls_metrics([yt[i] for i in np.where(mask)[0].tolist()], yp)
per_fold[f] = {"acc": m["acc"], "macro_f1": m["macro_f1"]}
out["composition"] = comp
out["per_fold"] = per_fold
accs = [v["acc"] for v in per_fold.values()]
macros = [v["macro_f1"] for v in per_fold.values()]
out["acc_mean"] = float(np.mean(accs))
out["acc_std"] = float(np.std(accs))
out["macro_f1_mean"] = float(np.mean(macros))
out["macro_f1_std"] = float(np.std(macros))
return out
def ece(yt, proba, n_bins=15):
conf = proba.max(1)
pred = proba.argmax(1)
acc = (pred == yt).astype(float)
bins = np.linspace(0, 1, n_bins + 1)
tot = 0.0
details = []
counts = 0
for i in range(n_bins):
lo, hi = bins[i], bins[i + 1]
m = (conf >= lo) & (conf < hi) if i < n_bins - 1 else conf >= lo
if m.sum() == 0:
continue
acc_m = acc[m].mean()
conf_m = conf[m].mean()
w = m.sum() / len(conf)
tot += w * abs(acc_m - conf_m)
counts += int(m.sum())
details.append({"bin": i, "lo": lo, "hi": hi, "conf": float(conf_m),
"acc": float(acc_m), "n": int(m.sum())})
return {"ece": float(tot), "n_bins": n_bins, "counted": counts, "bins": details}
def main():
pool, meta = load_pool_and_embeds()
pool.sort(key=lambda r: r["idx"])
print(f"pool: {len(pool)} rows")
from sklearn.metrics import brier_score_loss, log_loss
report = {"population": {}, "legacy": {}, "e5_linear": {}, "floors": {},
"family_holdouts": {}, "kmw": {}, "uncertain": {}, "confidence": {},
"ood": {}, "artifact": {}}
# ── §1 population ──────────────────────────────────────────────────────
cnt = {}
for r in pool:
cnt[r["y"]] = cnt.get(r["y"], 0) + 1
report["population"] = {
"n": len(pool),
"routes": cnt,
"family_ids": len(set(r["family_id"] for r in pool)),
"split_groups": len(set(r["split_group"] for r in pool)),
"folds": {str(f): int(sum(1 for r in pool if r["cv_fold"] == f)) for f in sorted(set(r["cv_fold"] for r in pool))},
"corpus": {k: v for k, v in meta.items() if k in
("dev_count", "residual_count", "fast_path_count",
"dimension", "embedder_id", "input_template", "pooling", "normalization")},
}
print("\n§1 population:", report["population"])
X = np.vstack([r["emb"] for r in pool])
y = np.array([r["y"] for r in pool])
folds = np.array([r["cv_fold"] for r in pool])
yt = np.array([CLASSES.index(c) for c in y])
# ── §2 legacy baselines ────────────────────────────────────────────────
import collections
for tag, fname in [("hash", "legacy.json"), ("heads", "legacy_heads.json")]:
path = os.path.join(OUT_DIR, fname)
if not os.path.exists(path):
continue
leg = json.load(open(path))
leg_by_idx = {r["idx"]: r for r in leg}
yp_leg = []
illegal = []
for r in pool:
lr = leg_by_idx[r["idx"]]
if lr["illegal_action_prediction"]:
illegal.append(lr)
yp_leg.append("action")
else:
yp_leg.append(lr["class"])
yp_leg = np.array(yp_leg)
# five-way: an 'action' prediction is an error (outside the label set)
yp5 = np.array([("uncertain" if p == "action" else p) for p in yp_leg])
m = cls_metrics(y, yp5)
m["illegal_action_prediction"] = len(illegal)
m["illegal_cases"] = [{"idx": i["idx"], "text": i["text"], "route": i["route"],
"intent": i["intent"], "producer": i["producer"],
"confidence": i["confidence"]} for i in illegal]
# per-cell confusion also shows 'action' column
conf_counts = collections.Counter(zip(y, yp_leg))
m["confusion_with_action"] = {f"{a}->{b}": int(c) for (a, b), c in conf_counts.items()}
report["legacy"][tag] = m
print(f"\n§2 legacy ({tag}) acc={m['acc']:.4f} macroF1={m['macro_f1']:.4f} "
f"illegal={len(illegal)}")
for c in CLASSES:
p = m["per_class"][c]
print(f" {c:<14} P={p['p']:.3f} R={p['r']:.3f} F1={p['f1']:.3f} n={p['n']}")
# grammar-pure residual: rows not resolved by any current stage-0 grammar
if os.path.exists(os.path.join(OUT_DIR, "legacy.json")):
leg = json.load(open(os.path.join(OUT_DIR, "legacy.json")))
gh = {r["idx"] for r in leg if r["producer"] == "grammar"}
gp_mask = np.array([r["idx"] not in gh for r in pool])
report["grammar_drift"] = {
"grammar_hits_in_pool": len(gh),
"grammar_pure_n": int(gp_mask.sum()),
}
# ── §3 e5-linear primary head ─────────────────────────────────────────
print("\n§3 e5-linear")
bestC, bestMac = 1.0, -1.0
grid = {}
for C in C_GRID:
p = oof_proba_grouped(X, y, folds, C=C)
mp = cls_metrics(y, p.argmax(1).tolist())
grid[float(C)] = {"acc": mp["acc"], "macro_f1": mp["macro_f1"]}
print(f" C={C} acc={mp['acc']:.4f} macroF1={mp['macro_f1']:.4f}")
if mp["macro_f1"] > bestMac:
bestMac, bestC = mp["macro_f1"], C
print(f" -> best C={bestC}")
p_best = oof_proba_grouped(X, y, folds, C=bestC)
m_best = cls_metrics(y, p_best.argmax(1).tolist())
m_best["C"] = bestC
m_best["C_grid"] = grid
m_best["folds"] = fold_report(yt, p_best, folds, y)
report["e5_linear"] = m_best
for f, v in m_best["folds"]["per_fold"].items():
print(f" fold {f}: acc={v['acc']:.4f} macroF1={v['macro_f1']:.4f}")
print(f" fold acc mean={m_best['folds']['acc_mean']:.4f} "
f"std={m_best['folds']['acc_std']:.4f}; "
f"macroF1 mean={m_best['folds']['macro_f1_mean']:.4f} "
f"std={m_best['folds']['macro_f1_std']:.4f}")
for c in CLASSES:
p_ = m_best["per_class"][c]
print(f" {c:<14} P={p_['p']:.3f} R={p_['r']:.3f} F1={p_['f1']:.3f} n={p_['n']}")
# grammar-pure sensitivity for the primary head
if "grammar_drift" in report:
mp_gp = cls_metrics(y[gp_mask], p_best[gp_mask].argmax(1).tolist())
report["e5_linear"]["grammar_pure"] = {
"acc": mp_gp["acc"], "macro_f1": mp_gp["macro_f1"], "n": int(gp_mask.sum())}
# ── §4 floors ─────────────────────────────────────────────────────────
print("\n§4 floors")
# majority floor
maj = CLASSES.index("knowledge")
ym = np.full(len(y), maj)
mm = cls_metrics(y, ym)
report["floors"]["majority"] = {"acc": mm["acc"], "macro_f1": mm["macro_f1"],
"per_class": mm["per_class"]}
print(f" majority (predict {CLASSES[maj]}): acc={mm['acc']:.4f} macroF1={mm['macro_f1']:.4f}")
# centroid floor: cosine to per-class mean of the training folds' embeddings
cf_proba = np.zeros((len(yt), len(CLASSES)))
folds_arr = np.asarray(folds)
for te_fold in sorted(set(folds_arr.tolist())):
tr = folds_arr != te_fold
te = folds_arr == te_fold
centroids = []
for c in CLASSES:
idxs = np.where(tr & (y == c))[0]
ctr = X[idxs].mean(axis=0)
ctr = ctr / np.linalg.norm(ctr)
centroids.append(ctr)
Cm = np.vstack(centroids)
sims = X[te] @ Cm.T
cf_proba[te] = sims
yc = cf_proba.argmax(1)
# accuracy + macroF1 with the same 5-way
mc = cls_metrics(y, yc.tolist())
report["floors"]["centroid"] = {"acc": mc["acc"], "macro_f1": mc["macro_f1"],
"per_class": mc["per_class"]}
print(f" centroid cosine: acc={mc['acc']:.4f} macroF1={mc['macro_f1']:.4f}")
# sparse word+char logistic (slice18 builder, grouped CV, five-way)
texts = [r["n_text"] for r in pool]
Xs, _vec = slice18_sparse.build_features(texts, "both")
psp = np.zeros((len(yt), len(CLASSES)))
for te_fold in sorted(set(folds_arr.tolist())):
tr = folds_arr != te_fold
te = folds_arr == te_fold
clf = slice18_sparse.LogisticRegression(
C=1.0, max_iter=2000, solver="lbfgs", random_state=42)
clf.fit(Xs[tr], yt[tr])
psp[te] = clf.predict_proba(Xs[te])
msp = cls_metrics(y, psp.argmax(1).tolist())
report["floors"]["sparse_word_char"] = {
"acc": msp["acc"], "macro_f1": msp["macro_f1"], "per_class": msp["per_class"],
"vocab": slice18_sparse.vocab_size(_vec)}
print(f" sparse both: acc={msp['acc']:.4f} macroF1={msp['macro_f1']:.4f} "
f"vocab={report['floors']['sparse_word_char']['vocab']}")
# ── §5 route-family holdouts ─────────────────────────────────────────
print("\n§5 route-family holdouts")
fam = np.array([r["family_id"] for r in pool])
holdouts = {}
all_fams = sorted(set(fam.tolist()))
for grp, fams in HOLDOUT_GROUPS.items():
if fams is None:
fams = [f for f in all_fams if f.startswith(grp + ":")]
mask = np.isin(fam, fams)
if mask.sum() == 0:
continue
tr = ~mask
clf = slice18_sparse.LogisticRegression(
C=bestC, max_iter=2000, solver="lbfgs", random_state=42)
clf.fit(X[tr], yt[tr])
ypgrp = clf.predict(X[mask])
m = cls_metrics([CLASSES.index(c) for c in y[mask]], ypgrp.tolist())
m["families"] = fams
m["rows"] = int(mask.sum())
holdouts[grp] = {"acc": m["acc"], "macro_f1": m["macro_f1"], "n": int(mask.sum()),
"per_class": m["per_class"]}
print(f" {grp:<14} n={m['rows']} acc={m['acc']:.4f} macroF1={m['macro_f1']:.4f}")
# full leave-one-family-out summary
lofo_accs = []
lofo_f1s = []
for f in all_fams:
mask = fam == f
tr = ~mask
clf = slice18_sparse.LogisticRegression(
C=bestC, max_iter=2000, solver="lbfgs", random_state=42)
clf.fit(X[tr], yt[tr])
ypf = clf.predict(X[mask])
m = cls_metrics([CLASSES.index(c) for c in y[mask]], ypf.tolist())
lofo_accs.append(m["acc"])
lofo_f1s.append(m["macro_f1"])
holdouts["_all_49_lo_"] = {"n_families": len(all_fams),
"acc_mean": float(np.mean(lofo_accs)),
"macro_f1_mean": float(np.mean(lofo_f1s))}
report["family_holdouts"] = holdouts
print(f" leave-one-family-out over {len(all_fams)} families: "
f"acc mean={np.mean(lofo_accs):.4f} macroF1 mean={np.mean(lofo_f1s):.4f}")
# ── §6 knowledge vs memory_write ─────────────────────────────────────
print("\n§6 knowledge vs memory_write")
# reuse e5-linear OOF: does the model put the higher probability on the
# right side (memory_write for a write, knowledge for a recall)?
conf_km = np.zeros((2, 2))
pk = p_best[:, CLASSES.index("knowledge")]
pmw = p_best[:, CLASSES.index("memory_write")]
for i in range(len(yt)):
t = y[i]
if t == "knowledge":
conf_km[0, 1 if pmw[i] > pk[i] else 0] += 1
elif t == "memory_write":
conf_km[1, 1 if pmw[i] >= pk[i] else 0] += 1
report["kmw"] = {"confusion_p_ordered": conf_km.tolist()}
# matched pairs with shared subject lexemes, corpus-justified
def build_pairs(subject, fam_k, fam_mw):
kr = [r for r in pool if r["family_id"] in fam_k]
mr = [r for r in pool if r["family_id"] in fam_mw]
pairs = []
for mw in mr:
for k in kr:
if subject in mw["n_text"] and subject in k["n_text"]:
pairs.append((mw["idx"], k["idx"], mw["n_text"], k["n_text"]))
return pairs
sets = {
"water": build_pairs("вод", ["knowledge:recall-fact"], ["fact:water"]),
"homelab": build_pairs("dns", ["knowledge:homelab-status"], ["note:homelab"])
+ build_pairs("сервер", ["knowledge:homelab-status"], ["note:homelab"])
+ build_pairs("vlan", ["knowledge:homelab-status"], ["note:homelab"]),
"task": build_pairs("задач", ["knowledge:task-check", "knowledge:deadline"],
["note:task"]),
}
idx_of = {r["idx"]: i for i, r in enumerate(pool)}
pair_rep = {}
for name, pairs in sets.items():
if not pairs:
continue
ok = 0
margins = []
bad = []
for mi, ki, mx, kx in pairs:
mi_i, ki_i = idx_of[mi], idx_of[ki]
# MW row should get a higher memory_write probability than the K row
mk = (pmw[mi_i] + 0.0)
if pmw[mi_i] > pmw[ki_i]:
ok += 1
else:
bad.append((mx[:46], round(float(pmw[mi_i]), 3), kx[:46], round(float(pmw[ki_i]), 3)))
margins.append(pmw[mi_i] - pmw[ki_i])
pair_rep[name] = {
"pairs": len(pairs),
"mw_over_k_order_acc": ok / len(pairs),
"mean_margin": float(np.mean(margins)),
"reversed_examples": bad[:6],
}
print(f" {name}: pairs={len(pairs)} order_acc={ok/len(pairs):.3f} "
f"mean_margin={np.mean(margins):+.3f}")
report["kmw"]["matched_pairs"] = pair_rep
# ── §7 uncertain as explicit class ────────────────────────────────────
print("\n§7 uncertain")
up = m_best["per_class"]["uncertain"]
uc = m_best["confusion"][CLASSES.index("uncertain")]
report["uncertain"] = {
"per_class": up,
"row_from_uncertain": {CLASSES[j]: int(uc[j]) for j in range(5)},
"row_to_uncertain": {CLASSES[j]: int(m_best["confusion"][j][CLASSES.index("uncertain")])
for j in range(5)},
}
print(f" uncertain n={up['n']} P={up['p']:.3f} R={up['r']:.3f} F1={up['f1']:.3f}")
print(" wrong-→label pulled from uncertain:", report["uncertain"]["row_from_uncertain"])
print(" →uncertain pulled from:", report["uncertain"]["row_to_uncertain"])
# ── §8 OOF confidence / calibration / abstention ─────────────────────
print("\n§8 confidence / calibration")
conf = p_best.max(1)
right = (p_best.argmax(1) == yt)
cer = {
"correct_conf_mean": float(conf[right].mean()),
"correct_conf_median": float(np.median(conf[right])),
"wrong_conf_mean": float(conf[~right].mean()),
"wrong_conf_median": float(np.median(conf[~right])),
"ece": ece(yt, p_best)["ece"],
"ece_bins": ece(yt, p_best)["bins"],
"log_loss": float(log_loss(yt, p_best, labels=[0, 1, 2, 3, 4])),
}
# Brier is label-set specific: one-vs-rest mean
briers = []
for i in range(5):
briers.append(brier_score_loss((yt == i).astype(int), p_best[:, i]))
cer["brier_macro"] = float(np.mean(briers))
report["confidence"] = cer
print(f" right conf mean={cer['correct_conf_mean']:.3f} "
f"wrong conf mean={cer['wrong_conf_mean']:.3f} ECE={cer['ece']:.4f}")
print(f" log_loss={cer['log_loss']:.4f} brier_macro={cer['brier_macro']:.4f}")
thr_grid = np.linspace(0.10, 0.98, 45)
abst = []
for t in thr_grid:
cov = (conf >= t).mean()
if cov == 0:
continue
keep = conf >= t
yt_k = yt[keep]
yp_k = p_best[keep].argmax(1)
mk_ = cls_metrics(yt_k.tolist(), yp_k.tolist())
abst.append({"threshold": round(float(t), 3), "coverage": float(cov),
"accuracy": mk_["acc"], "macro_f1": mk_["macro_f1"]})
report["confidence"]["abstention_curve"] = abst
print(" threshold | coverage | accuracy | macroF1 (first 6/45 + knee)")
for row in abst[::9]:
print(f" {row['threshold']:.2f} | {row['coverage']:.3f} | "
f"{row['accuracy']:.3f} | {row['macro_f1']:.3f}")
# ── §9 action OOD probes ──────────────────────────────────────────────
print("\n§9 action OOD")
ood_rows = [r for r in json.load(open(os.path.join(OUT_DIR, "ood.json")))]
emb_by_idx = {i: np.asarray(e["embedding"], dtype=np.float64)
for i, e in enumerate(slice18_sparse.filter_dev_pool(
slice18_sparse.load_data()[1]))}
Xo = np.vstack([emb_by_idx[r["idx"]] for r in ood_rows])
fold_models = []
for te_fold in sorted(set(folds_arr.tolist())):
tr = folds_arr != te_fold
clf = slice18_sparse.LogisticRegression(
C=bestC, max_iter=2000, solver="lbfgs", random_state=42)
clf.fit(X[tr], yt[tr])
fold_models.append(clf)
# XX
# comparable to the non-action in-fold behaviour
po = np.zeros((len(Xo), 5))
for clf in fold_models:
po += clf.predict_proba(Xo)
po /= len(fold_models)
ood_top = int(np.argmax(po.mean(0)))
ood_conf = po.max(1)
ood_pred = po.argmax(1)
top_dist = {CLASSES[i]: int((ood_pred == i).sum()) for i in range(5)}
confident_na = int((ood_conf > 0.9).sum())
report["ood"] = {
"n": len(ood_rows),
"top_class": CLASSES[int(ood_top)],
"top_class_dist": top_dist,
"conf_gt_0.9": confident_na,
"conf_gt_0.9_frac": float(confident_na / len(ood_rows)),
"conf_mean": float(ood_conf.mean()),
"conf_median": float(np.median(ood_conf)),
}
print(f" action OOD n={len(ood_rows)}: most-confident class={report['ood']['top_class']} "
f"dist={top_dist}")
print(f" conf>0.9: {confident_na} ({confident_na/len(ood_rows):.3f}) "
f"conf mean={report['ood']['conf_mean']:.3f}")
# ── §10 artifact cost ────────────────────────────────────────────────
print("\n§10 artifact")
n_params = len(CLASSES) * X.shape[1] + len(CLASSES)
fp32 = n_params * 4
report["artifact"] = {
"e5_dim": X.shape[1],
"head_params": n_params,
"head_fp32_bytes": fp32,
"head_fp32_kib": fp32 / 1024,
"head_int8_bytes": n_params,
}
# incremental latency of the linear head over a batch of 1 (µs)
clf = slice18_sparse.LogisticRegression(C=bestC, max_iter=2000, solver="lbfgs", random_state=42)
clf.fit(X, yt)
x1 = X[:1]
for _ in range(50):
clf.predict_proba(x1)
lat = []
for _ in range(2000):
t0 = time.perf_counter_ns()
clf.predict_proba(x1)
lat.append((time.perf_counter_ns() - t0) / 1e3)
lat = np.array(lat)
report["artifact"]["head_latency_us_mean"] = float(lat.mean())
report["artifact"]["head_latency_us_p50"] = float(np.median(lat))
print(f" head params={n_params} fp32={fp32/1024:.2f}KiB "
f"lat mean={lat.mean():.2f}us p50={np.median(lat):.2f}us")
with open(os.path.join(OUT_DIR, "results.json"), "w") as f:
json.dump(report, f, ensure_ascii=False, indent=1, default=float)
print(f"\nwrote {OUT_DIR}/results.json")
if __name__ == "__main__":
main()
+293
View File
@@ -0,0 +1,293 @@
{
"db_path": "/var/lib/maven/maven.db.enc",
"db_tmpfs": "/dev/shm/maven-plain.db",
"db_key_env": "MAVEN_DB_KEY",
"socket_path": "/run/maven/mavend.sock",
"state_dir": "/var/lib/maven",
"//disabled_rules": [
"Nudge rules that are not wired at all. Names come from loop.DefaultRules:",
"water, meal, break, service_down, netdata_critical.",
"service_down is back on: mavpoll now writes one fact per kuma monitor",
"(service_down:<name>), so the nudge names the service and pausing a monitor",
"in kuma silences that monitor. It is also edge-triggered, so a service that",
"stays down is one nudge, not one every fifteen minutes."
],
"disabled_rules": [],
"phraser": {
"model_path": "/opt/maven/models/llm/qwen3/Qwen3-1.7B-UD-Q4_K_XL.gguf",
"bin_path": "llama-server",
"n_gpu_layers": 99,
"n_ctx": 4096,
"cache_ram_mib": 512,
"timeout": "60s",
"llm_nudges": false
},
"//ntfy": [
"The second reach (V-649). Until 07-08-2026 telegram was the only one, and",
"telegram needs api.telegram.org, the socks relay below and a matching ufw",
"rule — three things in series that have each failed once, and when they do",
"a sev4 nudge has nowhere to go. ntfy shares none of them: it is reached",
"directly, no relay.",
"It is the preferred away reach. Reminders fall back to Telegram in order,",
"and every missing reach is recorded in the outbox rather than disappearing.",
"The credential is an ntfy access token, scoped write-only to this one",
"topic, so a popped sink can push to it and cannot read it back. Set it in",
"deploy/telegram.env beside the telegram secrets; that file is gitignored."
],
"ntfy": {
"disabled": true,
"base_url": "https://ntfy.kvmx.ru",
"topic": "maven",
"token": "${NTFY_TOKEN}"
},
"telegram": {
"bot_token": "${TELEGRAM_BOT_TOKEN}",
"chat_id": "${TELEGRAM_CHAT_ID}",
"//proxy": [
"api.telegram.org is not reachable directly from this box, so every send",
"timed out. The relay is the x-ui socks inbound on the host, port 10808;",
"192.168.240.1 is the maven_default bridge gateway, which is how a",
"container addresses the host. mavend is on that network.",
"This needs a matching ufw rule or the container's SYN is dropped:",
" ufw allow from 192.168.240.0/20 to any port 10808 proto tcp"
],
"proxy": "socks5://192.168.240.1:10808",
"//intake": [
"Read the chat as well as write to it (V-637). The poller long-polls",
"getUpdates through the same relay and accepts chat_id as the only",
"sender. Deleting this key turns inbound off again.",
"chat_id must be numeric here or the daemon refuses to start: an inbound",
"update names its chat by number, so an @-name would match nothing."
],
"intake": true
},
"//workstation": [
"The big model on the desk PC (workpc, 7900 GRE 16GB), fronted by",
"mavgpud on port 8080. It runs gemma-4-12b and it is preferred over the",
"resident Qwen3-1.7B for routing and replies whenever the card is free.",
"The machine is never assumed up: it sleeps, and the card is often held by",
"a CPT run, in which case mavgpud answers 503 and Maven falls back to the",
"resident model without saying so. Deleting this block restores exactly",
"the behaviour homesrv had before it existed.",
"Addressed by LAN address, not container name: mavgpud runs on another",
"machine and there is no shared docker network to name it on.",
"model_disabled keeps only this model arm dark until MAVEN_GPU_TOKEN is",
"provisioned; the independently authenticated STT arm below stays live."
],
"//workstation.stt": [
"CrisperWhisper 2.0 turbo on the same machine, a second service on port",
"8081 and not a second endpoint on mavgpud. whisper.cpp cannot load CW2 at",
"all: it derives its language count from the vocabulary size, and CW2's",
"51897 tokens shift seven special token ids. So it runs under transformers",
"there and mavsttd stays whisper.cpp here.",
"Worth the second service: CW2 turbo scores 10.4% WER in Russian against",
"27.5% for the ggml-small.bin mavsttd loads, measured on 200 Golos clips",
"in docs/evals/2026-08-09-crisperwhisper2-russian-wer.md.",
"Deleting this block sends every utterance to mavsttd, which is what the",
"box did before it existed. A worse transcript is still a turn, so the",
"fallback is silent and Kami is never told which machine heard him.",
"The token is what stops anything on the LAN posting audio to that port."
],
"workstation": {
"model_disabled": true,
"url": "http://192.168.1.105:8080",
"token": "${MAVEN_GPU_TOKEN}",
"probe": "15s",
"timeout": "90s",
"stt": {
"url": "http://192.168.1.105:8081/transcribe",
"token": "${MAVEN_STT_TOKEN}",
"probe": "15s",
"timeout": "10s"
}
},
"//search": [
"The live web, searched after his own notes and before Kiwix. Only the",
"query string leaves the box — never a note, a fact, the persona block or",
"the history — and a question about him never reaches here at all.",
"The instance must have `json` in search.formats (settings.yml); a stock",
"SearXNG answers 403 to format=json and every search then fails. It is",
"addressed by container name, so it needs the same maven_default",
"attachment kiwix has, and it must listen on 9563: 8080 is taken several",
"times over on this box. No instance reachable ⇒ she falls through to the",
"ZIMs and never says the search failed."
],
"search": {
"url": "http://searxng:9563",
"max_results": 4,
"snippet_runes": 1500,
"language": "auto",
"timeout": "8s"
},
"//kiwix": [
"The offline encyclopedia, searched after his own notes and before anything",
"on the network. kiwix-server publishes 8034 on loopback only, so a container",
"cannot reach it by address; it is attached to the maven_default network",
"instead and addressed by container name. That attachment is imperative and",
"does not survive recreating the kiwix stack — make it declarative there:",
" networks: [default, maven_default] # maven_default: external: true",
"The book is the catalog name from the /content/... href in",
"/catalog/v2/entries, not the display title. Others on the box:",
"ifixit_en_all_2025-06, devdocs_en_ansible_2025-10."
],
"kiwix": {
"url": "http://kiwix-server:8080",
"book": "wikipedia_en_all_maxi_2026-02",
"book_ru": "wikipedia_ru_all_maxi_2026-02",
"max_results": 5,
"snippet_runes": 1500
},
"//morning_routines": [
"The daily checklist (Vikunja #280). Each item is done when its fact_key",
"gets a non-voided fact inside the window, so 'выпил воды' closes water and",
"nothing has to be ticked by hand. nudge_at fires once, at the end of the",
"window, and only for what is still open. Weekdays empty = every day."
],
"morning_routines": [
{
"name": "утро",
"window_start": "08:00",
"window_end": "11:00",
"nudge_at": "10:30",
"severity": 1,
"items": [
{ "key": "medicine", "fact_key": "medicine", "label": "лекарство" },
{ "key": "water", "fact_key": "water", "label": "вода" },
{ "key": "pets", "fact_key": "pets", "label": "покормить кота" }
]
}
],
"//feeds": [
"RSS reading (Vikunja #258). Every item lands as a note with source",
"rss:<name>, which is also what puts entries in the intake journal that",
"/events reads. Only the feed URL leaves the box.",
"This is a starting pair, not a curated set — trim or extend it."
],
"feeds": {
"poll_interval": "30m",
"max_items": 5,
"max_age": "24h",
"sources": [
{ "name": "lwn", "url": "https://lwn.net/headlines/newrss", "category": "технологии" },
{ "name": "archlinux", "url": "https://archlinux.org/feeds/news/", "category": "технологии" }
]
},
"//crawl": [
"Reading a web page (Vikunja #259). on_demand answers 'посмотри <URL>'.",
"No allow_hosts, so any public host he names is readable; private",
"addresses are refused unconditionally by internal/webfetch and do not",
"need listing. Setting allow_hosts here would also narrow on-demand,",
"which is the point of leaving it empty."
],
"crawl": {
"on_demand": true,
"timeout": "10s",
"max_runes": 4000
},
"digest": {
"enabled": true,
"window": "30m",
"max_items": 5,
"severity_ceiling": 2
},
"pattern_proposals": {
"notify": false,
"cooldown": "24h"
},
"mcp": {
"timeout": "15s",
"servers": [
{
"name": "vikunja",
"url": "http://192.168.1.104:9100/mcp",
"allow_private": true,
"allow_tools": ["list_projects", "list_tasks", "get_task_details", "create_task"],
"max_tools": 6,
"enabled": false
}
]
},
"smarthome": {
"provider": "homeassistant",
"url": "http://192.168.1.50:8123",
"token": "${HA_TOKEN}",
"domains": ["light", "switch", "sensor"],
"max_entities": 40,
"timeout": "10s",
"refresh": "15m",
"enabled": false
},
"netscan": {
"subnets": ["192.168.1.0/24"],
"ports": [22, 80, 443, 8080],
"timeout": "400ms",
"rate": 100,
"max_hosts": 256,
"enabled": true
},
"nexus": { "url": "http://nexus:9740" },
"praxis": { "url": "http://praxis:8989" },
"hexis": { "url": "http://hexis:9741" },
"voice": {
"enabled": true,
"bind": "0.0.0.0:9100",
"lang": "ru",
"stt": { "socket": "/run/maven/stt.sock", "lang": "ru" },
"tts": { "socket": "/run/maven/tts.sock", "lang": "ru" },
"embedder": {
"model_path": "/opt/maven/models/embedder/multilingual-e5-small/model_quantized.onnx",
"tokenizer_path": "/opt/maven/models/embedder/multilingual-e5-small/tokenizer.json",
"lib_path": "/opt/maven/lib/libonnxruntime.so",
"heads_path": "/opt/maven/models/embedder/router-heads/router_heads.onnx"
},
"llm_router": true,
"query_min_score": 0.80,
"query_min_margin": 0.008,
"clarify_max_attempts": 3,
"tool_timeout": "30s",
"tools": [
{ "name": "status", "cmd": ["systemctl", "status"], "scope": "homelab", "destructive": false,
"aliases": ["статус", "покажи статус", "проверь статус"] },
{ "name": "ps", "cmd": ["docker", "ps"], "scope": "homelab", "destructive": false,
"aliases": ["статус докера", "лог докера", "покажи запущенные контейнеры", "покажи контейнеры", "список контейнеров", "что запущено"] },
{ "name": "uptime", "cmd": ["uptime"], "scope": "homelab", "destructive": false,
"aliases": ["покажи uptime", "аптайм", "как работает сервер", "сколько работает сервер"] },
{ "name": "disk", "cmd": ["df", "-h"], "scope": "homelab", "destructive": false,
"aliases": ["сколько места на диске", "сколько свободного места на диске", "место на диске", "покажи диск"] },
{ "name": "memory", "cmd": ["free", "-h"], "scope": "homelab", "destructive": false,
"aliases": ["свободная память", "сколько оперативной памяти свободно", "покажи память"] },
{ "name": "logs", "cmd": ["journalctl", "-n", "50", "-u"], "scope": "homelab", "destructive": false,
"aliases": ["покажи логи", "логи", "лог"] },
{ "name": "restart", "cmd": ["systemctl", "restart"], "scope": "homelab", "destructive": true,
"aliases": ["перезапусти", "перезагрузи", "рестарт"] },
{ "name": "stop", "cmd": ["systemctl", "stop"], "scope": "homelab", "destructive": true,
"aliases": ["останови", "останови сервис"] },
{ "name": "start", "cmd": ["systemctl", "start"], "scope": "homelab", "destructive": true,
"aliases": ["запусти", "запусти сервис"] },
{ "name": "docker-restart", "cmd": ["docker", "restart"], "scope": "homelab", "destructive": true,
"aliases": ["перезапусти контейнер", "перезагрузи контейнер"] },
{ "name": "docker-stop", "cmd": ["docker", "stop"], "scope": "homelab", "destructive": true,
"aliases": ["останови контейнер"] },
{ "name": "reboot", "cmd": ["systemctl", "reboot"], "scope": "homelab", "destructive": true,
"aliases": ["перезагрузи сервер", "перезагрузи хост"] }
]
}
}
+6
View File
@@ -11,6 +11,8 @@ The tier is the path, so staleness is visible from the filename.
| `docs/evals/` | dated measurements, one file per measurement. **Never edited after the day.** A newer number is a new file. Indexed in `docs/evals/CLAUDE.md`, which marks each one live or superseded. | forever |
| `docs/caveats/` | known limits, one entry per limit, each with a task id and a revisit trigger. Indexed in `docs/caveats/CLAUDE.md`. | until fixed, then deleted |
| `docs/plans/` | the plan for one piece of work, frozen once it starts. Indexed in `docs/plans/CLAUDE.md`. | until the work lands |
| `docs/capabilities/` | generated. The capability ledger and its probe harness, regenerated from `docs/spec.md` plus a named eval. **Never hand-edited**, except `domains.yaml`, `probes_field.json` and the scripts, which are its sources. Indexed in `docs/capabilities/README.md`. | until the spec or the measurement moves |
| `docs/architecture/` | generated. The architecture observation and its evidence pack, rebuilt from source by the scripts beside it. Indexed in `docs/architecture/README.md`. | until the shape changes |
| `docs/archive/` | dead. Read by nobody by default. | forever |
## Rules for this directory
@@ -21,6 +23,10 @@ The tier is the path, so staleness is visible from the filename.
correction. Do not append a changelog to it.
* A number in prose with no `docs/evals/` file behind it is an opinion.
* Fixing something deletes its caveat. It does not edit the eval that found it.
* **A generated tier is rebuilt, never corrected.** A wrong row in
`docs/capabilities/` or `docs/architecture/` is a bug in the generator or in
one of its hand-written inputs. Editing the output makes the next rebuild
silently undo the fix.
## Where a subsystem's reasoning lives
+143
View File
@@ -0,0 +1,143 @@
# docs/architecture
An observation of Maven as built, read at `5cae33a` on 2026-08-25. It describes
what the code does today. It proposes nothing.
This directory is a build output plus its sources. `index.html`,
`maven-architecture.json`, `anchors.md` and `diagrams/*.svg` are generated.
## Read it
| file | what it is |
|---|---|
| `index.html` | the viewer. Open it from the filesystem, no server needed. Seven views, the rendered diagram above each, click a component for its record. |
| `check_viewer.js` | the viewer's only check. Runs every view against a DOM stub, because a TypeError in a renderer shows as a blank panel and not as an error. |
| `findings.md` | the analysis. Kept apart from the facts on purpose. |
| `maven-architecture.json` | the inventory. 160 components, 204 relations. The factual source for everything else. |
| `anchors.md` | every symbol the inventory names, resolved to `path:line` with the line quoted. |
| `diagrams/*.mmd` | the five views as Mermaid source. `03a`, `03b` and `03c` are the three traced requests. |
| `diagrams/*.svg` | the same, rendered. |
## Rebuild it
```sh
python3 docs/architecture/build_inventory.py # → maven-architecture.json
python3 docs/architecture/verify_anchors.py # → anchors.md, exit 1 if stale
sh docs/architecture/render.sh # → diagrams/*.svg, then index.html, then check_viewer.js
python3 docs/architecture/build_viewer.py # → index.html alone
node docs/architecture/check_viewer.js # → every view rendered, no throw
```
Views 6 and 7 read `docs/capabilities/`, not this directory. View 6 is the
capability matrix, 51 rows against seven dimensions. View 7 is the twelve
cross-cutting invariants and the components that participate in each. Both are
inlined by `build_viewer.py`, which reads `ledger.yaml` and `invariants.yaml`
rather than deriving anything itself: the ledger's build is the only thing
allowed to decide a dimension.
`render.sh` drives mermaid-cli through the system chromium rather than letting
puppeteer download its own. It is also the only syntax check this repo has for a
`.mmd`.
## What is verified, and what is not
**Verified mechanically.** `verify_anchors.py` resolves all 692 claimed symbols
against the files their component names. Current state: 681 resolved to a line
and 0 unresolved, with 0 missing files. The other 11 are config keys and make
targets rather than Go identifiers, so they are skipped. The script exits
non-zero on any failure, which makes it a staleness gate.
Writing it caught 29 symbols filed under the wrong component and seven names
that were wrong outright. Two examples: `Store.RecordEvent` for what is really
`Store.CreateEvent`, and `media.Keeper` for what is really `media.Store`.
**Not verified.** That a symbol means what its `responsibility` says. An anchor
proves the identifier is on that line and nothing more. Judgements about
ownership, coupling and enforcement are readings of the code. A reading can be
wrong in a way grep cannot catch.
**Marked, not resolved.** Relations carry a `confidence` field. `medium` means
the wiring is in the source and the call path was not traced end to end. `low`
means it was inferred from one reference. The viewer can hide both. Four
relations are `medium` and one is `low`.
**Deployment-specific.** Sixteen components are `configured-off` against
`deploy/mavend.json` as it stood on the day, and that file was dirty in the
working tree. A different config makes different components live. `status` says
which, per component.
## The one thing to check first
`findings.md` 6.3 through 6.3d. They say the system has no single point that
decides whether an origin may cause an effect, and that the pieces which look
like that point are each answering a different question.
Revised on 2026-08-25 after an independent second pass. Four readings changed
and one earlier statement was wrong. Section 6.3 marks the corrections.
Start at `internal/tool/tool.go:181`, `cmd/mavend/ecosystem_acts.go:158` and
`internal/router/claim.go:38`.
## The evidence pack
`sh docs/architecture/pack_evidence.sh` builds `maven-evidence.zip` at the repo
root: this directory, the structural context, and whole source files for the
architectural seams. Whole files, never snippets, because a cut-down file loses
the call path that makes a claim checkable.
The path list is an allowlist, not an exclusion list. A denylist ships whatever
nobody thought to exclude, and this tree has a database key in it.
`architecture-evidence.txt` is the reviewer's index. It resolves a named symbol
list against the checkout and says plainly when a requested name does not exist.
It also re-runs the probe under every contradiction, so a claim and its grep
cannot drift apart.
One file is not verbatim. `docker-compose.yml` carries an uptime-kuma API key,
so a redacted copy ships in its place with that one value replaced. The script
diffs the two and aborts if anything else changed.
The scan at the end refuses to build on a credential-shaped hit rather than
printing a warning. Both of its first two versions were wrong in instructive
ways. The name filter deleted `internal/router/singletoken.go` for matching
`*token*`. The value scan flagged docker volume lines that name where a secret
would live and contain none.
## The authorization function as implemented
The reconstruction, at the one decision point that gates an act
(`internal/tool/tool.go:156`):
```
permit(tool, confirmed) =
row.status == "enabled" tool.go:164
AND tier != irreversible risk.go:74 VoiceMayRun:false
AND (tier == safe OR confirmed) risk.go:72,76
```
`tier` is `RiskOf(row)`. The reach is not an input: `Executor.Exec` takes
`(ctx, name, args, confirmed)` and no surface.
`confirmed` is unproven at this boundary too. The invariant that a confirmation
binds one capability, one target and an expiry lives in `pendingAct` and
`resolveConfirm`. `Exec` trusts the boolean.
The expression covers two of the three act paths. Hexis reuses it deliberately
(`cmd/mavend/ecosystem_acts.go:768`). The Praxis lifecycle path has no tier and
no confirm turn: `praxisItemAction.handle` calls straight through at
`ecosystem_acts.go:158`.
Behind the IPC boundary, `auth.Can(method, scope, params)` runs with
`scope.Surface` always `SurfaceCoreProcess` (`internal/auth/enrollment.go:65`)
and step-up held as one global timestamp that ignores `Scope`
(`internal/webauthn/session.go:38` and `:62`).
`auth` answers who may carry what authority. `tool` answers what effect a
capability has and what proof it demands. Those are orthogonal, not competing.
The decision combining them does not exist.
Two representations of reach exist and both are ignored. `server.go:198` only
defaults an empty `p.Surface`, so a client-asserted one survives and nothing
reads it. `server.go:148` hardcodes `Session.Surface`. Since `req.Surface` is
request payload on an unauthenticated wire, it must not become an authorization
input as it stands.
+290
View File
@@ -0,0 +1,290 @@
#!/usr/bin/env python3
"""Write architecture-evidence.txt: the reviewer's index into the pack.
It resolves a named symbol list against this checkout and prints where each one
is, or says plainly that it does not exist. A requested name that is absent is
evidence too, so nothing here is silently dropped or silently corrected.
Every contradiction is re-checked at generation time by running its own probe,
so the claim and the grep that supports it cannot drift apart in the pack.
python3 docs/architecture/build_evidence.py
"""
import os
import re
import subprocess
import sys
HERE = os.path.dirname(os.path.abspath(__file__))
ROOT = os.path.abspath(os.path.join(HERE, "..", ".."))
TREES = ["cmd", "internal"]
# The reviewer's list, verbatim on the left. Where a name does not exist in this
# repo, the right side is what it appears to mean. Resolution below reports both
# so a wrong name is visible rather than quietly fixed.
REQUESTED = [
("auth.TierFor", "auth.MaxLayer"),
("auth.Surface", None),
("auth.Layer", None),
("tool.Executor.Exec", None),
("tool.PolicyFor", None),
("tool.RiskOf", None),
("tool.RiskSafe", "tool.TierSafe"),
("tool.RiskDestructive", "tool.TierDestructive"),
("tool.RiskIrreversible", "tool.TierIrreversible"),
("router.ClaimOf", None),
("reactiveHandler", None),
("tickLoop", None),
]
# Added because the authorization function the reviewer wants to reconstruct
# runs through these and the list above does not reach them.
ALSO = [
"auth.Gate", "auth.Can", "auth.Requirement", "auth.Authority",
"auth.NewFloorEnrollment", "auth.StaticEnrollment", "auth.Scope",
"tool.Policy", "tool.RiskOfCapability", "tool.irreversibleVerbs",
"tool.ErrNeedsConfirm", "tool.ErrNeedsAuthedSurface", "tool.ErrNotEnabled",
"ipc.Server.Check", "ipc.CheckFunc",
"voice.PushToTalkReq", "voice.Sessions.Add", "voice.Session",
"PolicyFor", "RiskOf", "Executor.Exec",
"pendingAct", "resolveConfirm", "actionAct", "runTurn", "applyAction",
"querySources", "queryWalk", "StageZeroGrammars", "Router.Route",
]
PKG_DIR = {
"auth": "internal/auth", "tool": "internal/tool", "claim": "internal/claim",
"modes": "internal/modes", "router": "internal/router", "voice": "internal/voice",
"ipc": "internal/ipc", "store": "internal/store",
}
def go_files(rel):
full = os.path.join(ROOT, rel)
out = []
for base, _, names in os.walk(full):
for n in sorted(names):
if n.endswith(".go"):
out.append(os.path.relpath(os.path.join(base, n), ROOT))
return sorted(out)
def all_go():
out = []
for t in TREES:
out.extend(go_files(t))
return out
_CACHE = {}
def lines_of(rel):
"""Read once. resolve() sweeps every file per pattern per symbol, and
re-reading cmd/ and internal/ that many times took minutes."""
if rel not in _CACHE:
try:
_CACHE[rel] = open(os.path.join(ROOT, rel), errors="replace").read().splitlines()
except OSError:
_CACHE[rel] = []
return _CACHE[rel]
def resolve(sym):
"""Find the declaration of sym. Returns (path, line, text) or (None,)*3."""
tail = sym.split(".")[-1]
recv = sym.split(".")[-2] if sym.count(".") >= 1 else None
pats = [
re.compile(r"^func\s+\(\w+\s+\*?" + re.escape(recv or "\x00") + r"\)\s+" + re.escape(tail) + r"\b"),
re.compile(r"^func\s+" + re.escape(tail) + r"\b"),
re.compile(r"^type\s+" + re.escape(tail) + r"\b"),
re.compile(r"^\s*" + re.escape(tail) + r"\s+\w+\s*=\s"), # typed const
re.compile(r"^\s*" + re.escape(tail) + r"\s*=\s"),
re.compile(r"^(var|const)\s+" + re.escape(tail) + r"\b"),
re.compile(r"^\s*" + re.escape(tail) + r"\s+\w"), # struct field
]
pkg = sym.split(".")[0]
files = go_files(PKG_DIR[pkg]) if pkg in PKG_DIR else all_go()
files = [f for f in files if not f.endswith("_test.go")]
for pat in pats:
for rel in files:
for i, line in enumerate(lines_of(rel), 1):
if pat.match(line):
return rel, i, line.strip()
return None, None, None
def exported(pkg_rel):
"""Every exported declaration in a package, for the `claim.*` / `modes.*` asks."""
out = []
pat = re.compile(r"^(func|type|const|var)\s+\(?[^)]*\)?\s*([A-Z]\w*)")
fn = re.compile(r"^func\s+(\([^)]*\)\s*)?([A-Z]\w*)")
for rel in go_files(pkg_rel):
if rel.endswith("_test.go"):
continue
for i, line in enumerate(lines_of(rel), 1):
m = fn.match(line) or pat.match(line)
if m:
name = m.group(m.lastindex)
if name and name[0].isupper():
out.append((name, f"{rel}:{i}", line.strip()))
return out
def sh(cmd):
return subprocess.run(cmd, shell=True, cwd=ROOT, capture_output=True,
text=True).stdout.strip()
# Each probe is a shell command whose output IS the evidence. Re-run at pack
# time so the pack cannot claim something the checkout no longer shows.
CONTRADICTIONS = [
("auth surface/layer documented as control, not consumed on turn path",
# px.Surface is the Praxis lifecycle verb, an unrelated name collision, and
# Surfaced* are the read-out-item helpers. Excluded by name so the absence
# this probe reports is the auth Surface and not a filtering accident.
"grep -rnE 'req\\.Surface|sess\\.Surface|Session\\.Surface|auth\\.Surface|voice\\.Surface' cmd/mavend/*.go "
"| grep -v _test | grep -vE 'px\\.Surface|Surfaced' "
"|| echo '(no match: no file in cmd/mavend reads the auth Surface of a request or a session)'"),
("auth.Can runs only behind the IPC boundary",
"grep -rn 'auth\\.' cmd/ internal/ --include='*.go' | grep -v _test "
"| grep -v '^internal/auth/' | grep -vE ':[0-9]+:\\s*(//|\\*)'"),
("tool risk policy live on execution path",
"sed -n '176,190p' internal/tool/tool.go"),
("tool executor receives no reach/surface",
"grep -n 'func (e \\*Executor) Exec' internal/tool/tool.go"),
("voice server normalizes incoming surface to pc-client",
"grep -n 'SurfacePCClient' internal/voice/server.go"),
("claim abstraction exists but Route does not consume it",
"grep -rn 'ClaimOf' --include='*.go' cmd internal | grep -v _test || echo '(only the definition; no caller)'"),
("modes package is imported by nothing",
"grep -rn 'internal/modes' --include='*.go' cmd internal | grep -v '^internal/modes/' || echo '(no importer)'"),
("voice server DEFAULTS an empty surface, it does not overwrite a sent one",
"sed -n '193,201p' internal/voice/server.go"),
("session surface is hardcoded, independently of the request field",
"sed -n '146,149p' internal/voice/server.go"),
("HandlePushToTalk never reads req.Surface",
"sed -n '200,203p' cmd/mavend/voice.go"),
("hexis reuses the same risk policy",
"grep -n 'RiskOfCapability\\|PolicyFor' cmd/mavend/ecosystem_acts.go"),
("praxis lifecycle mutations bypass the risk policy entirely",
"sed -n '156,172p' cmd/mavend/ecosystem_acts.go"),
("Exec trusts a confirmed bool it cannot verify was bound",
"grep -n 'func (e \\*Executor) Exec' internal/tool/tool.go; grep -rn 'tools.Exec(' cmd/mavend/*.go | grep -v _test"),
("FloorEnrollment maps every same-uid caller to one surface",
"sed -n '63,72p' internal/auth/enrollment.go"),
("PasskeySession ignores Scope in both methods",
"grep -n 'func (s \\*PasskeySession) CurrentLayer\\|func (s \\*PasskeySession) Assert' internal/webauthn/session.go"),
("Claim.Coverage can be 1.0 with nothing extracted",
"grep -n 'func claimSpans' -A 4 internal/router/claim.go; grep -n 'd.Slots.Text = ex.Text' -B 2 internal/router/router.go; grep -n 'func (c Claim) Coverage' -A 7 internal/claim/claim.go"),
("claim_test asserts Band only, and every case sets Text == Utterance",
"grep -n 'Utterance:\\|Text:\\|want:' internal/router/claim_test.go | head -20"),
("systemctl reboot is destructive, not irreversible",
"grep -n 'irreversibleVerbs = map' -A 8 internal/tool/risk.go; grep -n 'reboot' deploy/mavend.json"),
]
def main() -> int:
out = []
w = out.append
w("architecture evidence pack")
w("=" * 72)
w("")
w("commit: " + sh("git rev-parse HEAD"))
w("date: " + sh("git log -1 --format=%cd --date=short"))
w("branch: " + sh("git rev-parse --abbrev-ref HEAD"))
w("")
w("working tree at pack time (git status --short):")
for line in (sh("git status --short") or "(clean)").splitlines():
w(" " + line)
w("")
w("The pack is built from the WORKING TREE, not from the commit. The lines")
w("above are the difference. deploy/mavend.json in particular is modified:")
w("phraser.model_path points at maven-instruct-b2, the committed value was")
w("Qwen3-1.7B-UD-Q4_K_XL. Sixteen 'configured-off' claims read this file.")
w("")
w("requested symbols")
w("-" * 72)
for name, actual in REQUESTED:
rel, line, text = resolve(name)
if rel:
w(f"- {name}")
w(f" {rel}:{line} {text}")
elif actual:
arel, aline, atext = resolve(actual)
w(f"- {name} -> DOES NOT EXIST in this repo")
if arel:
w(f" the name appears to be {actual}")
w(f" {arel}:{aline} {atext}")
else:
w(f" and neither does {actual}")
else:
w(f"- {name} -> NOT FOUND")
w("")
for pkg, rel in (("claim.*", "internal/claim"), ("modes.*", "internal/modes")):
w(f"{pkg} ({rel})")
w("-" * 72)
for name, anchor, text in exported(rel):
w(f"- {name}")
w(f" {anchor} {text}")
w("")
w("additional symbols on the authorization path")
w("-" * 72)
for name in ALSO:
rel, line, text = resolve(name)
w(f"- {name}")
w(f" {rel}:{line} {text}" if rel else " NOT FOUND")
w("")
w("known contradictions, each re-checked at pack time")
w("=" * 72)
w("The command under each claim was run against this checkout just now.")
w("Its output is what follows. Nothing here is transcribed by hand.")
w("")
for claim, cmd in CONTRADICTIONS:
w("- " + claim)
w(" $ " + cmd)
res = sh(cmd)
for line in (res or "(no output)").splitlines():
w(" " + line)
w("")
w("what the pack does NOT contain, and why")
w("=" * 72)
w("- .git, so no history and no gitignored working files travel with it.")
w("- deploy/telegram.env and deploy/db_key.env. The second holds the")
w(" database key. Both are gitignored and present in the working tree.")
w("- docker-compose.yml verbatim. It carries one live-looking credential on")
w(" line 132 (an uptime-kuma API key). The pack ships")
w(" docker-compose.redacted.yml with that one value replaced and nothing")
w(" else changed, so the mavcaldav and mavmaild claims stay checkable.")
w("- models/, deps/, *.db, *.onnx, *.gguf, certs, logs, node_modules.")
w("- internal/session, internal/db and tests/ from the requested list: none")
w(" of the three exists. Sessions live in internal/voice/session.go, the")
w(" store is internal/store, and tests sit beside their code as *_test.go.")
w("")
w("included test files, since the ask named them by subject:")
for pat, label in (
("internal/router", "routing and arbitration"),
("internal/tool", "risk and confirmation"),
("internal/auth", "authorization"),
("internal/claim", "claim"),
("cmd/mavend", "turn path, confirm gate, query chain"),
):
n = sh(f"find {pat} -name '*_test.go' | wc -l")
w(f" {label}: {n} *_test.go under {pat}/")
w("")
w("fixtures are synthetic, not captured speech: internal/router/eval/*.json")
w("and cmd/mavend/testdata/**.json are hand-written contracts. Named here")
w("because they are Russian utterances and look like personal data.")
path = os.path.join(HERE, "architecture-evidence.txt")
open(path, "w").write("\n".join(out) + "\n")
print(f"architecture-evidence.txt: {os.path.getsize(path)} bytes, {len(out)} lines")
return 0
if __name__ == "__main__":
sys.exit(main())
File diff suppressed because it is too large Load Diff
+60
View File
@@ -0,0 +1,60 @@
#!/usr/bin/env python3
"""Assemble index.html from the template, the inventory and the diagrams.
index.html is self-contained on purpose: it opens from the filesystem with no
server, and a browser at file:// refuses to fetch a sibling JSON. So the
inventory, every .mmd source and every rendered .svg are inlined here rather
than loaded at runtime.
Run it through docs/architecture/render.sh, which re-renders the SVGs first.
Running it alone rebuilds the viewer against whatever SVGs are already there.
"""
import json
import os
import yaml
HERE = os.path.dirname(os.path.abspath(__file__))
DIA = os.path.join(HERE, "diagrams")
CAPDIR = os.path.join(os.path.dirname(HERE), "capabilities")
def main() -> None:
arch = json.load(open(os.path.join(HERE, "maven-architecture.json")))
mermaid, svg = {}, {}
for name in sorted(os.listdir(DIA)):
path = os.path.join(DIA, name)
if name.endswith(".mmd"):
mermaid[name] = open(path).read()
elif name.endswith(".svg"):
svg[name] = open(path).read()
# The capability half. Generated beside this one and read here rather than
# re-derived: the ledger's build is the only thing allowed to decide a
# dimension, and a second derivation would drift from it silently.
ledger = yaml.safe_load(open(os.path.join(CAPDIR, "ledger.yaml")))
invariants = yaml.safe_load(
open(os.path.join(CAPDIR, "invariants.yaml")))["invariants"]
payload = (
"const ARCH = " + json.dumps(arch, ensure_ascii=False) + ";\n"
"const CAPS = " + json.dumps(ledger, ensure_ascii=False) + ";\n"
"const INV = " + json.dumps(invariants, ensure_ascii=False) + ";\n"
"const MERMAID = " + json.dumps(mermaid, ensure_ascii=False) + ";\n"
"const SVG = " + json.dumps(svg, ensure_ascii=False) + ";\n"
)
template = open(os.path.join(HERE, "viewer.template.html")).read()
if "/*__DATA__*/" not in template:
raise SystemExit("viewer.template.html has no /*__DATA__*/ marker")
out = os.path.join(HERE, "index.html")
open(out, "w").write(template.replace("/*__DATA__*/", payload))
print(
"index.html: %d bytes, %d components, %d relations, %d diagrams, "
"%d rendered, %d capabilities, %d invariants"
% (os.path.getsize(out), len(arch["components"]), len(arch["edges"]),
len(mermaid), len(svg), len(ledger["capabilities"]), len(invariants))
)
if __name__ == "__main__":
main()
+64
View File
@@ -0,0 +1,64 @@
// Smoke test for index.html's renderers, run by render.sh when node is present.
//
// The viewer has no test harness and a TypeError in a renderer produces a blank
// panel, not an error anyone sees. This runs every view's render function
// against a DOM stub and fails loudly on the first throw. It checks that the
// renderers run over the real data, not that the result looks right.
//
// node docs/architecture/check_viewer.js [path/to/index.html]
const fs = require('fs');
const path = process.argv[2] || __dirname + '/index.html';
const src = fs.readFileSync(path, 'utf8');
const js = src.match(/<script>([\s\S]*)<\/script>/)[1];
const el = () => {
const e = {
innerHTML: '', textContent: '', style: {}, checked: true, value: '',
_written: 0,
dataset: {}, classList: { add(){}, remove(){}, toggle(){} },
querySelectorAll: () => [], querySelector: () => null,
appendChild(){}, addEventListener(){}, scrollIntoView(){},
getBoundingClientRect: () => ({top:0,left:0,width:0,height:0}),
};
return e;
};
// One shared element per id, so a renderer's output can be read back. A stub
// that silently swallows innerHTML would let an empty render pass.
const els = {};
const document = {
getElementById: id => (els[id] = els[id] || el()), querySelectorAll: () => [], querySelector: () => null,
createElementNS: el, createElement: el, addEventListener(){},
};
const window = { addEventListener(){} };
const requestAnimationFrame = () => {};
// `const` inside a direct eval stays in the eval's own scope, so the checks are
// appended to the source and evaluated with it rather than run beside it.
const checks = `
let n = 0;
for (const v of VIEWS) {
setView(v.id);
n++;
}
// Every flow, and every capability's side panel: the branch a click takes.
for (const k of Object.keys(FLOWS)) { S.flow = k; renderFlow(el()); n++; }
setView('c1');
if (els.main.innerHTML.length < 5000) throw new Error('capability matrix rendered ' + els.main.innerHTML.length + ' chars');
for (const c of CAPS.capabilities) {
renderCapSide(c.id);
if (els.side.innerHTML.length < 400) throw new Error('thin panel for ' + c.id);
n++;
}
S.capBy = 'domain'; renderCaps(el()); n++;
setView('c2');
if (els.main.innerHTML.length < 4000) throw new Error('invariants view rendered ' + els.main.innerHTML.length + ' chars');
for (const iv of INV) { invRollup(iv); n++; }
for (const c of ARCH.components) { renderSide(c.id); n++; }
console.log('viewer: ' + n + ' render calls, ' + VIEWS.length + ' views, ' +
CAPS.capabilities.length + ' capabilities, ' + INV.length +
' invariants, no throw');
`;
eval(js + checks);
@@ -0,0 +1,120 @@
%% View 1 — System topology.
%% Runtime processes and external systems, with process boundaries drawn explicitly.
%% mavend is the centre because the code makes it one: it is the only key holder,
%% it owns the store, the IPC socket, the voice listener, the tick loop, eight
%% in-process background workers and the child llama-server.
%% Evidence: docker-compose.yml, cmd/mavend/main.go, cmd/mavend/boot.go,
%% deploy/mavwaked.service, deploy/mavgpud.service.
flowchart LR
subgraph WORKPC["workpc — systemd user units, never in docker-compose"]
direction TB
MAVWAKED["mavwaked<br/>process<br/>arecord · silero VAD · keyword head"]
MAVGPUD["mavgpud<br/>process<br/>GPU supervisor"]
LLAMA_W["llama-server<br/>model · workstation card"]
CW2["CrisperWhisper2 turbo<br/>model · port 8081"]
ALSA["arecord / aplay<br/>external"]
TUNNEL["maven-voice-tunnel.service<br/>ssh · the only path in"]
end
subgraph HOMESRV["homesrv — docker compose project `maven`"]
direction TB
subgraph MAVEND_P["mavend — process boundary · the only key holder"]
direction TB
IPCSRV["IPC server<br/>unix /run/maven/mavend.sock"]
VOICESRV["voice server<br/>TCP 0.0.0.0:9100"]
TURN["reactive handler<br/>the turn pipeline"]
TICK["tick loop<br/>60s"]
WORKERS["8 background workers<br/>tick · fact-enrichment · feed · crawl<br/>voice · mcp · home · memory-eval"]
STORE[("store<br/>sqlite, MaxOpenConns=1")]
end
LLAMA_H["llama-server<br/>model · resident<br/>child process of mavend"]
MAVSTTD["mavsttd<br/>process<br/>whisper.cpp"]
MAVTTSD["mavttsd<br/>process<br/>piper"]
MAVWEB["mavweb<br/>process<br/>HTTP 127.0.0.1:9201"]
MAVPOLL["mavpoll<br/>process<br/>network_mode: host"]
SEARX["SearXNG<br/>external"]
KIWIX["kiwix-server<br/>external"]
NETDATA["netdata<br/>external"]
KUMA["uptime-kuma<br/>external"]
end
subgraph OFF["built, not deployed — commented out in docker-compose.yml"]
direction TB
MAVCALDAV["mavcaldav<br/>process"]
MAVMAILD["mavmaild<br/>process"]
end
subgraph ECO["ecosystem network — external compose project"]
direction TB
NEXUS["Nexus<br/>external · identity"]
PRAXIS["Praxis<br/>external · attention"]
HEXIS["Hexis<br/>external · capabilities"]
end
subgraph NET["internet"]
direction TB
TG["Telegram Bot API<br/>external · via SOCKS relay"]
NTFY["ntfy<br/>external · DISABLED in config"]
ZM["zenmoney<br/>external · no token mounted"]
end
HA["Home Assistant<br/>external · enabled:false"]
%% ---- voice path
ALSA --- MAVWAKED
MAVWAKED -->|"PushToTalk · TCP"| TUNNEL
TUNNEL -->|"ssh to 127.0.0.1:9110"| VOICESRV
VOICESRV -->|"proactive Push on the same conn"| MAVWAKED
%% ---- module IPC
MAVWEB -->|"3 × ipc.Client · unix"| IPCSRV
MAVWEB -->|"POST /api/ptt · TCP mavend:9100"| VOICESRV
MAVPOLL -->|"WriteFact · unix"| IPCSRV
MAVCALDAV -.->|"WriteFact · unix"| IPCSRV
MAVMAILD -.->|"IngestMail · unix"| IPCSRV
%% ---- worker sockets
TURN -->|"worker · unix stt.sock"| MAVSTTD
TURN -->|"worker · unix tts.sock"| MAVTTSD
TURN -->|"HTTP · preferred, silent fallback"| CW2
%% ---- models
MAVEND_P ---|"spawns and owns"| LLAMA_H
MAVGPUD ---|"spawns and supervises"| LLAMA_W
MAVGPUD ---|"spawns and supervises"| CW2
TURN -.->|"llm.Pair · model_disabled:true"| MAVGPUD
%% ---- world and ecosystem
TURN -->|"HTTP · query string only"| SEARX
TURN -->|"HTTP"| KIWIX
TURN -->|"HTTP · v1 contract, correlation id"| NEXUS
TURN -->|"HTTP"| PRAXIS
TURN -->|"HTTP"| HEXIS
MAVWEB -->|"HTTP · read-only panel"| NEXUS
MAVWEB -->|"HTTP · read-only panel"| PRAXIS
MAVWEB -->|"HTTP · read-only panel"| HEXIS
TURN -.->|"HTTP · enabled:false"| HA
%% ---- reaches
TICK -->|"telegram sink"| TG
TG -->|"getUpdates long poll"| TURN
TICK -.->|"ntfy sink · nil, disabled"| NTFY
%% ---- pollers
MAVPOLL --> NETDATA
MAVPOLL --> KUMA
MAVPOLL -.-> ZM
classDef proc fill:#1f3a5f,stroke:#7fb3ff,color:#eaf2ff
classDef ext fill:#3d2f4f,stroke:#c39bd3,color:#f4ecf7
classDef model fill:#4a3a1f,stroke:#e0b050,color:#fff6e0
classDef store fill:#1f4a3a,stroke:#6ed0a8,color:#e8fff5
classDef off fill:#3a3a3a,stroke:#888,color:#ccc,stroke-dasharray:4 3
class MAVWAKED,MAVGPUD,MAVSTTD,MAVTTSD,MAVWEB,MAVPOLL,IPCSRV,VOICESRV,TURN,TICK,WORKERS proc
class ALSA,SEARX,KIWIX,NETDATA,KUMA,NEXUS,PRAXIS,HEXIS,TG,TUNNEL ext
class LLAMA_H,LLAMA_W,CW2 model
class STORE store
class MAVCALDAV,MAVMAILD,NTFY,ZM,HA off
@@ -0,0 +1,160 @@
%% View 2 — Core internals of mavend.
%% The real path, in the order runTurn actually runs it. The sequence is NOT
%% input → routing → intent → state → tools → response: eleven stateful
%% pre-emptors get first refusal BEFORE routing, and a query intent then enters
%% a second, longer arbitration of its own.
%% Evidence: cmd/mavend/voice.go runTurn, cmd/mavend/turnroute.go,
%% cmd/mavend/actions.go, cmd/mavend/actions_query.go, internal/router/router.go.
flowchart TB
subgraph IN["input — three reaches, one pipeline"]
A1["voice.Server<br/>HandlePushToTalk"]
A2["daemonAPI.Chat<br/>mavweb /api/chat"]
A3["telegram poller<br/>getUpdates"]
STT["stt seam<br/>Remote mavsttd · CW2 · Stub"]
end
A1 --> STT --> RT
A2 --> RT
A3 --> A2
RT["runTurn<br/>cmd/mavend/voice.go"]
RT --> D0["decision.With<br/>one arbitration record per turn"]
D0 --> TR0["turnRoute created<br/>sync.Once, on the context"]
subgraph PRE["pre-route ladder — 11 rungs, order load-bearing"]
direction TB
P1["1 expired-clarify notice"]
P2["2 confirm answer<br/>resolveConfirm"]
P3["3 targeted repair"]
P4["3b untargeted repair"]
P5["3c command prohibition"]
P6["4 clarify answer"]
P7["5 quiet toggle"]
P8["5b snooze"]
P9["5c ack"]
P10["5d reminder cancellation"]
P11["5e ordinal selection"]
P1-->P2-->P3-->P4-->P5-->P6-->P7-->P8-->P9-->P10-->P11
end
TR0 --> PRE
PRE -->|"any rung claims"| OUT
subgraph ROUTE["step 6 — the cascade · internal/router"]
direction TB
CONT["continuationDecision<br/>an elliptical follow-up is answered<br/>from the previous turn, not routed"]
S0["stage 0 grammars<br/>StageZeroGrammars · first match wins at 1.0<br/>the ONLY arm that may set SourceAnchored"]
SH["stage 0b routing heads<br/>ONNX softmax over the label set"]
SL["stage 1a LLM router<br/>resident model, grammar-constrained"]
SC["stage 1 classifier<br/>nearest centroid · THE FLOOR<br/>names no destination"]
SE["stage 2 extractor + stage 3 gate"]
CONT -->|"not a continuation"| S0
S0 -->|"no match"| SH
SH -->|"declines"| SL
SL -->|"error or unparsable"| SC
SH --> SE
SL --> SE
SC --> SE
end
PRE -->|"nobody claimed"| ROUTE
ROUTE --> DLG["step 7 dialogue merge<br/>followUpMerge · rememberTurn"]
DLG --> CLAR{"step 8<br/>dec.Clarify OR a required slot missing?"}
CLAR -->|"yes"| ASK["askClarify<br/>park the request, ask one question"]
ASK --> OUT
CLAR -->|"no"| ACT
subgraph ACT["step 9 — actionHandlers, 7 intents"]
direction TB
HF["fact<br/>actions_fact.go"]
HR["reminder<br/>actions_reminder.go"]
HA["act<br/>actions_act.go"]
HN["note"]
HC["chat"]
HS["system"]
HQ["query → the chain"]
end
subgraph QC["the query chain — a SECOND arbitration, 22 sources"]
direction TB
QW["queryWalk<br/>removes only guesses:true sources<br/>when the cascade named a destination"]
Q1["his data<br/>fact-by-key · day-plan · habits · tasks<br/>attention · list · money · history · feeds<br/>home · network · calendar · weather · self<br/>embed · memory · notes"]
QB["personal boundary<br/>the only source a stage 0 anchor may drop"]
Q2["the world<br/>search → kiwix → web → general-knowledge"]
QW --> Q1 --> QB --> Q2
end
HQ --> QC
HF -->|"question-shaped ⇒ re-route"| QC
HF -->|"complaint ⇒ re-route"| HC
subgraph STATE["state and memory"]
direction TB
DB[("store · sqlite<br/>facts · reminders · notes · tools<br/>tasks · lists · nudges")]
VEC[("memory_vectors<br/>brute-force cosine")]
DLGS[("dialogue_sessions<br/>persisted, TTL 2m")]
CLS["clarifyStore<br/>IN MEMORY ONLY, by design"]
PEND["pending act / routine / hexis<br/>3 single-slot registers, one mutex"]
SURF["surfacedItems<br/>last Praxis read-out order"]
RING["decision.Ring<br/>bounded, in memory"]
end
HF --> DB
HF --> VEC
HR --> DB
HN --> DB
HN --> VEC
Q1 --> DB
Q1 --> VEC
DLG --> DLGS
ASK --> CLS
HA --> PEND
QC --> SURF
D0 --> RING
subgraph TOOLS["act execution"]
direction TB
ALLOW[("tools table<br/>only status='enabled' runs")]
EXEC["tool.Executor<br/>+ MCP + Home Assistant callers"]
CONF["destructive confirm turn<br/>binds capability, entity, args, requester, expiry"]
HEX["Hexis capability<br/>entity id resolved via Nexus first"]
end
HA --> ALLOW --> EXEC
HA --> CONF
HA --> HEX
subgraph RESP["response generation"]
direction TB
REP["replier<br/>only when the handler returned \"\""]
PHR["phraser<br/>parseResponseMood is the one parser"]
TTS["tts seam<br/>Remote mavttsd · Stub"]
end
ACT --> RESP
QC --> RESP
REP --> PHR
OUT["reply text<br/>+ notice + resumed question"]
RESP --> OUT
OUT -->|"voice path only"| TTS
subgraph PROACT["the other half of the process — nothing above touches it"]
direction TB
TICKL["tick loop · 60s<br/>13 jobs in one function"]
GTH["loop.Gatherer<br/>one consistent snapshot"]
RUL["loop rules + restraint gate<br/>pure"]
DISP["delivery.Dispatcher<br/>ChannelsFor severity,presence"]
SNK["sinks: voice · ntfy · telegram"]
TICKL --> GTH --> RUL --> TICKL
TICKL --> DISP --> SNK
end
TICKL --> DB
SNK -->|"PushToMostRecent on the request conn"| A1
classDef stage fill:#1f3a5f,stroke:#7fb3ff,color:#eaf2ff
classDef store fill:#1f4a3a,stroke:#6ed0a8,color:#e8fff5
classDef mem fill:#4a3a1f,stroke:#e0b050,color:#fff6e0
classDef danger fill:#4f2626,stroke:#e08080,color:#ffecec
class S0,SH,SL,SC,SE,CONT stage
class DB,VEC,DLGS,ALLOW store
class CLS,PEND,SURF,RING mem
class QB,CONF danger
@@ -0,0 +1,78 @@
%% View 3a — Runtime flow: a reminder request.
%% Traced through cmd/mavend/voice.go runTurn, internal/router/stagezero.go,
%% cmd/mavend/clarify.go, cmd/mavend/actions_reminder.go, cmd/mavend/tick.go,
%% internal/loop/loop.go and internal/delivery/dispatcher.go.
%% Shows the branch where the hour is missing, the parked clarify, the answer
%% turn, the write and the eventual delivery with durable retry.
sequenceDiagram
autonumber
participant K as Owner
participant W as mavwaked
participant V as voice.Server
participant H as reactiveHandler.runTurn
participant PRE as pre-route ladder
participant R as router cascade
participant CL as clarifyStore
participant AR as actionReminder
participant DB as store
participant T as tick loop
participant D as dispatcher
Note over K,W: "Мэйвен, напомни позвонить маме"
K->>W: speech
W->>W: silero VAD + keyword head, score ≥ 0.999
W->>V: PushToTalkReq, one utterance
V->>H: HandlePushToTalk
H->>H: stt seam → text
H->>H: decision.With, turnRoute created
H->>PRE: 11 rungs
PRE-->>H: nobody claims
H->>R: rt.resolve
R->>R: stage 0 ReminderGrammar matches, Stage=0, conf 1.0
R-->>H: IntentReminder, Slots.Text="позвонить маме", HasTime=false
rect rgb(70,40,40)
Note over H,CL: BRANCH — missingFor names `time`, whatever the confidence
H->>H: dec.Clarify false BUT len missingFor > 0 → step 8 fires
H->>CL: Push a PendingQuestion, park the request
H-->>V: "во сколько напомнить?"
V-->>W: reply audio + text
end
Note over K,W: "в семь вечера"
K->>W: speech
W->>V: PushToTalkReq
V->>H: runTurn
H->>PRE: rung 4, resolveClarifyAnswer
PRE->>CL: Pop the parked question
PRE->>R: extractor parses the hour with the SAME parsers stage 2 uses
PRE->>AR: finishClarified → applyAction
Note right of AR: filling in an argument never grants authority
AR->>AR: router.ResolvedTheHour guard
AR->>DB: CreateReminder fire_ts, payload
DB-->>AR: id
AR-->>H: reminderConfirm, phrased FROM THE ROW not the utterance
H-->>V: "хорошо, напомню сегодня в 19:00."
Note over T,D: later — the proactive half, no shared code with the turn path
loop every 60s
T->>DB: Gatherer.GatherState, due reminders, collapsed by group
T->>T: loop.RemindDecisions — reminders BYPASS the restraint gate
alt not cached
T->>T: phraser.PhraseReminder
end
T->>D: DispatchReminder
D->>DB: BeginDeliveryAttempt BEFORE the external send
alt a voice session is live
D->>V: voicesink push on the request conn
else away
D->>D: ntfy is nil (disabled) → telegram
end
alt success
D->>DB: CompleteSuccessfulReminderAttempt + fire the originals, one txn
else failure
D->>DB: advance the persisted bounded backoff, next_attempt_ts
end
end
Note over T,DB: Recurring is NOT on this path. reminders.cron and next_fire_ts<br/>exist since migration #2 and no spoken path writes them.
@@ -0,0 +1,61 @@
%% View 3b — Runtime flow: a factual / state update.
%% Traced through cmd/mavend/actions_fact.go, cmd/mavend/ack.go,
%% cmd/mavend/patterns.go, cmd/mavend/factenrichment.go, cmd/mavend/intake.go
%% and internal/morning.
%% Shows the two re-route branches this handler owns, the vector prune-and-insert,
%% the nudge it can close, and the async entity resolution behind it.
sequenceDiagram
autonumber
participant K as Owner
participant H as runTurn
participant R as router cascade
participant AF as actionFact
participant API as CoreAPI · intakeAPI then storeAPI
participant DB as facts table
participant VEC as memory_vectors
participant BUS as event.Bus
participant FE as fact-enrichment worker
participant NX as Nexus
participant T as tick loop
Note over K,H: "выпил воды"
K->>H: utterance, src=tap:voice
H->>R: rt.resolve
R->>R: stage 0 declines → heads → LLM router → classifier
R-->>H: IntentFact, Slots.Key="water", Slots.Value=...
rect rgb(70,40,40)
Note over AF: two guards that RE-ROUTE rather than write
AF->>AF: router.IsQuestionShaped? → becomes actionQuery, Key cleared
AF->>AF: router.IsTransientComplaint? → becomes actionChat, nothing stored
end
AF->>AF: factConfidence — 1.0 only for a value he actually said
AF->>API: WriteFact kind=self, source=tap:voice, Subject=Key
API->>DB: append-only row
API->>BUS: publish one intake envelope
API-->>AF: factID
AF->>VEC: pruneFactVectors by key
AF->>VEC: EmbedPassage(FactRecallText) then Insert "fact:<key>:<unix>"
Note right of VEC: the FACT is embedded, not the utterance.<br/>The utterance rides along as provenance only
AF->>DB: RecordEvent action+object, for pattern detection
H->>H: step 9b ackFromFact — a fact answering a live nudge closes it as `acted`, silently
par asynchronous, minutes later
FE->>DB: read facts with resolution_state='pending'
FE->>NX: Resolve(Subject)
alt resolved
NX-->>FE: entity_id
FE->>DB: UPDATE entity_id, resolution_state='resolved'
else ambiguous
Note right of FE: candidates are NOT stored —<br/>ambiguity blocks, it does not pick
end
and the next tick
T->>DB: Gatherer reads the same row
T->>T: morning routine item `water` is now evidenced, so it will not nudge
T->>T: detectPatterns scans events for a stable interval
end
Note over DB,VEC: A wrong value is superseded, never overwritten:<br/>voids_id points at the row it cancels, and CorrectValue /<br/>VoidLatestFact drop the key's vectors so recall keeps exactly one.
@@ -0,0 +1,70 @@
%% View 3c — Runtime flow: a world query, tool-backed.
%% Traced through internal/router/worldquery.go, internal/router/source.go,
%% cmd/mavend/actions_query.go queryWalk + querySources, cmd/mavend/personalboundary.go,
%% cmd/mavend/searchwire.go, cmd/mavend/kiwixwire.go.
%% Shows destination anchoring, which sources are skipped and why, and the
%% four-step fallback to the model's own weights.
sequenceDiagram
autonumber
participant K as Owner
participant H as runTurn
participant R as router cascade
participant QC as actionQuery
participant W as queryWalk
participant LOC as local sources
participant PB as personal boundary
participant SX as SearXNG
participant KX as kiwix-server
participant PH as phraser / resident model
participant REC as decision record
Note over K,H: "что такое TCP?"
K->>H: utterance
H->>R: rt.resolve
R->>R: stage 0 — WorldQueryGrammars matches a literal definition frame
R->>R: d.SourceAnchored = true, set HERE and nowhere else
R-->>H: IntentQuery, Source=SourceWorld, anchored
H->>QC: applyAction → actionQuery
QC->>REC: Expect the full 22-source roster
QC->>W: queryWalk(SourceWorld, anchored=true)
rect rgb(70,40,40)
Note over W: removes ONLY sources with guesses:true whose dest ≠ world
W-->>REC: skipped: attention, list, feeds, home, network, weather, self
W-->>REC: skipped: personal boundary — anchored, so a literal pattern may drop it
Note right of W: a model or a softmax naming SourceWorld<br/>would NOT drop the boundary (V-666)
end
W-->>QC: the sources that LOOK still walk, in table order
loop first source to claim answers the turn
QC->>LOC: fact-by-key, day-plan, habits, tasks, money, history, calendar
LOC-->>QC: no rows → pass
QC->>LOC: embed → memory → notes (vector recall, gated by min score + margin)
LOC-->>QC: below the gate → pass
QC->>PB: personal boundary
PB-->>QC: SKIPPED this turn
QC->>SX: Search(utterance verbatim, max 4)
alt results
SX-->>QC: snippets
QC->>PH: phraseSource("search", utterance, evidence)
PH-->>QC: reply
QC->>REC: claimed by "search", and everyone below is NeverAsked
else empty or unreachable
QC->>KX: ZIM search, ru then en
alt hit
KX-->>QC: article snippet
QC->>PH: phraseSource("kiwix", ...)
else miss
QC->>QC: "web" claims only if he named a URL out loud
QC->>PH: queryGeneral — the model answers from its own weights, LAST
end
end
end
QC-->>H: reply text
H-->>K: spoken or written answer
Note over LOC,SX: What leaves the box is the query string and nothing else.<br/>His notes, his facts, the persona block and the history never travel.
Note over W,PB: With no destination named — the classifier arm sets none —<br/>the whole chain walks in table order. That is the floor.
@@ -0,0 +1,115 @@
%% View 4 — State ownership.
%% Every persistent and shared store, its authoritative owner, its writers and
%% readers, its synchronisation boundary and its lifecycle.
%% Red = written by components that do not know about each other.
%% Evidence: internal/store/schema.sql, internal/store/migrations.go,
%% internal/store/crypt.go, cmd/mavend/voice.go, cmd/mavend/tick.go,
%% cmd/mavweb/*.go, cmd/mavpoll/main.go, cmd/mavcaldav/main.go.
flowchart LR
subgraph OWNER["authoritative owner — mavend, the only key holder"]
STORE[("store.Store<br/>SetMaxOpenConns(1)<br/>every write serialised at the db")]
end
subgraph LIFE["lifecycle of the database itself"]
direction TB
ENC[("maven.db.enc<br/>AES-256-GCM at rest<br/>volume dbdata")]
TMP[("/dev/shm/maven-plain.db<br/>tmpfs working copy<br/>dies with the container")]
ENC -->|"Open: decrypt"| TMP
TMP -->|"Close: checkpoint, re-encrypt, atomic rename"| ENC
SEAL["mavseal<br/>recovery only, VACUUM INTO"]
TMP -.->|"when mavend was killed, not stopped"| SEAL
SEAL -.-> ENC
end
STORE --- TMP
%% ------------- multiply written tables
FACTS[("facts<br/>append-only, ts = valid-time<br/>correction sets voids_id")]:::multi
NOTES[("notes<br/>float32 blob, brute-force scan")]:::multi
TOOLS[("tools<br/>only status='enabled' executes")]:::multi
%% ------------- singly owned tables
REM[("reminders")]
NUD[("nudges — the restraint memory<br/>AND the only feedback input")]
VEC[("memory_vectors<br/>marked with the embedder id")]
PRES[("presence_state — singleton row")]
EV[("events")]
PROP[("proposed_routines")]
DIG[("digest_entries — gate-BLOCKED candidates")]
DEL[("delivery_attempts — the outbox")]
ACK[("ack_sends")]
DLGS[("dialogue_sessions — TTL 2m")]
TASKS[("tasks")]
LISTS[("list_items")]
RTR[("routing_traces — 14-day retention")]
RLB[("routing_labels")]
ETR[("ecosystem_traces")]
META[("meta — schema version + embedder marker")]
STORE --- FACTS & NOTES & TOOLS & REM & NUD & VEC & PRES & EV & PROP & DIG & DEL & ACK & DLGS & TASKS & LISTS & RTR & RLB & ETR & META
%% ------------- writers into facts
WF1["actionFact — tap:voice / tap:text"] --> FACTS
WF2["quiet toggle — config fact"] --> FACTS
WF3["mavpoll — poll:netdata, poll:uptimekuma,<br/>infer:wg, poll:zenmoney"] --> FACTS
WF4["mavcaldav — poll:caldav<br/>NOT DEPLOYED"]:::off -.-> FACTS
WF5["mavweb — /api/signal presence,<br/>/api/ambient meeting time"] --> FACTS
WF6["feed + crawl watermarks<br/>crawl:hash:*"] --> FACTS
WF7["fact-enrichment worker<br/>entity_id, resolution_state"] --> FACTS
WF8["tick loop tune()<br/>cooldown:<rule> feedback fact"] --> FACTS
WF9["mavweb /api/revert<br/>voids the latest fact for a key"] --> FACTS
%% ------------- writers into notes
WN1["actionNote"] --> NOTES
WN2["RSS poller — source rss:*"] --> NOTES
WN3["crawl watcher"] --> NOTES
WN4["meeting capture"]:::off -.-> NOTES
WN5["image description"]:::off -.-> NOTES
WN6["netscan record"] --> NOTES
%% ------------- writers into tools
WT1["seedTools from mavend.json"] --> TOOLS
WT2["MCP discovery — proposed"]:::off -.-> TOOLS
WT3["Home Assistant discovery<br/>proposed, always destructive"]:::off -.-> TOOLS
WT4["mavweb POST /tools<br/>the ONLY enable path"] --> TOOLS
%% ------------- readers
FACTS --> RD1["loop.Gatherer — the tick snapshot"]
FACTS --> RD2["queryFactByKey · money · history · morning"]
NOTES --> RD3["queryNotes · queryFeeds · recall"]
VEC --> RD4["queryMemory · queryEmbed"]
TOOLS --> RD5["tool.Matcher + tool.Executor"]
NUD --> RD6["restraint gate · TuneCooldown · UnackedTelegramRules"]
%% ------------- in-memory shared state
subgraph MEM["shared mutable state — process-local, no synchronisation boundary beyond a mutex"]
direction TB
CLS["clarifyStore<br/>per-reach stack · NOT persisted on purpose:<br/>a restart expires the open question"]
PEND["pending act / pendingRoutine / pendingHexis<br/>3 single-slot registers under handler.mu<br/>last-asked wins, TTL each"]
SURF["surfacedItems<br/>replaced by the next digest, NO TTL"]
RING["decision.Ring — bounded, diagnosis only"]
BUS["event.Bus — bounded journal, read surface only"]
TICKM["tickLoop: lastPhrase, lastTrace, digestQ,<br/>routineLast, morningLast, lastProposalAt"]
LASTR["lastRouted — the previous acted turn, for a spoken correction"]
end
H1["reactiveHandler<br/>one instance, called from per-conn goroutines"] --- CLS
H1 --- PEND
H1 --- SURF
H1 --- RING
H1 --- LASTR
TICKL["tickLoop"] --- TICKM
INTAKE["intakeAPI decorator"] --- BUS
%% ------------- outside the database
subgraph OUT["state outside the database"]
direction TB
PK[("passkeys.json<br/>OWNED BY mavweb, not mavend")]:::multi
WK[("wrapped key blob<br/>written by mavend WrapKeyFn,<br/>triggered by mavweb")]:::multi
MAIL[("mavmaild seen-UID file<br/>own volume · NOT DEPLOYED")]:::off
BLOB[("media blobs · retention loop")]:::off
end
PK -.->|"a v1 blob + this file together<br/>recover the database key with no authenticator"| WK
classDef multi fill:#4f2626,stroke:#e08080,color:#ffecec
classDef off fill:#3a3a3a,stroke:#888,color:#ccc,stroke-dasharray:4 3
@@ -0,0 +1,141 @@
%% View 5 — Dependency and boundary map.
%% Architectural components, not classes. Highlights the cycle, the cross-layer
%% calls, the duplicated responsibilities, the fan-in and fan-out hotspots, the
%% process and IPC boundaries, and where a failure propagates.
%% Evidence: cmd/mavend/boot.go, cmd/mavend/tick_api.go, cmd/mavend/voice.go,
%% cmd/mavend/voicewire.go, internal/ipc/server.go, internal/delivery/channel.go.
flowchart TB
subgraph B1["process boundary — mavend"]
direction TB
subgraph L_EDGE["entry layer"]
IPCS["ipc.Server<br/>fan-in: 6 processes<br/>+ 8 bypass function fields"]
VSRV["voice.Server"]
HTTPIN["telegram poller"]
end
subgraph L_API["API layer"]
DAPI["daemonAPI<br/>store adapter + 8 closures"]
IAPI["intakeAPI decorator"]
SAPI["ipc.NewStoreAPI"]
end
subgraph L_TURN["turn layer"]
RH["reactiveHandler<br/>GOD COMPONENT<br/>34 fields · fan-out ≈ 20"]
TRT["turnRoute"]
PRE["pre-route ladder · 11 rungs"]
ATBL["actionHandlers · 7"]
QCH["querySources · 22"]
end
subgraph L_ROUTE["routing layer"]
RTR["router.Router cascade"]
G0["stage 0 grammars · 22+"]
HDS["routing heads"]
LLMR["LLM router"]
CLF["classifier"]
end
subgraph L_PROACT["proactive layer"]
TICK["tickLoop<br/>13 jobs, one function<br/>fan-out ≈ 10"]
GATH["loop.Gatherer"]
RULES["loop rules + gate · pure"]
DISP["delivery.Dispatcher"]
end
subgraph L_WIRE["construction layer"]
WIRE["wireVoice<br/>builds 17 subsystems<br/>returns voiceWiring"]
BOOT["boot.go<br/>newDaemonAPI + startBackground"]
end
subgraph L_STATE["state layer"]
ST[("store.Store")]
end
end
subgraph B2["process boundary — modules"]
MSTT["mavsttd"]
MTTS["mavttsd"]
MWEB["mavweb"]
MPOLL["mavpoll"]
end
subgraph B3["process boundary — workstation"]
MWAKE["mavwaked"]
MGPU["mavgpud"]
end
subgraph B4["external services"]
EXT["SearXNG · kiwix · Nexus · Praxis · Hexis<br/>Telegram · ntfy · Home Assistant"]
end
%% ---------- boundaries
MWEB -.->|"UNIX IPC · 3 conns"| IPCS
MPOLL -.->|"UNIX IPC"| IPCS
MWAKE -.->|"TCP over ssh · plaintext, no auth"| VSRV
MWEB -.->|"TCP · /api/ptt"| VSRV
RH -.->|"UNIX worker"| MSTT
RH -.->|"UNIX worker"| MTTS
RH -.->|"HTTP"| EXT
RH -.->|"HTTP"| MGPU
DISP -.->|"HTTP"| EXT
%% ---------- the cycle
IPCS --> DAPI
DAPI -->|"chatFn = handler.handleText"| RH
RH -->|"h.api, back-patched by upgradeAPI"| DAPI
DAPI --> IAPI --> SAPI --> ST
%% ---------- turn layer
VSRV --> RH
HTTPIN --> DAPI
RH --> TRT --> RTR
RH --> PRE --> TRT
RH --> ATBL --> QCH
QCH --> ST
ATBL --> ST
RTR --> G0 & HDS & LLMR & CLF
%% ---------- cross-layer calls
QCH -->|"CROSS-LAYER: a query source reads the tick loop"| TICK
RH -->|"CROSS-LAYER: dataStore, the raw store beside the CoreAPI"| ST
DAPI -->|"reads tick state"| TICK
WIRE --> RH
WIRE --> RTR
WIRE --> DISP
BOOT --> DAPI
BOOT --> TICK
%% ---------- proactive
TICK --> GATH --> ST
TICK --> RULES
TICK --> DISP
DISP --> ST
DISP -->|"voicesink pushes on the request conn"| VSRV
%% ---------- annotations
DUP1["DUPLICATED RESPONSIBILITY<br/>two independent arbitrations decide a turn:<br/>the 22-grammar cascade, then the 22-source chain.<br/>Both are ordered lists; neither can compare scores."]:::note
DUP1 -.- RTR
DUP1 -.- QCH
DUP2["DUPLICATED RESPONSIBILITY<br/>restraint is decided twice:<br/>loop.Gate says whether a rule EMITS,<br/>delivery.ChannelsFor says where it LANDS.<br/>Deliberate, and documented in channel.go."]:::note
DUP2 -.- RULES
DUP2 -.- DISP
DUP3["DUPLICATED RESPONSIBILITY<br/>three unrelated components propose tool rows:<br/>config seeding, MCP discovery, HA discovery."]:::note
DUP3 -.- ST
FRAG1["FRAGILE PATH<br/>4 seams degrade silently:<br/>workstation model → resident model,<br/>CW2 → mavsttd, heads → LLM → classifier,<br/>search → kiwix → weights.<br/>Nothing on the turn says which one answered."]:::warn
FRAG1 -.- RTR
FRAG1 -.- QCH
FRAG2["FAILURE PROPAGATION<br/>ipc.Server holds long-lived conns from 4 modules.<br/>Before V-638 that deadlocked EVERY shutdown and<br/>the deployed ciphertext went 11 days stale."]:::warn
FRAG2 -.- IPCS
FRAG3["PLANNED, UNWIRED<br/>internal/claim + router.ClaimOf: a comparable<br/>unit of evidence for exactly the two arbitrations above.<br/>Nothing calls it. internal/modes: nothing imports it."]:::warn
FRAG3 -.- RTR
classDef note fill:#2a3f2a,stroke:#7fbf7f,color:#eaffea
classDef warn fill:#4f2626,stroke:#e08080,color:#ffecec
+699
View File
@@ -0,0 +1,699 @@
# Architecture findings: Maven as built
Read at commit `5cae33a`, 2026-08-25. Working tree dirty: `deploy/mavend.json`
swaps `phraser.model_path` to `maven-instruct-b2-Q4_K_XL.gguf`, plus an edited
`docs/evals/CLAUDE.md` and two untracked files.
This file is analysis. The factual inventory is
`docs/architecture/maven-architecture.json` and the diagrams under
`docs/architecture/diagrams/`. Nothing here proposes a new architecture.
**Revised 2026-08-25 after an independent second pass over the evidence pack.**
Four readings changed, and section 6.3 contained one statement that was wrong:
the voice server defaults an empty `Surface`, it does not overwrite the client's.
The sections marked below carry the corrections.
**The ranking changed with them.** The missing end-to-end authority model
(6.3 through 6.3d) is the first architectural issue, ahead of `reactiveHandler`
size (4.1) and the process boundaries (section 5). Those are refactors. This one
is a property nobody can state.
Each finding cites what it was read from. Where the repository already names a
problem in its own comments, that is said. A known defect and an undiscovered
one are different facts.
---
## 1. Unclear ownership
### 1.1 The `facts` table has nine writers and no owner
`internal/store/schema.sql` calls facts "substrate, all observations". Nine
components append to it, and no component owns the key namespace:
| Writer | Source tag | Evidence |
|---|---|---|
| `actionFact` | `tap:voice`, `tap:text` | `cmd/mavend/actions_fact.go` |
| quiet-hours toggle | `config` | `cmd/mavend/quiet_toggle.go` |
| mavpoll | `poll:netdata`, `poll:uptimekuma`, `infer:wg`, `poll:zenmoney` | `cmd/mavpoll/main.go` |
| mavcaldav | `poll:caldav` | `cmd/mavcaldav/main.go`, not deployed |
| mavweb | presence, ambient meeting time | `cmd/mavweb/facts.go`, `cmd/mavweb/ambient.go` |
| feed worker | RSS watermark | `cmd/mavend/feeds.go` |
| crawl worker | `crawl:hash:<name>` | `cmd/mavend/crawls.go` `hashKey` |
| fact-enrichment worker | mutates `entity_id`, `resolution_state` | `cmd/mavend/factenrichment.go` |
| tick loop autotune | `cooldown:<rule>` | `cmd/mavend/tick.go` `tune`, `internal/loop/feedback.go` `FeedbackKey` |
Two of these are not observations at all. `crawl:hash:*` is a fetch watermark
and `cooldown:<rule>` is a tuning parameter. Both live in the same append-only
table that recall embeds and that `queryFactByKey` reads back as an answer. The
`source` column is what keeps them apart, and it is a convention, not a
constraint: `schema.sql` documents the vocabulary in a comment and the `CHECK`
covers only `kind`.
### 1.2 `notes` has six writers and one of them is a LAN scan
`cmd/mavend/netscan.go` `writeScanRecord` writes a scan result as a note. Notes
are the recall corpus: `queryNotes` and `queryMemory` answer from them. So a
network scan record competes by cosine similarity with things he said.
### 1.3 `tools` is proposed by three unrelated components
Config seeding (`seedTools`), MCP discovery (`cmd/mavend/mcp.go` `propose`) and
Home Assistant discovery (`cmd/mavend/smarthome.go` `propose`) all write rows.
Only `mavweb` `POST /tools` can enable one, which is the invariant that holds.
But nothing arbitrates a name collision between the three proposers, and
`tools.name` is the primary key.
### 1.4 The day plan has no store and two owners
`queryDayPlan` is a query source. The day plan it reads is assembled by the tick
loop (`cmd/mavend/tick_morning.go` `dayPlan`). The bare store adapter cannot
answer it, which is why `upgradeAPI` exists at all (finding 3.1). So a read of
his calendar depends on a proactive scheduler being wired.
---
## 2. Duplicated responsibilities
### 2.1 Two independent arbitrations decide one turn
The cascade sorts an utterance into one of seven intents through four arms
(`internal/router/router.go` `Route`). An `IntentQuery` then enters a second
arbitration of twenty-two ordered sources (`cmd/mavend/actions_query.go`
`querySources`, counted in the source). Both are ordered lists. Neither can
compare scores across arms.
The repository states this itself, in `internal/router/source.go`:
> The cascade sorted an utterance into one of seven intents with stage 0 rules,
> the resident model and the classifier behind it, a fixture measuring it and
> the decision trace recording it. Then IntentQuery handed the turn to
> querySources in the daemon, a chain of twenty-two branches deciding by seed
> similarity in a fixed order, with none of that.
`Source` and `queryWalk` narrow the second arbitration with a decision from the
first. They do not merge the two.
### 2.2 A third arbitration runs before both
`runTurn` steps 1 through 5e are eleven stateful pre-emptors, each answering "is
this mine?" alone (`cmd/mavend/voice.go`, `preRouteLadder` in
`cmd/mavend/decisiontrace.go`). Their order is argued rung by rung in comments.
That is three ordered lists deciding one utterance, in three files, with three
different notions of confidence.
`internal/claim/claim.go` names exactly this and counts it:
> Maven's cascade has twenty-two stage-0 grammars, seven router intents,
> twenty-two query sources and seven stateful pre-emptors, and every one of them
> answers "is this mine?" alone. None can answer "is this more mine than
> yours?" … So list order is the whole arbitration.
The unit that would fix it is written, tested and called by nothing. See 6.1.
### 2.3 Restraint is decided twice, deliberately
`internal/loop/loop.go` `Gate` decides whether a rule emits.
`internal/delivery/channel.go` `ChannelsFor` decides where it lands, and drops
care nudges on away for its own reasons. `channel.go` argues the duplication:
> double authority is intentional: the gate decides whether a rule EMITS;
> delivery decides where it LANDS.
Recorded here as duplication that is owned, not as a defect.
### 2.4 Two digest mechanisms with the same word in the name
`tickLoop.digestQ` is an in-memory queue batching candidates the gate **allowed**.
`digest_entries` is a table durably holding candidates the gate **blocked**. Both
are flushed in the same `tick()` body, six lines apart
(`cmd/mavend/tick_digest.go`). The distinction is carried entirely by a comment.
---
## 3. Accidental coupling
### 3.1 A construction cycle between the API layer and the turn layer
Two back-patches, each documented, together forming a cycle:
- `cmd/mavend/boot.go`: `api.chatFn = d.voiceW.handler.handleText`
- `cmd/mavend/voice.go` `upgradeAPI`: `h.api = api`, the daemon's own CoreAPI
So `daemonAPI` holds the handler and the handler holds `daemonAPI`. The comment
on `upgradeAPI` states the reason and the safety argument:
> Wiring order forces this. wireVoice runs before the tick loop exists … main
> already back-patches the other direction … this is the same seam in reverse.
> Safe against the obvious loop: nothing in the voice path calls api.Chat.
The safety rests on a negative that nothing enforces. Adding a query source that
calls `api.Chat` would recurse.
### 3.2 The handler holds the raw store beside the mediated one
`reactiveHandler` carries both `api ipc.CoreAPI` and
`dataStore *store.Store`, "direct store access for event extraction + pattern
detection" (`cmd/mavend/voice.go`). `internal/ipc/frame.go` states the opposing
rule for the boundary:
> Core mediates, never hands back a db handle … Anything needing raw db access
> lives in core and is unreachable.
That holds across the process boundary and not inside it. The turn path has two
ways to reach the same tables, with different auditing.
### 3.3 The intake journal is bypassed by the one path that needed it
`cmd/mavend/intake.go` decorates `CoreAPI` so every intake write narrates
itself, and names its own exception:
> The exception is cmd/mavend/mail.go, which reaches past the interface to
> st.CaptureTask directly. It publishes explicitly.
One caller reaching past a decorator means the decorator is not the boundary it
claims to be.
### 3.4 A query source reads the proactive scheduler
`queryDayPlan``tickLoop.dayPlan`. The reactive and proactive halves otherwise
share only the store. This is the single call across that line, and it is the
reason for the `upgradeAPI` back-patch in 3.1.
---
## 4. God components
### 4.1 `reactiveHandler` has 34 fields
`cmd/mavend/voice.go:75`. One struct holds stt, tts, the router, the CoreAPI,
the raw store, the tool executor and matcher, the phraser, the replier, the
recall wiring, the crawler, the search client, the Kiwix client, the feeds flag,
the Home Assistant wiring, the LAN scanner, the weather provider and its default
location, the time parser, the dialogue session store, the decision ring, the
trace writer, the encoder id, the clarify store and its attempt cap, the
extractor, a mutex, `lastRouted`, three pending-confirmation registers,
`surfacedItems`, and the ecosystem clients.
`docs/handler-wiring.md` exists because grouping five of these into `recall`
was itself a task (Vikunja #433).
Every query source, every action handler and every pre-route resolver is a
method on this one type. There is no seam between "the thing that routes a
turn" and "the thing that knows the house is a Home Assistant".
### 4.2 `runTurn` is one function with eleven early returns
`cmd/mavend/voice.go:270`, about 226 lines. Two deferred finalisers, six numbered
steps with lettered sub-steps up to `5e`, and an explicit statement that the
ordering is load-bearing. Eleven of the returns are `return withNotice(...)`
from a pre-emptor.
### 4.3 `tick` runs thirteen jobs in one function
`cmd/mavend/tick.go:160`. Gather, save presence, pick a candidate, queue or
phrase-and-dispatch, flush the digest, enqueue gate-suppressed candidates,
expire stale digest, drain digest, fire routines, fire accepted routines, fire
morning routines, detect patterns, deliver reminders, repeat un-acked sev4
alarms. One 60s ticker drives all of it, so a slow phraser call delays every job
after it.
### 4.4 `wireVoice` is one constructor for seventeen subsystems
`cmd/mavend/voicewire.go:108`, about 270 lines, returning a `voiceWiring` struct
whose fields the rest of the daemon reaches into (`embedderOf`, `nexusOf`,
`d.voiceW.mcp`, `d.voiceW.home`, `d.voiceW.server`, `d.voiceW.handler`).
---
## 5. Process boundaries
### 5.1 Unnecessary: `mavsttd` and `mavttsd` at current scale
Both are justified in their own headers as "restart-free, key-free,
fail-independent". Both run in the same container image, on the same host, as
the same user, over a socket in a shared volume, and both are hard dependencies
of a turn: `HandlePushToTalk` returns an error reply when either is unavailable.
The key argument is real but partial. `internal/ipc/frame.go` says "a crashing
tts can't read the key page", and the same holds for any goroutine that never
touches the key.
The boundary earns itself for a different reason the docs do not lead with:
whisper.cpp and piper are cgo and subprocess dependencies, so an in-process
crash would be a daemon crash. Recorded as a boundary whose stated reason and
real reason differ.
### 5.2 Unnecessary: three IPC connections from one process
`cmd/mavweb/main.go` opens `core`, `swapConn` and `turnConn` to the same socket,
because `ipc.Client` serialises every call on one mutex and a model swap or a
chat turn would otherwise freeze every page. The comments say so. Connection
count is standing in for request concurrency.
### 5.3 Missing: the turn path and the tick loop are one process
They share `store.Store` at `SetMaxOpenConns(1)`, one `phraser.Phraser` and one
`llm.Gate`. A reminder being phrased and a spoken turn being answered contend
for the same llama-server through `internal/llm/gate.go`. Nothing isolates a
foreground turn from a background job beyond that gate.
### 5.4 Missing: the act executor runs in the key holder
`internal/tool/tool.go:238` is `exec.CommandContext(ctx, argv[0], argv[1:]...)`,
running inside mavend, the only process holding the database key.
`deploy/mavend.json` seeds twelve rows, five of them destructive, including
`systemctl restart`, `docker restart` and `systemctl reboot`.
The controls are the enabled allowlist, the risk tier (6.3b) and the confirm
turn. The process boundary is not one of them. `internal/tool/risk.go:84` says
so directly: "It is not a sandbox and it does not try to be one. An enabled row
can already run anything the daemon's user can run."
### 5.5 The one boundary that is load-bearing and undefended by itself
The voice TCP wire is plaintext with no auth (`internal/voice/server.go`). Its
security argument is entirely external: loopback publish plus an ssh tunnel
(`docker-compose.yml` `ports: ["127.0.0.1:9110:9100"]`,
`deploy/mavwaked.service` `Requires=maven-voice-tunnel.service`). Correct, and
it means a single compose edit silently removes the whole control.
---
## 6. Implementation disagreeing with apparent responsibility
### 6.1 `internal/claim` and `router.ClaimOf` are called by nothing
`internal/router/claim.go` says so in its own doc comment:
> Nothing in Route calls this yet. The arbiter that reads claims is V-560.
V-560 landed as `turnRoute` (memoise the route), not as an arbiter. The package
and its `router` adapter are complete and tested and are on no path.
### 6.2 `internal/modes` is imported by nothing outside itself
`grep -rn "internal/modes"` over `cmd/` and `internal/` returns only its own
test. It describes itself as "the roughly thirty distinct downstream behaviours
mavend has". That is an inventory of the very thing findings 2.1 and 2.2 are
about.
### 6.3 The auth tier system does not bind the turn path
`internal/auth/tier.go` documents "voice can never reach EnableTool, not
because we check the method, but because the surface can't carry the layer",
and `MaxLayer(SurfaceVoice)` returns `Layer0`. `cmd/mavwaked/main.go:275` duly
sends `Surface: voice.SurfaceVoice` on the wire.
Nothing in `cmd/mavend` reads it. `grep -rn "internal/auth" cmd/ internal/`
outside tests returns `cmd/mavend/main.go` (building the IPC `Gate`),
`cmd/mavweb/webauthn.go`, `internal/webauthn/session.go` and
`internal/voice/wire.go` (type aliases only). `actionAct`
(`cmd/mavend/actions_act.go`) contains no surface check.
**Two representations of reach exist, and both are ignored.** An earlier draft
of this file said the server overwrites the client's value. It does not.
1. **Client-asserted, and it survives.** `internal/voice/server.go:198` reads
`if p.Surface == "" { p.Surface = SurfacePCClient }`. That defaults an empty
field. `mavwaked`'s `SurfaceVoice` arrives intact and reaches
`HandlePushToTalk`, which ignores it (`cmd/mavend/voice.go:200`, the
parameter is `req` and only `req.Audio` is read).
2. **Server-created, and it is wrong.** `internal/voice/server.go:148` is
`sess := s.sessions.Add(c, SurfacePCClient)`, hardcoded for every connection
whatever the peer is. Nothing reads that either.
The consequence matters more than the finding. `req.Surface` is request payload
on a plaintext wire with no auth, so **any voice-wire client can claim
`"pc_client"`**. It must not become an authorization input as it stands. A reach
has to be derived from the transport or the session, never trusted from the
body.
`auth.Can` runs only in `ipc.Server.Check`, and `FloorEnrollment` maps every
same-uid caller there to `SurfaceCoreProcess` / `Layer3`
(`internal/auth/enrollment.go:65`).
The comments in `cmd/mavwaked/main.go`, `deploy/mavwaked.service` and
`CLAUDE.md` all present "SurfaceVoice caps acts at L0" as a live control. On the
reactive turn path, `internal/auth` is not what enforces it. Finding 6.3b is.
### 6.3b There is a second tier system, it is live, and it is not keyed on the reach
`internal/tool/risk.go` carries its own two-axis policy, and this one runs on
every act:
```go
policy := PolicyFor(RiskOf(t)) // internal/tool/tool.go:181
if !policy.VoiceMayRun { return "", ErrNeedsAuthedSurface }
if policy.Confirm && !confirmed { return "", ErrNeedsConfirm }
```
`RiskOf` sorts a row into `TierSafe`, `TierDestructive` or `TierIrreversible`.
`PolicyFor` maps those to `{Confirm:false, VoiceMayRun:true}`,
`{Confirm:true, VoiceMayRun:true}` and `{Confirm:true, VoiceMayRun:false}`
(`internal/tool/risk.go:69`). So the control that actually stops an act is real,
well argued, and fails safe on an unknown shape.
Two observations about it:
1. **`VoiceMayRun` is not conditioned on voice.** `Executor.Exec` takes
`(ctx, name, args, confirmed)` and no surface. The same policy is applied to
the mic, to telegram inbound and to `POST /api/chat` on the authed page. A
field named for a reach is evaluated identically for every reach.
2. **`systemctl reboot` is `TierDestructive`, not `TierIrreversible`.**
`irreversibleVerbs` (`internal/tool/risk.go:88`) lists `rm`, `mkfs`, `dd`,
`prune`, `truncate` and eleven more. `reboot` is not among them, and
`deploy/mavend.json` seeds it as an enabled row with `destructive: true`. So
it runs on the reactive path after one spoken "да", which is exactly what
`PolicyFor(TierDestructive)` says and is worth stating out loud.
So the repository has two tier systems: `Surface × Layer` in `internal/auth`,
unread on the turn path, and `Risk × Policy` in `internal/tool`, live.
They are **not two implementations of one idea**, which is how an earlier draft
of this file read. They are two orthogonal dimensions that never meet. `auth`
answers who or where may carry what authority. `tool` answers what effect a
capability has and what proof it demands. The decision that combines them does
not exist anywhere.
That both dimensions are also thin today makes the gap easier to see:
- `FloorEnrollment.Lookup` maps **every** same-uid IPC caller to
`SurfaceCoreProcess` (`internal/auth/enrollment.go:65`), so the process-radius
distinction behind IPC is a future contract, not a live one.
- `PasskeySession` is one global timestamp. `CurrentLayer` and `Assert` both
take a `Scope` and both ignore it (`internal/webauthn/session.go:38` and
`:62`), so step-up is per-daemon rather than per-scope.
### 6.3c The act policy is more distributed than one gate
`RiskOf → PolicyFor → Executor.Exec` is one of three act paths, not the act
path.
| path | risk policy? | evidence |
|---|---|---|
| local tool row | yes | `internal/tool/tool.go:181` |
| Hexis capability | yes, explicitly reused | `cmd/mavend/ecosystem_acts.go:768` `tool.RiskOfCapability` then `tool.PolicyFor` |
| Praxis lifecycle | **no** | `cmd/mavend/ecosystem_acts.go:158` `praxisItemAction.handle` calls `a.call(ctx, px, id)` directly |
Acknowledge, resolve, ignore and pin are remote mutations that run on first
hearing, with no tier and no confirm turn. They are reversible on the Praxis
side, which is a reason, and it is a reason nothing in the code states.
`Exec` also has no proof that its `confirmed bool` was bound correctly. The
invariant that a confirmation names one capability, one target and an expiry
lives in `pendingAct` and `resolveConfirm` (`cmd/mavend/confirm.go`), not at the
boundary that acts on it. `Exec` trusts the boolean because only two callers
exist today.
So the authorization function is spread across origin handling, routing, parked
confirm state, risk classification, allowlist state and execution. Section
"The authorization function as implemented" in `README.md` writes down the part
that is one expression. The rest is not.
### 6.3d `Claim.Coverage` returns 1.0 for a claim that extracted nothing
`ClaimOf` builds its consumed span from `claimSpans`, which includes
`d.Slots.Text` unconditionally (`internal/router/claim.go:38`).
`Router.fillSlots` backfills the raw utterance into `Text` for a note, a query
and a chat turn (`internal/router/router.go:334`, `if d.Slots.Text == "" &&
d.Intent != IntentReminder`).
`claim.Split` then marks every token of the utterance explained, and
`Coverage()` is `len(Consumed) / total` (`internal/claim/claim.go:122`). A query
claim that extracted nothing scores 1.0, and `MoreSpecificThan` reads coverage
first.
`filledSlots` in the same file already knows about this: it counts `Text` "only
when it differs from the whole utterance". `claimSpans`, four functions above
it, does not.
`internal/router/claim_test.go` does not catch it. All five cases in
`TestClaimOfBands` set `Text` equal to `Utterance`, and the test asserts `Band`
only. Coverage is never asserted anywhere.
This is why `internal/claim` is not yet an answer to "what competes for a turn".
It is the beginning of a vocabulary. It also has no production callers, no
builders for query sources or pre-route claimants, and it identifies only the
seven-intent destination rather than the roughly thirty behaviours
`internal/modes` enumerates. Keeping it unwired is the right state until that is
resolved, and the file's own comment already warns against it becoming a fourth
arbitration layer.
### 6.4 Two query sources do not do what their names say
Two of the twenty-two "query sources" have side effects or read a different
substrate than their name implies. `queryNetwork` triggers a live LAN scan
inside a read path (`cmd/mavend/netscan.go` `scanSummary`), and the scan writes
a note.
### 6.5 `actionFact` answers queries and chat
`cmd/mavend/actions_fact.go` re-routes a question-shaped utterance into
`actionQuery` and a complaint into `actionChat`. Both re-routes are argued and
correct in effect. The consequence is that the fact handler is one of three
entry points into the query chain.
### 6.6 `mavgpud`'s model arm is off and its STT arm is on
`deploy/mavend.json` sets `workstation.model_disabled: true` while
`workstation.stt` is live. One config block, two independently authenticated
services, one flag that turns off half of it. The block's own comment explains
this. A reader of the topology would not guess it.
---
## 7. Hidden shared state
### 7.1 Six context keys carry per-turn state
`querySourceKey`, `turnRouteKey`, `dialogueKey`, `ecosystemCorrelationKey`,
`traceIDKey` (all `cmd/mavend/`), and `recorderKey`
(`internal/decision/decision.go`). Plus `callerKey` in `internal/ipc/api.go`.
Every one is invisible in a function signature. `turnRouteFrom` returns nil
"when the caller is not inside runTurn, a unit test calling one resolver
directly, most often". That is the shape of the problem: a resolver behaves
differently depending on invisible context.
### 7.2 Three single-slot confirmation registers under one mutex
`reactiveHandler.pending`, `pendingRoutine`, `pendingHexis`
(`cmd/mavend/voice.go:170-180`). The comment states the posture: "single slot,
single-user box, a second act while one waits overwrites it (last-asked wins)".
Three separate registers, one shared mutex, and the pre-route ladder decides
between them by position rather than by comparing them.
### 7.3 `surfacedItems` has no TTL
Same struct. The comment argues it: a stale position resolves to an item Praxis
reports as already acknowledged, "which is a harmless answer, unlike a stale
confirmation". That is correct given Praxis is the arbiter. It also means an
ordinal can refer to a list read out an arbitrarily long time ago.
### 7.4 The tick loop's memory is in-process and unbounded in one place
`tickLoop.lastPhrase` is a `map[string]delivery.PhrasedNudge` keyed by rule
name, and rules are a fixed set, so it is bounded. `digestQ` is a slice with a
config `MaxItems`. `lastProposalAt` is deliberately not persisted: "a restart is
allowed to permit one more announcement".
### 7.5 The clarify store is deliberately not persisted, while the dialogue store is
`cmd/mavend/voicewire.go`: `dialogue.NewPersistentSessionStore` for follow-up
slots, `dialogue.NewClarifyStore` for the parked question. The reasoning is
recorded (Vikunja #385). The consequence is that a restart mid-clarify silently
drops a request the user believes is parked, and the "expired clarify notice"
path in `runTurn` step 1 cannot fire for it, because the store it reads is gone
too.
---
## 8. Fragile request paths
### 8.1 Four silent degradations stacked on one turn
| Seam | Falls back to | Told to the user? |
|---|---|---|
| workstation model → resident model | `internal/llm/remote.go` `Pair.Complete` | no, by design (`docs/offload.md`) |
| CW2 → mavsttd | `cmd/mavend/voicewire.go` `sttSeam` | no |
| routing heads → LLM router → classifier | `internal/router/router.go` | no |
| search → kiwix → named page → model weights | `cmd/mavend/actions_query.go` | no |
Each is individually argued. Together, a single answer can be the resident model
routing a worse transcript with the classifier as a floor and answering from its
own weights, and nothing in the reply distinguishes that from the best case. The
only instrument is the decision record and the query-source log line.
### 8.2 The reminder path depends on a table nobody writes
`queryCalendar` reads `facts(kind=env, source=caldav:*)`, and `mavcaldav` is
commented out in `docker-compose.yml`. `loop.State.CalendarBusy` reads the same
facts, so the "do not nag mid-meeting" suppressor is permanently false. The
compose comment says both of these explicitly, which makes it a known gap rather
than a hidden one.
### 8.3 Recurring reminders have storage, an IPC parameter, and no caller
`reminders.cron` and `reminders.next_fire_ts` exist since migration #2
(`internal/store/migrations.go`). `ipc.CreateReminder` takes a cron argument.
`actionReminder` passes `""`. Nothing on the spoken path can create one.
### 8.4 Shutdown is a known past failure with a bounded workaround
`cmd/mavend/main.go` carries the history: long-lived module connections
deadlocked every shutdown, `run()` never returned, `defer st.Close()` never
sealed, and "the deployed ciphertext was eleven days stale before anyone
noticed". The fix is `workerGrace = 4 * time.Second` plus tracked connections.
A worker parked in a model call still loses its tick, and the seal proceeds
without it.
### 8.5 One inbound worker is outside the assertable worker set
`backgroundWorkers` in `cmd/mavend/boot.go` exists so "a test can compare the
set the two paths would start without standing a daemon up". `wireTelegramIntake`
starts its poller with `wg.Add(1)` and a bare goroutine
(`cmd/mavend/telegramintake.go:41`), so it is not in that set. It is at least on
the outer `WaitGroup`, unlike the seven workers V-639 fixed.
### 8.6 The daemon is wired twice, in two places
`run()` wires everything at boot. `srv.UnlockFn` wires everything again after a
passkey assertion. `boot.go` exists because those two lists had already drifted:
"seven workers started untracked on the unlock path and two daemonAPI fields
were never set there, silently". Both paths now funnel through `newDaemonAPI`
and `startBackground`. But `wireRules`, `wireGatherer`, `wirePhraser`,
`wireEcosystem`, `wireVoice`, `wireDispatcher`, `wireTickLoop`, the four worker
constructors, `wireMailIntake`, `wireModelSwap`, `wireTelegramIntake`,
`wireVision`, `wireCapture` and `wireSpeaker` are still listed twice, by hand,
in the same file.
---
## 9. Difficult-to-test boundaries
### 9.1 A resolver's behaviour depends on invisible context
See 7.1. `turnRouteFrom(ctx)` returning nil is the documented test case, and it
changes what the resolver does.
### 9.2 The single-instance handler is the unit under test for ~60 behaviours
Twenty-two query sources, seven action handlers, eleven pre-route resolvers and
the recall gate are all methods on `*reactiveHandler`. Testing one requires
constructing a struct with 34 fields, most of them nil.
### 9.3 The static gates pass against a baseline, and the baseline records the debt
`scripts/analyzers/deadcode.baseline` accepts thirteen unreachable symbols,
eleven of them from the 2026-08-10 audit (V-686), with three marked as
"must stay". `make audit` is a git-grep inventory and is explicitly not a
reachability check (`CLAUDE.md`).
### 9.4 Measurement needs weights that are not in the tree
`make t` self-skips the four `TestONNX*` measurements without `MAVEN_ONNX_LIB`,
and still prints `ok` (`CLAUDE.md`). The routing heads, the embedder, silero and
the keyword head are all ONNX files under `models/`, bind-mounted from
`/mnt/hdd1/llms` in the case of the gguf. A checkout alone cannot reproduce a
routing measurement.
### 9.5 Only 5 of 51 spec entries cite a scenario that exists
Recorded in the previous session's handoff, from `docs/spec.md` and
`cmd/mavend/testdata/scenarios/`. Not re-verified here.
---
## 10. Excessive fan-in and fan-out
**Fan-in.** `ipc.Server` is reached by six processes (mavweb ×3 connections,
mavpoll, mavcaldav, mavmaild, mavupdate, e2eprobe) and carries eight function
fields that bypass `CoreAPI` entirely: `StepUp`, `UnlockFn`, `WrapKeyFn`,
`IngestMailFn`, `SwapModelFn`, `ModelStatusFn`, `DescribeImageFn` and the four
`Capture*` fields. Each is nil unless its config block exists, so the wire
surface of the daemon depends on `deploy/mavend.json`.
**Fan-out.** `reactiveHandler` reaches roughly twenty distinct subsystems
(4.1). `tickLoop` reaches ten (4.3). `wireVoice` constructs seventeen (4.4).
**Failure propagation.** The store is the shared point: `SetMaxOpenConns(1)`
means every writer in the daemon and every module over IPC serialises through
one connection. The measurement backing that cap is
`docs/evals/2026-08-07-store-connection-cap.md` (V-642), cited in
`internal/ipc/server.go` and not re-run here.
---
## 11. What is dark, and what that costs
Sixteen components are wired in code and off in the deployed configuration:
`ntfy`, `zenmoney`, Home Assistant, MCP, the weather provider, the workstation
model arm, vision, meeting capture, speaker identification, mail intake, model
swap, memory evaluation, and the `mavcaldav` and `mavmaild` services.
Three of these have a visible cost:
1. **ntfy disabled** means the away reach is telegram alone, through a SOCKS
relay, through `api.telegram.org`. `deploy/mavend.json` documents that this
was exactly the fragility ntfy was added to remove: "three things in series
that have each failed once, and when they do a sev4 nudge has nowhere to go."
2. **mavcaldav absent** disables both the calendar answer and the busy
suppressor (8.2).
3. **The weather provider is a stub.** `wireVoice` selects Open-Meteo only when
`cfg.Voice.Weather.Provider == "open-meteo"`, and the deployed `voice` block
has no `weather` key at all. `weather` is nevertheless a live query source with
`guesses: true`, so it can claim a turn and answer it from a stub.
---
# Questions the current architecture raises
1. **Which of the three ordered lists is the arbiter?** Stage 0 grammars, the
query-source chain and the pre-route ladder each decide by position. If
`internal/claim` is the answer, what stops it being a fourth list rather than
the thing that collapses the other three?
2. **Where is the one point that decides whether this authenticated origin may
perform this specific effect using this specific evidence?** Today there is
no such point. `reboot` shows why the question is not "which tier system
wins": it is correctly classified as not irreversible, and that does not
imply a room microphone plus "да" should carry reboot authority.
Reversibility, effect severity, reach authority and confirmation strength are
four dimensions, and `TierDestructive → VoiceMayRun:true` collapses them into
one.
3. **What owns the `facts` key namespace?** Nine writers, two of which store
watermarks and tuning parameters in the table that recall embeds. Is `source`
meant to be a partition, and if so what enforces it?
4. **Should the executor live in the key holder?** `systemctl reboot` is a
seeded, enabled row in a process holding the unlocked database. The controls
are an allowlist and a spoken confirm. Is that the intended trust boundary,
or the one that happened?
5. **Should the tick loop and the turn path share one llama-server?**
`internal/llm/gate.go` exists to arbitrate them. What is the acceptable
latency a foreground turn may pay for a background nudge being phrased?
6. **Is a silent four-level degradation still honest?** Each fallback is argued
separately. Nothing tells the user when all four fire at once. The M1 honesty
milestone in `docs/roadmap.md` is about the turn path. Does it cover this?
7. **What is `docker-compose.yml` the source of truth for?** Two complete
services are commented out in it with their reasoning, and one of them
silently disables two behaviours elsewhere. Should absence be expressible in
`deploy/mavend.json` where the rest of the capability switches live?
8. **Why is the daemon wired twice?** `boot.go` fixed the drift that had already
happened. Fifteen `wire*` calls are still listed by hand on both paths. Is
cold-start unlock worth a second wiring path, or should the locked daemon
wire everything and gate at the `Check` hook alone?
9. **What is a query source allowed to do?** One triggers a live LAN scan and
writes a note. If a source may have side effects, what does "first source to
claim answers the turn" guarantee about the sources that ran before it?
10. **Is `mavsttd`/`mavttsd`'s process boundary about the key or about cgo?**
The stated reason is key isolation. The operative reason looks like crash
isolation from cgo and subprocesses. Which one governs whether the next
model caller gets its own process?
File diff suppressed because it is too large Load Diff
+139
View File
@@ -0,0 +1,139 @@
#!/usr/bin/env bash
# Build maven-evidence.zip: the architecture package plus the source seams a
# reviewer needs to test its claims, and nothing else.
#
# sh docs/architecture/pack_evidence.sh
#
# Three rules this script exists to enforce:
#
# 1. Whole files, never snippets. A cut-down file loses the call path that
# makes a claim checkable, which is the whole point of sending source.
# 2. Allowlist, not denylist. Paths are named one by one below. A denylist
# ships whatever nobody thought to exclude, and this tree has a database
# key in it.
# 3. Refuse rather than warn. The scan at the end aborts on a hit instead of
# printing something a tired person scrolls past.
#
# The one file that is not verbatim is docker-compose.yml. It carries a live
# uptime-kuma API key, so a redacted copy goes in its place and the redaction is
# recorded in architecture-evidence.txt and printed here.
set -euo pipefail
here=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
root=$(CDPATH= cd -- "$here/../.." && pwd)
cd "$root"
out=maven-evidence.zip
stage=$(mktemp -d)
trap 'rm -rf "$stage"' EXIT
echo "== regenerating the architecture package"
python3 "$here/build_inventory.py"
python3 "$here/verify_anchors.py" # exits 1 if any claim no longer resolves
python3 "$here/build_evidence.py"
python3 "$here/build_viewer.py"
echo "== structural context"
# `tree` here is an eza alias in the owner's shell and absent in a plain sh, so
# the listing is generated with find and does not depend on either.
{
echo "# find -L internal cmd -maxdepth 3 -type d"
echo
find internal cmd -maxdepth 3 -type d | sort
echo
echo "# go files per package"
echo
find internal cmd -name '*.go' ! -name '*_test.go' | sed 's|/[^/]*$||' | sort | uniq -c | sort -rn
echo
echo "# test files per package"
echo
find internal cmd -name '*_test.go' | sed 's|/[^/]*$||' | sort | uniq -c | sort -rn
} > "$here/tree.txt"
echo "== redacting the one credential in docker-compose.yml"
sed 's/"uk5_[^"]*"/"<REDACTED: uptime-kuma api key>"/' docker-compose.yml \
> "$here/docker-compose.redacted.yml"
if grep -q 'uk5_' "$here/docker-compose.redacted.yml"; then
echo "pack_evidence.sh: redaction failed, refusing to build" >&2; exit 1
fi
diff <(sed 's/"uk5_[^"]*"/X/' docker-compose.yml) \
<(sed 's/"<REDACTED: uptime-kuma api key>"/X/' "$here/docker-compose.redacted.yml") \
>/dev/null || { echo "pack_evidence.sh: redacted copy differs by more than the key" >&2; exit 1; }
# ---- the allowlist -------------------------------------------------------
# Requested and present. internal/session, internal/db and tests/ are absent
# from this repo; architecture-evidence.txt says where their contents live.
paths=(
docs/architecture
CLAUDE.md
docs/CLAUDE.md
go.mod
deploy/mavend.json # ${VAR} placeholders only; 16 off-claims read it
cmd/mavend
internal/auth
internal/tool
internal/claim
internal/modes
internal/router
internal/voice
internal/ipc # the boundary auth.Can actually runs on
internal/store
internal/dialogue # clarify + session state the turn path parks in
internal/decision # the arbitration record
internal/delivery/channel.go
internal/loop
internal/webauthn # the other half of the auth story
cmd/mavwaked/main.go # the client that sends Surface
cmd/mavweb/main.go # the six unguarded surfaces
)
echo "== staging"
for p in "${paths[@]}"; do
if [ ! -e "$p" ]; then echo " MISSING $p (skipped)"; continue; fi
mkdir -p "$stage/$(dirname "$p")"
cp -r "$p" "$stage/$(dirname "$p")/"
done
# Generated-in-place files that must not travel, and anything that is a secret,
# a model, a database or a build artefact regardless of how it got staged.
#
# The name filters skip .go on purpose: internal/router/singletoken.go matched
# '*token*' and was deleted out of the first build of this pack. That is exactly
# the silent hole an allowlist exists to prevent, and a Go source file is never
# the thing this clause is for.
find "$stage" ! -name '*.go' \( \
-name '*.db' -o -name '*.sqlite*' -o -name '*.enc' \
-o -name '*.pem' -o -name '*.key' -o -name '*.crt' -o -name '*.p12' \
-o -name '.env*' -o -name '*.token' -o -name '*.secret' -o -name '*.password' \
-o -name '*.onnx' -o -name '*.gguf' -o -name '*.bin' -o -name '*.wav' \
-o -name '*.zip' -o -name '*.log' -o -name '.git' \
\) -print -exec rm -rf {} + 2>/dev/null || true
echo "== scanning the staged tree"
# A value-shaped assignment: a credential word, a delimiter, then twelve or more
# characters of value. The value must NOT begin with a slash or a dot, because a
# docker volume line pairs a host path with a container path and both halves end
# in the same secret-sounding filename while containing no secret. Three of those
# in docker-compose.yml tripped the first version of this scan.
hits=$(grep -rInE '(api[_-]?key|secret|passwo?r?d|bearer|token)["'"'"' ]*[:=]["'"'"' ]*[A-Za-z0-9+_-][A-Za-z0-9/+_-]{11,}' "$stage" \
| grep -vE '\$\{|<REDACTED|example|EXAMPLE|xxx|XXX|your-|changeme' \
| grep -vE '_test\.go|\.md:' \
| grep -vE ':[0-9]+:[[:space:]]*(#|//)' || true)
if [ -n "$hits" ]; then
echo "pack_evidence.sh: possible credentials in the staged tree, refusing to build:" >&2
echo "$hits" >&2
exit 1
fi
echo "== building $out"
rm -f "$out"
( cd "$stage" && zip -qr "$root/$out" . )
echo
printf '%s %s %s files\n' "$out" \
"$(du -h "$out" 2>/dev/null | cut -f1)" \
"$(unzip -l "$out" | tail -1 | awk '{print $2}')"
echo
echo "redacted: docker-compose.yml -> docs/architecture/docker-compose.redacted.yml"
echo " one uptime-kuma api key, nothing else"
echo "excluded: .git, deploy/telegram.env, deploy/db_key.env, models, deps, databases"
+52
View File
@@ -0,0 +1,52 @@
#!/bin/sh
# Re-render every diagram in diagrams/*.mmd to a committed SVG beside it, then
# rebuild index.html so the viewer carries the new pictures.
#
# mermaid-cli drives a real browser through puppeteer. It downloads its own
# chrome-headless-shell by default, which fails behind a proxy and wastes
# 150 MB; PUPPETEER_EXECUTABLE_PATH points it at the system chromium instead.
# --no-sandbox is required because that chromium is not the one puppeteer
# provisioned and has no sandbox helper of its own here.
#
# sh docs/architecture/render.sh
#
# Run it from anywhere. A parse error in one file leaves the others alone and
# prints FAIL with the mermaid error, which is the only way this repo has to
# syntax-check a .mmd.
set -eu
here=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
dia="$here/diagrams"
cfg=$(mktemp)
trap 'rm -f "$cfg"' EXIT
printf '{"args":["--no-sandbox","--disable-gpu"]}' > "$cfg"
: "${PUPPETEER_EXECUTABLE_PATH:=$(command -v chromium || command -v chromium-browser || command -v google-chrome-stable || true)}"
if [ -z "$PUPPETEER_EXECUTABLE_PATH" ]; then
echo "render.sh: no chromium found. Install one, or set PUPPETEER_EXECUTABLE_PATH." >&2
exit 1
fi
export PUPPETEER_EXECUTABLE_PATH
for f in "$dia"/*.mmd; do
n=$(basename "$f" .mmd)
err=$(mktemp)
if npx --yes @mermaid-js/mermaid-cli@11 -p "$cfg" -t dark -b '#0e1116' \
-i "$f" -o "$dia/$n.svg" >/dev/null 2>"$err" && [ -s "$dia/$n.svg" ]; then
echo "OK $n"
else
echo "FAIL $n"
grep -m1 -A3 'Parse error' "$err" || tail -3 "$err"
fi
rm -f "$err"
done
python3 "$here/build_viewer.py"
# The only check the viewer has. A TypeError in a renderer shows as a blank
# panel, not as an error, so run every view against a DOM stub before shipping.
if command -v node >/dev/null 2>&1; then
node "$here/check_viewer.js" || exit 1
else
echo "SKIP check_viewer.js: no node"
fi
+155
View File
@@ -0,0 +1,155 @@
#!/usr/bin/env python3
"""Resolve every claim in maven-architecture.json to a file and a line.
The inventory names files and symbols. A reader has to take on trust that the
symbol is in the file and that the file still exists. This script removes the
trust: it looks up every symbol in the component's own files and writes
anchors.md, a table of component, symbol, path:line and the verbatim line.
Exit code is 1 when anything fails to resolve, so it doubles as a staleness
gate. A symbol that moved to another file, or a file that was deleted, fails
here rather than in a reader's head.
python3 docs/architecture/verify_anchors.py # write anchors.md
python3 docs/architecture/verify_anchors.py --quiet # gate only
What it deliberately does NOT check: that the symbol means what the
responsibility says it means. That is the human pass this file exists to make
cheap.
"""
import json
import os
import re
import sys
HERE = os.path.dirname(os.path.abspath(__file__))
ROOT = os.path.abspath(os.path.join(HERE, "..", ".."))
# Symbols the inventory names that are not Go identifiers in this repo: config
# keys, make targets, flags, wire strings, table names. Looking them up in a .go
# file would fail for the wrong reason, so they are resolved against the file
# they belong to when possible and skipped when not.
NON_GO = re.compile(r"^(make |-|/|\$)|\.(json|sql|service|yml)$| ")
def candidates(sym: str):
"""Search forms for one symbol, longest first.
A dotted symbol like `Store.WriteFact` or `voice.NewServer` is written as a
method or a qualified call, so the tail is what appears in a definition and
the whole string is what appears at a call site. Try both.
"""
forms = [sym]
if "." in sym:
forms.append(sym.split(".")[-1])
return forms
def find(paths, sym):
for form in candidates(sym):
needle = re.compile(r"\b" + re.escape(form) + r"\b")
for rel in paths:
full = os.path.join(ROOT, rel)
if not os.path.isfile(full):
continue
try:
lines = open(full, errors="replace").read().splitlines()
except OSError:
continue
# Three passes, best anchor first: a definition, then any code
# line, then a comment. Without the comment pass being last, a
# const whose doc comment names it anchors on the prose rather
# than on the declaration.
for rank in (0, 1, 2):
for i, line in enumerate(lines, 1):
if not needle.search(line):
continue
bare = line.strip()
comment = bare.startswith(("//", "#", "--", "%%", "*"))
isdef = bool(re.match(
r"\s*(func|type|const|var)\b", line)) or bool(re.match(
r"\s*\"?" + re.escape(form) + r"\"?\s*[:=]", line))
got = 2 if comment else (0 if isdef else 1)
if got == rank:
return rel, i, bare
return None, None, None
def expand(rel):
"""A directory in the inventory stands for the files under it."""
full = os.path.join(ROOT, rel)
if os.path.isdir(full):
return sorted(
os.path.join(rel, f) for f in os.listdir(full)
if f.endswith((".go", ".json", ".sql")) and not f.endswith("_test.go")
)
return [rel]
def main() -> int:
quiet = "--quiet" in sys.argv
arch = json.load(open(os.path.join(HERE, "maven-architecture.json")))
rows, missing_files, unresolved = [], [], []
for c in arch["components"]:
paths = []
for f in c["files"]:
if not os.path.exists(os.path.join(ROOT, f)):
missing_files.append((c["id"], f))
continue
paths.extend(expand(f))
for sym in c["symbols"]:
if NON_GO.search(sym):
rows.append((c["id"], sym, "", "", "not a Go identifier, not looked up"))
continue
rel, line, text = find(paths, sym)
if rel is None:
unresolved.append((c["id"], sym))
rows.append((c["id"], sym, "", "", "UNRESOLVED"))
else:
rows.append((c["id"], sym, f"{rel}:{line}", text, ""))
resolved = sum(1 for r in rows if r[2])
if not quiet:
with open(os.path.join(HERE, "anchors.md"), "w") as fh:
fh.write("# Claim anchors\n\n")
fh.write(
"Generated by `docs/architecture/verify_anchors.py`. Every symbol the\n"
"inventory names, resolved to a file and a line in this checkout, with the\n"
"line quoted. Regenerate after any edit to the inventory or the code.\n\n"
)
fh.write(
f"- components: {len(arch['components'])}\n"
f"- symbols claimed: {len(rows)}\n"
f"- resolved to a line: {resolved}\n"
f"- unresolved: {len(unresolved)}\n"
f"- missing files: {len(missing_files)}\n\n"
)
if unresolved:
fh.write("## Unresolved\n\n")
for cid, sym in unresolved:
fh.write(f"- `{cid}` claims `{sym}` and it is in none of its files\n")
fh.write("\n")
if missing_files:
fh.write("## Missing files\n\n")
for cid, f in missing_files:
fh.write(f"- `{cid}` names `{f}`, which does not exist\n")
fh.write("\n")
fh.write("## Anchors\n\n| component | symbol | anchor | line |\n|---|---|---|---|\n")
for cid, sym, anchor, text, note in rows:
shown = (text or note).replace("|", "\\|")
if len(shown) > 120:
shown = shown[:117] + "..."
fh.write(f"| `{cid}` | `{sym}` | {anchor or ''} | `{shown}` |\n")
print(f"symbols {len(rows)}, resolved {resolved}, unresolved {len(unresolved)}, "
f"missing files {len(missing_files)}")
for cid, sym in unresolved[:20]:
print(f" UNRESOLVED {cid} :: {sym}")
for cid, f in missing_files[:20]:
print(f" MISSING {cid} :: {f}")
return 1 if (unresolved or missing_files) else 0
if __name__ == "__main__":
sys.exit(main())
+691
View File
@@ -0,0 +1,691 @@
<!doctype html>
<html lang="en" data-theme="dark">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Maven architecture — as built</title>
<style>
:root{
--bg:#0e1116; --panel:#141922; --panel2:#1a2130; --line:#26304a; --line2:#38456b;
--fg:#dfe6f2; --dim:#8d9bb5; --dim2:#5f6c85;
--proc:#7fb3ff; --procbg:#16263f;
--svc:#8fd0ff; --svcbg:#132433;
--store:#6ed0a8; --storebg:#0f2a20;
--model:#e0b050; --modelbg:#2a2210;
--adapter:#b79bf0; --adapterbg:#221a33;
--ext:#c39bd3; --extbg:#241a2b;
--bnd:#ff9f6b; --bndbg:#2c1c12;
--warn:#e08080; --warnbg:#2e1616;
--ok:#7fbf7f;
}
*{box-sizing:border-box}
html,body{margin:0;height:100%}
body{background:var(--bg);color:var(--fg);font:14px/1.5 ui-sans-serif,-apple-system,"Segoe UI",Roboto,sans-serif;overflow:hidden}
code,.mono{font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace}
#app{display:grid;grid-template-columns:250px 1fr 420px;grid-template-rows:auto 1fr;height:100vh}
header{grid-column:1/-1;display:flex;align-items:center;gap:18px;padding:10px 16px;border-bottom:1px solid var(--line);background:var(--panel)}
header h1{font-size:15px;margin:0;font-weight:650;letter-spacing:.2px}
header .meta{color:var(--dim2);font-size:12px}
header .meta b{color:var(--dim);font-weight:500}
nav{border-right:1px solid var(--line);background:var(--panel);overflow-y:auto;padding:12px 10px}
nav h2{font-size:10px;letter-spacing:.12em;text-transform:uppercase;color:var(--dim2);margin:14px 6px 6px}
nav h2:first-child{margin-top:0}
.viewbtn{display:block;width:100%;text-align:left;background:transparent;border:1px solid transparent;color:var(--dim);
padding:7px 9px;border-radius:6px;cursor:pointer;font:inherit;font-size:13px}
.viewbtn:hover{background:var(--panel2);color:var(--fg)}
.viewbtn.on{background:#1d2c47;border-color:var(--line2);color:#fff}
.viewbtn small{display:block;color:var(--dim2);font-size:11px;line-height:1.35;margin-top:2px}
.toggle{display:flex;align-items:center;gap:8px;padding:5px 7px;color:var(--dim);font-size:12.5px;cursor:pointer;border-radius:5px}
.toggle:hover{background:var(--panel2)}
.toggle input{accent-color:#5b8ff9}
.legend{display:flex;flex-wrap:wrap;gap:5px;padding:4px 6px}
.legend span{font-size:10.5px;padding:2px 6px;border-radius:99px;border:1px solid var(--line2);color:var(--dim)}
/* capability matrix and invariants */
.capsel{display:flex;gap:6px;margin:0 0 14px}
.capsel button{background:var(--panel2);border:1px solid var(--line);color:var(--dim);padding:5px 11px;border-radius:6px;cursor:pointer;font:inherit;font-size:12.5px}
.capsel button.on{background:#1d2c47;border-color:var(--line2);color:#fff}
table.mx{border-collapse:collapse;width:100%;font-size:12.5px}
table.mx th{text-align:left;font-weight:500;color:var(--dim2);font-size:10px;letter-spacing:.1em;text-transform:uppercase;padding:0 6px 7px;vertical-align:bottom}
table.mx th.d{text-align:center;width:64px}
table.mx tr.grp td{padding:16px 6px 5px;color:var(--dim);font-size:10.5px;letter-spacing:.1em;text-transform:uppercase;border-bottom:1px solid var(--line)}
table.mx tbody tr.cap{cursor:pointer}
table.mx tbody tr.cap:hover td{background:var(--panel2)}
table.mx tbody tr.cap.sel td{background:#1b2b45}
table.mx td{padding:4px 6px;border-bottom:1px solid #1b2230}
table.mx td.n{font-weight:600}
table.mx td.n em{font-style:normal;color:var(--dim2);font-weight:400;font-size:11px;margin-left:7px}
table.mx td.d{text-align:center}
.dot{display:inline-block;width:11px;height:11px;border-radius:3px;border:1px solid #0006}
.dot.yes{background:#4f9d69}.dot.partial{background:#c8992e}.dot.no{background:#3a4256}
.dot.speconly{background:#3a4256;border-style:dashed;border-color:#6b7896}
.gapc{font-size:10.5px;color:var(--dim2)}
.gapc.missing{color:#e08080}.gapc.unreachable{color:#e0a060}.gapc.partial{color:#c8992e}
.mark{font-size:10px;letter-spacing:.06em;text-transform:uppercase;padding:2px 7px;border-radius:99px;border:1px solid var(--line2)}
.mark.explicit{color:#7fbf7f;border-color:#3d6b43}
.mark.implied{color:#e0b050;border-color:#6b5a26}
.mark.unresolved{color:#e08080;border-color:#6b3838}
.inv{background:var(--panel2);border:1px solid var(--line);border-radius:9px;padding:12px 14px;margin-bottom:12px}
.inv h4{margin:0 0 6px;font-size:14px;display:flex;align-items:center;gap:10px}
.inv h4 span.n{color:var(--dim2);font-weight:400}
.inv .q{color:#e0b8b8;font-size:12.5px;margin:8px 0 0}
.bars{display:flex;gap:14px;margin:9px 0 4px;flex-wrap:wrap}
.bar{font-size:10.5px;color:var(--dim2)}
.bar b{display:block;font-size:9.5px;letter-spacing:.1em;text-transform:uppercase;color:var(--dim2);font-weight:500;margin-bottom:3px}
.bar .v{color:var(--fg);font-size:12px}
.bar .v.warn{color:#e08080}.bar .v.mid{color:#c8992e}.bar .v.ok{color:#7fbf7f}
.cchip{display:inline-block;background:var(--panel);border:1px solid var(--line);border-radius:6px;padding:3px 8px;margin:3px 4px 0 0;
font-size:11.5px;cursor:pointer;color:var(--dim)}
.cchip:hover{border-color:var(--line2);color:var(--fg)}
.cchip.off{border-color:#6b5a26;color:#e0b050}
.cchip.pl{border-color:#6b3838;color:#e08080}
.crit{border-left:2px solid var(--line2);padding:0 0 0 10px;margin:0 0 11px}
.crit .vd{font-size:10px;letter-spacing:.08em;text-transform:uppercase;margin-right:8px}
.crit .vd.pass{color:#7fbf7f}.crit .vd.fail{color:#e08080}.crit .vd.blocked{color:#e0a060}
.crit .vd.untested{color:var(--dim2)}.crit .vd.unknown{color:#b79bf0}
.crit .rs{color:var(--dim2);font-size:11px}
.crit p{margin:5px 0 0;color:var(--dim);font-size:12px}
main{position:relative;overflow:auto;padding:18px 20px 60px}
.lane{margin-bottom:20px}
.lane-h{display:flex;align-items:baseline;gap:10px;margin:0 0 8px;cursor:pointer;user-select:none}
.lane-h b{font-size:11px;letter-spacing:.1em;text-transform:uppercase;color:var(--dim)}
.lane-h i{font-style:normal;color:var(--dim2);font-size:11px}
.lane-h .caret{color:var(--dim2);font-size:11px;width:10px}
.chips{display:flex;flex-wrap:wrap;gap:8px}
.chip{position:relative;background:var(--panel2);border:1px solid var(--line);border-radius:8px;padding:7px 10px;cursor:pointer;
max-width:280px;transition:border-color .12s,background .12s}
.chip:hover{border-color:var(--line2)}
.chip.sel{border-color:#7fb3ff;background:#1b2b45;box-shadow:0 0 0 1px #7fb3ff44}
.chip.rel{border-color:#4a5f8f}
.chip.dim{opacity:.28}
.chip .nm{font-weight:600;font-size:13px}
.chip .ty{font-size:10.5px;color:var(--dim2);letter-spacing:.04em;text-transform:uppercase}
.chip .badges{display:flex;gap:4px;margin-top:4px;flex-wrap:wrap}
.b{font-size:9.5px;padding:1px 5px;border-radius:99px;border:1px solid currentColor;letter-spacing:.03em}
.b.off{color:#c9a227}.b.nd{color:#c98a27}.b.pl{color:#a07fe0}.b.tmp{color:#8d9bb5}.b.pw{color:#e08080}
.b.lo{color:#e08080}.b.me{color:#c9a227}
.chip[data-t=process]{border-left:3px solid var(--proc)}
.chip[data-t=service],.chip[data-t=worker],.chip[data-t=handler]{border-left:3px solid var(--svc)}
.chip[data-t=arbitration],.chip[data-t=query_source]{border-left:3px solid #ffd479}
.chip[data-t=storage],.chip[data-t=table]{border-left:3px solid var(--store)}
.chip[data-t=model]{border-left:3px solid var(--model)}
.chip[data-t=adapter]{border-left:3px solid var(--adapter)}
.chip[data-t=external]{border-left:3px solid var(--ext)}
.chip[data-t=boundary]{border-left:3px solid var(--bnd)}
.chip[data-t="shared-state"]{border-left:3px solid #ff9ec7}
.chip[data-t=planned]{border-left:3px solid #a07fe0}
.chip[data-t=config],.chip[data-t=test]{border-left:3px solid var(--dim2)}
svg.wires{position:absolute;inset:0;pointer-events:none;overflow:visible}
aside{border-left:1px solid var(--line);background:var(--panel);overflow-y:auto;padding:16px 16px 60px}
aside .empty{color:var(--dim2);font-size:13px;margin-top:30px;line-height:1.7}
aside h3{margin:0 0 2px;font-size:16px}
aside .sub{color:var(--dim2);font-size:11px;text-transform:uppercase;letter-spacing:.08em;margin-bottom:10px}
aside section{margin-top:16px}
aside section > h4{font-size:10px;letter-spacing:.12em;text-transform:uppercase;color:var(--dim2);margin:0 0 6px}
aside p{margin:0 0 8px;color:#c9d3e6}
.note{background:#1c1f14;border-left:2px solid #c9a227;padding:8px 10px;border-radius:0 5px 5px 0;color:#ded6b6;font-size:12.5px}
ul.plain{list-style:none;margin:0;padding:0}
ul.plain li{padding:3px 0;border-bottom:1px solid #1d2433;font-size:12.5px}
ul.plain li:last-child{border-bottom:0}
.rel{display:block;width:100%;text-align:left;background:transparent;border:0;color:#a9c6f5;cursor:pointer;font:inherit;font-size:12.5px;padding:3px 0}
.rel:hover{color:#fff;text-decoration:underline}
.rel .k{display:inline-block;min-width:66px;color:var(--dim2);font-size:10.5px;text-transform:uppercase;letter-spacing:.05em}
.rel .ev{display:block;color:var(--dim2);font-size:11px;margin-left:66px;line-height:1.4}
pre.mm{white-space:pre-wrap;word-break:break-word;background:#0b0e13;border:1px solid var(--line);border-radius:6px;
padding:12px;font-size:11.5px;color:#b8c6de;overflow-x:auto;max-height:none}
.searchbox{width:100%;background:var(--panel2);border:1px solid var(--line);border-radius:6px;color:var(--fg);
padding:7px 9px;font:inherit;font-size:12.5px}
.searchbox:focus{outline:none;border-color:var(--line2)}
.results{margin-top:6px;max-height:280px;overflow:auto}
.results button{display:block;width:100%;text-align:left;background:transparent;border:0;color:var(--dim);
padding:5px 7px;border-radius:5px;cursor:pointer;font:inherit;font-size:12px}
.results button:hover{background:var(--panel2);color:#fff}
.results button em{font-style:normal;color:#ffd479}
.flowsel{display:flex;gap:6px;margin-bottom:14px;flex-wrap:wrap}
.flowsel button{background:var(--panel2);border:1px solid var(--line);color:var(--dim);border-radius:6px;
padding:6px 11px;cursor:pointer;font:inherit;font-size:12.5px}
.flowsel button.on{background:#1d2c47;border-color:var(--line2);color:#fff}
ol.steps{counter-reset:s;list-style:none;margin:0;padding:0;max-width:1000px}
ol.steps li{position:relative;padding:9px 12px 9px 44px;border-left:2px solid var(--line);margin-left:14px}
ol.steps li:before{counter-increment:s;content:counter(s);position:absolute;left:-13px;top:9px;width:24px;height:24px;
border-radius:99px;background:var(--panel2);border:1px solid var(--line2);color:var(--dim);font-size:11px;
display:flex;align-items:center;justify-content:center}
ol.steps li.branch{border-left-color:var(--warn);background:#1e1414}
ol.steps li.branch:before{border-color:var(--warn);color:#e08080}
ol.steps b{color:#fff}
ol.steps .who{display:inline-block;background:#1d2c47;border:1px solid var(--line2);border-radius:4px;
padding:0 6px;font-size:11px;color:#a9c6f5;cursor:pointer;margin-right:8px}
ol.steps .who:hover{color:#fff;border-color:#7fb3ff}
ol.steps .ev{display:block;color:var(--dim2);font-size:11.5px;margin-top:3px}
.viewnote{max-width:1000px;color:var(--dim);font-size:12.5px;background:var(--panel2);border:1px solid var(--line);
border-radius:7px;padding:11px 13px;margin-bottom:18px}
.viewnote b{color:var(--fg)}
.dia{margin-bottom:20px;border:1px solid var(--line);border-radius:8px;background:#0b0e13;overflow:hidden}
.dia-h{display:flex;align-items:center;gap:10px;padding:8px 12px;background:var(--panel2);border-bottom:1px solid var(--line);cursor:pointer;user-select:none}
.dia-h b{font-size:11px;letter-spacing:.1em;text-transform:uppercase;color:var(--dim)}
.dia-h .fn{color:var(--dim2);font-size:11px}
.dia-h .zoom{margin-left:auto;display:flex;gap:4px}
.dia-h .zoom button{background:var(--panel);border:1px solid var(--line);color:var(--dim);border-radius:4px;
width:24px;height:22px;cursor:pointer;font:inherit;font-size:12px;line-height:1}
.dia-h .zoom button:hover{color:#fff;border-color:var(--line2)}
.dia-body{overflow:auto;max-height:70vh;padding:10px}
.dia-body > div{transform-origin:0 0}
.dia-body svg{max-width:none;height:auto;display:block}
</style>
</head>
<body>
<div id="app">
<header>
<h1>Maven — architecture as built</h1>
<div class="meta">commit <b id="commit"></b> · <b id="gen"></b> · <b id="counts"></b></div>
<div class="meta" id="dirty"></div>
</header>
<nav>
<h2>Views</h2>
<div id="views"></div>
<h2>Search</h2>
<input class="searchbox" id="q" placeholder="component, file or symbol">
<div class="results" id="results"></div>
<div id="archctl">
<h2>Filters</h2>
<label class="toggle"><input type="checkbox" id="tLow" checked> show low-confidence relations</label>
<label class="toggle"><input type="checkbox" id="tMed" checked> show medium-confidence relations</label>
<label class="toggle"><input type="checkbox" id="tOff" checked> show configured-off</label>
<label class="toggle"><input type="checkbox" id="tUndeployed" checked> show built-not-deployed</label>
<label class="toggle"><input type="checkbox" id="tPlanned" checked> show planned / unwired</label>
<label class="toggle"><input type="checkbox" id="tWires" checked> draw relation wires</label>
<h2>Type</h2>
<div class="legend" id="legend"></div>
</div>
</nav>
<main id="main"></main>
<aside id="side"><div class="empty">Select a component to see its responsibility, the files and symbols it was read from, and every relation in and out.<br><br>Every claim here cites a file. Nothing is inferred from a directory name.</div></aside>
</div>
<script>
/*__DATA__*/
const byId = Object.fromEntries(ARCH.components.map(c => [c.id, c]));
const S = { view: 'v1', sel: null, selCap: null, capBy: 'section', flow: 'reminder', collapsed: {}, diaClosed: false, diaZoom: 1 };
/* ---------------- view definitions ---------------- */
const VIEWS = [
{ id:'v1', name:'1 · System topology', hint:'Processes and external systems, with process boundaries drawn.',
note:'<b>mavend is the centre because the code makes it one.</b> It is the only holder of the database key, it owns the store, the IPC socket, the voice TCP listener, the tick loop, eight background workers and the child llama-server. Every other daemon is key-free and fail-independent. Five services run under docker-compose; two more are built and commented out; two run under systemd on the workstation.',
lanes:[
['homesrv — docker compose', c => c.group==='homesrv' && ['process','model','boundary','external'].includes(c.type)],
['workpc — systemd user units', c => c.group==='workpc'],
['ecosystem network', c => c.group==='ecosystem'],
['internet / LAN', c => ['internet','lan'].includes(c.group)],
['dev and recovery binaries', c => c.group==='dev' || ['proc.mavseal','proc.mavupdate'].includes(c.id)],
['configuration boundary', c => c.type==='config'],
]},
{ id:'v2', name:'2 · Core internals', hint:'The real path through mavend, in the order runTurn runs it.',
note:'The implementation does <b>not</b> follow input → routing → intent → state → tools → response. Eleven stateful pre-emptors get first refusal <b>before</b> routing; the routing cascade is four arms deep; and an <code>IntentQuery</code> then enters a <b>second</b> arbitration of twenty-two ordered sources. The proactive half shares no code with any of it.',
lanes:[
['input and entry', c => ['core.voice_server','core.ipc_server','core.daemon_api','core.intake_api','core.store_api','core.stt_seam','core.telegram_intake','core.auth_gate','core.daemon_lock'].includes(c.id)],
['turn pipeline', c => ['core.reactive_handler','core.turn_route','core.preroute','core.action_table'].includes(c.id)],
['routing cascade', c => c.id.startsWith('router.')],
['intent handlers', c => c.type==='handler' || c.id==='core.query_chain'],
['query sources — the second arbitration, in table order', c => c.type==='query_source'],
['response generation', c => ['core.replier','core.phraser','core.tts_seam','core.model_seam','core.recall','core.topics'].includes(c.id)],
['proactive half', c => ['core.tick_loop','core.gatherer','core.rules','core.pattern','core.morning','core.routines','core.dispatcher','core.sink_voice','core.sink_ntfy','core.sink_telegram'].includes(c.id)],
['background workers', c => c.type==='worker' && c.id!=='core.tick_loop' && c.id!=='core.telegram_intake'],
['dark capabilities — wired, no config block', c => ['core.vision','core.capture','core.speaker','core.mail_intake','core.modelswap','core.netscan','core.memory_eval'].includes(c.id)],
['construction and diagnosis', c => ['core.wiring','core.decision_trace','core.event_bus'].includes(c.id)],
]},
{ id:'v3', name:'3 · Runtime flow', hint:'Three representative requests traced through real code.', flow:true },
{ id:'v4', name:'4 · State ownership', hint:'Every persistent and shared store, its owner, writers and readers.',
note:'One process owns the database and every write is serialised at it: <code>SetMaxOpenConns(1)</code>. Three tables are nevertheless written by components that do not know about each other — <b>facts</b> by nine, <b>notes</b> by six, <b>tools</b> by four. Two files live outside the database entirely, and together they weaken the at-rest key.',
lanes:[
['authoritative owner', c => ['proc.mavend','state.db'].includes(c.id)],
['database lifecycle', c => ['state.db_file','state.db_tmpfs','state.wrapped_key','proc.mavseal'].includes(c.id)],
['tables written by unrelated components', c => ['state.facts','state.notes','state.tools'].includes(c.id)],
['singly-owned tables', c => c.type==='table' && !['state.facts','state.notes','state.tools'].includes(c.id)],
['shared mutable state — process-local', c => c.type==='shared-state'],
['state outside the database', c => ['state.passkey_file','state.maildata','state.media_blobs'].includes(c.id)],
['writers', c => (c.writes||[]).length>0 && c.type!=='table'],
['readers', c => (c.reads||[]).length>0 && c.type!=='table' && !(c.writes||[]).length],
]},
{ id:'v5', name:'5 · Dependency and boundary map', hint:'Components, not classes. Cycles, cross-layer calls, fan-in and fan-out.',
note:'The one <b>cycle</b> is deliberate and documented at both ends: <code>daemonAPI.chatFn = handler.handleText</code> and <code>handler.api</code> back-patched by <code>upgradeAPI</code>. The <b>fan-in</b> hotspot is <code>ipc.Server</code>, reached by six processes and carrying eight function fields that bypass CoreAPI entirely. The <b>fan-out</b> hotspots are <code>reactiveHandler</code> (34 fields) and <code>tickLoop</code> (thirteen jobs in one function).',
lanes:[
['process and network boundaries', c => c.type==='boundary'],
['entry layer', c => ['core.ipc_server','core.voice_server','core.telegram_intake','core.auth_gate'].includes(c.id)],
['API layer', c => c.type==='adapter'],
['turn layer — fan-out hotspot', c => ['core.reactive_handler','core.turn_route','core.preroute','core.action_table','core.query_chain'].includes(c.id)],
['routing layer', c => c.id.startsWith('router.')],
['proactive layer — fan-out hotspot', c => ['core.tick_loop','core.gatherer','core.rules','core.dispatcher'].includes(c.id)],
['construction layer', c => ['core.wiring'].includes(c.id)],
['state layer', c => ['state.db'].includes(c.id) || c.type==='shared-state'],
['evaluation and gates', c => c.type==='test'],
]},
{ id:'c1', name:'6 · Capabilities', hint:'Every capability against the seven dimensions. Sorted by the spec, or by domain.', caps:true,
note:'<b>Nothing here is asserted.</b> The six build dimensions come from the <code>status</code> field of every component the capability maps to, and <code>verified</code> comes from a probe run against the deployed stack. A capability can be coded and unwired, wired and unconfigured, or configured and undeployed, and those are three different pieces of work — which is why this is not one <code>implemented</code> column. Source: <code>docs/capabilities/ledger.yaml</code>.' },
{ id:'c2', name:'7 · Invariants', hint:'The twelve cross-cutting rules, and which components participate in each.', inv:true,
note:'These rules run across all 51 capabilities and no capability\'s definition of done states any of them, so breaking one breaks many at once without producing a single failing criterion. <b>Target</b> is whether the rule is written down. <b>Implementation</b> is the status of the components that participate. <b>Runtime</b> is what the probe run observed for the capabilities it touches. Source: <code>docs/capabilities/invariants.yaml</code>, prose and evidence in <code>invariants.md</code>.' },
];
/* ---------------- runtime flows ---------------- */
const FLOWS = {
reminder: { name:'A reminder request', file:'03a-flow-reminder.mmd', steps:[
{who:['proc.mavwaked'], t:'The keyword head scores the utterance at or above 0.999 and silero VAD closes it. One clean blob ships over the ssh tunnel.', ev:'cmd/mavwaked/wakeword.go, deploy/mavwaked.service'},
{who:['core.voice_server','core.reactive_handler'], t:'The conn already has a Session; HandlePushToTalk transcribes and enters runTurn.', ev:'internal/voice/server.go, cmd/mavend/voice.go'},
{who:['core.decision_trace'], t:'A decision record is installed on the context before anything can claim the turn, so the mic, telegram and the web leave the same trail.', ev:'cmd/mavend/voice.go step 0'},
{who:['core.preroute'], t:'Eleven rungs get first refusal. None claims "напомни позвонить маме".', ev:'cmd/mavend/voice.go steps 1 to 5e, preRouteLadder'},
{who:['router.stage0','router.cascade'], t:'ReminderGrammar matches at stage 0 and wins outright at confidence 1.0. The extractor then fills the slots the grammar did not match.', ev:'internal/router/stagezero.go, router.go fillMatchedSlots'},
{who:['core.reactive_handler'], t:'BRANCH — the route is confident and incomplete. missingFor names `time`, so step 8 fires even though dec.Clarify is false.', ev:'cmd/mavend/voice.go step 8, Vikunja #557', branch:true},
{who:['state.clarify_store'], t:'The request is parked as a PendingQuestion and she asks one question about one thing. The store is in memory on purpose: a restart expires it.', ev:'internal/dialogue/clarify.go, cmd/mavend/clarify.go'},
{who:['core.preroute'], t:'The next utterance is claimed by rung 4, resolveClarifyAnswer, and parsed with the same parsers stage 2 uses.', ev:'cmd/mavend/clarify.go finishClarified'},
{who:['core.action_reminder'], t:'ResolvedTheHour guards a time the parser did not really read. The row is written, and the confirmation is phrased FROM THE ROW, never from the utterance.', ev:'cmd/mavend/actions_reminder.go, Vikunja #507'},
{who:['state.reminders'], t:'One append. cron and next_fire_ts exist as columns and this path never sets them.', ev:'internal/store/reminders.go, migrations.go #2', branch:true},
{who:['core.tick_loop','core.gatherer'], t:'Later, on a 60s ticker: the gatherer collapses due reminders by delivery group and RemindDecisions bypasses the restraint gate.', ev:'internal/loop/gather.go collapseReminders, loop.go RemindDecisions'},
{who:['core.dispatcher','state.delivery_attempts'], t:'The outbox records intent BEFORE the external send, so a crash leaves a pending row rather than silence.', ev:'internal/delivery/dispatcher.go beginReminderOutbox'},
{who:['core.sink_voice','core.sink_telegram'], t:'Voice when a session is live; away, ntfy is nil because the config disables it, so telegram carries it. A definite failure advances the persisted bounded backoff.', ev:'internal/delivery/channel.go ChannelsFor, cmd/mavend/main.go wireNtfySink'},
]},
fact: { name:'A factual / state update', file:'03b-flow-fact.mmd', steps:[
{who:['core.reactive_handler','router.cascade'], t:'"выпил воды" reaches the cascade. Stage 0 declines, the heads or the LLM router or the classifier names IntentFact with a key and a value.', ev:'internal/router/router.go Route'},
{who:['core.action_fact'], t:'BRANCH — a question-shaped utterance is never a fact. It is re-routed into actionQuery with the model-guessed key cleared, and the stage 0 world destination reconstructed so the boundary cannot claim it.', ev:'cmd/mavend/actions_fact.go, Vikunja #470', branch:true},
{who:['core.action_fact','core.action_chat'], t:'BRANCH — a passing complaint is not a fact either. It becomes chat and stores nothing, because recall reads a self row back later as if it were still true.', ev:'cmd/mavend/actions_fact.go, Vikunja #481', branch:true},
{who:['core.intake_api','state.facts'], t:'WriteFact appends kind=self, source=tap:voice, Subject=Key. Confidence is 1.0 only for a value he actually said. The decorator publishes one intake envelope.', ev:'cmd/mavend/actions_fact.go factConfidence, cmd/mavend/intake.go'},
{who:['state.memory_vectors'], t:'The keys old vectors are pruned, then the FACT text is embedded with the passage prefix and inserted. The utterance rides along as provenance and is never embedded.', ev:'cmd/mavend/actions_fact.go pruneFactVectors, Vikunja #493'},
{who:['core.reactive_handler'], t:'Step 9b: a fact that answers a live nudge closes it as `acted`, silently. The fact reply stands.', ev:'cmd/mavend/ack.go ackFromFact'},
{who:['core.fact_enrichment','ext.nexus'], t:'Asynchronously, the enrichment worker resolves Subject to a canonical entity id with per-fact backoff. An ambiguous result is NOT stored.', ev:'cmd/mavend/factenrichment.go resolveOne'},
{who:['core.tick_loop','core.morning'], t:'On the next tick the morning routine sees the item evidenced inside its window and will not nudge for it.', ev:'cmd/mavend/tick_morning.go gatherMorningFacts'},
{who:['core.pattern','state.events'], t:'detectPatterns scans every action+object pair for a stable interval and may propose a routine. notify is false in the deployed config, so it proposes silently.', ev:'cmd/mavend/tick_routines.go, deploy/mavend.json pattern_proposals'},
{who:['state.facts'], t:'A wrong value is superseded, never overwritten: voids_id points at the row it cancels, and both correction paths drop the keys vectors.', ev:'internal/store/schema.sql, internal/store/facts.go'},
]},
world: { name:'A world query, tool-backed', file:'03c-flow-world-query.mmd', steps:[
{who:['router.stage0'], t:'"что такое TCP?" matches WorldQueryGrammars, a literal definition frame. That match sets Source=SourceWorld AND SourceAnchored, which happens here and nowhere else in the cascade.', ev:'internal/router/router.go d.SourceAnchored = d.Source != SourceUnknown'},
{who:['core.query_chain','core.decision_trace'], t:'actionQuery declares the full 22-source roster to the record, so a reader can tell "looked and passed" from "never asked".', ev:'cmd/mavend/actions_query.go decision.Expect'},
{who:['core.query_chain'], t:'queryWalk removes only the sources marked guesses:true whose destination is not world. Sources that LOOK are all still asked, because a named destination is evidence and not a promise.', ev:'cmd/mavend/actions_query.go queryWalk'},
{who:['core.q.personal'], t:'BRANCH — the personal boundary is dropped only because a literal pattern named the destination. A model or a softmax naming SourceWorld would NOT drop it.', ev:'cmd/mavend/actions_query.go queryWalk anchored, V-666', branch:true},
{who:['core.q.factbykey','core.q.memory','core.q.notes'], t:'His own data still gets its turn: fact-by-key, the day plan, tasks, money, history, the calendar, then the three recall passes gated by min score 0.80 and min margin 0.008.', ev:'cmd/mavend/actions_query.go querySources, deploy/mavend.json'},
{who:['core.q.search','ext.searxng'], t:'SearXNG is asked verbatim, with no rewriter. Only the query string leaves the box: no note, no fact, no persona block, no history.', ev:'cmd/mavend/actions_query.go querySearch'},
{who:['core.phraser'], t:'The snippets are handed over as evidence for the question, trimmed under one budget, and phrased. With no phraser the best snippet is read back rather than pretending the search did not happen.', ev:'cmd/mavend/actions_query.go phraseSource, readBack'},
{who:['core.q.kiwix','ext.kiwix'], t:'BRANCH — empty or unreachable falls through to the offline ZIMs, Russian first. No results there is not announced.', ev:'cmd/mavend/actions_query.go queryKiwix', branch:true},
{who:['core.q.web'], t:'A page he named by URL is read only if he actually said a URL, and it sits AFTER the ZIMs on purpose.', ev:'cmd/mavend/actions_query.go queryWeb, Vikunja #259'},
{who:['core.q.general'], t:'Last: the resident model answers from its own weights. Response.Empty() is the whole gate on a world answer; there is no quality threshold in front of it.', ev:'cmd/mavend/actions_query.go queryGeneral, CLAUDE.md'},
{who:['core.query_chain'], t:'Whichever source claimed is logged and noted on the turn sink, so /chat can show it. Everyone below the winner is recorded as NeverAsked.', ev:'cmd/mavend/querysource.go noteQuerySource, Vikunja #474'},
]},
};
/* ---------------- filters ---------------- */
const T = id => document.getElementById(id).checked;
function statusHidden(st){
if (st==='configured-off') return !T('tOff');
if (st==='built-not-deployed') return !T('tUndeployed');
if (st==='planned-unwired'||st==='dead') return !T('tPlanned');
return false;
}
function edgeHidden(e){
if (e.confidence==='low' && !T('tLow')) return true;
if (e.confidence==='medium' && !T('tMed')) return true;
return statusHidden(e.status);
}
const edgesOf = id => ARCH.edges.filter(e => e.from===id || e.to===id);
/* ---------------- rendering ---------------- */
function badges(c){
const out=[];
if (c.status!=='implemented') out.push(`<span class="b ${ {'configured-off':'off','built-not-deployed':'nd','planned-unwired':'pl','temporary':'tmp','partially-wired':'pw','dead':'pl'}[c.status]||'tmp'}">${c.status}</span>`);
if (c.confidence==='low') out.push('<span class="b lo">uncertain</span>');
if (c.confidence==='medium') out.push('<span class="b me">medium confidence</span>');
return out.length?`<div class="badges">${out.join('')}</div>`:'';
}
function chipHTML(c){
return `<div class="chip" data-id="${c.id}" data-t="${c.type}">
<div class="nm">${c.id.split('.').pop().replace(/_/g,' ')}</div>
<div class="ty">${c.type} · ${c.id}</div>${badges(c)}</div>`;
}
function renderView(){
const v = VIEWS.find(x=>x.id===S.view);
const main = document.getElementById('main');
if (v.flow) return renderFlow(main);
if (v.caps) return renderCaps(main);
if (v.inv) return renderInv(main);
let html = v.note ? `<div class="viewnote">${v.note}</div>` : '';
html += diagramPanel(diagramFileFor());
const used = new Set();
v.lanes.forEach(([title, pred], i) => {
const items = ARCH.components.filter(c => !used.has(c.id) && pred(c) && !statusHidden(c.status));
items.forEach(c=>used.add(c.id));
if (!items.length) return;
const key = v.id+':'+i, open = !S.collapsed[key];
html += `<div class="lane"><div class="lane-h" data-lane="${key}">
<span class="caret">${open?'▾':'▸'}</span><b>${title}</b><i>${items.length}</i></div>
<div class="chips" ${open?'':'style="display:none"'}>${items.map(chipHTML).join('')}</div></div>`;
});
html += `<svg class="wires" id="wires"></svg>`;
main.innerHTML = html;
main.querySelectorAll('.lane-h').forEach(h=>h.onclick=()=>{ S.collapsed[h.dataset.lane]=!S.collapsed[h.dataset.lane]; renderView(); paint(); });
main.querySelectorAll('.chip').forEach(ch=>ch.onclick=()=>select(ch.dataset.id));
wireDiagram();
requestAnimationFrame(drawWires);
}
/* ---------------- capabilities and invariants ---------------- */
const DIMS = ['designed','code_present','wired','configured','deployed','reachable','verified'];
const DIMH = {designed:'design',code_present:'code',wired:'wired',configured:'config',
deployed:'deploy',reachable:'reach',verified:'verified'};
const capById = Object.fromEntries(CAPS.capabilities.map(c => [c.id, c]));
// A criterion is blocked when it was observed blocked, or when its reason names
// something outside the code as the thing in the way.
const BLOCKREASON = new Set(['configuration missing','deployment missing',
'external dependency unavailable','scenario missing']);
const dotCls = v => v==='spec-only' ? 'speconly' : v;
// The ledger carries markdown inline code, because docs/spec.md does. Rendering
// it literally puts backticks on screen next to every path.
const md = t => (t||'').replace(/[&<>]/g, m => ({'&':'&amp;','<':'&lt;','>':'&gt;'}[m]))
.replace(/`([^`]+)`/g, '<code>$1</code>')
.replace(/\*\*([^*]+)\*\*/g, '<b>$1</b>');
function capRow(c){
const cls = {'capability missing':'missing','capability exists but unreachable':'unreachable',
'capability partial':'partial'}[c.implementation.gap_class] || '';
const gap = c.implementation.gap_class==='none' ? '' : c.implementation.gap_class;
return `<tr class="cap" data-cap="${c.id}">
<td class="n">${c.title}${c.scope!=='v1'?'<em>deferred</em>':''}
<em class="gapc ${cls}">${gap}</em></td>
${DIMS.map(d=>`<td class="d"><span class="dot ${dotCls(c.implementation[d])}" title="${d}: ${c.implementation[d]}"></span></td>`).join('')}
</tr>`;
}
function renderCaps(main){
const v = VIEWS.find(x=>x.id===S.view);
const by = S.capBy;
const groups = {};
CAPS.capabilities.forEach(c => {
const keys = by==='domain' ? (c.domain.length?c.domain:['unassigned']) : [c.section];
keys.forEach(k => (groups[k] = groups[k]||[]).push(c));
});
const order = by==='domain' ? Object.keys(groups).sort()
: [...new Set(CAPS.capabilities.map(c=>c.section))];
main.innerHTML = `<div class="viewnote">${v.note}</div>
<div class="capsel">
<button data-by="section" class="${by==='section'?'on':''}">by spec section</button>
<button data-by="domain" class="${by==='domain'?'on':''}">by domain</button>
</div>
<table class="mx"><thead><tr><th>capability</th>
${DIMS.map(d=>`<th class="d">${DIMH[d]}</th>`).join('')}</tr></thead>
<tbody>${order.map(g=>`<tr class="grp"><td colspan="8">${g} · ${groups[g].length}</td></tr>`
+ groups[g].map(capRow).join('')).join('')}</tbody></table>`;
main.querySelectorAll('.capsel button').forEach(b=>b.onclick=()=>{S.capBy=b.dataset.by;renderCaps(main);});
main.querySelectorAll('tr.cap').forEach(r=>r.onclick=()=>selectCap(r.dataset.cap));
}
function selectCap(id){
S.sel = null; S.selCap = id;
document.querySelectorAll('tr.cap').forEach(r=>r.classList.toggle('sel', r.dataset.cap===id));
renderCapSide(id);
}
function compChip(cid){
const c = byId[cid];
const k = c && (c.status==='configured-off' ? 'off'
: ['planned-unwired','dead','partially-wired'].includes(c.status) ? 'pl' : '');
return `<button class="cchip ${k}" data-id="${cid}" title="${c?c.status:'unknown'}">${cid}</button>`;
}
function renderCapSide(id){
const c = capById[id], side = document.getElementById('side');
const blockers = c.criteria.filter(cr => cr.verified==='blocked' || BLOCKREASON.has(cr.reason));
const qs = INV.filter(iv => iv.capabilities.includes(id) && iv.question);
const scen = c.scenarios || [];
side.innerHTML = `
<h3>${c.title}</h3>
<div class="sub">${c.section} · ${c.domain.join(', ')} · ${c.scope}${
c.implementation.gap_class==='none'?'':' · '+c.implementation.gap_class}</div>
<p>${md(c.state)}</p>
<section><h4>Implementation</h4>
<div class="bars">${DIMS.map(d=>{
const v = c.implementation[d];
const k = v==='yes'?'ok':v==='partial'?'mid':'warn';
return `<div class="bar"><b>${DIMH[d]}</b><span class="v ${k}">${v}</span></div>`;
}).join('')}</div></section>
<section><h4>Components — ${(c.components||[]).length}</h4>
${(c.components||[]).length ? c.components.map(compChip).join('')
: '<div class="empty" style="margin:0">Nothing carries this capability. That is the finding.</div>'}</section>
<section><h4>Definition of done — ${c.criteria.length}</h4>
${c.criteria.map(cr=>`<div class="crit">
<span class="vd ${cr.verified}">${cr.verified}</span><span class="rs">${cr.reason}</span>
<p>${md(cr.text)}</p>
${cr.detail?`<p style="color:var(--dim2)">${md(cr.detail)}</p>`:''}
${(cr.evidence||[]).length?`<p class="mono" style="font-size:11px;color:var(--dim2)">${cr.evidence.join('<br>')}</p>`:''}
</div>`).join('')}</section>
<section><h4>Blockers — ${blockers.length}</h4>
${blockers.length ? '<ul class="plain">'+blockers.map(b=>`<li>${b.reason} · <span class="mono">${b.id}</span></li>`).join('')+'</ul>'
: '<div class="empty" style="margin:0">none. Nothing outside the code is in the way.</div>'}</section>
<section><h4>Scenarios — ${scen.length}</h4>
${scen.length ? '<ul class="plain">'+scen.map(x=>`<li class="mono">${x.name} ${x.exists?'':'<span class="b off">absent from disk</span>'}</li>`).join('')+'</ul>'
: '<div class="empty" style="margin:0">none named</div>'}</section>
<section><h4>Unresolved product questions — ${qs.length}</h4>
${qs.length ? qs.map(iv=>`<div class="crit"><span class="mark ${iv.mark}">invariant ${iv.id}</span>
<p>${iv.question}</p></div>`).join('')
: '<div class="empty" style="margin:0">none</div>'}</section>`;
side.querySelectorAll('.cchip').forEach(b=>b.onclick=()=>{S.sel=b.dataset.id;renderSide(b.dataset.id);});
}
function invRollup(iv){
const comps = iv.components.map(x=>byId[x]).filter(Boolean);
const bad = comps.filter(c=>c.status!=='implemented'&&c.status!=='temporary').length;
const caps = iv.capabilities.map(x=>capById[x]).filter(Boolean);
const ver = caps.filter(c=>c.implementation.verified==='yes').length;
const part = caps.filter(c=>c.implementation.verified==='partial').length;
return {comps, bad, caps, ver, part};
}
function renderInv(main){
const v = VIEWS.find(x=>x.id===S.view);
main.innerHTML = `<div class="viewnote">${v.note}</div>` + INV.map(iv=>{
const r = invRollup(iv);
const mk = iv.mark==='explicit'?'ok':iv.mark==='implied'?'mid':'warn';
const ik = r.bad?'mid':'ok';
const vk = r.ver===r.caps.length?'ok':(r.ver+r.part)?'mid':'warn';
return `<div class="inv" data-inv="${iv.id}">
<h4><span class="n">${iv.id}</span> ${iv.title}
<span class="mark ${iv.mark}">${iv.mark}${iv.split?' · split':''}</span></h4>
<div class="bars">
<div class="bar"><b>target</b><span class="v ${mk}">${
iv.mark==='explicit'?'written down':iv.mark==='implied'?'not stated':'no answer exists'}</span></div>
<div class="bar"><b>implementation</b><span class="v ${ik}">${r.comps.length} components, ${r.bad} not live</span></div>
<div class="bar"><b>runtime verification</b><span class="v ${vk}">${r.ver} of ${r.caps.length} capabilities verified${r.part?', '+r.part+' partly':''}</span></div>
</div>
${iv.question?`<p class="q">${iv.question}</p>`:''}
<div style="margin-top:8px">${iv.capabilities.map(c=>`<button class="cchip" data-cap="${c}">${c}</button>`).join('')}</div>
<div style="margin-top:4px">${iv.components.map(compChip).join('')}</div>
</div>`;
}).join('');
main.querySelectorAll('.cchip[data-id]').forEach(b=>b.onclick=()=>{S.sel=b.dataset.id;renderSide(b.dataset.id);});
main.querySelectorAll('.cchip[data-cap]').forEach(b=>b.onclick=()=>renderCapSide(b.dataset.cap));
}
function renderFlow(main){
const f = FLOWS[S.flow];
main.innerHTML = `${diagramPanel(FLOWS[S.flow].file)}<div class="viewnote"><b>Three requests, traced through real code.</b> Steps marked in red are branches, fallbacks or refusals the implementation actually takes. Click a component name to open its record. The Mermaid sequence source for each flow is under the panel on the right.</div>
<div class="flowsel">${Object.entries(FLOWS).map(([k,x])=>`<button data-f="${k}" class="${k===S.flow?'on':''}">${x.name}</button>`).join('')}</div>
<ol class="steps">${f.steps.map(s=>`<li class="${s.branch?'branch':''}">
${s.who.map(w=>`<span class="who" data-id="${w}">${byId[w]?byId[w].id:w}</span>`).join('')}
${s.t}<span class="ev">${s.ev}</span></li>`).join('')}</ol>`;
main.querySelectorAll('.flowsel button').forEach(b=>b.onclick=()=>{S.flow=b.dataset.f;renderFlow(main);});
main.querySelectorAll('.who').forEach(b=>b.onclick=()=>select(b.dataset.id));
wireDiagram();
}
function diagramFileFor(){
return {v1:'01-system-topology.mmd',v2:'02-core-internals.mmd',
v4:'04-state-ownership.mmd',v5:'05-dependency-boundary.mmd'}[S.view] || null;
}
// The rendered picture, from the committed SVG beside the .mmd. Absent SVG ⇒
// no panel at all, rather than an empty frame: `sh docs/architecture/render.sh`
// is what fills it, and a missing file means that has not been run.
function diagramPanel(mmFile){
if (!mmFile) return '';
const svg = SVG[mmFile.replace(/\.mmd$/, '.svg')];
if (!svg) return '';
const open = !S.diaClosed;
return `<div class="dia">
<div class="dia-h" id="diaH"><span class="caret">${open?'▾':'▸'}</span><b>Rendered diagram</b>
<span class="fn">diagrams/${mmFile.replace(/\.mmd$/,'.svg')}</span>
<span class="zoom"><button data-z="-1" title="zoom out"></button><button data-z="0" title="fit"></button><button data-z="1" title="zoom in">+</button></span>
</div>
<div class="dia-body" id="diaBody" ${open?'':'style="display:none"'}><div id="diaScale">${svg}</div></div>
</div>`;
}
function wireDiagram(){
const h = document.getElementById('diaH'); if (!h) return;
const body = document.getElementById('diaBody'), scale = document.getElementById('diaScale');
// The mermaid SVG carries width="100%" and a viewBox, so it fills whatever
// box it is given. Widening the wrapper past 100% is the zoom, and the
// .dia-body scrollbar is what makes the extra width reachable. A CSS
// transform would scale the scrollport too and clip the bottom of a tall
// flowchart, which 01-system-topology is at 2304x3542.
const apply = () => { scale.style.width = (S.diaZoom*100)+'%'; };
h.onclick = ev => {
const z = ev.target.closest('button');
if (z){ ev.stopPropagation();
const d = +z.dataset.z;
S.diaZoom = d===0 ? 1 : Math.min(3, Math.max(.25, S.diaZoom + d*0.2));
apply(); return; }
S.diaClosed = !S.diaClosed; renderView(); if (S.sel) select(S.sel);
};
apply();
}
function drawWires(){
const svg = document.getElementById('wires');
if (!svg) return;
svg.innerHTML='';
if (!T('tWires') || !S.sel) return;
const main = document.getElementById('main'), mr = main.getBoundingClientRect();
const pos = id => { const el = main.querySelector(`.chip[data-id="${id}"]`); if(!el) return null;
const r = el.getBoundingClientRect();
return {x:r.left-mr.left+main.scrollLeft+r.width/2, y:r.top-mr.top+main.scrollTop+r.height/2}; };
const a = pos(S.sel); if (!a) return;
edgesOf(S.sel).filter(e=>!edgeHidden(e)).forEach(e=>{
const other = e.from===S.sel ? e.to : e.from, b = pos(other); if (!b) return;
const out = e.from===S.sel;
const col = e.confidence==='low' ? '#e08080' : e.confidence==='medium' ? '#c9a227' : (out?'#7fb3ff':'#6ed0a8');
const mx = (a.x+b.x)/2;
const p = document.createElementNS('http://www.w3.org/2000/svg','path');
p.setAttribute('d',`M${a.x},${a.y} C${mx},${a.y} ${mx},${b.y} ${b.x},${b.y}`);
p.setAttribute('stroke',col); p.setAttribute('stroke-width','1.4'); p.setAttribute('fill','none');
p.setAttribute('opacity','.75');
if (e.status!=='implemented') p.setAttribute('stroke-dasharray','5 4');
svg.appendChild(p);
});
}
function select(id){
S.sel = id;
document.querySelectorAll('.chip').forEach(ch=>{
ch.classList.remove('sel','rel','dim');
if (ch.dataset.id===id) ch.classList.add('sel');
});
const rel = new Set(edgesOf(id).filter(e=>!edgeHidden(e)).map(e=>e.from===id?e.to:e.from));
document.querySelectorAll('.chip').forEach(ch=>{
if (ch.dataset.id!==id) ch.classList.add(rel.has(ch.dataset.id)?'rel':'dim');
});
renderSide(id);
drawWires();
}
function relRow(e, id){
const out = e.from===id, other = out?e.to:e.from, oc = byId[other];
const marks=[];
if (e.confidence!=='high') marks.push(`<span class="b ${e.confidence==='low'?'lo':'me'}">${e.confidence}</span>`);
if (e.status!=='implemented') marks.push(`<span class="b off">${e.status}</span>`);
return `<button class="rel" data-id="${other}"><span class="k">${out?'→':'←'} ${e.kind}</span>${oc?oc.id:other} ${marks.join('')}
<span class="ev">${e.label}${e.evidence?' · '+e.evidence:''}</span></button>`;
}
function renderSide(id){
const c = byId[id], side = document.getElementById('side');
if (!c){ side.innerHTML = `<div class="empty">No record for <code>${id}</code>.</div>`; return; }
const es = edgesOf(id).filter(e=>!edgeHidden(e));
const outE = es.filter(e=>e.from===id), inE = es.filter(e=>e.to===id);
const mmFile = S.view==='v3' ? FLOWS[S.flow].file : {v1:'01-system-topology.mmd',v2:'02-core-internals.mmd',v4:'04-state-ownership.mmd',v5:'05-dependency-boundary.mmd'}[S.view];
side.innerHTML = `
<h3>${c.id}</h3>
<div class="sub">${c.type} · ${c.group} · ${c.status} · ${c.confidence} confidence</div>
<p>${c.responsibility}</p>
${c.notes?`<div class="note">${c.notes}</div>`:''}
<section><h4>Files</h4><ul class="plain">${c.files.map(f=>`<li class="mono">${f}</li>`).join('')}</ul></section>
<section><h4>Symbols</h4><ul class="plain">${c.symbols.map(s=>`<li class="mono">${s}</li>`).join('')}</ul></section>
<section><h4>Outgoing — ${outE.length}</h4>${outE.map(e=>relRow(e,id)).join('')||'<div class="empty" style="margin:0">none</div>'}</section>
<section><h4>Incoming — ${inE.length}</h4>${inE.map(e=>relRow(e,id)).join('')||'<div class="empty" style="margin:0">none</div>'}</section>
${mmFile?`<section><h4>Mermaid source — ${mmFile}</h4><pre class="mm">${MERMAID[mmFile].replace(/[&<>]/g,m=>({'&':'&amp;','<':'&lt;','>':'&gt;'}[m]))}</pre></section>`:''}`;
side.querySelectorAll('.rel').forEach(b=>b.onclick=()=>{
const t=b.dataset.id;
if (!document.querySelector(`.chip[data-id="${t}"]`)) { renderSide(t); S.sel=t; drawWires(); }
else select(t);
});
}
/* ---------------- search ---------------- */
function search(){
const q = document.getElementById('q').value.trim().toLowerCase();
const box = document.getElementById('results');
if (q.length<2){ box.innerHTML=''; return; }
const hits=[];
for (const c of ARCH.components){
const where=[];
if (c.id.toLowerCase().includes(q)) where.push('id');
if (c.responsibility.toLowerCase().includes(q)) where.push('responsibility');
const f = c.files.filter(x=>x.toLowerCase().includes(q));
const s = c.symbols.filter(x=>x.toLowerCase().includes(q));
if (f.length) where.push('file: '+f[0]);
if (s.length) where.push('symbol: '+s[0]);
if ((c.notes||'').toLowerCase().includes(q)) where.push('note');
if (where.length) hits.push([c, where]);
}
box.innerHTML = hits.slice(0,60).map(([c,w])=>`<button data-id="${c.id}">${c.id}<br><em>${w.join(' · ')}</em></button>`).join('')
|| '<button disabled style="color:#5f6c85">no match</button>';
box.querySelectorAll('button[data-id]').forEach(b=>b.onclick=()=>{
const el = document.querySelector(`.chip[data-id="${b.dataset.id}"]`);
if (el){ select(b.dataset.id); el.scrollIntoView({block:'center',behavior:'smooth'}); }
else { S.sel=b.dataset.id; renderSide(b.dataset.id); }
});
}
/* ---------------- boot ---------------- */
document.getElementById('commit').textContent = ARCH.commit.slice(0,7);
document.getElementById('gen').textContent = ARCH.generated;
document.getElementById('counts').textContent = `${ARCH.components.length} components · ${ARCH.edges.length} relations`;
document.getElementById('dirty').textContent = ARCH.working_tree;
document.getElementById('views').innerHTML = VIEWS.map(v=>`<button class="viewbtn" data-v="${v.id}">${v.name}<small>${v.hint}</small></button>`).join('');
document.getElementById('legend').innerHTML = [...new Set(ARCH.components.map(c=>c.type))].sort().map(t=>`<span>${t}</span>`).join('');
function setView(id){ S.view=id; S.sel=null; S.selCap=null;
// The relation filters and the component-type legend do nothing in the
// capability views. Leaving them visible reads as controls that are broken.
const vw = VIEWS.find(x=>x.id===id);
document.getElementById('archctl').style.display = (vw.caps || vw.inv) ? 'none' : '';
document.querySelectorAll('.viewbtn').forEach(b=>b.classList.toggle('on', b.dataset.v===id));
renderView();
const v = VIEWS.find(x=>x.id===id);
document.getElementById('side').innerHTML = v.caps
? '<div class="empty">Select a capability to see its definition of done, every verdict and its evidence, the components that carry it, its blockers and the product questions it waits on.<br><br>A dot is never an opinion. Six of the seven come from component status, the seventh from a probe run.</div>'
: v.inv
? '<div class="empty">Twelve rules that run across all 51 capabilities. Click a capability or a component to open its record.<br><br>An unresolved rule is a product question, not a defect.</div>'
: '<div class="empty">Select a component to see its responsibility, the files and symbols it was read from, and every relation in and out.</div>';
}
document.querySelectorAll('.viewbtn').forEach(b=>b.onclick=()=>setView(b.dataset.v));
['tLow','tMed','tOff','tUndeployed','tPlanned','tWires'].forEach(k=>
document.getElementById(k).onchange=()=>{ renderView(); if(S.sel) select(S.sel); });
document.getElementById('q').oninput = search;
document.getElementById('main').addEventListener('scroll', drawWires);
window.addEventListener('resize', drawWires);
setView('v1');
</script>
</body>
</html>
+626
View File
@@ -0,0 +1,626 @@
# Maven Current-State Audit
**Date:** 2026-09-05
**Scope:** Read-only code + tests investigation. No inference from names or docs.
**Source of truth:** code and tests only.
---
## A. Current End-to-End Flow Diagram
```
Telegram Bot API (long-poll)
|
| api.Chat(ctx, "telegram:<id>", text)
v
mavweb POST /api/chat --> ipc.Client (unix socket) --> daemonAPI.Chat()
|
| chatFn = handler.handleText
v
Voice TCP :9100 --> HandlePushToTalk --> STT --> +------------------+
| runTurn() |
| voice.go:270 |
+------------------+
| 0. decision record (ctx)
| 0b. turnRoute (computed once)
| 1. expired clarify notice
| 2. resolveConfirm (y/n)
| 3. resolveRepair (correction)
| 3b. resolveUntargetedRepair
| 3c. resolveCommandProhibition
| 4. resolveClarifyAnswer
| 5. resolveQuietToggle
| 5b. resolveSnooze
| 5c. resolveAck
| 5d. resolveReminderCancellation
| 5e. resolveCandidate (ordinal)
| 6. ROUTE (cascade)
| 7. dialogue merge (followUpMerge)
| 8. clarify (missing slots)
| 9. applyAction (per-intent dispatch)
| 10. replier (LLM or stub)
+------------------+
|
v
reply string
```
### The routing cascade (step 6) in detail:
```
utterance
|
v
Stage 0: Grammars (stagezero.go:24-91)
| 22 ordered regex/structural grammars. First match wins.
| Confidence = 1.0, Stage = 0. fillMatchedSlots runs.
| NO MATCH -> fall through
v
Stage 0b: Routing Heads (heads.go, ONNX)
| 4 linear heads over mean-pooled e5-small: intent, destination, slot BIO, clarify
| Below headsThreshold (0.6) -> decline, cascade continues
| Intent head softmax -> Decision{Intent, Confidence, Source, Clarify}
| completeParsedSingleVerbFact can overrule clarify
| NO WIRE / ERROR / DECLINE -> fall through
v
Stage 1a: LLM Router (llmrouter.go, Qwen3-1.7B via llama-server)
| GBNF grammar-constrained JSON output
| gateLLMDecision: thin confidence (0.3) for incomplete slots
| ERROR / PARSE FAIL / "unknown" -> fall through
v
Stage 1: Nearest-Centroid Classifier (classifier.go)
| Cosine similarity over ONNX embeddings against seed centroids
| Best intent wins (ties broken by name)
v
Stage 2: Slot Extraction (slots.go:55-91)
| Dispatch on intent: DateTimeParser (reminder), ActMatcher (act), FactParser (fact)
v
Stage 3: Confidence Gate (router.go:244-248)
| Confidence < threshold (0.55) -> Clarify = true, Stage = 3
```
---
## B. Package/File Ownership Map
### Command packages (cmd/)
| Package | Binary | Role |
|---------|--------|------|
| cmd/mavend | mavend | Core daemon: DB, IPC, routing, actions, voice server, tick loop |
| cmd/mavweb | mavweb | Web UI + HTTP server (serves /, /dash, /history, /trace, /notifications, /tools) |
| cmd/mavwaked | mavwaked | Wake-word + voice activity detection (silero VAD, energy threshold) |
| cmd/mavsttd | mavsttd | Speech-to-text daemon (whisper.cpp / remote worker) |
| cmd/mavttsd | mavttsd | Text-to-speech daemon (piper) |
| cmd/mavgpud | mavgpud | GPU proxy daemon (workpc inference) |
| cmd/mavcaldav | mavcaldav | CalDAV sync daemon |
| cmd/mavmaild | mavmaild | Mail ingestion daemon |
| cmd/mavpoll | mavpoll | Polling daemon (Kuma monitors) |
| cmd/mavenclient | mavenclient | CLI client |
| cmd/mavupdate | mavupdate | Self-update tool |
| cmd/mavseal | mavseal | Seal/encryption tool |
| cmd/e2eprobe | e2eprobe | End-to-end probe |
| cmd/labelgen | labelgen | Label generation tool |
| cmd/mavend/seedtest | (test helper) | Test seeder |
### Internal packages relevant to routing/action
| Package | Key files | Role |
|---------|-----------|------|
| internal/router | router.go, intent.go, stage0.go, stagezero.go, heads.go, llmrouter.go, classifier.go, slots.go, source.go, embedder.go, onnxembedder.go, question.go, singletoken.go, notecapture.go, claim.go (package-level), decisiontrace.go | Routing cascade, intent taxonomy, slot extraction, confidence |
| internal/claim | claim.go | Band/Claim/arbitration types (imported by router and dialogue) |
| internal/dialogue | session.go, clarify.go, pending.go, pending.go | Session state, clarification, PendingAction/Capability |
| internal/tool | tool.go, risk.go | Tool execution, risk tiers, policy |
| internal/phraser | phraser.go, llmphraser.go, fallbacks.go, replier.go | Response phrasing (LLM or stub) |
| internal/voice | server.go, wire.go, replier.go | Voice TCP server, wire protocol |
| internal/memory | store.go | Vector memory (cosine similarity search) |
| internal/llm | remote.go | Two-tier LLM client (resident + workstation) |
| internal/lexicon | lexicon.go | Closed Russian word sets (embedded JSON) |
| internal/morph | morph.go | Russian morphology (golem lemmatizer) |
| internal/loop | loop.go | Proactive nudge rules, gates |
| internal/store | facts.go, reminders.go, tools.go | SQLite persistence |
| internal/ipc | coreapi.go, api.go | IPC interface + wire types |
| internal/modes | modes.go | Mode inventory (30 routing classes) |
| internal/mcp | manager.go, client.go, allowlist.go | MCP tool discovery and execution |
| internal/smarthome | client.go | Home Assistant service calls |
### Key cmd/mavend files (the "glue" layer)
| File | Responsibility |
|------|---------------|
| voice.go | runTurn() pipeline, reactiveHandler, replySystem, applyAction |
| turnroute.go | turnRoute memo (computed once per turn) |
| actions.go | actionHandlers dispatch table (7 intents) |
| actions_act.go | actionAct: tool matching, ecosystem interception, execution |
| actions_fact.go | actionFact: write fact + embed + vector insert |
| actions_reminder.go | actionReminder: parse time + create reminder |
| actions_note.go | actionNote: write note + embed + vector insert |
| actions_query.go | actionQuery: 20+ source chain, queryTurn, queryWalk |
| confirm.go | resolveConfirm: y/n for destructive acts, Hexis, routines |
| clarify.go | askClarify, resolveClarifyAnswer, wantedSlots |
| followup.go | followUpMerge: slot inheritance across turns |
| continuation.go | continuationDecision: elliptical follow-ups |
| boot.go | Daemon wiring: connects all pieces |
| voicewire.go | Voice server wiring: STT, router, replier, tools, sessions |
| tick.go | Proactive tick loop: nudge delivery, reminders, routines |
| tick_routines.go | Routine firing, pattern detection |
| tick_digest.go | Digest queue and flush |
---
## C. Ingress -> Routing -> Action Call Trace
### Text path (web/telegram)
```
1. mavweb POST /api/chat (cmd/mavweb/chat.go:84)
-> strings.TrimSpace(r.FormValue("text"))
-> core.Chat(ctx, "web", text) (ipc/coreapi.go:205-213)
2. daemonAPI.Chat (cmd/mavend/tick_api.go:95)
-> chatFn(ctx, conversation, text)
-> handler.handleText(ctx, conversation, text) (voice.go:245)
3. handleText (voice.go:245-248)
-> runTurn(withDialogueID(ctx, ...), text, sourceText)
4. runTurn (voice.go:270-481) -- see Section A for step-by-step
```
### Voice path (mic)
```
1. Voice TCP Server receives PushToTalk frame (internal/voice/server.go:187-215)
-> handler.HandlePushToTalk(ctx, req, sid) (cmd/mavend/voice.go:200-218)
2. HandlePushToTalk:
-> stt.Transcribe(ctx, req.Audio) -- whisper.cpp or remote worker
-> runTurn(ctx, text, sourceVoice) -- same pipeline as text
-> tts.Synthesize(ctx, replyText) -- piper
-> return voice.PushToTalkResp{ReplyText, ReplyAudio}
```
### runTurn step-by-step (voice.go:270-481)
```
Step 0 (276): decision.Record installed on context (V-564)
Step 0b (292): turnRoute computed once, shared via context
Step 1 (313): clarifyExpiredNotice -- parked question TTL ran out
Step 2 (318): resolveConfirm -- y/n for parked destructive act
-> classifyConfirm(text) (confirm.go:264)
-> confirmResolvers chain (confirm.go:114):
1. pendingRoutineConfirm
2. pendingHexisExec
3. pendingAct -> tools.Exec(ctx, fn, args, true)
Step 3 (327): resolveRepair -- "нет, это был вопрос"
Step 3b (334): resolveUntargetedRepair -- "нет, не так"
Step 3c (344): resolveCommandProhibition -- "не отменяй..."
Step 4 (356): resolveClarifyAnswer -- answer to parked question
Step 5 (366): resolveQuietToggle -- "тихий режим"
Step 5b (374): resolveSnooze -- "не сейчас" / "потом"
Step 5c (381): resolveAck -- "готово"
Step 5d (389): resolveReminderCancellation -- cancel verb + noun
Step 5e (397): resolveCandidate -- "второй" (ordinal)
Step 6 (406): ROUTE
-> turnRoute.resolve(ctx) (turnroute.go:68)
-> continuationDecision(prev, text, now) OR router.Route(ctx, text, now)
Step 7 (425): followUpMerge(prev, dec, now) -- slot inheritance
Step 8 (445): clarify -- missing required slots
-> hexisBeforeClarify -- try Hexis before asking
-> askClarify -> park PendingQuestion
Step 9 (466): applyAction -> actionHandlers[dec.Intent]
Step 9b (474): ackFromFact -- close live nudge
Step 10 (477): replier -- phrase the reply (LLM or stub)
```
### Action dispatch (voice.go:496-504 -> actions.go:48-56)
```
applyAction(ctx, dec):
if dec.Clarify -> return "" (Replier phrases)
actionHandlers[dec.Intent](h, ctx, dec):
IntentFact -> actionFact (actions_fact.go:17)
-> coreAPI.WriteFact + memStore.Insert + detectPattern
IntentReminder -> actionReminder (actions_reminder.go:16)
-> coreAPI.CreateReminder
IntentAct -> actionAct (actions_act.go:17)
-> refusesCommand check
-> matcher.Match (fuzzy prefix over enabled tool names)
-> task_status intercept
-> praxis intercept
-> hexis intercept (resolve entity -> discover capabilities -> risk -> exec)
-> proposeGap (if no match)
-> tools.Exec (tool/tool.go:156):
LookupTool -> RiskOf -> PolicyFor(tier) -> dispatch:
MCP -> mcp.CallPositional
HA -> smarthome.CallService
Process -> exec.CommandContext
IntentChat -> actionChat (actions.go:58)
-> phraser.PhraseChat(ctx, utterance, history)
IntentSystem -> actionSystem (actions.go:77)
-> replySystem: keyword match on utterance
IntentNote -> actionNote (actions_note.go:24)
-> coreAPI.WriteNote + memStore.Insert
IntentQuery -> actionQuery (actions_query.go:varies)
-> querySources chain (20+ sources, first claim wins)
-> queryWalk narrows by destination
```
---
## D. Inventory of Existing Machinery
### D1. Deterministic fast-path recognizers
**Status: EXISTS, extensive, production-critical**
| Component | File:line | What it does |
|-----------|-----------|-------------|
| 22 stage-0 grammars | router/stagezero.go:24-91 | Ordered regex/structural rules. First match wins at confidence 1.0 |
| Grammar type | router/stage0.go:20-29 | Pattern+Build (regex) or Decide (structural) |
| Wake-word strip | router/stage0.go:66-78 | StripWakeToken: removes "мавен" in any script |
| Act allowlist fast path | router/stage0.go:55,80 | "мавен, restart nginx" -> act at stage 0 |
| Command prohibition | stagezero.go:30 | "don't restart nginx" -> refusal sentinel |
| System time/date | stagezero.go:35 | "сколько времени", "который час" |
| Agenda query | stagezero.go:39 | "что у меня сегодня" |
| Reminder | stagezero.go:59 | "напомни через час" with time extraction |
| Fact capture | stagezero.go:80 | "запиши купить молоко" with note body extraction |
| Possession statement | stagezero.go:89 | "у меня кончилась вода" |
| Lexicon (closed word sets) | lexicon/lexicon.go | ~40 embedded Russian word sets |
| Morphology | morph/morph.go | Lemma(), IsVerbForm(), SameWord() via golem |
| Question detection | router/question.go | IsQuestionShaped(), IsOpenQuestionShaped(), CarriesCaptureVerb() |
| Single-token analysis | router/singletoken.go | thinSingleToken() with completeSingles escape |
| Note capture parser | router/notecapture.go | ParseNoteCapture(): strips capture frame |
| DateTimeParser | router/slots.go:19-21 | Interface; production is dateparser (not shown) |
| DefaultFactParser | router/slots.go:154-180 | Lemma-based fact extraction (water/meal/shower/break/sleep) |
| DefaultActMatcher | router/slots.go:106-149 | Exact phrase prefix + aliases, longest-first |
### D2. Learned routing/NLU
**Status: EXISTS, multi-layered**
| Component | File:line | What it does |
|-----------|-----------|-------------|
| RouterHeads (ONNX) | router/heads.go:55-64 | 4 heads over e5-small: intent, destination, slot BIO, clarify |
| RouterHeads.Route | router/heads.go:120+ | Softmax classification, threshold=0.6, single forward pass |
| LLMRouter | router/llmrouter.go:20-22 | GBNF-constrained JSON from Qwen3-1.7B |
| LLMRouter.Route | router/llmrouter.go:120+ | System prompt + grammar -> routeAction structs |
| gateLLMDecision | router/router.go:352-373 | Thin confidence (0.3) for incomplete slots |
| Classifier | router/classifier.go:40-45 | Nearest-centroid over ONNX embeddings |
| Classifier.Classify | router/classifier.go:50+ | Cosine similarity, sorted best-first |
| Classifier.AddExample | router/classifier.go | Append-only correction (grows classifier) |
| ONNX Embedder | router/onnxembedder.go | multilingual-e5-small, 384-dim, query/passage prefix |
| HashEmbedder | router/embedder.go:79-116 | Fallback bag-of-words (deterministic, weak) |
| Embedder interface | router/embedder.go:17-21 | Dim(), Embed(), Close() |
| Intent taxonomy | router/intent.go:38-63 | 7 intents: act, reminder, fact, note, query, chat, system |
| Source taxonomy | router/source.go | 12 destinations: recall, calendar, tasks, list, money, weather, home, network, feeds, attention, self, world |
| Slots struct | router/intent.go:66-94 | Time, Fn, Args, Key, Value, Text + Has* flags |
| Decision struct | router/intent.go:100-128 | Utterance, Stage, Intent, Confidence, Slots, Clarify, Source, SourceAnchored |
| Modes inventory | modes/modes.go | ~30 distinct downstream behaviors, embedded JSON |
| Confidence gate | router/router.go:244-248 | threshold=0.55, below -> Clarify=true |
### D3. Claim/arbitration system
**Status: EXISTS, not wired into cascade (per claim.go:138-140)**
| Component | File:line | What it does |
|-----------|-----------|-------------|
| Band enum | claim/claim.go:38-69 | BandUnknown, BandVetoed, BandNearest, BandStructural, BandAnchored |
| Claim struct | claim/claim.go:88-117 | Claimant, Intent, Filled, Consumed, Unexplained, Band, Veto |
| Coverage() | claim/claim.go:122-128 | Consumed / (Consumed + Unexplained) |
| MoreSpecificThan() | claim/claim.go:141-147 | Coverage first, Band breaks ties |
| Tokens() | claim/claim.go:156-169 | Tokenize utterance for coverage |
| Split() | claim/claim.go:178-192 | Partition tokens into consumed/unexplained |
Note: claim.go:138-140 explicitly states this is "Deliberately NOT wired into the cascade by V-565." It is here so the ordering is one function with tests rather than duplicated logic.
### D4. Action path
**Status: EXISTS, with multiple dispatch paths**
| Component | File:line | What it does |
|-----------|-----------|-------------|
| actionHandlers table | actions.go:48-56 | 7-intent dispatch map |
| applyAction | voice.go:496-504 | Short-circuits on Clarify, dispatches via table |
| actionAct | actions_act.go:17-114 | Full act cascade: refuse -> match -> task_status -> praxis -> hexis -> propose -> exec |
| tool.Executor.Exec | tool/tool.go:156-235 | LookupTool -> RiskOf -> PolicyFor -> dispatch (MCP/HA/process) |
| RiskOf | tool/risk.go | Derives tier from Tool row |
| PolicyFor | tool/risk.go | TierSafe/TierDestructive/TierIrreversible -> Confirm/VoiceMayRun |
| Matcher.Match | tool/tool.go:251+ (tool package) | Fuzzy prefix over enabled tool names |
| MCP dispatch | tool/tool.go:191-198 | mcp.ParseCmd -> mcp.CallPositional |
| HA dispatch | tool/tool.go:205-221 | smarthome.ParseCmd -> smarthome.CallService |
| Process dispatch | tool/tool.go:231-234 | exec.CommandContext (no shell) |
| Hexis integration | ecosystem_acts.go:660+ | resolve entity -> discover capabilities -> risk -> confirm/exec |
| Praxis integration | ecosystem_acts.go (handlePraxisAct) | Attention/item lifecycle |
| UnknownTargetError | tool/tool.go:90-96 | Named error with target word |
| Tool store | store/tools.go | ProposeTool, EnableTool, DisableTool, LookupTool, ReconcileMCPTool |
### D5. Confirmation/risk handling
**Status: EXISTS, comprehensive**
| Component | File:line | What it does |
|-----------|-----------|-------------|
| resolveConfirm | confirm.go:78-100 | classifyConfirm + chain of resolvers |
| classifyConfirm | confirm.go:264+ | Closed yes/no lexicon, entire utterance must match |
| confirmResolvers | confirm.go:114-186 | 3 slots: routine proposal, Hexis exec, local tool |
| confirmTTL | confirm.go:59 | 90s |
| pendingAct | confirm.go:48-54 | fn, args, phrase, expiry |
| pendingHexisExec | confirm.go:21-35 | capabilityID, entityID, correlationID, expiry |
| pendingRoutineConfirm | confirm.go:39-46 | routineID, action, object, interval, phrase, expiry |
| park() | confirm.go:63-67 | Stores pendingAct |
### D6. Non-action paths
| Component | File:line | What it does |
|-----------|-----------|-------------|
| actionChat | actions.go:58-75 | phraser.PhraseChat with dialogue history |
| replySystem | voice.go:509-569 | Keyword matching on utterance for time/date/status |
| actionQuery chain | actions_query.go:99-188 | 20+ sources, first claim wins |
| queryWalk | actions_query.go:190+ | Narrows chain by destination |
| Best recall | recall.go:8-65 | Vector search with confidence gate (minScore + minMargin) |
| Clarify store | dialogue/clarify.go | PendingQuestion stack (max depth 2), 90s TTL |
| Follow-up merge | followup.go:99-194 | Slot inheritance across same-intent turns |
| Continuation | continuation.go:57-98 | Elliptical follow-ups ("а завтра?") |
| Tick loop | tick.go | Proactive nudge delivery, reminders, routines, patterns, digest |
### D7. Tests/data inventory
**Status: Extensive (354 test files)**
Key test fixtures:
| Fixture | File | Cases |
|---------|------|-------|
| Routing contract | internal/router/eval/ru_routing_v1.json | ~70 held-out utterances with intent/source/time/clarity expectations |
| Ecosystem reach | internal/router/eval/ru_ecosystem_v1.json | ~50 act utterances with service/capability expectations |
| Nudge phrasing | internal/phraser/eval/nudges_v1.json | 15 cases, on-topic + property checks |
| Talk phrasing | internal/phraser/eval/talk_v1.json | 27 cases (chat/query/knowledge) |
| Recall contract | internal/memory/recalleval/ru_recall_v1.json | ~40 cases with note sets |
| Personal boundary | cmd/mavend/testdata/personal_boundary_v1.json | 72 cases |
| Safety scenarios | cmd/mavend/testdata/system_safety_scenarios.json | 4 scenarios |
| Simulator scenarios | cmd/mavend/testdata/scenarios/*.json | 5 full scripted scenarios |
| Usage transcript | scripts/testdata/usage-turns.txt | 167-line simulated usage |
| STT golden | cmd/mavsttd/testdata/golden_v1.json | 4 cases with WER bounds |
Key eval harnesses:
| Harness | File |
|---------|------|
| Routing eval | internal/router/eval/eval.go |
| Heads eval | internal/router/eval/heads_test.go |
| LLM router eval | internal/router/eval/llmrouter_test.go |
| Ecosystem reach eval | internal/router/eval/reach_test.go |
| Claim scoring | internal/router/eval/claims_test.go |
| Nudge phrasing eval | internal/phraser/eval/eval.go |
| Talk phrasing eval | internal/phraser/eval/talk.go |
| Recall eval | internal/memory/recalleval/recalleval.go |
| Kiwix rewrite eval | internal/kiwix/rewrite_eval.go |
| Simulator | cmd/mavend/simulator_test.go |
Classifier seed phrases: models/seeds/{query,fact,chat,act,note,reminder,system}.txt
---
## E. Existing Contracts/Types We Can Reuse
### Already well-typed (reuse as-is or thin wrapper)
| Type | File:line | Notes |
|------|-----------|-------|
| `router.Intent` | intent.go:38 | String enum: act, reminder, fact, note, query, chat, system |
| `router.Slots` | intent.go:66-94 | Typed: Time, Fn, Args, Key, Value, Text + Has* flags |
| `router.Decision` | intent.go:100-128 | Utterance, Stage, Intent, Confidence, Slots, Clarify, Source, SourceAnchored |
| `router.Source` | source.go | String enum: 12 destinations |
| `router.Grammar` | stage0.go:20-29 | Pattern+Build or Decide |
| `router.Extractor` | slots.go:45-49 | Time, Acts, Facts parsers |
| `claim.Claim` | claim.go:88-117 | Band-based arbitration (not wired yet) |
| `claim.Band` | claim.go:36-69 | Ordinal evidence kinds |
| `dialogue.PendingAction` | pending.go:58 | Capability, slots, missing, utterance, TTL |
| `dialogue.Capability` | pending.go:16-46 | 7 capability strings |
| `tool.Executor` | tool/tool.go:116-122 | Exec with policy checks |
| `tool.Policy` | tool/risk.go | Confirm, VoiceMayRun per tier |
| `ipc.ChatReply` | ipc/api.go:752-761 | Reply, Source, TraceID |
| `ipc.Tool` | ipc/api.go:699 | Wire shape of tool row |
### Partially exists (needs extension)
| Concept | Current form | Gap |
|---------|-------------|-----|
| NormalizedInput | Raw string in, `text string` parameter | No NormalizedInput struct; STT output passes as-is |
| FastPathResult | Stage-0 grammar Decision | Not a separate type; embedded in Decision |
| RouteDecision | `router.Decision` | Already carries Stage (0/1/2/3), Intent, Confidence, Slots. Could become RouteDecision |
| ActionCandidate | `router.Decision` + `actionHandlers` dispatch | No explicit ActionCandidate type; intent + slots + handler selection are implicit |
### Missing (must be created if needed)
| Concept | Notes |
|---------|-------|
| Schema validation | No JSON-schema or struct validation on incoming slots before execution |
| Confidence/risk policy | Risk tiers exist for tools but not for routing confidence. Stage 3 gate exists but is a simple threshold |
| Post-execution verification | No explicit verification step after tool execution (success/failure is the extent) |
---
## F. Gaps Against the Proposed First-Stage Design
### Proposed pipeline vs current reality
| Proposed stage | Current state | Gap |
|---------------|---------------|-----|
| **NormalizedInput** | Raw `text string` everywhere | No normalization struct. Preprocessing is scattered: StripWakeToken in router, lowercase in matchers, trim in entry points |
| **Deterministic fast-path** | Stage-0 grammars (22 rules) | EXISTS and is production-critical. However: grammars produce `Decision` directly, not a separate FastPathResult type. No schema validation on slots before returning |
| **Learned routing** | Heads -> LLM -> Classifier cascade | EXISTS and complex. But: multiple confidence scales (stage-0=1.0, heads=softmax, LLM=1.0/0.3, classifier=cosine). No unified confidence model |
| **RouteDecision** | `router.Decision` | EXISTS under another name. Carries Stage, Intent, Confidence, Slots, Clarify, Source. Could be wrapped/renamed |
| **ActionCandidate** | Implicit in `router.Decision` + `actionHandlers` | Missing as explicit type. The decision arrives at applyAction and is dispatched by intent. No schema validation of slots before dispatch |
| **Schema validation** | NONE | Slots are filled by extractors and used directly. No validation that e.g. reminder has both Text and Time before actionReminder runs |
| **Confidence/risk policy** | Threshold gate (0.55) for routing; risk tiers for tools | No unified confidence policy. Routing confidence and tool risk are separate systems. No policy that says "if confidence < X, require confirmation for action" |
| **Confirmation** | resolveConfirm with 3 pending slots | EXISTS for destructive tools, Hexis, and routines. Not applied to routing confidence (a low-confidence act just gets proposed, not confirmed) |
| **Execution** | tool.Executor.Exec | EXISTS with MCP/HA/process dispatch. Well-structured. But: actionFact, actionReminder, actionNote bypass tool.Executor entirely (they call CoreAPI directly) |
| **Post-execution verification** | Success/failure error handling in actionAct | Partially EXISTS. Tool execution returns (out, err). Error types are handled specifically. But: no structured verification step, no retry policy, no rollback |
### Architectural problems identified
1. **Fast paths that execute directly**: actionFact, actionReminder, actionNote call CoreAPI.WriteFact/CreateReminder/WriteNote directly from the action handler, bypassing tool.Executor. This means they skip the risk tier system, the confirm gate, and the allowlist. This is by design (facts/reminders are user-stated, not tool invocations) but means the "all actions through one pipeline" goal requires either wrapping these in tool-like abstractions or explicitly exempting them.
2. **Multiple confidence scales**: Stage-0 = 1.0 (hardcode), heads = softmax float, LLM = 1.0 or 0.3 (thin), classifier = cosine similarity. These are not on the same scale and cannot be compared. The claim system (claim.Band) explicitly addresses this by making confidence ordinal (Band) rather than graded. The proposed design should preserve this insight.
3. **Routing code that also performs tool selection**: actionAct at actions_act.go:29-33 runs the matcher inline when HasFn is false. The matcher is also the stage-2 ActMatcher. So tool selection happens both in the router (stage 2) and in the action handler (actionAct). The actionAct path is the fallback for LLM-routed acts where the verb didn't go through stage-0.
4. **Implicit fallthrough**: The router cascade is explicitly designed as fallthrough (each stage may decline). The query source chain is also fallthrough (first claim wins). The confirm resolver chain is also fallthrough. This is a consistent pattern, not a bug, but means the "stage-to-stage" architecture must preserve explicit decline semantics.
5. **Clarify bypasses action pipeline**: When dec.Clarify is true, applyAction returns "" immediately (voice.go:497-499). The clarifier can also call hexisBeforeClarify (voice.go:446) to try Hexis before asking, which is a hidden action path that runs before the normal action dispatch.
6. **No schema validation**: Slots filled by stage-2 extraction or stage-0 grammars are used directly by action handlers. actionReminder (actions_reminder.go) checks HasTime itself. actionFact checks HasKey. But there is no shared validation layer; each handler does its own checks.
7. **Voice-specific behavior**: HandlePushToTalk wraps runTurn with STT before and TTS after. The turnSource tag ("tap:voice" vs "tap:text") propagates into fact sources but does not change routing or action behavior. However: the voice path has barge-in, session management, and wake-word detection that the text path lacks entirely. The semantic behavior is the same; the infrastructure is different.
8. **Existing useful code to preserve**:
- Stage-0 grammars: 22 ordered rules, battle-tested, each with extensive comments about why it sits where it sits. Moving or reordering them breaks routing.
- The claim/Band system: explicitly designed for the problem of incomparable confidence scales. Not wired yet but well-tested.
- The clarify store with stack support: handles nested clarification flows.
- The query source chain with destination narrowing: 20+ sources with guessers vs lookups distinction.
- The tool risk tier system: well-tested, with voice-specific authority limits.
- The turnRoute memo pattern (V-560): computed once, shared via context, prevents routing disagreement.
---
## G. Smallest Behavior-Preserving Refactor Boundary
The smallest refactor that aligns with the proposed architecture without changing behavior:
**Wrap Decision in RouteDecision + add NormalizedInput as thin alias**
```
Current: router.Route(ctx, utterance, now) -> (Decision, error)
Proposed: router.Route(ctx, NormalizedInput, now) -> (RouteDecision, error)
```
Where:
- `NormalizedInput` is `type NormalizedInput struct { Text string; Source string }` -- a thin wrapper, not a transformation
- `RouteDecision` is `type RouteDecision Decision` -- or just `Decision` with a type alias
- The existing Stage field (0/1/2/3) already encodes which stage produced the result
- The existing Confidence field already carries the per-stage confidence
This changes zero behavior. It names what exists. It creates the typed boundary the future stages need.
**Second step: extract action candidates**
Currently actionAct does tool matching inline. The matcher result (fn, args) should be a typed ActionCandidate returned by the router or by a post-route step, not discovered inside the action handler. But this changes the call structure of actionAct, which is a larger refactor.
**Third step: schema validation**
Add a Validate(slots) step between routing and action dispatch. Currently each handler validates its own slots; this would centralize it. Minimal behavior change: the same checks, in one place.
---
## H. Recommended Implementation Order
1. **Type the boundaries** (1-2 hours)
- Define NormalizedInput, RouteDecision as thin wrappers
- Route() signature change (internal callers only)
- Zero behavior change
2. **Pin current behavior with regression tests** (2-3 hours)
- Run existing eval fixtures and record baselines
- Add integration tests for the full runTurn pipeline (text + voice paths)
- Add tests for each action handler with representative inputs
3. **Extract ActionCandidate from actionAct** (3-4 hours)
- Move tool matching out of actionAct into a post-route step
- Return ActionCandidate{Fn, Args, Source} from routing
- actionAct consumes ActionCandidate instead of re-matching
4. **Add schema validation layer** (2-3 hours)
- Validate slots before action dispatch
- Centralize the per-handler checks
- Fail-closed: missing required slot -> clarify, not runtime error
5. **Unify confidence presentation** (3-4 hours)
- Map per-stage confidence to ordinal Band (leverage existing claim.Band)
- Expose in RouteDecision for downstream policy
- Do NOT try to make confidence comparable across stages
6. **Wire claim.Band into cascade** (4-6 hours)
- Replace ad-hoc precedence with MoreSpecificThan
- This is the V-558/V-565 work already planned
---
## I. Tests That Should Pin Current Behavior Before Refactoring
### High-value regression pins
| Test | What it pins | File |
|------|-------------|------|
| Router cascade stage ordering | Stage-0 wins, heads decline correctly, LLM fallback, classifier floor | internal/router/router_test.go |
| Held-out routing contract | ~70 utterances with intent/source/time expectations | internal/router/eval/eval_test.go |
| Ecosystem reach contract | ~50 act utterances routing to correct service | internal/router/eval/reach_test.go |
| Simulator scripted day | Full pipeline: STT -> router -> store -> phraser | cmd/mavend/simulator_test.go |
| Safety scenarios | Destructive acts require confirmation, ambiguous entities clarified | cmd/mavend/eval_scenarios_test.go |
| Tool risk assessment | Tier derivation from tool rows | internal/tool/ risk_test.go (implied) |
| Confirm flow | y/n for parked acts, TTL expiry, chain ordering | cmd/mavend/confirm_test.go |
| Clarify flow | Missing slots -> question -> answer -> continue | cmd/mavend/clarify_test.go |
| Follow-up merge | Slot inheritance across turns | cmd/mavend/followup_test.go |
| Query source chain | First-claim-wins, destination narrowing | cmd/mavend/querywalk_test.go |
| Personal boundary | 72-case held-out fixture | cmd/mavend/personalboundary_test.go |
| Recall contract | ~40 cases with paraphrased queries | internal/memory/recalleval/recalleval_test.go |
| Action act risk | Destructive/irreversible classification | cmd/mavend/actions_act_risk_test.go |
| Degradation | Each ecosystem service unreachable | cmd/mavend/ecosystem_degraded_test.go |
### What to run before and after each refactor step
```sh
make test # full suite
go test ./internal/router/eval/ -run Eval # routing contract
go test ./cmd/mavend/ -run Simulator # integration
go test ./cmd/mavend/ -run Eval # safety scenarios
go test ./cmd/mavend/ -run PersonalBoundary # boundary fixture
go test ./internal/phraser/eval/ -run Eval # phrasing contract
go test ./internal/memory/recalleval/ -run Eval # recall contract
```
---
## J. Unknowns That Cannot Be Established From Code/Tests
1. **Actual production accuracy numbers**: The eval fixtures measure held-out accuracy, but production routing traces (routing_traces table) are the real measure. We cannot inspect the production DB from code.
2. **Whether the LLM router is currently enabled in production**: The config shows `llm_router` settings but we cannot confirm the daemon is running with it wired. The heads may be the actual fast path.
3. **Real-world confirmation rates**: How often do users get asked to confirm? How often do they decline? This is behavioral data, not code.
4. **Whether the claim system should be wired**: claim.go says "Deliberately NOT wired by V-565" but the current ad-hoc precedence works. The claim system is tested but untested in production.
5. **Token budget pressure on the resident model**: The 4096 context window is shared between routing, phraser, and chat. We cannot tell from code whether context pressure causes routing failures in production.
6. **Whether stage-0 grammars overlap or shadow each other**: The ordering is documented, but no test measures "if grammar A were removed, which utterances would fall through differently." The cascade hides contention by design.
7. **Performance characteristics of the ONNX embedder in production**: Tests measure p50 (20.6ms for classifier). Production numbers on the actual hardware may differ.
8. **Whether the ecology of pre-route resolvers (steps 1-5e) can be unified**: Seven resolvers each claim the turn independently, in order. Whether they could be replaced by a single arbiter (the claim system) is a design question, not a code question.
---
## Proposed Mapping: Future Stage/Contract -> Current Implementation
| Future stage/contract | Current implementation | Reuse/wrap/move/replace | Reason |
|----------------------|----------------------|------------------------|--------|
| **NormalizedInput** | Raw `text string` parameter in handleText/HandlePushToTalk/runTurn | **wrap** | Create struct, pass through. No transformation needed yet. Existing preprocessing (StripWakeToken, trim) stays inside the router. |
| **Deterministic fast path** | Stage-0 grammars (router/stagezero.go, stage0.go) | **reuse as-is** | 22 battle-tested rules with load-bearing ordering. Output is Decision at confidence 1.0. Naming it "fast path" is cosmetic. |
| **FastPathResult** | Decision with Stage=0 | **wrap** | Type alias or thin struct. The Stage field already identifies the source. |
| **RouteDecision** | `router.Decision` (intent.go:100-128) | **reuse (rename or alias)** | Already carries all needed fields: Intent, Confidence, Slots, Clarify, Source, Stage. The Stage field (0/1/2/3) tells which cascade stage produced it. |
| **ActionCandidate** | Implicit: Decision.Intent + Decision.Slots + actionHandlers dispatch | **move** | Extract from actionAct into a post-route step. Currently, actionAct:29-33 re-runs the matcher when HasFn is false. This should produce an ActionCandidate that actionAct consumes. |
| **Schema validation** | Per-handler checks (actionReminder checks HasTime, actionFact checks HasKey) | **move + centralize** | Currently scattered across action handlers. Centralize into a Validate(Decision) step before applyAction. |
| **Confidence/risk policy** | Stage-3 gate (threshold 0.55) for routing; RiskOf/PolicyFor for tools | **extend** | These are separate systems today. The routing gate Clarify flag. The tool policy returns ErrNeedsConfirm/ErrNeedsAuthedSurface. A unified policy would map confidence bands to action policies. |
| **Confirmation** | resolveConfirm (confirm.go:78) with 3 pending slots | **reuse** | Already handles destructive tools, Hexis, and routines. Would need extension if low-confidence acts should also confirm. |
| **Execution** | tool.Executor.Exec (tool/tool.go:156) + per-intent handlers | **reuse** | Well-structured with MCP/HA/process dispatch. The per-intent handlers (actionFact, actionReminder) bypass Executor by design -- they write to the store, not run tools. |
| **Post-execution verification** | Error handling in actionAct (actions_act.go:64-108) | **extend** | Currently: success -> "done", specific error -> specific reply. No structured verification step. Adding one would be a new layer. |
+125
View File
@@ -0,0 +1,125 @@
# docs/capabilities/
Generated. Regenerated from `docs/spec.md` plus a named eval. Not hand-edited.
`docs/spec.md` says what Maven should do. This directory says how much of that
exists, measured rather than asserted. The predecessor audit had a green test
suite while 22 of 39 capabilities were not live, which is the failure mode the
whole directory is built against.
## The files
| file | what it is | hand-edited |
| --- | --- | --- |
| `ledger.yaml` | the ledger: 51 capabilities, 156 DoD criteria, one verdict per criterion, seven implementation dimensions per capability | no |
| `build_ledger.py` | extracts the ledger from `docs/spec.md` and joins the two inputs | yes, it is the source |
| `domains.yaml` | the domain axis, one of the two judgment calls in the extraction | yes |
| `implementation.yaml` | capability to component mapping, the other judgment call | yes |
| `verdicts.json` | one verdict per criterion id, produced by scoring a probe run | no, scored |
| `probes_field.json` | 25 multi-turn probes: the owner's real week | yes |
| `probes_dod.json` | probes derived from the ledger's criteria | no, generated |
| `run_probes.py` | drives a probe file through the deployed stack | yes |
| `store_counts.py` | row counts per store, over IPC | yes |
| `invariants.md` | the twelve cross-cutting rules the 51 capabilities imply, with the prose and the evidence | yes |
| `invariants.yaml` | the machine-readable half of the same twelve: mark, capabilities, components | yes |
| `gaps.md` | eight gap classes and the one ranked priority list | yes, except classes 1-4 |
| `out/` | raw probe output, one JSON object per line | no |
## Rebuilding
```sh
python3 docs/capabilities/build_ledger.py # spec.md + domains.yaml + implementation.yaml + verdicts.json + maven-architecture.json -> ledger.yaml
```
The generator is also the checker. It fails, loudly and non-zero, on a
capability with no DoD criteria, a capability with no `State` line, a criterion
id collision, a capability with no domain or more than two, an unknown domain
name, a `domains.yaml` row naming a capability that does not exist, a
`verdicts.json` row scoring a criterion that does not exist, a verdict word
outside the five, and a reason outside the plan's list. It caught the domain
reconciler silently dropping `recall` from its 51.
It fails on an invariant whose `## N. Title` heading is absent from
`invariants.md`, on a count mismatch between the two files, on an unknown
capability or component in `invariants.yaml`, and on an `unresolved` invariant
carrying no product question. The two files exist separately so the viewer can
read one and a person can read the other, and they drift the moment nothing
checks them.
It also fails on a capability missing from `implementation.yaml`, a component id
that `docs/architecture/maven-architecture.json` does not carry, and a component
status the dimension table does not know. A capability absent from the mapping
would read `no` on every dimension, which is indistinguishable from a capability
nothing carries.
## The seven dimensions
Never one `implemented` boolean. Coded and unwired, wired and unconfigured, and
configured and undeployed are three different pieces of work.
`designed`, `code_present`, `wired`, `configured`, `deployed` and `reachable`
come from the `status` field of every component mapped to the capability, rolled
up as all yes, none no, otherwise partial. `verified` comes from the criteria
verdicts: yes when every one passes, partial when some do.
`designed` answers a narrower question, because `docs/spec.md` states all 51 of
them. It reads `yes` when a living doc owns the subsystem, and `spec-only` when
the State line says no living doc or no package.
A component the mapping does not use is reported by name at the end of a build.
Shared infrastructure is excluded on purpose: mapping `core.reactive_handler` to
everything would give all 51 rows the same status and say nothing.
## Running the probes
The probes run **on homesrv**, where mavweb is on `127.0.0.1:9201` and the
mavend socket is reachable from inside the container.
```sh
# Build the probe binary. It needs CGO and the target's glibc, so build it in a
# trixie container: both the golang image and the mavend image are trixie.
docker run --rm -v "$PWD":/src -w /src \
-e CGO_ENABLED=1 -e GOFLAGS=-mod=vendor \
-e GOCACHE=/src/.cache/gocache -e GOPATH=/src/.cache/gopath \
golang:1.25-trixie go build -buildvcs=false -o /src/.cache/e2eprobe ./cmd/e2eprobe
docker cp .cache/e2eprobe maven-mavend-1:/tmp/e2eprobe
python3 docs/capabilities/store_counts.py # before
python3 docs/capabilities/run_probes.py probes_field.json > out/field.raw.jsonl
python3 docs/capabilities/store_counts.py # after
```
## Two things the harness learned the hard way
**Probes must be isolated.** `mavweb` hardcodes one conversation id for the
whole web reach, so a clarify parked by one probe is still parked for the next.
The first run measured the previous probe, not the current one: the park set at
turn 8 appended `Сейчас 01:07. В какой день?` to turns 9 through 13, five
unrelated turns in a row, including plain statements. `run_probes.py` now sends `отмена` before every
probe. The contaminated run is kept at `out/field.contaminated.jsonl`, because
the leak is a finding and not only an artifact.
**Readback is the contract, not the file.** The plaintext database copy at
`/dev/shm/maven-plain.db` would answer every question faster and would bypass
the IPC contract the ledger exists to measure. The mavweb pages are a
second-hand rendering of the same thing.
## What a verdict means
`pass` comes only from `live` evidence: the deployed build, the real model, real
store rows. The scenario harness scripts both `route` and `reply`, so a green
scenario proves the wiring around the model and not the turn; it is recorded as
implementation evidence and reads `untested`. `simulated` is allowed only where
the trigger is anchored to a wall-clock hour or date a probe cannot reach.
## The viewer
`docs/architecture/index.html` views 6 and 7 read this directory.
`build_viewer.py` inlines `ledger.yaml` and `invariants.yaml` and derives
nothing: the ledger's build is the only thing allowed to decide a dimension, and
a second derivation would drift from it silently.
```sh
python3 docs/architecture/build_viewer.py
node docs/architecture/check_viewer.js
```
+592
View File
@@ -0,0 +1,592 @@
#!/usr/bin/env python3
"""Extract the target side of the capability ledger from docs/spec.md.
Mechanical. No implementation judgment, no verification status, no ranking.
Domain assignment is the one human input and lives in domains.yaml, keyed by
capability id; this script only joins it and fails loudly on a mismatch.
Run from the repo root: python3 docs/capabilities/build_ledger.py
"""
import hashlib
import json
import pathlib
import re
import sys
ROOT = pathlib.Path(__file__).resolve().parents[2]
SPEC = ROOT / "docs" / "spec.md"
OUT = ROOT / "docs" / "capabilities" / "ledger.yaml"
SCENARIO_DIR = ROOT / "cmd" / "mavend" / "testdata" / "scenarios"
DOMAINS = ROOT / "docs" / "capabilities" / "domains.yaml"
VERDICTS = ROOT / "docs" / "capabilities" / "verdicts.json"
IMPL = ROOT / "docs" / "capabilities" / "implementation.yaml"
ARCH = ROOT / "docs" / "architecture" / "maven-architecture.json"
INV_YAML = ROOT / "docs" / "capabilities" / "invariants.yaml"
INV_MD = ROOT / "docs" / "capabilities" / "invariants.md"
# Sections of docs/spec.md whose ### headings are capabilities. Every other ##
# is prose about how to read the file.
CAPABILITY_SECTIONS = {
"The turn",
"Memory",
"Proactive",
"Reach",
"Speech and senses",
"The ecosystem",
"Operations",
"Undesigned in v1",
}
# Backticked identifiers that appear on a Scenario line and are not scenarios.
NOT_SCENARIOS = {"mavseal", "docker"}
DOMAIN_NAMES = {
"perception", "memory", "attention", "deliberation",
"initiative", "action", "interaction", "governance", "operations",
}
def slug(title):
s = re.sub(r"[^a-z0-9]+", "-", title.lower()).strip("-")
return s
def crit_id(cap_slug, text):
h = hashlib.sha256(text.encode("utf-8")).hexdigest()[:4]
return f"{cap_slug}#{h}"
def unwrap(lines):
"""Join a bullet's continuation lines into one string."""
return " ".join(l.strip() for l in lines).strip()
def parse():
lines = SPEC.read_text(encoding="utf-8").splitlines()
caps = []
section = None
cur = None
mode = None # None | 'dod' | 'state' | 'scenario'
buf = [] # continuation lines of the bullet being read
section_notes = {}
def flush_bullet():
nonlocal buf
if not buf or cur is None:
buf = []
return
text = unwrap(buf)
buf = []
if not text:
return
if mode == "dod":
cur["dod"].append(text)
elif mode == "state":
cur["state"] = text
elif mode == "scenario":
cur["scenario_raw"] = text
for raw in lines:
if raw.startswith("## "):
flush_bullet()
mode = None
cur = None
section = raw[3:].strip()
continue
if raw.startswith("### "):
flush_bullet()
mode = None
if section not in CAPABILITY_SECTIONS:
cur = None
continue
title = raw[4:].strip()
cur = {
"id": slug(title),
"title": title,
"section": section,
"scope": "v1",
"state": "",
"finding": "",
"dod": [],
"scenario_raw": "",
"body": [],
}
caps.append(cur)
continue
if cur is None:
# Section-level prose. Keep a Finding paragraph, it belongs to the
# whole cluster (Memory has one).
if section in CAPABILITY_SECTIONS and "**Finding**" in raw:
section_notes.setdefault(section, []).append(raw.strip())
elif section in section_notes and raw.strip() and not raw.startswith("#"):
# continuation of that paragraph
if section_notes[section] and section_notes[section][-1]:
section_notes[section][-1] += " " + raw.strip()
continue
cur["body"].append(raw)
stripped = raw.strip()
if stripped.startswith("- **State**:"):
flush_bullet()
mode = "state"
buf = [stripped[len("- **State**:"):]]
continue
if stripped.startswith("- **DoD**"):
flush_bullet()
mode = "dod"
continue
if stripped.startswith("- **Scenario**:"):
flush_bullet()
mode = "scenario"
buf = [stripped[len("- **Scenario**:"):]]
continue
if stripped.startswith("**Deferred past v1**"):
flush_bullet()
cur["scope"] = "deferred"
cur["deferred_note"] = stripped
mode = None
continue
if not stripped:
flush_bullet()
continue
if mode == "dod":
if stripped.startswith("- "):
flush_bullet()
buf = [stripped[2:]]
else:
buf.append(stripped)
continue
if mode in ("state", "scenario"):
if stripped.startswith("- "):
flush_bullet()
mode = None
else:
buf.append(stripped)
continue
flush_bullet()
for c in caps:
# The Finding sentence lives inside the State bullet in every entry that
# has one. Split it out so a gap is a field, not prose.
m = re.search(r"\*\*Finding\*\*:\s*(.*)$", c["state"], re.S)
if m:
c["finding"] = m.group(1).strip()
c["state"] = c["state"][: m.start()].strip()
c["state"] = c["state"].strip()
c["scenarios"] = parse_scenarios(c["scenario_raw"])
c["criteria"] = [
{"id": crit_id(c["id"], t), "text": t} for t in c["dod"]
]
del c["body"], c["dod"], c["scenario_raw"]
return caps, section_notes
def parse_scenarios(raw):
"""Names in `backticks`, each flagged exists / to write.
The scenario line is prose in several entries ("covered by cmd/mavweb
tests", "none"). Keep the prose verbatim as `note` rather than guessing.
"""
out = []
for m in re.finditer(r"`([a-z0-9_]+)`(\s*\*\(to write\)\*)?", raw):
name = m.group(1)
# A path, a package or a binary named in prose is not a scenario name.
if "/" in name or "." in name or name in NOT_SCENARIOS:
continue
claimed = m.group(2) is None
on_disk = (SCENARIO_DIR / f"{name}.json").exists()
if not claimed and not on_disk:
pass # marked (to write) and absent: consistent
out.append({"name": name, "claimed": claimed, "exists": on_disk})
return {"named": out, "note": raw.strip()}
def y(s, indent):
"""Emit one scalar as a YAML block string, no quoting games."""
pad = " " * indent
body = "\n".join(pad + " " + l for l in s.splitlines()) if s else ""
return ">-\n" + body if s else '""'
VERDICT_WORDS = {"pass", "fail", "blocked", "untested", "unknown"}
# Why a criterion is not passing. The plan's list, and nothing outside it.
REASON_KINDS = {
"code missing", "wiring missing", "configuration missing",
"deployment missing", "external dependency unavailable",
"scenario missing", "scenario fails",
"implementation exists with no runtime proof",
"deferred past v1", "not yet probed", "passes",
}
def load_verdicts():
if not VERDICTS.exists():
return {}
return json.loads(VERDICTS.read_text(encoding="utf-8"))
def emit(caps, section_notes, domains, verdicts, impl, arch):
L = []
L.append("# Capability ledger, target side.")
L.append("#")
L.append("# GENERATED by docs/capabilities/build_ledger.py from docs/spec.md.")
L.append("# Do not hand-edit. Domain assignment is the one human input and")
L.append("# lives in docs/capabilities/domains.yaml.")
L.append("#")
L.append("# Verification is per criterion, from verdicts.json. Implementation is")
L.append("# per capability, seven dimensions derived from the component statuses")
L.append("# in docs/architecture/maven-architecture.json through the mapping in")
L.append("# docs/capabilities/implementation.yaml. Never one boolean.")
L.append("")
L.append(f"source: docs/spec.md")
L.append(f"capability_count: {len(caps)}")
L.append(f"criterion_count: {sum(len(c['criteria']) for c in caps)}")
L.append("")
if section_notes:
L.append("section_findings:")
for sec, notes in section_notes.items():
L.append(f" - section: {sec!r}")
L.append(" finding: " + y(" ".join(notes), 4))
L.append("")
L.append("capabilities:")
for c in caps:
L.append(f" - id: {c['id']}")
L.append(f" title: {c['title']!r}")
L.append(f" section: {c['section']!r}")
L.append(f" scope: {c['scope']}")
d = domains.get(c["id"], [])
L.append(" domain: [" + ", ".join(d) + "]")
L.append(" state: " + y(c["state"], 4))
if arch:
dims = dimensions(c, impl.get(c["id"], []), arch, verdicts)
L.append(" implementation:")
for k in ("designed", "code_present", "wired", "configured",
"deployed", "reachable", "verified", "gap_class"):
# Quoted: bare yes/no are YAML booleans and the
# round-trip check reads them back as True/False.
L.append(f" {k}: {dims[k]!r}")
comps = impl.get(c["id"], [])
L.append(" components: [" + ", ".join(comps) + "]")
if c["finding"]:
L.append(" finding: " + y(c["finding"], 4))
if c.get("deferred_note"):
L.append(" deferred_note: " + y(c["deferred_note"], 4))
L.append(" scenarios:")
for s in c["scenarios"]["named"]:
L.append(f" - name: {s['name']}")
L.append(f" exists: {str(s['exists']).lower()}")
if s["claimed"] != s["exists"]:
L.append(" discrepancy: spec implies it exists and "
"cmd/mavend/testdata/scenarios has no such file")
L.append(" scenario_note: " + y(c["scenarios"]["note"], 4))
L.append(" criteria:")
for cr in c["criteria"]:
L.append(f" - id: {cr['id']!r}")
L.append(" text: " + y(cr["text"], 8))
v = verdicts.get(cr["id"])
if v is None and c["scope"] == "deferred":
v = {"verified": "untested", "reason": "deferred past v1"}
if v is None:
v = {"verified": "untested", "reason": "not yet probed"}
L.append(f" verified: {v['verified']}")
L.append(f" reason: {v['reason']!r}")
if v.get("detail"):
L.append(" detail: " + y(v["detail"], 8))
ev = v.get("evidence", [])
if ev:
L.append(" evidence:")
for e in ev:
L.append(f" - {e!r}")
L.append("")
return "\n".join(L) + "\n"
# --- Implementation dimensions -------------------------------------------
#
# Never one `implemented` boolean. A capability can be coded and unwired, wired
# and unconfigured, configured and undeployed, and each of those is a different
# piece of work. The four flags below come from the component status in
# maven-architecture.json, which was read from code, config and compose.
#
# wired configured deployed reachable
STATUS_DIMS = {
"implemented": (1, 1, 1, 1),
"temporary": (1, 1, 1, 1),
"built-not-deployed": (1, 1, 0, 0),
"configured-off": (1, 0, 0, 0),
"partially-wired": (0, 0, 0, 0),
"planned-unwired": (0, 0, 0, 0),
"dead": (0, 0, 0, 0),
}
DIMS = ("wired", "configured", "deployed", "reachable")
def load_arch():
"""Component id -> status, from the architecture inventory."""
if not ARCH.exists():
return {}
d = json.loads(ARCH.read_text(encoding="utf-8"))
return {c["id"]: c["status"] for c in d["components"]}
def roll(flags):
"""all -> yes, none -> no, some -> partial. Empty -> no."""
if not flags:
return "no"
if all(flags):
return "yes"
if not any(flags):
return "no"
return "partial"
def dimensions(cap, comps, arch, verdicts):
"""The seven dimensions for one capability. Never collapsed."""
known = [c for c in comps if c in arch]
out = {}
# designed: the spec states every one of these, so the question this
# dimension answers is narrower. Does a living doc own the subsystem.
st = cap["state"].lower()
if "no package" in st:
out["designed"] = "spec-only"
elif "no living doc" in st or "no capture client" in st:
out["designed"] = "spec-only"
else:
out["designed"] = "yes"
out["code_present"] = roll([1] * len(known)) if comps else "no"
for i, name in enumerate(DIMS):
out[name] = roll([STATUS_DIMS[arch[c]][i] for c in known])
vs = [verdicts.get(cr["id"], {}).get("verified", "untested")
for cr in cap["criteria"]]
# The gap class, for docs/capabilities/gaps.md. Order matters: nothing built
# outranks nothing reachable, which outranks something reachable and wrong.
if out["code_present"] == "no":
out["gap_class"] = "capability missing"
elif out["reachable"] != "yes":
out["gap_class"] = "capability exists but unreachable"
elif any(v == "fail" for v in vs):
out["gap_class"] = "capability partial"
elif all(v == "pass" for v in vs):
out["gap_class"] = "none"
else:
out["gap_class"] = "capability exists but unverified"
if vs and all(v == "pass" for v in vs):
out["verified"] = "yes"
elif any(v == "pass" for v in vs):
out["verified"] = "partial"
else:
out["verified"] = "no"
return out
def load_flat(path):
"""`key: [a, b]` per line, # comments stripped. domains and implementation."""
if not path.exists():
return {}
out = {}
for line in path.read_text(encoding="utf-8").splitlines():
line = line.split("#", 1)[0].strip()
if not line or ":" not in line:
continue
k, v = line.split(":", 1)
vals = [x.strip() for x in v.strip().strip("[]").split(",") if x.strip()]
out[k.strip()] = vals
return out
def main():
caps, section_notes = parse()
domains = load_flat(DOMAINS)
impl = load_flat(IMPL)
arch = load_arch()
verdicts = load_verdicts()
errs = []
seen = {}
for c in caps:
for cr in c["criteria"]:
if cr["id"] in seen:
errs.append(f"criterion id collision: {cr['id']}")
seen[cr["id"]] = cr["text"]
if not c["criteria"]:
errs.append(f"{c['id']}: no DoD criteria extracted")
if not c["state"]:
errs.append(f"{c['id']}: no State line extracted")
d = domains.get(c["id"])
if domains:
if not d:
errs.append(f"{c['id']}: no domain assigned")
elif len(d) > 2:
errs.append(f"{c['id']}: {len(d)} domains, max is 2")
else:
for x in d:
if x not in DOMAIN_NAMES:
errs.append(f"{c['id']}: unknown domain {x!r}")
cap_ids = {c["id"] for c in caps}
for k in domains:
if k not in cap_ids:
errs.append(f"domains.yaml names unknown capability {k!r}")
# The mapping is the whole basis of the implementation columns. A capability
# missing from it reads as `no` on every dimension, which is indistinguishable
# from a capability nothing carries. Refuse rather than guess which.
if impl:
if not arch:
errs.append("implementation.yaml is present and "
"docs/architecture/maven-architecture.json is not")
for k in impl:
if k not in cap_ids:
errs.append(f"implementation.yaml names unknown capability {k!r}")
for c in caps:
if c["id"] not in impl:
errs.append(f"{c['id']}: no row in implementation.yaml")
for k, comps in impl.items():
for comp in comps:
if arch and comp not in arch:
errs.append(f"{k}: unknown component {comp!r}")
for comp, st in arch.items():
if st not in STATUS_DIMS:
errs.append(f"maven-architecture.json: unknown status {st!r} on {comp}")
# A verdict cites evidence by path, and a path that resolves to nothing is
# worse than no citation: it reads as verified and is not. Section refs are
# checked too, because writing "§ Something" that no heading matches is the
# easy way to make an unsupported claim look sourced.
for cid, v in verdicts.items():
for e in v.get("evidence", []):
path, _, section = e.partition(" § ")
# An evidence string is "<path>[:line] [locator]" or
# "<path> § <heading>". The locator points inside the file
# (a probe id, a readback key) and is not part of the path.
path = path.strip().split()[0].split(":")[0]
f = ROOT / path
if not f.exists():
errs.append(f"{cid}: evidence path does not exist: {path}")
elif section and f.suffix == ".md" and section.strip() not in f.read_text(encoding="utf-8"):
errs.append(f"{cid}: evidence names a section not in {path}: {section}")
for cid, v in verdicts.items():
if cid not in seen:
errs.append(f"verdicts.json scores unknown criterion {cid!r}")
elif v.get("verified") not in VERDICT_WORDS:
errs.append(f"{cid}: verdict {v.get('verified')!r} is not one of {sorted(VERDICT_WORDS)}")
elif v.get("reason") not in REASON_KINDS:
errs.append(f"{cid}: reason {v.get('reason')!r} is not one of the plan's kinds")
elif v["verified"] == "pass" and v["reason"] != "passes":
errs.append(f"{cid}: a pass carries reason {v['reason']!r}")
elif v["verified"] != "pass" and v["reason"] == "passes":
errs.append(f"{cid}: reason 'passes' on a {v['verified']} verdict")
elif v["verified"] == "fail" and v["reason"] == "implementation exists with no runtime proof":
# That reason means nothing was observed. A fail was observed, or it
# is not a fail. Mixing them is how a wrong diagnosis survives.
errs.append(f"{cid}: a fail cannot rest on 'no runtime proof'")
# The invariants exist twice on purpose: prose and evidence in the .md, the
# machine-readable half in the .yaml for the viewer. They drift the moment
# nothing checks them, so check them.
if INV_YAML.exists():
try:
import yaml as _y
except ImportError:
errs.append("pyyaml absent: invariants.yaml was not checked")
else:
inv = _y.safe_load(INV_YAML.read_text(encoding="utf-8"))["invariants"]
md = INV_MD.read_text(encoding="utf-8") if INV_MD.exists() else ""
if not md:
errs.append("invariants.yaml exists and invariants.md does not")
n_md = md.count("\n## ") - md.count("\n## What this file")
if md and n_md != len(inv):
errs.append(f"invariants: {len(inv)} in the yaml, {n_md} headings in the md")
for iv in inv:
head = f"## {iv['id']}. {iv['title']}"
if md and head not in md:
errs.append(f"invariant {iv['id']}: no heading {head!r} in invariants.md")
if iv["mark"] not in {"explicit", "implied", "unresolved"}:
errs.append(f"invariant {iv['id']}: mark {iv['mark']!r} is not one of three")
for c in iv["capabilities"]:
if c not in cap_ids:
errs.append(f"invariant {iv['id']}: unknown capability {c!r}")
for c in iv["components"]:
if arch and c not in arch:
errs.append(f"invariant {iv['id']}: unknown component {c!r}")
if iv["mark"] == "unresolved" and not iv.get("question"):
errs.append(f"invariant {iv['id']}: unresolved with no product question")
OUT.write_text(emit(caps, section_notes, domains, verdicts, impl, arch), encoding="utf-8")
# The emitter hand-writes YAML, so it can produce something that reads fine
# and does not parse. It did once: evidence came out as a bare list item
# inside a mapping. Parse what was just written.
try:
import yaml
except ImportError:
errs.append("pyyaml absent: the output was written without a parse check")
else:
try:
doc = yaml.safe_load(OUT.read_text(encoding="utf-8"))
except yaml.YAMLError as e:
errs.append(f"the emitted ledger is not valid YAML: {e}")
else:
n = sum(len(c["criteria"]) for c in doc["capabilities"])
if n != len(seen):
errs.append(f"round trip lost criteria: wrote {len(seen)}, read back {n}")
for c in doc["capabilities"]:
if c["scope"] != "v1":
continue
for cr in c["criteria"]:
if cr["verified"] != "untested" and not cr.get("evidence"):
errs.append(f"{cr['id']}: scored {cr['verified']} with no evidence")
print(f"{len(caps)} capabilities, {len(seen)} criteria -> {OUT.relative_to(ROOT)}")
print(f" v1: {sum(1 for c in caps if c['scope'] == 'v1')}, "
f"deferred: {sum(1 for c in caps if c['scope'] == 'deferred')}")
scen = {s["name"]: s["exists"] for c in caps for s in c["scenarios"]["named"]}
ghosts = sorted(n for n, e in scen.items() if not e
and any(s["claimed"] for c in caps for s in c["scenarios"]["named"] if s["name"] == n))
if ghosts:
print(f" named as existing but absent from disk: {', '.join(ghosts)}")
print(f" scenarios named: {len(scen)}, existing: {sum(scen.values())}, "
f"to write: {len(scen) - sum(scen.values())}")
if verdicts:
from collections import Counter
tally = Counter(v["verified"] for v in verdicts.values())
print(" verdicts: " + ", ".join(f"{k} {n}" for k, n in sorted(tally.items())))
if arch:
from collections import Counter as _C
for k in ("code_present", "wired", "configured", "deployed", "reachable"):
t = _C(dimensions(c, impl.get(c["id"], []), arch, verdicts)[k] for c in caps)
print(f" {k}: " + ", ".join(f"{a} {n}" for a, n in sorted(t.items())))
# A capability nothing carries that still scores a pass. Always a
# negative criterion passing by absence. Worth seeing, not an error.
for c in caps:
d_ = dimensions(c, impl.get(c["id"], []), arch, verdicts)
if d_["code_present"] == "no" and d_["verified"] != "no":
print(f" ANOMALY {c['id']}: nothing carries it and it scores "
f"verified={d_['verified']} (a negative criterion passing by absence)")
t = _C(dimensions(c, impl.get(c["id"], []), arch, verdicts)["gap_class"]
for c in caps if c["scope"] == "v1")
print(" v1 gap classes: " + ", ".join(f"{a} {n}" for a, n in sorted(t.items())))
used = {x for v in impl.values() for x in v}
orphan = sorted(set(arch) - used)
print(f" components serving no capability: {len(orphan)}")
for o in orphan:
print(f" {o} ({arch[o]})")
print(f" no living doc: {sum(1 for c in caps if 'No living doc' in c['state'] or 'no living doc' in c['state'].lower())}")
if errs:
print("\nERRORS:", file=sys.stderr)
for e in errs:
print(" " + e, file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
+166
View File
@@ -0,0 +1,166 @@
# Domain assignment for the capability ledger. The one human input to
# build_ledger.py; everything else in ledger.yaml is mechanical.
#
# At most two domains, PRIMARY FIRST. Three independent passes ran under
# different lenses (bottom-up from the DoD, from the owner's experience,
# from state ownership and effect), then one reconciler re-read the DoD of
# every contested row and broke the tie. A row marked (contested) is one the
# three passes did not agree on; its note names what broke the tie.
#
# 35 of 51 were unanimous. Goes to the owner once, before any probe runs.
# Unanimous: every criterion interprets an utterance into intent plus source and records which stage decided.
route-an-utterance: [deliberation]
# Contested; broken by criterion count: two of three DoD lines are parked-turn dialogue lifecycle (cancel by "отмена", survive an interleaved turn and resume), and none is a permission or confirmation, so interaction beats governance for second. (contested)
ask-instead-of-guessing: [deliberation, interaction]
# Unanimous: every criterion polices what wording reaches the outbound wire.
speak-as-herself: [interaction]
# Contested; broken by criterion dbeb, which is the privacy ordering rule (owner's sources before anything outside, every time), not an interpretation step, so governance beats deliberation. (contested)
answer-from-your-own-data: [memory, governance]
# Unanimous: retrieve an external answer, bounded by what may leave the box.
answer-from-the-world: [action, governance]
# Unanimous: pick the right Kiwix book and retrieve a topically correct article — pure retrieval.
read-an-encyclopedia: [action]
# Unanimous: call a configured provider, with the follow-up city parked as a clarify rather than guessed.
weather: [action, deliberation]
# Contested; I overrule the two perception-first votes on the ledger's own weather logic: the DoD is an offloaded vision tool call over content he supplied, with a silent fallback, so it is a tool call first and sensing second. (contested)
see-an-image: [action, perception]
# Contested; broken by reading the criteria: write, honest confirmation, supersede and Nexus-resolved subject are all fact-store integrity, and none is a permission, privacy or confirmation-binding rule, so governance drops. (contested)
facts: [memory]
# Contested; broken by criterion count: capture, recall and delete are the store's lifecycle, and "a question is not stored as a statement" is a routing defect already owned by route-an-utterance, so deliberation drops. (contested)
notes: [memory]
# Unanimous across all three passes: it reads the note and fact store through
# the embedder, with the personal boundary deciding what that read may cross
# into. Recovered from the three passes' journal: the reconciler dropped this
# row from its 51, and build_ledger.py's guard caught the omission.
recall: [memory, governance]
# Contested; broken by the ledger finding that the evaluator cannot speak: nothing surfaces to him, so initiative cannot hold, and the DoD's checkable conclusions over notes it read are deliberation. (contested)
memory-evaluation: [memory, deliberation]
# Unanimous: hold a future commitment, fire it at its time, and get it delivered across reaches.
reminders: [attention, interaction]
# Unanimous: the whole DoD is whether an unprompted item may break in, keyed on presence and severity.
interruption-policy: [initiative, perception]
# Unanimous: a suppressed nudge candidate must resurface unprompted in a later digest, without acting.
digest-of-held-nudges: [initiative, attention]
# Unanimous: an unprompted plan inside its window that must find another reach rather than be dropped.
morning-routine: [initiative, interaction]
# Unanimous: propose from observed repeated behaviour and store the decline.
routine-proposals: [initiative, memory]
# Unanimous: open items ordered by deadline and urgency over a stored work list.
tasks: [attention, memory]
# Unanimous: fetch configured feeds and find the matching item on request, explicitly never unprompted.
rss-and-news: [action]
# Unanimous: outbound delivery with an outbox row and continuous inbound reading — a reach.
telegram: [interaction]
# Unanimous: a push reach whose criteria are its credential and not looping when refused.
ntfy: [interaction]
# Unanimous: a live speech reach, capped at L0 and bound to loopback.
voice: [interaction, governance]
# Unanimous: a page per capability with step-up standing in front of a destructive write.
web-ui: [interaction, governance]
# Unanimous: inbound ingests desktop events as low-confidence facts; the outbound half is an undecided fourth reach.
desk-notifications: [perception, interaction]
# Contested; broken by the DoD being accuracy-enough-to-route plus a silent fallback arm on the voice surface, with nothing about context, so interaction leads and perception stays second. (contested)
speech-to-text: [interaction, perception]
# Unanimous: the reply rendered as Russian speech with times and numbers expanded for the ear.
text-to-speech: [interaction]
# Contested; broken by criterion 10ca being the detection itself ("Мэйвен" wakes her, a near-miss does not) — the session it opens belongs to voice, so perception leads. (contested)
wake-word: [perception, interaction]
# Contested; broken by the DoD being a microphone capture client existing at all — sensing that must exist before any surface — so perception leads over the reach it feeds. (contested)
hearing: [perception, interaction]
# Contested (order only); broken by criterion 3110 being the recognition itself, with the act-path gate the second criterion it feeds, so perception leads. (contested)
speaker-recognition: [perception, governance]
# Unanimous: free text resolves to a canonical entity or she asks, and that resolution precedes any mutating call.
nexus: [deliberation, governance]
# Unanimous: the source of items needing attention, bound by lifecycle words and the no-auto-act rule.
praxis: [attention, governance]
# Unanimous: the only path that changes the world, bound by a confirmation LLM output cannot supply.
hexis: [action, governance]
# Unanimous: control a device through Hexis on a Nexus-resolved entity, never by free text.
smart-home: [action, governance]
# Contested; broken by what the rate-limit criterion actually is — a politeness bound in config, not an authorization gate — so governance drops and action stands alone. (contested)
network-scans: [action]
# Unanimous: connect and disconnect a paired radio device through the act path.
bluetooth-control: [action]
# Unanimous: external tools callable through the act path with the allowlist as the only door.
mcps: [action, governance]
# Unanimous: compose services on the current build and a lossless restart — infrastructure, filed under Operations.
the-deployed-stack: [operations]
# Contested; broken by all three criteria being secrecy mechanisms (at-rest encryption, key held only by mavend, passwords from files), which is governance's privacy clause carried by infrastructure. (contested)
encrypted-database: [operations, governance]
# Unanimous: no privileged gate fail-open and step-up per-request — authorization, filed under Operations.
passkey-and-step-up: [governance, operations]
# Unanimous: which gguf may load and whether the swap survives a restart.
model-swap: [operations]
# Unanimous: updating the deployment from inside it and rolling back a failure.
self-update: [operations]
# Unanimous: the build and analyzer gate itself, no agent behaviour in it.
tests-and-analyzers: [operations]
# Unanimous: a mail workflow whose open criteria are candidates staying candidates and no content leaving the box.
email-triage: [action, governance]
# Contested; broken by criteria 4047 and 5f7c both refusing to proceed on an under-determined request (ask for the slot, name the conflict), which is deliberation, not proactive attention. (contested)
calendar-management: [action, deliberation]
# Contested; broken by criterion count: two of four are constraints (robots and politeness, only the URL and utterance leave) against one for the watch, so governance takes second — the watch does pull at initiative. (contested)
web-crawling: [action, governance]
# Unanimous: pull a named source and condense it, refusing a summary that would invent content.
summaries: [action, governance]
# Contested; broken by criterion 5134 naming authentication for both directions explicitly, which outweighs the inbound half's perception flavour. (contested)
webhooks: [interaction, governance]
# Unanimous: a user-set schedule that runs acts under the same confirmation rules as a spoken act.
cron-jobs: [action, governance]
# Unanimous: a correction stored as a readable, deletable outcome whose only effect is later phrasing.
learning-the-style: [memory, interaction]
# Unanimous: stored outcomes from dismissals and repairs that change the next decision.
learning-from-mistakes: [memory, deliberation]
# Contested; broken by criterion 77b5 stating that a chain containing an act confirms each act separately, an explicit confirmation rule that outranks the generic "performs both" pull toward action. (contested)
command-chaining: [deliberation, governance]
+241
View File
@@ -0,0 +1,241 @@
# Gaps: what is missing, and what the architecture does about it
Hand-written, except the four capability classes, which are derived.
This file compares responsibilities. It never compares package names. A package
existing is not a capability, and a capability can be spread over six packages
and still be missing.
## Where each class comes from
Classes 1 through 4 are the `gap_class` field in `docs/capabilities/ledger.yaml`,
derived from the seven implementation dimensions and the criteria verdicts.
Rebuild them:
```sh
python3 docs/capabilities/build_ledger.py
```
Classes 5 through 8 are read from `docs/architecture/findings.md` and
`docs/capabilities/invariants.md`. **Every entry names the capability or
invariant it affects.** An entry affecting neither is marked non-blocking
cleanup, in those words, and it is the whole of class 8.
Counts are over the 46 v1 capabilities. The 5 deferred ones are excluded.
---
## 1. Capability missing (5)
Nothing carries it. `code_present: no`.
| capability | criteria | note |
| --- | --- | --- |
| `summaries` | 0 pass, 3 fail | no package. Wanted by `email-triage`, `web-crawling`, `hearing` and `rss-and-news`, each of which would consume it |
| `webhooks` | 0 pass, 3 fail | no package. Telegram's own inbound channel is mapped to `telegram`, not here |
| `command-chaining` | 0 pass, 3 fail | no package. The `chain` in `internal/router` is the world chain and the source chain |
| `learning-the-style` | 1 pass, 2 fail | the pass is a negative criterion satisfied by absence. The build reports it as an anomaly |
| `learning-from-mistakes` | 0 pass, 2 fail | no package |
The last two are invariant 10, and it is `unresolved`. Whether behavioural
learning is wanted is a product question, so these two are not automatically
work.
## 2. Capability partial (21)
Reachable, and at least one criterion was observed failing. This is the class
that matters most, because a user can get to all 21 today and 21 misbehave.
`route-an-utterance`, `ask-instead-of-guessing`, `answer-from-your-own-data`,
`answer-from-the-world`, `read-an-encyclopedia`, `facts`, `notes`, `recall`,
`reminders`, `voice`, `web-ui`, `desk-notifications`, `wake-word`, `nexus`,
`praxis`, `the-deployed-stack`, `encrypted-database`, `passkey-and-step-up`,
`tests-and-analyzers`, `web-crawling`, `cron-jobs`.
`wake-word` is the sharpest: reachable on every dimension and 0 of 2 criteria
pass.
## 3. Capability exists but unreachable (9)
Built, and the deployed configuration does not reach it.
| capability | why | class of fix |
| --- | --- | --- |
| `speak-as-herself` | `reachable: partial` on `core.model_seam` | configuration |
| `weather` | no `weather` key in the deployed `voice` block | configuration |
| `see-an-image` | no media block | configuration |
| `memory-evaluation` | worker is `configured-off` | configuration |
| `ntfy` | present in the config and disabled there | configuration |
| `hearing` | `capture.enabled` false and no capture client ships (V-514) | configuration and code |
| `mcps` | no MCP server configured, and V-478 blocks the one candidate | deployment |
| `email-triage` | `mavmaild` is not in `docker-compose.yml` | deployment |
| `calendar-management` | `mavcaldav` is not in `docker-compose.yml` | deployment |
None of these nine is a code defect. Seven are one config block and two are one
compose entry. `docs/spec.md` says this about the audit's four and it still
holds for these nine.
## 4. Capability exists but unverified (11)
Reachable, nothing observed failing, and not all criteria pass. These are
measurement gaps, not defects.
`interruption-policy`, `digest-of-held-nudges`, `morning-routine`,
`routine-proposals`, `tasks`, `rss-and-news`, `telegram`, `speech-to-text`,
`text-to-speech`, `hexis`, `network-scans`.
Four of them, the whole Proactive cluster, are untested on every criterion,
because a proactive behaviour cannot be probed by sending an utterance. That is
the shape of the gap and it needs a different harness, not more probes.
---
## 5. Duplicated mechanism (6)
| what | affects | owned? |
| --- | --- | --- |
| Two independent arbitrations decide one turn: seven intents, then twenty-two ordered query sources (`findings.md` 2.1) | invariant 11, `route-an-utterance`, `answer-from-your-own-data` | no |
| A third arbitration runs before both: eleven stateful pre-emptors in the pre-route ladder (`findings.md` 2.2) | invariant 11, invariant 6, `ask-instead-of-guessing` | no |
| Two tier systems. `internal/auth` does not bind the turn path, `internal/tool` is not keyed on the reach (`findings.md` 6.3, 6.3b) | invariant 8, `hexis`, `passkey-and-step-up` | no |
| Two representations of reach, both ignored (`findings.md` 6.3) | invariant 8, `voice` | no |
| Two digest mechanisms with the same word in the name, flushed six lines apart (`findings.md` 2.4) | invariant 5, `digest-of-held-nudges` | no |
| Restraint decided twice: the gate decides whether a rule emits, delivery decides where it lands (`findings.md` 2.3) | `interruption-policy` | **yes**, argued in `channel.go` |
The last row is duplication that is owned. It is listed so it is not
rediscovered as a defect.
## 6. Missing shared mechanism (6)
| what is missing | affects |
| --- | --- |
| A single point that decides whether this origin may perform this effect with this evidence. `origin × effect × evidence → permit` is the target and nothing computes it | invariant 8, `hexis`, `praxis`, `voice`, `passkey-and-step-up` |
| An owner for a key namespace. `facts` has nine writers, `notes` six, `tools` three unrelated proposers (`findings.md` 1.1, 1.2, 1.3) | invariant 2, `facts`, `notes`, `recall` |
| A comparable unit of evidence, so claimants can answer "is this more mine than yours?". `internal/claim` is that unit, written, tested and called by nothing (`findings.md` 6.1) | invariant 11, `command-chaining` |
| A conversation that spans reaches. `mavweb` instead hardcodes one conversation id for the whole web reach | invariant 1, `web-ui`, `voice`, `telegram` |
| A stated rule for what survives a restart. Six stores, six independent choices, two of them argued | invariant 12, `ask-instead-of-guessing` |
| A summariser. Four capabilities would consume one and none exists | `summaries`, `email-triage`, `web-crawling`, `hearing` |
## 7. Current architecture conflicts with target behavior (10)
The class where the code works as written and the written thing is not what the
spec asks for.
| conflict | affects |
| --- | --- |
| Four silent degradations stack on one turn, and `docs/spec.md` writes every v1 DoD at "honest" (`findings.md` 8.1) | invariant 7, `answer-from-the-world`, `speech-to-text`, `route-an-utterance`, `speak-as-herself` |
| `praxisItemAction.handle` calls straight through: acknowledge, resolve, ignore and pin run on first hearing with no tier and no confirm turn (`findings.md` 6.3c, `cmd/mavend/ecosystem_acts.go:158`) | invariant 8, `praxis` |
| `weather` is a live query source with `guesses: true` and the deployed config selects no provider, so it can claim a turn and answer from a stub (`findings.md` 11.3) | invariant 7, `weather` |
| `loop.State.CalendarBusy` reads facts `mavcaldav` never writes, so the do-not-nag-mid-meeting suppressor is permanently false (`findings.md` 8.2) | invariant 3, `interruption-policy`, `calendar-management` |
| Recurring reminders have a column, an IPC parameter and no caller. `actionReminder` passes `""` (`findings.md` 8.3) | `reminders`, `cron-jobs` |
| The clarify store is not persisted and the expired-clarify notice reads the store that is gone (`findings.md` 7.5) | invariant 6, invariant 12, `ask-instead-of-guessing` |
| `Claim.Coverage` returns 1.0 for a claim that extracted nothing (`findings.md` 6.3d) | invariant 11. Latent: it corrupts the fix for class 6 row 3 before that fix ships |
| The act executor runs inside the key holder, and the process boundary is not one of the controls (`findings.md` 5.4) | invariant 8, `hexis`, `encrypted-database` |
| The voice wire's whole security argument is external: loopback publish plus an ssh tunnel, so one compose edit removes it (`findings.md` 5.5) | invariant 8, `voice` |
| `make test` is green with the four `TestONNX*` measurements silently skipped, because the recipe does not set `MAVEN_ONNX_LIB` | `tests-and-analyzers`, `recall`. It is why the predecessor audit had a green suite and 22 dead capabilities |
## 8. Architecture concern with no current product impact (11)
**Every row here is non-blocking cleanup.** None names a capability or an
invariant, which is the test for belonging in this class rather than in 5, 6 or
7.
- `reactiveHandler` has 34 fields (`findings.md` 4.1).
- `runTurn` is one function with eleven early returns (4.2).
- `tick` runs thirteen jobs in one function (4.3).
- `wireVoice` is one constructor for seventeen subsystems (4.4).
- `mavsttd` and `mavttsd` are separate processes at a scale that does not need it (5.1).
- Three IPC connections from one process (5.2).
- A construction cycle between the API layer and the turn layer (3.1).
- The handler holds the raw store beside the mediated one (3.2).
- `queryDayPlan` reads the proactive scheduler, the single call across that line (3.4).
- `internal/modes` is imported by nothing outside itself (6.2).
- The daemon is wired twice, in two places (8.6).
Two entries were considered for this class and moved out. `queryNetwork`
triggering a live LAN scan inside a read path (6.4) affects `network-scans`,
whose two remaining criteria are `unknown`. `actionFact` re-routing into the
query chain (6.5) affects `facts` and `route-an-utterance`, which is where the
"меня зовут Ками" misroute lives.
---
# Priority
One list. The rank is the plan's, and it is about impact today, not about how
ugly the code is. An unwired or unreachable future defect never outranks a live
user-visible failure because its architecture is offensive.
## 1. Prevents intended everyday use today
1. **`speak-as-herself` fails all three criteria.** The deployed resident model,
`maven-instruct-b2-Q4_K_XL`, produces Russian sentences that no longer hold
together, and the phrasing checks that would catch it run in the eval and not
on the outbound path. Everything that asks the model to write a sentence
inherits this. Formal `вас` and `вы` reached the wire while `CheckFeminine`
passed.
2. **His own name is not stored as a fact.** "меня зовут Ками" routes to chat,
so nothing is written, and "что ты помнишь обо мне?" routes to chat too.
Two of the seven audit probes, still broken and now broken differently.
3. **`wake-word` fails both criteria while reachable on every dimension.** Voice
is the spine of v1 and the always-on half of it does not work.
4. **Nine capabilities are one config block or one compose entry from
reachable.** `weather`, `ntfy`, `see-an-image`, `memory-evaluation`,
`email-triage`, `calendar-management` and `mcps` are the cheap ones. This is
the highest ratio of capability to work in the whole list.
## 2. Makes existing behavior incorrect or unreliable
5. **The Praxis lifecycle path has no gate.** Four remote mutations run on first
hearing. This is live today and needs no new wiring to matter.
6. **`weather` answers from a stub and is allowed to claim the turn.** A source
marked `guesses: true` with no provider is worse than a named gap.
7. **The busy suppressor is permanently false.** Every interruption decision
that should have deferred to a meeting fails open.
8. **Four silent degradations stack**, and nothing in a reply distinguishes the
worst case from the best. Invariant 7 has no written boundary between "only
better" and "cannot do the job".
9. **A parked clarify survived five consecutive turns** and was released by a
path other than `отмена`. Invariant 6 says nobody owns closing it.
## 3. Blocks multiple capabilities
10. **No single authorization point.** Invariant 8, `unresolved`, and the third
of the three questions the freeze was called to answer. It blocks `hexis`,
`praxis`, `voice` and `passkey-and-step-up`, and it is the one property
nobody can currently state.
11. **No owner for a key namespace.** A fetch watermark and a tuning parameter
live in the table recall embeds and `queryFactByKey` reads back as an
answer.
12. **No comparable unit of evidence.** Three ordered lists decide one turn.
`command-chaining` cannot be built on top of them, and `internal/claim`
carries a live defect before it is wired.
13. **No summariser.** Four capabilities would consume one.
## 4. Prevents verification
14. **`make test` is green with four measurements skipped.** The recipe does not
set `MAVEN_ONNX_LIB`. This is the exact trap `CLAUDE.md` describes, and the
baseline walked into it while measuring whether other things had.
15. **The whole Proactive cluster is untested on every criterion.** A proactive
behaviour cannot be probed by sending an utterance. It needs a clock-driving
harness, not more probes.
16. **26 of 31 named scenarios do not exist on disk.** Only 5 of 51 spec entries
cite a scenario that is there (`findings.md` 9.5).
17. **`POST /api/ptt` was called unreachable in an earlier draft and is not.**
Four speech criteria were filed `deployment missing` when the deployment is
present and the probe was never written.
## 5. Architectural cleanup with no present user impact
18. Everything in class 8, in any order. None of it blocks a capability or an
invariant, and that is why it is last.
---
## What this file does not do
It does not schedule. `docs/roadmap.md` orders the work and this file feeds it.
It does not decide the four `unresolved` invariants. Authority and confirmation,
learning from outcomes, capability composition and the shelf life of a held
nudge are the owner's, and items 10, 12 and 13 above stall on them.
+88
View File
@@ -0,0 +1,88 @@
# Capability -> component mapping. HAND-WRITTEN. This is the judgment call.
#
# Component ids come from docs/architecture/maven-architecture.json, whose
# `status` field was read from code, config and compose and audited against
# them. build_ledger.py derives the six implementation dimensions from those
# statuses and refuses an id that file does not carry.
#
# What is mapped is what CARRIES the capability, never the infrastructure every
# capability shares. core.reactive_handler, core.wiring, core.action_table,
# core.daemon_api and bnd.ipc are deliberately absent: mapping them everywhere
# would give all 51 rows the same status and say nothing.
#
# An empty list means no component carries it. That is the finding, not a hole
# in this file.
# --- The turn ---
route-an-utterance: [router.cascade, router.stage0, router.heads, router.llm, router.classifier, router.embedder, router.extractor, core.turn_route, core.topics, core.decision_trace, state.decision_ring, state.routing_traces, state.routing_labels]
ask-instead-of-guessing: [core.preroute, state.clarify_store, state.dialogue_sessions]
speak-as-herself: [core.phraser, core.replier, core.action_chat, core.model_seam, svc.llama_server, eval.phrasing]
answer-from-your-own-data: [core.query_chain, core.q.embed, core.q.memory, core.q.factbykey, core.q.notes, core.q.history, core.q.list, core.q.self, core.q.personal, state.list_items]
answer-from-the-world: [core.q.search, core.q.web, core.q.general, core.q.personal, ext.searxng]
read-an-encyclopedia: [core.q.kiwix, ext.kiwix]
weather: [core.q.weather, ext.openmeteo]
see-an-image: [core.vision, state.media_blobs]
# --- Memory ---
facts: [state.facts, core.action_fact, core.fact_enrichment, core.store_api]
notes: [state.notes, core.action_note]
recall: [core.recall, state.memory_vectors, router.embedder, core.q.memory, core.q.notes]
memory-evaluation: [core.memory_eval]
# --- Proactive ---
reminders: [state.reminders, core.action_reminder, core.dispatcher, state.delivery_attempts]
interruption-policy: [core.rules, core.dispatcher, core.gatherer, state.presence_state, state.nudges, state.tick_memo]
digest-of-held-nudges: [state.digest_entries, core.tick_loop, core.rules]
morning-routine: [core.morning, core.q.dayplan]
routine-proposals: [core.pattern, core.routines, state.proposed_routines, state.events]
tasks: [state.tasks, core.q.tasks]
rss-and-news: [core.feed_worker, core.q.feeds]
# --- Reach ---
telegram: [core.sink_telegram, core.telegram_intake, ext.telegram, state.ack_sends]
ntfy: [core.sink_ntfy, ext.ntfy]
voice: [core.voice_server, bnd.voice_tcp, core.sink_voice, proc.mavenclient]
web-ui: [proc.mavweb, bnd.http_web]
desk-notifications: [core.event_bus, proc.mavweb]
# --- Speech and senses ---
speech-to-text: [core.stt_seam, proc.mavsttd, ext.whispercpp, ext.cw2_stt, bnd.worker]
text-to-speech: [core.tts_seam, proc.mavttsd, ext.piper, bnd.worker]
wake-word: [proc.mavwaked, cfg.systemd, ext.alsa]
hearing: [core.capture, state.media_blobs]
speaker-recognition: [core.speaker]
# --- The ecosystem ---
nexus: [ext.nexus, core.ecosystem, bnd.http_ecosystem, state.ecosystem_traces]
praxis: [ext.praxis, core.ecosystem, core.praxis_acts, core.q.attention, state.surfaced_items, state.ecosystem_traces]
hexis: [ext.hexis, core.ecosystem, core.ecosystem_hexis_gate, core.action_act, core.risk_policy, state.tools, state.pending_act, state.ecosystem_traces]
smart-home: [ext.homeassistant, core.home_worker, core.q.home]
network-scans: [core.netscan, core.q.network]
# No package, no component. The finding, not an omission.
bluetooth-control: []
mcps: [core.mcp_worker, ext.vikunja_mcp]
# --- Operations ---
the-deployed-stack: [cfg.compose, cfg.mavend, proc.mavend, proc.mavweb, proc.mavsttd, proc.mavttsd, proc.mavpoll, proc.mavgpud, ext.netdata, ext.uptimekuma]
encrypted-database: [state.db_file, state.db, state.db_tmpfs, proc.mavseal]
passkey-and-step-up: [state.wrapped_key, state.passkey_file, core.daemon_lock, core.auth_gate, bnd.http_web]
model-swap: [core.modelswap, svc.llama_server]
self-update: [proc.mavupdate]
tests-and-analyzers: [eval.gates, eval.router, eval.phrasing]
# --- Undesigned in v1 ---
email-triage: [proc.mavmaild, core.mail_intake, state.maildata]
calendar-management: [proc.mavcaldav, core.q.calendar]
web-crawling: [core.crawl_worker, core.q.web]
summaries: []
# Empty on purpose. core.telegram_intake is Telegram's own inbound channel and
# is mapped to `telegram`. Mapping it here too would make this row read as
# built and deployed when all three of its criteria fail on code missing.
webhooks: []
cron-jobs: [core.routines, core.tick_loop]
learning-the-style: []
# state.routing_labels holds owner corrections of a route and is deliberately
# NOT mapped here. It is route learning, not behavioural learning, and mapping
# it would make this row read as partially built when nothing reads it back.
learning-from-mistakes: []
command-chaining: []
+281
View File
@@ -0,0 +1,281 @@
# The cross-cutting rules the 51 capabilities imply
Hand-written. The one file in this directory that is not generated.
`docs/spec.md` states 51 capabilities one at a time. Twelve rules run across all
of them, and no capability's definition of done states any of these. A rule
broken here breaks many capabilities at once, which is why it does not show up
as one failing criterion.
Each rule carries a mark. A rule split between what is written down and what
is not carries both, and says which half is which.
| mark | meaning |
| --- | --- |
| `explicit` | a source states the rule and names its enforcement point |
| `implied` | capabilities depend on it, no source states it, and the code decides it case by case |
| `unresolved` | the sources do not answer it. A product question, not a defect |
Nothing wanted is invented where the sources are silent. An `unresolved` rule
needs the owner, not a commit.
Evidence is `docs/architecture/findings.md` for the code reading,
`docs/evals/2026-08-26-capability-baseline.md` for what ran, and the file itself
where the rule is written down.
---
## 1. Continuity across turns and across reaches
**Implied.** Continuity within one reach is built. Continuity across reaches is
not, and nothing states whether it should be.
`dialogue.NewPersistentSessionStore` carries follow-up slots across turns and
across a restart. The clarify store is a per-reach stack and is deliberately not
persisted (`findings.md` 7.5, Vikunja #385).
Across reaches there is no shared thread. `mavweb` hardcodes one conversation id
for the whole web reach, which is not continuity but the absence of separation:
a clarify parked by one probe was still parked for the next, and the first field
run had to be discarded for it (`docs/capabilities/README.md`, "Two things the
harness learned the hard way").
**What breaks:** a question asked by voice and answered on the web has no thread
to attach to. No capability's DoD asks for one, so nothing scores this.
**The product question:** is a conversation per reach, or one conversation the
reaches are windows onto?
## 2. Memory and correction semantics
**Explicit for the row, implied for the namespace.**
Supersede is written down and enforced: a correction points `voids_id` at the row
it replaces, and valid-time is the `ts` column (`internal/store/schema.sql`).
`CLAUDE.md` states the embedder contract, `EmbedQuery` and `EmbedPassage`, and
calling plain `Embed` on a note is named as a bug.
Who may write a key is not written anywhere. `facts` has nine writers and no
owner, and two of them store things that are not observations: `crawl:hash:*` is
a fetch watermark and `cooldown:<rule>` is a tuning parameter (`findings.md`
1.1). The `source` column keeps them apart by convention, and the `CHECK`
constraint covers only `kind`. `notes` has six writers, one of them a LAN scan
whose records then compete by cosine similarity with things he said
(`findings.md` 1.2).
**What breaks:** recall answers a question about him with a fetch watermark.
`queryFactByKey` reads the same table back as an answer.
## 3. Current context and presence
**Explicit and partly false at runtime.**
Presence is one hysteresis bucket rewritten each tick (`state.presence_state`),
and the dispatcher's routing table is a pure function of severity and presence.
One input is permanently wrong. `loop.State.CalendarBusy` reads
`facts(kind=env, source=caldav:*)` and `mavcaldav` is commented out of
`docker-compose.yml`, so the "do not nag mid-meeting" suppressor is always false
(`findings.md` 8.2). The compose file says so, which makes it a known gap.
**What breaks:** every interruption decision that should have deferred to a
meeting. It fails open, toward interrupting.
## 4. Proactive attention
**Explicit, and the one prohibition is stated.**
`CLAUDE.md`: no automatic attention-to-action path. Digestion may summarise
Praxis and may not call Hexis. At most one nudge candidate per tick, and the
restraint gate is a pure function over the rule set (`core.rules`).
Restraint is decided twice on purpose (`findings.md` 2.3), and blocked candidates
are held durably in `digest_entries` rather than dropped.
**What breaks:** nothing observed. This is the best-specified rule in the list.
## 5. Interruption policy
**Explicit for the choice, implied for the outcome.**
`docs/handler-wiring.md` owns the dispatch decision, and the table over
(severity, presence) is pure. Delivery intent is recorded in
`delivery_attempts` before the external send, so a crash leaves a pending row
rather than a lost one.
What is not stated is what a held nudge owes the user later. `digest_entries`
holds blocked candidates and two separate mechanisms carry the word digest
(`findings.md` 2.4). Nothing says when a held item expires instead of
resurfacing.
**The product question:** does a held nudge have a shelf life?
## 6. Clarification and follow-up ownership
**Implied.** Who owns an open question, and for how long, is decided by three
components and stated by none.
`runTurn` step 1 fires an expired-clarify notice, the clarify store is a
per-reach stack, and the pre-route ladder may claim the turn before routing
(`findings.md` 2.2). A restart drops a parked request silently, because the
notice path reads the store that is gone (`findings.md` 7.5).
Measured: one park survived five consecutive turns, turns 9 through 13, and was
released by a path other than `отмена`
(`docs/evals/2026-08-26-capability-baseline.md`).
**What breaks:** a question she asked stays open across unrelated turns, and
neither the ladder nor the store says whose job it is to close it.
## 7. Degradation and honesty
**Explicit as a rule, and the rule contradicts itself in practice.**
`CLAUDE.md` states both halves. Fall back silently when the fallback would only
do the job better. Name the gap when the resident model cannot do the job at
all. `docs/spec.md` writes every v1 DoD at "voice-reachable and honest", where
honest means naming the gap and never filling it with a guess.
Four silent degradations stack on one turn: workstation model to resident model,
CW2 to mavsttd, routing heads to LLM router to classifier, and search to Kiwix to
a named page to the model's own weights (`findings.md` 8.1). Each is argued
individually. Together a reply can be the resident model routing a worse
transcript with the classifier as a floor, answering from its weights, and
nothing in the reply distinguishes that from the best case.
**What breaks:** the boundary between "only better" and "cannot do the job" is
not drawn anywhere, so the stack decides it by accident.
**The product question:** at what depth of fallback does silence stop being
honest?
## 8. Authority and confirmation
**Unresolved, and this is the largest hole in the list.**
Two systems each answer half and never meet. `internal/auth` answers who may
carry what authority and does not bind the reactive turn path at all
(`findings.md` 6.3). `internal/tool` answers what effect a capability has and
what proof it demands, runs on every act, and is not keyed on the reach
(`findings.md` 6.3b). Neither has the other's reach.
Two representations of reach exist and both are ignored.
`internal/voice/server.go:198` defaults an empty `p.Surface` and a
client-asserted one survives to a handler that never reads it. `:148` hardcodes
`SurfacePCClient` for every connection. `req.Surface` is request payload on a
plaintext wire with no auth, so any client can claim `pc_client`. It must not
become an authorization input as it stands.
One path has no gate at all. `praxisItemAction.handle`
(`cmd/mavend/ecosystem_acts.go:158`) reads `dec.Slots.Value` and calls straight
through. Acknowledge, resolve, ignore and pin are remote mutations that run on
first hearing, with no tier and no confirm turn.
`CLAUDE.md` states the rule the code does not implement: LLM output is not
authorization, and a confirmation binds capability id, target entity, arguments,
requester and expiry.
**What breaks:** no one can currently state the authority property of a Maven
turn. `origin × effect × evidence → permit` is the target shape and nothing
computes it.
## 9. Privacy boundaries
**Explicit, and it is the best-enforced rule here.**
`CLAUDE.md`: the owner's data first, then the world. His notes and facts are
never search input, only the utterance leaves the box. The personal boundary is a
query source with `boundary: true`, and `queryWalk` reads
`Decision.SourceAnchored` for that source and no other.
The exception is deliberate and recorded. The boundary guesses, so naming
`SourceWorld` drops it, and only a stage 0 grammar may do that (owner's call,
V-666). No component reads another component's database, and Praxis attention
comes over HTTP rather than from its SQLite file.
**What breaks:** nothing observed. The one caveat is that `queryWalk` takes
sources out and moves none, which is the safety argument, and it holds only as
long as the table's order stays load-bearing.
## 10. Learning from outcomes
**Unresolved.** One loop exists, two are specified with no package, and nothing
says whether learning is a product goal.
Built: `state.nudges` is the restraint memory and the only input to the tick
loop's autotune, which writes `cooldown:<rule>` back into `facts`.
`state.routing_labels` holds owner corrections of a route.
Not built: `learning-the-style` and `learning-from-mistakes` have no package and
no component. `docs/spec.md` gives each a DoD written at what done would look
like. Both score `code_present: no`.
One criterion passes by absence. "No model weights change and no training set is
built" is a negative, and nothing being built satisfies it. The generator reports
this as an anomaly rather than counting it as progress.
**The product question:** is behavioural learning wanted, or is the negative
criterion the whole of the intent?
## 11. Capability composition
**Implied and absent.** Every capability is specified alone and the turn is
single-claim by construction.
`core.action_table` dispatches one intent to one handler, and a handler returning
the empty string hands the turn on. `queryWalk` stops at the first source that
claims. Two independent arbitrations already decide one turn, with a third
running before both (`findings.md` 2.1, 2.2).
`command-chaining` fails all three of its criteria with reason `wiring missing`.
The `chain` in `internal/router` is the world chain and the source chain, not
command chaining.
`internal/claim` is the beginning of a vocabulary for this and is called by
nothing (`findings.md` 6.1). It carries a live defect: `Claim.Coverage` returns
1.0 for a claim that extracted nothing, because `claimSpans` includes
`Slots.Text` unconditionally and `fillSlots` backfills the raw utterance into
`Text` (`findings.md` 6.3d).
**What breaks:** "напомни мне и запиши это" performs one of the two and says
nothing about the other.
**The product question:** what is the single unit that competes for a turn. This
is one of the three the freeze was called to answer.
## 12. Persistence across restart
**Implied.** Four stores made four different choices and no source states the
rule.
| state | survives a restart | evidence |
| --- | --- | --- |
| dialogue sessions | yes | `dialogue.NewPersistentSessionStore` |
| clarify store | no, deliberately | `findings.md` 7.5, Vikunja #385 |
| decision ring | no, in-memory bounded at 25 | `internal/decision/ring.go:11` |
| routing traces | yes, retained 14 days | `CLAUDE.md` |
| tick memo | no, in-process and argued for one field | `findings.md` 7.4 |
| surfaced items | no, and no TTL | `findings.md` 7.3 |
Two of these are principled. The decision ring holds his words and is bounded on
purpose. The clarify store's reasoning is filed. The other four are not decided
anywhere.
**What breaks:** less than it looks. `surfacedItems` has no TTL, and the source
comment argues that a stale ordinal resolves to an item Praxis reports as already
acknowledged, which is harmless because Praxis is the arbiter (`findings.md`
7.3). The cost is that the same absence of a written rule produced one argued
choice and three unargued ones.
---
## What this file is for
Session 2 step 2 of `docs/plans/26-capability-ledger-and-baseline.md`. It feeds
`docs/capabilities/gaps.md`, where every architecture concern must name the
capability or invariant it affects.
Four rules are `unresolved` and they are the owner's, not a commit's: authority
and confirmation, learning from outcomes, capability composition, and the shelf
life of a held nudge. Two of the three questions the freeze was called to answer
appear here as invariant 8 and invariant 11.
+91
View File
@@ -0,0 +1,91 @@
# The structured half of docs/capabilities/invariants.md. HAND-WRITTEN.
#
# The prose, the evidence and the reasoning live in the .md. This file carries
# only what a machine needs: the mark, which capabilities the rule touches, and
# which components participate in it. build_ledger.py checks the two agree, so
# an invariant cannot exist in one and not the other.
#
# mark: explicit | implied | unresolved. `split` means the rule is written down
# in one half and not in the other, and the .md says which half is which.
invariants:
- id: 1
title: Continuity across turns and across reaches
mark: implied
question: Is a conversation per reach, or one conversation the reaches are windows onto?
capabilities: [web-ui, voice, telegram, ask-instead-of-guessing]
components: [state.dialogue_sessions, state.clarify_store, proc.mavweb, core.voice_server, core.sink_telegram]
- id: 2
title: Memory and correction semantics
mark: explicit
split: true
capabilities: [facts, notes, recall]
components: [state.facts, state.notes, state.memory_vectors, core.recall, router.embedder, core.q.factbykey, core.fact_enrichment, core.netscan]
- id: 3
title: Current context and presence
mark: explicit
capabilities: [interruption-policy, calendar-management, morning-routine]
components: [state.presence_state, core.gatherer, core.q.calendar, proc.mavcaldav, core.dispatcher]
- id: 4
title: Proactive attention
mark: explicit
capabilities: [interruption-policy, digest-of-held-nudges, routine-proposals, praxis]
components: [core.tick_loop, core.rules, state.digest_entries, state.nudges, core.q.attention, ext.praxis]
- id: 5
title: Interruption policy
mark: explicit
split: true
question: Does a held nudge have a shelf life?
capabilities: [interruption-policy, digest-of-held-nudges, telegram, ntfy]
components: [core.dispatcher, state.delivery_attempts, state.digest_entries, core.rules, core.sink_telegram, core.sink_ntfy, core.sink_voice]
- id: 6
title: Clarification and follow-up ownership
mark: implied
capabilities: [ask-instead-of-guessing, route-an-utterance]
components: [core.preroute, state.clarify_store, state.dialogue_sessions, core.turn_route]
- id: 7
title: Degradation and honesty
mark: explicit
split: true
question: At what depth of fallback does silence stop being honest?
capabilities: [answer-from-the-world, speech-to-text, route-an-utterance, speak-as-herself, read-an-encyclopedia, weather]
components: [core.model_seam, core.stt_seam, router.cascade, router.classifier, core.query_chain, core.phraser, core.q.kiwix, core.q.general]
- id: 8
title: Authority and confirmation
mark: unresolved
question: Where is the one point that decides whether this origin may perform this effect with this evidence?
capabilities: [hexis, praxis, voice, passkey-and-step-up, encrypted-database]
components: [core.auth_gate, core.risk_policy, state.pending_act, state.tools, core.action_act, core.ecosystem_hexis_gate, core.praxis_acts, core.voice_server, bnd.voice_tcp, core.daemon_lock]
- id: 9
title: Privacy boundaries
mark: explicit
capabilities: [answer-from-the-world, answer-from-your-own-data, recall, read-an-encyclopedia]
components: [core.q.personal, core.query_chain, core.q.search, core.q.kiwix, bnd.http_ecosystem]
- id: 10
title: Learning from outcomes
mark: unresolved
question: Is behavioural learning wanted, or is the negative criterion the whole of the intent?
capabilities: [learning-the-style, learning-from-mistakes, interruption-policy, route-an-utterance]
components: [state.nudges, state.routing_labels, core.tick_loop, core.rules]
- id: 11
title: Capability composition
mark: implied
question: What is the single unit that competes for a turn?
capabilities: [command-chaining, route-an-utterance, answer-from-your-own-data, ask-instead-of-guessing]
components: [core.action_table, core.query_chain, core.preroute, router.cascade, router.stage0, router.claim, router.modes]
- id: 12
title: Persistence across restart
mark: implied
capabilities: [ask-instead-of-guessing, praxis, route-an-utterance]
components: [state.dialogue_sessions, state.clarify_store, state.decision_ring, state.routing_traces, state.tick_memo, state.surfaced_items]
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,36 @@
{
"facts": {
"count": 13041
},
"notes": {
"count": 185
},
"reminders": {
"count": 98
},
"pending_reminders": {
"error": "e2eprobe: ipc: unknown method:"
},
"tasks_live": {
"count": 6
},
"nudges": {
"count": 5
},
"tools": {
"count": 13
},
"decisions": {
"count": 25
},
"events": {
"count": 62
},
"eco_traces": {
"count": 115
},
"delivery_attempts": {
"count": 200,
"note": "at e2eprobe's hardcoded 200 cap, true count is >= 200"
}
}
@@ -0,0 +1,35 @@
{
"facts": {
"count": 1
},
"notes": {
"count": 4
},
"reminders": {
"count": 0
},
"pending_reminders": {
"error": "e2eprobe: ipc: unknown method:"
},
"tasks_live": {
"count": 0
},
"nudges": {
"count": 0
},
"tools": {
"count": 12
},
"decisions": {
"count": 0
},
"events": {
"count": 4
},
"eco_traces": {
"count": 0
},
"delivery_attempts": {
"count": 0
}
}
File diff suppressed because one or more lines are too long
View File
File diff suppressed because one or more lines are too long
@@ -0,0 +1,34 @@
field:reminder:create-and-persist напомни мне завтра в 9 утра позвонить в клинику хорошо, напомню завтра в 09:00. 0.29
field:reminder:clarify-the-missing-slot напомни мне позвонить маме Сейчас 01:09. Когда? 0.26
field:reminder:clarify-the-missing-slot в шесть вечера Сейчас 01:09. Поняла: в шесть вечера. В какой день? 0.62
field:reminder:cancel-mid-clarify напомни мне заехать за кормом Сейчас 01:09. Когда? 0.26
field:reminder:cancel-mid-clarify отмена Хорошо, отменила. 0.27
field:reminder:correct-the-time напомни мне в семь вечера вынести мусор Сейчас 01:09. В какой день? 0.3
field:reminder:correct-the-time нет, в восемь Сейчас 01:09. В какой день? 0.6
field:reminder:recurring-from-speech напоминай мне каждый день в девять вечера дать собаке таблетку Сейчас 01:09. В какой день? 0.29
field:reminder:recurring-from-speech какие у меня повторяющиеся напоминания? Все напоминания: 0. На какое время поставить напоминание? memory 3.36
field:fact:record-and-confirm меня зовут Ками У тебя есть 35 сообщений. 4.14
field:fact:supersede я работаю в Тинькофф не знаю — не нашла у тебя такой записи. personal 0.12
field:fact:supersede я больше не работаю в Тинькофф, теперь я в Яндексе не знаю — не нашла у тебя такой записи. personal 0.13
field:fact:supersede где я работаю? не знаю — не нашла у тебя такой записи. personal 0.1
field:fact:what-do-you-remember что ты помнишь обо мне? Мне известно о тебе с 2017 года. В настоящее время у меня нет информации об удалении работы в Тинькофф и возобновлений работ по этой компании. 2.98
field:note:question-is-not-a-statement я рассказывал тебе про байкал? Давайте проверим. memory 4.78
field:note:capture-then-recall запиши: ключи от гаража лежат в верхнем ящике стола "Записала", "проверила" 2.65
field:note:capture-then-recall где лежат ключи от гаража? В главе с Долорес он спрашивает: «где лежат ключи от гаража?» Источник утверждает, что они находятся рядом со кухней на первом этаже. search 7.52
field:note:recall-miss-is-honest что я говорил тебе про ремонт балкона? Уточни пожалуйста имя человека из которого ты будешь учиться или посмотреть информацию. memory 2.4
field:query:own-data-first что у меня сегодня по плану? на 26.08.2026 ничего не запланировано. day-plan 0.01
field:query:the-world-summary почему небо голубое? Почему небо голубое? - В статье разберемся, почему днем небо кажется голубым (см., например). Рассказываем про солнечный свет. search 6.29
field:query:arithmetic-must-not-escape сколько будет два плюс два? Похоже у вас есть интерес к математике! 2 + 2 = ? Это довольно простая задачка для ребенка с первого класса, а вы можете продолжить? Или вам нужны советы по образованию? search 8.17
field:query:follow-up-context какая сейчас погода? для какого города? weather 0.17
field:query:follow-up-context Самара Я тебя не разобрала. Повтори, пожалуйста. 0.04
field:query:encyclopedia-language расскажи про Байкал Всего лишь одна фразаЕсли у тебя есть желание узнать больше о Байкале или его влиянии как гидроэнергетического комплекс, посетите сайт «Энциклопедии Кирилла и Мефодия» https://www.km.ru/encyclopedia/view.php?ID=602. memory 5.0
field:query:feeds-on-request что нового в новостях? вот что нового: mklinux-v7.0-mk2 released; [$] Old-school calendaring at the command line with Remind; Vanilla OS 3 released feeds 1.67
field:query:network-scan-as-prose какие устройства сейчас в сети? нашла 3 устройства, из них 2 с вебом, 2 с ssh. список записала. network 27.01
field:action:target-must-resolve перезапусти контейнер с базой «с» — не знаю такой цели. назови её как в системе. 0.02
field:action:confirm-binds выключи свет в спальне «с» — не знаю такой цели. назови её как в системе. 0.04
field:action:report-the-actual-result покажи что запущено в докере «с» — не знаю такой цели. назови её как в системе. 0.06
field:action:praxis-lifecycle-gate что требует моего внимания? Нет ничего нового. 3.66
field:action:praxis-lifecycle-gate отметь первое как выполненное Такую команду я не знаю. 0.04
field:action:capture-a-task добавь в задачи: заказать корм для собаки внесла в список: заказать корм для собаки 0.01
field:action:capture-a-task какие у меня задачи? сначала: проверить интейк форму на боксе (просрочено на 17 дней); купить хлеб (давно в списке); купить молоко (давно в списке); полить цветы (давно в списке); починить кран (давно в списке); и ещё 1 задача. 1 задача просрочено; 5 задач лежит дольше 10 дней. tasks 0.01
field:proactive:delivery-reaches-telegram напомни мне через две минуты MVNPROBE проверка доставки хорошо, напомню сегодня в 01:13. 0.33
Can't render this file because it contains an unexpected character in line 16 and column 143.
+130
View File
@@ -0,0 +1,130 @@
{
"note": "field probes: the owner's real week, drawn from the five vertical slices, the three ugly conversations of the 2026-08-13 audit verbatim, the five existing scenarios and what deploy/mavend.json is actually configured for. A failing field probe is an unresolved product question or a missing-criterion finding against docs/spec.md, never a DoD verdict.",
"probes": [
{ "id": "field:reminder:create-and-persist", "origin": "field", "kind": "live", "slice": "reminder",
"utterances": ["напомни мне завтра в 9 утра позвонить в клинику"],
"readback": {"reminders": ["reminders", "10"], "decisions": ["decisions", "1"]},
"expect": "The reply states the time back, and a reminders row exists for 09:00 tomorrow with that text." },
{ "id": "field:reminder:clarify-the-missing-slot", "origin": "field", "kind": "live", "slice": "reminder",
"utterances": ["напомни мне позвонить маме", "в шесть вечера"],
"readback": {"reminders": ["reminders", "10"], "decisions": ["decisions", "2"]},
"expect": "Turn 1 asks only for the time, not for all slots. Turn 2 fills it and a row lands at 18:00. The audit saw this pass; it is here to catch a regression." },
{ "id": "field:reminder:cancel-mid-clarify", "origin": "field", "kind": "live", "slice": "reminder",
"utterances": ["напомни мне заехать за кормом", "отмена"],
"readback": {"reminders": ["reminders", "10"]},
"expect": "Turn 2 drops the parked turn and says so. No reminder row is written for the kibble." },
{ "id": "field:reminder:correct-the-time", "origin": "field", "kind": "live", "slice": "reminder",
"utterances": ["напомни мне в семь вечера вынести мусор", "нет, в восемь"],
"readback": {"reminders": ["reminders", "10"]},
"expect": "One reminder at 20:00, not two rows and not one at 19:00. Correction of a just-set reminder is the slice's real shape." },
{ "id": "field:reminder:recurring-from-speech", "origin": "field", "kind": "live", "slice": "reminder",
"utterances": ["напоминай мне каждый день в девять вечера дать собаке таблетку", "какие у меня повторяющиеся напоминания?"],
"readback": {"reminders": ["reminders", "20"]},
"expect": "The schedule is stated back and a row carries Cron. The spec says storage and delivery are finished and no caller in cmd/mavend passes a cron, so this is expected to fail; the probe records HOW it fails." },
{ "id": "field:fact:record-and-confirm", "origin": "field", "kind": "live", "slice": "fact/note",
"utterances": ["меня зовут Ками"],
"readback": {"facts": ["facts", "20"], "decisions": ["decisions", "1"]},
"expect": "The confirmation is feminine, and a facts row actually exists. The audit caught 'я записала информацию о тебе' confirming a write that never happened." },
{ "id": "field:fact:supersede", "origin": "field", "kind": "live", "slice": "fact/note",
"utterances": ["я работаю в Тинькофф", "я больше не работаю в Тинькофф, теперь я в Яндексе", "где я работаю?"],
"readback": {"facts": ["facts", "30"]},
"expect": "Turn 3 answers Yandex, not Tinkoff and not both. Both rows are readable and the old value is retired." },
{ "id": "field:fact:what-do-you-remember", "origin": "field", "kind": "live", "slice": "fact/note",
"utterances": ["что ты помнишь обо мне?"],
"readback": {"decisions": ["decisions", "1"], "notes": ["notes", "5"]},
"expect": "Routes to query, not remember, and the reply uses informal singular ты. The planning session got formal вас/вы here while the phrasing eval passed. Two of the audit's seven probes misrouted on exactly this shape." },
{ "id": "field:note:question-is-not-a-statement", "origin": "field", "kind": "live", "slice": "fact/note",
"utterances": ["я рассказывал тебе про байкал?"],
"readback": {"notes": ["notes", "5"], "decisions": ["decisions", "1"]},
"expect": "No note row is written. The audit stored this question as a statement, and the personal boundary scored it as world." },
{ "id": "field:note:capture-then-recall", "origin": "field", "kind": "live", "slice": "fact/note",
"utterances": ["запиши: ключи от гаража лежат в верхнем ящике стола", "где лежат ключи от гаража?"],
"readback": {"notes": ["notes", "10"], "decisions": ["decisions", "1"]},
"expect": "Turn 2 returns the drawer from the note captured in turn 1, not a world answer and not a recall miss." },
{ "id": "field:note:recall-miss-is-honest", "origin": "field", "kind": "live", "slice": "fact/note",
"utterances": ["что я говорил тебе про ремонт балкона?"],
"readback": {"decisions": ["decisions", "1"]},
"expect": "She says she does not remember. A world answer here is the failure: the personal boundary must stop a question about him from reaching outside." },
{ "id": "field:query:own-data-first", "origin": "field", "kind": "live", "slice": "query",
"utterances": ["что у меня сегодня по плану?"],
"readback": {"decisions": ["decisions", "1"], "plan": ["plan"]},
"expect": "The real checklist and its open items come back, and the decision trace shows an owner source winning before any world source was asked." },
{ "id": "field:query:the-world-summary", "origin": "field", "kind": "live", "slice": "query",
"utterances": ["почему небо голубое?"],
"readback": {"decisions": ["decisions", "1"]},
"expect": "A Russian summary that does not invent physics. The audit's answer was 'корочковатые цветы отражают длинноволны'. Verbatim from the audit, so the two are comparable." },
{ "id": "field:query:arithmetic-must-not-escape", "origin": "field", "kind": "live", "slice": "query",
"utterances": ["сколько будет два плюс два?"],
"readback": {"decisions": ["decisions", "1"]},
"expect": "The answer is 4. Measured this session: the arithmetic-query stage 0 grammar declined, the turn reached external search, and the reply was 'Во-первых - это два плюса двойки'." },
{ "id": "field:query:follow-up-context", "origin": "field", "kind": "live", "slice": "query",
"utterances": ["какая сейчас погода?", "Самара"],
"readback": {"decisions": ["decisions", "2"]},
"expect": "Turn 2 is understood as the city for turn 1. The audit's follow-up died with 'Я тебя не разобрала'. There is no weather block in the config, so a named gap is the honest pass and a guessed forecast is the failure." },
{ "id": "field:query:encyclopedia-language", "origin": "field", "kind": "live", "slice": "query",
"utterances": ["расскажи про Байкал"],
"readback": {"decisions": ["decisions", "1"]},
"expect": "A Russian question lands on the Russian book. The claiming source in the trace is kiwix, and the article is about the lake." },
{ "id": "field:query:feeds-on-request", "origin": "field", "kind": "live", "slice": "query",
"utterances": ["что нового в новостях?"],
"readback": {"decisions": ["decisions", "1"]},
"expect": "Configured feed items are read back. Two sources are configured; a dead feed must name itself dead and the other still answer." },
{ "id": "field:query:network-scan-as-prose", "origin": "field", "kind": "live", "slice": "query",
"utterances": ["какие устройства сейчас в сети?"],
"readback": {"decisions": ["decisions", "1"]},
"expect": "Hosts and open ports come back as prose, not as a table dump. netscan.enabled is true with subnets, ports and rate set." },
{ "id": "field:action:target-must-resolve", "origin": "field", "kind": "live", "slice": "action",
"utterances": ["перезапусти контейнер с базой"],
"readback": {"eco-traces": ["eco-traces", "10"], "decisions": ["decisions", "1"]},
"expect": "Free text does not reach a mutating Hexis call. Nexus holds no entities, so the honest outcome is a named gap or a clarify, never a guessed target and never an execution." },
{ "id": "field:action:confirm-binds", "origin": "field", "kind": "live", "slice": "action",
"utterances": ["выключи свет в спальне"],
"readback": {"eco-traces": ["eco-traces", "10"], "decisions": ["decisions", "1"]},
"expect": "smarthome.enabled is false, so this is a named gap. The failure is a reply that claims the light was switched." },
{ "id": "field:action:report-the-actual-result", "origin": "field", "kind": "live", "slice": "action",
"utterances": ["покажи что запущено в докере"],
"readback": {"eco-traces": ["eco-traces", "10"], "tools": ["tools"]},
"expect": "Either the real container list or a named gap. A plausible invented list is the failure this slice exists to catch." },
{ "id": "field:action:praxis-lifecycle-gate", "origin": "field", "kind": "live", "slice": "action",
"utterances": ["что требует моего внимания?", "отметь первое как выполненное"],
"readback": {"eco-traces": ["eco-traces", "20"], "decisions": ["decisions", "2"]},
"expect": "Turn 1 calls Surface, never Acknowledge. Turn 2 is a remote mutation: the architecture pass found praxisItemAction.handle calls straight through with no tier and no confirm turn. The probe records whether it runs on first hearing." },
{ "id": "field:action:capture-a-task", "origin": "field", "kind": "live", "slice": "action",
"utterances": ["добавь в задачи: заказать корм для собаки", "какие у меня задачи?"],
"readback": {"tasks": ["tasks"], "decisions": ["decisions", "2"]},
"expect": "The task appears in the list, ordered by deadline and urgency, and the capture is visible over IPC." },
{ "id": "field:proactive:delivery-reaches-telegram", "origin": "field", "kind": "live", "slice": "proactive",
"utterances": ["напомни мне через две минуты MVNPROBE проверка доставки"],
"readback": {"reminders": ["reminders", "10"], "delivery-attempts": ["delivery-attempts"]},
"expect": "Within a compressed live horizon the reminder fires and a delivery-attempts row records the reach that took it. TickInterval defaults to 60s and no override is configured, so two minutes is two ticks. The marker keeps it from reading as a real nudge." },
{ "id": "field:proactive:failure-retries-into-another-reach", "origin": "field", "kind": "live", "slice": "proactive",
"utterances": [],
"readback": {"delivery-attempts": ["delivery-attempts"], "nudges": ["nudges", "20"], "events": ["events", "50"]},
"expect": "Read the existing attempt rows: a failed delivery retried into another reach rather than looping. The audit watched the ntfy failure run once a minute until 03:05, then Telegram took reminder #83 at 03:06 and nothing retried." }
]
}
+137
View File
@@ -0,0 +1,137 @@
#!/usr/bin/env python3
"""Drive probes through the deployed stack and record what came back.
Runs ON homesrv, where 127.0.0.1:9201 is mavweb and the mavend socket is
reachable from inside maven-mavend-1.
Transport is POST /api/chat, which runs a real turn: router, resident model,
query walk, act path. The reply rides back on the 303 Location as
?q=..&r=..&s=<claiming query source>&t=<trace id>.
Readback is e2eprobe over the IPC socket, never the plaintext sqlite copy.
python3 run_probes.py probes.json > raw.jsonl
"""
import json
import re
import subprocess
import sys
import time
import urllib.parse
WEB = "http://127.0.0.1:9201"
SOCK = "/run/maven/mavend.sock"
CONTAINER = "maven-mavend-1"
PROBE_BIN = "/tmp/e2eprobe"
def sh(args, timeout=180):
p = subprocess.run(args, capture_output=True, text=True, timeout=timeout)
return p.returncode, p.stdout, p.stderr
def chat(text):
"""One turn. Returns the parsed redirect, or the failure verbatim."""
t0 = time.time()
rc, out, err = sh([
"curl", "-s", "-o", "/dev/null", "-w", "%{http_code}\t%{redirect_url}",
"-X", "POST", "--data-urlencode", f"text={text}", f"{WEB}/api/chat",
])
dt = round(time.time() - t0, 2)
if rc != 0:
return {"utterance": text, "error": err.strip() or f"curl rc={rc}", "seconds": dt}
code, _, loc = out.partition("\t")
r = {"utterance": text, "http": code, "seconds": dt}
if code != "303":
r["error"] = f"expected 303, got {code}"
return r
q = urllib.parse.parse_qs(urllib.parse.urlparse(loc).query)
r["reply"] = q.get("r", [""])[0]
r["source"] = q.get("s", [""])[0]
r["trace"] = q.get("t", [""])[0]
return r
def probe(cmd):
"""One e2eprobe readback. Returns parsed JSON or the error verbatim."""
rc, out, err = sh(["docker", "exec", CONTAINER, PROBE_BIN, "-sock", SOCK] + cmd)
if rc != 0:
return {"error": (err or out).strip()}
try:
return json.loads(out)
except json.JSONDecodeError:
return {"raw": out.strip()}
def shell(cmd):
"""One read-only command on homesrv, for a configuration or deployment fact.
A criterion about whether a config block exists is not observable through a
turn, and chasing it through one measures the router instead.
"""
t0 = time.time()
p = subprocess.run(["sh", "-c", cmd], capture_output=True, text=True, timeout=300)
return {
"cmd": cmd,
"rc": p.returncode,
"stdout": p.stdout[-8000:],
"stderr": p.stderr[-2000:],
"seconds": round(time.time() - t0, 2),
}
def normalise_readback(rb):
"""Accept both shapes: {name: argv} and [{name, argv}]."""
if isinstance(rb, dict):
return list(rb.items())
return [(d["name"], d["argv"]) for d in (rb or [])]
def reset():
"""Clear any parked clarify before the next probe.
mavweb hardcodes one conversation id for the whole web reach, so a clarify
parked by one probe is still parked for the next one. Measured: an
unanswered park appended "Сейчас 01:07. В какой день?" to eleven unrelated
turns in a row, including plain statements the router should have taken as
facts. Without this the run measures the previous probe, not this one.
The leak itself is a finding, recorded separately from a run that isolates.
"""
r = chat("отмена")
return {"reply": r.get("reply", ""), "http": r.get("http", ""), "error": r.get("error", "")}
def main():
spec = json.load(open(sys.argv[1]))
for p in spec["probes"]:
rec = {
"id": p["id"],
"origin": p.get("origin", "dod" if p["id"].startswith("dod:") else "field"),
"criteria": p.get("criteria", []),
"slice": p.get("slice", ""),
"expect": p.get("expect", ""),
"kind": p.get("kind", "live"),
"method": p.get("method", "chat"),
"turns": [],
"readback": {},
}
if rec["kind"] == "blocked" or rec["method"] == "none":
rec["blocked_reason"] = p.get("blocked_reason", "")
print(json.dumps(rec, ensure_ascii=False), flush=True)
continue
if p.get("isolate", True) and rec["method"] == "chat":
rec["reset_before"] = reset()
time.sleep(1)
if rec["method"] == "shell":
rec["shell"] = shell(p["shell"])
for text in p.get("utterances", []):
rec["turns"].append(chat(text))
time.sleep(p.get("gap", 1))
for name, cmd in normalise_readback(p.get("readback")):
rec["readback"][name] = probe(cmd)
print(json.dumps(rec, ensure_ascii=False), flush=True)
if __name__ == "__main__":
main()
+59
View File
@@ -0,0 +1,59 @@
#!/usr/bin/env python3
"""Row counts per store, read over IPC. Runs on homesrv.
The eval's first artifact. Read before any probe writes, and again after, so
what the probes added is attributable. Not the sqlite file: the point is to
measure the contract, and the plaintext copy in tmpfs bypasses it.
"""
import json
import subprocess
import sys
C = "maven-mavend-1"
S = "/run/maven/mavend.sock"
BIN = "/tmp/e2eprobe"
# command, argv, and whether the reply is capped by a limit e2eprobe hardcodes.
READS = [
("facts", ["facts", "100000"], False),
("notes", ["notes", "100000"], False),
("reminders", ["reminders", "100000"], False),
("pending_reminders", ["pending-reminders", "100000"], False),
("tasks_live", ["tasks"], False),
("nudges", ["nudges", "100000"], False),
("tools", ["tools"], False),
("decisions", ["decisions", "100000"], False),
("events", ["events", "100000"], False),
("eco_traces", ["eco-traces", "100000"], False),
("delivery_attempts", ["delivery-attempts"], True), # capped at 200 in e2eprobe
]
def main():
out = {}
for name, argv, capped in READS:
p = subprocess.run(["docker", "exec", C, BIN, "-sock", S] + argv,
capture_output=True, text=True, timeout=180)
if p.returncode != 0:
out[name] = {"error": (p.stderr or p.stdout).strip()[:300]}
continue
try:
d = json.loads(p.stdout)
except json.JSONDecodeError:
out[name] = {"error": "not json", "raw": p.stdout.strip()[:300]}
continue
if d is None:
out[name] = {"count": 0}
elif isinstance(d, list):
r = {"count": len(d)}
if capped and len(d) >= 200:
r["note"] = "at e2eprobe's hardcoded 200 cap, true count is >= 200"
out[name] = r
else:
out[name] = {"value": d}
json.dump(out, sys.stdout, ensure_ascii=False, indent=2)
print()
if __name__ == "__main__":
main()

Some files were not shown because too many files have changed in this diff Show More