docs: tier the tree by lifetime, so staleness shows in the path (V-446)
Seventeen markdown files at the repo root, twelve of them dated one-shot reports sitting next to CLAUDE.md. That is why stale docs read as current: nothing in the path said which was which. Root now keeps CLAUDE.md and AGENTS.md. Living docs move under docs/ and carry a Last verified line. Dated measurements move to docs/evals/ ISO-prefixed, and are never edited after the day, so a newer number is a new file. The senior review moves to docs/archive/. Every reference was rewritten across markdown, Go comments, the Makefile and the recall fixture. The touched Go packages still build. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,362 @@
|
||||
# Maven — senior review, 2026-07-30
|
||||
|
||||
Scope: code, config, docs, deploy path, test suite. Runtime claims below come from
|
||||
tests, config and static reading — the daemons were not run and routing latency was
|
||||
not measured. Working tree at review time: uncommitted `internal/morning` work plus
|
||||
`cmd/mavend/eval_scenarios_test.go` (both look correct, see "keep").
|
||||
|
||||
---
|
||||
|
||||
## verdict
|
||||
|
||||
**Healthy engineering, misaligned direction.** The code is unusually disciplined for a
|
||||
personal project — clean boundaries, pure-core/impure-driver separation, argv-only tool
|
||||
execution, 12k lines of real tests, `go vet` clean. The problem is not code quality. It is
|
||||
that **the single most important architectural decision has four contradictory sources of
|
||||
truth and the intended design is switched off in production**, plus a broken green-build
|
||||
signal and a deploy path that does not enforce the security boundary its own comments
|
||||
assume.
|
||||
|
||||
## what the project is now
|
||||
|
||||
```
|
||||
purpose: self-hosted RU/EN voice assistant; 8 Go daemons over unix sockets,
|
||||
SQLite, local models
|
||||
intended users: one person, one box (homesrv, Ryzen 5 5600U, Vega iGPU, 14GB)
|
||||
actual users: same — confirmed by hardcoded TZ, single-passkey auth,
|
||||
/home/kami replace directives in go.mod
|
||||
critical workflows: voice turn (mic→STT→route→tool/reply→TTS); proactive nudges
|
||||
(tick→dispatcher→telegram/ntfy/voice); web PWA
|
||||
(/dash /chat /tools /ecosystem)
|
||||
current state: all 34 test packages pass; vet clean; `make test` still exits 1
|
||||
(finding 2)
|
||||
known failures: weak RU query routing — docs/rearchitecture.md names the cause; the named fix
|
||||
is wired `nil`
|
||||
maintenance burden: 4,518 lines of root markdown vs 33,319 lines of Go; 15 top-level
|
||||
.md files, 3 of them dated session logs; several contradict
|
||||
each other
|
||||
constraints: CPU-only deploy, ~14GB shared RAM, resident model must stay small;
|
||||
never phones home; not autonomous; feminine RU persona
|
||||
what works well: internal/tool (argv-only, store allowlist, confirm gate); pure
|
||||
engines (loop, routine, morning, pattern, dialogue — 86–100%
|
||||
coverage); delivery outbox (Begin-before-send/Complete-after);
|
||||
vendored deps (5 direct, tiny surface)
|
||||
what's obsolete: llmrouter.go (built, tested, never wired); classifier seed-phrase
|
||||
cascade (interim stopgap, now 4 weeks old);
|
||||
SESSION-05/06-07-2026.md; PLANS.md; ROADMAP.md
|
||||
```
|
||||
|
||||
**Classification: healthy + misaligned.** Not fragile, not overbuilt, not abandoned. The
|
||||
architecture in `docs/rearchitecture.md` is sound and mostly *built* — it just is not *connected*.
|
||||
|
||||
## what it should become
|
||||
|
||||
The thing `docs/rearchitecture.md` already describes, with the switch flipped and the drift removed:
|
||||
one resident small model doing both routing and phrasing, classifier demoted from the live
|
||||
path to the failure floor, embedder demoted to RAG hint. No new architecture is needed.
|
||||
**The gap is a config/wiring decision plus doc convergence, not a redesign.**
|
||||
|
||||
---
|
||||
|
||||
## main findings
|
||||
|
||||
### 1. Four competing sources of truth on the resident model — and the target router is dead code
|
||||
|
||||
`architecture` / `repair`
|
||||
|
||||
**problem:** The most load-bearing design decision in the project is stated four different,
|
||||
incompatible ways, and the code path `docs/rearchitecture.md` calls "the linchpin" is disabled.
|
||||
|
||||
**evidence** (all confirmed):
|
||||
|
||||
- `cmd/mavend/voice.go:211` — `rtr := buildRouter(emb, matcher, threshold, nil) // LLM router disabled`,
|
||||
with comment *"the classifier handles routing reliably."*
|
||||
- `docs/rearchitecture.md:11` says the same classifier is *"the structural cause of 'she messes up
|
||||
queries.'"* **The code comment and the design doc make opposite claims about the same
|
||||
component.**
|
||||
- `internal/router/llmrouter.go` (139 lines) + `llmrouter_test.go` — fully built and
|
||||
tested, zero non-test callers.
|
||||
- Model identity, four ways: docs say **Qwen3-1.7B** (`CLAUDE.md:6`, `docs/rearchitecture.md:15`,
|
||||
`SPEC.md:46`, `AGENTS.md:79`, `docs/ecosystem.md:72`);
|
||||
`deploy/mavend.json:9` says **Qwen3.5-2B-UD-Q4_K_XL**; `models/llm/` on disk holds
|
||||
**LFM2.5-1.2B-Thinking**; code comments in 5 files still say **LFM**.
|
||||
- `deploy/mavend.json:11` sets `"n_gpu_layers": 99` while `CLAUDE.md:4` states the target
|
||||
is **CPU-only**. (Interpretation: compose does pass `/dev/dri` + render gid for Vulkan,
|
||||
so 99 may be intentional — but then the CPU-only claim is stale. One of the two is
|
||||
wrong; the repo alone cannot say which.)
|
||||
|
||||
**impact:** You cannot answer "which model is deployed and does it route?" from this repo.
|
||||
Every future routing decision starts by re-deriving the ground truth — that is the actual
|
||||
maintenance cost, and it is the reason a 4-week-old "interim stopgap" is still the
|
||||
committed default. The 2,453 lines of router package carry two engines where one is wanted.
|
||||
|
||||
**recommended action:** Pick the deployed model, then in one commit: set
|
||||
`phraser.model_path` to it, wire `NewLLMRouter(llmClient)` into `buildRouter` at
|
||||
`voice.go:211` behind a config flag (`voice.llm_router: true`) so the classifier stays the
|
||||
documented failure floor, and update the model name in all 5 docs + 5 code comments to
|
||||
match. Delete nothing from `internal/router` yet — the classifier is the fallback by design.
|
||||
|
||||
**why this level of change:** The components exist and are tested. This is wiring + truth
|
||||
reconciliation, not a refactor. **Do not rewrite the router.**
|
||||
|
||||
**alternatives:** Delete `llmrouter.go` and commit to the classifier — only defensible if
|
||||
the eval harness shows the classifier is actually adequate, which contradicts `docs/rearchitecture.md`.
|
||||
|
||||
**risk:** Low-moderate. LLM route failures already fall through to the classifier
|
||||
(`router.go:88-96`), so a bad model cannot break a turn. The real risk is CPU latency.
|
||||
|
||||
**verification:** `cmd/mavend/eval_scenarios_test.go` + `testdata/` (already in the working
|
||||
tree, uncommitted) is exactly the right harness — run RU query scenarios against both
|
||||
paths and compare before flipping the default.
|
||||
|
||||
---
|
||||
|
||||
### 2. `make test` always fails, so there is no green signal
|
||||
|
||||
`repair` — smallest high-value fix in the repo
|
||||
|
||||
**problem:** Every test passes; `make test` exits 1 anyway.
|
||||
|
||||
**evidence** (confirmed, root-caused): `make test` ends with
|
||||
`# github.com/kami/maven/cmd/mavenclient` / `go: no such tool "covdata"` →
|
||||
`make: *** Error 1`. Cause chain:
|
||||
|
||||
- `deps/go/go/VERSION` = **go1.23.4**; `go.mod` requires **go 1.25.5** → `GOTOOLCHAIN=auto`
|
||||
re-execs into `/home/kami/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.5.linux-amd64`.
|
||||
- That cached toolchain ships only 7 of ~17 tools: `asm cgo compile cover link preprofile
|
||||
vet`. **`covdata` is missing** (source present, binary absent), as are `pprof`, `nm`,
|
||||
`objdump`, `trace`, `test2json`.
|
||||
- `-coverprofile` invokes `covdata` for the two packages with no test files
|
||||
(`cmd/mavenclient`, `cmd/mavend/seedtest`) → failure. `GOTOOLCHAIN=local` confirms the
|
||||
vendored 1.23.4 is too old to substitute
|
||||
(`go.mod requires go >= 1.25.5 (running go 1.23.4)`).
|
||||
|
||||
**impact:** A permanently-red test target trains you to ignore the exit code — the one
|
||||
signal protecting a 12k-line test suite. `go tool pprof` is also unavailable, which matters
|
||||
on a latency-sensitive CPU-only box.
|
||||
|
||||
**recommended action:** `rm -rf '/home/kami/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.5.linux-amd64'`
|
||||
and let Go re-download it complete — the cache entry is truncated, not a Makefile bug.
|
||||
Then bump `deps/go` to a real 1.25.5 so the vendored toolchain is self-sufficient and
|
||||
`GOTOOLCHAIN=local` works, which is the whole point of vendoring it.
|
||||
|
||||
**risk:** None. Worst case the re-download fails and you are where you started.
|
||||
|
||||
**verification:** `make test` exits 0.
|
||||
|
||||
---
|
||||
|
||||
### 3. Deploy path publishes an unauthenticated RCE surface; the code's stated mitigation is not in the repo
|
||||
|
||||
`security`
|
||||
|
||||
**problem:** The tool-enable gate is deliberately fail-open, documented as safe because it
|
||||
"sits behind wg+nginx+auth" — but the repo's own deploy path publishes the port with no
|
||||
such layer and no WebAuthn.
|
||||
|
||||
**evidence** (confirmed):
|
||||
|
||||
- `cmd/mavweb/main.go:918` — `if session != nil && !session.IsStepUp()`. `stepUpSession`
|
||||
is nil unless `-webauthn-origin` **and** `-webauthn-rpid` are both set (`main.go:419`).
|
||||
Nil ⇒ **gate skipped entirely.**
|
||||
- `docker-compose.yml:82` passes neither flag, and `ports: ["9201:9201"]` publishes
|
||||
mavweb on the host.
|
||||
- `handleTools` `enable` takes `name` + `cmd` from form fields and calls `core.EnableTool`
|
||||
— so this is not "approve a proposal," it is **define arbitrary argv**. `internal/tool`
|
||||
then executes it. `destructive` is a checkbox the caller sets.
|
||||
- No auth middleware wraps the mux. `/ptt` (`main.go:1037`) proxies straight to the voice
|
||||
server, so an unauthenticated caller also gets voice turns — and can answer its own
|
||||
confirm turn.
|
||||
- Separately: `docker-compose.yml` contains a literal committed credential,
|
||||
`-kuma-key uk5_mavpoll-key`.
|
||||
|
||||
**attack path:** anyone who reaches `homesrv:9201` → `POST /tools` with
|
||||
`action=enable&name=x&cmd=<any binary + args>` → `POST /ptt` uttering `x` → arbitrary
|
||||
command execution as the mavend user. No credential needed.
|
||||
|
||||
**impact:** The trust boundary the code carefully reasons about (`internal/tool` doc
|
||||
comment: *"a compromised router can't grant itself a capability"*) is real, but the outer
|
||||
boundary it depends on is an unwritten deployment assumption. Code safe, deployment not.
|
||||
|
||||
**recommended action:** Two small changes, not a redesign.
|
||||
(a) Bind mavweb to loopback in compose — `ports: ["127.0.0.1:9201:9201"]` — so reaching it
|
||||
requires the wg tunnel by construction, not by convention.
|
||||
(b) Make the fail-open explicit and loud: if `-webauthn-origin` is unset, log a startup
|
||||
warning naming `/tools` and `/api/revert` as unguarded, and add a `-require-stepup` flag
|
||||
that fails closed for when you do want it.
|
||||
Move the kuma key into `deploy/telegram.env` (already gitignored) and rotate it.
|
||||
|
||||
**note:** The fail-open is a **documented deliberate choice** (`main.go:411-418` explains
|
||||
it, and it is correct that gating on an unassertable session would 403 permanently). The
|
||||
finding is that the compensating control is absent from the repo, not that the reasoning
|
||||
is wrong.
|
||||
|
||||
**risk:** Low. Loopback binding breaks off-box access only if something other than wg
|
||||
currently reaches 9201 directly — worth confirming on the box first.
|
||||
|
||||
---
|
||||
|
||||
### 4. `cmd/mavend/voice.go` is 1,715 lines doing six unrelated jobs
|
||||
|
||||
`refactor` — later, not now
|
||||
|
||||
**problem:** One file holds daemon wiring (`wireVoice`, 240 lines), the turn handler,
|
||||
intent dispatch (`applyAction`, 280 lines), Praxis/Hexis ecosystem acts (~250 lines), RU
|
||||
text utilities (`ruPlural`, `stripWake`, `hasDurationWords`, `formatTime`), classifier
|
||||
seeding, and weather parsing.
|
||||
|
||||
**evidence:** confirmed — 33 functions, largest file in the project by 40%; `cmd/mavend`
|
||||
coverage is 29.8%, the lowest of any substantial package, precisely because this file mixes
|
||||
wiring (hard to test) with pure logic (trivial to test).
|
||||
|
||||
**impact:** Not urgent — nothing is broken. But it is the reason the router flip in finding
|
||||
1 feels risky: the wiring you must touch sits in the same file as the dispatch logic you
|
||||
must not break.
|
||||
|
||||
**recommended action:** Split along seams that already exist: `wire_voice.go`
|
||||
(construction), `handler.go` (turn), `acts_ecosystem.go` (Praxis/Hexis — already partly in
|
||||
`ecosystem.go`), and move the RU helpers to `internal/ttsnorm` or a new `internal/rutext`.
|
||||
Pure-function extraction lifts `cmd/mavend` coverage with no behavior change. **Do this
|
||||
after finding 1, not before** — and skip it if it stays purely cosmetic.
|
||||
|
||||
---
|
||||
|
||||
### 5. Documentation outweighs its own usefulness
|
||||
|
||||
`cleanup` / `deletion`
|
||||
|
||||
**problem:** 15 root markdown files, 4,518 lines, several stale or superseded, at least
|
||||
three pairs contradicting each other.
|
||||
|
||||
**evidence:** `ROADMAP.md` (759) + `docs/ecosystem.md` (884) +
|
||||
`PROGRESS.md` (456) + `maven.md` (413) + `20-07-2026-BACKLOG.md` (396) +
|
||||
`SESSION-05-07-2026.md` + `SESSION-06-07-2026.md` (477 combined) + `PLANS.md` (25) +
|
||||
`docs/operations.md` + `SPEC.md` + `docs/protocol.md`. `PROGRESS.md:61` annotates its own staleness:
|
||||
*"Older LFM references below describe the currently deployed..."*. `docs/rearchitecture.md` announces it
|
||||
"supersedes" a model still described as current elsewhere.
|
||||
|
||||
**impact:** The doc set is the reason finding 1 exists. When five documents describe the
|
||||
architecture, the code becomes the only trustworthy one — which defeats the purpose of
|
||||
having them.
|
||||
|
||||
**recommended action:** Keep `CLAUDE.md` (agent contract), `docs/rearchitecture.md` (target
|
||||
architecture), `AGENTS.md` (recipes), `docs/protocol.md` (wire format),
|
||||
`20-07-2026-BACKLOG.md` (live queue). Delete the two `SESSION-*.md` and `PLANS.md` — git
|
||||
history holds them. Fold `SPEC.md` + `maven.md` + `ROADMAP.md` into one `docs/design.md` and
|
||||
mark superseded sections instead of leaving them to read as current. Target ~1,500 lines.
|
||||
|
||||
---
|
||||
|
||||
## keep
|
||||
|
||||
- `internal/tool` — argv-only executor, store allowlist, confirm gate. Well-reasoned; the
|
||||
doc comment explains *why* rather than *what*.
|
||||
- `internal/loop`, `routine`, `morning`, `pattern`, `dialogue` — pure engines, impure
|
||||
drivers in `tick.go`, 86–100% coverage. That pattern is working; keep applying it.
|
||||
- The delivery outbox (Begin-before-send / Complete-after / unknown-on-crash).
|
||||
- Dependency discipline: 5 direct deps for a voice assistant is excellent.
|
||||
- The uncommitted `internal/morning` work — follows the established pattern correctly, and
|
||||
its rationale comment ("why not four timers") is the right kind of comment.
|
||||
|
||||
## remove
|
||||
|
||||
`SESSION-05-07-2026.md`, `SESSION-06-07-2026.md`, `PLANS.md`. Stale `LFM` comments in 5
|
||||
files once the model is pinned. The committed kuma key. `n_gpu_layers: 99` **or** the
|
||||
CPU-only claim in `CLAUDE.md` — whichever is false.
|
||||
|
||||
## repair now
|
||||
|
||||
1. **Toolchain / `make test` exit code** (finding 2) — ~10 minutes, restores the safety
|
||||
net. Do this first; everything else is verified through it.
|
||||
2. **Bind mavweb to loopback + startup warning on unguarded step-up + rotate the kuma key**
|
||||
(finding 3) — ~30 minutes, closes an unauthenticated RCE path.
|
||||
3. **Pin the model and reconcile all five docs + config** (finding 1, first half) — the
|
||||
decision is yours; the edit is mechanical once made.
|
||||
4. **Flip the LLM router behind a config flag and evaluate with `eval_scenarios_test.go`**
|
||||
(finding 1, second half) — the change that addresses the actual user-facing complaint.
|
||||
|
||||
## refactor later
|
||||
|
||||
Split `voice.go` (finding 4). Consolidate docs (finding 5). Raise `internal/phraser`
|
||||
coverage from 16.5% — lowest in the project, and it owns the llama-server subprocess
|
||||
lifecycle, exactly where a leak or orphan would hide (the `Pdeathsig` handling looks right,
|
||||
but it is nearly untested).
|
||||
|
||||
## rewrite only if
|
||||
|
||||
Nothing here justifies a rewrite. `internal/router` is the only candidate and it fails the
|
||||
test: the target engine is already written and tested, the fallback is deliberate, and
|
||||
there is no compatibility burden. **Incremental correction is clearly cheaper.** Revisit
|
||||
only if the LLM router needs a fundamentally different `Decision` shape than the cascade
|
||||
produces — evaluate that with the scenario harness before touching structure.
|
||||
|
||||
```
|
||||
incremental repair cost: one wiring change + config flag + doc sweep
|
||||
rewrite cost: re-derive 7-intent contract, slot extraction, RU
|
||||
grammars, stage-0 fast path
|
||||
migration cost: n/a — single user, no external consumers
|
||||
behavior at risk: stage-0 fast path, RU slot extraction, confirm turns
|
||||
tests available: router_test, slots_ru_test, eval_scenarios_test
|
||||
hidden knowledge in code: high — RU qualifier pre-processing, seed collisions,
|
||||
wake-token stripping across scripts
|
||||
expected maintenance gain: none over the incremental path
|
||||
```
|
||||
|
||||
## verification plan
|
||||
|
||||
1. `make test` exits 0 (currently broken, so this is step zero).
|
||||
2. `make build` — all 8 binaries.
|
||||
3. `go vet ./internal/... ./cmd/...` stays clean.
|
||||
4. Run `eval_scenarios_test.go` against classifier-only and LLM-router paths; compare RU
|
||||
query accuracy.
|
||||
5. `start-maven.sh` locally; exercise a voice turn and a destructive-tool confirm turn.
|
||||
6. Confirm `curl homesrv:9201/tools` is unreachable from off-box after the bind change.
|
||||
|
||||
## uncertainties
|
||||
|
||||
- **Which model is actually running on homesrv.** Repo evidence points three ways;
|
||||
unresolvable without the box. Everything in finding 1 depends on the answer.
|
||||
- **Whether `n_gpu_layers: 99` is correct.** Compose passes `/dev/dri` and the render gid,
|
||||
suggesting Vulkan offload is intended and working — but `CLAUDE.md` says CPU-only. Likely
|
||||
the doc is stale, not the config; unverified.
|
||||
- **Whether the classifier is genuinely adequate.** `voice.go:211` asserts it is;
|
||||
`docs/rearchitecture.md` asserts it is not. Both are claims, neither is measured. The uncommitted eval
|
||||
harness is the instrument to settle it — resolve before flipping the router, not after.
|
||||
- **Whether wg+nginx+auth actually fronts 9201 in production.** Not in this repo. If it
|
||||
does, finding 3 drops from "unauthenticated RCE" to "the control is not reproducible from
|
||||
the repo" — still worth fixing, much less urgent.
|
||||
- The daemons were not run and routing latency was not measured.
|
||||
|
||||
---
|
||||
|
||||
## second opinion (Claude, 2026-07-30)
|
||||
|
||||
Key claims independently re-verified against the working tree; the verdict stands.
|
||||
|
||||
**Confirmed directly:**
|
||||
|
||||
- Finding 1: `voice.go:211` passes `nil` for the LLM router; `deploy/mavend.json:9`
|
||||
points at `Qwen3.5-2B-UD-Q4_K_XL.gguf`; `models/llm/` on disk holds only
|
||||
`LFM2.5-1.2B-Thinking-Q4_K_M.gguf`; `CLAUDE.md` says Qwen3-1.7B. At least three live
|
||||
contradictions, exactly as stated.
|
||||
- Finding 2: reproduced — all packages pass, `make: *** Error 1` anyway; vendored
|
||||
toolchain is go1.23.4 vs `go 1.25.5` in `go.mod`. Correctly sequenced first.
|
||||
- Finding 3: `docker-compose.yml` publishes `9201:9201` with no webauthn flags, and the
|
||||
kuma key `uk5_mavpoll-key` is committed verbatim. Attack path is coherent.
|
||||
|
||||
**Nuances to the review's framing:**
|
||||
|
||||
- Finding 3's severity depends entirely on network topology. If 9201 is only reachable
|
||||
over WireGuard on homesrv, this is hygiene, not an emergency — but the loopback bind
|
||||
and startup warning are cheap insurance either way, so do them regardless.
|
||||
- Finding 1's "flip the router" step should be gated harder on measurement.
|
||||
`docs/rearchitecture.md`'s claim that the classifier causes weak RU queries is itself unmeasured —
|
||||
the review admits this under uncertainties, but the "repair now" ordering buries it.
|
||||
Run `eval_scenarios_test.go` against both paths **before** deciding to flip, not
|
||||
after. A 2B model on CPU may add enough latency that the classifier wins in practice
|
||||
even if it is dumber.
|
||||
- Finding 5 (doc consolidation) is what prevents finding 1 from recurring. Do the doc
|
||||
sweep as part of the model-pinning commit, not "later."
|
||||
|
||||
**Agreed without reservation:** the no-rewrite conclusion, the keep list, and the
|
||||
overall classification (healthy engineering, misaligned direction).
|
||||
+803
@@ -0,0 +1,803 @@
|
||||
# Maven — Design
|
||||
|
||||
*Last verified: 2026-08-02 @ 7079a24. Living doc: correct it in place, do not append.*
|
||||
|
||||
> Folded 2026-07-30 from `SPEC.md` (north star, 2026-07-03), `maven.md`
|
||||
> (consolidated decisions, 2026-06-30) and `ROADMAP.md` (execution plan,
|
||||
> 2026-07-06). Those three files are gone; git history holds them.
|
||||
> This is the single design document: principles, target state, and the
|
||||
> execution ledger. `docs/rearchitecture.md` remains authoritative wherever it disagrees
|
||||
> with anything here. Everything the three sources asserted that is no longer
|
||||
> the intended design is preserved under **§ Superseded** — do not read that
|
||||
> section as current.
|
||||
|
||||
---
|
||||
|
||||
## Identity
|
||||
|
||||
**Maven** — self-hosted personal assistant. Manages your day, acts on your
|
||||
homelab. One daemon on homesrv (always-on, not the workstation), multiple
|
||||
client surfaces. Inference and data stay on the box; she may READ external
|
||||
sources (see Non-goals — "never phones home" is deprecated).
|
||||
|
||||
Primary name is "Maven", with feminine-gendered Russian self-reference
|
||||
("она", "меня", "помогла"). Clients may choose their own UI label. Consistent
|
||||
character — tone, values, phrasing — pinned in prompt; it is what makes
|
||||
restraint legible.
|
||||
|
||||
## Non-goals
|
||||
|
||||
Outside boundary: not Alexa/Siri on steroids, not a smart device.
|
||||
|
||||
Inside boundary — the ones that actually constrain the build:
|
||||
|
||||
- **Not autonomous** — suggests and acts on command. Proactive triggers never
|
||||
get unsandboxed action rights. "backup failed, rerun it?" — never reruns it
|
||||
herself. suggest ≠ act is the safety model.
|
||||
- **Not a guesser-of-truth** — inference changes whether she asks, never what
|
||||
she records. A confident wrong fact is worse than a known gap.
|
||||
- **Not a nag** — she'd rather miss a nudge than be mutable. Shuts up when
|
||||
uncertain. Load-bearing.
|
||||
- **Not a stranger** — runs on your stuff, your model, your data. No
|
||||
telemetry, no cloud model, no third-party account. She may READ external
|
||||
sources to answer world questions (Kiwix first, then optional search); she
|
||||
never reports anything about you to anyone, and your notes and facts are
|
||||
never used as search input. **"Never phones home" as an absolute is
|
||||
deprecated** — owner's call, 2026-07-31: a small model does not know enough
|
||||
to be useful without reading.
|
||||
- **Not a relationship** — mom-tone is a function that makes nudges land, not
|
||||
emotional company. Names the drift a warm small model falls into.
|
||||
|
||||
## Capabilities
|
||||
|
||||
- **Reactive** — converse (voice in → STT → router → LLM → TTS, and text);
|
||||
act (function calls into the homelab).
|
||||
- **Proactive** — health nudges (hydration, meals, breaks, shower, sleep,
|
||||
cleanup); user reminders (stated future intent, fires once); deliver (voice
|
||||
when near, ntfy/telegram when away); restrain (quiet hours, per-rule
|
||||
cooldowns, snooze-memory, self-quieting).
|
||||
- **Capture** — throw facts/notes/tasks at it mid-flow.
|
||||
- **State** — self (timestamped facts about you), presence (inferred, decaying
|
||||
confidence, never one signal), activity, environment (homelab health,
|
||||
calendar, weather).
|
||||
- **Memory** — long-term recall and personalization.
|
||||
- **Feedback** — nudge outcomes (acted/snoozed/ignored) tune the rules;
|
||||
corrections are recorded; self-quieting falls out of this.
|
||||
- **Surface** — you talk to it (phone page, PC client), it reaches you, and it
|
||||
can prove it's you (auth).
|
||||
|
||||
## Users
|
||||
|
||||
| Phase | Users | Data model |
|
||||
|-------|-------|------------|
|
||||
| Now (MVP) | just me | single-user, no namespace |
|
||||
| Soon | me + gf | per-user namespace (facts/notes/reminders partitioned by speaker attribution) |
|
||||
|
||||
Per-user means: when the router attributes an utterance to user X, writes go
|
||||
into X's partition, and reads are user-scoped. Shared state (house chores,
|
||||
shared calendar busyness) is explicitly cross-partition via a `shared` /
|
||||
`household` namespace. The router owns attribution — speaker recognition for
|
||||
voice, surface ownership for text.
|
||||
|
||||
**This is post-MVP and fenced.** The schema has no `user_id` columns. Adding
|
||||
them later is a migration, not a rewrite, because append-only means no
|
||||
existing row needs updating. **An agent must not introduce user-scoping
|
||||
mechanisms while single-user is the only operational mode.** Revisit when a
|
||||
second person is actually in the house — speaker attribution needs the second
|
||||
voice to train against.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
- Daemon lives on homesrv (always-on), not the workstation.
|
||||
- The trigger loop is dumb: ticks ~60s, no LLM, evaluates deterministic
|
||||
predicates against state.
|
||||
- The LLM wakes only when a predicate fires; its job is narrow — phrase,
|
||||
never drive the loop.
|
||||
- Presence is a decaying confidence score over multiple weak signals with
|
||||
hysteresis; never trust one source.
|
||||
- Self-facts → care nudges. World-facts → ops + context. Same engine.
|
||||
- Rules as code, not a config DSL — revisit at ~30 rules.
|
||||
- Proactive triggers: read + suggest only, never action rights.
|
||||
|
||||
**Build order:** the state layer is first. Nothing proactive works without
|
||||
state to evaluate predicates against — it is the floor, built before the
|
||||
loop, phrasing, or delivery.
|
||||
|
||||
### Storage — sqlite
|
||||
|
||||
Single-user, no concurrent writers, on a box already tight on RAM → sqlite,
|
||||
not postgres. Library not a process, no port to harden, backup is `cp`.
|
||||
Giving up postgres `LISTEN/NOTIFY` is a non-loss: the loop polls anyway.
|
||||
|
||||
At-rest encryption is AES-256-GCM with a tmpfs working copy
|
||||
(`internal/store/crypt.go`), **not** sqlcipher. The key is read at daemon
|
||||
start, never hardcoded.
|
||||
|
||||
```sql
|
||||
PRAGMA journal_mode=WAL;
|
||||
PRAGMA synchronous=NORMAL;
|
||||
PRAGMA foreign_keys=ON;
|
||||
PRAGMA busy_timeout=5000;
|
||||
```
|
||||
|
||||
### Schema — append-only, three shapes
|
||||
|
||||
Never UPDATE a value. A wrong fact is superseded, not overwritten — this
|
||||
keeps the audit trail. Current value = latest non-voided row for a key.
|
||||
|
||||
**facts** — substrate, all observations (self + env):
|
||||
|
||||
```sql
|
||||
facts (
|
||||
id, ts, -- ts = valid-time (true-as-of), not insert-time
|
||||
kind, -- 'self' | 'env' | 'config'
|
||||
key, value, -- value json if structured
|
||||
source, -- tap:* | infer:* | poll:* | ambient | promote | feedback
|
||||
confidence, -- 1.0 taps only; <1 inferred
|
||||
voids_id -- correction points at the fact it cancels
|
||||
)
|
||||
-- index (key, ts desc)
|
||||
```
|
||||
|
||||
**reminders** — user intent, fires once:
|
||||
|
||||
```sql
|
||||
reminders ( id, created_ts, fire_ts, payload, status ) -- pending|fired|cancelled
|
||||
```
|
||||
|
||||
**nudges** — every proactive send + outcome. This table IS the restraint
|
||||
memory:
|
||||
|
||||
```sql
|
||||
nudges ( id, ts, rule, channel, message, outcome, outcome_ts ) -- pending|acted|snoozed|ignored
|
||||
```
|
||||
|
||||
**Presence is not a table** — it is a pure function over recent facts,
|
||||
computed each tick. The only stateful bit is hysteresis:
|
||||
|
||||
```sql
|
||||
presence_state ( last_bucket, last_score, updated_ts )
|
||||
```
|
||||
|
||||
Facts additionally carry `Subject`/`EntityID`/`ResolutionState` for
|
||||
entity-aware resolution against Nexus (see `docs/ecosystem.md`).
|
||||
|
||||
### Trigger model
|
||||
|
||||
- The loop ticks ~60s, no LLM. 99% of ticks evaluate a few predicates and die
|
||||
for free.
|
||||
- A predicate is `(State) -> Boolean`, **pure, no I/O** → unit-tests with a
|
||||
fake State, zero infra.
|
||||
- `since(key)==null` → don't fire. Silence on no-data is "shuts up when
|
||||
uncertain."
|
||||
- **The gate is universal, applied by the loop, never per-rule** —
|
||||
quiet-hours, presence, cooldown, snooze, calendar-busy all live in one
|
||||
`fires()`. Cross-cutting restraint lives in one place or it drifts.
|
||||
- One nudge per tick (max severity), never dogpile.
|
||||
|
||||
### Rules decide, LLM phrases
|
||||
|
||||
The rule decides whether Maven speaks — absolute, deterministic. The LLM only
|
||||
words it: input `(rule, severity, context)`, output `message`. **No
|
||||
`send`/veto bool** — a nondeterministic small model never gets to silently
|
||||
kill a greenlit nudge. Suppression context ("don't nag mid-meeting") moves
|
||||
INTO the gate as an env predicate, not the LLM's job.
|
||||
|
||||
### User reminders — a separate class
|
||||
|
||||
- Relative → absolute **at capture** ("in 4h" → store `now+4h`, never the
|
||||
string).
|
||||
- Reuses the loop, not a second scheduler — just a predicate:
|
||||
`fire_ts <= now AND pending`.
|
||||
- **Bypasses the restraint gate** — "wake me 7" fires in quiet hours; that's
|
||||
the point. Snooze still applies. Two delivery paths.
|
||||
|
||||
---
|
||||
|
||||
## Reactive path — routing
|
||||
|
||||
**Target design: LLM-as-router** (see `docs/rearchitecture.md` and `CLAUDE.md`). One
|
||||
resident model emits GBNF-constrained structured JSON, and the same model
|
||||
phrases replies; the embedder is a RAG hint, not a routing gate. The
|
||||
committed default today is the classifier/embedder cascade, which is an
|
||||
interim stopgap — see **§ Superseded**.
|
||||
|
||||
### A cascade, not one decider
|
||||
|
||||
Not alternatives — layers:
|
||||
|
||||
- **Stage 0 — exact match (regex/grammar).** Wake-word + known command
|
||||
grammar. "maven, restart nginx" hits the allowlist directly and skips
|
||||
everything downstream. Lowest latency; boring high-frequency acts for free.
|
||||
- **Stage 1 — route decision.** The resident LLM (target) or the
|
||||
nearest-centroid classifier (current stopgap). Any LLM error falls through
|
||||
to the classifier so a turn never breaks on the model.
|
||||
- **Stage 2 — slot extraction, per intent.** Classification gives *what
|
||||
kind*, not *the args*. Reminders need a datetime, acts need fn + params.
|
||||
- **Stage 3 — confidence gate.** Below threshold → clarify, don't guess. Same
|
||||
pattern as `since(key)==null → don't fire`. A misroute is a confident wrong
|
||||
write, which is worse than a gap.
|
||||
|
||||
Router contract: `[{"intent":<enum>, key?, value?, text?, verb?}, ...]` over
|
||||
7 intents (`fact, reminder, note, query, act, chat, system`).
|
||||
|
||||
### save-where — the two-memory routing axis
|
||||
|
||||
One discriminator: **does the loop evaluate a predicate against it?**
|
||||
|
||||
| intent | example | lands in | why |
|
||||
|---|---|---|---|
|
||||
| act | "restart the backup" | function call (allowlist) | command now, not stored |
|
||||
| reminder | "wake me 7", "vet tuesday" | `reminders` | has a fire-time |
|
||||
| fact | "drank water", "slept 6h" | `facts` | structured state the loop reasons over |
|
||||
| note | "gpu driver fixed the flicker" | semantic store | recall/preference, no predicate touches it |
|
||||
| query | "is the backup up?" | LLM over the stores | answer, don't store |
|
||||
|
||||
fact-vs-note is the whole line: a predicate will read it → structured `facts`
|
||||
row; "recall when relevant" → semantic store. Reminder splits off by future
|
||||
timestamp; act splits off by being imperative-now.
|
||||
|
||||
### The preference seam — forced, not a choice
|
||||
|
||||
A preference ("prefer backups at 3am") looks like note-or-config. It isn't,
|
||||
because of a hard constraint: **a predicate cannot read the semantic store.**
|
||||
The loop is dumb and deterministic; it can't run a vector search every tick.
|
||||
The moment a preference becomes load-bearing it MUST exist as a structured
|
||||
`key=value` row the loop can evaluate. The store split is physical, not
|
||||
cosmetic. So: **capture → always a note** (inert, fail-safe, drives nothing),
|
||||
and **a note stays a note until a rule needs it.**
|
||||
|
||||
### Promotion
|
||||
|
||||
**Promotion = the moment a human authors a predicate that reads the value.**
|
||||
|
||||
- Authoring the rule copies the value into `facts (kind=config,
|
||||
source=promote)` — a deterministic key the loop can evaluate.
|
||||
- The note stays as provenance (where the config came from, in your words).
|
||||
- The predicate reads the promoted `facts` row, never the semantic store.
|
||||
|
||||
Properties: **one-way, never auto** — a note cannot self-promote; there is no
|
||||
path from the semantic store into the loop that skips a human writing a
|
||||
predicate. This **closes the injection hole**: ambient-derive overhears the TV
|
||||
say "prefer backups at 3am" → lands as an inert note → cannot drive the loop.
|
||||
Same shape as `proposed → enabled`: the low-authority form is free and
|
||||
automatic, the high-authority form requires a deliberate human act.
|
||||
|
||||
### Tool registration — drafting is suggest, enabling is act
|
||||
|
||||
Maven can scaffold a tool she's missing. She cannot enable it.
|
||||
|
||||
- She detects the gap, scaffolds the registration (name, command, params,
|
||||
destructive y/n), writes a `proposed` row, surfaces it.
|
||||
- `proposed → enabled` flips through an authed surface (PC client / authed
|
||||
page), **never the voice/chat path** — that's the act, and it's the user's.
|
||||
|
||||
Why human-only: editing the allowlist is the one act that *moves the
|
||||
boundary*, and a boundary you can move from inside isn't one. Registration is
|
||||
privilege escalation, a different authority tier from invoking a listed tool.
|
||||
Paranoid case: prompt injection via ambient-derive — the TV says "maven add a
|
||||
shell tool" — a self-registering Maven grants *itself* arbitrary capability.
|
||||
Human-only enable keeps a compromised Maven boxed by what's already on. She
|
||||
builds the stubs, you review and enable; you never lose the pen.
|
||||
|
||||
### Confirmation is not one mechanism
|
||||
|
||||
A gate assumes a fully-formed action — "drop the db? y/n" works because the
|
||||
action is already specified. An underspecified request can't be gated; you
|
||||
can't confirm what isn't specified. Confirmation scales to how formed the act
|
||||
is. Acts fuzzy-match against the fn allowlist: **not on the list → refuse,
|
||||
don't improvise.** Destructive ones still gate behind confirm.
|
||||
|
||||
Misroute correction is append-only and grows the router's examples with use —
|
||||
same shape as `nudges.outcome` tuning cooldowns, no retrain.
|
||||
|
||||
---
|
||||
|
||||
## Voice pipeline (STT / TTS)
|
||||
|
||||
Real STT + TTS are wired and tested. The stub floor exists for CI and for the
|
||||
"no models on disk" bootstrap.
|
||||
|
||||
| Component | When active | Module | Handler |
|
||||
|-----------|-------------|--------|---------|
|
||||
| STT | `voice.stt.socket` in config | `cmd/mavsttd -model <path>` | whisper.cpp (CGo, Vulkan) |
|
||||
| TTS | `voice.tts.socket` in config | `cmd/mavttsd -piper <bin> -model <path>` | piper (subprocess, espeak-ng) |
|
||||
| STT stub | socket unset / no `-model` | in-process `stt.Stub` or `mavsttd` stub | hash + template |
|
||||
| TTS stub | socket unset / no `-piper` | in-process `tts.Stub` or `mavttsd` stub | 200ms tone |
|
||||
|
||||
The server has an iGPU + Vulkan. whisper.cpp uses Vulkan; piper uses CPU
|
||||
(lightweight, real-time). Voice defaults to the stock piper RU voice
|
||||
(`ru_RU-irina-medium.onnx`); a custom trained voice is a model-file swap
|
||||
(`-model` / `VoiceConfig.Tts.Voice`), not a code change.
|
||||
|
||||
## Resident language model
|
||||
|
||||
One llama-server process serves both the grammar-constrained route contract
|
||||
and the persona/response contract. The target artifact is produced by RU
|
||||
continued pretraining followed by joint persona/router SFT. Stage-0 grammar,
|
||||
the classifier, and stub phrasing remain availability fallbacks. A larger
|
||||
on-demand reasoner is deferred until the main feature set is complete.
|
||||
|
||||
Models are per-component and downloaded separately (gitignored `models/`);
|
||||
no model is baked into a binary. llama-server runs with `-ngl -1`.
|
||||
|
||||
**Resident checkpoint (resolved 2026-07-30, task #318).** Currently
|
||||
**Qwen3.5-0.8B** (`Q4_K_M`) — the smallest checkpoint in the gguf library,
|
||||
chosen for latency on the deploy box. The **target** is the locally CPT'd
|
||||
**Qwen3-1.7B**, whose training is still in flight (Vikunja #122); until that
|
||||
produces a gguf, 0.8B is what runs.
|
||||
|
||||
The library lives at `/mnt/hdd1/llms`, bind-mounted to
|
||||
`/opt/maven/models/llm`, which **shadows** the repo's `models/llm/` — the
|
||||
LFM2.5-1.2B gguf in the repo tree is a leftover and is never loaded. Earlier
|
||||
docs claiming LFM2.5 or a 2B Qwen as resident described states that are no
|
||||
longer current; see § Superseded.
|
||||
|
||||
Persona is configurable: `voice.persona` in `mavend.json` is prepended to
|
||||
every system prompt; empty means the built-in feminine-gendered Russian
|
||||
persona.
|
||||
|
||||
## ML & hardware profile
|
||||
|
||||
| Resource | Available | Used by |
|
||||
|----------|-----------|---------|
|
||||
| CPU | Ryzen 5 5600U, 13GB RAM | loop, router, delivery |
|
||||
| iGPU | Vega, Vulkan | whisper.cpp (STT), piper (TTS), llama-server offload |
|
||||
| llama-server | `n_gpu_layers: 99` | resident model — Qwen3.5-0.8B now, Qwen3-1.7B target (#122) |
|
||||
|
||||
---
|
||||
|
||||
## State — signal sources
|
||||
|
||||
**Explicit taps set truth. Passive signals drive prompting.** A passive
|
||||
signal can never write a self-fact — it only makes Maven ask. The boundary
|
||||
lives in `source`; rules trust provenance.
|
||||
|
||||
- **self / taps (1.0):** water, meal, shower — phone / voice / telegram.
|
||||
- **self / passive (activity, not truth):** desk-idle, voice_active, sleep.
|
||||
- **presence (weak, decaying, multi-source):** wg handshake, page heartbeat,
|
||||
desk-not-idle. *No LAN sweeps* — too invasive; wg + heartbeat get ~90%.
|
||||
- **env / polled:** healthcheck (disk/service/backup/cert), calendar
|
||||
(CalDAV), weather.
|
||||
- **provenance-scoping:** a rule on `service_down` trusts only
|
||||
`source=poll:healthcheck`. A compromised poller must not be able to forge a
|
||||
trigger.
|
||||
|
||||
### Presence — concrete scoring
|
||||
|
||||
**Combiner — noisy-OR, not weighted sum.** These are independent-ish positive
|
||||
indicators of one latent binary ("kami here?").
|
||||
|
||||
```
|
||||
p_i = weight_i · exp(-Δt_i / τ_i) # one signal's decayed contribution
|
||||
P = 1 − Π (1 − p_i) # combine
|
||||
```
|
||||
|
||||
Diminishing returns on stacking weak signals; never exceeds 1.0. "Any one
|
||||
signal raises confidence, no single one owns it" falls straight out.
|
||||
Weighted-sum was rejected: heartbeat+wg fresh would peg identical to
|
||||
everything-fresh — overcounts.
|
||||
|
||||
**Signals.** `Δt` = `now − latest fact ts` for that key. No fact → drops out
|
||||
of the product (not a zero).
|
||||
|
||||
| signal | source | fresh weight | τ (min) | rationale |
|
||||
|---|---|---|---|---|
|
||||
| desk active | `infer:hyprland` (not-idle) | 0.90 | 8 | input = human at keyboard. strongest. τ forgives read/think gaps |
|
||||
| page heartbeat | `infer:heartbeat` | 0.60 | 4 | a surface you use is open + alive. pings ~30s; 4min gap = gone |
|
||||
| wg handshake | `infer:wg` | 0.40 | 20 | device on tunnel. coarse — pocket or three rooms away |
|
||||
|
||||
**Threshold + hysteresis (schmitt trigger):**
|
||||
|
||||
```
|
||||
ENTER (away → present): P ≥ 0.55
|
||||
EXIT (present → away): P < 0.30
|
||||
cold start: away # fail-closed; same as since(key)==null → don't fire
|
||||
```
|
||||
|
||||
A wide band is stable. A lone fresh wg (0.40) can't *enter* present but
|
||||
*holds* it while decaying — phone-on-network alone never declares you here.
|
||||
|
||||
| scenario | P | bucket |
|
||||
|---|---|---|
|
||||
| at desk, typing | ~0.90 | present |
|
||||
| at desk, 5min reading (client open) | ~0.79 | present |
|
||||
| desk-only, ~9min zero input | <0.30 | → away |
|
||||
| couch, phone page open, no desk | 0.60 | present |
|
||||
| left house, only wg lingering | decays ~20min | → away |
|
||||
| cold boot, nothing | 0 | away |
|
||||
|
||||
Implemented in core (`internal/store/presence.go`) — presence reads State
|
||||
under the lock; it is predicate input, not a module. Each tick: score →
|
||||
resolve against the last bucket → persist `presence_state`. Decay uses
|
||||
wall-clock Δt, so tick jitter causes no drift.
|
||||
|
||||
Boundaries are **hand-tuned, NOT feedback-tuned** — keep presence numbers out
|
||||
of the auto-tuner or a weird week drifts you silently invisible. **Presence =
|
||||
reachability, not wakefulness** — sleep/quiet-hours are handled separately in
|
||||
the gate. Stated caveat: noisy-OR assumes independence and desk+heartbeat
|
||||
correlate; co-firing slightly overcounts, which is fine — genuinely co-firing
|
||||
IS stronger evidence.
|
||||
|
||||
---
|
||||
|
||||
## Proactive
|
||||
|
||||
### Listening — three modes, not one
|
||||
|
||||
"Ambient listening" was the wrong frame; there are three capabilities split
|
||||
by trigger + retention. The threat model is local-only: the file at rest and
|
||||
who can reach the box.
|
||||
|
||||
1. **address-capture** — "maven, note this" out loud. Hands-free voice path,
|
||||
no tap. Raw audio dies after deriving the fact. **← MVP pick.** Lowest
|
||||
retention, highest daily payoff.
|
||||
2. **meeting-record** — deliberate start/stop, verbatim transcript *kept*.
|
||||
Retention is the point (overrides any "always delete" rule). Post-MVP.
|
||||
3. **ambient-derive** — background overhearing, low-confidence candidate
|
||||
facts, raw ephemeral, only the derived fact survives. Needs the confidence
|
||||
model working first. Last.
|
||||
|
||||
Only mode 3 is always-on. Modes 1–2 fire heavy transcription on an explicit
|
||||
trigger → the CPU idles otherwise. Always-on means lightweight VAD (+ maybe
|
||||
owner-detect) only. **Speaker-scoping is attribution metadata, not a
|
||||
kill-gate** — tag `source=ambient:self | ambient:other`, so per-person
|
||||
retention becomes a `WHERE` clause.
|
||||
|
||||
`cmd/mavwaked/` ships as energy-VAD only, no wake-word model — every
|
||||
utterance fires, capped at `SurfaceVoice` (L0). Its 30ms/16kHz frame shape
|
||||
matches silero-vad ONNX input 1:1, so swapping in a real wake-word model
|
||||
(silero-vad / openWakeWord) is a local change in `vad.go`. Hardware topology
|
||||
is settled: mavwaked is a *client* binary (desk PC / Pi with a mic, systemd
|
||||
user unit), not a homesrv daemon.
|
||||
|
||||
### Delivery / channel routing
|
||||
|
||||
Routing = `f(severity, presence)`. Presence decides *reachability*, severity
|
||||
decides *insistence*. Both are needed.
|
||||
|
||||
| | present | away |
|
||||
|---|---|---|
|
||||
| **sev1–2** (care) | voice | **drop** |
|
||||
| **sev3** (ops, soft) | voice | ntfy, once |
|
||||
| **sev4** (ops, hard) | voice + ntfy | telegram, repeat til ack |
|
||||
|
||||
sev ≤ 2 drops on away, sev ≥ 3 holds: a missed water nudge is noise, a missed
|
||||
backup failure isn't. Away-channels (ntfy/telegram) leave the box — the one
|
||||
path that leaves the box for a person to see, through your own relay. **Minimal
|
||||
body** — "disk low on homesrv," not detail; don't make notifications a
|
||||
shoulder-surf exfil surface.
|
||||
|
||||
The same table governs runtime fallthrough: when the dispatcher chose voice
|
||||
but no session is live at push time (`ErrNoSession`), it falls through to the
|
||||
next channel on this table rather than stopping. Delivery is durable —
|
||||
`BeginDeliveryAttempt` before `Send`, `CompleteDeliveryAttempt` after, with a
|
||||
stale `pending` row reconciled to `unknown` at startup (never silently
|
||||
resent or dropped).
|
||||
|
||||
### Feedback loop (outcomes → tune cooldowns)
|
||||
|
||||
The `nudges.outcome` column IS the signal — no new storage. Cooldown becomes
|
||||
a function of recent outcomes, not a constant.
|
||||
|
||||
- mostly `ignored` → nagging into the void → lengthen cooldown / raise
|
||||
threshold
|
||||
- mostly `acted` → landing → leave it, or cautiously shorten
|
||||
- mostly `snoozed` → right nudge, wrong *time* → shift the window, not the
|
||||
frequency
|
||||
|
||||
**Tunes parameters, never logic.** It can widen a cooldown, nudge a
|
||||
threshold, shift a window; it cannot rewrite a predicate or invent a rule.
|
||||
Bounded knobs (`cooldown ∈ [min,max]`) so a weird week can't mutate Maven
|
||||
silent or stalker. Dead simple at MVP: a ratio over the last N, not a learned
|
||||
model — `ignored_rate > 0.7 → cooldown *= 1.5, capped`. **Persist the
|
||||
adjusted cooldown as a fact** (`source=feedback`) — it survives restart and
|
||||
stays visible; why Maven went quiet should be a query, not a mystery.
|
||||
|
||||
### Quiet hours
|
||||
|
||||
The loop reads a `quiet_hours` config fact. A voice toggle ("тихий режим")
|
||||
writes it; a time-window schedule in config and calendar-busy also gate the
|
||||
same way, written at tick boundaries.
|
||||
|
||||
---
|
||||
|
||||
## Auth
|
||||
|
||||
### A cascade, not a pick-one
|
||||
|
||||
Same shape as `confirmation is not one mechanism` — each layer answers a
|
||||
different question.
|
||||
|
||||
| layer | question | mechanism | surface |
|
||||
|---|---|---|---|
|
||||
| 0 — network | on the tunnel at all? | WireGuard | everything. floor |
|
||||
| 1 — device | enrolled box? | mTLS client cert, terminated at proxy (optional) | PC client, authed page |
|
||||
| 2 — session | you, this session? | passkey / WebAuthn | PC client, authed page |
|
||||
| 3 — step-up | you, *right now*, for this act? | passkey user-verification gesture | registration-enable, destructive acts, **core cold-start unlock** |
|
||||
|
||||
wg is necessary-not-sufficient: an unlocked laptop inside the tunnel is
|
||||
"authed" at layer 0 only — that gap is why the upper layers exist. **Passkey
|
||||
over password/token** because step-up is load-bearing: WebAuthn gives
|
||||
per-assertion user verification for free, and the biometric/PIN gesture IS the
|
||||
human-in-the-loop. No shared secret on the box to steal; the private key stays
|
||||
in the enclave/TPM. **mTLS (layer 1) is the optional one** — if dropping a
|
||||
layer, drop mTLS, never the passkey.
|
||||
|
||||
`internal/webauthn/` does real WebAuthn (ES256, sign-count regression) and
|
||||
`cmd/mavweb/webauthn.go` serves enroll + assert; a successful assert bumps the
|
||||
session to L3 for 5 minutes.
|
||||
|
||||
### The invariant — surface caps authority
|
||||
|
||||
**Auth tier is a property of the surface, and the surface caps maximum
|
||||
authority.** You cannot step up past what the channel structurally carries.
|
||||
|
||||
- **voice** presents layer 0 + speaker attribution and STOPS. Speaker
|
||||
verification is attribution, not auth. A room mic is reachable by anyone
|
||||
present → voice is *structurally incapable* of layer 3.
|
||||
- **telegram inbound** = possession of a telegram account + a chat-id
|
||||
allowlist; telegram's auth, outside our control. Weak tier → read + soft
|
||||
acts, never destructive, never registration.
|
||||
|
||||
So voice/chat can never reach registration-enable — **not because auth
|
||||
"failed" but because the channel can't carry the proof.** Destructive acts
|
||||
always gate behind an authed surface for the final confirm.
|
||||
|
||||
### Key provenance
|
||||
|
||||
**The theorem:** no unattended key source survives a powered-on stolen box.
|
||||
Anything the daemon fetches with no human present, a thief who grabs the
|
||||
running laptop fetches too. The question was never "find the secure source" —
|
||||
it's **pick the failure mode:** unattended-but-loses-to-running-theft, or
|
||||
theft-resistant-but-attended. Structural; you can't have both.
|
||||
|
||||
**The trap — TPM alone:** TPM sealing binds release to PCRs and defeats
|
||||
offline disk extraction (real, worth having), but against full-box theft it
|
||||
does nothing — PCRs still match, the thief boots, the TPM unseals on cue. A
|
||||
laptop is portable, so full-box theft is the *likely* case.
|
||||
|
||||
**The resolution — cold-start IS a layer-3 act:** unlocking the DB makes
|
||||
Maven's entire memory readable, the single highest-authority op. "Always-on"
|
||||
means it doesn't need babysitting during *normal operation*; it does NOT mean
|
||||
it survives a cold boot with nobody around. Unlock is **remote-attended** —
|
||||
`systemd-ask-password` over ssh, or pushed through the passkey-authed page.
|
||||
Theft = the box reboots into a locked daemon and stays there.
|
||||
|
||||
| layer | mechanism | buys |
|
||||
|---|---|---|
|
||||
| disk | LUKS2, `systemd-cryptenroll` **TPM2 + PIN** | offline extraction dead (TPM), powered theft needs the PIN in your head |
|
||||
| db key | not at rest: supplied at daemon start, sourced from the authed surface, held in process memory only | unlock is a deliberate gesture, never a file to steal |
|
||||
|
||||
TPM+PIN is the honest middle; a yubikey is the upgrade path *if carried on
|
||||
your body*. **Runtime:** key in daemon RAM → `mlock` the page (no
|
||||
swap-to-disk), swap off or encrypted, zero on shutdown. **The trade:** a cold
|
||||
reboot needs you (remotely) present; in exchange a stolen laptop — running or
|
||||
off — is a brick holding ciphertext.
|
||||
|
||||
Implemented shape (`internal/webauthn/keywrap.go`): **the key is wrapped, not
|
||||
derived.** A passkey assertion doesn't produce deterministic bytes (WebAuthn
|
||||
signatures are randomized), so at enrollment a random 32-byte AES key is
|
||||
generated, wrapped with HKDF-SHA256(credential public key, salt) +
|
||||
AES-256-GCM, and stored on disk; at cold-start the assertion unwraps it. The
|
||||
daemon boots *locked* — the IPC server serves only the unlock method
|
||||
(`MethodStoreEncryptionKey`/`MethodUnlock`), and the loop, voice and delivery
|
||||
don't start until unlock succeeds. mavweb serves the passkey page while
|
||||
locked; other pages return 503. An env-key fallback is preserved for dev/CI
|
||||
and recovery. This is disk-theft protection, not RAM-capture protection: root
|
||||
on the host can still dump the key after unlock, but a stolen disk or a
|
||||
`docker inspect` no longer yields it.
|
||||
|
||||
### Core/module key isolation
|
||||
|
||||
The convenience (reboot attendance collapses to *core cold-start only*) is
|
||||
contingent on one thing: **the key lives in core's address space and nowhere
|
||||
else.**
|
||||
|
||||
- **core = the only key-holder.** The daemon holding the unlocked DB + the
|
||||
trigger loop. Unlocked once (remote-attended), runs for weeks. It reboots
|
||||
almost never *because it was deliberately given nothing that churns* — no
|
||||
tool code, no model weights.
|
||||
- **modules = restart-free, key-free, fail-independent.** STT/TTS, phrasing,
|
||||
tool executors, delivery. Update/crash/swap one → none touch the unlock.
|
||||
"Update the tool module, no attendance" is correct *by construction*: the
|
||||
module never had the key.
|
||||
|
||||
Enforced by an **IPC boundary, not a shared address space** (unix domain
|
||||
socket, local-only). **Core mediates and never hands back a DB handle** —
|
||||
modules send requests *to* core ("write this fact" / "read presence").
|
||||
**Module compromise ≤ module authority:** the worst a popped TTS does is send
|
||||
garbage audio. The discipline: **nothing enters core's process unless it must
|
||||
read state under the lock.** The loop and predicates qualify; phrasing,
|
||||
routing, transcription, tool execution and delivery all read *derived* data.
|
||||
Erode this and you buy back the attendance you just eliminated. systemd
|
||||
topology: core = one unit, each module its own unit, `After=core.socket`,
|
||||
socket-activated, `Restart=on-failure`.
|
||||
|
||||
### The through-line
|
||||
|
||||
Network → box → process: the same question at three radii.
|
||||
|
||||
- **network (wg):** who reaches the box
|
||||
- **box (LUKS+TPM+PIN, at-rest encryption):** what a dead/stolen box gives up
|
||||
- **process (core/module socket):** what a compromised module reaches
|
||||
|
||||
Every cut is the same instinct — *a boundary you can move from inside isn't
|
||||
one*, *compromised X can't forge Y*, *attribution is not auth*. Auth didn't
|
||||
add a new principle; it applied the existing one at smaller and smaller scope.
|
||||
|
||||
---
|
||||
|
||||
## Calendar
|
||||
|
||||
Integration with **Radicale** (self-hosted CalDAV), not Nextcloud. Scope is
|
||||
read + write: read to detect busy/available (gating nudges) and answer "what's
|
||||
on my calendar"; write to schedule and move events. The feed is a separate
|
||||
binary, `cmd/mavcaldav`, which polls Radicale and writes `calendar_busy` plus
|
||||
per-event facts through CoreAPI, on value change only (same append-only
|
||||
discipline as `mavpoll`).
|
||||
|
||||
## Deployment
|
||||
|
||||
| Phase | Mechanism | Notes |
|
||||
|-------|-----------|-------|
|
||||
| Then | scripts (`start-maven.sh`, `kill-maven.sh`) | manual start/stop in tmux |
|
||||
| Now | Docker (one image, several daemon containers) | `docker-compose.yml` |
|
||||
| Alt | systemd user units | one per binary, socket-activated modules |
|
||||
|
||||
Invariant: **core is rarely redeployed, components are.** The IPC boundary
|
||||
(worker STT/TTS sockets, `internal/ipc` CoreAPI socket) means `mavsttd`,
|
||||
`mavttsd`, `mavpoll`, `mavweb` restart independently without touching the
|
||||
daemon.
|
||||
|
||||
## Client protocol
|
||||
|
||||
The voice wire protocol (length-prefixed JSON frames over TCP) is designed for
|
||||
**multiple client implementations**. The reference PWA at `cmd/mavweb` is one
|
||||
client; any app (phone, desktop CLI, smartwatch) can implement the same frame
|
||||
protocol. The published spec is `docs/protocol.md` — **generated from
|
||||
`internal/voice/wire.go`**, not composed freehand, so it can't drift from
|
||||
code. It covers transport (4-byte big-endian length prefix), methods
|
||||
(`PushToTalk`, `Pong`), push kinds (`AudioNudge`), surface identity
|
||||
(header field, cap enforced server-side), error codes, and how passkey
|
||||
assertions are carried for step-up.
|
||||
|
||||
The act allowlist is config-driven (`deploy/mavend.json` seeds a homelab set:
|
||||
read-only status/ps/uptime/df/free/logs, gated restart/stop/reboot).
|
||||
Broadening to home automation, media or comms is JSON, not code.
|
||||
|
||||
---
|
||||
|
||||
## Execution ledger
|
||||
|
||||
Condensed from `ROADMAP.md` (2026-07-06). The live queue is
|
||||
`20-07-2026-BACKLOG.md`; current state is `PROGRESS.md`.
|
||||
|
||||
| # | Item | Prio | Status |
|
||||
|---|------|------|--------|
|
||||
| 1.1 | Kuma API key for `service_down` polling | P1 | done `eda434f` |
|
||||
| 1.2 | Voice bind verify + stale comment fix | P1 | done `eda434f` |
|
||||
| 1.3 | desk_active presence script on desk PC | P1 | **not done** — operator action on `linux` (systemd user timer + hypridle listener); 0 facts ever written, presence runs on `page_heartbeat` alone |
|
||||
| 2.1 | Cold-start unlock (passkey → L3 key seam) | P2 | code done `b0932a1`+`15fe7bb`, **tests missing** — wrap/unwrap round-trip, wrong-cred unwrap fails, locked-mode IPC rejects non-unlock methods |
|
||||
| 3.1 | Always-on listening | P3 | MVP `e57647c` (energy-VAD only); remaining: wake-word model in `vad.go` |
|
||||
| 3.2 | Conversation depth (multi-turn) | P3 | done `05236ad` — anaphora resolver + cross-intent `followUpMerge` + `Session.History` |
|
||||
| 3.3 | Latency / streaming (streaming STT/TTS, barge-in) | P3 | not started; recommended path is WebSocket voice, keeping TCP for non-browser clients |
|
||||
| 4.1 | Routing quality (dev embedder) | P4 | done `b7eb53a` — `make download-embedder`, configurable `voice.query_min_score` |
|
||||
| 4.2 | Act surface broadening | P4 | not a code item (operator config) |
|
||||
| 4.3 | LTM ANN index | P4 | deferred — `memory.Store` is the swap point; brute-force cosine is sub-ms at single-user scale. Revisit past ~10k rows |
|
||||
| 4.4 | Persona prompt | P4 | done `b7eb53a` — `voice.persona` |
|
||||
| 4.5 | Custom TTS voice | P4 | not started; operator work (record ~50–100 clips, train a piper voice), code already supports the swap |
|
||||
| 5.1 | Multi-user | P5 | deferred by design — do not start without an explicit operator decision |
|
||||
|
||||
Also landed from the SPEC's original open items: protocol doc, away-channel
|
||||
fallthrough, CalDAV poller, quiet-hours schedule, tools enable/disable page
|
||||
(`/tools`, gated at step-up), and note RAG (the `query` intent phrases from
|
||||
gated top-k notes instead of dumping a verbatim note).
|
||||
|
||||
Conventions retained from the roadmap: **done when** = a checkable finish
|
||||
criterion; name the exact files, not "somewhere in internal/"; every code item
|
||||
ends with `make test` green (gofmt + vet + `-race`), no exceptions.
|
||||
|
||||
---
|
||||
|
||||
## Open questions
|
||||
|
||||
Router+invocation, two-memory routing, presence, and auth were once listed
|
||||
here and are resolved by the sections above.
|
||||
|
||||
**Impactful — gate behaviour or the MVP surface:**
|
||||
|
||||
- **stage-3 confidence threshold** — the gate-or-clarify number. Defines how
|
||||
often Maven asks vs. guesses on free-form input.
|
||||
- **quiet-hours definition** — fixed clock vs derived from sleep facts.
|
||||
- **confirmation tier model** — only the forced pin is settled (registration
|
||||
is out-of-band, human-only). The rest is a sketch: confirmation scales with
|
||||
specification + reversibility (just-do / binary-confirm /
|
||||
clarify-then-confirm / out-of-band). Open sub-item: destructive-act confirm
|
||||
— policy-level vs per-function flag.
|
||||
- **ask-password transport** — `systemd-ask-password` over ssh vs a
|
||||
passkey-authed page push for cold-start unlock. Both work; unpicked.
|
||||
- **wake-word hardware** — USB mic on a client box (fast path) vs an ESP32-S3
|
||||
room device (the "real" version). The code is the same either way; the
|
||||
hardware changes the deploy — and it decides where an ambient reply is
|
||||
spoken (speaker on the capture device vs the PWA if a session is live).
|
||||
|
||||
**Plumbing / deferred:**
|
||||
|
||||
- **passkey enrollment bootstrap** — the first credential on a fresh device,
|
||||
before a passkey exists to authenticate with (trust-on-first-use gap).
|
||||
- **mTLS in or out** — provisioning cost on mobile vs paranoia payoff.
|
||||
Leaning optional.
|
||||
- **PIN vs yubikey for LUKS** — PIN-in-head pinned for now.
|
||||
- **session lifetime / re-auth cadence** — unset.
|
||||
- **compound captures** — "slept 6h, fan noise wrecked it" = one fact + one
|
||||
note in one utterance. Needs a second pass or it loses half.
|
||||
- **query read-path** — semantic RAG vs a structured read, depending on the
|
||||
ask.
|
||||
- **presence — away tap override** — an explicit `away` tap as a hard
|
||||
override. Clean extension, deferred; scoring stands without it.
|
||||
- **presence — weights/τ hand-tuning** — first-guess numbers; expect tuning
|
||||
against real signal traces.
|
||||
- **presence — wg home-vs-cellular** — let wg carry more weight when clearly
|
||||
home. Needs the signal to exist first.
|
||||
- **listening modes 2–3** — meeting-record + ambient-derive; ambient-derive
|
||||
needs the confidence model first.
|
||||
- **LLM dialogue manager** — the router/phraser deciding "ask for X" vs "act."
|
||||
Blocked on the resident-model question (task #318).
|
||||
|
||||
---
|
||||
|
||||
## Superseded
|
||||
|
||||
Kept for provenance. **None of this is the current or intended design.**
|
||||
|
||||
- **Classifier-owns-the-route.** `maven.md` argued the route decision must
|
||||
stay deterministic — "classifier owns the route, the SLM stays in its
|
||||
phrasing lane" — with an embedding + nearest-centroid stage 1 over ~10
|
||||
examples per intent, and misroutes appended as new centroid examples.
|
||||
*Replaced by* LLM-as-router (`docs/rearchitecture.md`): one resident model emits
|
||||
GBNF-constrained JSON and also phrases replies; the embedder is demoted to
|
||||
a RAG hint. *Landed 2026-07-31:* the LLM router is on by default and set
|
||||
`true` in `deploy/mavend.json`. The classifier cascade stays as the failure
|
||||
floor — it runs when there is no llama-server to talk to and on any per-turn
|
||||
LLM error — but routing by seed similarity is the known cause of weak RU
|
||||
query handling and is not a design to extend.
|
||||
- **Named STT/TTS model picks.** `maven.md` picked faster-whisper small/int8
|
||||
as primary STT with vosk RU for a low-latency command grammar, and silero
|
||||
(license unverified) as TTS with piper RU as the floor, all on
|
||||
onnxruntime/CPU. *Replaced by* whisper.cpp (CGo, Vulkan) in `cmd/mavsttd`
|
||||
and piper as the production TTS in `cmd/mavttsd`. Several Go doc comments
|
||||
still cite the old picks by way of `maven.md § stt/tts`.
|
||||
- **Small-model phrasing claim.** `maven.md` specified "lfm2.5 / sub-1b for
|
||||
phrasing — prompted, not trained," and `SPEC.md` named a specific resident
|
||||
size. Both are superseded by the RU-CPT + joint persona/router SFT plan.
|
||||
*Resolved 2026-07-30 (#318), revised 2026-07-31:* the resident checkpoint is
|
||||
stock **Qwen3-1.7B** (`UD-Q4_K_XL`, `n_ctx` 4096), which replaced
|
||||
Qwen3.5-0.8B after measuring better on both fixtures
|
||||
(`docs/evals/2026-07-31-model-bakeoff.md`). The CPT'd **Qwen3-1.7B** remains the target
|
||||
(#122); what stock gets wrong is the persona, not the Russian. Note the resident
|
||||
model is no longer described as untrained — the target is trained
|
||||
end-to-end, which is the substantive change from the old claim.
|
||||
- **sqlcipher at rest.** `maven.md` specified sqlcipher with the key read at
|
||||
daemon start. *Replaced by* AES-256-GCM with a tmpfs working copy
|
||||
(`internal/store/crypt.go`). The key-provenance argument above survives
|
||||
unchanged; only the cipher layer differs.
|
||||
- **Kotlin/Spring implementation sketches.** `maven.md` gave the presence
|
||||
scorer as Kotlin (`data class Signal`, `presenceScore`, `resolve`) and cited
|
||||
Spring Security's passkey support as in-stack. *Replaced by* Go throughout;
|
||||
the presence math is unchanged and lives in `internal/store/presence.go`.
|
||||
- **obsidian → chroma for long-term memory.** `maven.md` specified Obsidian
|
||||
as canonical markdown with a derived Chroma embedding index, and listed the
|
||||
chunking mechanics as unbuilt. *Replaced by* sqlite-backed vector storage
|
||||
(`internal/store/memory.go` behind `internal/memory.Store`); no Chroma, no
|
||||
Obsidian. Wherever this document says "semantic store," that is what it
|
||||
means.
|
||||
- **Script-based deployment.** `start-maven.sh` / `kill-maven.sh` in tmux was
|
||||
the "now" row of the SPEC deployment table. *Replaced by* the Docker
|
||||
deployment (one image, several daemon containers).
|
||||
- **`FloorEnrollment` as the auth floor.** `SPEC.md`'s week-1 floor granted
|
||||
full L3 to any same-uid caller with only the wg tunnel underneath.
|
||||
*Replaced by* real WebAuthn enroll/assert plus the wrapped-key cold-start
|
||||
path; the passkey step-up item is landed.
|
||||
@@ -0,0 +1,292 @@
|
||||
# Deterministic logic around a small model
|
||||
|
||||
*Last verified: 2026-08-02 @ 7079a24. Living doc: correct it in place, do not append.*
|
||||
|
||||
Written 2026-08-02. Branch `fix/integrated`.
|
||||
|
||||
## The question
|
||||
|
||||
Where does deterministic code attach, so that it helps the resident 1.7B now and
|
||||
does not fight a larger model later.
|
||||
|
||||
## The mistake to avoid
|
||||
|
||||
Everything deterministic we have added so far sits in front of the model and
|
||||
preempts it. Stage 0 matches, the model never sees the turn. That shape helps a
|
||||
weak model and blocks a strong one, silently.
|
||||
|
||||
The fix is not to remove it. The fix is to know which rules are safe in that
|
||||
position and to have a way to measure the rest.
|
||||
|
||||
## Four attachment points
|
||||
|
||||
**Bypass, before the model.** The only shape that saves the 2.7s p50. Safe when
|
||||
the rule is a decision procedure over a closed set, not a guess over an open one.
|
||||
Exact match and clock queries qualify.
|
||||
|
||||
**Evidence, beside the model.** Extractors emit candidate slots as a prior, not a
|
||||
verdict. The prompt carries the prior and the validator reuses it. A small model
|
||||
leans on it, a large one overrides it correctly.
|
||||
|
||||
**Grammar, around the model.** GBNF built from live state rather than hardcoded.
|
||||
Costs nothing at runtime and prevents the error instead of catching it.
|
||||
|
||||
**Repair, after the model.** Validation failure re-asks with the specific error
|
||||
rather than overriding. Self-retiring, because a better model trips it less.
|
||||
|
||||
## The constraint that ranks them
|
||||
|
||||
Latency must stay minimal. That demotes repair and promotes grammar.
|
||||
|
||||
- Grammar first. Zero runtime cost, immediate gain.
|
||||
- Bypass keeps its place. It is the only thing that avoids a model call at all.
|
||||
- Repair only where failure is rare, capped at one retry.
|
||||
- Evidence is correct but costs a model call where a bypass costs none.
|
||||
- Ecosystem calls belong in the snapshot path, in parallel, on strict deadlines.
|
||||
Never serial before routing.
|
||||
|
||||
## The line that never moves
|
||||
|
||||
Separate policy from capability compensation. They look alike and age oppositely.
|
||||
|
||||
Capability compensation exists because the model is weak. It should be measurable
|
||||
and retirable.
|
||||
|
||||
Policy exists because we decided. The personal boundary, the feminine persona, the
|
||||
never-search-his-notes rule, the Hexis allowlist and confirmation binding. None of
|
||||
those yield to a smarter model. A larger model is more dangerous there, not less.
|
||||
|
||||
## What we keep
|
||||
|
||||
Every stage-0 rule stays exactly as it is. Retiring them was the wrong call and it
|
||||
would throw away a day of measured gains.
|
||||
|
||||
- exact-match fast path
|
||||
- clock rules
|
||||
- `SystemTimeDateGrammars`
|
||||
- `AgendaQueryGrammars`
|
||||
- `thinSingleToken`, including the social lexicon and the verb-ending test
|
||||
|
||||
Three additions, none of which change behaviour:
|
||||
|
||||
1. Each rule gets an id and a fixture subset.
|
||||
2. Each rule writes one trace line when it fires.
|
||||
3. Each rule carries a comment saying whether its set is closed or open.
|
||||
|
||||
That preserves today's accuracy and buys the option to revisit later with numbers.
|
||||
|
||||
## What we build
|
||||
|
||||
**Dynamic grammars from live state.** The grammars today are static: `routeGrammar`
|
||||
fixes the 7 intents, `responseGrammar` fixes the mood enum, kiwix `queryGrammar`
|
||||
fixes word shape. Everything else is a free string.
|
||||
|
||||
Candidates in order of payoff:
|
||||
|
||||
- **Hexis capability ids.** After discovery the exact list is known. As an enum,
|
||||
the model cannot name a capability that does not exist.
|
||||
- **Act fn allowlist.** Hits the two remaining false clarifies directly. They are
|
||||
the act-with-no-allowlisted-fn arm of `gateLLMDecision`, firing on invented verbs.
|
||||
- **Calendar names.** Enumerate the real ones for agenda and query slots.
|
||||
- **Known fact keys.** A read-back matches a stored key instead of inventing a
|
||||
synonym. This is the general form of the read side we scrapped on 2026-08-01.
|
||||
- **Nexus display names as act targets.** Only while the list stays small.
|
||||
|
||||
Two rules or it backfires:
|
||||
|
||||
- **Always include an escape value.** A closed enum with no `other` forces a wrong
|
||||
pick instead of a decline. The escape is what feeds the clarify gate.
|
||||
- **Cache the grammar string, keyed on the state that built it.** Rebuilding per
|
||||
turn is fine. Recompiling a large grammar per turn is not.
|
||||
|
||||
## Retrieval over regex
|
||||
|
||||
Resolve against stores that already exist rather than adding patterns.
|
||||
|
||||
Identity is the worked example. Nexus is authoritative, `actionFact` already sets
|
||||
`Subject`, and `cmd/mavend/factenrichment.go` resolves it in the background. The
|
||||
scrapped work invented a parallel key namespace with nothing reconciling the two.
|
||||
|
||||
A table that grows with real data beats patterns that grow with our patience.
|
||||
|
||||
Note a real gap: the personal boundary in `cmd/mavend/actions_query.go` guards
|
||||
Maven's own store only. It does not know Nexus or Praxis exist.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Both are offline and need no deploy. Neither existed on 2026-08-01, and that is
|
||||
why the day cost what it did.
|
||||
|
||||
**Failure taxonomy over the 77-case fixture.** Classify every miss as model
|
||||
ignorance, contract loss, prompt ambiguity, or our own bug. Only model ignorance
|
||||
deserves deterministic compensation. The other three get fixed once, for every
|
||||
model size. Inference, not verified: much of what we patched was the last two.
|
||||
|
||||
**Per-assist ablation runner.** One command toggles each assist off and reports the
|
||||
accuracy delta on its fixture subset. Then retiring an assist is a config flip and
|
||||
a number, not an argument.
|
||||
|
||||
## Held
|
||||
|
||||
Not now, and nothing gets built for it.
|
||||
|
||||
- A 12B or 35B model. It may be cloud or the 16GB workstation, and cloud crosses
|
||||
the current no-third-party line.
|
||||
- The escalation tier in `gateLLMDecision`.
|
||||
- Converting the stage-0 heuristics to evidence.
|
||||
|
||||
One thing carries forward for free: deterministic assists emit confidence, never a
|
||||
verdict. A verdict cannot escalate.
|
||||
|
||||
## Ruling on the idea list
|
||||
|
||||
Nineteen ideas, judged on whether they earn a place in Maven. Checked against the
|
||||
tree on 2026-08-02, not from memory.
|
||||
|
||||
### Build. Absent, and worth it.
|
||||
|
||||
**SQLite FTS over embeddings.** No `fts5` anywhere in the tree. Lexical search is
|
||||
faster than the ONNX embedder, deterministic, and strongest exactly where the
|
||||
embedder is weakest, which is exact Russian names. Semantic search becomes the
|
||||
fallback rather than the gate. This is the highest-value absent item.
|
||||
|
||||
**Cached TTS phrases.** No cache in `internal/tts`. Confirmations, clarifies and
|
||||
refusals repeat constantly and their text is already fixed. Pre-rendering them is
|
||||
cheap and pays straight into the minimal-latency constraint.
|
||||
|
||||
**Synthetic router dataset.** Generate Russian tool-calling examples from the same
|
||||
schemas that will build the dynamic grammars. One source of truth for both, so the
|
||||
model is trained on exactly the shapes it will be constrained to at inference.
|
||||
|
||||
**Command STT separate from dictation STT.** One path today. Commands want latency
|
||||
and a small vocabulary, meeting capture wants accuracy and can take its time.
|
||||
`internal/capture` already spools to disk, so the split follows the existing seam.
|
||||
Medium priority, behind the four above.
|
||||
|
||||
### Polish. Present, incomplete.
|
||||
|
||||
**Strict JSON everywhere.** Done 02-08-2026. The replier sends
|
||||
`phraser.ResponseGrammar`. It is exported once so its two copies cannot drift.
|
||||
The meeting summariser is wrapped and unwrapped in the daemon's Completer, so
|
||||
`internal/capture` stays text-in/text-out. Every model call now carries a grammar.
|
||||
|
||||
**Evidence-first prompting.** Done 02-08-2026, in the evidence branch of
|
||||
`PhraseQuery`. Sources arrive numbered, one per line. The system prompt no longer
|
||||
calls them all "заметки" and no longer lets the model add anything of its own.
|
||||
Blank sources now take the knowledge branch instead of asking for an answer from
|
||||
an empty list. Not yet measured against a live model. Run `make eval-phrasing`,
|
||||
and watch the case `query-notes-do-not-answer`.
|
||||
|
||||
**Progressive inference.** What exists is a fallback cascade, not escalation. On
|
||||
error it drops to something weaker. It never escalates on ambiguity. The upgrade is
|
||||
held with the bigger-model question, and `gateLLMDecision` is the hook.
|
||||
|
||||
**Entity dictionaries.** Nexus is the canonical store, `behavior_ru.go` has
|
||||
`KeyAliases`, `ecosystem_acts.go` has verb aliases. Morphology is the gap, and the
|
||||
code says so in three places. Russian needs it and there is no stemmer in the repo.
|
||||
|
||||
**Assistant state machine.** `confirm.go` models pending confirmation and
|
||||
`dialogue.Session` models the turn. There is no unified task state. Reuse the
|
||||
Praxis vocabulary rather than inventing one, because surfaced, acknowledged and
|
||||
resolved already mean something precise here.
|
||||
|
||||
**Session memory compiler.** The digestion tick, `internal/memory` and
|
||||
`followUpMerge` do parts of this. Not a coherent compile step.
|
||||
|
||||
**Background memory maintenance.** Mostly present, and the absent part is small.
|
||||
What exists: `internal/memeval` reads recent memory on a loop and writes
|
||||
observations, deduped against its own prior output, unable to speak or act. Facts
|
||||
supersede at write time through `voidsID`, `CorrectValue` and `VoidLatestFact`,
|
||||
and `RecentActiveFactsByKind` reads only live rows. Digests, ecosystem traces and
|
||||
media all prune.
|
||||
|
||||
Three gaps remain, all in the fact store:
|
||||
|
||||
- Superseding is turn-driven. Nothing reconciles two live facts that contradict
|
||||
unless a turn corrects one of them.
|
||||
- Duplicates written by different sources or phrasings stay as separate live rows.
|
||||
There is no key-level merge pass.
|
||||
- Nothing expires. The log is append-only and grows without bound, and no fact
|
||||
ever ages out on its own.
|
||||
|
||||
Worth doing, and smaller than it looked. Not urgent.
|
||||
|
||||
**Qwen CPT then narrow SFT.** In flight as #122. One disagreement with the idea as
|
||||
written: do not drop persona from the SFT. Persona is the stated reason the CPT
|
||||
exists, because stock writes `рад` where Maven needs `рада`. Keep the joint
|
||||
router-plus-persona tune.
|
||||
|
||||
**Offline job queue.** The tick loop, `factEnrichmentWorker` with backoff, and the
|
||||
media prune already defer work. A general queue is tidier, not more capable. Low
|
||||
priority.
|
||||
|
||||
**Response templates.** Present in `clarify.go`, in the `replySystem` arms and in
|
||||
`StubPhraser`. Worth extending to high-frequency confirmations, where latency and
|
||||
persona correctness both matter. Do not extend further. Templating the
|
||||
conversational reply removes the reason she is worth having.
|
||||
|
||||
### Reject.
|
||||
|
||||
**Grammar-first routing as a replacement for the LLM router.** Already measured.
|
||||
The classifier scores 36.8% full accuracy against 72.7% through the cascade.
|
||||
Replacing the model with rules halves the accuracy. Grammar-first as an ordering is
|
||||
what stage 0 already is, and that stays.
|
||||
|
||||
**Hierarchical intent classification.** Seven intents is already the coarse layer.
|
||||
The specialisation stage exists as per-intent slot filling in `Extractor.Extract`.
|
||||
Adding a tier buys structure, not accuracy.
|
||||
|
||||
**Tool-specific micro-models.** Contradicts the one-resident-model constraint,
|
||||
needs per-domain training data nobody has, and multiplies model loads on a single
|
||||
Vega iGPU. Dynamic grammars give the same domain narrowing at zero runtime cost.
|
||||
|
||||
**Local knowledge graph.** Nexus owns entities and relationships. Building a second
|
||||
graph in Maven breaks the ecosystem line and creates two answers to one question.
|
||||
If graph traversal is wanted, it is a Nexus feature request.
|
||||
|
||||
**Preemptible training.** Training runs in a separate workspace, not on the serving
|
||||
box. This only becomes real if CPT moves onto homesrv, and that is not the plan.
|
||||
|
||||
## Known open, carried over from the deleted handoff
|
||||
|
||||
Found live on 2026-08-01, not fixed. Everything else in that file was stale.
|
||||
|
||||
- **Kiwix ranks badly on a correct query.** The stop-word pass eats the "and". So
|
||||
"кто написал войну и мир" reaches Kiwix as "war peace author". The top hit is
|
||||
"List of peace activists" and she summarises that as the answer. The tag
|
||||
`scrapped/fact-and-kiwix-phrases` fixes the query text. The ranking is the ZIM
|
||||
search and is untouched either way.
|
||||
- **Chat drags prior turns into an answer.** One live reply mixed the greeting, the
|
||||
height statement and a world question. It named Левитан as the author of Война и
|
||||
мир.
|
||||
- **`safeKey` drops Cyrillic**, so Russian calendar events on one day collide.
|
||||
Vikunja #443 with three fix options. It is a migration, not a patch.
|
||||
|
||||
## Open after the 02-08-2026 deploy
|
||||
|
||||
SearXNG runs on homesrv at `http://searxng:9563`, on `maven_default`, and the
|
||||
rebuilt `mavend` wires it. "кто написал войну и мир?" now routes to query, takes
|
||||
4 results off the search, and answers Толстой with the lookup opener. Two things
|
||||
that turn left unsettled.
|
||||
|
||||
- **The turn was slow, and nobody knows yet whether that is real.** Route 7s,
|
||||
search 1s, phrasing 15s. The p50 in `docs/evals/2026-07-31-routing.md` is 825ms. It
|
||||
was the first turn after a cold start with the model still warming, so it
|
||||
proves nothing either way. Re-run the same question warm before treating it as
|
||||
a regression. Do not plan latency work off this number.
|
||||
- **The personal boundary has never run live.** `queryPersonal` in
|
||||
`cmd/mavend/actions_query.go` stops a question about him from reaching
|
||||
SearXNG. The question above is not one, so only tests cover it. Ask something
|
||||
about him on the deployed box and confirm from the log that no `voice: search`
|
||||
line appears for it.
|
||||
|
||||
## Sequence
|
||||
|
||||
1. Failure taxonomy over the fixture.
|
||||
2. Ablation runner, plus ids and trace lines for the existing rules.
|
||||
3. Dynamic grammar for the act fn allowlist.
|
||||
4. Dynamic grammar for Hexis capability ids.
|
||||
5. Dynamic grammar for calendar names and known fact keys.
|
||||
6. Extend the personal boundary to the ecosystem stores.
|
||||
|
||||
Steps 1 and 2 come before anything is written in the router.
|
||||
@@ -0,0 +1,887 @@
|
||||
# Maven Ecosystem Architecture
|
||||
|
||||
*Last verified: 2026-08-02 @ 7079a24. Living doc: correct it in place, do not append.*
|
||||
|
||||
## 1. Purpose
|
||||
|
||||
This document defines Maven's role in the local ecosystem formed by:
|
||||
|
||||
- **Maven** — conversational control center and personal assistant
|
||||
- **Nexus** — canonical identity and relationship service
|
||||
- **Praxis** — operational state and attention service
|
||||
- **Hexis** — capability registry and controlled execution service
|
||||
|
||||
The design keeps Maven useful without turning it into the owner of every concern.
|
||||
|
||||
The core rule is:
|
||||
|
||||
```text
|
||||
Nexus identifies.
|
||||
Praxis observes.
|
||||
Hexis acts.
|
||||
Maven understands and coordinates.
|
||||
```
|
||||
|
||||
Maven is the user-facing control center, but not the source of truth for identities, operational state, or execution.
|
||||
|
||||
---
|
||||
|
||||
## 2. Maven's role
|
||||
|
||||
Maven provides the human interface over the other systems.
|
||||
|
||||
It is responsible for:
|
||||
|
||||
- interpreting Russian and English utterances
|
||||
- deciding whether the user wants information, memory access, or an action
|
||||
- assembling relevant world-state context
|
||||
- querying Praxis for operational attention
|
||||
- resolving references through Nexus
|
||||
- discovering and invoking capabilities through Hexis
|
||||
- managing conversational clarification and confirmation
|
||||
- phrasing structured results naturally
|
||||
- exposing the same state through voice, Telegram, web, and other supported channels
|
||||
- retaining personal memory, reminders, and conversational history
|
||||
|
||||
Maven is not responsible for:
|
||||
|
||||
- owning canonical identities
|
||||
- ingesting every external notification directly
|
||||
- monitoring all infrastructure and agent sessions itself
|
||||
- storing execution definitions
|
||||
- executing arbitrary commands
|
||||
- deciding that an attention item should trigger an action automatically
|
||||
- sharing databases with Nexus, Praxis, or Hexis
|
||||
- treating LLM output as authorization
|
||||
|
||||
---
|
||||
|
||||
## 3. Existing Maven architecture
|
||||
|
||||
Maven remains a set of Go daemons connected through Unix sockets.
|
||||
|
||||
| Binary | Role |
|
||||
|---|---|
|
||||
| `mavend` | Core router, phraser, memory, reminders, digestion, integrations, and IPC owner |
|
||||
| `mavweb` | HTTP UI and PWA |
|
||||
| `mavsttd` | Speech-to-text through whisper.cpp |
|
||||
| `mavttsd` | Text-to-speech through Piper |
|
||||
| `mavwaked` | Wake-word and VAD gate |
|
||||
| `mavenclient` | Voice interaction loop |
|
||||
| `mavpoll` | Telegram reach |
|
||||
| `mavcaldav` | CalDAV synchronization |
|
||||
|
||||
The resident model (Qwen3.5-0.8B now, CPT'd Qwen3-1.7B as the target — #122)
|
||||
remains bounded to:
|
||||
|
||||
- structured routing
|
||||
- concise natural-language phrasing
|
||||
- bounded digestion synthesis
|
||||
|
||||
It does not directly query databases or execute external operations.
|
||||
|
||||
---
|
||||
|
||||
## 4. Ecosystem topology
|
||||
|
||||
```text
|
||||
user
|
||||
voice / web / Telegram / text
|
||||
|
|
||||
v
|
||||
+------------------+
|
||||
| Maven |
|
||||
| conversation and |
|
||||
| coordination |
|
||||
+--------+---------+
|
||||
|
|
||||
+----------------+----------------+
|
||||
| | |
|
||||
v v v
|
||||
+-------------+ +-------------+ +-------------+
|
||||
| Nexus | | Praxis | | Hexis |
|
||||
| identity | | operational | | capabilities|
|
||||
| relations | | attention | | execution |
|
||||
+-------------+ +-------------+ +-------------+
|
||||
^ ^ |
|
||||
| | |
|
||||
+----------------+----------------+
|
||||
shared canonical entity IDs
|
||||
```
|
||||
|
||||
Maven talks to each service through a versioned client contract.
|
||||
|
||||
No component reads another component's SQLite database.
|
||||
|
||||
---
|
||||
|
||||
## 5. Core interaction model
|
||||
|
||||
A Maven turn follows this shape:
|
||||
|
||||
```text
|
||||
utterance
|
||||
-> stage-0 deterministic fast path
|
||||
-> world-state snapshot
|
||||
-> router
|
||||
-> reference resolution
|
||||
-> information query or capability discovery
|
||||
-> confirmation when required
|
||||
-> execution or response
|
||||
-> phrasing
|
||||
-> delivery
|
||||
-> trace
|
||||
```
|
||||
|
||||
Expanded:
|
||||
|
||||
```text
|
||||
1. receive utterance
|
||||
2. normalize text and collect channel metadata
|
||||
3. build bounded world-state context
|
||||
4. route into a structured intent
|
||||
5. resolve referenced entities through Nexus
|
||||
6. query Praxis or Hexis as required
|
||||
7. stop and clarify on ambiguity
|
||||
8. request explicit confirmation for protected actions
|
||||
9. execute through Hexis when authorized
|
||||
10. phrase the structured result
|
||||
11. deliver through the originating or selected channel
|
||||
12. record a complete trace
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Maven and Nexus
|
||||
|
||||
### 6.1 Purpose
|
||||
|
||||
Maven uses Nexus whenever an utterance refers to a real entity:
|
||||
|
||||
- project
|
||||
- repository
|
||||
- device
|
||||
- service
|
||||
- application
|
||||
- person
|
||||
- pet
|
||||
- location
|
||||
- agent session
|
||||
- household object
|
||||
- external task project
|
||||
|
||||
Examples:
|
||||
|
||||
```text
|
||||
"correx"
|
||||
"the correx repository"
|
||||
"коррекс"
|
||||
```
|
||||
|
||||
All may resolve to the same canonical entity.
|
||||
|
||||
### 6.2 Resolution flow
|
||||
|
||||
```text
|
||||
utterance
|
||||
-> Maven extracts reference text and expected entity types
|
||||
-> Nexus resolves candidates
|
||||
-> Maven receives resolved / ambiguous / not_found
|
||||
```
|
||||
|
||||
Resolved:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "resolved",
|
||||
"entity_id": "ent_correx",
|
||||
"entity_type": "project.software",
|
||||
"display_name": "Correx"
|
||||
}
|
||||
```
|
||||
|
||||
Ambiguous:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "ambiguous",
|
||||
"candidates": [
|
||||
{"entity_id": "ent_muzick_indexer", "label": "Muzick indexer"},
|
||||
{"entity_id": "ent_manga_indexer", "label": "Manga indexer"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Maven must ask for clarification instead of choosing silently.
|
||||
|
||||
### 6.3 Rules
|
||||
|
||||
- canonical entity IDs are used in all downstream calls
|
||||
- free-text names must not reach mutating Hexis operations
|
||||
- low-confidence mappings may be shown as suggestions but not used for mutation
|
||||
- Maven may submit user-confirmed alias or mapping feedback to Nexus
|
||||
- Maven must not create entities automatically unless the operation is explicit
|
||||
- entity display names are presentation data, not authorization data
|
||||
|
||||
---
|
||||
|
||||
## 7. Maven and Praxis
|
||||
|
||||
### 7.1 Purpose
|
||||
|
||||
Praxis gives Maven a normalized view of external operational state.
|
||||
|
||||
Examples:
|
||||
|
||||
- coding agent waiting for input
|
||||
- coding agent failed
|
||||
- service degraded
|
||||
- notification still unresolved
|
||||
- task or project item requiring attention
|
||||
- source integration stale
|
||||
- required routine item not completed
|
||||
- execution succeeded but recovery not yet observed
|
||||
|
||||
Maven asks Praxis questions such as:
|
||||
|
||||
```text
|
||||
what needs attention?
|
||||
what changed since this morning?
|
||||
what is unresolved for Correx?
|
||||
which agent is waiting?
|
||||
what happened after the restart?
|
||||
```
|
||||
|
||||
### 7.2 Query contract
|
||||
|
||||
Recommended Maven-facing operations:
|
||||
|
||||
```text
|
||||
praxis.list_attention
|
||||
praxis.list_changes
|
||||
praxis.list_items
|
||||
praxis.get_item
|
||||
praxis.search
|
||||
praxis.acknowledge
|
||||
praxis.resolve
|
||||
praxis.ignore
|
||||
praxis.pin
|
||||
praxis.list_item_capabilities
|
||||
praxis.list_item_executions
|
||||
```
|
||||
|
||||
These may be exposed through a native API client or a compact internal tool abstraction.
|
||||
|
||||
### 7.3 Item lifecycle
|
||||
|
||||
Maven must preserve Praxis semantics:
|
||||
|
||||
```text
|
||||
surfaced != acknowledged
|
||||
acknowledged != resolved
|
||||
execution_succeeded != recovered
|
||||
stale != resolved
|
||||
ignored != deleted
|
||||
```
|
||||
|
||||
Examples:
|
||||
|
||||
- reading an item aloud marks it surfaced, not acknowledged
|
||||
- the user saying "got it" may acknowledge it
|
||||
- the user saying "done" may resolve it
|
||||
- a successful restart command does not resolve a service failure
|
||||
- a later healthy observation may resolve it
|
||||
|
||||
### 7.4 Digestion integration
|
||||
|
||||
Maven's digestion worker may query Praxis for:
|
||||
|
||||
- newly unresolved attention
|
||||
- repeated operational patterns
|
||||
- items surfaced but never acknowledged
|
||||
- failed sources affecting visibility
|
||||
- recently recovered items
|
||||
|
||||
Digestion may propose or summarize.
|
||||
|
||||
It must not invoke Hexis automatically.
|
||||
|
||||
---
|
||||
|
||||
## 8. Maven and Hexis
|
||||
|
||||
### 8.1 Purpose
|
||||
|
||||
Hexis exposes controlled capabilities.
|
||||
|
||||
Examples:
|
||||
|
||||
- inspect service status
|
||||
- restart a registered service
|
||||
- inspect logs
|
||||
- query repository state
|
||||
- trigger a local application operation
|
||||
- perform a predefined filesystem action
|
||||
- invoke a registered HTTP API action
|
||||
|
||||
Hexis exposes:
|
||||
|
||||
- native API
|
||||
- CLI
|
||||
- MCP adapter
|
||||
|
||||
Maven should normally use the native API or a typed local client.
|
||||
|
||||
The MCP adapter exists primarily for coding agents and other MCP clients.
|
||||
|
||||
### 8.2 Capability flow
|
||||
|
||||
```text
|
||||
Maven
|
||||
-> resolve target through Nexus
|
||||
-> ask Hexis for capabilities applicable to entity
|
||||
-> select capability from router output or deterministic mapping
|
||||
-> validate arguments
|
||||
-> request confirmation when required
|
||||
-> execute through Hexis
|
||||
-> receive structured result
|
||||
-> phrase result
|
||||
```
|
||||
|
||||
### 8.3 Execution rules
|
||||
|
||||
Maven must not:
|
||||
|
||||
- generate shell commands for Hexis
|
||||
- bypass capability schemas
|
||||
- submit arbitrary target strings for mutating operations
|
||||
- interpret a successful command as operational recovery
|
||||
- retry unknown mutation outcomes automatically
|
||||
- treat MCP tool availability as permission
|
||||
|
||||
### 8.4 Confirmation
|
||||
|
||||
Confirmation is bound to:
|
||||
|
||||
- capability ID and version
|
||||
- canonical target entity
|
||||
- normalized arguments
|
||||
- requester identity
|
||||
- risk
|
||||
- expiry
|
||||
|
||||
Conversation state stores the pending confirmation.
|
||||
|
||||
Example:
|
||||
|
||||
```text
|
||||
User: restart the Muzick indexer.
|
||||
Maven: the restart affects Muzick indexer on homesrv. proceed?
|
||||
User: yes.
|
||||
```
|
||||
|
||||
The final `yes` is accepted only when a valid pending confirmation exists.
|
||||
|
||||
---
|
||||
|
||||
## 9. Maven as control center
|
||||
|
||||
Maven is a control center in the human-interface sense.
|
||||
|
||||
It provides one place to:
|
||||
|
||||
- ask what is happening
|
||||
- inspect attention
|
||||
- identify affected systems
|
||||
- request safe actions
|
||||
- understand results
|
||||
- move between voice, Telegram, and web
|
||||
- access personal memory and reminders
|
||||
|
||||
It is not a central orchestrator in the infrastructure sense.
|
||||
|
||||
The other services remain independently usable:
|
||||
|
||||
- Nexus through API and CLI
|
||||
- Praxis through API, CLI, and web
|
||||
- Hexis through API, CLI, and MCP
|
||||
|
||||
If Maven is down:
|
||||
|
||||
- identities remain available
|
||||
- observations continue
|
||||
- attention state remains visible
|
||||
- Hexis capabilities remain callable by authorized clients
|
||||
|
||||
---
|
||||
|
||||
## 10. Maven internal packages
|
||||
|
||||
Recommended additions or extensions:
|
||||
|
||||
```text
|
||||
pkg/
|
||||
context/
|
||||
snapshot.go
|
||||
providers.go
|
||||
nexus.go
|
||||
praxis.go
|
||||
hexis.go
|
||||
|
||||
integrations/
|
||||
nexus/
|
||||
praxis/
|
||||
hexis/
|
||||
|
||||
dialogue/
|
||||
pending_reference.go
|
||||
pending_confirmation.go
|
||||
continuation.go
|
||||
|
||||
tools/
|
||||
nexus_tools.go
|
||||
praxis_tools.go
|
||||
hexis_tools.go
|
||||
|
||||
trace/
|
||||
external_call.go
|
||||
entity_resolution.go
|
||||
execution.go
|
||||
```
|
||||
|
||||
The exact package layout may follow existing repository conventions.
|
||||
|
||||
### 10.1 Context providers
|
||||
|
||||
Each external service contributes a bounded context fragment.
|
||||
|
||||
```go
|
||||
type Provider interface {
|
||||
Name() string
|
||||
Collect(ctx context.Context) (Fragment, error)
|
||||
}
|
||||
```
|
||||
|
||||
Fragments include:
|
||||
|
||||
- freshness
|
||||
- source
|
||||
- compact structured content
|
||||
- degradation state
|
||||
- token-budget estimate
|
||||
|
||||
External service failure must not break a turn.
|
||||
|
||||
### 10.2 Typed clients
|
||||
|
||||
Each integration uses a typed client with:
|
||||
|
||||
- Unix socket and optional HTTP transport
|
||||
- context cancellation
|
||||
- strict deadlines
|
||||
- version negotiation
|
||||
- typed errors
|
||||
- bounded retries for safe reads
|
||||
- no automatic retries for uncertain mutations
|
||||
- correlation IDs
|
||||
|
||||
---
|
||||
|
||||
## 11. Router contract
|
||||
|
||||
The resident model emits a fixed structured action.
|
||||
|
||||
Conceptual shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"tool": "praxis.list_attention",
|
||||
"args": {
|
||||
"entity_id": "ent_correx"
|
||||
},
|
||||
"escalate": false
|
||||
}
|
||||
```
|
||||
|
||||
or:
|
||||
|
||||
```json
|
||||
{
|
||||
"tool": "hexis.execute",
|
||||
"args": {
|
||||
"capability_id": "cap_service_restart_muzick",
|
||||
"target_entity_id": "ent_muzick_indexer",
|
||||
"arguments": {}
|
||||
},
|
||||
"escalate": false
|
||||
}
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- tool names are versioned
|
||||
- exposed tool schemas are compact
|
||||
- only relevant tools are placed in router context
|
||||
- canonical entity IDs are injected after deterministic resolution where possible
|
||||
- the model does not authorize execution
|
||||
- all arguments are validated by the receiving system
|
||||
- malformed output falls back to the deterministic classifier path
|
||||
|
||||
---
|
||||
|
||||
## 12. World-state snapshot
|
||||
|
||||
Every route receives one immutable snapshot.
|
||||
|
||||
Example:
|
||||
|
||||
```go
|
||||
type Snapshot struct {
|
||||
CapturedAt time.Time
|
||||
|
||||
Time TimeContext
|
||||
Presence PresenceContext
|
||||
Calendar CalendarContext
|
||||
|
||||
Attention PraxisContext
|
||||
Entities NexusContext
|
||||
Actions HexisContext
|
||||
|
||||
Degraded []DependencyFailure
|
||||
}
|
||||
```
|
||||
|
||||
The snapshot must remain small.
|
||||
|
||||
It should contain summaries such as:
|
||||
|
||||
```text
|
||||
Praxis:
|
||||
- 2 high-attention items
|
||||
- Correx agent waiting
|
||||
- Maven calendar source stale
|
||||
|
||||
Hexis:
|
||||
- 4 applicable read capabilities
|
||||
- 1 mutating capability unavailable
|
||||
|
||||
Nexus:
|
||||
- current project resolved as Correx
|
||||
```
|
||||
|
||||
Full data is fetched only after routing selects a relevant tool.
|
||||
|
||||
---
|
||||
|
||||
## 13. Trace design
|
||||
|
||||
Maven's `/trace` must include cross-system calls.
|
||||
|
||||
A turn trace records:
|
||||
|
||||
```text
|
||||
input received
|
||||
stage-0 decision
|
||||
world-state snapshot
|
||||
router prompt contract version
|
||||
router output
|
||||
Nexus resolution request and result
|
||||
Praxis query and result references
|
||||
Hexis capability discovery
|
||||
confirmation proposal
|
||||
Hexis execution ID
|
||||
phrasing input and output
|
||||
delivery result
|
||||
```
|
||||
|
||||
Required fields:
|
||||
|
||||
- turn ID
|
||||
- correlation ID
|
||||
- causation ID
|
||||
- service
|
||||
- operation
|
||||
- duration
|
||||
- status
|
||||
- contract version
|
||||
- redacted request summary
|
||||
- redacted response summary
|
||||
- fallback reason
|
||||
- error code
|
||||
|
||||
Raw secrets, credentials, and unbounded payloads must never enter trace storage.
|
||||
|
||||
---
|
||||
|
||||
## 14. Failure handling
|
||||
|
||||
### Nexus unavailable
|
||||
|
||||
Maven may:
|
||||
|
||||
- answer using already verified entity IDs in current context
|
||||
- query Praxis by direct item ID
|
||||
- inspect existing pending confirmations
|
||||
|
||||
Maven must not:
|
||||
|
||||
- resolve new free-text action targets
|
||||
- execute mutations against unresolved targets
|
||||
- invent mappings
|
||||
|
||||
### Praxis unavailable
|
||||
|
||||
Maven may:
|
||||
|
||||
- continue personal memory, reminders, calendar, and direct Hexis operations
|
||||
- explain that operational attention is unavailable
|
||||
|
||||
It must not claim that there is nothing requiring attention.
|
||||
|
||||
### Hexis unavailable
|
||||
|
||||
Maven may:
|
||||
|
||||
- inspect Praxis
|
||||
- describe the action that would be applicable
|
||||
- report that execution is unavailable
|
||||
|
||||
It must not queue speculative mutating actions.
|
||||
|
||||
### Partial failure
|
||||
|
||||
The response should distinguish known state from unavailable state.
|
||||
|
||||
Example:
|
||||
|
||||
```text
|
||||
Muzick indexer is marked failed in Praxis. Hexis is unavailable, so I cannot inspect or restart it right now.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 15. Security model
|
||||
|
||||
Maven is not trusted to authorize itself.
|
||||
|
||||
Security boundaries:
|
||||
|
||||
- Nexus authoritatively resolves identities
|
||||
- Hexis authoritatively validates and executes capabilities
|
||||
- Praxis authoritatively owns operational lifecycle
|
||||
- Maven owns conversation and user intent
|
||||
|
||||
Requirements:
|
||||
|
||||
- separate service credentials
|
||||
- Unix sockets by default
|
||||
- authenticated HTTP when used
|
||||
- no browser-visible Nexus or Hexis credentials
|
||||
- no direct database access
|
||||
- no arbitrary command transport
|
||||
- no execution based solely on LLM output
|
||||
- no automatic attention-to-action chain
|
||||
- explicit user confirmation for protected actions
|
||||
- requester identity propagated to Hexis
|
||||
- all cross-service requests carry correlation IDs
|
||||
- all contracts are versioned
|
||||
|
||||
---
|
||||
|
||||
## 16. Web UI role
|
||||
|
||||
`mavweb` becomes the conversational control surface, not a replacement for the administrative UIs of the sibling systems.
|
||||
|
||||
Recommended additions:
|
||||
|
||||
### `/dash`
|
||||
|
||||
- compact Praxis attention summary
|
||||
- degraded dependency summary
|
||||
- active confirmations
|
||||
- recent Hexis executions
|
||||
- relevant personal reminders
|
||||
|
||||
### `/history`
|
||||
|
||||
- Maven conversation history
|
||||
- linked Praxis items and Hexis executions
|
||||
|
||||
### `/trace`
|
||||
|
||||
- full cross-service turn trace
|
||||
|
||||
### `/notifications`
|
||||
|
||||
- Maven delivery and reach state
|
||||
- references to Praxis items where applicable
|
||||
|
||||
### `/tools`
|
||||
|
||||
- compact user-facing view of relevant Hexis capabilities
|
||||
- capability availability
|
||||
- risk
|
||||
- canonical target
|
||||
- no raw command definitions
|
||||
|
||||
Administrative identity editing should remain in Nexus or Praxis administration.
|
||||
|
||||
Provider and capability registration should remain in Hexis administration.
|
||||
|
||||
---
|
||||
|
||||
## 17. Example scenarios
|
||||
|
||||
### 17.1 Morning summary
|
||||
|
||||
1. Maven digestion queries Praxis.
|
||||
2. Praxis returns:
|
||||
- Correx agent waiting
|
||||
- Muzick agent failed
|
||||
- calendar source stale
|
||||
3. Maven adds personal reminders and calendar state.
|
||||
4. Maven phrases one bounded summary.
|
||||
5. Included Praxis items are marked surfaced.
|
||||
6. They are not automatically acknowledged.
|
||||
|
||||
### 17.2 Inspect waiting agent
|
||||
|
||||
```text
|
||||
User: what is Correx waiting for?
|
||||
```
|
||||
|
||||
1. Nexus resolves `Correx`.
|
||||
2. Praxis finds the waiting agent item for that project.
|
||||
3. Maven reads the bounded terminal evidence.
|
||||
4. Maven phrases the exact question.
|
||||
5. Hexis is not involved.
|
||||
|
||||
### 17.3 Restart service
|
||||
|
||||
```text
|
||||
User: restart the Muzick indexer.
|
||||
```
|
||||
|
||||
1. Nexus resolves the service.
|
||||
2. Hexis returns applicable restart capability.
|
||||
3. Maven requests confirmation according to risk.
|
||||
4. User confirms.
|
||||
5. Hexis executes.
|
||||
6. Maven reports command success.
|
||||
7. Praxis waits for independent recovery.
|
||||
8. Maven later reports whether recovery occurred.
|
||||
|
||||
### 17.4 Ambiguous target
|
||||
|
||||
```text
|
||||
User: restart the indexer.
|
||||
```
|
||||
|
||||
1. Nexus returns Muzick and Manga indexers.
|
||||
2. Maven asks which one.
|
||||
3. No Hexis execution request is created.
|
||||
|
||||
### 17.5 Dependency outage
|
||||
|
||||
```text
|
||||
User: what needs attention?
|
||||
```
|
||||
|
||||
Praxis is unavailable.
|
||||
|
||||
Maven responds that operational attention cannot currently be read, while still reporting personal reminders and local Maven state.
|
||||
|
||||
---
|
||||
|
||||
## 18. Implementation phases
|
||||
|
||||
### Phase 1 — Read-only integration
|
||||
|
||||
- typed Nexus client
|
||||
- typed Praxis client
|
||||
- typed Hexis client
|
||||
- world-state provider framework
|
||||
- trace integration
|
||||
- read-only Maven tools:
|
||||
- resolve entity
|
||||
- list attention
|
||||
- list changes
|
||||
- inspect item
|
||||
- list capabilities
|
||||
- inspect capability
|
||||
|
||||
### Phase 2 — Safe state transitions
|
||||
|
||||
- acknowledge Praxis item
|
||||
- resolve Praxis item
|
||||
- ignore and pin operations
|
||||
- dialogue continuation
|
||||
- explicit entity disambiguation
|
||||
- cross-service correlation IDs
|
||||
|
||||
### Phase 3 — Controlled execution
|
||||
|
||||
- Hexis execution request
|
||||
- confirmation lifecycle
|
||||
- execution status polling
|
||||
- unknown-outcome handling
|
||||
- Praxis execution correlation
|
||||
- web execution history
|
||||
|
||||
### Phase 4 — Digestion integration
|
||||
|
||||
- Praxis change cursor
|
||||
- bounded operational summaries
|
||||
- surfacing semantics
|
||||
- repeated-pattern detection
|
||||
- source degradation awareness
|
||||
- no autonomous execution
|
||||
|
||||
### Phase 5 — Hardening
|
||||
|
||||
- protocol compatibility tests
|
||||
- degraded-mode tests
|
||||
- replay fixtures
|
||||
- cross-service integration harness
|
||||
- latency budgets
|
||||
- security tests
|
||||
- model routing evaluation with ecosystem tools
|
||||
|
||||
---
|
||||
|
||||
## 19. Acceptance criteria
|
||||
|
||||
1. Maven can resolve Russian and English names through Nexus.
|
||||
2. Ambiguous identities always cause clarification before mutation.
|
||||
3. Maven can summarize Praxis attention without duplicating Praxis state.
|
||||
4. Maven preserves surfaced, acknowledged, and resolved semantics.
|
||||
5. Maven can discover Hexis capabilities for canonical entities.
|
||||
6. Maven cannot submit arbitrary commands to Hexis.
|
||||
7. Protected Hexis actions require bound confirmation.
|
||||
8. Successful execution is reported separately from observed recovery.
|
||||
9. Praxis remains responsible for operational item resolution.
|
||||
10. All external calls appear in Maven trace.
|
||||
11. Nexus, Praxis, and Hexis outages degrade independently.
|
||||
12. No component database is accessed directly.
|
||||
13. The resident model is used only for routing, phrasing, and bounded synthesis.
|
||||
14. LLM output never bypasses deterministic validation or policy.
|
||||
15. Maven remains useful when all three sibling systems are unavailable.
|
||||
16. Nexus, Praxis, and Hexis remain useful when Maven is unavailable.
|
||||
17. Cross-service contracts are versioned.
|
||||
18. Requests propagate requester identity and correlation IDs.
|
||||
19. No automatic Praxis-to-Hexis execution path exists.
|
||||
20. The full integration works without cloud services.
|
||||
|
||||
---
|
||||
|
||||
## 20. Final invariant
|
||||
|
||||
```text
|
||||
Maven may coordinate the ecosystem,
|
||||
but it never replaces the authority of the system it calls.
|
||||
```
|
||||
|
||||
- Nexus is authoritative for identity.
|
||||
- Praxis is authoritative for operational attention and lifecycle.
|
||||
- Hexis is authoritative for capabilities and execution.
|
||||
- Maven is authoritative for conversation, personal context, and user-facing coordination.
|
||||
@@ -0,0 +1,230 @@
|
||||
# Resident model bake-off — 31-07-2026
|
||||
|
||||
**Outcome: the resident model is stock Qwen3-1.7B** (`UD-Q4_K_XL`). Two sweeps ran this
|
||||
evening and the second one changed the answer — read to the end before acting on any table
|
||||
here. [Second sweep](#second-sweep-same-evening--five-models-and-a-resident-model-change)
|
||||
is the one that holds.
|
||||
|
||||
## First sweep — LFM2.5-1.2B vs Qwen3.5-0.8B
|
||||
|
||||
**Verdict, scoped to this pair: keep Qwen3.5-0.8B over LFM2.5-1.2B.** LFM2.5-1.2B is worse
|
||||
at routing (52.6% vs 60.5% intent accuracy), and the loss is almost entirely Russian
|
||||
(18/61 vs 22/61 RU, while EN is a wash). It is also 2.4× slower. The Thinking variant is
|
||||
far worse again. This verdict still stands as written — it rejects LFM2.5-1.2B. It is
|
||||
**not** a recommendation to keep 0.8B as the resident model; the second sweep replaced it
|
||||
with Qwen3-1.7B.
|
||||
|
||||
Settles Vikunja **#278 / #250**.
|
||||
|
||||
- Same fixture and scorer as `docs/evals/2026-07-31-routing.md`: `internal/router/eval/`
|
||||
(`ru_routing_v1.json`, 76 held-out cases).
|
||||
- Reproduce: `MAVEN_LLM_URL=http://127.0.0.1:<port> make eval-router`
|
||||
(`TestLLMRouterBaseline`). (This line used to say there is no `make eval-models` target.
|
||||
There is one now — start a server with the gguf you want, then
|
||||
`make eval-models MAVEN_LLM_URL=http://127.0.0.1:<port>`. It runs only the LLM test, since
|
||||
the classifier baselines do not depend on the model.)
|
||||
- All three models served by the same `llama-server` flags — `-c 2048 -ngl 99 -t 6`, only
|
||||
`-m` and `--port` differ. One server at a time on an otherwise idle box, so latencies are
|
||||
real and not contention.
|
||||
- Measured on top of the router prompt fix (`origin/overnight/router-prompt` merged in), so
|
||||
the Qwen column is directly comparable to the numbers already recorded.
|
||||
|
||||
## Results
|
||||
|
||||
`llm-only` — the model alone. This is the column that measures the model.
|
||||
|
||||
| | Qwen3.5-0.8B | LFM2.5-1.2B Instruct | LFM2.5-1.2B Thinking |
|
||||
|---|---|---|---|
|
||||
| **intent-only accuracy** | **60.5%** | 52.6% | 36.8% |
|
||||
| full accuracy (intent+slots+gate) | **36.8%** | 32.9% | 21.1% |
|
||||
| **RU** | **22/61** | 18/61 | 10/61 |
|
||||
| EN | 6/15 | **7/15** | 6/15 |
|
||||
| route errors | 0 | 0 | 0 |
|
||||
| **p50 / p95 latency** | **1.05s / 1.71s** | 2.47s / 3.62s | 2.42s / 3.24s |
|
||||
| missed clarify | 6 / 6 | 6 / 6 | 6 / 6 |
|
||||
|
||||
`cascade+llm` — stage-0 → model → classifier floor, what #320 would actually ship. Same
|
||||
ordering.
|
||||
|
||||
| | Qwen3.5-0.8B | LFM2.5-1.2B Instruct | LFM2.5-1.2B Thinking |
|
||||
|---|---|---|---|
|
||||
| intent-only accuracy | **61.8%** | 55.3% | 38.2% |
|
||||
| full accuracy | **46.1%** | 42.1% | 30.3% |
|
||||
| RU / EN | **27/61** / 8/15 | 23/61 / **9/15** | 15/61 / 8/15 |
|
||||
| route errors | 0 | 0 | 0 |
|
||||
| p50 / p95 latency | **1.28s / 1.94s** | 2.18s / 2.72s | 2.27s / 3.19s |
|
||||
|
||||
Full logs: the three runs are archived in the session scratchpad
|
||||
(`qwen08.txt`, `lfm-instruct.txt`, `lfm-thinking.txt`).
|
||||
|
||||
## Russian-specific failures — the owner's worry is confirmed
|
||||
|
||||
LFM2.5's Russian loss is not spread out. It has one large, specific failure: **it hears
|
||||
almost any Russian imperative or short phrase as `reminder`.**
|
||||
|
||||
- `перезапусти докер` → reminder (want act)
|
||||
- `включи вытяжку` → reminder (want act)
|
||||
- `закрой жалюзи` → reminder (want act)
|
||||
- `заметка: продлить домен в августе` → reminder (want note)
|
||||
- `запиши что кран на кухне снова капает` → reminder (want note)
|
||||
- `доброе утро` → reminder (want chat)
|
||||
- `спасибо тебе` → reminder (want note/chat)
|
||||
- `переходи в тихий режим` → reminder (want system)
|
||||
|
||||
That is `note→reminder ×4`, `act→reminder ×4`, `chat→reminder ×2` in one run. Qwen's
|
||||
equivalent failure axis is `query→fact ×8`, which is a narrower and already-understood bug.
|
||||
|
||||
Two more Russian-side problems worth naming:
|
||||
|
||||
1. **Fact keys come back empty or wrong in Russian.** `воды попил наконец`, `поужинал`,
|
||||
`поспал часов пять` and `отметь что я позавтракал овсянкой` all returned an empty key.
|
||||
`сходил в душ` and `отдохнул минут двадцать` both returned `water`. Qwen does not do this.
|
||||
2. **It leaked German.** `slept about seven hours` produced the fact key
|
||||
`"7 Stunden geschlafen"`. Grammar-valid, semantically garbage — a sign the multilingual
|
||||
mix is not anchored where Maven needs it.
|
||||
|
||||
The claimed tool-calling advantage did not show up here. `act` is the closest thing this
|
||||
fixture has to a tool call, and LFM2.5 got it wrong more often than Qwen, mostly by calling
|
||||
it a reminder. It also produced no `fn` slot on any act, same as Qwen.
|
||||
|
||||
## The Thinking variant
|
||||
|
||||
Not viable. 36.8% intent accuracy, 10/61 Russian, and no latency saving over Instruct — the
|
||||
thinking trace costs time without buying accuracy on a short enum classification. With the
|
||||
`enable_thinking=false` diagnostic it collapsed further to 28.9% with 2 route errors
|
||||
(`query→reminder ×12`). Do not pursue.
|
||||
|
||||
## Notes
|
||||
|
||||
- Nothing crashed, nothing ignored the GBNF grammar, and no model produced unparseable JSON
|
||||
in the shippable configurations. Zero route errors for both Instruct and Thinking in
|
||||
`llm-only` and `cascade+llm`. The problem with LFM2.5 is what it decides, not whether it
|
||||
can emit the contract.
|
||||
- The `6 / 6` missed clarify is unchanged across all three models. No model fixes the missing
|
||||
refusal lane — that is `Confidence: 1.0` hardcoded in `llmrouter.go` (Vikunja #359), not a
|
||||
model property.
|
||||
- The report labels every configuration `(0.8B)`; that string is hardcoded in the test, not a
|
||||
reflection of which gguf was loaded. Model identity was confirmed per run via `/v1/models`.
|
||||
- No Go code was changed for this measurement, and no bug was found that needed one.
|
||||
|
||||
## What this does not settle
|
||||
|
||||
Routing only. LFM2.5 might still phrase better, and phrasing is the resident model's other
|
||||
job — that needs its own fixture. But routing is the load-bearing path and Maven is
|
||||
Russian-first, so on the evidence here the switch is not worth making.
|
||||
|
||||
---
|
||||
|
||||
# Second sweep, same evening — five models, and a resident-model change
|
||||
|
||||
The sections above compared LFM2.5-1.2B against Qwen3.5-0.8B on routing and concluded
|
||||
"the switch is not worth making". That still holds. This sweep asked a different
|
||||
question — whether a *smaller* model could work, since LFM2.5's published
|
||||
instruction-following scores beat Qwen3.5-0.8B badly — and answered it, plus found a
|
||||
better resident model by accident.
|
||||
|
||||
**Outcome: the resident model is now stock Qwen3-1.7B.** Sub-500M is a dead end.
|
||||
|
||||
## Routing — 77 Russian cases, one run each
|
||||
|
||||
| model | on disk | llm-only (full) | llm-only (intent) | cascade + fallback |
|
||||
|---|---|---|---|---|
|
||||
| LFM2.5-230M-Q8_0 | 246 MB | 23.4% | 33.8% | 36.4% |
|
||||
| LFM2.5-350M-Q8_0 | 379 MB | 2.6% | **5.2%** | 20.8% |
|
||||
| Qwen3.5-0.8B-Q4_K_M | 527 MB | 36.4% | 59.7% | 61.0% |
|
||||
| Qwen3.5-2B-UD-Q4_K_XL | 1.34 GB | 42.9% | 62.3% | 63.6% |
|
||||
| **Qwen3-1.7B-UD-Q4_K_XL (stock)** | 1.13 GB | **44.2%** | **67.5%** | **72.7%** |
|
||||
|
||||
Qwen3-1.7B wins every column, including against a model 20% larger than it.
|
||||
|
||||
## Talk fixture — 27 cases, three runs each, idle box
|
||||
|
||||
| | Qwen3.5-0.8B | Qwen3-1.7B stock |
|
||||
|---|---|---|
|
||||
| composite | 13, 11, 8 | **20, 21, 18** |
|
||||
| address | 21, 18, 18 | **26, 25, 23** |
|
||||
| feminine | 27, 25, 26 | 26, 27, 26 |
|
||||
| lang | 27, 27, 26 | 26, 27, 27 |
|
||||
| ontopic | 16, 19, 19 | **22, 23, 23** |
|
||||
| canned fallbacks | 8, 5, 6 | **0, 2, 0** |
|
||||
|
||||
This also fills the row `docs/evals/2026-07-31-talk.md` had to void for contamination:
|
||||
**600ch/1024tok on Qwen3.5-0.8B scores 13, 11, 8.**
|
||||
|
||||
`address` is the headline. It sat at 18-22 of 27 on the 0.8B no matter how the prompt
|
||||
was worded — the prompt explicitly forbids "вы" and the model writes `вашей`,
|
||||
`подождите`, `делаете` anyway. That was read as "prompting is out of levers", and it
|
||||
was really "0.8B is out of capacity". The 1.7B mostly holds the constraint.
|
||||
|
||||
The fallback column matters too: 5-8 of 27 turns on the 0.8B end in a hardcoded
|
||||
`"не знаю."`, meaning it failed to emit parseable JSON about a quarter of the time.
|
||||
The 1.7B does that 0-2 times.
|
||||
|
||||
## Latency — the long tail is not the Thinking block
|
||||
|
||||
> **Stale, corrected 2026-08-02.** The p50 figures in this table are contention on a
|
||||
> shared llama-server, not the model's cost. The router measures p50 825ms / p95 1.2s /
|
||||
> max 3.0s in `docs/evals/2026-07-31-routing.md`, which says so at line 61. Read this table for
|
||||
> the shape of the tail only. Take absolute latency from the routing eval.
|
||||
|
||||
| | p50 | p95 |
|
||||
|---|---|---|
|
||||
| Qwen3.5-0.8B | 2.4s, 2.9s, 2.0s | 17.4s, 17.6s, 17.4s |
|
||||
| Qwen3-1.7B stock | 2.7s, 2.6s, 2.8s | 16.4s, 6.6s, 3.9s |
|
||||
|
||||
p50 is flat across a 2× size difference. The first instinct on seeing the 1.7B's
|
||||
16s p95 was "that is the reasoning trace, cap it" — wrong. The 0.8B's p95 is a
|
||||
consistent 17s and the 1.7B beat it in two of three runs. The tail is shared and
|
||||
lives somewhere else. Do not spend time on `/no_think` on this evidence.
|
||||
|
||||
## Sub-500M: not close, and the benchmarks say otherwise for a reason
|
||||
|
||||
LFM2.5-350M publishes IFEval 76.96 against Qwen3.5-0.8B's 59.94, and BFCLv3 44.11
|
||||
against 35.08 — better at instruction-following and structured output, at 2/3 the
|
||||
size. Those numbers are real and they are **English**. Every benchmark in that
|
||||
table except Multi-IF is English-only.
|
||||
|
||||
In Russian, with a 300-token budget and temperature 0:
|
||||
|
||||
- **350M**, «Столица Франции? Ответь кратко.» → *«Сторзит в Париже.»* — `Сторзит` is
|
||||
not a word; it is invented morphology.
|
||||
- **350M**, asked to read back a reminder → a fortune cookie about being attentive
|
||||
and confident. No reminder in it.
|
||||
- **230M**, «Привет, как дела?» → answered **in Spanish**.
|
||||
|
||||
The 230M beating the 350M six-fold on routing (33.8% vs 5.2%) is the other tell:
|
||||
when the larger sibling collapses like that it is format compliance failing, not
|
||||
reasoning.
|
||||
|
||||
This is a pretraining gap, not a fine-tuning gap. Teaching Russian to a 350M from
|
||||
near-zero is not an afternoon on a Colab, which was the premise worth checking.
|
||||
|
||||
## Why this vindicates the 1.7B CPT
|
||||
|
||||
Stock Qwen3-1.7B, untrained and unprompted, answers all three probes in fluent
|
||||
correct Russian. What it gets wrong is the persona: *«Привет! Я рад, что ты здесь»*
|
||||
— `рад` is masculine and Maven needs `рада`. That is the right kind of remaining
|
||||
problem, and it is exactly what the CPT (Vikunja #122) is for.
|
||||
|
||||
The 1.7B was the correct model choice. What was wrong was treating it as a
|
||||
**blocker**: stock already beats what was deployed, so it ships now and gets
|
||||
swapped again when the CPT lands.
|
||||
|
||||
## Caveats
|
||||
|
||||
- Routing is one run per model, not three. The gaps between families are far larger
|
||||
than the run-to-run spread seen on the talk fixture, but the 2B-vs-1.7B gap (62.3
|
||||
vs 67.5) is not safe to call on one run.
|
||||
- ~~The routing numbers only reach production once the LLM router is wired on. It is
|
||||
still `nil`.~~ **Resolved the same evening:** the LLM router is wired at `voice.go:214`
|
||||
behind `voice.llm_router`, the default is on, and `deploy/mavend.json` sets it `true`.
|
||||
These numbers are the production path now. **Corrected 2026-08-02: the p50 ≈2.7s in the
|
||||
latency table above WAS a bench artifact.** It is contention on the shared llama-server,
|
||||
not the model. `docs/evals/2026-07-31-routing.md` line 61 says so, and measures the router at
|
||||
p50 825ms / p95 1.2s / max 3.0s. Cite that file for latency, not this one.
|
||||
- ~~`/mnt/hdd1/llms/LFM2.5/Qwen3-1.7B-UD-Q4_K_XL.gguf` is a 293 MB truncated download
|
||||
in the wrong directory.~~ **Deleted 2026-07-31.** The good 1.13 GB copy in `qwen3/` is
|
||||
what `deploy/mavend.json` loads.
|
||||
- Harness: `scratchpad/bakeoff.sh`, one server at a time, health-checked before each
|
||||
run, `/v1/models` recorded per run. Never run two LLM consumers at once — see the
|
||||
contamination note in `docs/evals/2026-07-31-talk.md`.
|
||||
@@ -0,0 +1,169 @@
|
||||
# Phrasing evaluation — 31-07-2026
|
||||
|
||||
How Maven words a nudge, measured instead of argued. Counterpart to
|
||||
`docs/evals/2026-07-31-routing.md`.
|
||||
|
||||
- Fixture + scorer: `internal/phraser/eval/` (`nudges_v1.json`, 15 cases; `eval.go`, `checks.go`)
|
||||
- Reproduce: `MAVEN_LLM_URL=http://127.0.0.1:18099 make eval-phrasing`
|
||||
- Model: Qwen3.5-0.8B Q4_K_M, the resident model. Not swapped.
|
||||
- Commit: `a40bc55` (prompt fix)
|
||||
|
||||
Every check is a string or length test a human can read and disagree with. No model
|
||||
grades another model here.
|
||||
|
||||
## Result
|
||||
|
||||
| | before | after |
|
||||
|---|---|---|
|
||||
| **cases passing every check** | **0/15** | **13/15** |
|
||||
| mood in enum | 6/15 | 15/15 |
|
||||
| Russian | 2/15 | 14/15 |
|
||||
| length (≤120 chars, ≤16 words) | 13/15 | 15/15 |
|
||||
| feminine self-reference | 15/15 | 15/15 |
|
||||
| no cringe | 13/15 | 15/15 |
|
||||
| on topic | 6/15 | 13/15 |
|
||||
| p50 latency | 11.4s | 11.4s |
|
||||
|
||||
Latency did not move and is not good. 11s to word one nudge on this box.
|
||||
|
||||
## The bug reproduced
|
||||
|
||||
Yes, exactly as reported. 7 of 15 messages were the literal string `"..."`, and one was
|
||||
`"full voice message"`. Both are text copied straight out of the prompt.
|
||||
|
||||
The system prompt said:
|
||||
|
||||
```
|
||||
Respond ONLY with valid JSON: {"response": "full voice message", "mood": "neutral"}
|
||||
```
|
||||
|
||||
and the user prompt said:
|
||||
|
||||
```
|
||||
Respond as JSON: {"response": "...", "mood": "..."}
|
||||
```
|
||||
|
||||
A 0.8B does not read `"..."` as "put your answer here". It reads it as the answer. The
|
||||
prompt was a worked example whose worked part was blank, so the model filled the slot by
|
||||
copying. This is the whole of finding 1.
|
||||
|
||||
## What else was wrong
|
||||
|
||||
Four separate faults, all prompt-side:
|
||||
|
||||
1. **Placeholder echo** (7 cases) — above.
|
||||
2. **Wrong language** (13/15 failed the language check). The prompt was entirely English
|
||||
and said "in the user's language (Russian or English)". The model picked English. It is
|
||||
never English: the nudge is spoken by a Russian piper voice.
|
||||
3. **Rule names are English identifiers.** `netdata_critical`, `service_down`, `break` went
|
||||
into the prompt raw. The model cannot nudge about a topic it has not been told in words,
|
||||
so 9/15 were off topic. The daemon knows what its own rules mean; now it says so.
|
||||
4. **Mood invented** (`"warm"`, twice). The enum was listed in a parenthesis at the end of
|
||||
an English sentence. Now it is its own line: "ровно одно из: neutral, happy, thinking,
|
||||
tired, confused."
|
||||
|
||||
Plus two non-prompt faults the run exposed:
|
||||
|
||||
- **The no-parse fallback was English.** When the model returned nothing usable, the body
|
||||
became `fmt.Sprintf("%s — %s", rule, sev)` — `"water — care"` — and that string went to
|
||||
a Russian TTS. Now it falls back to plain Russian.
|
||||
- **Durations were English.** `humanDur` returns "3 hours"; it was landing verbatim inside
|
||||
Russian sentences. Nudges now use a Russian formatter.
|
||||
|
||||
## Three iterations, and what each taught
|
||||
|
||||
| | score | change |
|
||||
|---|---|---|
|
||||
| baseline | 0/15 | — |
|
||||
| iter 1 | 2/15 | Russian prompt, filled-in examples, Russian durations |
|
||||
| iter 2 | 11/15 | required keyword per rule, one example instead of five, Russian fallback |
|
||||
| iter 3 | **13/15** | examples moved to topics that are not rules |
|
||||
|
||||
The interesting step is 1 → 2. Fixing the placeholder did not fix the disease, it moved it:
|
||||
the model stopped copying `"..."` and started copying my first example instead. Five nudges
|
||||
in a row came back as `"Ты не пил воду три часа. Налей стакан."` regardless of the rule.
|
||||
|
||||
**A small model copies the nearest concrete text in its prompt.** That is one failure mode
|
||||
with two symptoms. The fix that stuck was making the examples about laundry and a laptop
|
||||
battery — topics no rule ever produces, so copying them is visible in the score rather than
|
||||
invisibly passing the water cases.
|
||||
|
||||
## Do not oversell 13/15
|
||||
|
||||
Seven of the thirteen passes are the **deterministic fallback**, not the model:
|
||||
`"Напоминаю: таблетки."`, `"Сервис не отвечает."`, `"Критический алярм: проверь диск."`,
|
||||
`"Ты давно не пил воду."`. Those are strings this commit added to Go. The model returned
|
||||
nothing parseable and the fallback scored.
|
||||
|
||||
So the honest reading is roughly **6/15 from the model, 7/15 from a fallback, 2/15 failing**.
|
||||
The prompt fix is real — `"..."` is nearly gone and the language and mood checks are clean —
|
||||
but a large part of the jump is that failure now degrades into Russian instead of into
|
||||
`"water — care"`. That is a genuine improvement for the operator and a weak one for the model.
|
||||
|
||||
The two remaining failures: one `"..."` recurrence (`routine-stretch`) and one meal nudge
|
||||
that never says food.
|
||||
|
||||
## Tried and reverted: an example-led nudge prompt (#393)
|
||||
|
||||
The idea was that a 0.8B copies examples better than it follows rules, so the nudge prompt
|
||||
was rewritten to lead with five on-topic examples (water, break, pills, morning, service) and
|
||||
the prose rules were compressed to pay for the tokens: 1190 chars down to 986.
|
||||
|
||||
It measured **worse**, three runs each side, same llama-server, same fixture:
|
||||
|
||||
| run | before | after |
|
||||
|---|---|---|
|
||||
| 1 | 12/15 (address 14) | 11/15 (address 13) |
|
||||
| 2 | 13/15 (address 15) | 12/15 (address 15) |
|
||||
| 3 | 14/15 (address 15) | 11/15 (address 12) |
|
||||
|
||||
`feminine` and `hisgender` were 15/15 on all six runs, so they measure nothing here. The
|
||||
regression is all in `address`: 44/45 before, 40/45 after. Formal "вы"/"ваше" and plural
|
||||
imperatives came back, and so did `"..."`.
|
||||
|
||||
Two likely causes, both about the same thing — **examples do not carry a prohibition**. The
|
||||
old prompt spent a whole sentence on «говоришь на "ты", в единственном числе»; the new one
|
||||
demoted that to one item in a long "никогда" list, and the model stopped obeying it. And
|
||||
making the examples on-topic let their *wording* leak: a break case came back as
|
||||
«Вы давно не пили воду. Выпей стакан.» — the water example, verbatim, in the wrong slot.
|
||||
That is exactly the failure the laundry/laptop examples were chosen to avoid.
|
||||
|
||||
Change reverted. What survives is the measurement: a rule the model must obey needs its own
|
||||
sentence, and examples must stay off-topic. Also note the before side alone spans 12–14 of
|
||||
15 — this fixture cannot resolve anything smaller than about three cases.
|
||||
|
||||
## Broken, found, not fixed
|
||||
|
||||
1. ~~**`checkFeminine` only catches half the constraint.**~~ **Fixed** (#381). It scanned for
|
||||
masculine self-reference only, so three messages that addressed the *owner* in the feminine
|
||||
("ты давно не отдыхал**а**") scored clean. There is now a second check, `hisgender`: a
|
||||
feminine past-tense verb (-ла/-лась) in a sentence addressed to him ("ты", "тебе", "твой")
|
||||
fails, unless the verb is hers ("я заметила", "напомнила тебе"). It is a suffix rule, not a
|
||||
parser — see the comment in `checks.go` for what it misses. A fresh 15-case run after adding
|
||||
it scored **12/15** with `hisgender` 15/15; the model did not repeat the feminine address in
|
||||
that sample, and the check is pinned by unit tests on the recorded bad strings instead.
|
||||
2. **Grammar is not checked at all, and it is bad.** `"Он не ел 11 дней"` (it was 11 hours),
|
||||
`"Сонуждились 7 дней"` (not a word), `"Они забыли воду"` (wrong person entirely). Every
|
||||
one of these passes all six checks. The fixture measures properties, not fluency, and at
|
||||
0.8B fluency is the binding constraint.
|
||||
3. **Unit confusion.** The model turns hours into days about a third of the time. The
|
||||
prompt now says "11 ч"; it reads it as days.
|
||||
4. **11s p50.** Unchanged and untouched here. A nudge the model takes eleven seconds to
|
||||
word has missed its moment. Worth its own task.
|
||||
5. **The keyword hint is close to teaching to the test.** `ruleKeywords` names the word the
|
||||
on-topic check looks for. It is defensible — the daemon genuinely knows its rule topics
|
||||
and the model genuinely cannot infer them from `netdata_critical` — but the on-topic
|
||||
number is softer than the others because of it.
|
||||
|
||||
## Next steps
|
||||
|
||||
1. ~~**Add a second-person gender check**~~ — done, `hisgender` in `checks.go` (#381).
|
||||
2. **Decide whether the fallback should count as a pass.** Right now `Score` cannot tell a
|
||||
model answer from a fallback. Either mark fallback bodies in `PhrasedNudge` or count them
|
||||
in their own column. Without that, any future prompt change can score well by failing
|
||||
more.
|
||||
3. **Attack the 11s.** Nudge phrasing is short and non-interactive; thinking off is the first
|
||||
thing to try, as it was for routing (#376).
|
||||
4. **Re-measure when #122 lands.** The CPT'd Qwen3-1.7B is the target. 13/15 with seven
|
||||
fallbacks is the floor it has to beat, and the fluency problems above are the ones a
|
||||
bigger, Russian-trained checkpoint should actually fix.
|
||||
@@ -0,0 +1,250 @@
|
||||
# Note recall evaluation — 31-07-2026
|
||||
|
||||
The operator's goal is that Maven "memorize/note things … and know more about me/world". This
|
||||
measures whether the note/recall path delivers that.
|
||||
|
||||
- Fixture + scorer: `internal/memory/recalleval/` (`ru_recall_v1.json`, 30 cases)
|
||||
- Reproduce: `make eval-recall` — hash ratchet always, ONNX when `deps/` is present
|
||||
- Commit: `43470ab` (harness)
|
||||
|
||||
Each case inserts its own 3 notes **plus 12 shared filler notes** into a fresh store, embeds the
|
||||
query, takes the top 3 — the read path `cmd/mavend/voice.go` runs for `IntentQuery`. Filler is
|
||||
load-bearing: with 3 notes and a top-3 search, recall@3 is 100% by construction. 25 answerable
|
||||
cases (paraphrased queries, homelab and preference content, 9 with a plausible second note) and 5
|
||||
that must recall **nothing**. `TestFixtureIsParaphrased` fails the build if a query shares over half
|
||||
its words with its note; equal-score ties count as ties, not recall.
|
||||
|
||||
## Results
|
||||
|
||||
| | recall+hash (CI ratchet) | recall+onnx (deployed) |
|
||||
|---|---|---|
|
||||
| **recall@1** | 36.0% (9/25) | **60.0% (15/25)** |
|
||||
| recall@3 | 76.0% (19/25) | 80.0% (20/25) |
|
||||
| **answered after the 0.55 gate** | **0.0% (0/25)** | **48.0% (12/25)** |
|
||||
| wrong note on top / tie on top | 9 / 7 | 10 / 0 |
|
||||
| ranked first, then silenced by the gate | 9 | 3 |
|
||||
| **false recall** | 0/5 | **1/5 (20%)** |
|
||||
| top-1 score when right, min / median | n/a | 0.559 / 0.678 |
|
||||
| top-1 when it must stay silent, median / max | 0.000 / 0.144 | 0.470 / **0.567** |
|
||||
| RU / EN / `hard` cases passed | 4/24 / 1/6 / 0/11 | 13/24 / 3/6 / 2/11 |
|
||||
| latency p50 / p95 / max | 49µs / 70µs | 59ms / 148ms / 194ms |
|
||||
|
||||
Never compare a hash-embedder number to an ONNX one — the hash floor is lexical and exists only so
|
||||
CI has a deterministic ratchet with no model files.
|
||||
|
||||
## Findings
|
||||
|
||||
### 1. Real recall is 48%, not 60%
|
||||
|
||||
The right note ranks first 60% of the time, but the daemon only *says* it 48% of the time — three
|
||||
more cases rank first and are then silenced by `voice.go:776`'s `queryMinScore`. **Roughly one
|
||||
useful question in two gets "не знаю".** This is not a working memory yet.
|
||||
|
||||
### 2. The gate cannot separate a real recall from a false one — the distributions overlap
|
||||
|
||||
Right-note top-1 scores start at **0.559**. Must-stay-silent top-1 scores reach **0.567**. No
|
||||
threshold keeps every real recall and rejects every false one. From the sweep: gate 0.50 → 13/25
|
||||
answered, 1/5 false; **0.55 (default) → 12/25, 1/5**; **0.60 → 10/25, 0/5**; 0.70 → 5/25, 0/5. What
|
||||
the data says about `DefaultQueryMinScore` (`internal/config/config.go:392`): **0.55 is
|
||||
slightly too loose** — it admits one confident wrong answer ("как зовут сестру моего коллеги"
|
||||
recalls "выучил пару аккордов на гитаре" at 0.567), which the spec ranks as worse than a gap. 0.60
|
||||
silences all five and costs 8 points of real recall. Left alone as instructed; the overlap means
|
||||
the threshold is the wrong dial anyway (finding 3).
|
||||
|
||||
### 3. Filler notes outrank the right answer — the model scores similarity, not relevance
|
||||
|
||||
`models/embedder/` is **paraphrase-multilingual-MiniLM-L12-v2** (`Makefile:119`), a *symmetric*
|
||||
paraphrase model. It scores "do these sentences look alike", not "does this passage answer this
|
||||
question", so question-shaped queries drift to whatever note is stylistically closest. "из-за чего
|
||||
кончилось место" and "откуда берётся токен бота" both return `выучил пару аккордов на гитаре`
|
||||
(0.730, 0.729); "как я восстановил конфиги" returns a bootloader note at 0.703 with the right note
|
||||
not even in the top 3. An unrelated guitar note beating a homelab note at 0.73 is not a tuning
|
||||
problem — an asymmetric retrieval model (`multilingual-e5-small`, with `query:` / `passage:`
|
||||
prefixes) is the targeted fix, and it would move findings 1 and 2 together. Separately:
|
||||
`deploy/mavend.json:39` loads a 470MB fp32 `model.onnx` while `make download-embedder` fetches
|
||||
`model_quantized.onnx` — not the same file.
|
||||
|
||||
`hard` cases score **2/11**: every one is a query where the operator did not reuse his own words.
|
||||
That is the normal case weeks later, and exactly what docs/design.md's "recall when relevant" promises.
|
||||
|
||||
### 4. The memory-store recall branch is dead for notes
|
||||
|
||||
`voice.go:776` only reaches `h.memStore.Search` when the notes-RAG top score is already below
|
||||
`queryMinScore`, and `bestRecall` (`cmd/mavend/recall.go:19`) then applies the **same** gate to the
|
||||
same vector. A note is indexed in both places with the same embedding, so if it failed the gate in
|
||||
`QueryNotes` it fails again here — the branch can only ever return a **fact**. Its comment calls it
|
||||
"additive"; for notes it is not.
|
||||
|
||||
**Fixed (Vikunja #373).** The memory pass now runs *first*, as one search over notes and facts with
|
||||
one gate, so whichever memory is clearly the best match answers — note or fact. The notes-only pass
|
||||
stays behind it for notes the vector index does not hold. No threshold changed, so the set of
|
||||
questions Maven answers is the same; only which memory answers them. The fixture gained two mixed
|
||||
note+fact cases (`ru-mixed-031`, `ru-mixed-032`), which is why the counts below are out of 27
|
||||
answerable cases and not 25: hash recall@1 36.0% (9/25) → 37.0% (10/27), e5 recall@1 72.0% (18/25) →
|
||||
70.4% (19/27) with answered-after-gate 68.0% → 66.7% and false recall unchanged at 1/5.
|
||||
|
||||
### 5. Ranking has no recency or type signal, and the store is not the bottleneck
|
||||
|
||||
`internal/store/notes.go:67` sorts by cosine and uses `ts` only to break an exact float tie, which
|
||||
never happens; `kind` never enters the ranking. Meanwhile `TestPersistentStoreScoresTheSame` scores
|
||||
sqlite-backed `store.MemoryStore` and `memory.InMemoryStore` identically — both full-scan cosine
|
||||
(`internal/store/memory.go:64`) at ~150µs over 42 rows against a ~59ms query embed. An ANN index is
|
||||
not the problem to solve.
|
||||
|
||||
## Re-measured after the embedder swap — 31-07-2026, later the same day
|
||||
|
||||
Changed: `models/embedder/` is now **multilingual-e5-small** (quantized, 118MB), with `query: ` in
|
||||
front of a question and `passage: ` in front of a stored note (Vikunja #371). `deploy/mavend.json`
|
||||
and `make download-embedder` now name the same file, and it is the quantized one — that is what the
|
||||
column below measures (Vikunja #372). Everything else is unchanged: same fixture, same store, same
|
||||
0.55 gate. The old column is the baseline and is left as it was.
|
||||
|
||||
| | recall+onnx, MiniLM (baseline) | recall+onnx, e5-small (new) |
|
||||
|---|---|---|
|
||||
| **recall@1** | 60.0% (15/25) | **72.0% (18/25)** |
|
||||
| recall@3 | 80.0% (20/25) | 84.0% (21/25) |
|
||||
| **answered after the 0.55 gate** | 48.0% (12/25) | **72.0% (18/25)** |
|
||||
| wrong note on top / tie on top | 10 / 0 | 7 / 0 |
|
||||
| ranked first, then silenced by the gate | 3 | 0 |
|
||||
| **false recall** | 1/5 (20%) | **5/5 (100%)** |
|
||||
| top-1 score when right, min / median | 0.559 / 0.678 | 0.791 / 0.857 |
|
||||
| top-1 when it must stay silent, median / max | 0.470 / 0.567 | 0.815 / 0.835 |
|
||||
| RU / EN / `hard` cases passed | 13/24 / 3/6 / 2/11 | 14/24 / 4/6 / 5/11 |
|
||||
| latency p50 / p95 / max | 59ms / 148ms / 194ms | 18ms / 37ms / 49ms |
|
||||
|
||||
### What moved
|
||||
|
||||
Ranking got better and got faster. Half the previously-unwinnable `hard` cases now pass (2/11 →
|
||||
5/11), the guitar note no longer beats the docker-logs note, and the gate stops silencing notes that
|
||||
already ranked first. The quantized e5 is also ~3x quicker than the fp32 MiniLM it replaces.
|
||||
|
||||
### What got worse: the gate is now a no-op
|
||||
|
||||
e5 packs every cosine into a narrow high band. Right-note scores start at 0.791; must-stay-silent
|
||||
scores reach 0.835. **The distributions still overlap, and now they overlap above the gate**, so
|
||||
0.55 admits everything and false recall goes from 1/5 to 5/5. The sweep:
|
||||
|
||||
```
|
||||
gate 0.50–0.70: answered 18/25 (72%) false recall 5/5
|
||||
gate 0.80: answered 17/25 (68%) false recall 4/5
|
||||
gate 0.90: answered 0/25 ( 0%) false recall 0/5
|
||||
```
|
||||
|
||||
There is no value that keeps real recall and rejects made-up questions — same conclusion as before,
|
||||
now with a wider band and no room at all. `query_min_score` was left at 0.55 as instructed. **The
|
||||
recommendation is to leave it there and stop tuning it**: any number under ~0.79 is a no-op and
|
||||
anything above starts cutting real recall long before it stops the false ones. The fix is a margin
|
||||
gate (`top1 − top2 > δ`), next-steps item 3, which is now the top item.
|
||||
|
||||
### The prefixes did not do the work
|
||||
|
||||
A control run with both prefixes set to the empty string scored the **same** recall@1 (72%), a
|
||||
slightly better recall@3 (88%) and the same 5/5 false recall. So on this fixture the gain comes from
|
||||
the model, not from the `query:` / `passage:` split. The prefixes are kept because they are how e5
|
||||
was trained and the split is the right shape for the read path, but they are not worth defending on
|
||||
this evidence — a bigger fixture may say otherwise.
|
||||
|
||||
### Stored vectors from the old model are now junk
|
||||
|
||||
Cosine between a MiniLM vector and an e5 vector means nothing. Every row already in `notes` and in
|
||||
the vector memory table was written by the old model, so after this deploy they will score as noise
|
||||
against a new query. A live database needs every note and fact re-embedded before recall works at
|
||||
all. Filed as its own task.
|
||||
|
||||
## Margin gate — 31-07-2026, third run
|
||||
|
||||
Next-steps item 3, done. The absolute gate is replaced by a **margin gate**: answer only when the
|
||||
top hit beats the runner-up by more than delta (`top1 − top2 > δ`). Same fixture, same e5 embedder,
|
||||
same store as the run above. `internal/memory/gate.go` holds the check; both read paths call it
|
||||
(`cmd/mavend/recall.go` and the notes-RAG branch in `voice.go`). New knob `voice.query_min_margin`
|
||||
in `deploy/mavend.json`, default 0.008.
|
||||
|
||||
### Why the absolute gate could not work, in one line of data
|
||||
|
||||
The harness now prints the margin distributions, and they barely overlap where the raw scores
|
||||
overlap completely:
|
||||
|
||||
| | top-1 score | margin (top1 − top2) |
|
||||
|---|---|---|
|
||||
| right note first (n=18) | min 0.810, median 0.862, max 0.890 | min 0.001, median 0.029, max 0.053 |
|
||||
| must stay silent (n=5) | min 0.795, median 0.815, max 0.835 | min 0.000, median 0.002, **max 0.019** |
|
||||
|
||||
Four of the five must-be-silent cases have a margin at or under 0.002 — when there is nothing to
|
||||
recall, e5 finds several notes equally close and no clear winner. That is the signal the absolute
|
||||
score throws away.
|
||||
|
||||
### The delta sweep
|
||||
|
||||
Absolute gate held at 0.55 throughout.
|
||||
|
||||
```
|
||||
delta 0.000: answered 18/25 (72%) false recall 5/5
|
||||
delta 0.002: answered 17/25 (68%) false recall 3/5
|
||||
delta 0.005: answered 17/25 (68%) false recall 2/5
|
||||
delta 0.008: answered 17/25 (68%) false recall 1/5 <- chosen
|
||||
delta 0.010: answered 15/25 (60%) false recall 1/5
|
||||
delta 0.012: answered 14/25 (56%) false recall 1/5
|
||||
delta 0.015: answered 12/25 (48%) false recall 1/5
|
||||
delta 0.020: answered 11/25 (44%) false recall 0/5
|
||||
delta 0.025: answered 9/25 (36%) false recall 0/5
|
||||
delta 0.030: answered 8/25 (32%) false recall 0/5
|
||||
delta 0.040: answered 4/25 (16%) false recall 0/5
|
||||
delta 0.050: answered 2/25 ( 8%) false recall 0/5
|
||||
delta 0.060: answered 0/25 ( 0%) false recall 0/5
|
||||
```
|
||||
|
||||
### Chosen: δ = 0.008
|
||||
|
||||
It is the best point on the frontier, not a taste call. **0.008 dominates 0.010, 0.012 and 0.015
|
||||
outright** — same 1/5 false recall, 8 to 20 points more real recall. Everything below it buys recall
|
||||
back only by admitting more false recalls (0.005 → 2/5, 0.002 → 3/5). The next real improvement is
|
||||
0.020 at 0/5 false, and it costs 24 points of recall to get there.
|
||||
|
||||
The brief's bar was "recall above 60% with false recall at 1/5 or better". 0.008 clears it with room:
|
||||
68% and 1/5.
|
||||
|
||||
### Before / after
|
||||
|
||||
| | absolute gate 0.55 (previous) | margin gate δ=0.008 |
|
||||
|---|---|---|
|
||||
| recall@1 (ranking, ungated) | 72.0% (18/25) | 72.0% (18/25) — unchanged, the gate does not rank |
|
||||
| **answered after the gate** | 72.0% (18/25) | **68.0% (17/25)** |
|
||||
| **false recall** | **5/5 (100%)** | **1/5 (20%)** |
|
||||
| fixture cases passed | 18/30 | **21/30** |
|
||||
|
||||
Four false recalls removed for one real answer. That is the trade the spec asks for — she is not a
|
||||
guesser-of-truth. The one survivor is `en-pref-025` ("should i be offered wine"), which recalls a
|
||||
filler note at 0.796 with a 0.019 margin: the widest silent-case margin in the fixture, and it sits
|
||||
inside the real-recall range, so no delta removes it without taking real answers with it.
|
||||
|
||||
### Does the absolute cutoff still earn its keep? Marginally — kept
|
||||
|
||||
On this fixture with e5 it is a **no-op**: the lowest right-note score is 0.791, so 0.55 rejects
|
||||
nothing the margin does not already reject. It is kept for two reasons, neither glamorous. It still
|
||||
does real work for the hash embedder (its own sweep shows answers dropping from 16% to 0% between
|
||||
0.30 and 0.50), and it is the only thing standing between the user and a reply built from a store
|
||||
where everything is far away but one row happens to be a little less far — a near-empty database, or
|
||||
the stale-vector case below. Cheap insurance, no measured cost. If a later embedder makes it bite,
|
||||
the sweep is one command.
|
||||
|
||||
### Caveat on the numbers
|
||||
|
||||
Five must-be-silent cases is a thin basis for a 4-point decision. 1/5 and 2/5 differ by one case.
|
||||
The shape of the frontier is trustworthy — margins separate, absolute scores do not — but δ=0.008
|
||||
itself should be re-read off a bigger fixture (next-steps item 6) before anyone defends the third
|
||||
decimal.
|
||||
|
||||
## Next steps — ordered by value-to-risk; nothing here is a decision
|
||||
|
||||
1. **Swap the embedder to `multilingual-e5-small` with `query:`/`passage:` prefixes.** One config
|
||||
change plus a prefix in `onnxembedder.go`, re-measurable in one command.
|
||||
2. **Re-run `make eval-recall`, then set the gate from the sweep** — not before. Any
|
||||
`query_min_score` picked against today's embedder describes a model on its way out.
|
||||
3. ~~**Replace the absolute-score gate with a margin gate**~~ — done, see the section above.
|
||||
δ=0.008, false recall 5/5 → 1/5.
|
||||
4. **Delete or repair the dead `memStore` branch** at `voice.go:776` — search before the gate,
|
||||
gate it separately, or restrict it to facts and say so.
|
||||
5. **Add a mild time decay to ranking** — the newest statement of a preference is the true one.
|
||||
6. **Grow the fixture from real misses.** 30 cases can rank two embedders, not trust 4 points.
|
||||
7. **Re-measure end to end.** Recall is gated twice — the utterance must first route to `query`,
|
||||
which the routing eval puts at ~50%. The product is ~24%, and that is what he experiences.
|
||||
@@ -0,0 +1,300 @@
|
||||
# Routing evaluation — 31-07-2026
|
||||
|
||||
Settles Vikunja **#319** ("measure classifier vs LLM router before flipping"). Everything
|
||||
below is measured against one held-out fixture, not argued from the code.
|
||||
|
||||
- Fixture + scorer: `internal/router/eval/` (`ru_routing_v1.json`, 76 cases; `eval.go`)
|
||||
- Reproduce: `make eval-router` (classifier baselines) and
|
||||
`MAVEN_LLM_URL=http://127.0.0.1:18099 make eval-router` (adds the LLM configurations)
|
||||
- Commits: `c7c4422` (fixture), `d34fdf4` (ONNX baseline), `46259b4` (LLM baseline)
|
||||
|
||||
## Why a new fixture
|
||||
|
||||
`cmd/mavend/eval_scenarios_test.go` could not answer #319: it asserts daemon-side *safety*
|
||||
invariants over already-normalized decisions, so it never exercises routing. And the only
|
||||
utterance corpus that existed — `models/seeds/*.txt` — is the classifier's own training set.
|
||||
Scoring a nearest-centroid classifier there measures memorisation of frozen centroids, which
|
||||
is exactly the illusion behind `voice.go:211`'s "the classifier handles routing reliably".
|
||||
|
||||
`TestFixtureIsHeldOut` fails the build if any fixture utterance appears verbatim in the seed
|
||||
corpus. The fixture is a **contract, not a snapshot**: cases the cascade fails today stay in
|
||||
the file and fail loudly.
|
||||
|
||||
## Results
|
||||
|
||||
| | classifier+hash | classifier+onnx | llm-only (0.8B) | cascade+llm (0.8B) |
|
||||
|---|---|---|---|---|
|
||||
| **intent-only accuracy** | 17.1% | 36.8% | 48.7% | **50.0%** |
|
||||
| full accuracy (intent+slots+gate) | 17.1% | 36.8% | 23.7% | 32.9% |
|
||||
| RU | 10/61 | 25/61 | 13/61 | 18/61 |
|
||||
| EN | 3/15 | 3/15 | 5/15 | 7/15 |
|
||||
| `hard` tag | 0/11 | 4/11 | — | — |
|
||||
| false clarify (asked, shouldn't) | 63 | 21 | 0 | 2 |
|
||||
| **missed clarify (guessed, shouldn't)** | **0 / 6** | **5 / 6** | **6 / 6** | **6 / 6** |
|
||||
| route errors | 0 | 0 | 2 | 0 |
|
||||
| **p50 / p95 / max latency** | 9µs / 14µs | **31ms / 71ms** | 850ms / 1.56s / 3.1s | **825ms / 1.20s / 3.0s** |
|
||||
|
||||
`classifier+hash` is the CI ratchet (deterministic, no model files). `classifier+onnx` is what
|
||||
homesrv runs today. `cascade+llm` is the wiring #320 proposes: stage-0 grammar → resident
|
||||
model → classifier as failure floor.
|
||||
|
||||
Never compare a hash-embedder run to an ONNX one.
|
||||
|
||||
## Re-measured after the prompt fix
|
||||
|
||||
The table above is the **baseline at commit `46259b4`**, kept as-is. The prompt fix (query
|
||||
tested before fact, plus `repeat_penalty` and a bounded grammar string) was then measured on
|
||||
an otherwise idle box — no other eval sharing llama-server, so these latencies are real
|
||||
rather than contention.
|
||||
|
||||
| | llm-only (0.8B) | cascade+llm (0.8B) | llm-only, thinking off |
|
||||
|---|---|---|---|
|
||||
| **intent-only accuracy** | 48.7% → **61.8%** | 50.0% → **63.2%** | **67.1%** |
|
||||
| full accuracy (intent+slots+gate) | 23.7% → **38.2%** | 32.9% → **47.4%** | **42.1%** |
|
||||
| route errors | 2 → **0** | 0 → 0 | **0** |
|
||||
| p50 / p95 latency | **1.08s / 1.55s** | **1.04s / 1.53s** | **0.93s / 1.41s** |
|
||||
|
||||
Three things this run settles:
|
||||
|
||||
1. **The prompt fix holds.** An earlier contended run reported 60.5% / 36.8% for llm-only;
|
||||
the quiet run gives 61.8% / 38.2%. Close enough to call the gain real, and the earlier
|
||||
run's 4-5s latency figures were contention, not the model.
|
||||
2. **`query→fact` fell from ×15 to ×7**, and both unparseable replies are gone. Zero route
|
||||
errors in every LLM configuration.
|
||||
3. **`note→fact ×4` is real, not noise.** It shows up in the quiet run too. The agent that
|
||||
wrote the prompt fix suspected its own change might have caused it by pulling assertive
|
||||
`запиши что…` phrasings toward fact, and that suspicion stands — all five `ru-note-*`
|
||||
cases now land on fact. Tracked as Vikunja #375.
|
||||
|
||||
The `thinking off` column above read as the best configuration measured so far (Vikunja #376).
|
||||
**It was wrong** — see the controlled re-run below. Ignore that column.
|
||||
|
||||
Still `6 / 6` missed clarify — the router has no way to say "I don't know" (Vikunja #359).
|
||||
That is unchanged by anything here.
|
||||
|
||||
## Thinking off — 31-07-2026, controlled re-run (Vikunja #376)
|
||||
|
||||
The "thinking off wins by 6 points" observation above **does not hold**. It was a measurement
|
||||
artefact, and the earlier table's `thinking off` column should be ignored.
|
||||
|
||||
The thinking-off variant was scored by a hand-rolled HTTP client living in the test file
|
||||
instead of `llm.Client`. That copy did not send `repeat_penalty`, which the real router does
|
||||
send (`routeRepeatPenalty = 1.15`). So the two columns differed on two axes at once, and the
|
||||
one that mattered was the penalty, not the thinking mode.
|
||||
|
||||
Re-measured with everything else held equal — same fixture, same prompt, same grammar, same
|
||||
sampling, same idle box, the three configurations run back to back and never concurrently:
|
||||
|
||||
| | llm-only, thinking on | llm-only, thinking off | cascade+llm |
|
||||
|---|---|---|---|
|
||||
| intent-only accuracy | 59.2% (45/76) | 59.2% (45/76) | 61.8% (47/76) |
|
||||
| full accuracy (intent+slots+gate) | 38.2% (29/76) | 38.2% (29/76) | 57.9% (44/76) |
|
||||
| route errors | 3 | 3 | 0 |
|
||||
| grammar violations | 3 (all 3 route errors) | 3 (same 3 cases) | 0 |
|
||||
| missed clarify | 5 / 6 | 5 / 6 | 5 / 6 |
|
||||
| p50 latency | 836ms | 920ms | 810ms |
|
||||
| p95 latency | 1.41s | 2.00s | 1.31s |
|
||||
|
||||
Thinking off is not just a tie on the headline numbers — it is identical case for case, with
|
||||
the same confusion matrix and the same three unparseable replies. The latency difference is
|
||||
run-to-run noise on one box, and it points the wrong way here.
|
||||
|
||||
The reason is simpler than any accuracy argument: **this llama-server build ignores the
|
||||
request-level thinking switch for this model.** Probed directly against the running server
|
||||
with `chat_template_kwargs.enable_thinking = false`, `chat_template_kwargs.thinking = false`
|
||||
and top-level `reasoning_budget = 0` — all three return a byte-identical answer with the
|
||||
thinking trace still in `reasoning_content`, and the server reports the prompt prefix as
|
||||
cached, meaning the rendered template did not change. There was never anything being turned
|
||||
off, which is also why the numbers match exactly.
|
||||
|
||||
Nothing was defaulted. `internal/llm` still has no `chat_template_kwargs` field, `VoiceConfig`
|
||||
has no thinking flag, and `deploy/mavend.json` is unchanged. The misleading third
|
||||
configuration is removed from `internal/router/eval` so the table it produced cannot be quoted
|
||||
again.
|
||||
|
||||
Two caveats worth saying out loud:
|
||||
|
||||
- **The fixture is 76 cases.** A 6-point difference on 76 cases is roughly 4-5 cases and would
|
||||
not have been worth trusting even if it had reproduced. This one was exactly 0 cases, which
|
||||
is a much easier call.
|
||||
- **This is one server build and one checkpoint** (`b9351`, Qwen3.5-0.8B Q4_K_M). If the
|
||||
#122 checkpoint or a newer llama.cpp does honour the switch, the question reopens — but it
|
||||
reopens as an unmeasured question, not as a 6-point win.
|
||||
|
||||
Phrasing was **not** measured. Whether thinking helps there is still open, and now also blocked
|
||||
on the same "can we even turn it off" question.
|
||||
|
||||
## Clock and calendar rule — 31-07-2026 (Vikunja #374)
|
||||
|
||||
`routeSystem` never said whether "который час" or "какое число завтра" are `system` or
|
||||
`query`, and `system→query ×4` showed up in every run. The rule added says: the clock and the
|
||||
calendar date themselves are `system`; what is *written in* the calendar or in memory
|
||||
("что у меня завтра", "какие есть напоминания") stays `query`; and a time named inside a
|
||||
request ("напомни завтра…") is just a detail of the request, not a reason for `system`.
|
||||
|
||||
That split is not a preference. In `cmd/mavend/voice.go` only `replySystem` owns the clock and
|
||||
the date formatter, so a clock question routed to `query` falls into the embedder + note RAG
|
||||
and answers "не знаю". The agenda, on the other hand, is answered by `ParseCalendarDate` +
|
||||
`CalendarEvents` *inside* the `query` branch, so that side has to stay `query`. The rule sits
|
||||
above the question test because every one of these utterances carries a question word and a
|
||||
later rule would never be reached.
|
||||
|
||||
The fixture is now 77 cases: one calendar-agenda case was added
|
||||
(`ru-query-019` "что у меня стоит в календаре на послезавтра", intent `query`) specifically so
|
||||
an over-broad system rule cannot pass unnoticed. The clock/date cases (`ru-sys-001/002/005`,
|
||||
`en-sys-001`) already existed.
|
||||
|
||||
Three runs, same box, back to back, never concurrently:
|
||||
|
||||
| | baseline | first rule (too broad) | rule as committed |
|
||||
|---|---|---|---|
|
||||
| llm-only intent-only | 59.2% (45/76) | 54.5% (42/77) | 59.7% (46/77) |
|
||||
| llm-only full | 38.2% | 35.1% | 39.0% |
|
||||
| llm-only route errors | 3 | 4 | 5 |
|
||||
| llm-only p50 | 1.09s | 0.91s | 0.93s |
|
||||
| cascade+llm intent-only | 61.8% (47/76) | 58.4% | 62.3% (48/77) |
|
||||
| cascade+llm full | 57.9% | 54.5% | 59.7% |
|
||||
| cascade+llm route errors | 0 | 0 | 0 |
|
||||
| cascade+llm p50 | 0.91s | 0.80s | 1.04s |
|
||||
|
||||
**The targeted bug is fixed and the headline number did not move.** `system→query ×4` is gone
|
||||
in both LLM configurations — the `time` and `date` tags go from 0/2 and 0/2 to 2/2 and 2/2 —
|
||||
but the model then over-applies the rule, and `query→system ×5` plus `reminder→system ×2`
|
||||
appear where they did not exist before. Net accuracy is a wash, inside the noise of a 77-case
|
||||
fixture.
|
||||
|
||||
The first attempt is shown because it is the honest history: it said "спрашивает время, дату
|
||||
или день недели → system" with no scope, which swept up reminders, and it cost 3-5 points. It
|
||||
was tightened once, on the reasoning that a rule capturing "напомни завтра в 7" is simply
|
||||
wrong, and not tuned further. The remaining `query/reminder → system` over-trigger is a new,
|
||||
separate weakness of the sub-1B model and deserves its own task rather than more prompt
|
||||
kneading against a held-out fixture.
|
||||
|
||||
The rule is kept. It is correct about what the daemon can answer, and the failure it replaces
|
||||
was silent ("не знаю" to "который час") while the one it introduces is loud.
|
||||
|
||||
## Findings
|
||||
|
||||
### 1. The resident model does route better — 50.0% vs 36.8%
|
||||
|
||||
docs/rearchitecture.md's premise holds; `voice.go:211`'s comment does not. **But the classifier is only
|
||||
~37% correct on held-out utterances, and the model only ~50%.** Neither is "reliable". The
|
||||
gap between them is real but both are far from a system you would describe as working.
|
||||
|
||||
### 2. It costs 27× the latency
|
||||
|
||||
p50 825ms vs 31ms, p95 1.2s, max 3.0s — on the same llama-server the phraser needs, before
|
||||
any phrasing happens. On the CPU/iGPU deploy target this is a trade, not a free win. The
|
||||
review's second-opinion caution was justified.
|
||||
|
||||
### 3. `query→fact ×15` is the dominant LLM failure — and it is a prompt bug
|
||||
|
||||
Four times the classifier's `×4` on the same axis. `routeSystem`'s decision order in
|
||||
`internal/router/llmrouter.go` reads:
|
||||
|
||||
```
|
||||
3. Сообщает или обновляет текущее состояние/событие → fact
|
||||
4. Хочет получить информацию → query
|
||||
```
|
||||
|
||||
Any utterance naming a fact key matches rule 3 first, so a *question about* past state
|
||||
("сколько воды я выпил с утра", "сколько раз я ел вчера") is classified as an *assertion of*
|
||||
that state — and a query becomes a confident wrong write. Reordering query above fact, or
|
||||
adding an explicit interrogative test, is the cheapest accuracy win available and needs no
|
||||
model change.
|
||||
|
||||
### 4. Neither path can refuse — the refusal lane is currently fiction
|
||||
|
||||
| | missed clarify | why |
|
||||
|---|---|---|
|
||||
| classifier+hash | 0 / 6 | cosine never clears 0.55 — refuses by accident |
|
||||
| classifier+onnx | 5 / 6 | better embeddings raise cosine everywhere; the gate stops separating |
|
||||
| LLM (any) | 6 / 6 | `llmrouter.go` hardcodes `Confidence: 1.0`, so stage 3 can never fire |
|
||||
|
||||
The deployed config confidently routes `сделай это` → **act** at 0.847, `ну это` → chat at
|
||||
0.808, `бэкап` → chat at 0.755, `потом` → system at 0.739. `сделай это` → act with unresolved
|
||||
anaphora is the destructive direction; the daemon's confirm gate is the only thing left.
|
||||
|
||||
This is the finding that should block #320. Flipping to the LLM router as-is does not improve
|
||||
the refusal lane — it removes it. Tracked as **#359**.
|
||||
|
||||
### 5. The 50.0% → 32.9% gap is entirely slots
|
||||
|
||||
The LLM path fills neither `Fn` nor `Time`: it returns `Slots.Text` for acts (the verb string,
|
||||
not an allowlist match), and `Extractor.Extract` never runs on an LLM decision at all. Any
|
||||
flip needs the extractor wired onto the LLM branch or every act and reminder arrives without
|
||||
its arguments.
|
||||
|
||||
### 6. The 2 route errors are a missing `RepeatPenalty`, not a grammar flaw
|
||||
|
||||
Both failures (`ru-act-006` "закрой жалюзи", `ru-chat-003` "расскажи анекдот про
|
||||
программистов") are the sub-1B repetition loop *inside* the grammar's `text` field:
|
||||
|
||||
> "Закрывание жалюзи — это действие, которое нужно выполнить. Если это не действие, то это
|
||||
> сообщение пользователя. Если это не действие, то это сообщение пользователя. …"
|
||||
|
||||
It runs to `MaxTokens: 128`, truncates the JSON mid-string, and `parseActions` fails →
|
||||
fallback to the classifier. `llm.Req` already has a `RepeatPenalty` field added for exactly
|
||||
this ("curbs the sub-1B 'тоже тоже тоже' loop") and `LLMRouter.Route` does not set it. Two
|
||||
lines.
|
||||
|
||||
Note the grammar's `string ::= "\"" ([^"\\] | "\\" .)* "\""` is unbounded, so nothing stops a
|
||||
1000-character `text`. Worth a length bound as well.
|
||||
|
||||
### 7. Two hypotheses tested and closed
|
||||
|
||||
- **Thinking mode is a non-issue.** Confirmed twice now, the second time properly — see the
|
||||
controlled re-run section. Grammar-constrained JSON lands in `reasoning_content` with
|
||||
`content` empty and `llm.Client`'s fallback handles it; the request-level switch does
|
||||
nothing on this build. `internal/llm` deliberately does **not** grow a
|
||||
`chat_template_kwargs` field.
|
||||
- **Runaway array repetition does not reproduce.** An isolated smoke test with a stripped
|
||||
grammar emitted `{"intent":"reminder"}` until `MaxTokens`; under the real `routeSystem`
|
||||
prompt the few-shot examples anchor it to one object. 2 errors in 76, not 76.
|
||||
|
||||
### 8. Incidental
|
||||
|
||||
- `ReminderGrammar` deliberately skips the extractor at stage 0; the daemon's `applyAction`
|
||||
parses the time downstream. The scorer counts those as `SlotsDeferred` rather than misses.
|
||||
- A local llama-server must bypass `http_proxy` — this box proxies loopback through a SOCKS
|
||||
bridge that answers 503. `noProxyLoopback` in the test handles it.
|
||||
- The onnxruntime `.so` was already vendored at `deps/onnxruntime-linux-x64-1.26.0`.
|
||||
|
||||
## Next steps
|
||||
|
||||
Ordered by ratio of value to risk. Nothing here is a decision — #320 stays open.
|
||||
|
||||
1. **Fix `routeSystem`'s decision order** (query above fact, or an explicit interrogative
|
||||
test). Largest single accuracy move, no model change, re-measurable in one command.
|
||||
Expected: most of `query→fact ×15`.
|
||||
2. **Set `RepeatPenalty` in `LLMRouter.Route`** and bound the grammar's `string` length.
|
||||
Removes both route errors.
|
||||
3. **Give the router a refusal signal — #359.** Blocks #320.
|
||||
- Classifier: the absolute-cosine gate does not survive a better embedder. A **margin**
|
||||
gate (`top1 − top2 > δ`) is the likely fix — ambiguous utterances should show flat
|
||||
distributions, which absolute cosine cannot see.
|
||||
- LLM: `Confidence: 1.0` must go. Either add an `unclear` intent to the grammar enum, or
|
||||
read logprobs, or gate on the classifier's margin *behind* the LLM decision.
|
||||
- Bar: `MissedClarify ≤ 1` without regressing full accuracy below 28/76.
|
||||
4. **Wire `Extractor.Extract` onto the LLM branch** so acts get `Fn` and reminders get
|
||||
`Time`. Closes the 50.0% → 32.9% slot gap.
|
||||
5. **Re-measure, then decide #320.** At p50 825ms a wholesale swap is probably the wrong
|
||||
shape; the honest candidate is LLM-for-queries with the classifier keeping the fast
|
||||
deterministic paths (stage-0 grammar hits, `system`, exact acts). That hypothesis is
|
||||
testable against this fixture by scoring a per-intent split.
|
||||
6. **Grow the fixture** as failures get understood. 76 cases with ≥5 per intent is enough to
|
||||
rank paths, not enough to trust a 2-point difference. Add cases from real misroutes
|
||||
(`CorrectMisroute` is already the append-only hook).
|
||||
7. **Second checkpoint when #122 lands.** The CPT'd Qwen3-1.7B is the target resident model;
|
||||
the same three configurations should be re-scored against it before it deploys. 0.8B's
|
||||
50.0% is the floor that checkpoint has to beat, and its latency is the number that decides
|
||||
whether the target is affordable at all.
|
||||
|
||||
## Open question worth naming
|
||||
|
||||
Both paths are under 50%. That is low enough that the interesting question may not be
|
||||
"classifier or model" but whether one-shot classification of a bare utterance is the right
|
||||
frame at all — `сделай это`, `потом`, `бэкап` are unanswerable without dialogue context, and
|
||||
`internal/router` currently sees none (`AnaphoraResolver` exists in `slots.go` but the
|
||||
cascade never calls it). A router that could ask one clarifying question and re-route on the
|
||||
answer would beat both numbers here without a better model.
|
||||
@@ -0,0 +1,150 @@
|
||||
# Conversational phrasing eval — 31-07-2026
|
||||
|
||||
Every score measured tonight, on the three paths the nudge eval never touched:
|
||||
chat, query-with-notes, and general knowledge.
|
||||
|
||||
**Short version: the plumbing got fixed and the score barely moved.** Grammar and
|
||||
Russian prompts together took the composite from ~9 to ~14 of 27. Everything
|
||||
still failing is the model not knowing things or not holding a constraint, and
|
||||
prompting is out of levers. Settles the measurement half of Vikunja #395 / #398 /
|
||||
#400.
|
||||
|
||||
## How to reproduce
|
||||
|
||||
```sh
|
||||
# llama-server: -c 4096 -ngl 99 -t 6, model /mnt/hdd1/llms/qwen3.5/Qwen3.5-0.8B.Q4_K_M.gguf
|
||||
MAVEN_LLM_URL=http://127.0.0.1:18099 no_proxy=127.0.0.1,localhost \
|
||||
deps/go/go/bin/go test -count=1 -timeout 40m \
|
||||
-run TestLLMTalkBaseline ./internal/phraser/eval/ -v
|
||||
```
|
||||
|
||||
Three runs per configuration, always. The fixture is 27 cases, so one reply
|
||||
changing moves the composite by 3.7 points — a single run cannot tell a real
|
||||
change from sampling noise. This was learned the expensive way: an earlier claim
|
||||
that "one nudge case fails every run" turned out to be three different cases
|
||||
across three runs.
|
||||
|
||||
**Run the box otherwise idle.** See the contamination note at the bottom.
|
||||
|
||||
## Composite, per configuration
|
||||
|
||||
| config | overall /27 | chat /9 | query /9 | knowledge /9 | canned fallbacks |
|
||||
|---|---|---|---|---|---|
|
||||
| baseline, no grammar | 7, 12, 7 | 1, 1, 0 | 2, 4, 2 | 4, 7, 5 | 0, 0, 0 |
|
||||
| + GBNF grammar (#398) | 14, 15, 8 | 1, 3, 0 | 5, 6, 3 | 8, 6, 5 | 0, 0, 0 |
|
||||
| + Russian prompts (#400) | 11, 17, 15 | 1, 5, 3 | 5, 6, 8 | 5, 6, 4 | 0, 0, 0 |
|
||||
| + truncation fix, 1000ch/768tok | 12, 13, 10 | 2, 2, 1 | 7, 7, 5 | 3, 4, 4 | 3, 3, 6 |
|
||||
| + rebalanced, 600ch/1024tok | **void — contaminated** | | | | |
|
||||
|
||||
"Canned fallbacks" counts replies that came back as the hardcoded `"не знаю."`
|
||||
or `"поговорили."`. It is not a check, it is a health signal: those strings mean
|
||||
the phraser gave up, and the eval scores them as ordinary bad replies.
|
||||
|
||||
## Per-check
|
||||
|
||||
| check | no grammar | + grammar | + RU prompts | + truncation fix |
|
||||
|---|---|---|---|---|
|
||||
| nonempty | 27, 27, 27 | 27, 27, 27 | 27, 27, 27 | 27, 27, 27 |
|
||||
| ellipsis | 20, 19, 23 | 27, 27, 27 | 27, 27, 27 | 27, 27, 27 |
|
||||
| lang | 13, 16, 15 | 23, 26, 26 | 25, 26, 25 | 26, 27, 27 |
|
||||
| feminine | — | — | 25, 24, 26 | 25, 25, 27 |
|
||||
| address | — | — | 21, 22, 22 | 22, 21, 22 |
|
||||
| ontopic | — | — | 17, 24, 18 | 17, 19, 14 |
|
||||
|
||||
`nonempty` reading 27/27 everywhere is not good news — it was a broken check.
|
||||
It tested for a non-blank string, so replies of literally `{` and `"15-16"`
|
||||
passed it. Fixed on `overnight/fix-truncation`; it needs a letter now.
|
||||
|
||||
## What each change actually bought
|
||||
|
||||
**GBNF grammar (#398) — the biggest single win.** Qwen3.5-0.8B writes
|
||||
`Thinking Process:` as plain text with no tags, `stripThink` only handles
|
||||
`</think>`, so the JSON never closed and the plain-text fallback shipped the
|
||||
literal reasoning. `ellipsis` went 20→27 and `lang` 13→26. The router had been
|
||||
using a grammar for ages; the phraser asking nicely in the prompt was the
|
||||
oversight.
|
||||
|
||||
**Russian prompts (#400) — modest, plus a large latency win.** Chat 1.3→3.0
|
||||
average, query 4.7→6.3, knowledge 6.3→5.0. All inside the run-to-run spread, so
|
||||
"probably better on the paths it targeted, not provable in three runs". p50
|
||||
latency dropped from ~11.5s to ~2.3s and that part is consistent across all
|
||||
three runs — shorter prompts, and she stopped emitting English reasoning first.
|
||||
|
||||
**Truncation fix — necessary, and did not help the score.** Two real bugs
|
||||
(replies of `{`, and a `nonempty` check that passed them), both fixed, and the
|
||||
composite went nowhere. A complete rambling wrong answer fails the same checks a
|
||||
truncated one did. Worth doing anyway: the daemon was shipping `{` to a
|
||||
text-to-speech voice.
|
||||
|
||||
## The truncation bug, since the cause was counter-intuitive
|
||||
|
||||
The grammar's `string ::= ... {0,400}` rule was the cause, not the token cap.
|
||||
Measured against Qwen3.5-0.8B at three caps — 256, 768 and 2048 — the reply came
|
||||
back **exactly 400 characters every time, cut mid-word** (`"Нужно записать и,"`).
|
||||
|
||||
Then I raised the bound to 1000 while the cap was 768 tokens and made it worse:
|
||||
Russian runs ~1.5 characters per token here, so generation died on the *token*
|
||||
cap instead, mid-object, and the new guard correctly refused it and shipped
|
||||
`"не знаю."` — 3, 3 and 6 fallbacks per run, from zero. **The two limits have to
|
||||
agree.** 600 characters needs ~400 tokens; the cap is 1024.
|
||||
|
||||
## Where the remaining failures live
|
||||
|
||||
`address` is stuck at 21-22 of 27 and `ontopic` at 14-19. Both resist prompting.
|
||||
|
||||
**The prompt now explicitly forbids exactly what she does.** It says never "вы",
|
||||
use the singular — and she writes `вашей`, `подождите`, `делаете`, `хотите`,
|
||||
`напишите`. Telling a 0.8B "never do X" does not work. Same for
|
||||
`feminine`: `я готов`, `я понял`, `я нашел`, `я заметил`, `я сказал`.
|
||||
|
||||
**Some of `ontopic` is the fixture, not the model.** `chat-how-are-you` got
|
||||
`"Привет! Я здесь, чтобы поговорить. Как дела сегодня?"` — a fine reply that
|
||||
fails because `want_any` is `[норм, хорош, порядк, тут, работ]`. It fails in
|
||||
every run, so it inflates the count. The `ontopic` column currently measures the
|
||||
fixture as much as the model. Not fixed yet, deliberately: changing it would
|
||||
break comparability with the runs above.
|
||||
|
||||
**Two replies worth reading, because they are not fixable by prompting:**
|
||||
|
||||
- Thunder and lightning: *"Скорость молнии — 8-10 тысяч километров в секунду, но
|
||||
звук — 300 метров в секунду, что делает молнию громче."* Confidently wrong,
|
||||
and it concludes lightning is *louder* rather than sound being *slower*.
|
||||
- "расскажи обо мне": *"Ты — прекрасное существо, с душой и вниманием… Спасибо за
|
||||
твою улыбку… О тебе — заповедь любви."* Sycophantic filler, zero information,
|
||||
and precisely the "not a relationship" non-goal.
|
||||
- Boiling an egg: `"15-16"` one run, `"1"` another. No unit, wrong number.
|
||||
|
||||
The first argues for reading instead of recalling (#403 — Kiwix retrieval scores
|
||||
8/8 on the same questions given English keywords). The second and third argue
|
||||
for templates on the paths where correctness matters (#392).
|
||||
|
||||
## Contamination note — how the last row got voided
|
||||
|
||||
I started the query-rewrite agent against the same llama-server the sweep was
|
||||
using, and assumed contention would only affect latency. It did not. The
|
||||
knowledge path collapsed to 0 of 9 with eight canned `"не знаю."` replies, p95
|
||||
tripled to 23.7s, and **the report still said "0 errors"**.
|
||||
|
||||
That is Vikunja #397, and it is worse than filed: a merely *busy* server
|
||||
produces a clean-looking report with a third of the fixture silently answering
|
||||
`"не знаю."`. `PhraseChat` and `PhraseQuery` swallow every failure and return a
|
||||
hardcoded string, so infrastructure trouble is indistinguishable from bad
|
||||
phrasing in the score. The talk test guards the *start* and *end* of a run with
|
||||
a model check, which catches a dead server but not a loaded one.
|
||||
|
||||
**Until #397 is fixed, treat any run made on a busy box as void.**
|
||||
|
||||
## Next
|
||||
|
||||
- Re-run 600ch/1024tok clean, to fill the void row.
|
||||
- Score `Qwen3.5-2B-UD-Q4_K_XL` (already at `/mnt/hdd1/llms/qwen3.5/`, never
|
||||
measured) on this fixture and the router fixture. Not the 4B — too big for
|
||||
this box, owner's call.
|
||||
- Newer sub-500M candidates (LFM2.5 200M/300M) are worth a run for routing.
|
||||
Note `docs/evals/2026-07-31-model-bakeoff.md` found LFM2.5-**1.2B** worse than
|
||||
Qwen3.5-0.8B at Russian routing and 2.4× slower — but those are a different,
|
||||
older generation, so that result does not predict the small ones.
|
||||
- Fix `chat-how-are-you`'s `want_any`, and re-baseline once, so `ontopic`
|
||||
measures the model.
|
||||
- #397 first if anything, since it decides whether any of the above is
|
||||
trustworthy.
|
||||
@@ -0,0 +1,187 @@
|
||||
# Start Commands
|
||||
|
||||
*Last verified: 2026-08-02 @ 7079a24. Living doc: correct it in place, do not append.*
|
||||
|
||||
All commands assume `ROOT=/home/kami/apps/Maven` and the local Go toolchain at `$ROOT/deps/go/go/bin/go`.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
```bash
|
||||
export ROOT=/home/kami/apps/Maven
|
||||
export CGO_CFLAGS="-I$ROOT/deps/include -I$ROOT/deps/whisper.cpp/ggml/include"
|
||||
export CGO_LDFLAGS="-L$ROOT/deps/lib -Wl,-rpath,$ROOT/deps/lib"
|
||||
export LD_LIBRARY_PATH="$ROOT/deps/lib"
|
||||
export PATH="$ROOT/deps/go/go/bin:$PATH"
|
||||
```
|
||||
|
||||
## Build everything
|
||||
|
||||
```bash
|
||||
cd "$ROOT"
|
||||
go build ./cmd/mavend/
|
||||
go build ./cmd/mavsttd/ # needs CGO (whisper.cpp)
|
||||
go build ./cmd/mavttsd/ # pure Go
|
||||
```
|
||||
|
||||
## mavend — daemon (core)
|
||||
|
||||
```bash
|
||||
cd "$ROOT"
|
||||
./mavend -config mavend.json
|
||||
```
|
||||
|
||||
Config path: `~/.config/maven/mavend.json`. Full example with all options.
|
||||
|
||||
> The `phraser.model_path` below is an example — point it at whatever GGUF you
|
||||
> have locally. The deployed value lives in `deploy/mavend.json`, currently
|
||||
> `Qwen3.5-0.8B.Q4_K_M.gguf`; the target is the CPT'd Qwen3-1.7B (#122).
|
||||
|
||||
```json
|
||||
{
|
||||
"db_path": "/home/kami/.local/share/maven/maven.db",
|
||||
"socket_path": "/run/user/1000/maven/mavend.sock",
|
||||
"tick_interval": "60s",
|
||||
"repeat_interval": "5m",
|
||||
"ntfy": {
|
||||
"base_url": "https://ntfy.kvmx.ru",
|
||||
"topic": "maven"
|
||||
},
|
||||
"phraser": {
|
||||
"model_path": "/mnt/hdd1/llms/Qwen3-Maven-1.7B-Q8_0.gguf",
|
||||
"bin_path": "/usr/local/bin/llama-server",
|
||||
"n_gpu_layers": -1
|
||||
},
|
||||
"voice": {
|
||||
"enabled": true,
|
||||
"bind": "127.0.0.1:9100",
|
||||
"lang": "ru",
|
||||
"embedder": {
|
||||
"model_path": "models/embedder/model.onnx",
|
||||
"tokenizer_path": "models/embedder/tokenizer.json",
|
||||
"lib_path": "deps/onnxruntime-linux-x64-1.17.1/lib/libonnxruntime.so.1.17.1"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Omit the `embedder` block entirely to use the deterministic HashEmbedder floor (no ML, no ONNX runtime dependency). Useful for testing or low-resource setups.
|
||||
|
||||
## mavsttd — STT worker (optional, remote whisper.cpp)
|
||||
|
||||
Requires `LD_LIBRARY_PATH` to include deps/lib (for libwhisper.so, libggml-vulkan.so).
|
||||
|
||||
```bash
|
||||
cd "$ROOT"
|
||||
export LD_LIBRARY_PATH="$ROOT/deps/lib"
|
||||
./mavsttd -socket /run/user/$UID/maven/stt.sock -model models/stt/ggml-small.bin
|
||||
```
|
||||
|
||||
Without `-model` it runs as a stub (deterministic, no ML).
|
||||
|
||||
## mavttsd — TTS worker (optional, remote Piper)
|
||||
|
||||
Requires `LD_LIBRARY_PATH` to include deps/piper (for Piper's espeak-ng).
|
||||
|
||||
```bash
|
||||
cd "$ROOT"
|
||||
export LD_LIBRARY_PATH="$ROOT/deps/piper"
|
||||
./mavttsd -socket /run/user/$UID/maven/tts.sock -piper deps/piper/piper -model models/tts/ru_RU-irina-medium.onnx -espeak_data deps/piper/espeak-ng-data
|
||||
```
|
||||
|
||||
Without `-piper` it runs as a stub.
|
||||
|
||||
## mavweb — PWA voice bridge (WebSocket ↔ TCP)
|
||||
|
||||
No CGo, no deps; builds with stock Go.
|
||||
|
||||
```bash
|
||||
cd "$ROOT"
|
||||
go build ./cmd/mavweb/
|
||||
./mavweb -addr :9200 -voice 127.0.0.1:9100
|
||||
```
|
||||
|
||||
To also receive proactive nudges in-app, pass the ntfy WebSocket subscribe URL
|
||||
(the PWA connects to it directly; the auth token stays server-side config):
|
||||
|
||||
```bash
|
||||
./mavweb -addr :9200 -voice 127.0.0.1:9100 \
|
||||
-ntfy 'wss://ntfy.kvmx.ru/maven/ws?auth=<base64-token>'
|
||||
```
|
||||
|
||||
`<base64-token>` is a *read*-capable ntfy access token, base64url-encoded
|
||||
(ntfy's browser-WS auth: `Bearer tk_...` can't set a header, so ntfy takes it as
|
||||
the `?auth=` query param). Without `-ntfy`, the PWA stays voice-only.
|
||||
|
||||
### presence-signal ingest (`-core`)
|
||||
|
||||
Pass mavend's IPC socket so mavweb can feed presence via `/api/signal`:
|
||||
|
||||
```bash
|
||||
./mavweb -addr :9200 -voice 127.0.0.1:9100 \
|
||||
-core /run/user/1000/maven/mavend.sock
|
||||
```
|
||||
|
||||
- **page_heartbeat** (weak, τ=4min) — the PWA auto-pings every 30s. Nothing to do.
|
||||
- **desk_active** (strongest, τ=8min) — a *workstation* signal (hyprland), so it
|
||||
can't be a homesrv module. Run `scripts/desk-active.sh` on the PC via a
|
||||
systemd-user timer, gated by hypridle (see the script header). Posts over wg.
|
||||
- **wg_handshake** (coarse, τ=20min) — still unfed; it's homesrv-local
|
||||
(`wg show latest-handshakes`), a natural small poller to add next.
|
||||
|
||||
Allowlisted keys only; without `-core`, `/api/signal` returns 503 and presence
|
||||
stays cold-start `away`.
|
||||
|
||||
Open http://10.42.0.1:9200/ (or http://voice.kvmx.ru:9200/) on your phone from
|
||||
inside the WireGuard tunnel. Tap & hold to speak; release to send; the reply
|
||||
plays automatically.
|
||||
|
||||
## mavpoll — env poller (netdata + uptime-kuma → facts)
|
||||
|
||||
Thin adapter: reads netdata alarms + kuma monitor status and writes `env` facts
|
||||
through core's IPC socket. This is what makes `service_down` (sev4) and
|
||||
`netdata_critical` (sev3) rules fire on real data. Runs on homesrv where both
|
||||
services live — hit them on localhost, not the public `.kvmx.ru` names.
|
||||
|
||||
```bash
|
||||
cd "$ROOT"
|
||||
go build ./cmd/mavpoll/
|
||||
# netdata only (kuma disabled until its API key exists):
|
||||
./mavpoll -socket /run/user/$UID/maven/mavend.sock -netdata http://127.0.0.1:19999
|
||||
# with kuma: create an API key in Kuma → Settings → API Keys, then:
|
||||
./mavpoll -socket /run/user/$UID/maven/mavend.sock \
|
||||
-netdata http://127.0.0.1:19999 \
|
||||
-kuma http://127.0.0.1:3001/metrics -kuma-key <API_KEY>
|
||||
```
|
||||
|
||||
Writes only on value change (append-only, no per-tick churn). `service_down`
|
||||
aggregates any monitor reading 0 as "down"; per-service granularity is a later
|
||||
add. netdata `-timeout`/`-interval` tunable; defaults 8s / 60s.
|
||||
|
||||
### nginx (optional)
|
||||
|
||||
```bash
|
||||
sudo cp cmd/mavweb/nginx.conf /etc/nginx/sites-available/voice.kvmx.ru
|
||||
sudo ln -sf /etc/nginx/sites-available/voice.kvmx.ru /etc/nginx/sites-enabled/
|
||||
sudo nginx -t && sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
## Quick smoke test (stubs, no models)
|
||||
|
||||
```bash
|
||||
cd "$ROOT"
|
||||
./mavend -config mavend.json # voice enabled, no stt/tts/embedder config → all stubs
|
||||
```
|
||||
|
||||
## Run all tests
|
||||
|
||||
```bash
|
||||
cd "$ROOT"
|
||||
go test ./internal/router/ ./internal/delivery/... ./cmd/mavend/ ./cmd/mavsttd/ ./cmd/mavttsd/
|
||||
```
|
||||
|
||||
## Benchmark
|
||||
|
||||
```bash
|
||||
cd "$ROOT"
|
||||
go test -bench=. ./internal/router/ ./cmd/mavsttd/ ./cmd/mavttsd/
|
||||
```
|
||||
@@ -6,7 +6,7 @@
|
||||
> use the embedded LFM model paths or old single-object examples as current ops
|
||||
> guidance; see `2026-07-18-qwen3-resident-training-eval.md`.
|
||||
|
||||
> Scope from `REARCH.md`. Make Maven trustworthy: the LFM becomes the router
|
||||
> Scope from `docs/rearchitecture.md`. Make Maven trustworthy: the LFM becomes the router
|
||||
> (fixes "messes up queries" / "doesn't take notes"), the engine actually runs
|
||||
> (fixes stub replies), dates stop being read as "number dot number dot number",
|
||||
> and telegram becomes a reach channel. NOT in scope: on-demand 4B reasoner,
|
||||
@@ -693,7 +693,7 @@ ssh kami@192.168.1.104 'curl -s localhost:9201/api/chat -d "{\"text\":\"запо
|
||||
4. `docker compose up -d mavend && docker logs -f maven-mavend-1` — confirm the
|
||||
phraser spawns and no `phraser: NewStub` path. Run the two verify curls.
|
||||
5. Update `AGENTS.md`: LFM model download + note that routing is now LFM-first
|
||||
with classifier fallback (`REARCH.md` is the design of record).
|
||||
with classifier fallback (`docs/rearchitecture.md` is the design of record).
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,260 @@
|
||||
# Maven Voice Protocol
|
||||
|
||||
*Last verified: 2026-08-02 @ 7079a24. Living doc: correct it in place, do not append.*
|
||||
|
||||
> Auto-generated from `internal/voice/wire.go`, `internal/voice/errors.go`,
|
||||
> `internal/voice/frame.go`, `internal/voice/client.go`. If this file and
|
||||
> those files disagree, the code wins.
|
||||
|
||||
## Transport
|
||||
|
||||
TCP, length-prefixed JSON. Each frame is:
|
||||
|
||||
```
|
||||
[4 bytes big-endian uint32 length][JSON payload]
|
||||
```
|
||||
|
||||
The length is the number of **bytes** of the JSON payload that follows
|
||||
(excludes the 4-byte length prefix itself). Maximum frame size is **64 MiB**
|
||||
(`maxFrame = 64 << 20`), enough for ~33 minutes of 16k mono int16 PCM audio.
|
||||
|
||||
The server binds a TCP address inside the WireGuard tunnel (production) or
|
||||
`127.0.0.1:9100` (local smoke test). The reference server port is configured
|
||||
via `voice.bind` in `mavend.json`.
|
||||
|
||||
## Frame types
|
||||
|
||||
Three frame shapes share the same length-prefixed envelope. A reader
|
||||
distinguishes them by shape rather than a type tag:
|
||||
|
||||
| Frame | Has `id`? | Has `kind`? | Direction |
|
||||
|-------|-----------|-------------|-----------|
|
||||
| Request | yes (`id` + `m`) | no | client → server |
|
||||
| Response | yes (`id`) | no | server → client |
|
||||
| Push | no | yes (`kind`) | server → client (async) |
|
||||
|
||||
### Request (client → server)
|
||||
|
||||
```json
|
||||
{
|
||||
"id": 1,
|
||||
"m": "push_to_talk",
|
||||
"p": { ... }
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `id` | uint64 | Chosen by client, monotonically increasing per connection. Server echoes it back in the matching Response |
|
||||
| `m` | string | Method name (see Methods below) |
|
||||
| `p` | object | Method-specific params (omitempty) |
|
||||
|
||||
### Response (server → client)
|
||||
|
||||
```json
|
||||
{
|
||||
"id": 1,
|
||||
"r": { ... },
|
||||
"e": null
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `id` | uint64 | Matches the Request this replies to |
|
||||
| `r` | object | Result payload (omitempty). Set exactly when `e` is null |
|
||||
| `e` | object | Error payload (omitempty). See Error codes |
|
||||
|
||||
A Response always matches a prior Request. It is written on the same
|
||||
connection immediately after handling. The client can block reading — there
|
||||
is exactly one Response per Request for today's synchronous methods.
|
||||
|
||||
### Push (server → client, async)
|
||||
|
||||
```json
|
||||
{
|
||||
"kind": "audio_nudge",
|
||||
"p": { ... }
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `kind` | string | Push kind (see Push kinds below) |
|
||||
| `p` | object | Push-specific params (omitempty) |
|
||||
|
||||
A Push is server-initiated. It can arrive on a connection that is also
|
||||
awaiting a Response. The client distinguishes them by checking whether `id`
|
||||
is present (Response) or `kind` is present (Push).
|
||||
|
||||
## Methods
|
||||
|
||||
### `push_to_talk`
|
||||
|
||||
The reactive round-trip: client sends captured audio, server replies with
|
||||
synthesised audio + reply text.
|
||||
|
||||
**Request params** (`PushToTalkReq`):
|
||||
|
||||
```json
|
||||
{
|
||||
"audio": {
|
||||
"fmt": "pcm_16k_mono",
|
||||
"data": "<base64-encoded PCM bytes>"
|
||||
},
|
||||
"lang": "ru",
|
||||
"surface": "pc_client"
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `audio` | Audio | Captured PCM. `fmt` is one of `pcm_16k_mono`, `pcm_44k_stereo`, etc. (see `internal/audio`). `data` is base64-encoded raw PCM bytes |
|
||||
| `lang` | string | Recognition language hint: `"ru"`, `"en"`, `"mixed"`, or unset for daemon default |
|
||||
| `surface` | string | Client's auth surface. Today the floor always sets `"pc_client"`; future mTLS/passkey handshakes populate this |
|
||||
|
||||
**Response result** (`PushToTalkResp`):
|
||||
|
||||
```json
|
||||
{
|
||||
"reply_audio": {
|
||||
"fmt": "pcm_16k_mono",
|
||||
"data": "<base64>"
|
||||
},
|
||||
"reply_text": "тихий режим включён. буду реже напоминать.",
|
||||
"transcript": "тихий режим",
|
||||
"routed_channels": null
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `reply_audio` | Audio | TTS-synthesised reply audio (always present on success) |
|
||||
| `reply_text` | string | Same reply in plain text |
|
||||
| `transcript` | string | STT transcription of the input audio (omitempty) |
|
||||
| `routed_channels` | [string] | Away channels the dispatcher also delivered to (empty for today's reactive-only path) |
|
||||
|
||||
### `pong`
|
||||
|
||||
Liveness response. The client sends this in reply to a `ping` Push to
|
||||
refresh its last-active timestamp on the server.
|
||||
|
||||
**Request params:** none (empty `"p": null` or omitted).
|
||||
|
||||
**Response result:** `null`.
|
||||
|
||||
## Push kinds
|
||||
|
||||
### `audio_nudge`
|
||||
|
||||
Server has a proactive nudge to deliver. The client should play the audio.
|
||||
|
||||
**Params** (`AudioNudgePush`):
|
||||
|
||||
```json
|
||||
{
|
||||
"rule_name": "water",
|
||||
"severity": 2,
|
||||
"audio": {
|
||||
"fmt": "pcm_16k_mono",
|
||||
"data": "<base64>"
|
||||
},
|
||||
"text": "кажется, ты давно не пил воду.",
|
||||
"ts": "2026-07-03T12:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `rule_name` | string | The proactive rule that fired |
|
||||
| `severity` | int | 1–4 where 4 = most urgent |
|
||||
| `audio` | Audio | TTS-synthesised nudge body |
|
||||
| `text` | string | Same body as plain text |
|
||||
| `ts` | datetime | When the nudge was sent (RFC3339) |
|
||||
|
||||
### `ping`
|
||||
|
||||
Liveness probe from the server. The client should respond with a `pong`
|
||||
Request to keep its session alive.
|
||||
|
||||
**Params:** none (`"p": null` or omitted).
|
||||
|
||||
## Audio format
|
||||
|
||||
The `Audio` type is always:
|
||||
|
||||
```json
|
||||
{
|
||||
"fmt": "pcm_16k_mono",
|
||||
"data": "<base64>"
|
||||
}
|
||||
```
|
||||
|
||||
- **Sample rate:** 16000 Hz
|
||||
- **Channels:** 1 (mono)
|
||||
- **Sample format:** signed 16-bit little-endian int (int16)
|
||||
- **Encoding:** raw PCM, no header
|
||||
- **Wire encoding:** base64 inside the JSON frame
|
||||
|
||||
The reference client wraps/unwraps WAV headers at the file edge
|
||||
(`audio.WAVFromPCM`, `audio.PCMFromWAV`). The wire never carries WAV.
|
||||
|
||||
## Error codes
|
||||
|
||||
Errors are returned as an object in the Response `e` field:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": 1,
|
||||
"r": null,
|
||||
"e": {
|
||||
"c": "unknown_method",
|
||||
"m": "optional diagnostic"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Code | Meaning | Has message? |
|
||||
|------|---------|-------------|
|
||||
| `unknown_method` | The method name is not recognised | yes (the method name) |
|
||||
| `bad_params` | Params failed to parse / validate | yes (parse error text) |
|
||||
| `forbidden` | The surface is not authorised for this method | no |
|
||||
| `internal` | Server-side error (transient, retryable) | yes (diagnostic only) |
|
||||
|
||||
The `message` field is never authority-bearing. Auth refusals carry
|
||||
`forbidden` with no message.
|
||||
|
||||
## Auth surface
|
||||
|
||||
The server enforces capability scopes per surface. Today's floor assigns
|
||||
`SurfacePCClient` to every connection (full L3 access). Future passkey
|
||||
handshakes will set the surface from `mTLS` metadata / `WebAuthn`
|
||||
enrollment.
|
||||
|
||||
Known surface values:
|
||||
- `voice` — voice/chat channel (structurally capped below destructive acts)
|
||||
- `pc_client` — reference desktop client
|
||||
- `authed_page` — mavweb /tools page
|
||||
- `telegram` — Telegram inbound
|
||||
|
||||
## Session lifecycle
|
||||
|
||||
1. Client connects via TCP to the voice address.
|
||||
2. Server registers a Session (assigns an opaque ID, records `lastActive`).
|
||||
3. Client sends Requests and receives Responses + Pushes on the same conn.
|
||||
4. Proactive delivery routes to the most-recently-active session by
|
||||
`lastActive` timestamp. If no session is live, the dispatcher falls
|
||||
through to away channels.
|
||||
5. On disconnect (EOF / read error / shutdown), the session is removed.
|
||||
|
||||
## Reference client
|
||||
|
||||
`cmd/mavenclient` implements this protocol. Use it to smoke-test:
|
||||
|
||||
```shell
|
||||
# one-shot: send audio.wav → get reply.wav
|
||||
mavenclient -addr 127.0.0.1:9100 -in audio.wav -out reply.wav
|
||||
|
||||
# listen mode: stay connected, write incoming Pushes to disk
|
||||
mavenclient -addr 127.0.0.1:9100 -listen -out-prefix /tmp/nudge-
|
||||
```
|
||||
+186
@@ -0,0 +1,186 @@
|
||||
# QA plan: checking Maven properly
|
||||
|
||||
*Last verified: 2026-08-02 @ 7079a24. Living doc: correct it in place, do not append.*
|
||||
|
||||
Written 2026-08-01, after the 35-PR stack landed and the box came back up.
|
||||
|
||||
44 of the 50 open Vikunja tasks are `QA:` tasks. They are verification work, not
|
||||
build work. Most sat unverifiable while Maven was down for 11 days. That
|
||||
blocker is gone.
|
||||
|
||||
This plan orders them by what unblocks what. Do sessions 1 and 2 first. Almost everything
|
||||
downstream assumes the voice loop works, and nobody has confirmed that since
|
||||
the redeploy.
|
||||
|
||||
---
|
||||
|
||||
## Before you start
|
||||
|
||||
Two things bite anyone running these checks on homesrv.
|
||||
|
||||
**curl needs `--noproxy '*'`.** The shell exports `http_proxy=http://127.0.0.1:18080`.
|
||||
Without the flag, every local check returns 503 from the proxy and looks like a
|
||||
dead service. This cost me a false regression report today.
|
||||
|
||||
**The database is not readable with sqlite3.** Four older QA steps say
|
||||
`docker compose exec mavend sqlite3 /data/maven.db "select ..."`. That cannot
|
||||
work: the container has no `sqlite3` binary, and the store is AES-256-GCM at
|
||||
rest with a tmpfs working copy. Read state through mavweb instead, at
|
||||
`/history`, `/trace`, `/routines` and `/dash`.
|
||||
|
||||
---
|
||||
|
||||
## Session 1: the voice loop (half a day)
|
||||
|
||||
Nothing here has been confirmed since the redeploy, and everything else assumes
|
||||
it works. Do this first.
|
||||
|
||||
Closes or advances: **44** (conversation), **45** (text chat), **287** (voice
|
||||
session quality), **321** steps 3-5 (quiet mode), **288** (STT fixtures).
|
||||
|
||||
1. Open `http://127.0.0.1:9201/chat` and hold a short conversation in Russian.
|
||||
Watch for three things: she answers in feminine forms (`рада`, `поняла`), she
|
||||
says `ты` and never `вы`, and no pet names appear.
|
||||
2. Press push-to-talk on `/dash`. Say `привет`. Confirm a spoken reply comes
|
||||
back. This is the only check that covers mic to STT to core to TTS to
|
||||
speaker as one path. It is also the path the eleven-day outage most likely
|
||||
broke.
|
||||
3. Say `тихий режим`. Expect `тихий режим включён. буду реже напоминать.`
|
||||
4. Say `выключи тихий режим`. Expect `тихий режим выключен.` Negation must win.
|
||||
5. Say `в комнате тихо`. Quiet mode must NOT flip. Confirm on `/history` that no
|
||||
`quiet_hours` fact was written.
|
||||
6. Say `включи режим тишины`, then `сделай потише`. Both must flip quiet mode
|
||||
on. These are the noun form and the comparative, added 01-08-2026.
|
||||
7. Wait for a nudge, then say `потом` within twenty minutes. Expect `хорошо,
|
||||
вернусь к этому позже.` and the nudge row on `/notifications` reading
|
||||
`snoozed`. Say `потом` again with nothing pending: it must route as an
|
||||
ordinary utterance, not be swallowed.
|
||||
8. Wait for the water nudge, then say `выпил воды`. Expect the ordinary fact
|
||||
reply and nothing extra — she must not congratulate you. Check
|
||||
`/notifications`: the row reads `acted`. Then trigger another nudge and say
|
||||
`готово`; expect `отлично, отметила.` and the same outcome.
|
||||
9. Note anything where she is slow, cuts off, or talks over herself. That is
|
||||
287's whole content and it has no written acceptance criteria yet.
|
||||
|
||||
**319 is fixed** (01-08-2026). Single-word Russian utterances no longer come
|
||||
back as `не совсем поняла — можешь переформулировать?`. `привет` and `поужинал`
|
||||
both pass now: `thinSingleToken` spares social singles and any token carrying a
|
||||
verb ending, and only thins a bare nominal like `вода`. If a one-word utterance
|
||||
still gets clarified during the smoke test, that is a new case for the lexicon,
|
||||
not the old bug.
|
||||
|
||||
---
|
||||
|
||||
## Session 2: measurement (half a day, mostly waiting)
|
||||
|
||||
Closes or advances: **320** items 2-4, **278** (make the eval lab routine),
|
||||
**319** (gate recalibration).
|
||||
|
||||
The resident llama-server cannot be reached by the eval harness. It binds
|
||||
`--host 127.0.0.1 --port 0` inside the container, so the port is kernel-assigned
|
||||
and never published. Start a second one on a fixed port instead:
|
||||
|
||||
```sh
|
||||
llama-server -m /mnt/hdd1/llms/qwen3/Qwen3-1.7B-UD-Q4_K_XL.gguf \
|
||||
--host 127.0.0.1 --port 18100 -c 4096 -ngl 99 --no-webui
|
||||
```
|
||||
|
||||
`-c 4096` matters. The recorded numbers were measured at that context size, and
|
||||
a mismatch invalidates the comparison.
|
||||
|
||||
Then:
|
||||
|
||||
```sh
|
||||
make eval-models MAVEN_LLM_URL=http://127.0.0.1:18100 # want ~72.7% cascade
|
||||
make eval-router # classifier baseline
|
||||
MAVEN_LLM_URL=http://127.0.0.1:18100 make eval-phrasing # persona checks, slow
|
||||
make eval-recall
|
||||
```
|
||||
|
||||
A large miss against 72.7% means the deploy differs from the bench harness.
|
||||
|
||||
Two things to decide while the numbers are in front of you:
|
||||
|
||||
- **319's gate recalibration.** The single-token rule needs narrowing or
|
||||
dropping. This needs your judgement, not a threshold sweep. The fixture and the
|
||||
daemon disagree about what is correct on two of the three false clarifies.
|
||||
- **278's real ask** is making the eval lab routine rather than building it. It
|
||||
is built. Decide whether it runs on a timer, on every merge, or on demand, and
|
||||
the task can close.
|
||||
|
||||
Item 4 of **320** needs a permission I do not have. Kill the `llama-server`
|
||||
pid under `maven-mavend-1`, post a turn, and confirm it still completes
|
||||
through the classifier. Either grant it or run it yourself. It is the only
|
||||
check that the failure floor catches a mid-session model death.
|
||||
|
||||
---
|
||||
|
||||
## Session 3: the interaction batch (a day, or three sittings)
|
||||
|
||||
These need real use rather than a command, grouped by what one sitting covers.
|
||||
|
||||
**Morning and delivery** (**280**, **281**, **128**, **282**): open `/morning`,
|
||||
walk the seven required behaviours, then check the four interruption outcomes
|
||||
and the digest gap. **282** needs the `desk_active` script enabled on the desk
|
||||
PC first, which is **15** and needs you at that machine.
|
||||
|
||||
**Tasks and calendar** (**129**, **130**, **127**, **126**, **246**): capture a
|
||||
task by voice, confirm it lands, check prioritisation ordering is not nonsense.
|
||||
**246** (mail reader) also exercises the `IngestMail` rung that moved to
|
||||
`AuthWrite` this morning.
|
||||
|
||||
**Routines and patterns** (**43**, **46**, **247**, **254**): these need history
|
||||
to detect against. If the database is thin after the outage, they may have
|
||||
nothing to propose, which is not a failure. Check `/routines` before
|
||||
concluding anything.
|
||||
|
||||
**Ecosystem** (**272**, **273**, **276**): nexus, hexis and praxis are wired and
|
||||
logged clean at boot. **276** is the degraded-mode suite, which means taking
|
||||
siblings down on purpose. Worth doing while you are already in there.
|
||||
|
||||
---
|
||||
|
||||
## Housekeeping (one sitting, no box needed)
|
||||
|
||||
Four QA tasks will not close no matter how long they sit, because they are
|
||||
gated on something that does not exist:
|
||||
|
||||
- **125** zenmoney: needs a token you have not minted.
|
||||
- **256** Home Assistant: needs HA configured.
|
||||
- **257** Bluetooth: BLOCKED, no bluez on the box. Says so in the title.
|
||||
- **288** STT golden audio: needs fixtures generated.
|
||||
|
||||
Relabel these so they stop reading as backlog. They are not verification work
|
||||
that is pending, they are work that has not started.
|
||||
|
||||
Same treatment for the five plan-only tasks (**251** MCP, **252** vision,
|
||||
**253** hearing, **255** speaker recognition, **259** crawler). A `QA:` prefix on
|
||||
a plan is misleading.
|
||||
|
||||
---
|
||||
|
||||
## Needs you specifically
|
||||
|
||||
Not QA. These are blocked on a decision or a credential only you have.
|
||||
|
||||
| # | what |
|
||||
|---|---|
|
||||
| 16 | Create the Kuma API key. `-kuma-key uk5_mavpoll-key` in `docker-compose.yml` is still the placeholder. |
|
||||
| 15 | Deploy `desk_active` on the desk PC. Blocks **282**. |
|
||||
| 122 | Finish the CPT run for Qwen3-1.7B. The persona fix depends on it. |
|
||||
| 355 | Deploy the Hexis auth change. Was blocked on Maven being under construction, which it no longer is. The client half is vendored and wired. |
|
||||
| 357 | Decide whether entity-existence validation is the permanent target guard or whether blessing lands in Nexus. |
|
||||
| 275 | Hexis native API and MCP parity. |
|
||||
| — | Decide on `-require-stepup`. Making it the default needs WebAuthn configured first, or it locks you out of your own admin surfaces. See **317**. |
|
||||
| — | Three nginx sites bind wildcard `:80` (`acme.conf`, `matrix`, `panel`), so the ecosystem's bind-level protection is not in effect and `allow`/`deny` is carrying it alone. See **354**. |
|
||||
|
||||
---
|
||||
|
||||
## Suggested order
|
||||
|
||||
1. Session 1. If the voice loop is broken, nothing else matters.
|
||||
2. The `-require-stepup` and Kuma decisions. Five minutes, unblocks **317** fully
|
||||
and **16**.
|
||||
3. Session 2. The numbers tell you whether the router is worth its 90x latency.
|
||||
4. Housekeeping. Cheap, and it makes the remaining backlog honest.
|
||||
5. Session 3, split whichever way suits you.
|
||||
@@ -0,0 +1,98 @@
|
||||
# Maven — Re-architecture (Qwen3 resident model, revised 2026-07-18)
|
||||
|
||||
*Last verified: 2026-08-02 @ 7079a24. Living doc: correct it in place, do not append.*
|
||||
|
||||
> Supersedes the classifier-first routing model. Agreed in a design session
|
||||
> after diagnosing that homesrv deploys with a **stub phraser** (no LLM
|
||||
> running) and an embedder-classifier that routes by nearest-neighbor between
|
||||
> frozen seed phrases — the structural cause of "she messes up queries."
|
||||
>
|
||||
> Hardware reality: homesrv = Ryzen 5 5600U laptop, Vega iGPU, 14 GB shared
|
||||
> RAM. Workstation (RX 7900 XT) is NOT the deploy target and is often busy.
|
||||
> So: small models, on-demand where heavy, always-on where cheap.
|
||||
|
||||
## Principle
|
||||
|
||||
The LLM is **not** the center of everything. Deterministic tools handle the
|
||||
bulk. One locally trained **Qwen3-1.7B** resident model is used for routing and
|
||||
talking back. Its system prompt selects one of two independently evaluated
|
||||
contracts: route actions or persona responses.
|
||||
|
||||
**If the router is good, Maven feels good.** Routing is the linchpin.
|
||||
|
||||
## The spine
|
||||
|
||||
```
|
||||
utterance
|
||||
→ [world-state context] cheap: time, presence, calendar_busy, weather (no LLM)
|
||||
→ ROUTER = Qwen3-1.7B (always-on, grammar-constrained)
|
||||
reads utterance + context + tool schema, emits a STRUCTURED action:
|
||||
• call a tool (deterministic) • answer directly
|
||||
→ tools (deterministic, fast)
|
||||
→ PHRASER = Qwen3-1.7B (same resident process) → TTS / text
|
||||
```
|
||||
|
||||
- **Router = Phraser = one resident Qwen3-1.7B llama-server**, with separate
|
||||
route and persona prompts/contracts. Always warm; classifier/stub remain the
|
||||
failure floors.
|
||||
- The resident checkpoint is trained end-to-end as: Qwen3-1.7B-Base → RU CPT →
|
||||
joint persona/router SFT → merge → GGUF. Larger on-demand reasoning models
|
||||
are deferred until the main feature set is complete.
|
||||
- **Embedder demoted from router to tool** — it now backs `memory.search`
|
||||
(RAG) and gives the router a cheap "similar past notes/intents" hint. The
|
||||
router no longer depends on it clearing a threshold. Upgrade MiniLM → bge-m3
|
||||
for better RU retrieval later (model swap, not architecture).
|
||||
|
||||
### Router output
|
||||
- Constrained structured JSON action `{tool, args, escalate}` — NOT free-form
|
||||
multi-step function-calling. Sub-1B is far more reliable emitting a fixed
|
||||
schema. Enforce with a **GBNF grammar** in llama.cpp (near-bulletproof).
|
||||
- Keep the existing **stage-0 exact-match fast-path** for dead-obvious commands
|
||||
(skips the router entirely) — cheap insurance, already built.
|
||||
|
||||
## The proactive / memory half — one background engine
|
||||
|
||||
"Take notes," "remember," "reflect," "suggest do you want to add X?", "remind"
|
||||
are NOT request-path features. They are one **digestion worker**:
|
||||
|
||||
```
|
||||
DIGESTION WORKER (periodic + event-driven, off the request path)
|
||||
• reads new facts/notes since last pass
|
||||
• RAG-consolidates: dedupe, link, summarize into durable memory
|
||||
• reflects: detect patterns ("mentioned X three times")
|
||||
• proposes: "want me to add X / remind you about Y?" → nudge dispatcher
|
||||
• surfaces due reminders
|
||||
uses the resident Qwen3 model for bounded synthesis — never blocks a turn
|
||||
```
|
||||
|
||||
Notes capture is a deterministic Tier-0 tool; making notes *mean something
|
||||
later* is the worker + RAG.
|
||||
|
||||
## Layer table
|
||||
|
||||
| Layer | What | Runs |
|
||||
|---|---|---|
|
||||
| Context | world-state (time/presence/calendar/weather) | always, no LLM |
|
||||
| **Router** | Qwen3 structured-action orchestrator — linchpin | **always-on** |
|
||||
| Tools | note/reminder/memory/calendar/weather/act (deterministic) | always |
|
||||
| Reasoner | larger specialist model | **deferred** |
|
||||
| Phraser | Qwen3 persona response | **always-on (same proc as router)** |
|
||||
| Digestion worker | reflection → suggestions/nudges/memory | **background** |
|
||||
| Reach | telegram (+ existing ntfy/voice) | quick win |
|
||||
| Voice quality | custom/better TTS | **deferred** (workstation GPU busy) |
|
||||
|
||||
## Build order
|
||||
|
||||
1. **Foundation + router** — completed shared llama-server client, router,
|
||||
fallback, replier, TTS normalization and telegram reach.
|
||||
2. **Qwen3 resident checkpoint** — finish RU CPT, pass the raw-vs-CPT gate,
|
||||
jointly SFT persona/router contracts, merge, quantize and deploy.
|
||||
3. **Main features** — reflection, proactive suggestions, memory
|
||||
consolidation, RAG read-back.
|
||||
4. **Deferred work** — larger reasoner, custom Piper voice and other expansions.
|
||||
|
||||
## Non-goals (unchanged)
|
||||
Not a nag. Not autonomous. Feminine-gendered RU self-ref. No telemetry, no
|
||||
cloud model, no third-party account — but she MAY read external sources to
|
||||
answer world questions (Kiwix first, search optional). "Never phones home" as
|
||||
an absolute is deprecated, owner's call 2026-07-31; see CLAUDE.md § Non-goals.
|
||||
Reference in New Issue
Block a user