76a6a007ef
The most load-bearing decision in the project was stated four incompatible ways: the docs said Qwen3-1.7B, deploy/mavend.json said Qwen3.5-2B, the repo's models/llm/ held an LFM2.5-1.2B gguf, and five code comments still said LFM. Answering "which model is deployed" meant re-deriving it from scratch every time. Two facts the review missed, found while resolving it: - /mnt/hdd1/llms is bind-mounted over /opt/maven/models/llm, which shadows the repo's models/llm/. The LFM2.5 gguf sitting there was never loaded by anything, so it was not evidence of the deployed model at all. - That library holds Qwen3.5-0.8B, -2B and -4B, and no Qwen3-1.7B. The config pointed at a file that does exist; the docs' Qwen3-1.7B was the stale claim, the reverse of the assumed direction. Qwen3-1.7B is the CPT target, and that training is still in flight (Vikunja #122), so no such gguf exists yet. phraser.model_path moves to Qwen3.5-0.8B (Q4_K_M) — the smallest checkpoint on disk, chosen for latency, and relevant to whether the LLM router is affordable on this box. Docs and comments now say the same thing in one voice: 0.8B resident now, CPT'd Qwen3-1.7B as the target, and the bind-mount shadowing written down so the next reader does not mistake models/llm/ for ground truth. Comments name the model, never a filename, so a swap stays a one-line config change. n_gpu_layers: 99 is correct and stays — compose passes /dev/dri and the render gid for Vulkan offload to the Vega iGPU. CLAUDE.md's "CPU-only" was the stale half of that contradiction and is corrected. phraser.go also dropped a wrong "sub-1b, prompted not trained" size claim: the target is trained end-to-end (RU CPT + joint persona/router SFT). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X5JApcrCRVGmqrxnhynSik
402 lines
26 KiB
Markdown
402 lines
26 KiB
Markdown
## Maven — current state (updated 2026-07-18)
|
||
|
||
Architecture decision: the target resident router/phraser is the locally
|
||
trained Qwen3-1.7B model. Older LFM references below describe the currently
|
||
deployed/historical stack, not the target checkpoint. RU CPT has a successful
|
||
full-weight checkpoint at step 1000/8077; evaluation and Qwen3 SFT tooling are
|
||
tracked in `docs/plans/2026-07-18-qwen3-resident-training-eval.md`.
|
||
|
||
Consolidated status. The reactive↔proactive core is closed and testable through
|
||
the web PWA. The SPEC's open items 1–7 are landed (protocol doc, away-channel
|
||
fallthrough, CalDAV poller, quiet-hours schedule, tools enable/disable, note RAG,
|
||
passkey step-up); item 8 (multi-user) is deliberately deferred — see the tail.
|
||
The two big infra gaps from the jul5 revision are closed on `overnight-jul5`:
|
||
**at-rest encryption** (AES-256-GCM, tmpfs working copy — not sqlcipher, see
|
||
`internal/store/crypt.go`) and **Docker deployment** (one image, six daemon
|
||
containers). The `overnight-jul6` session (now on `master`) closed the biggest
|
||
*query-surface* gaps — **calendar querying, general-knowledge answers, and
|
||
weather** — plus a populated homelab act allowlist and two pure scaffolds
|
||
(dialogue state, long-term-memory vector store). ~15.2k LOC + ~8.5k test, 303
|
||
tests, `-race` in `make test`.
|
||
|
||
### Access model
|
||
|
||
- **Phone** → needs the wg tunnel to reach homesrv (no homesrv DNS otherwise;
|
||
raw IP or a DNS tweak can bypass, not the default).
|
||
- **PC** → uses homesrv DNS, resolves the domains over local-net, **no wg needed**.
|
||
- nginx + ufw both scope to `10.42.0.0/24` (wg) + `192.168.1.0/24` (LAN), deny all else.
|
||
- **Surface in use now: the web PWA (`mavweb`).** Voice PTT + in-app nudges both ride it.
|
||
|
||
### Works end-to-end (tested)
|
||
|
||
- **Reactive voice:** PWA record → Whisper STT (`mavsttd`) → ONNX classifier →
|
||
resident phraser (llama-server subprocess; Qwen3.5-0.8B as of 2026-07-30 —
|
||
this line historically named "LFM 2.5-1.2B") → Piper TTS
|
||
(`mavttsd`) → reply.
|
||
HTTP POST path (mobile-Chrome drops WS for the audio).
|
||
- **Capture:** `fact` (EN **and RU** — root-substring recognizers) + `reminder`
|
||
persist through CoreAPI (`source=tap:voice`). This is the substrate the care
|
||
rules read.
|
||
- **Notes / query (semantic recall, sqlite — no chroma):** `note` → embed (the
|
||
classifier's ONNX embedder) → `notes` table. `query` → embed → brute-force
|
||
cosine top-k → confidence-gated (below `queryMinScore` 0.55 ⇒ "no note", not a
|
||
guess). **Note RAG (SPEC item 6):** the gated top-k feed the phraser
|
||
(`PhraseQuery`) to compose a natural answer ("вот что я нашла: …") instead of
|
||
a verbatim dump; raw-notes fallback on any LLM error. Stub is deterministic.
|
||
- **Monitoring (`/dash`):** mavweb server-renders presence + recent nudges (by
|
||
outcome) + recent facts from the append-only store via CoreAPI. Read-only,
|
||
meta-refresh, no JS.
|
||
- **Proactive loop:** 60s dumb ticker, pure predicates over a State snapshot,
|
||
universal gate (quiet-hours/presence/cooldown/snooze/calendar), one-nudge-per-
|
||
tick max-severity, reminders (gate-bypassing), sev4 repeat-til-ack, feedback
|
||
auto-tuner (outcome ratio → bounded cooldown, persisted as `source=feedback`).
|
||
- **Rules:** water/meal/break (sev1–2 care), service_down (sev4, `poll:uptimekuma`),
|
||
netdata_critical (sev3, `poll:netdata`).
|
||
- **Routines (`internal/routine`):** operator-declared clockwork — the third
|
||
proactive class beside reminders (user-stated) and care rules (world-state).
|
||
Config `routines[]` (cron + literal RU body + severity) fire through the normal
|
||
dispatcher on schedule (an 08:00 briefing, a 22:00 wind-down). Bodies are
|
||
literal (not LLM-phrased ⇒ can't hallucinate); rule name `routine:<name>` so
|
||
they don't pollute the care autotuner; cold-start guard seeds on first sight so
|
||
a restart never replays a missed schedule. Pure `routine.Due`, unit-tested; the
|
||
tick driver holds the last-fired map.
|
||
- **Env facts (`mavpoll`):** netdata alarms → `netdata_alarm` (fires immediately
|
||
on a real CRITICAL); kuma monitor_status → `service_down`. Writes only on
|
||
value-change (no append-only churn).
|
||
- **Presence:** noisy-OR decay + Schmitt hysteresis. Live via `page_heartbeat`
|
||
(PWA auto-pings `/api/signal` every 30s → present when a tab's open).
|
||
- **Delivery:** ntfy / telegram / voice by `f(severity, presence)`; minimal body
|
||
on away channels. PWA subscribes to ntfy over **WebSocket** for in-app nudges.
|
||
- **Away-channel fallthrough (SPEC item 2):** when the router picks voice but no
|
||
live session exists at push time (presence guess was wrong), the dispatcher
|
||
reroutes through the AWAY table — sev3→ntfy, sev4→telegram-repeat-til-ack,
|
||
sev≤2→drop — instead of silently dropping. Covers nudges + reminders.
|
||
- **Calendar busy (SPEC item 3, `mavcaldav`):** new poller queries a self-hosted
|
||
**Radicale** CalDAV server on an interval, writes `calendar_busy` + event facts
|
||
through CoreAPI (value-change only). The loop gate already consumes `calendar_busy`.
|
||
- **Quiet-hours schedule (SPEC item 4):** the gate reads `quiet_hours`; a config
|
||
time window (`voice.quiet_hours`, HH:MM, midnight-crossing handled) now sets it
|
||
on each tick — in addition to the "тихий режим" voice toggle. Both activate quiet.
|
||
- **Client protocol (SPEC item 1):** the voice wire format (length-prefixed JSON
|
||
frames) is published in `PROTOCOL.md`, generated from `internal/voice/wire.go`
|
||
so third-party clients don't need the Go source.
|
||
- **Passkey step-up (SPEC item 7):** `internal/webauthn` does real WebAuthn —
|
||
ES256/P-256 register + assert, ecdsa signature verification, rpIdHash + UP/UV
|
||
flag binding (UV = the gesture), sign-count regression check. `PasskeySession`
|
||
bumps the auth session L2→L3 for a TTL on assert. mavweb serves `/auth/passkey`
|
||
(enroll + step-up) + the begin/finish endpoints. Crypto is round-trip tested
|
||
(incl. tampered-sig / missing-UV / wrong-origin negatives).
|
||
- **Stability:** llama-server orphan leak fixed (`Pdeathsig` kills the child on
|
||
any mavend death); `kill-maven.sh` reaps strays (matches the model, not a
|
||
bogus `llama-server.*maven` pattern); `start-maven.sh` wires `-core` + poller.
|
||
|
||
### Wired but needs a deploy action (not code)
|
||
|
||
- **`desk_active`** (strongest presence signal) — `scripts/desk-active.sh` runs
|
||
on the **desk PC** (hypridle-gated systemd timer), posts over wg to mavweb.
|
||
- **`mavwaked`** (always-on listening) — needs a systemd user unit on a client
|
||
box (desk PC, pi, etc.) where the mic is attached. Connects to mavend over wg
|
||
or local net via `-addr`. Deferred until a client box is wired with a mic.
|
||
|
||
Caveats / gotchas:
|
||
- **desk_active is a workstation deploy, not code** — 0 facts ever written; presence
|
||
runs on page_heartbeat alone (dash reads "away"/"never at desk"). `scripts/desk-active.sh`
|
||
+ a hypridle-gated `maven-desk` timer must be installed on the desk PC (not homesrv).
|
||
- **Notes recall needs the ONNX embedder** — under the HashEmbedder floor, cosine is
|
||
lexical (token overlap), not semantic; scores are low, so most RU commands sit under
|
||
the 0.35 route threshold and clarify. Configure `voice.embedder` for confident recall+routing.
|
||
(The floor now at least tokenizes Cyrillic — see below — so it ranks correctly, just weakly.)
|
||
- **Switching the embedder model silently breaks old notes** — different dim ⇒
|
||
cosine 0 ⇒ they stop matching; brute-force can't re-embed. Re-embed on a model change.
|
||
- **`wg_handshake` is OFF and should stay off** — in this topology the phone only
|
||
runs wg when *outside*, so a fresh handshake means AWAY, not here. The `mavpoll
|
||
-wg` flag exists (defaults `""`) and could later back the spec's "away override"
|
||
by flipping the sign; as a presence-*here* signal it's inverted. desk_active +
|
||
page_heartbeat cover home presence.
|
||
- **Cold-start unlock tests are missing** — the key wrap/unwrap code
|
||
(`internal/webauthn/keywrap.go`) and locked-mode IPC gating (`cmd/mavend/main.go`)
|
||
are correct but have **zero test coverage**. The roadmap (item 2.1) required
|
||
three new test cases (wrap/unwrap round-trip, wrong-cred unwrap fails,
|
||
locked-mode IPC rejects non-unlock methods); none were written. `make test`
|
||
is green by omission. Write these before relying on the cold-start path with
|
||
real keys.
|
||
|
||
### Done since last revision (overnight-jul6, 2026-07-06)
|
||
|
||
Seven tasks (session board `SESSION-06-07-2026.md`, deleted 2026-07-30 — see git history), one commit each, merged to `master`.
|
||
This session was run through **opencode**, not Claude Code (co-author trailer).
|
||
|
||
Since then (**2026-07-06, second session**):
|
||
|
||
- **Always-on listening (gap 1, MVP)** — `cmd/mavwaked/`: 825 lines, 10 `-race`
|
||
tests. Energy-based VAD over 30ms windows (same RMS threshold as mavsttd's
|
||
`gateReason`), adaptive noise floor, speech→silence state machine. Captures
|
||
PCM from arecord(1) subprocess, sends `PushToTalk` with `Surface=SurfaceVoice`
|
||
(L0 — no destructive acts). Reply plays through aplay(1). No wake word yet
|
||
(pure VAD trigger); the 30ms frame shape matches silero-vad ONNX input 1:1,
|
||
so swapping energy-threshold for ONNX inference is a local change in vad.go.
|
||
`Makefile` `build-waked` target. Runs on client boxes (not docker/homesrv)
|
||
via systemd user unit; connects to mavend over wg or local net.
|
||
|
||
Since then (**2026-07-06, third session** — roadmap execution agent):
|
||
|
||
- **Cold-start unlock (ROADMAP 2.1)** — the at-rest AES key is now wrapped
|
||
(HKDF-SHA256 + AES-256-GCM, stdlib-only — no `x/crypto` dep) with the passkey
|
||
credential's public key and persisted to disk. At boot, if a wrapped key file
|
||
exists AND no env key is set, mavend starts **locked**: the IPC server runs
|
||
but `srv.Check` rejects everything except `MethodAssertStepUp` +
|
||
`MethodUnlock`. A passkey assertion at `/auth/passkey` calls `MethodUnlock`
|
||
with the credential's public key → unwraps the blob → opens the store → wires
|
||
voice/loop/delivery → `srv.SetAPI` swaps the locked stub for the real
|
||
CoreAPI. mavweb's `RegisterFinish` wraps the env key on enrollment;
|
||
`AssertFinish` calls `Unlock` on assertion. Env-key fallback preserved
|
||
(dev/CI path unchanged). **Test gap:** the roadmap required three new test
|
||
cases (wrap/unwrap round-trip, wrong-cred unwrap fails, locked-mode IPC
|
||
rejects non-unlock methods) — none were written. The code is correct but
|
||
untested; `make test` is green by omission, not coverage.
|
||
- **Conversation depth (ROADMAP 3.2)** — cross-intent anaphora + fact-by-key
|
||
lookup. `AnaphoraResolver` in `router/slots.go` detects RU pronouns
|
||
(это/он/она/оно/тот/мой + inflected forms). `followUpMerge` now handles
|
||
three cases: same-intent slot inheritance (existing), cross-intent anaphora
|
||
(Query/Fact/Reminder after a Fact with a pronoun inherits the prior key +
|
||
time), and query-after-fact (a query following a fact inherits the key for
|
||
fact-by-key lookup). `Session.History []Turn` added as the multi-turn
|
||
scaffold (capped at 4). 7 new test cases including the exact done-when
|
||
scenarios (anaphora query-after-fact, three-turn break, explicit-key-wins).
|
||
- **Routing quality + persona (ROADMAP 4.1/4.4)** — `QueryMinScore` is now a
|
||
config knob (`voice.query_min_score`, default 0.55) instead of a hardcoded
|
||
const. `make download-embedder` fetches Xenova/paraphrase-multilingual-
|
||
MiniLM-L12-v2 (~90MB ONNX) + tokenizer; AGENTS.md documents the embedder +
|
||
libonnxruntime setup. `Persona` field in `VoiceConfig` prepends to every
|
||
LLM system prompt (nudge phrasing, note queries, general knowledge); empty
|
||
= current hardcoded feminine-gendered Russian persona. Also fixed two
|
||
pre-existing data races found by `-race`: `voice/server.go` wg.Add vs
|
||
wg.Wait (accept mutex), `mavweb/server.go` s.api field (atomic.Value).
|
||
|
||
- **Calendar querying (task 3)** — "что у меня завтра?" now answers from the
|
||
CalDAV facts the poller already writes. Added `store.CalendarEvents(from,to)`,
|
||
a RU date-scope parser («сегодня»/«завтра») in `router/slots.go`, and an
|
||
IPC `CalendarEvents` RPC (api/client/server/wire) feeding the `IntentQuery`
|
||
handler. Empty day → «на сегодня ничего нет». Previously calendar only *gated*
|
||
nudges; it's now queryable.
|
||
- **General-knowledge routing (task 4)** — when notes-RAG misses `queryMinScore`,
|
||
the query now falls through to the phraser with an anti-hallucination system
|
||
prompt (`router.KnowledgePrompt`, single tested source) instead of giving up.
|
||
Empty/errored/Stub phraser → «не знаю.», never a fabrication.
|
||
- **Weather (task 5)** — new `internal/weather/`: `Provider` interface, a stub
|
||
(«погода не настроена»), and a real **keyless Open-Meteo** provider (geocode +
|
||
current_weather, injectable `*http.Client`, mocked in tests — no live network).
|
||
Wired into `IntentQuery` (keywords погода/градус/температура) with a ~5s
|
||
context timeout; selected by `voice.weather.provider` ("open-meteo" | "" → stub).
|
||
- **Homelab act allowlist (task 2)** — `voice.tools` seeded with read-only acts
|
||
(`systemctl status`, `docker ps`, `uptime`, `df`, `free`, `journalctl` reads)
|
||
as `destructive:false` and mutating ones (restart/stop/start/reboot,
|
||
docker-restart/stop) as `destructive:true`. Guardrail verified: no dangerous
|
||
verb is `destructive:false`. RU phrasings seeded in `act.txt`.
|
||
- **Embedder config validation (task 1)** — a partially-filled `voice.embedder`
|
||
block (some of model/tokenizer/lib paths missing) is now a load error instead
|
||
of a silent fall-through to the Hash floor; the floor fallback logs explicitly.
|
||
- **Dialogue state scaffold (task 6)** — `internal/dialogue/`: `Session` +
|
||
TTL `SessionStore` + pure `InheritSlots`. **Now wired** (post-merge follow-up):
|
||
the voice handler carries slots across same-intent turns within a 2-min window
|
||
(`followUpMerge`, unit-tested) — bounded gap-filling, not full multi-turn yet.
|
||
- **Long-term memory interface (task 7)** — `internal/memory/`: `Store` interface
|
||
+ `InMemoryStore` (cosine). Wired into `IntentNote` (best-effort insert) and,
|
||
post-merge, into `IntentFact` (facts indexed) + `IntentQuery` (read-back after
|
||
notes-RAG misses). In-memory only — no persistent backend yet (gap #8).
|
||
|
||
Follow-ups (Claude Code, post-merge): gofmt'd `handlers_test.go` (the jul6
|
||
verification commit left it misaligned, so `gofmt -l` still flagged it despite the
|
||
"all gates green" claim); deduped the task-4 knowledge prompt to the single tested
|
||
`router.KnowledgePrompt()`. Tree is now genuinely green (gofmt/vet/303 tests).
|
||
|
||
### Done since the jul5 revision (overnight-jul5, 2026-07-05)
|
||
|
||
The overnight session (`SESSION-05-07-2026.md`, deleted 2026-07-30 — see git history; 25 tasks) closed the previous
|
||
"not built yet" items 1–3 and added feature depth:
|
||
|
||
- **At-rest encryption** — the on-disk db is AES-256-GCM ciphertext; the daemon
|
||
works on a tmpfs (RAM) plaintext copy, sealed back atomically on close. Wrong
|
||
key / tamper ⇒ fail closed, never a plaintext fallback. Legacy plaintext dbs
|
||
upgrade on first clean shutdown. Key via config/env (`db_key_env`); no KDF —
|
||
raw 32-byte key, base64. The passkey cold-start unlock plugs into the same
|
||
`store.OpenEncrypted` seam later.
|
||
- **Docker deployment** — single image, one container per daemon
|
||
(`docker-compose.yml`); only mavend mounts the key + db volume; IPC over a
|
||
shared socket volume. `ipc.DialWait` (boot-order tolerance) + redial-on-drop
|
||
(core restarts don't kill modules). `deploy/README.md` has the runbook.
|
||
- **Tests** — mavcaldav, mavttsd, voicesink, mavweb main/handlers covered;
|
||
`make test` runs `-race -coverprofile`.
|
||
- **Recurring reminders** — `cron` + `next_fire_ts` on reminders; recurring ones
|
||
reschedule (instead of mark-fired) after successful delivery.
|
||
- **Notification digest/batching** — low-severity nudges queue and flush as one
|
||
digest per window/max-items (`digest` config block); stale-reminder bursts on
|
||
boot collapse into a single digest reminder, completed only after delivery.
|
||
- **Rule trace engine** — `ExplainTick`/`ExplainGate` record per-rule
|
||
predicate/gate/selection results each tick; served over IPC (`tick_trace`)
|
||
and rendered at mavweb `/trace` ("why didn't she nudge me").
|
||
- **Web UI** — new `/history` (facts + revert buttons), `/notifications` (nudge
|
||
history), `/trace` pages; nav links on `/dash`; RU/EN cheatsheet toggle in the
|
||
PWA; manifest icons (`icon.svg`). POST `/tools` now requires an in-process
|
||
passkey step-up when WebAuthn is configured.
|
||
- **Revert/undo** — `RevertFact` voids the latest fact for a key (append-only
|
||
void-marker, audit trail intact); exposed at `/api/revert` from `/history`.
|
||
- **Tool scopes** — `scope` column on tools, threaded through propose/enable/UI.
|
||
`DisableTool` raised to AuthStepUp alongside Enable.
|
||
- **Passkey persistence** — mavweb credentials in a JSON file (`-passkey-file`),
|
||
surviving restarts; rollback-on-persist-failure keeps memory and disk in sync.
|
||
- **STT silence gate** — min-duration + RMS floor drop non-speech before whisper
|
||
hallucinates on it (`-min-ms`, `-silence-rms` flags on mavsttd).
|
||
- **Housekeeping** — `db_key.env` gitignored (+`.env.example`), `build-caldav`
|
||
target, zero-timestamp "never" fix on /dash.
|
||
|
||
### Not built yet (ranked by ROI)
|
||
|
||
1. **Multi-user (SPEC item 8)** — deliberately deferred, see the tail.
|
||
|
||
Closed (jul6 follow-ups): `/api/revert` now sits behind the same passkey
|
||
step-up as POST `/tools`; `go.mod` direct deps (`onnxruntime_go`,
|
||
`coder/websocket`, `robfig/cron`) are labeled correctly — `go mod tidy` can't
|
||
run here because it walks the vendored `deps/go` toolchain tree.
|
||
Purge+rotate leaked db key (#12) — investigated and closed: the key was
|
||
**never committed** to git history (gitignored at introduction, no commit
|
||
ever tracked `deploy/db_key.env`), so nothing to scrub. File stays on disk
|
||
and in deploy env by design — at-rest encryption needs it at boot.
|
||
|
||
Done earlier (2026-07-03): **act tool executor, store-backed, full flow**
|
||
(`internal/tool` + `internal/store/tools.go` + `tools` CoreAPI methods).
|
||
- **Execution:** IntentAct runs the matched fn against the store's ENABLED
|
||
allowlist. argv, no shell → STT text can't inject. Live store read, so a
|
||
newly-enabled tool runs without a daemon restart.
|
||
- **proposed→enabled→disabled (SPEC item 5):** an act whose verb isn't enabled is
|
||
scaffolded as a `proposed` tool (maven suggests). A human enables it (fills argv
|
||
+ destructive) on the authed **`mavweb /tools`** page — never voice — and can
|
||
disable it back to `proposed` (kept in the store, won't run). `EnableTool`/
|
||
`DisableTool` sit at `AuthStepUp`; the gate is now **live** via `PasskeySession`,
|
||
so /tools enable requires a passkey assertion at `/auth/passkey` first.
|
||
- **Confirm turn:** a destructive enabled tool replies "выполнить X? да/нет" and
|
||
parks; the next utterance (ru/en yes-no) confirms or cancels (90s TTL).
|
||
- **Config:** `voice.tools` seeds enabled tools at boot (editing mavend.json =
|
||
the human enable act); mavweb enables ad-hoc ones on top.
|
||
- **Russian:** fixed grammar in reply strings + seed files; maven's self-
|
||
reference is feminine ("she") — [[maven-persona-gender]].
|
||
|
||
Also fixed:
|
||
- **HashEmbedder was blind to Cyrillic** (`tokenize` iterated bytes, kept only
|
||
`a-z0-9`) → every RU utterance embedded to the zero vector → cosine 0 across
|
||
all intents → misrouted to `act` (alphabetical tie-break). Now rune-based
|
||
(`unicode.IsLetter`). This was the real cause of "Найди заметку" (a query)
|
||
landing in `notes`; added note-retrieval query seeds too.
|
||
- **Notes are now browsable on `/dash`** — `RecentNotes` plumbed through the
|
||
store + CoreAPI; voice-captured notes were previously only reachable via
|
||
semantic `query`.
|
||
Earlier: notes/query recall, `/dash` monitoring, `wg_handshake` poller (NO-OP).
|
||
|
||
### Gaps — why "voice assistant" is still aspirational (2026-07-06)
|
||
|
||
What separates Maven today from the thing the spec describes. Dealbreakers
|
||
first — these define the category:
|
||
|
||
1. **Always-on listening is code-complete (MVP).** `cmd/mavwaked` captures
|
||
PCM from arecord → energy-based VAD → PushToTalk with `Surface=SurfaceVoice`
|
||
(L0). Gap narrowed: no wake word yet (pure voice-activity trigger; every
|
||
utterance fires). The 30ms frame shape and 16kHz PCM match silero-vad's
|
||
ONNX input exactly, so a wake-word model swap is a local change in vad.go.
|
||
Hardware: the mic lives on a client box (desk PC, pi, etc.) — never the
|
||
homesrv. Deploy action: systemd user unit on whichever box has the mic,
|
||
connects to mavend over wg or local net.
|
||
2. **Conversation is deeper now, still not full dialogue.** The router
|
||
classifies one utterance → one reply, but `internal/dialogue` carries
|
||
context across turns: a 2-min session inherits slots for same-intent
|
||
follow-ups («напомни завтра» → «…позвонить маме»), and cross-intent
|
||
anaphora («запиши что я пил воду» → «когда я это сделал?») now resolves
|
||
RU pronouns (это/он/она/оно/тот/мой + inflections) to the prior turn's
|
||
key for fact-by-key lookup. `Session.History []Turn` is the scaffold for
|
||
real multi-turn. Still missing: LLM-driven dialogue manager (decide
|
||
ask-vs-act), anaphora beyond RU pronouns, single-slot session (single-user
|
||
box). The sub-1B phraser only words replies.
|
||
3. **Latency/shape of a turn.** Clip-based STT (record → upload → whisper →
|
||
route → phrase → piper → play). No streaming either direction, no barge-in;
|
||
every exchange is a full round trip.
|
||
|
||
Capability-class gaps — built but thin:
|
||
|
||
4. **Act surface is a small argv allowlist.** propose→enable works and the
|
||
allowlist now ships a homelab starter set (jul6 task 2 — status/ps/uptime/
|
||
df/free/logs read-only, restart/stop/reboot gated). Still bounded to what's
|
||
seeded; broadening it is config, not code.
|
||
5. **Query answers now cover notes + calendar + weather + general knowledge**
|
||
(jul6 tasks 3/4/5). Calendar querying, keyless Open-Meteo weather, and a
|
||
phraser knowledge-fallback all landed; caveat — general-knowledge quality is
|
||
only as good as the sub-1B phraser, and weather needs `voice.weather.provider`
|
||
set. The cheatsheet and router are now roughly aligned.
|
||
6. **Routing quality depends on the ONNX embedder being configured** — the
|
||
HashEmbedder floor makes RU recall lexical/weak; many commands fall to
|
||
"clarify". `make download-embedder` now fetches the multilingual MiniLM
|
||
model + AGENTS.md documents libonnxruntime setup; `voice.query_min_score`
|
||
is a config knob (default 0.55) so the floor can be tuned without recompile.
|
||
7. **Presence is effectively one signal** (page_heartbeat); desk_active is
|
||
still an undeployed script — "voice when near" routing runs on a guess.
|
||
8. **Long-term memory is now persistent (store-backed), not the spec's chroma.**
|
||
`internal/memory` has a `Store` interface; the daemon now wires
|
||
`store.MemoryStore` (`internal/store/memory.go`) — a **persistent** backend
|
||
in the **same encrypted sqlite db** (survives restarts; recall text inherits
|
||
at-rest encryption, so no plaintext sidecar). Vectors are float32 blobs,
|
||
search is brute-force cosine (fine at single-user scale; ANN is the later
|
||
swap behind the same interface). Notes **and facts** are indexed on capture;
|
||
`IntentQuery` reads it back (after notes-RAG misses, before general-knowledge)
|
||
— fact recall («когда я пил воду?») is its distinct payoff. The in-memory
|
||
impl remains the test/no-store floor. Remaining: an ANN/external index is
|
||
optional-scale, not a gap. Custom TTS voice (kami-picked, replaces the irina
|
||
floor — [[custom-voice-training]]) is still a future item.
|
||
|
||
Ops footnote: voice-over-web verified 2026-07-06 — mavend binds 0.0.0.0:9100
|
||
and mavweb reaches it cross-container at mavend:9100 (nc -z confirmed).
|
||
mavpoll uses network_mode=host to reach localhost services (netdata, kuma).
|
||
|
||
### Future / logged, not now
|
||
|
||
Custom TTS voice training (kami-picked voice, replaces irina floor); listening
|
||
modes 2–3 (meeting-record, ambient-derive).
|
||
|
||
### Services & layout
|
||
|
||
- `mavend` (core, IPC unix socket) — store + loop + phraser; the only key-holder.
|
||
- `mavsttd` / `mavttsd` — STT/TTS worker modules (unix sockets).
|
||
- `mavweb` — PWA bridge (HTTP), `/api/ptt` voice, `/api/signal` presence ingest,
|
||
`/api/ntfy` WS-subscribe config, `/dash` read-only monitoring.
|
||
- `mavpoll` — env poller (netdata/kuma → facts via CoreAPI).
|
||
- `mavcaldav` — CalDAV poller (Radicale → `calendar_busy` + events via CoreAPI).
|
||
- All behind wg + nginx deny-all; no phone-home. CGo only in `mavsttd`.
|
||
- Start/stop: `./start-maven.sh [build]`, `./kill-maven.sh`.
|
||
- Config: `~/.config/maven/mavend.json` (or `mavend.json` in repo root).
|
||
|
||
### Key files
|
||
|
||
- `cmd/mavend/{main,tick,voice}.go` — daemon wiring, loop driver, voice handler
|
||
- `internal/loop/{loop,rules,gather,feedback}.go` — proactive engine
|
||
- `internal/store/` — append-only facts/reminders/nudges/presence/notes
|
||
- `cmd/mavweb/{main.go,dash.html}` — PWA bridge + `/dash` monitoring
|
||
- `internal/router/{classifier,slots,stage0}.go` — reactive routing + slot parse
|
||
- `internal/delivery/` — dispatcher + ntfy/telegram/voice sinks
|
||
- `internal/auth/` — scope/gate/policy; `FloorEnrollment` (same-uid = device
|
||
trust) + `webauthn.PasskeySession` (real step-up for L3)
|
||
- `internal/webauthn/`, `cmd/mavweb/webauthn.go` — passkey register/assert
|
||
- `cmd/mavcaldav/`, `cmd/mavpoll/`, `scripts/desk-active.sh` — env producers
|
||
|
||
### Why multi-user (SPEC item 8) is deferred
|
||
|
||
Not neglect — the one item where doing nothing now beats doing something:
|
||
|
||
- **No second user exists yet** (the "gf phase"). Building per-user partitioning
|
||
now means code exercised by zero users and validated by nobody — YAGNI.
|
||
- **The append-only schema makes it a migration, not a rewrite.** No row is ever
|
||
mutated, so adding `facts/notes/reminders.user_id` later is add-columns +
|
||
backfill-to-"kami" — no reshaping, no dual-write window. Deferral is cheap.
|
||
- **The hard part is speaker attribution, and it needs the second voice.** A
|
||
voice-print discriminator (kami vs gf vs unknown) can't be trained or tuned
|
||
with one voice in the house. Plumbing before the model is pipe with no water.
|
||
- **It's fenced deliberately** (`DO NOT TOUCH THIS PHASE` in `DESIGN.md` § Users) so an
|
||
autonomous agent doesn't add `user_id` columns while touching the store and
|
||
commit us to a schema before the constraints that shape it exist.
|