diff --git a/PROGRESS.md b/PROGRESS.md index 8487e4b..3f710a1 100644 --- a/PROGRESS.md +++ b/PROGRESS.md @@ -86,8 +86,6 @@ tests, `-race` in `make test`. - **`desk_active`** (strongest presence signal) — `scripts/desk-active.sh` runs on the **desk PC** (hypridle-gated systemd timer), posts over wg to mavweb. -- **Kuma `service_down`** — needs an API key created in Kuma → Settings → API Keys, - passed to `mavpoll -kuma-key`. - **`mavwaked`** (always-on listening) — needs a systemd user unit on a client box (desk PC, pi, etc.) where the mic is attached. Connects to mavend over wg or local net via `-addr`. Deferred until a client box is wired with a mic. @@ -107,6 +105,13 @@ Caveats / gotchas: -wg` flag exists (defaults `""`) and could later back the spec's "away override" by flipping the sign; as a presence-*here* signal it's inverted. desk_active + page_heartbeat cover home presence. +- **Cold-start unlock tests are missing** — the key wrap/unwrap code + (`internal/webauthn/keywrap.go`) and locked-mode IPC gating (`cmd/mavend/main.go`) + are correct but have **zero test coverage**. The roadmap (item 2.1) required + three new test cases (wrap/unwrap round-trip, wrong-cred unwrap fails, + locked-mode IPC rejects non-unlock methods); none were written. `make test` + is green by omission. Write these before relying on the cold-start path with + real keys. ### Done since last revision (overnight-jul6, 2026-07-06) @@ -125,6 +130,41 @@ Since then (**2026-07-06, second session**): `Makefile` `build-waked` target. Runs on client boxes (not docker/homesrv) via systemd user unit; connects to mavend over wg or local net. +Since then (**2026-07-06, third session** — roadmap execution agent): + +- **Cold-start unlock (ROADMAP 2.1)** — the at-rest AES key is now wrapped + (HKDF-SHA256 + AES-256-GCM, stdlib-only — no `x/crypto` dep) with the passkey + credential's public key and persisted to disk. At boot, if a wrapped key file + exists AND no env key is set, mavend starts **locked**: the IPC server runs + but `srv.Check` rejects everything except `MethodAssertStepUp` + + `MethodUnlock`. A passkey assertion at `/auth/passkey` calls `MethodUnlock` + with the credential's public key → unwraps the blob → opens the store → wires + voice/loop/delivery → `srv.SetAPI` swaps the locked stub for the real + CoreAPI. mavweb's `RegisterFinish` wraps the env key on enrollment; + `AssertFinish` calls `Unlock` on assertion. Env-key fallback preserved + (dev/CI path unchanged). **Test gap:** the roadmap required three new test + cases (wrap/unwrap round-trip, wrong-cred unwrap fails, locked-mode IPC + rejects non-unlock methods) — none were written. The code is correct but + untested; `make test` is green by omission, not coverage. +- **Conversation depth (ROADMAP 3.2)** — cross-intent anaphora + fact-by-key + lookup. `AnaphoraResolver` in `router/slots.go` detects RU pronouns + (это/он/она/оно/тот/мой + inflected forms). `followUpMerge` now handles + three cases: same-intent slot inheritance (existing), cross-intent anaphora + (Query/Fact/Reminder after a Fact with a pronoun inherits the prior key + + time), and query-after-fact (a query following a fact inherits the key for + fact-by-key lookup). `Session.History []Turn` added as the multi-turn + scaffold (capped at 4). 7 new test cases including the exact done-when + scenarios (anaphora query-after-fact, three-turn break, explicit-key-wins). +- **Routing quality + persona (ROADMAP 4.1/4.4)** — `QueryMinScore` is now a + config knob (`voice.query_min_score`, default 0.55) instead of a hardcoded + const. `make download-embedder` fetches Xenova/paraphrase-multilingual- + MiniLM-L12-v2 (~90MB ONNX) + tokenizer; AGENTS.md documents the embedder + + libonnxruntime setup. `Persona` field in `VoiceConfig` prepends to every + LLM system prompt (nudge phrasing, note queries, general knowledge); empty + = current hardcoded feminine-gendered Russian persona. Also fixed two + pre-existing data races found by `-race`: `voice/server.go` wg.Add vs + wg.Wait (accept mutex), `mavweb/server.go` s.api field (atomic.Value). + - **Calendar querying (task 3)** — "что у меня завтра?" now answers from the CalDAV facts the poller already writes. Added `store.CalendarEvents(from,to)`, a RU date-scope parser («сегодня»/«завтра») in `router/slots.go`, and an @@ -204,10 +244,7 @@ The overnight session (`SESSION-05-07-2026.md`, 25 tasks) closed the previous ### Not built yet (ranked by ROI) -1. **Cold-start unlock** — the at-rest key still comes from env/config; the - passkey→key L3 dance is a documented seam, not a feature. Until then the key - sits in the container env. -2. **Multi-user (SPEC item 8)** — deliberately deferred, see the tail. +1. **Multi-user (SPEC item 8)** — deliberately deferred, see the tail. Closed (jul6 follow-ups): `/api/revert` now sits behind the same passkey step-up as POST `/tools`; `go.mod` direct deps (`onnxruntime_go`, @@ -260,13 +297,16 @@ first — these define the category: Hardware: the mic lives on a client box (desk PC, pi, etc.) — never the homesrv. Deploy action: systemd user unit on whichever box has the mic, connects to mavend over wg or local net. -2. **Conversation is thin, not absent.** The router still classifies one - utterance → one reply and there's no anaphora resolution or LLM-driven - dialogue. But `internal/dialogue` is now **wired** (jul6 task 6 + follow-up): - a 2-min session carries slots across turns, so a same-intent follow-up - («напомни завтра» → «…позвонить маме») inherits the earlier time. Bounded to - same-intent gap-filling — cross-intent anaphora and real multi-turn dialogue - are still future. The sub-1B phraser only words replies. +2. **Conversation is deeper now, still not full dialogue.** The router + classifies one utterance → one reply, but `internal/dialogue` carries + context across turns: a 2-min session inherits slots for same-intent + follow-ups («напомни завтра» → «…позвонить маме»), and cross-intent + anaphora («запиши что я пил воду» → «когда я это сделал?») now resolves + RU pronouns (это/он/она/оно/тот/мой + inflections) to the prior turn's + key for fact-by-key lookup. `Session.History []Turn` is the scaffold for + real multi-turn. Still missing: LLM-driven dialogue manager (decide + ask-vs-act), anaphora beyond RU pronouns, single-slot session (single-user + box). The sub-1B phraser only words replies. 3. **Latency/shape of a turn.** Clip-based STT (record → upload → whisper → route → phrase → piper → play). No streaming either direction, no barge-in; every exchange is a full round trip. @@ -284,7 +324,9 @@ Capability-class gaps — built but thin: set. The cheatsheet and router are now roughly aligned. 6. **Routing quality depends on the ONNX embedder being configured** — the HashEmbedder floor makes RU recall lexical/weak; many commands fall to - "clarify". + "clarify". `make download-embedder` now fetches the multilingual MiniLM + model + AGENTS.md documents libonnxruntime setup; `voice.query_min_score` + is a config knob (default 0.55) so the floor can be tuned without recompile. 7. **Presence is effectively one signal** (page_heartbeat); desk_active is still an undeployed script — "voice when near" routing runs on a guess. 8. **Long-term memory is now persistent (store-backed), not the spec's chroma.** @@ -297,8 +339,8 @@ Capability-class gaps — built but thin: `IntentQuery` reads it back (after notes-RAG misses, before general-knowledge) — fact recall («когда я пил воду?») is its distinct payoff. The in-memory impl remains the test/no-store floor. Remaining: an ANN/external index is - optional-scale, not a gap. Persona prompt and custom TTS voice (kami-picked, - replaces the irina floor — [[custom-voice-training]]) are still future items. + optional-scale, not a gap. Custom TTS voice (kami-picked, replaces the irina + floor — [[custom-voice-training]]) is still a future item. Ops footnote: voice-over-web verified 2026-07-06 — mavend binds 0.0.0.0:9100 and mavweb reaches it cross-container at mavend:9100 (nc -z confirmed). @@ -306,8 +348,8 @@ mavpoll uses network_mode=host to reach localhost services (netdata, kuma). ### Future / logged, not now -Personality prompt; custom TTS voice training (kami-picked voice, replaces irina -floor); listening modes 2–3 (meeting-record, ambient-derive). +Custom TTS voice training (kami-picked voice, replaces irina floor); listening +modes 2–3 (meeting-record, ambient-derive). ### Services & layout diff --git a/ROADMAP.md b/ROADMAP.md index 4a60f99..b50beab 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -27,6 +27,11 @@ that unlock already-built features. Highest ROI per minute on the list. ### 1.1 Kuma API key for `service_down` polling +**Status:** done (`eda434f`, 2026-07-06). Key `uk5_mavpoll-key` created, wired +into mavpoll. Agent caught a real bug: kuma expects the key as the *password* +field, not username. mavpoll switched to `network_mode: host` (compose bridge +couldn't reach localhost netdata/kuma). + **Vikunja:** #16 (prio 2) **Est:** ~5 min **Deps:** none @@ -58,6 +63,9 @@ The ServiceDownRule then fires a sev4 nudge through the normal dispatcher. ### 1.2 Voice bind verification + stale comment fix +**Status:** done (`eda434f`, 2026-07-06). Stale comment replaced with verified +note. `nc -z` confirmed mavweb reaches `mavend:9100` cross-container. + **Vikunja:** #18 (prio 2) **Est:** ~30 min **Deps:** none @@ -90,6 +98,12 @@ fixed. Update PROGRESS.md "Ops footnote" (line 284-286) to "verified." ### 1.3 Deploy desk_active presence script on desk PC +**Status:** not done. Script exists (`scripts/desk-active.sh`, complete) but +the systemd user timer + hypridle listener were not installed on the desk PC +(`linux`). This is an operator action on a different machine — the agent +couldn't reach it from the homesrv context. 0 facts ever written; `/dash` +presence still runs on `page_heartbeat` alone. + **Vikunja:** #15 (prio 3) **Est:** ~1 h **Deps:** none (the script is complete; this is a workstation deploy) @@ -153,6 +167,16 @@ Presence is now 2 signals (desk_active + page_heartbeat) instead of 1. ### 2.1 Cold-start unlock (passkey → L3 key seam) +**Status:** code done, **tests missing** (`b0932a1` + `15fe7bb`, 2026-07-06). +`internal/webauthn/keywrap.go` (HKDF-SHA256 + AES-256-GCM, stdlib-only), +locked-mode boot in `cmd/mavend/main.go` (`lockedAPI` stub, `srv.Check` +allowlist), `MethodStoreEncryptionKey`/`MethodUnlock` IPC, mavweb +`RegisterFinish` wraps + `AssertFinish` unlocks. Env-key fallback preserved. +**Gap:** the roadmap's done-when #4 required three new test cases +(wrap/unwrap round-trip, wrong-cred unwrap fails, locked-mode IPC rejects +non-unlock methods) — none were written. `make test` is green by omission, +not coverage. Write these before relying on the cold-start path with real keys. + **Vikunja:** #14 (prio 2) **Est:** ~1 day **Deps:** none (the seam is documented in code comments, just not wired) @@ -276,6 +300,15 @@ architecture item, not a config tweak. ### 3.1 Always-on listening (wake word + ambient capture) +**Status:** MVP (`e57647c` + `5d1850c`, 2026-07-06). `cmd/mavwaked/` (main + +vad + vad_test, 10 `-race` tests) ships as **energy-VAD only, no wake-word +model** — every utterance fires. `SurfaceVoice` (L0) caps it. 30ms/16kHz +frame shape matches silero-vad ONNX input 1:1, so the wake-word swap is a +local change in `vad.go`. Hardware topology settled: client box with mic +(desk PC / pi), not homesrv. mavwaked is a client binary (systemd user unit), +not a docker daemon — the agent first added it to Docker, then reverted +(`5d1850c`). Remaining: wake-word model (openWakeWord / silero-vad ONNX). + **Est:** 1-2 weeks + hardware **Deps:** a capture device (USB mic or a dedicated ESP32-S3 box) @@ -360,6 +393,14 @@ device); the hardware changes the deploy. ### 3.2 Conversation depth (multi-turn dialogue) +**Status:** done (`05236ad`, 2026-07-06). Path 1 (rule-based deepening) +implemented: `AnaphoraResolver` in `router/slots.go` (RU pronouns: +это/он/она/оно/тот/мой + inflections), `followUpMerge` extended for +cross-intent (Query/Fact/Reminder after Fact with anaphora inherits key + +time), `Session.History []Turn` added, fact-by-key lookup in `applyAction`. +7 new test cases including the exact done-when scenarios. Path 2 (LLM +dialogue manager) remains future — the sub-1B phraser can't drive it. + **Est:** 3-5 days **Deps:** none (the dialogue scaffold is wired; this deepens it) @@ -425,6 +466,9 @@ test` green with the new cases. ### 3.3 Latency / streaming +**Status:** not started. Correctly deferred — the roadmap itself flagged this +as "most likely to be deferred" and lowest-ROI of the dealbreakers. + **Est:** 1-2 weeks **Deps:** none (architecture rework) @@ -495,6 +539,12 @@ better but doesn't change what maven *is*. ### 4.1 Routing quality (dev embedder) +**Status:** done (`b7eb53a`, 2026-07-06). `make download-embedder` fetches +Xenova/paraphrase-multilingual-MiniLM-L12-v2 (~90MB ONNX) + tokenizer. +AGENTS.md documents embedder + libonnxruntime setup. `queryMinScore` is now +configurable (`voice.query_min_score`, default 0.55) instead of a hardcoded +const. + **Est:** 2-4 h **Deps:** none @@ -528,6 +578,11 @@ correctly, not to "clarify"). `make test` green. ### 4.2 Act surface broadening +**Status:** not a code item — operator config. The seeded homelab set +(status/ps/uptime/df/free/logs read-only, restart/stop/reboot gated) ships +in `deploy/mavend.json`. Broadening to home-automation/media/comms is +editing JSON, not code. + **Est:** ongoing config **Deps:** none @@ -550,6 +605,10 @@ after adding the tool to `mavend.json` and the act seed file. ### 4.3 LTM ANN (approximate nearest neighbor) +**Status:** deferred (correctly). The `memory.Store` interface is the swap +point; brute-force cosine is sub-ms at single-user scale. Not started until +note+fact count exceeds ~10k and `Search` latency shows up in profiles. + **Est:** ~1 day **Deps:** none (the interface is the swap point) @@ -574,6 +633,10 @@ passes the existing `memory_test.go` suite and shows <1ms latency at ### 4.4 Persona prompt +**Status:** done (`b7eb53a`, 2026-07-06). `Persona` field in `VoiceConfig`, +`llmphraser` prepends to `systemPrompt()` + `querySystemPrompt()`. Empty = +current hardcoded feminine-gendered Russian persona (backward compat). + **Est:** 2-4 h **Deps:** none @@ -616,6 +679,10 @@ without recompiling. Empty = current behavior. `make test` green. ### 4.5 Custom TTS voice +**Status:** not started. Mostly operator work (record ~50-100 clips, train a +piper model). The code already supports it — `-model` flag takes any piper +voice file, `VoiceConfig.Tts.Voice` names it. + **Est:** ~1 day + training time **Deps:** none (piper supports custom voices) @@ -651,10 +718,10 @@ The code already supports it — it's a model-file swap. ### 5.1 Multi-user (SPEC item 8) -**Status:** `SPEC.md:264-270` fences this explicitly: "DO NOT TOUCH THIS -PHASE." No second user exists. The append-only schema (`schema.sql`) -makes it a migration (add `user_id` columns + backfill to "kami"), not a -rewrite. Speaker attribution needs the second voice to train against. +**Status:** deferred by design. SPEC fences this explicitly (`DO NOT TOUCH +THIS PHASE`). No second user exists. The append-only schema makes it a +migration (add `user_id` columns + backfill), not a rewrite. Speaker +attribution needs the second voice to train against. **When to revisit:** when a second person is actually in the house and using maven. Not before. @@ -668,23 +735,25 @@ exist. ## Summary table -| # | Item | Prio | Est | Type | Deps | -|---|------|------|-----|------|------| -| 1.1 | Kuma API key | P1 | 5m | ops | — | -| 1.2 | Voice bind verify + comment fix | P1 | 30m | ops | — | -| 1.3 | desk_active deploy | P1 | 1h | ops | — | -| 2.1 | Cold-start unlock | P2 | 1d | code | — | -| 3.1 | Always-on listening | P3 | 1-2w | code+hw | hardware decision | -| 3.2 | Conversation depth | P3 | 3-5d | code | — | -| 3.3 | Latency/streaming | P3 | 1-2w | code | — | -| 4.1 | Routing quality (dev embedder) | P4 | 2-4h | code+docs | — | -| 4.2 | Act surface | P4 | ongoing | config | — | -| 4.3 | LTM ANN | P4 | 1d | code | scale problem | -| 4.4 | Persona prompt | P4 | 2-4h | code | — | -| 4.5 | Custom TTS voice | P4 | 1d+train | ops | — | -| 5.1 | Multi-user | P5 | deferred | — | second user | +| # | Item | Prio | Est | Type | Deps | Status | +|---|------|------|-----|------|------|--------| +| 1.1 | Kuma API key | P1 | 5m | ops | — | done `eda434f` | +| 1.2 | Voice bind verify + comment fix | P1 | 30m | ops | — | done `eda434f` | +| 1.3 | desk_active deploy | P1 | 1h | ops | — | not done (operator action on `linux`) | +| 2.1 | Cold-start unlock | P2 | 1d | code | — | code done `b0932a1`+`15fe7bb`, **tests missing** | +| 3.1 | Always-on listening | P3 | 1-2w | code+hw | hardware decision | MVP `e57647c` (VAD only, no wake word) | +| 3.2 | Conversation depth | P3 | 3-5d | code | — | done `05236ad` | +| 3.3 | Latency/streaming | P3 | 1-2w | code | — | not started (deferred) | +| 4.1 | Routing quality (dev embedder) | P4 | 2-4h | code+docs | — | done `b7eb53a` | +| 4.2 | Act surface | P4 | ongoing | config | — | not a code item (config) | +| 4.3 | LTM ANN | P4 | 1d | code | scale problem | deferred (scale) | +| 4.4 | Persona prompt | P4 | 2-4h | code | — | done `b7eb53a` | +| 4.5 | Custom TTS voice | P4 | 1d+train | ops | — | not started (ops) | +| 5.1 | Multi-user | P5 | deferred | — | second user | deferred by design | -**Recommended order:** 1.1 → 1.2 → 1.3 (today, ~2h) → 2.1 (security, -~1d) → 4.1 + 4.4 (quick depth, same day) → 3.2 (conversation, ~3-5d) → -3.1 (always-on, needs hardware pick first) → 3.3 (streaming, lowest -ROI of the dealbreakers) → 4.5 (custom voice, when recording is done). +**Remaining work (in priority order):** +1. **2.1 tests** — write the 3 missing keywrap/locked-mode test cases (~30 min) +2. **1.3 desk_active** — install systemd timer + hypridle on `linux` (~1h, your hands) +3. **3.1 wake word** — swap energy-VAD for silero-vad/openWakeWord ONNX in `vad.go` +4. **3.3 streaming** — lowest ROI, defer until 3.1 is real +5. **4.5 custom voice** — when recording is done