progress+roadmap: sync with third-session work, mark stale items
PROGRESS.md: - remove Kuma from 'Wired but needs deploy' (key wired ineda434f) - remove cold-start unlock from 'Not built yet' (built in b0932a1+15fe7bb) - add third-session block: cold-start unlock, conversation depth, routing+persona — with the cold-start test gap called out - update gap #2: anaphora + cross-intent landed (05236ad) - update gap #6: note download-embedder + query_min_score knob - remove 'Personality prompt' from Future (landed inb7eb53a) - add caveat: cold-start unlock tests missing (3 required cases absent) ROADMAP.md: - add Status line under every item header with commit SHAs + verdict - summary table gains a Status column - replace stale 'Recommended order' with 'Remaining work' priority list
This commit is contained in:
kami
+60
-18
@@ -86,8 +86,6 @@ tests, `-race` in `make test`.
|
||||
|
||||
- **`desk_active`** (strongest presence signal) — `scripts/desk-active.sh` runs
|
||||
on the **desk PC** (hypridle-gated systemd timer), posts over wg to mavweb.
|
||||
- **Kuma `service_down`** — needs an API key created in Kuma → Settings → API Keys,
|
||||
passed to `mavpoll -kuma-key`.
|
||||
- **`mavwaked`** (always-on listening) — needs a systemd user unit on a client
|
||||
box (desk PC, pi, etc.) where the mic is attached. Connects to mavend over wg
|
||||
or local net via `-addr`. Deferred until a client box is wired with a mic.
|
||||
@@ -107,6 +105,13 @@ Caveats / gotchas:
|
||||
-wg` flag exists (defaults `""`) and could later back the spec's "away override"
|
||||
by flipping the sign; as a presence-*here* signal it's inverted. desk_active +
|
||||
page_heartbeat cover home presence.
|
||||
- **Cold-start unlock tests are missing** — the key wrap/unwrap code
|
||||
(`internal/webauthn/keywrap.go`) and locked-mode IPC gating (`cmd/mavend/main.go`)
|
||||
are correct but have **zero test coverage**. The roadmap (item 2.1) required
|
||||
three new test cases (wrap/unwrap round-trip, wrong-cred unwrap fails,
|
||||
locked-mode IPC rejects non-unlock methods); none were written. `make test`
|
||||
is green by omission. Write these before relying on the cold-start path with
|
||||
real keys.
|
||||
|
||||
### Done since last revision (overnight-jul6, 2026-07-06)
|
||||
|
||||
@@ -125,6 +130,41 @@ Since then (**2026-07-06, second session**):
|
||||
`Makefile` `build-waked` target. Runs on client boxes (not docker/homesrv)
|
||||
via systemd user unit; connects to mavend over wg or local net.
|
||||
|
||||
Since then (**2026-07-06, third session** — roadmap execution agent):
|
||||
|
||||
- **Cold-start unlock (ROADMAP 2.1)** — the at-rest AES key is now wrapped
|
||||
(HKDF-SHA256 + AES-256-GCM, stdlib-only — no `x/crypto` dep) with the passkey
|
||||
credential's public key and persisted to disk. At boot, if a wrapped key file
|
||||
exists AND no env key is set, mavend starts **locked**: the IPC server runs
|
||||
but `srv.Check` rejects everything except `MethodAssertStepUp` +
|
||||
`MethodUnlock`. A passkey assertion at `/auth/passkey` calls `MethodUnlock`
|
||||
with the credential's public key → unwraps the blob → opens the store → wires
|
||||
voice/loop/delivery → `srv.SetAPI` swaps the locked stub for the real
|
||||
CoreAPI. mavweb's `RegisterFinish` wraps the env key on enrollment;
|
||||
`AssertFinish` calls `Unlock` on assertion. Env-key fallback preserved
|
||||
(dev/CI path unchanged). **Test gap:** the roadmap required three new test
|
||||
cases (wrap/unwrap round-trip, wrong-cred unwrap fails, locked-mode IPC
|
||||
rejects non-unlock methods) — none were written. The code is correct but
|
||||
untested; `make test` is green by omission, not coverage.
|
||||
- **Conversation depth (ROADMAP 3.2)** — cross-intent anaphora + fact-by-key
|
||||
lookup. `AnaphoraResolver` in `router/slots.go` detects RU pronouns
|
||||
(это/он/она/оно/тот/мой + inflected forms). `followUpMerge` now handles
|
||||
three cases: same-intent slot inheritance (existing), cross-intent anaphora
|
||||
(Query/Fact/Reminder after a Fact with a pronoun inherits the prior key +
|
||||
time), and query-after-fact (a query following a fact inherits the key for
|
||||
fact-by-key lookup). `Session.History []Turn` added as the multi-turn
|
||||
scaffold (capped at 4). 7 new test cases including the exact done-when
|
||||
scenarios (anaphora query-after-fact, three-turn break, explicit-key-wins).
|
||||
- **Routing quality + persona (ROADMAP 4.1/4.4)** — `QueryMinScore` is now a
|
||||
config knob (`voice.query_min_score`, default 0.55) instead of a hardcoded
|
||||
const. `make download-embedder` fetches Xenova/paraphrase-multilingual-
|
||||
MiniLM-L12-v2 (~90MB ONNX) + tokenizer; AGENTS.md documents the embedder +
|
||||
libonnxruntime setup. `Persona` field in `VoiceConfig` prepends to every
|
||||
LLM system prompt (nudge phrasing, note queries, general knowledge); empty
|
||||
= current hardcoded feminine-gendered Russian persona. Also fixed two
|
||||
pre-existing data races found by `-race`: `voice/server.go` wg.Add vs
|
||||
wg.Wait (accept mutex), `mavweb/server.go` s.api field (atomic.Value).
|
||||
|
||||
- **Calendar querying (task 3)** — "что у меня завтра?" now answers from the
|
||||
CalDAV facts the poller already writes. Added `store.CalendarEvents(from,to)`,
|
||||
a RU date-scope parser («сегодня»/«завтра») in `router/slots.go`, and an
|
||||
@@ -204,10 +244,7 @@ The overnight session (`SESSION-05-07-2026.md`, 25 tasks) closed the previous
|
||||
|
||||
### Not built yet (ranked by ROI)
|
||||
|
||||
1. **Cold-start unlock** — the at-rest key still comes from env/config; the
|
||||
passkey→key L3 dance is a documented seam, not a feature. Until then the key
|
||||
sits in the container env.
|
||||
2. **Multi-user (SPEC item 8)** — deliberately deferred, see the tail.
|
||||
1. **Multi-user (SPEC item 8)** — deliberately deferred, see the tail.
|
||||
|
||||
Closed (jul6 follow-ups): `/api/revert` now sits behind the same passkey
|
||||
step-up as POST `/tools`; `go.mod` direct deps (`onnxruntime_go`,
|
||||
@@ -260,13 +297,16 @@ first — these define the category:
|
||||
Hardware: the mic lives on a client box (desk PC, pi, etc.) — never the
|
||||
homesrv. Deploy action: systemd user unit on whichever box has the mic,
|
||||
connects to mavend over wg or local net.
|
||||
2. **Conversation is thin, not absent.** The router still classifies one
|
||||
utterance → one reply and there's no anaphora resolution or LLM-driven
|
||||
dialogue. But `internal/dialogue` is now **wired** (jul6 task 6 + follow-up):
|
||||
a 2-min session carries slots across turns, so a same-intent follow-up
|
||||
(«напомни завтра» → «…позвонить маме») inherits the earlier time. Bounded to
|
||||
same-intent gap-filling — cross-intent anaphora and real multi-turn dialogue
|
||||
are still future. The sub-1B phraser only words replies.
|
||||
2. **Conversation is deeper now, still not full dialogue.** The router
|
||||
classifies one utterance → one reply, but `internal/dialogue` carries
|
||||
context across turns: a 2-min session inherits slots for same-intent
|
||||
follow-ups («напомни завтра» → «…позвонить маме»), and cross-intent
|
||||
anaphora («запиши что я пил воду» → «когда я это сделал?») now resolves
|
||||
RU pronouns (это/он/она/оно/тот/мой + inflections) to the prior turn's
|
||||
key for fact-by-key lookup. `Session.History []Turn` is the scaffold for
|
||||
real multi-turn. Still missing: LLM-driven dialogue manager (decide
|
||||
ask-vs-act), anaphora beyond RU pronouns, single-slot session (single-user
|
||||
box). The sub-1B phraser only words replies.
|
||||
3. **Latency/shape of a turn.** Clip-based STT (record → upload → whisper →
|
||||
route → phrase → piper → play). No streaming either direction, no barge-in;
|
||||
every exchange is a full round trip.
|
||||
@@ -284,7 +324,9 @@ Capability-class gaps — built but thin:
|
||||
set. The cheatsheet and router are now roughly aligned.
|
||||
6. **Routing quality depends on the ONNX embedder being configured** — the
|
||||
HashEmbedder floor makes RU recall lexical/weak; many commands fall to
|
||||
"clarify".
|
||||
"clarify". `make download-embedder` now fetches the multilingual MiniLM
|
||||
model + AGENTS.md documents libonnxruntime setup; `voice.query_min_score`
|
||||
is a config knob (default 0.55) so the floor can be tuned without recompile.
|
||||
7. **Presence is effectively one signal** (page_heartbeat); desk_active is
|
||||
still an undeployed script — "voice when near" routing runs on a guess.
|
||||
8. **Long-term memory is now persistent (store-backed), not the spec's chroma.**
|
||||
@@ -297,8 +339,8 @@ Capability-class gaps — built but thin:
|
||||
`IntentQuery` reads it back (after notes-RAG misses, before general-knowledge)
|
||||
— fact recall («когда я пил воду?») is its distinct payoff. The in-memory
|
||||
impl remains the test/no-store floor. Remaining: an ANN/external index is
|
||||
optional-scale, not a gap. Persona prompt and custom TTS voice (kami-picked,
|
||||
replaces the irina floor — [[custom-voice-training]]) are still future items.
|
||||
optional-scale, not a gap. Custom TTS voice (kami-picked, replaces the irina
|
||||
floor — [[custom-voice-training]]) is still a future item.
|
||||
|
||||
Ops footnote: voice-over-web verified 2026-07-06 — mavend binds 0.0.0.0:9100
|
||||
and mavweb reaches it cross-container at mavend:9100 (nc -z confirmed).
|
||||
@@ -306,8 +348,8 @@ mavpoll uses network_mode=host to reach localhost services (netdata, kuma).
|
||||
|
||||
### Future / logged, not now
|
||||
|
||||
Personality prompt; custom TTS voice training (kami-picked voice, replaces irina
|
||||
floor); listening modes 2–3 (meeting-record, ambient-derive).
|
||||
Custom TTS voice training (kami-picked voice, replaces irina floor); listening
|
||||
modes 2–3 (meeting-record, ambient-derive).
|
||||
|
||||
### Services & layout
|
||||
|
||||
|
||||
+92
-23
@@ -27,6 +27,11 @@ that unlock already-built features. Highest ROI per minute on the list.
|
||||
|
||||
### 1.1 Kuma API key for `service_down` polling
|
||||
|
||||
**Status:** done (`eda434f`, 2026-07-06). Key `uk5_mavpoll-key` created, wired
|
||||
into mavpoll. Agent caught a real bug: kuma expects the key as the *password*
|
||||
field, not username. mavpoll switched to `network_mode: host` (compose bridge
|
||||
couldn't reach localhost netdata/kuma).
|
||||
|
||||
**Vikunja:** #16 (prio 2)
|
||||
**Est:** ~5 min
|
||||
**Deps:** none
|
||||
@@ -58,6 +63,9 @@ The ServiceDownRule then fires a sev4 nudge through the normal dispatcher.
|
||||
|
||||
### 1.2 Voice bind verification + stale comment fix
|
||||
|
||||
**Status:** done (`eda434f`, 2026-07-06). Stale comment replaced with verified
|
||||
note. `nc -z` confirmed mavweb reaches `mavend:9100` cross-container.
|
||||
|
||||
**Vikunja:** #18 (prio 2)
|
||||
**Est:** ~30 min
|
||||
**Deps:** none
|
||||
@@ -90,6 +98,12 @@ fixed. Update PROGRESS.md "Ops footnote" (line 284-286) to "verified."
|
||||
|
||||
### 1.3 Deploy desk_active presence script on desk PC
|
||||
|
||||
**Status:** not done. Script exists (`scripts/desk-active.sh`, complete) but
|
||||
the systemd user timer + hypridle listener were not installed on the desk PC
|
||||
(`linux`). This is an operator action on a different machine — the agent
|
||||
couldn't reach it from the homesrv context. 0 facts ever written; `/dash`
|
||||
presence still runs on `page_heartbeat` alone.
|
||||
|
||||
**Vikunja:** #15 (prio 3)
|
||||
**Est:** ~1 h
|
||||
**Deps:** none (the script is complete; this is a workstation deploy)
|
||||
@@ -153,6 +167,16 @@ Presence is now 2 signals (desk_active + page_heartbeat) instead of 1.
|
||||
|
||||
### 2.1 Cold-start unlock (passkey → L3 key seam)
|
||||
|
||||
**Status:** code done, **tests missing** (`b0932a1` + `15fe7bb`, 2026-07-06).
|
||||
`internal/webauthn/keywrap.go` (HKDF-SHA256 + AES-256-GCM, stdlib-only),
|
||||
locked-mode boot in `cmd/mavend/main.go` (`lockedAPI` stub, `srv.Check`
|
||||
allowlist), `MethodStoreEncryptionKey`/`MethodUnlock` IPC, mavweb
|
||||
`RegisterFinish` wraps + `AssertFinish` unlocks. Env-key fallback preserved.
|
||||
**Gap:** the roadmap's done-when #4 required three new test cases
|
||||
(wrap/unwrap round-trip, wrong-cred unwrap fails, locked-mode IPC rejects
|
||||
non-unlock methods) — none were written. `make test` is green by omission,
|
||||
not coverage. Write these before relying on the cold-start path with real keys.
|
||||
|
||||
**Vikunja:** #14 (prio 2)
|
||||
**Est:** ~1 day
|
||||
**Deps:** none (the seam is documented in code comments, just not wired)
|
||||
@@ -276,6 +300,15 @@ architecture item, not a config tweak.
|
||||
|
||||
### 3.1 Always-on listening (wake word + ambient capture)
|
||||
|
||||
**Status:** MVP (`e57647c` + `5d1850c`, 2026-07-06). `cmd/mavwaked/` (main +
|
||||
vad + vad_test, 10 `-race` tests) ships as **energy-VAD only, no wake-word
|
||||
model** — every utterance fires. `SurfaceVoice` (L0) caps it. 30ms/16kHz
|
||||
frame shape matches silero-vad ONNX input 1:1, so the wake-word swap is a
|
||||
local change in `vad.go`. Hardware topology settled: client box with mic
|
||||
(desk PC / pi), not homesrv. mavwaked is a client binary (systemd user unit),
|
||||
not a docker daemon — the agent first added it to Docker, then reverted
|
||||
(`5d1850c`). Remaining: wake-word model (openWakeWord / silero-vad ONNX).
|
||||
|
||||
**Est:** 1-2 weeks + hardware
|
||||
**Deps:** a capture device (USB mic or a dedicated ESP32-S3 box)
|
||||
|
||||
@@ -360,6 +393,14 @@ device); the hardware changes the deploy.
|
||||
|
||||
### 3.2 Conversation depth (multi-turn dialogue)
|
||||
|
||||
**Status:** done (`05236ad`, 2026-07-06). Path 1 (rule-based deepening)
|
||||
implemented: `AnaphoraResolver` in `router/slots.go` (RU pronouns:
|
||||
это/он/она/оно/тот/мой + inflections), `followUpMerge` extended for
|
||||
cross-intent (Query/Fact/Reminder after Fact with anaphora inherits key +
|
||||
time), `Session.History []Turn` added, fact-by-key lookup in `applyAction`.
|
||||
7 new test cases including the exact done-when scenarios. Path 2 (LLM
|
||||
dialogue manager) remains future — the sub-1B phraser can't drive it.
|
||||
|
||||
**Est:** 3-5 days
|
||||
**Deps:** none (the dialogue scaffold is wired; this deepens it)
|
||||
|
||||
@@ -425,6 +466,9 @@ test` green with the new cases.
|
||||
|
||||
### 3.3 Latency / streaming
|
||||
|
||||
**Status:** not started. Correctly deferred — the roadmap itself flagged this
|
||||
as "most likely to be deferred" and lowest-ROI of the dealbreakers.
|
||||
|
||||
**Est:** 1-2 weeks
|
||||
**Deps:** none (architecture rework)
|
||||
|
||||
@@ -495,6 +539,12 @@ better but doesn't change what maven *is*.
|
||||
|
||||
### 4.1 Routing quality (dev embedder)
|
||||
|
||||
**Status:** done (`b7eb53a`, 2026-07-06). `make download-embedder` fetches
|
||||
Xenova/paraphrase-multilingual-MiniLM-L12-v2 (~90MB ONNX) + tokenizer.
|
||||
AGENTS.md documents embedder + libonnxruntime setup. `queryMinScore` is now
|
||||
configurable (`voice.query_min_score`, default 0.55) instead of a hardcoded
|
||||
const.
|
||||
|
||||
**Est:** 2-4 h
|
||||
**Deps:** none
|
||||
|
||||
@@ -528,6 +578,11 @@ correctly, not to "clarify"). `make test` green.
|
||||
|
||||
### 4.2 Act surface broadening
|
||||
|
||||
**Status:** not a code item — operator config. The seeded homelab set
|
||||
(status/ps/uptime/df/free/logs read-only, restart/stop/reboot gated) ships
|
||||
in `deploy/mavend.json`. Broadening to home-automation/media/comms is
|
||||
editing JSON, not code.
|
||||
|
||||
**Est:** ongoing config
|
||||
**Deps:** none
|
||||
|
||||
@@ -550,6 +605,10 @@ after adding the tool to `mavend.json` and the act seed file.
|
||||
|
||||
### 4.3 LTM ANN (approximate nearest neighbor)
|
||||
|
||||
**Status:** deferred (correctly). The `memory.Store` interface is the swap
|
||||
point; brute-force cosine is sub-ms at single-user scale. Not started until
|
||||
note+fact count exceeds ~10k and `Search` latency shows up in profiles.
|
||||
|
||||
**Est:** ~1 day
|
||||
**Deps:** none (the interface is the swap point)
|
||||
|
||||
@@ -574,6 +633,10 @@ passes the existing `memory_test.go` suite and shows <1ms latency at
|
||||
|
||||
### 4.4 Persona prompt
|
||||
|
||||
**Status:** done (`b7eb53a`, 2026-07-06). `Persona` field in `VoiceConfig`,
|
||||
`llmphraser` prepends to `systemPrompt()` + `querySystemPrompt()`. Empty =
|
||||
current hardcoded feminine-gendered Russian persona (backward compat).
|
||||
|
||||
**Est:** 2-4 h
|
||||
**Deps:** none
|
||||
|
||||
@@ -616,6 +679,10 @@ without recompiling. Empty = current behavior. `make test` green.
|
||||
|
||||
### 4.5 Custom TTS voice
|
||||
|
||||
**Status:** not started. Mostly operator work (record ~50-100 clips, train a
|
||||
piper model). The code already supports it — `-model` flag takes any piper
|
||||
voice file, `VoiceConfig.Tts.Voice` names it.
|
||||
|
||||
**Est:** ~1 day + training time
|
||||
**Deps:** none (piper supports custom voices)
|
||||
|
||||
@@ -651,10 +718,10 @@ The code already supports it — it's a model-file swap.
|
||||
|
||||
### 5.1 Multi-user (SPEC item 8)
|
||||
|
||||
**Status:** `SPEC.md:264-270` fences this explicitly: "DO NOT TOUCH THIS
|
||||
PHASE." No second user exists. The append-only schema (`schema.sql`)
|
||||
makes it a migration (add `user_id` columns + backfill to "kami"), not a
|
||||
rewrite. Speaker attribution needs the second voice to train against.
|
||||
**Status:** deferred by design. SPEC fences this explicitly (`DO NOT TOUCH
|
||||
THIS PHASE`). No second user exists. The append-only schema makes it a
|
||||
migration (add `user_id` columns + backfill), not a rewrite. Speaker
|
||||
attribution needs the second voice to train against.
|
||||
|
||||
**When to revisit:** when a second person is actually in the house and
|
||||
using maven. Not before.
|
||||
@@ -668,23 +735,25 @@ exist.
|
||||
|
||||
## Summary table
|
||||
|
||||
| # | Item | Prio | Est | Type | Deps |
|
||||
|---|------|------|-----|------|------|
|
||||
| 1.1 | Kuma API key | P1 | 5m | ops | — |
|
||||
| 1.2 | Voice bind verify + comment fix | P1 | 30m | ops | — |
|
||||
| 1.3 | desk_active deploy | P1 | 1h | ops | — |
|
||||
| 2.1 | Cold-start unlock | P2 | 1d | code | — |
|
||||
| 3.1 | Always-on listening | P3 | 1-2w | code+hw | hardware decision |
|
||||
| 3.2 | Conversation depth | P3 | 3-5d | code | — |
|
||||
| 3.3 | Latency/streaming | P3 | 1-2w | code | — |
|
||||
| 4.1 | Routing quality (dev embedder) | P4 | 2-4h | code+docs | — |
|
||||
| 4.2 | Act surface | P4 | ongoing | config | — |
|
||||
| 4.3 | LTM ANN | P4 | 1d | code | scale problem |
|
||||
| 4.4 | Persona prompt | P4 | 2-4h | code | — |
|
||||
| 4.5 | Custom TTS voice | P4 | 1d+train | ops | — |
|
||||
| 5.1 | Multi-user | P5 | deferred | — | second user |
|
||||
| # | Item | Prio | Est | Type | Deps | Status |
|
||||
|---|------|------|-----|------|------|--------|
|
||||
| 1.1 | Kuma API key | P1 | 5m | ops | — | done `eda434f` |
|
||||
| 1.2 | Voice bind verify + comment fix | P1 | 30m | ops | — | done `eda434f` |
|
||||
| 1.3 | desk_active deploy | P1 | 1h | ops | — | not done (operator action on `linux`) |
|
||||
| 2.1 | Cold-start unlock | P2 | 1d | code | — | code done `b0932a1`+`15fe7bb`, **tests missing** |
|
||||
| 3.1 | Always-on listening | P3 | 1-2w | code+hw | hardware decision | MVP `e57647c` (VAD only, no wake word) |
|
||||
| 3.2 | Conversation depth | P3 | 3-5d | code | — | done `05236ad` |
|
||||
| 3.3 | Latency/streaming | P3 | 1-2w | code | — | not started (deferred) |
|
||||
| 4.1 | Routing quality (dev embedder) | P4 | 2-4h | code+docs | — | done `b7eb53a` |
|
||||
| 4.2 | Act surface | P4 | ongoing | config | — | not a code item (config) |
|
||||
| 4.3 | LTM ANN | P4 | 1d | code | scale problem | deferred (scale) |
|
||||
| 4.4 | Persona prompt | P4 | 2-4h | code | — | done `b7eb53a` |
|
||||
| 4.5 | Custom TTS voice | P4 | 1d+train | ops | — | not started (ops) |
|
||||
| 5.1 | Multi-user | P5 | deferred | — | second user | deferred by design |
|
||||
|
||||
**Recommended order:** 1.1 → 1.2 → 1.3 (today, ~2h) → 2.1 (security,
|
||||
~1d) → 4.1 + 4.4 (quick depth, same day) → 3.2 (conversation, ~3-5d) →
|
||||
3.1 (always-on, needs hardware pick first) → 3.3 (streaming, lowest
|
||||
ROI of the dealbreakers) → 4.5 (custom voice, when recording is done).
|
||||
**Remaining work (in priority order):**
|
||||
1. **2.1 tests** — write the 3 missing keywrap/locked-mode test cases (~30 min)
|
||||
2. **1.3 desk_active** — install systemd timer + hypridle on `linux` (~1h, your hands)
|
||||
3. **3.1 wake word** — swap energy-VAD for silero-vad/openWakeWord ONNX in `vad.go`
|
||||
4. **3.3 streaming** — lowest ROI, defer until 3.1 is real
|
||||
5. **4.5 custom voice** — when recording is done
|
||||
|
||||
Reference in New Issue
Block a user