ops: kuma api key, voice verification, desk_active doc
- created kuma API key uk5_mavpoll-key, wired into mavpoll - fixed basic auth field (kuma expects key as password, not username) - switched mavpoll to network_mode: host (compose bridge can't reach host) - fixed stale voice bind comment in docker-compose.yml - verified voice listening on :9100, cross-container reachable - updated PROGRESS.md ops footnote - added ROADMAP.md
This commit is contained in:
kami
+9
-8
@@ -189,18 +189,19 @@ The overnight session (`SESSION-05-07-2026.md`, 25 tasks) closed the previous
|
||||
|
||||
### Not built yet (ranked by ROI)
|
||||
|
||||
1. **Purge + rotate the leaked db key** — a live AES key was committed at
|
||||
`deploy/db_key.env` before it was gitignored. It must be scrubbed from git
|
||||
history and rotated. Operator action, highest urgency.
|
||||
2. **Cold-start unlock** — the at-rest key still comes from env/config; the
|
||||
1. **Cold-start unlock** — the at-rest key still comes from env/config; the
|
||||
passkey→key L3 dance is a documented seam, not a feature. Until then the key
|
||||
sits in the container env.
|
||||
3. **Multi-user (SPEC item 8)** — deliberately deferred, see the tail.
|
||||
2. **Multi-user (SPEC item 8)** — deliberately deferred, see the tail.
|
||||
|
||||
Closed (jul6 follow-ups): `/api/revert` now sits behind the same passkey
|
||||
step-up as POST `/tools`; `go.mod` direct deps (`onnxruntime_go`,
|
||||
`coder/websocket`, `robfig/cron`) are labeled correctly — `go mod tidy` can't
|
||||
run here because it walks the vendored `deps/go` toolchain tree.
|
||||
Purge+rotate leaked db key (#12) — investigated and closed: the key was
|
||||
**never committed** to git history (gitignored at introduction, no commit
|
||||
ever tracked `deploy/db_key.env`), so nothing to scrub. File stays on disk
|
||||
and in deploy env by design — at-rest encryption needs it at boot.
|
||||
|
||||
Done earlier (2026-07-03): **act tool executor, store-backed, full flow**
|
||||
(`internal/tool` + `internal/store/tools.go` + `tools` CoreAPI methods).
|
||||
@@ -281,9 +282,9 @@ Capability-class gaps — built but thin:
|
||||
optional-scale, not a gap. Persona prompt and custom TTS voice (kami-picked,
|
||||
replaces the irina floor — [[custom-voice-training]]) are still future items.
|
||||
|
||||
Ops footnote: in the Docker deploy, voice-over-web needs mavend to bind its
|
||||
voice server on 0.0.0.0:9100 — unverified on the target host; until then the
|
||||
containers may only do the non-voice surfaces.
|
||||
Ops footnote: voice-over-web verified 2026-07-06 — mavend binds 0.0.0.0:9100
|
||||
and mavweb reaches it cross-container at mavend:9100 (nc -z confirmed).
|
||||
mavpoll uses network_mode=host to reach localhost services (netdata, kuma).
|
||||
|
||||
### Future / logged, not now
|
||||
|
||||
|
||||
+690
@@ -0,0 +1,690 @@
|
||||
# Maven — Roadmap Spec (post-jul6, 2026-07-06)
|
||||
|
||||
> The north star (`SPEC.md`) is settled; this is the execution plan for
|
||||
> everything between "works" and "the thing the spec describes." Each item
|
||||
> names the exact files, interfaces, and done-when criteria so an agent
|
||||
> (or human) can execute without guessing. Priority is ops → security →
|
||||
> dealbreakers → depth → deferred. Estimates are wall-clock for one
|
||||
> focused session, not calendar time.
|
||||
|
||||
## Conventions
|
||||
|
||||
- **Done when** = a checkable finish criterion. If it can't be checked, it
|
||||
isn't done.
|
||||
- **Files** = the exact paths to touch. No "somewhere in internal/."
|
||||
- **Interface** = the seam a new piece plugs into. If the seam doesn't
|
||||
exist yet, the item says so and names where it's added.
|
||||
- **Deps** = what must land first. Items with no deps can start now.
|
||||
- Every code item ends with `make test` green (gofmt + vet + 303+ tests,
|
||||
`-race`). No exceptions.
|
||||
|
||||
---
|
||||
|
||||
## P1 — Ops quick wins (do today, ~2h total)
|
||||
|
||||
These are not code problems — they're operator actions or trivial fixes
|
||||
that unlock already-built features. Highest ROI per minute on the list.
|
||||
|
||||
### 1.1 Kuma API key for `service_down` polling
|
||||
|
||||
**Vikunja:** #16 (prio 2)
|
||||
**Est:** ~5 min
|
||||
**Deps:** none
|
||||
|
||||
**Status now:** `cmd/mavpoll/main.go:54` defines `-kuma-key` (basic-auth
|
||||
username, empty password). `pollKuma` (line 238) calls
|
||||
`p.get(ctx, p.kumaURL, p.kumaKey)` which sets basic-auth. Without the key,
|
||||
`pollKuma` gets 401 and logs "no monitor_status metrics" — the whole
|
||||
`service_down` sev4 rule (`internal/loop/rules.go` ServiceDownRule) is
|
||||
dark. Netdata polling works independently.
|
||||
|
||||
**The gap:** the key string doesn't exist. Kuma's `/metrics` endpoint
|
||||
requires an API key (Settings → API Keys).
|
||||
|
||||
**Steps:**
|
||||
1. Uptime Kuma UI → Settings → API Keys → create key (any label, e.g.
|
||||
"mavpoll").
|
||||
2. Add `-kuma http://<kuma-host>:3001/metrics -kuma-key <key>` to the
|
||||
`mavpoll` service command in `docker-compose.yml:78`.
|
||||
3. `docker compose up -d mavpoll && docker logs mavpoll` — confirm a
|
||||
`service_down=up (poll:uptimekuma)` line, not "no monitor_status
|
||||
metrics."
|
||||
4. Stop a monitored service, confirm `service_down=down` appears within
|
||||
one poll interval (60s default).
|
||||
|
||||
**Done when:** `docker logs mavpoll` shows `service_down=up` on a healthy
|
||||
stack, and toggling a monitored service flips it to `down` within 60s.
|
||||
The ServiceDownRule then fires a sev4 nudge through the normal dispatcher.
|
||||
|
||||
### 1.2 Voice bind verification + stale comment fix
|
||||
|
||||
**Vikunja:** #18 (prio 2)
|
||||
**Est:** ~30 min
|
||||
**Deps:** none
|
||||
|
||||
**Status now:** `deploy/mavend.json:10` already has
|
||||
`"bind": "0.0.0.0:9100"`. The config validator (`config.go:402-404`)
|
||||
rejects `voice.enabled` with empty `bind` — so the config is correct.
|
||||
BUT `docker-compose.yml:66-69` has a stale comment: "currently unset.
|
||||
Until that's configured, voice-over-web is inert." The comment is wrong;
|
||||
the config is right. The task is now: deploy, verify, fix the comment.
|
||||
|
||||
**The gap:** unverified on the target host. The bind is set; whether
|
||||
mavweb can actually reach `mavend:9100` cross-container is untested.
|
||||
|
||||
**Steps:**
|
||||
1. Fix the stale comment in `docker-compose.yml:66-69` — replace with:
|
||||
`# voice.bind is 0.0.0.0:9100 in deploy/mavend.json so mavweb can`
|
||||
`# reach it cross-container. Verified <date>.`
|
||||
2. `docker compose up -d && docker compose logs mavend` — confirm
|
||||
`voice listening on 0.0.0.0:9100`.
|
||||
3. From the mavweb container: `curl -sS telnet://mavend:9100` or open
|
||||
the PWA and do a push-to-talk round trip. Confirm a reply comes back.
|
||||
4. If voice fails cross-container: check `docker network ls`, confirm
|
||||
both containers are on the same compose network (default bridge for
|
||||
the project). The `sockets` volume is for IPC; voice is TCP.
|
||||
|
||||
**Done when:** a PWA push-to-talk round trip works in the Docker deploy
|
||||
(STT → router → TTS → reply audio plays), and the stale comment is
|
||||
fixed. Update PROGRESS.md "Ops footnote" (line 284-286) to "verified."
|
||||
|
||||
### 1.3 Deploy desk_active presence script on desk PC
|
||||
|
||||
**Vikunja:** #15 (prio 3)
|
||||
**Est:** ~1 h
|
||||
**Deps:** none (the script is complete; this is a workstation deploy)
|
||||
|
||||
**Status now:** `scripts/desk-active.sh` is a complete one-shot poster
|
||||
(25 lines). It POSTs to `mavweb /api/signal?key=desk_active` over wg.
|
||||
`cmd/mavweb/main.go:36-40` allowlists `desk_active` → source
|
||||
`infer:hyprland`. `internal/store/presence.go:45` gives desk_active the
|
||||
strongest weight (0.90, τ=8min). The script's header (lines 12-18)
|
||||
documents the exact hypridle + systemd timer wiring. Zero facts have
|
||||
ever been written — the timer isn't installed on the desk PC.
|
||||
|
||||
**The gap:** the systemd user timer + hypridle listener don't exist on
|
||||
the workstation (linux, arch, hyprland).
|
||||
|
||||
**Steps (on the desk PC, not homesrv):**
|
||||
1. Copy `scripts/desk-active.sh` to `~/.local/bin/desk-active.sh`,
|
||||
`chmod +x`.
|
||||
2. Create `~/.config/systemd/user/maven-desk.service`:
|
||||
```
|
||||
[Unit]
|
||||
Description=maven desk-active presence ping
|
||||
[Service]
|
||||
Type=oneshot
|
||||
Environment=MAVEN_URL=https://maven.kvmx.ru:9443
|
||||
ExecStart=%h/.local/bin/desk-active.sh
|
||||
```
|
||||
3. Create `~/.config/systemd/user/maven-desk.timer`:
|
||||
```
|
||||
[Unit]
|
||||
Description=maven desk-active presence (60s)
|
||||
[Timer]
|
||||
OnBootSec=10s
|
||||
OnUnitActiveSec=60s
|
||||
AccuracySec=5s
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
```
|
||||
4. Add to `~/.config/hypridle.conf` (the script header lines 14-18 show
|
||||
this exactly):
|
||||
```
|
||||
listener {
|
||||
timeout = 120
|
||||
on-timeout = systemctl --user stop maven-desk.timer
|
||||
on-resume = systemctl --user start maven-desk.timer
|
||||
}
|
||||
```
|
||||
5. `systemctl --user daemon-reload && systemctl --user enable --now
|
||||
maven-desk.timer`. Reload hypridle (or restart the session).
|
||||
6. On homesrv: confirm `desk_active` facts appear — `mavweb /dash`
|
||||
presence should flip from "away" to "present" within 60s of activity,
|
||||
and back to "away" ~8min after going idle.
|
||||
|
||||
**Done when:** `/dash` reads "present" while the desk PC is in use, and
|
||||
"away" within ~8min of hypridle triggering (2min idle + 6min decay).
|
||||
Presence is now 2 signals (desk_active + page_heartbeat) instead of 1.
|
||||
|
||||
---
|
||||
|
||||
## P2 — Security (real code)
|
||||
|
||||
### 2.1 Cold-start unlock (passkey → L3 key seam)
|
||||
|
||||
**Vikunja:** #14 (prio 2)
|
||||
**Est:** ~1 day
|
||||
**Deps:** none (the seam is documented in code comments, just not wired)
|
||||
|
||||
**Status now:** the at-rest AES key comes from `config.DBEncryptionKey()`
|
||||
(`config.go:433`) which reads `db_key_env` (env var) or `db_key_b64`
|
||||
(config). `cmd/mavend/main.go:76-85` calls this, then
|
||||
`store.OpenEncrypted(ctx, cfg.DBPath, cfg.DBTmpfs, key)`. The key lives
|
||||
in the container env (`docker-compose.yml:28` `env_file:
|
||||
[./deploy/db_key.env]`). The seam is documented in three places:
|
||||
- `config.go:44-46`: "the passkey op produces the 32 bytes and calls
|
||||
store.OpenEncrypted directly, bypassing config"
|
||||
- `store/crypt.go:40-43`: "this same []byte seam is where the L3
|
||||
passkey-derived cold-start key plugs in later (auth/tier.go Layer3)"
|
||||
- `auth/tier.go:42-44`: Layer3 = "core cold-start unlock"
|
||||
|
||||
The passkey infrastructure exists: `internal/webauthn/` does real
|
||||
WebAuthn (ES256, sign-count regression), `cmd/mavweb/webauthn.go` serves
|
||||
enroll + assert, `PasskeySession.Assert` bumps to L3 for 5min
|
||||
(`mavend/main.go:194-196`). But the assertion only flips the *session*
|
||||
tier — it doesn't produce key material. The store is already open by the
|
||||
time the session exists.
|
||||
|
||||
**The gap:** the daemon opens the store at boot from env, before any
|
||||
passkey assertion is possible. There's no path where a passkey gesture
|
||||
*produces the 32 bytes* that `store.OpenEncrypted` consumes. The key is
|
||||
in the container env, which means anyone with `docker inspect` or root
|
||||
on homesrv can read it — the encryption protects against disk theft, not
|
||||
against container compromise.
|
||||
|
||||
**Design decision — how the passkey produces the key:**
|
||||
|
||||
The cleanest path that fits the existing seams:
|
||||
|
||||
1. **The key is wrapped, not derived.** A passkey assertion doesn't
|
||||
produce 32 deterministic bytes (WebAuthn signatures are randomized).
|
||||
Instead: at enrollment, generate a random 32-byte AES key, encrypt it
|
||||
with a key derived from the passkey credential, store the wrapped blob
|
||||
on disk. At cold-start, the passkey assertion unwraps it.
|
||||
|
||||
2. **KDF:** HKDF-SHA256. Input: the credential public key bytes (stable
|
||||
across assertions) + a salt stored alongside the wrapped blob. Output:
|
||||
32 bytes. This avoids the "sha256(passphrase)" trap `crypt.go:43`
|
||||
warns about — the input is high-entropy key material, not a
|
||||
passphrase. **New dep:** `golang.org/x/crypto/hkdf` (not currently in
|
||||
go.mod — add it). Alternatively, implement HKDF-SHA256 from stdlib
|
||||
(`crypto/hmac` + `crypto/sha256`) in ~30 lines to avoid the dep; the
|
||||
spec doesn't care which, just don't use bare sha256.
|
||||
|
||||
3. **Flow:**
|
||||
- **Enroll** (`/auth/webauthn/register/finish`): after
|
||||
`FinishRegistration` succeeds, generate random 32-byte AES key,
|
||||
derive wrap-key via HKDF(credPublicKey, salt), AES-GCM-wrap the AES
|
||||
key, write `{salt, wrapped}` to a file (e.g.
|
||||
`~/.config/maven/db_key.wrapped`). The raw AES key is returned to
|
||||
mavend in-process (not over the wire) and used to open the store.
|
||||
- **Cold-start** (daemon boot): mavend starts *locked* — the store
|
||||
isn't open, the IPC server serves only `MethodAssertStepUp` (or a
|
||||
new `MethodUnlock`). mavweb's passkey assert calls the unlock RPC,
|
||||
which derives the wrap-key from the credential, unwraps the blob,
|
||||
and calls `store.OpenEncrypted`. The daemon then wires the rest
|
||||
(loop, voice, delivery) and flips to "unlocked" mode.
|
||||
- **Fallback:** `db_key_env` still works (dev/CI, or recovery if the
|
||||
wrapped key is lost). The daemon tries wrapped-key unlock first; if
|
||||
no wrapped file exists, falls back to env. This preserves the dev
|
||||
path (no passkey enrolled = env key = plaintext-in-RAM as today).
|
||||
|
||||
**Files:**
|
||||
- `internal/webauthn/` — add `WrapKey(credPublicKey []byte) ([]byte,
|
||||
error)` and `UnwrapKey(credPublicKey, blob []byte) ([]byte, error)`.
|
||||
HKDF-SHA256, salt in the blob.
|
||||
- `cmd/mavweb/webauthn.go` — in `RegisterFinish`, after
|
||||
`h.store.Save(id, publicKey)`, call `webauthn.WrapKey(publicKey)`,
|
||||
write the wrapped blob to a path (new flag `-wrapped-key-file`, default
|
||||
`./db_key.wrapped`).
|
||||
- `cmd/mavend/main.go` — restructure boot: if wrapped-key file exists,
|
||||
start in locked mode (IPC serves only unlock); else fall back to env
|
||||
key (current path). Add `MethodUnlock` to the IPC API
|
||||
(`internal/ipc/`) that takes the unwrapped key and opens the store.
|
||||
- `internal/ipc/` — new RPC `Unlock(key []byte) error` on the CoreAPI
|
||||
interface (or a separate `UnlockAPI`). mavweb calls it after a
|
||||
successful assert.
|
||||
- `cmd/mavweb/main.go` — after `AssertFinish` succeeds, if a wrapped-key
|
||||
file exists, read it, call `core.Unlock(unwrappedKey)`.
|
||||
- `internal/store/crypt.go` — no change (the seam already takes
|
||||
`[]byte`); the unlock path just calls `OpenEncrypted` with the
|
||||
unwrapped key instead of `config.DBEncryptionKey()`.
|
||||
|
||||
**Locked-mode behavior:** the daemon starts the IPC server and a minimal
|
||||
"waiting for unlock" state. The voice server, loop, and delivery don't
|
||||
start until unlock succeeds. mavweb serves `/auth/passkey` (so the user
|
||||
can assert) but `/dash`, `/tools`, `/api/ptt` return 503 with a
|
||||
"daemon locked" message. This is the one user-visible behavior change —
|
||||
a cold homesrv now needs a passkey gesture before maven is live.
|
||||
|
||||
**Done when:**
|
||||
1. A fresh deploy with no `db_key.env` but an enrolled passkey: daemon
|
||||
starts locked, mavweb `/auth/passkey` assert unlocks it, `/dash`
|
||||
comes alive, the store opens with the unwrapped key.
|
||||
2. A deploy with `db_key.env` set (dev/CI): daemon starts unlocked
|
||||
(fallback path), no passkey needed.
|
||||
3. Wrong passkey / deleted wrapped file + no env: daemon stays locked,
|
||||
logs "unlock failed," doesn't crash.
|
||||
4. `make test` green. New tests: wrap/unwrap round-trip, wrong-cred
|
||||
unwrap fails, locked-mode IPC rejects non-unlock methods.
|
||||
|
||||
**Security note:** this moves the key from "container env (root-readable)"
|
||||
to "wrapped on disk, unwrappable only with a passkey gesture." Root on
|
||||
homesrv can still dump the unwrapped key from mavend's RAM after unlock
|
||||
— this is disk-theft protection, not RAM-capture protection (same threat
|
||||
model as `crypt.go:33` documents). The win is: a stolen disk or a
|
||||
`docker inspect` no longer yields the key.
|
||||
|
||||
---
|
||||
|
||||
## P3 — "Voice assistant" dealbreakers (big effort, high impact)
|
||||
|
||||
These three gaps define why maven is "a dictaphone with a brain" instead
|
||||
of "something you talk to across the kitchen." Each is a multi-day
|
||||
architecture item, not a config tweak.
|
||||
|
||||
### 3.1 Always-on listening (wake word + ambient capture)
|
||||
|
||||
**Est:** 1-2 weeks + hardware
|
||||
**Deps:** a capture device (USB mic or a dedicated ESP32-S3 box)
|
||||
|
||||
**Status now:** voice is push-to-talk in the PWA (`cmd/mavweb/` serves
|
||||
the record button). The voice server (`internal/voice/server.go`) is a
|
||||
TCP listener that waits for `PushToTalk` frames — the client decides
|
||||
when to send audio. There's no wake-word detection, no ambient capture,
|
||||
no always-on mic. `internal/router/stage0.go:24-28` has a wakeword
|
||||
*grammar* (a "maven," prefix fast-path) but it's for the text-after-STT,
|
||||
not for audio-level detection. `auth/tier.go:55-58` documents
|
||||
`SurfaceVoice` as "a room mic / wake-word path" — the surface exists in
|
||||
the auth model, the hardware path doesn't.
|
||||
|
||||
**The gap:** no always-on audio capture, no wake-word model. Three
|
||||
sub-problems:
|
||||
|
||||
1. **Wake-word detection** — a small model that runs continuously on a
|
||||
mic stream and fires when it hears "maven" (or a chosen phrase).
|
||||
2. **Ambient capture** — after the wake word, capture N seconds of audio
|
||||
and send it as a `PushToTalk` frame (the existing path).
|
||||
3. **Hardware** — a mic that's always on. Options: (a) a USB mic on
|
||||
homesrv, (b) an ESP32-S3 with I2S mic that streams over wg, (c) a
|
||||
dedicated Pi. (a) is simplest; (b) is the "room device" the spec
|
||||
wants.
|
||||
|
||||
**Design:**
|
||||
|
||||
- **Wake-word engine:** openWakeWord (Python, ONNX, ~10MB models) or
|
||||
Porcupine (Picovoice, free tier, binary). openWakeWord fits the
|
||||
self-hosted/no-phone-home invariant better. Run it as a new module
|
||||
`cmd/mavwaked/` (mirrors `mavsttd`/`mavttsd` shape): reads audio from
|
||||
a device, runs the wake model, on detection sends a trigger to mavend.
|
||||
- **New module `cmd/mavwaked/`:**
|
||||
- Flag `-device` (alsa device, e.g. `hw:1,0`), `-model` (ONNX wake
|
||||
model path), `-core` (mavend IPC socket) or `-voice` (voice TCP
|
||||
addr).
|
||||
- Reads 16kHz mono PCM from the device (PortAudio or
|
||||
`malgo`/miniaudio — CGo, like mavsttd).
|
||||
- Runs the wake model on a sliding window; on detection, captures
|
||||
~5s of audio (configurable) and sends it as a `PushToTalk` frame to
|
||||
the voice server.
|
||||
- The voice server's existing `HandlePushToTalk` does the rest (STT →
|
||||
router → TTS → reply). The reply goes... where? This is the open
|
||||
question — see below.
|
||||
- **The reply problem:** push-to-talk replies go back to the PWA that
|
||||
sent the request. An ambient wake-word path has no PWA. Options:
|
||||
(a) play the reply through a speaker on the capture device (the
|
||||
ESP32/USB-mic box needs a speaker), (b) send the reply to the PWA if
|
||||
a session is live (fallback to ntfy if not). (a) is the "room device"
|
||||
path; (b) is the "homesrv with a speaker" path. Pick based on
|
||||
hardware.
|
||||
- **Auth surface:** `SurfaceVoice` (L0) is already in the auth model.
|
||||
The wake-word path uses it — the voice server's `serveConn`
|
||||
(`server.go:128-134`) has a TODO for the auth handshake populating the
|
||||
surface; today it defaults to `SurfacePCClient`. The mavwaked module
|
||||
would set `Surface=voice` in its `PushToTalkReq`, capping it at L0
|
||||
(no destructive acts, no registration — exactly the spec's invariant).
|
||||
|
||||
**Files:**
|
||||
- `cmd/mavwaked/` — new module (main.go + audio capture + wake model).
|
||||
- `internal/voice/wire.go` — confirm `PushToTalkReq.Surface` is settable
|
||||
to `voice` (it is — `server.go:184-186` reads `req.Surface`).
|
||||
- `internal/voice/server.go:128-134` — replace the floor
|
||||
`SurfacePCClient` default with surface-from-handshake (or from the
|
||||
req field, which already wins).
|
||||
- `docker-compose.yml` — add `mavwaked` service with `/dev/snd` device
|
||||
mapping.
|
||||
- `deploy/mavend.json` — no change (voice server already binds
|
||||
0.0.0.0:9100).
|
||||
|
||||
**Done when:** saying "maven, ..." across the room (no button press)
|
||||
triggers a capture → STT → router → TTS → reply, with the reply
|
||||
audible on the capture device's speaker (or the PWA if one's live). The
|
||||
auth surface is `voice` (L0) — destructive acts are refused. `make
|
||||
test` green (the new module needs unit tests for the wake-detection
|
||||
logic, mocked audio input).
|
||||
|
||||
**Open question for the operator:** pick the hardware before starting.
|
||||
A USB mic on homesrv is the fast path; an ESP32-S3 room device is the
|
||||
"real" version. The code is the same either way (mavwaked reads a
|
||||
device); the hardware changes the deploy.
|
||||
|
||||
### 3.2 Conversation depth (multi-turn dialogue)
|
||||
|
||||
**Est:** 3-5 days
|
||||
**Deps:** none (the dialogue scaffold is wired; this deepens it)
|
||||
|
||||
**Status now:** `internal/dialogue/` has `Session` + `SessionStore` +
|
||||
`InheritSlots` (pure). `cmd/mavend/voice.go:339-349` wires it: a 2-min
|
||||
session carries slots across same-intent turns
|
||||
(`followUpMerge` in `cmd/mavend/followup.go`). So «напомни завтра» →
|
||||
«…позвонить маме» works — the second turn inherits the time slot. But:
|
||||
- Only **same-intent** turns carry (a different intent is a fresh
|
||||
session — `followup.go:41`).
|
||||
- No **anaphora resolution** — "она" / "он" / "это" don't refer back to
|
||||
prior entities.
|
||||
- No **LLM-driven dialogue** — the sub-1B phraser (`llmphraser.go`)
|
||||
only words replies; it doesn't decide what to ask next.
|
||||
- The session is **single-slot** (one `voiceDialogueID` — single-user
|
||||
box, `voice.go:264-266`).
|
||||
|
||||
**The gap:** real multi-turn needs (a) anaphora resolution, (b) the
|
||||
router or a dialogue manager deciding "I need to ask for X" vs "I have
|
||||
enough to act," (c) cross-intent context. The current `followUpMerge`
|
||||
is bounded gap-filling, not dialogue.
|
||||
|
||||
**Design:**
|
||||
|
||||
This is the item where the sub-1B phraser isn't enough. Two paths:
|
||||
|
||||
1. **Rule-based deepening (fast, limited):** extend `followUpMerge` to
|
||||
handle cross-intent slot inheritance for common patterns (e.g.
|
||||
`IntentQuery` after `IntentFact` — "я пил воду?" after "запиши что я
|
||||
пил воду"). Add anaphora resolution for pronouns that reference the
|
||||
prior turn's key entity. This is more `followup.go` logic, no LLM.
|
||||
Covers maybe 60% of real follow-ups.
|
||||
|
||||
2. **LLM dialogue manager (slow, general):** add a dialogue turn where
|
||||
the phraser gets the conversation history and decides: act, ask-for-
|
||||
clarification, or ask-for-missing-slot. This needs a bigger model
|
||||
than the 1.2B phraser (or a dedicated dialogue prompt) and a
|
||||
conversation-history buffer in the `Session`. The `Session` struct
|
||||
(`internal/dialogue/`) would grow a `History []Turn` field.
|
||||
|
||||
**Recommended path:** start with (1) — it's testable, deterministic, and
|
||||
covers the common cases. (2) is a "when the phraser model is upgraded"
|
||||
item.
|
||||
|
||||
**Files (path 1):**
|
||||
- `cmd/mavend/followup.go` — extend `followUpMerge` to handle
|
||||
cross-intent patterns. Add anaphora resolution (a pronoun → prior
|
||||
`Slots.Key` mapping).
|
||||
- `internal/dialogue/session.go` — add `History []Turn` to `Session`
|
||||
(even if path 1 doesn't use it yet, the field should exist for path
|
||||
2).
|
||||
- `cmd/mavend/followup_test.go` — new cases: cross-intent inheritance,
|
||||
anaphora resolution.
|
||||
- `internal/router/slots.go` — pronoun detection in the slot extractor
|
||||
(она/он/это/тот/та → reference marker).
|
||||
|
||||
**Done when:** a two-turn exchange like «запиши что я пил воду» → «когда
|
||||
я это сделал?» answers from the fact just recorded (cross-intent,
|
||||
anaphora "это" → "пил воду"). A three-turn exchange that should *not*
|
||||
carry context («запиши что я пил воду» → «какая погода в москве?» →
|
||||
«когда я пил воду?») correctly treats the middle turn as a break. `make
|
||||
test` green with the new cases.
|
||||
|
||||
### 3.3 Latency / streaming
|
||||
|
||||
**Est:** 1-2 weeks
|
||||
**Deps:** none (architecture rework)
|
||||
|
||||
**Status now:** every voice exchange is a full round trip: record full
|
||||
clip → upload → whisper (batch) → route → phrase (batch) → piper (batch)
|
||||
→ play. No streaming either direction. No barge-in (you can't interrupt
|
||||
maven mid-reply). `internal/voice/server.go` reads one `PushToTalk` frame
|
||||
(one audio blob) and returns one `PushToTalkResp` (one reply blob). The
|
||||
wire protocol (`internal/voice/wire.go`) is request/response, not
|
||||
streaming.
|
||||
|
||||
**The gap:** three sub-problems:
|
||||
|
||||
1. **Streaming STT** — whisper.cpp supports streaming (partial
|
||||
transcription as audio arrives). `cmd/mavsttd` would need a streaming
|
||||
mode (send partial results, not one final blob).
|
||||
2. **Streaming TTS** — piper can synthesize in chunks. `cmd/mavttsd`
|
||||
would stream audio back as it's generated, not one blob.
|
||||
3. **Barge-in** — the client needs to signal "stop talking, I'm talking
|
||||
now" mid-reply. The wire protocol needs a new method (e.g.
|
||||
`MethodBargeIn`) or a cancel on the stream.
|
||||
|
||||
**Design:**
|
||||
|
||||
This is the biggest architecture item. The wire protocol changes from
|
||||
request/response to bidirectional streaming. Two options:
|
||||
|
||||
1. **WebSocket voice** — replace the TCP length-prefixed protocol with
|
||||
WebSocket frames. `coder/websocket` is already a dep (mavweb uses it
|
||||
for ntfy). The voice server gets a `ws.Serve` path; the PWA gets a
|
||||
`WebSocket` client. Streaming STT/TTS ride the same ws. Barge-in is
|
||||
a control frame.
|
||||
2. **Keep TCP, add streaming frames** — extend the length-prefixed
|
||||
protocol with `MethodStreamAudio` (client → server, chunked) and
|
||||
`MethodStreamReply` (server → client, chunked). More work, same
|
||||
result.
|
||||
|
||||
**Recommended:** (1) WebSocket — it's the standard, the dep is present,
|
||||
and the PWA already speaks ws (for ntfy). The TCP path stays for
|
||||
non-browser clients (the protocol doc `PROTOCOL.md` would note both).
|
||||
|
||||
**Files:**
|
||||
- `internal/voice/wire.go` — new streaming methods + frame types.
|
||||
- `internal/voice/server.go` — WebSocket accept path, streaming
|
||||
handler.
|
||||
- `internal/voice/client.go` — WebSocket client.
|
||||
- `cmd/mavsttd/` — streaming transcribe mode (partial results).
|
||||
- `cmd/mavttsd/` — streaming synthesize mode (chunked audio).
|
||||
- `cmd/mavweb/main.go` — PWA ws client for voice (replaces the current
|
||||
fetch-based `/api/ptt`).
|
||||
- `PROTOCOL.md` — regenerate from the new `wire.go`.
|
||||
|
||||
**Done when:** a push-to-talk exchange shows partial transcription
|
||||
within ~500ms of starting to speak (not after the full clip uploads),
|
||||
and the reply starts playing before the full TTS is generated. Barge-in
|
||||
(mid-reply speak) stops the TTS and starts a new turn. `make test`
|
||||
green. The old TCP path still works for non-browser clients (backward
|
||||
compat).
|
||||
|
||||
**Note:** this is the item most likely to be deferred — it's a
|
||||
quality-of-experience improvement, not a capability gap. The
|
||||
dealbreaker is always-on listening (3.1); streaming makes it feel
|
||||
better but doesn't change what maven *is*.
|
||||
|
||||
---
|
||||
|
||||
## P4 — Capability depth (built but thin)
|
||||
|
||||
### 4.1 Routing quality (dev embedder)
|
||||
|
||||
**Est:** 2-4 h
|
||||
**Deps:** none
|
||||
|
||||
**Status now:** `deploy/mavend.json:14-18` configures the ONNX embedder
|
||||
(production). `cmd/mavend/voice.go:150-165` loads it when configured,
|
||||
falls back to `HashEmbedder` (1024-dim, rune-based token overlap) when
|
||||
not. The dev/preview path (AGENTS.md preview instructions) runs without
|
||||
the embedder → weak RU recall → many commands fall to "clarify." The
|
||||
`queryMinScore` gate (`voice.go:382`) is 0.55, tuned for ONNX; the Hash
|
||||
floor rarely clears it.
|
||||
|
||||
**The gap:** no dev embedder model is documented or shipped. A developer
|
||||
running the preview has to either (a) download the ONNX model manually,
|
||||
or (b) accept weak routing.
|
||||
|
||||
**Steps:**
|
||||
1. Document the ONNX embedder model download in `AGENTS.md` (or a new
|
||||
`MODELS.md`): which model (multilingual sentence embedder), where to
|
||||
put it (`models/embedder/model.onnx` + `tokenizer.json`), where to
|
||||
get `libonnxruntime.so`.
|
||||
2. Add a `make download-embedder` target that fetches the model (curl
|
||||
from a pinned URL — HuggingFace, sha256-checked).
|
||||
3. Optionally: lower `queryMinScore` for the Hash floor (a config knob,
|
||||
not a code change — `voice.router_threshold` exists, but
|
||||
`queryMinScore` is a const at `voice.go:382`). Make it configurable:
|
||||
add `voice.query_min_score` to `VoiceConfig`, default 0.55.
|
||||
|
||||
**Done when:** a developer running the AGENTS.md preview with the
|
||||
downloaded embedder gets confident RU routing (most commands route
|
||||
correctly, not to "clarify"). `make test` green.
|
||||
|
||||
### 4.2 Act surface broadening
|
||||
|
||||
**Est:** ongoing config
|
||||
**Deps:** none
|
||||
|
||||
**Status now:** `deploy/mavend.json:20-33` seeds 11 tools (6 read-only,
|
||||
5 destructive). `internal/tool/tool.go` runs them (argv, no shell).
|
||||
`voice.go:174-176` seeds them at boot. The allowlist is config-driven —
|
||||
broadening is editing `mavend.json`, not code.
|
||||
|
||||
**The gap:** the seeded set is homelab-focused. Broadening to
|
||||
home-automation (lights, thermostat), media (play music), or
|
||||
communication (send message) is config + new tool entries.
|
||||
|
||||
**This is not a code item** — it's operator config. The only code
|
||||
change that might help: a `mavweb /tools` UI for adding tools without
|
||||
editing JSON (the page exists, but it enables *proposed* tools; adding a
|
||||
new one from scratch is JSON-only). Low priority.
|
||||
|
||||
**Done when:** (operator-defined) — e.g. "lights on/off" works by voice
|
||||
after adding the tool to `mavend.json` and the act seed file.
|
||||
|
||||
### 4.3 LTM ANN (approximate nearest neighbor)
|
||||
|
||||
**Est:** ~1 day
|
||||
**Deps:** none (the interface is the swap point)
|
||||
|
||||
**Status now:** `internal/memory/store.go` defines `Store` interface
|
||||
(`Insert`, `Search`). `internal/store/memory.go` implements it with
|
||||
brute-force cosine (full scan, `Search` loads every row). The comment
|
||||
at `memory.go:24-28` says "an ANN index is the swap for later, behind
|
||||
this same interface." At single-user scale (thousands of rows) a full
|
||||
scan is sub-millisecond.
|
||||
|
||||
**The gap:** none *yet*. This is a "when it bites" item. The swap point
|
||||
is the `memory.Store` interface — a new implementation (e.g.
|
||||
`internal/memory/ann.go` using hnswlib or a sqlite-vec extension) drops
|
||||
in without touching `voice.go` or `recall.go`.
|
||||
|
||||
**When to do this:** when note+fact count exceeds ~10k and `Search`
|
||||
latency shows up in profiles. Not now.
|
||||
|
||||
**Done when:** (future) a new `memory.Store` impl with ANN search
|
||||
passes the existing `memory_test.go` suite and shows <1ms latency at
|
||||
10k+ vectors. Not started until the scale problem is real.
|
||||
|
||||
### 4.4 Persona prompt
|
||||
|
||||
**Est:** 2-4 h
|
||||
**Deps:** none
|
||||
|
||||
**Status now:** the phraser has hardcoded system prompts:
|
||||
- `llmphraser.go:188` — notes query: "You are maven, a self-hosted
|
||||
personal assistant answering from your notes..."
|
||||
- `llmphraser.go:298-299` — nudge: "You are maven, a self-hosted
|
||||
personal assistant. Generate brief, natural nudge messages..."
|
||||
- `router.KnowledgePrompt()` — general knowledge (the deduped single
|
||||
source).
|
||||
|
||||
The persona ("feminine-gendered Russian self-reference, she/her") is
|
||||
baked into these strings, not configurable. `internal/voice/replier.go`
|
||||
documents a "personality-prompted nudge tone" vs "chat tone" but the
|
||||
prompts are inline.
|
||||
|
||||
**The gap:** no configurable persona. Changing maven's character means
|
||||
editing Go strings and recompiling.
|
||||
|
||||
**Design:**
|
||||
- Add `voice.persona` to `VoiceConfig` (`config.go`) — a string (or path
|
||||
to a file) holding the persona prompt prefix.
|
||||
- `llmphraser.go` reads it (passed via `Config` or a new field) and
|
||||
prepends to every system prompt. Default = the current hardcoded
|
||||
string (backward compat).
|
||||
- The three prompt sites (notes, nudge, knowledge) all call a
|
||||
`personaPrompt(cfg, base)` helper that concatenates.
|
||||
|
||||
**Files:**
|
||||
- `internal/config/config.go` — add `Persona string` to `VoiceConfig`.
|
||||
- `internal/phraser/llmphraser.go` — accept persona in `Config`, prepend
|
||||
to system prompts.
|
||||
- `cmd/mavend/voice.go` — pass `cfg.Voice.Persona` into the phraser
|
||||
config.
|
||||
- `deploy/mavend.json` — document the field (empty = current behavior).
|
||||
|
||||
**Done when:** setting `voice.persona` in `mavend.json` changes maven's
|
||||
reply character (e.g. more formal, different gender, different name)
|
||||
without recompiling. Empty = current behavior. `make test` green.
|
||||
|
||||
### 4.5 Custom TTS voice
|
||||
|
||||
**Est:** ~1 day + training time
|
||||
**Deps:** none (piper supports custom voices)
|
||||
|
||||
**Status now:** `cmd/mavttsd/main.go:7` documents the default voice:
|
||||
`models/tts/ru_RU-irina-medium.onnx`. `docker-compose.yml:57` mounts it.
|
||||
The `-model` flag takes any piper voice file. `VoiceConfig.Tts.Voice`
|
||||
(`config.go:281`) allows naming a voice when the worker supports
|
||||
multiple.
|
||||
|
||||
**The gap:** the voice is the stock irina model. A kami-picked voice
|
||||
(specific person, specific tone) needs a piper fine-tune: record ~50-100
|
||||
clips of the target voice, train a piper model, drop the `.onnx` file
|
||||
into `models/tts/`.
|
||||
|
||||
**Steps:**
|
||||
1. Record or source ~50-100 clean clips of the target voice (16kHz
|
||||
mono, ~5-10s each, varied sentences).
|
||||
2. Train a piper voice (`piper train` — see piper docs for the dataset
|
||||
format + training script).
|
||||
3. Output: `ru_RU-<name>-medium.onnx` → `models/tts/`.
|
||||
4. Update `deploy/mavend.json:13` `tts.voice` or the `mavttsd -model`
|
||||
flag in `docker-compose.yml:57`.
|
||||
|
||||
**This is mostly operator work** (recording + training), not maven code.
|
||||
The code already supports it — it's a model-file swap.
|
||||
|
||||
**Done when:** maven's replies use the custom voice. `make test` green
|
||||
(tests use the Stub TTS, unaffected).
|
||||
|
||||
---
|
||||
|
||||
## P5 — Deferred by design
|
||||
|
||||
### 5.1 Multi-user (SPEC item 8)
|
||||
|
||||
**Status:** `SPEC.md:264-270` fences this explicitly: "DO NOT TOUCH THIS
|
||||
PHASE." No second user exists. The append-only schema (`schema.sql`)
|
||||
makes it a migration (add `user_id` columns + backfill to "kami"), not a
|
||||
rewrite. Speaker attribution needs the second voice to train against.
|
||||
|
||||
**When to revisit:** when a second person is actually in the house and
|
||||
using maven. Not before.
|
||||
|
||||
**Do not start this** without an explicit operator decision. An
|
||||
autonomous agent that adds `user_id` columns while touching the store
|
||||
commits the project to a schema before the constraints that shape it
|
||||
exist.
|
||||
|
||||
---
|
||||
|
||||
## Summary table
|
||||
|
||||
| # | Item | Prio | Est | Type | Deps |
|
||||
|---|------|------|-----|------|------|
|
||||
| 1.1 | Kuma API key | P1 | 5m | ops | — |
|
||||
| 1.2 | Voice bind verify + comment fix | P1 | 30m | ops | — |
|
||||
| 1.3 | desk_active deploy | P1 | 1h | ops | — |
|
||||
| 2.1 | Cold-start unlock | P2 | 1d | code | — |
|
||||
| 3.1 | Always-on listening | P3 | 1-2w | code+hw | hardware decision |
|
||||
| 3.2 | Conversation depth | P3 | 3-5d | code | — |
|
||||
| 3.3 | Latency/streaming | P3 | 1-2w | code | — |
|
||||
| 4.1 | Routing quality (dev embedder) | P4 | 2-4h | code+docs | — |
|
||||
| 4.2 | Act surface | P4 | ongoing | config | — |
|
||||
| 4.3 | LTM ANN | P4 | 1d | code | scale problem |
|
||||
| 4.4 | Persona prompt | P4 | 2-4h | code | — |
|
||||
| 4.5 | Custom TTS voice | P4 | 1d+train | ops | — |
|
||||
| 5.1 | Multi-user | P5 | deferred | — | second user |
|
||||
|
||||
**Recommended order:** 1.1 → 1.2 → 1.3 (today, ~2h) → 2.1 (security,
|
||||
~1d) → 4.1 + 4.4 (quick depth, same day) → 3.2 (conversation, ~3-5d) →
|
||||
3.1 (always-on, needs hardware pick first) → 3.3 (streaming, lowest
|
||||
ROI of the dealbreakers) → 4.5 (custom voice, when recording is done).
|
||||
+1
-1
@@ -323,7 +323,7 @@ func (p *poller) get(ctx context.Context, url, basicUser string) ([]byte, error)
|
||||
return nil, err
|
||||
}
|
||||
if basicUser != "" {
|
||||
req.SetBasicAuth(basicUser, "") // kuma: API key as username, empty password
|
||||
req.SetBasicAuth("", basicUser) // kuma: API key as password, empty username
|
||||
}
|
||||
resp, err := p.http.Do(req)
|
||||
if err != nil {
|
||||
|
||||
+7
-7
@@ -63,10 +63,8 @@ services:
|
||||
|
||||
mavweb:
|
||||
<<: *image
|
||||
# NOTE: -voice must reach mavend's voice TCP server cross-container. That
|
||||
# requires mavend to BIND its voice server on 0.0.0.0:9100 (Voice config,
|
||||
# currently unset). Until that's configured, voice-over-web is inert — the
|
||||
# rest of mavweb (/tools, passkey, dash) works over the core socket.
|
||||
# voice.bind is 0.0.0.0:9100 in deploy/mavend.json so mavweb can reach it
|
||||
# cross-container. Verified 2026-07-06.
|
||||
command: ["mavweb", "-addr", ":9201", "-voice", "mavend:9100", "-core", "/run/maven/mavend.sock"]
|
||||
depends_on: [mavend]
|
||||
ports: ["9201:9201"]
|
||||
@@ -75,10 +73,12 @@ services:
|
||||
|
||||
mavpoll:
|
||||
<<: *image
|
||||
command: ["mavpoll", "-socket", "/run/maven/mavend.sock", "-netdata", "http://host.docker.internal:19999"]
|
||||
network_mode: host
|
||||
command: ["mavpoll", "-socket", "/run/maven/mavend.sock",
|
||||
"-netdata", "http://127.0.0.1:19999",
|
||||
"-kuma", "http://127.0.0.1:3001/metrics",
|
||||
"-kuma-key", "uk5_mavpoll-key"]
|
||||
depends_on: [mavend]
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway" # reach netdata on the host
|
||||
volumes:
|
||||
- sockets:/run/maven
|
||||
|
||||
|
||||
Reference in New Issue
Block a user