Maven/docs/qa.md

QA plan: checking Maven properly

Last verified: 2026-08-02 @ 5c05163. Living doc: correct it in place, do not append.

Written 2026-08-01, after the 35-PR stack landed and the box came back up. Refreshed 2026-08-02 against the live list, after PRs #85-#90.

42 of the 50 open Vikunja tasks are QA: tasks. They are verification work, not build work. Most sat unverifiable while Maven was down for 11 days. That blocker is gone.

The plan as written on 2026-08-01 named 40 task numbers. Ten open QA: tasks were missing and two of the named ones had closed. Every open task now appears below, the eight non-QA ones in the last two sections.

This plan orders them by what unblocks what. Do sessions 1 and 2 first. Almost everything downstream assumes the voice loop works, and nobody has confirmed that since the redeploy.

---

Before you start

Two things bite anyone running these checks on homesrv.

curl needs --noproxy '*'. The shell exports http_proxy=http://127.0.0.1:18080. Without the flag, every local check returns 503 from the proxy and looks like a dead service. This cost me a false regression report today.

The database is not readable with sqlite3. Four older QA steps say docker compose exec mavend sqlite3 /data/maven.db "select ...". That cannot work: the container has no sqlite3 binary, and the store is AES-256-GCM at rest with a tmpfs working copy. Read state through mavweb instead, at /history, /trace, /routines and /dash.

---

Session 1: the voice loop (half a day)

Nothing here has been confirmed since the redeploy, and everything else assumes it works. Do this first.

Closes or advances: 44 (conversation), 45 (text chat), 287 (voice session quality), 321 steps 3-5 (quiet mode), 288 (STT fixtures).

1. Open http://127.0.0.1:9201/chat and hold a short conversation in Russian. Watch for three things: she answers in feminine forms (рада, поняла), she says ты and never вы, and no pet names appear.
2. Press push-to-talk on /dash. Say привет. Confirm a spoken reply comes back. This is the only check that covers mic to STT to core to TTS to speaker as one path. It is also the path the eleven-day outage most likely broke.
3. Say тихий режим. Expect тихий режим включён. буду реже напоминать.
4. Say выключи тихий режим. Expect тихий режим выключен. Negation must win.
5. Say в комнате тихо. Quiet mode must NOT flip. Confirm on /history that no quiet_hours fact was written.
6. Say включи режим тишины, then сделай потише. Both must flip quiet mode on. These are the noun form and the comparative, added 01-08-2026.
7. Wait for a nudge, then say потом within twenty minutes. Expect хорошо, вернусь к этому позже. and the nudge row on /notifications reading snoozed. Say потом again with nothing pending: it must route as an ordinary utterance, not be swallowed.
8. Wait for the water nudge, then say выпил воды. Expect the ordinary fact reply and nothing extra. She must not congratulate you. Check /notifications: the row reads acted. Then trigger another nudge and say готово. Expect отлично, отметила. and the same outcome.
9. Note anything where she is slow, cuts off, or talks over herself. That is 287's whole content and it has no written acceptance criteria yet.

319's single-token bug is fixed (01-08-2026). Single-word Russian utterances no longer come back as не совсем поняла — можешь переформулировать?. привет and поужинал both pass now: thinSingleToken spares social singles and any token carrying a verb ending, and only thins a bare nominal like вода. A one-word utterance that still gets clarified in this session is a new case for the lexicon, not the old bug.

---

Session 2: measurement (half a day, mostly waiting)

Closes or advances: 320 items 2-4, 278 (make the eval lab routine). Also 248 (memory evaluation), 319 (the margin gate) and 323 (the startup timeout arm).

The resident llama-server cannot be reached by the eval harness. It binds --host 127.0.0.1 --port 0 inside the container, so the port is kernel-assigned and never published. Start a second one on a fixed port instead:

llama-server -m /mnt/hdd1/llms/qwen3/Qwen3-1.7B-UD-Q4_K_XL.gguf \
  --host 127.0.0.1 --port 18100 -c 4096 -ngl 99 --no-webui

-c 4096 matters. The recorded numbers were measured at that context size, and a mismatch invalidates the comparison.

Then:

make eval-models MAVEN_LLM_URL=http://127.0.0.1:18100   # want ~72.7% cascade
make eval-router                                         # classifier baseline
MAVEN_LLM_URL=http://127.0.0.1:18100 make eval-phrasing  # persona checks, slow
make eval-recall

A large miss against 72.7% means the deploy differs from the bench harness.

Then three things to decide while the numbers are in front of you:

• 319's gate recalibration is done. 359 gave the LLM path a real confidence signal. thinSingleToken was narrowed on 01-08-2026, and agenda questions moved to stage 0. Missed clarify sits at 1 of 6 and false clarifies at 2. What is left on 319 is item 2 point 2: re-derive QueryMinMargin, or show the distribution the hand-picked 0.008 came from.
• 278's real ask is making the eval lab routine rather than building it. It is built. Decide whether it runs on a timer, on every merge, or on demand, and the task can close.
• 248 is the memory evaluation loop. It ships, it writes notes, and it cannot speak. make eval-recall covers the retrieval half. The open question is whether a written evaluation nobody reads is worth the tick.

323 is down to one check. PR #90 covered the spawn path and took phraser coverage to 76.9%. Only the 60s startup timeout arm is untested, because testing it needs a StartupTimeout field on Config rather than a test-only hack. While you are on the box, time a cold 1.7B load off spinning disk. If it runs near 60s, the default is too tight and the field earns itself twice.

Item 4 of 320 needs a permission I do not have. Kill the llama-server pid under maven-mavend-1, post a turn, and confirm it still completes through the classifier. Either grant it or run it yourself. It is the only check that the failure floor catches a mid-session model death.

---

Session 3: the interaction batch (a day, or three sittings)

These need real use rather than a command, grouped by what one sitting covers.

Morning and delivery (280, 281, 128, 282, 283, 285): open /morning, walk the seven required behaviours, then check the four interruption outcomes and the digest gap. 282 needs the desk_active script enabled on the desk PC first, which is 15 and needs you at that machine. 283 is the event intake envelope every reach shares, so a delivery check exercises it whether you name it or not. 285 is not verification: the bridge framework works and the remaining ask is more adapters. Decide which reach comes next, or park it.

Query sources (258, 286): ask her something the RSS feeds answer and something only a ZIM answers, with the search block on. Live search leads and the ZIMs are the fallback since 02-08-2026. A ZIM answer to a current-events question means the search leg failed silently. 286's remaining half is doc and git ingestion, which is build work, not a check.

Tasks and calendar (129, 130, 127, 126, 246): capture a task by voice, confirm it lands, check prioritisation ordering is not nonsense. 246 (mail reader) also exercises the IngestMail rung that moved to AuthWrite this morning.

Routines and patterns (43, 46, 247, 254): these need history to detect against. If the database is thin after the outage, they may have nothing to propose, which is not a failure. Check /routines before concluding anything.

Ecosystem (272, 273, 276): nexus, hexis and praxis are wired and logged clean at boot. 276 is the degraded-mode suite, which means taking siblings down on purpose. Worth doing while you are already in there.

Operations (249, 250): these bite hardest if they are broken, and nobody has pulled either lever on this box. Roll 249 forward and back once, then swap phraser.model_path and confirm 250 reloads without a restart. Do this sitting last, because both checks can take the box down.

---

Housekeeping (one sitting, no box needed)

Six QA tasks will not close no matter how long they sit, because they are gated on something that does not exist:

• 125 zenmoney: needs a token you have not minted.
• 256 Home Assistant: needs HA configured.
• 257 Bluetooth: BLOCKED, no bluez on the box. Says so in the title.
• 288 STT golden audio: needs fixtures generated.
• 14 cold-start unlock: the -wrapped-key-file seam exists, the passkey to L3 half does not. lockedAPI was deleted as dead code in PR #50, so there is nothing to verify.
• 284 replayable full-system simulator: nothing is built. This one is a design task wearing a QA: prefix.

Relabel these so they stop reading as backlog. They are not verification work that is pending, they are work that has not started.

Same treatment for the five plan-only tasks (251 MCP, 252 vision, 253 hearing, 255 speaker recognition, 259 crawler). A QA: prefix on a plan is misleading.

---

Needs you specifically

Not QA. These are blocked on a decision or a credential only you have.

#	what
16	Create the Kuma API key. -kuma-key uk5_mavpoll-key in docker-compose.yml is still the placeholder.
15	Deploy desk_active on the desk PC. Blocks 282.
122	Finish the CPT run for Qwen3-1.7B. The persona fix depends on it.
355	Deploy the Hexis auth change. Was blocked on Maven being under construction, which it no longer is. The client half is vendored and wired.
357	Decide whether entity-existence validation is the permanent target guard or whether blessing lands in Nexus.
275	Hexis native API and MCP parity.
—	Decide on -require-stepup. Making it the default needs WebAuthn configured first, or it locks you out of your own admin surfaces.

317 and 354 closed on 01-08-2026. The step-up gate now covers POST /api/chat and /routines, and the nginx template is locked down with a maven.<domain> block for mavweb. The -require-stepup default is still your call.

---

Not this repo

Two open tasks sit on the Maven board and are not Maven work. Move them or note where they land, so the board stops reading as 50 things Maven owes.

• 358 replace the rowid execution cursor with a real seq column. This is Hexis, and it must land before any execution retention or pruning does.
• 362 mirror the router prompt reorder into the relabelling prompt. This is the training workspace, enforced by llm/check_prompt_parity.py there, not here.

---

Suggested order

1. Session 1. If the voice loop is broken, nothing else matters.
2. The -require-stepup and Kuma decisions. Five minutes, and it unblocks 16.
3. Session 2. The numbers tell you whether the router is worth its ~27× latency. The 90× figure that stood here came from a 2.7s p50 that was contention, not the model. Real p50 is 825ms against the classifier's 31ms.
4. Housekeeping. Cheap, and it makes the remaining backlog honest.
5. Session 3, split whichever way suits you.