# QA plan: checking Maven properly

*Last verified: 2026-08-02 @ af9d213. Living doc: correct it in place, do not append.*

Written 2026-08-01, after the 35-PR stack landed and the box came back up.
Refreshed 2026-08-02 against the live list, after PRs #85-#90.

42 of the 50 open Vikunja tasks are `QA:` tasks. They are verification work, not
build work. Most sat unverifiable while Maven was down for 11 days. That
blocker is gone.

The plan as written on 2026-08-01 named 40 task numbers. Ten open `QA:` tasks were
missing and two of the named ones had closed. Every open task now appears below,
the eight non-QA ones in the last two sections.

This plan orders them by what unblocks what. Do sessions 1 and 2 first. Almost everything
downstream assumes the voice loop works, and nobody has confirmed that since
the redeploy.

---

## Before you start

Two things bite anyone running these checks on homesrv.

**curl needs `--noproxy '*'`.** The shell exports `http_proxy=http://127.0.0.1:18080`.
Without the flag, every local check returns 503 from the proxy and looks like a
dead service. This cost me a false regression report today.

**The database is not readable with sqlite3.** Four older QA steps say
`docker compose exec mavend sqlite3 /data/maven.db "select ..."`. That cannot
work: the container has no `sqlite3` binary, and the store is AES-256-GCM at
rest with a tmpfs working copy. Read state through mavweb instead, at
`/history`, `/trace`, `/routines` and `/dash`.

---

## Session 1: the voice loop (half a day)

Nothing here has been confirmed since the redeploy, and everything else assumes
it works. Do this first.

Closes or advances: **44** (conversation), **45** (text chat), **287** (voice
session quality), **321** steps 3-5 (quiet mode), **288** (STT fixtures).

Steps 1 and 3-6 were run on 02-08-2026 and pass. Steps 2 and 7-9 still need a
person at the box, because they need a microphone or a nudge to arrive.

Steps 1 and 3-6 do not need a browser. `POST /api/chat` takes a form-encoded
`text=` field and a cookie jar, and answers with the rendered `/chat` page:

```sh
curl -s --noproxy '*' -c jar -b jar -L -X POST \
  http://127.0.0.1:9201/api/chat --data-urlencode 'text=привет'
```

Parse the whole page, not the last text node. The page carries nav and footer
text. A naive tail of the Cyrillic nodes returns the wrong string, which makes
turns look misaligned when they are not.

1. Open `http://127.0.0.1:9201/chat` and hold a short conversation in Russian.
   Watch for three things: she answers in feminine forms (`рада`, `поняла`), she
   says `ты` and never `вы`, and no pet names appear.
   **Passes** (02-08-2026, five turns): `я рада`, `поняла`, `помогла`,
   `проверила`, `записала`, `грустна`, `ты` throughout, no pet names.
2. Press push-to-talk on `/dash`. Say `привет`. Confirm a spoken reply comes
   back. This is the only check that covers mic to STT to core to TTS to
   speaker as one path. It is also the path the eleven-day outage most likely
   broke.
3. Say `тихий режим`. Expect `тихий режим включён. буду реже напоминать.` **Passes.**
4. Say `выключи тихий режим`. Expect `тихий режим выключен.` Negation must win. **Passes.**
5. Say `в комнате тихо`. Quiet mode must NOT flip. Confirm on `/history` that no
   `quiet_hours` fact was written. **Passes**: no row written. She answers `пока
   не умею отвечать на этот вопрос.`, so it lands on `IntentSystem` with no arm.
6. Say `включи режим тишины`, then `сделай потише`. Both must flip quiet mode
   on. These are the noun form and the comparative, added 01-08-2026. **Both pass.**
7. Wait for a nudge, then say `потом` within twenty minutes. Expect `хорошо,
   вернусь к этому позже.` and the nudge row on `/notifications` reading
   `snoozed`. Say `потом` again with nothing pending: it must route as an
   ordinary utterance, not be swallowed.
8. Wait for the water nudge, then say `выпил воды`. Expect the ordinary fact
   reply and nothing extra. She must not congratulate you. Check
   `/notifications`: the row reads `acted`. Then trigger another nudge and say
   `готово`. Expect `отлично, отметила.` and the same outcome.
9. Note anything where she is slow, cuts off, or talks over herself. That is
   287's whole content and it has no written acceptance criteria yet.
   **First evidence, in text** (02-08-2026): nothing breaks, but answers wander
   and stitch unrelated topics. Asked whether he should move flats, she opened
   with the weather. That is 287, and it is a phrasing problem, not a loop problem.

**The wake path cannot be checked as deployed.** `mavwaked` and `mavenclient`
appear in no compose file and run as no host process. Step 2 covers only
push-to-talk, from `/dash` through mavsttd and mavttsd. Wake word and VAD
are untested by construction. Decide whether they belong in compose or on a
client machine, and say which in the deploy docs.

**319's single-token bug is fixed** (01-08-2026). Single-word Russian utterances no longer come
back as `не совсем поняла — можешь переформулировать?`. `привет` and `поужинал`
both pass now: `thinSingleToken` spares social singles and any token carrying a
verb ending, and only thins a bare nominal like `вода`. A one-word utterance that
still gets clarified in this session is a new case for the lexicon, not the old bug.

---

## Session 2: measurement (half a day, mostly waiting)

Closes or advances: **320** items 2-4, **278** (make the eval lab routine).
Also **248** (memory evaluation), **319** (the margin gate) and **323** (the
startup timeout arm).

The resident llama-server cannot be reached by the eval harness. It binds
`--host 127.0.0.1 --port 0` inside the container, so the port is kernel-assigned
and never published. Start a second one on a fixed port instead:

```sh
llama-server -m /mnt/hdd1/llms/qwen3/Qwen3-1.7B-UD-Q4_K_XL.gguf \
  --host 127.0.0.1 --port 18100 -c 4096 -ngl 99 --no-webui
```

`-c 4096` matters. The recorded numbers were measured at that context size, and
a mismatch invalidates the comparison.

Then:

```sh
make eval-models MAVEN_LLM_URL=http://127.0.0.1:18100   # want ~72.7% cascade
make eval-router                                         # classifier baseline
MAVEN_LLM_URL=http://127.0.0.1:18100 make eval-phrasing  # persona checks, slow
make eval-recall
```

A large miss against 72.7% means the deploy differs from the bench harness.

**Run on 02-08-2026 @ af9d213. The deploy matches the bench.** `eval-models`
scored 56 of 77: 72.7% full, 77.9% intent-only, 2 false clarifies and 1 missed.
That is the recorded figure to the decimal, and calendar sat at 2 of 2, so the
stage 0 agenda rules hold. `eval-phrasing` scored 21 of 27 on the talk fixture
against a recorded 20, and the 15 nudge templates passed every check.

Two numbers in this repo were wrong, and both flattered the resident model.

- **The classifier is not 36.8% and not 31ms.** `make eval-router` reports
  `classifier+onnx: 53/77 (68.8% full)` at p50 16.6µs. The figure repeated here
  and in `CLAUDE.md` predates the stage 0 rules and the seed additions. Both now
  score inside that baseline. The accuracy gap the router buys is
  roughly 4 points, not 36. Re-argue the trade on the real numbers.
- **Router latency was measured under contention again.** p50 1.126s, p95 1.58s,
  max 3.24s, against a recorded p50 825ms. The resident model was serving the
  daemon on the same iGPU throughout. Do not record this as a regression, and do
  not record it as a measurement either. Stop the stack before timing the router.

`classifier+hash` scores 19.5%, which is the no-ONNX degraded path and is not the
failure floor the deploy uses. Do not quote it as the classifier baseline.

Then three things to decide while the numbers are in front of you:

- **319 is done.** 359 gave the LLM path a real confidence signal.
  `thinSingleToken` was narrowed on 01-08-2026, and agenda questions moved to
  stage 0. Missed clarify sits at 1 of 6 and false clarifies at 2. Item 2 point 2
  closed on 02-08-2026: the `make eval-recall` margin sweep is the distribution
  that was asked for, and `0.008` sits at the knee.

  | delta | answered | false recall |
  |---|---|---|
  | 0.005 | 18/27 | 2/5 |
  | **0.008** | **18/27** | **1/5** |
  | 0.010 | 16/27 | 1/5 |

  It removes four of five false recalls at no cost in answers, and the next step
  costs two answers for nothing. The hand-picked value survives on evidence.
- **278's real ask** is making the eval lab routine rather than building it. It
  is built. Decide whether it runs on a timer, on every merge, or on demand, and
  the task can close.
- **248** is the memory evaluation loop. It ships, it writes notes, and it cannot
  speak. `make eval-recall` covers the retrieval half. The open question is whether
  a written evaluation nobody reads is worth the tick.

**323 is down to one check.** PR #90 covered the spawn path and took phraser
coverage to 76.9%. Only the 60s startup timeout arm is untested, because testing it
needs a `StartupTimeout` field on `Config` rather than a test-only hack. While you
are on the box, time a cold 1.7B load off spinning disk. If it runs near 60s, the
default is too tight and the field earns itself twice.

Warm, it is nowhere near. A second llama-server answered `/health` 1.8s after
launch at `n_ctx 4096` on 02-08-2026. That is page cache, so it does not settle
the question. A cold read needs a cache drop, which needs root.

**`CheckFeminine` has a false positive.** On 02-08-2026 it failed
`query-notes-do-not-answer` for `ты заплатил`, calling it masculine
self-reference. Masculine second person is correct, because the owner is male.
The check matches a masculine
past-tense verb before `за` without confirming the subject is `я`. Fix it in
`internal/phraser/eval/checks.go` before trusting a phrasing score to the case.
The real talk-fixture score on that run is 22 of 27, not 21.

Item 4 of **320** needs a permission I do not have. Kill the `llama-server`
pid under `maven-mavend-1`, post a turn, and confirm it still completes
through the classifier. Either grant it or run it yourself. It is the only
check that the failure floor catches a mid-session model death.

---

## Session 3: the interaction batch (a day, or five sittings)

These need real use rather than a command, grouped by what one sitting covers.

**Morning and delivery** (**280**, **281**, **128**, **282**, **283**, **285**):
open `/morning`, walk the seven required behaviours, then check the four
interruption outcomes and the digest gap. **282** needs the `desk_active` script
enabled on the desk PC first, which is **15** and needs you at that machine.
**283** is the event intake envelope every reach shares, so a delivery check
exercises it whether you name it or not. **285** is not verification: the bridge
framework works and the remaining ask is more adapters. Decide which reach comes
next, or park it.

**Query sources** (**258**, **286**): ask her something the RSS feeds answer and
something only a ZIM answers, with the search block on. Live search leads and the
ZIMs are the fallback since 02-08-2026. A ZIM answer to a current-events question
means the search leg failed silently. **286**'s remaining half is doc and
git ingestion, which is build work, not a check.

**Tasks and calendar** (**129**, **130**, **127**, **126**, **246**): capture a
task by voice, confirm it lands, check prioritisation ordering is not nonsense.
**246** (mail reader) also exercises the `IngestMail` rung that moved to
`AuthWrite` this morning.

**Routines and patterns** (**43**, **46**, **247**, **254**): these need history
to detect against. If the database is thin after the outage, they may have
nothing to propose, which is not a failure. Check `/routines` before
concluding anything.

**Ecosystem** (**272**, **273**, **276**): nexus, hexis and praxis are wired and
logged clean at boot. **276** is the degraded-mode suite, which means taking
siblings down on purpose. Worth doing while you are already in there.

**Operations** (**249**, **250**): these bite hardest if they are broken, and
nobody has pulled either lever on this box. Roll **249** forward and back once,
then swap `phraser.model_path` and confirm **250** reloads without a restart. Do
this sitting last, because both checks can take the box down.

---

## Housekeeping (one sitting, no box needed)

Six QA tasks will not close no matter how long they sit, because they are
gated on something that does not exist:

- **125** zenmoney: needs a token you have not minted.
- **256** Home Assistant: needs HA configured.
- **257** Bluetooth: BLOCKED, no bluez on the box. Says so in the title.
- **288** STT golden audio: needs fixtures generated.
- **14** cold-start unlock: the `-wrapped-key-file` seam exists, the passkey to L3
  half does not. `lockedAPI` was deleted as dead code in PR #50, so there is
  nothing to verify.
- **284** replayable full-system simulator: nothing is built. This one is a
  design task wearing a `QA:` prefix.

Relabel these so they stop reading as backlog. They are not verification work
that is pending, they are work that has not started.

Same treatment for the five plan-only tasks (**251** MCP, **252** vision,
**253** hearing, **255** speaker recognition, **259** crawler). A `QA:` prefix on
a plan is misleading.

---

## Needs you specifically

Not QA. These are blocked on a decision or a credential only you have.

| # | what |
|---|---|
| 16 | Create the Kuma API key. `-kuma-key uk5_mavpoll-key` in `docker-compose.yml` is still the placeholder. |
| 15 | Deploy `desk_active` on the desk PC. Blocks **282**. |
| 122 | Finish the CPT run for Qwen3-1.7B. The persona fix depends on it. |
| 355 | Deploy the Hexis auth change. Was blocked on Maven being under construction, which it no longer is. The client half is vendored and wired. |
| 357 | Decide whether entity-existence validation is the permanent target guard or whether blessing lands in Nexus. |
| 275 | Hexis native API and MCP parity. |
| — | Decide on `-require-stepup`. Making it the default needs WebAuthn configured first, or it locks you out of your own admin surfaces. |

317 and 354 closed on 01-08-2026. The step-up gate now covers `POST /api/chat` and
`/routines`, and the nginx template is locked down with a `maven.<domain>` block for
mavweb. The `-require-stepup` default is still your call.

---

## Not this repo

Two open tasks sit on the Maven board and are not Maven work. Move them or note
where they land, so the board stops reading as 50 things Maven owes.

- **358** replace the rowid execution cursor with a real seq column. This is Hexis,
  and it must land before any execution retention or pruning does.
- **362** mirror the router prompt reorder into the relabelling prompt. This is the
  training workspace, enforced by `llm/check_prompt_parity.py` there, not here.

---

## Suggested order

1. Session 1. If the voice loop is broken, nothing else matters.
2. The `-require-stepup` and Kuma decisions. Five minutes, and it unblocks **16**.
3. Session 2. **Run on 02-08-2026.** The numbers came back worse for the router
   than the docs claimed. The classifier is 68.8%, not 36.8%, and 16.6µs, not
   31ms. The router buys about 4 points of accuracy for four orders of magnitude
   of latency. Whether that still earns its place is now an open question.
4. Housekeeping. Cheap, and it makes the remaining backlog honest.
5. Session 3, split whichever way suits you.