Merge pull request 'Docs: refresh the QA plan against the live task list' (#91) from task/459-docs-refresh-the-qa-plan-against-the-liv into master
Reviewed-on: #91
This commit was merged in pull request #91.
This commit is contained in:
@@ -40,6 +40,9 @@ deploy/telegram.env
|
||||
deploy/zenmoney.token
|
||||
# IMAP password, read by mavmaild (never in argv, never committed)
|
||||
deploy/imap.password
|
||||
# Compose interpolation secrets — MAVEN_AMBIENT_TOKEN today. docker compose
|
||||
# reads this file itself; it is not an env_file on any service.
|
||||
/.env
|
||||
|
||||
# Temp files
|
||||
/tmp/
|
||||
|
||||
@@ -127,11 +127,13 @@ on in deploy** — this section used to say it was wired `nil`, which stopped be
|
||||
Cascade order: `stage0.go` exact-match fast-path → LLM router (when non-nil) → classifier
|
||||
fallback. Any LLM error falls through to the classifier so a turn never breaks on the model.
|
||||
|
||||
Measured on the 77-case RU fixture (`docs/evals/2026-07-31-model-bakeoff.md`): the classifier scores
|
||||
36.8% full accuracy at p50 31ms; Qwen3-1.7B scores 67.5% intent-only / 72.7% through the
|
||||
cascade at p50 ≈825ms. Accuracy roughly doubled, latency is ~27× worse, and that trade was
|
||||
accepted deliberately. **The ≈2.7s figure that stood here until 2026-08-02 was contention,
|
||||
not the model.** See `docs/evals/2026-07-31-routing.md` line 61, which measures the LLM router at
|
||||
Measured on the 77-case RU fixture. **Re-measured 2026-08-02: the classifier scores 68.8%
|
||||
full accuracy at p50 16.6µs**, not the 36.8% at p50 31ms that stood here from
|
||||
`docs/evals/2026-07-31-model-bakeoff.md`. That older figure predates the stage 0 rules and the
|
||||
seed additions, both of which now score inside the classifier baseline. Qwen3-1.7B scores
|
||||
77.9% intent-only / 72.7% through the cascade. So the router buys about 4 points of accuracy,
|
||||
not a doubling, and the trade is worth re-arguing rather than assuming. **The ≈2.7s figure
|
||||
that stood here until 2026-08-02 was contention, not the model.** See `docs/evals/2026-07-31-routing.md` line 61, which measures the LLM router at
|
||||
p50 825ms / p95 1.2s / max 3.0s and the full cascade at p50 0.80-1.04s. Do not plan latency
|
||||
work off the bakeoff table. `Confidence: 1.0` used to be hardcoded in `llmrouter.go`, so the LLM
|
||||
path could never ask for clarification (6/6 refusal cases missed on the fixture) — Vikunja
|
||||
|
||||
+51
-1
@@ -76,6 +76,56 @@
|
||||
"snippet_runes": 1500
|
||||
},
|
||||
|
||||
"//morning_routines": [
|
||||
"The daily checklist (Vikunja #280). Each item is done when its fact_key",
|
||||
"gets a non-voided fact inside the window, so 'выпил воды' closes water and",
|
||||
"nothing has to be ticked by hand. nudge_at fires once, at the end of the",
|
||||
"window, and only for what is still open. Weekdays empty = every day."
|
||||
],
|
||||
"morning_routines": [
|
||||
{
|
||||
"name": "утро",
|
||||
"window_start": "08:00",
|
||||
"window_end": "11:00",
|
||||
"nudge_at": "10:30",
|
||||
"severity": 1,
|
||||
"items": [
|
||||
{ "key": "medicine", "fact_key": "medicine", "label": "лекарство" },
|
||||
{ "key": "water", "fact_key": "water", "label": "вода" },
|
||||
{ "key": "pets", "fact_key": "pets", "label": "покормить кота" }
|
||||
]
|
||||
}
|
||||
],
|
||||
|
||||
"//feeds": [
|
||||
"RSS reading (Vikunja #258). Every item lands as a note with source",
|
||||
"rss:<name>, which is also what puts entries in the intake journal that",
|
||||
"/events reads. Only the feed URL leaves the box.",
|
||||
"This is a starting pair, not a curated set — trim or extend it."
|
||||
],
|
||||
"feeds": {
|
||||
"poll_interval": "30m",
|
||||
"max_items": 5,
|
||||
"max_age": "24h",
|
||||
"sources": [
|
||||
{ "name": "lwn", "url": "https://lwn.net/headlines/newrss", "category": "технологии" },
|
||||
{ "name": "archlinux", "url": "https://archlinux.org/feeds/news/", "category": "технологии" }
|
||||
]
|
||||
},
|
||||
|
||||
"//crawl": [
|
||||
"Reading a web page (Vikunja #259). on_demand answers 'посмотри <URL>'.",
|
||||
"No allow_hosts, so any public host he names is readable; private",
|
||||
"addresses are refused unconditionally by internal/webfetch and do not",
|
||||
"need listing. Setting allow_hosts here would also narrow on-demand,",
|
||||
"which is the point of leaving it empty."
|
||||
],
|
||||
"crawl": {
|
||||
"on_demand": true,
|
||||
"timeout": "10s",
|
||||
"max_runes": 4000
|
||||
},
|
||||
|
||||
"digest": {
|
||||
"enabled": true,
|
||||
"window": "30m",
|
||||
@@ -119,7 +169,7 @@
|
||||
"timeout": "400ms",
|
||||
"rate": 100,
|
||||
"max_hosts": 256,
|
||||
"enabled": false
|
||||
"enabled": true
|
||||
},
|
||||
|
||||
"nexus": { "url": "http://nexus:9740" },
|
||||
|
||||
@@ -79,7 +79,16 @@ services:
|
||||
<<: *image
|
||||
# voice.bind is 0.0.0.0:9100 in deploy/mavend.json so mavweb can reach it
|
||||
# cross-container. Verified 2026-07-06.
|
||||
# -ambient-token turns on POST /api/ambient (Vikunja #126): the phone posts
|
||||
# notification text, mavweb keeps only a meeting time. Empty ⇒ no route at
|
||||
# all, which is what a missing MAVEN_AMBIENT_TOKEN gives. The value comes
|
||||
# from the gitignored .env docker compose reads for interpolation, NOT from
|
||||
# an env_file — flags are interpolated before any service env exists.
|
||||
# Weakness worth naming: mavweb takes this as a flag, so it is visible in
|
||||
# `ps` inside this container, unlike the zenmoney and IMAP secrets which are
|
||||
# read from files.
|
||||
command: ["mavweb", "-addr", ":9201", "-voice", "mavend:9100", "-core", "/run/maven/mavend.sock",
|
||||
"-ambient-token", "${MAVEN_AMBIENT_TOKEN:-}",
|
||||
"-nexus", "http://nexus:9740", "-praxis", "http://praxis:8989", "-hexis", "http://hexis:9741"]
|
||||
depends_on: [mavend]
|
||||
# loopback-only on purpose: /tools defines+executes arbitrary argv and
|
||||
|
||||
+415
-44
@@ -1,19 +1,62 @@
|
||||
# QA plan: checking Maven properly
|
||||
|
||||
*Last verified: 2026-08-02 @ 7079a24. Living doc: correct it in place, do not append.*
|
||||
*Last verified: 2026-08-02 @ 20aa2d5. Living doc: correct it in place, do not append.*
|
||||
|
||||
Written 2026-08-01, after the 35-PR stack landed and the box came back up.
|
||||
Refreshed 2026-08-02 against the live list, after PRs #85-#90.
|
||||
|
||||
44 of the 50 open Vikunja tasks are `QA:` tasks. They are verification work, not
|
||||
42 of the 50 open Vikunja tasks are `QA:` tasks. They are verification work, not
|
||||
build work. Most sat unverifiable while Maven was down for 11 days. That
|
||||
blocker is gone.
|
||||
|
||||
The plan as written on 2026-08-01 named 40 task numbers. Ten open `QA:` tasks were
|
||||
missing and two of the named ones had closed. Every open task now appears below,
|
||||
the eight non-QA ones in the last two sections.
|
||||
|
||||
This plan orders them by what unblocks what. Do sessions 1 and 2 first. Almost everything
|
||||
downstream assumes the voice loop works, and nobody has confirmed that since
|
||||
the redeploy.
|
||||
|
||||
---
|
||||
|
||||
## What the 02-08-2026 run found
|
||||
|
||||
Sessions 1, 2 and 3 all ran. Read these five before picking anything up.
|
||||
|
||||
- **470: a question writes invented knowledge into memory.** Recall then serves
|
||||
it back. `что дальше?` lands on `IntentFact` and stores the model's answer as a
|
||||
`self` fact at confidence 1.00. Two junk rows then claimed seven unrelated
|
||||
world questions through recall, outranking the search leg. A question about the
|
||||
capital of Australia was answered `какая последняя версия языка Go?`. Two bad
|
||||
writes silently disabled world answering, with nothing logged.
|
||||
- **466: a pending clarify is global.** One unanswerable clarify swallowed the
|
||||
next three utterances from three separate sessions. With ntfy, telegram and
|
||||
voice all live, a clarify raised on web chat eats the next telegram message.
|
||||
- **467: spoken task capture is dead.** The router calls the capture marker an
|
||||
`act`, and capture is reachable only from the `note` intent.
|
||||
- **The classifier baseline in this repo was wrong**, and it flattered the
|
||||
router. See session 2 and **464**.
|
||||
- **477: the model swap and the self-update cannot be triggered on this box.**
|
||||
Both are built and both are correct in test. The swap needs a passkey and
|
||||
WebAuthn is unconfigured. `mavupdate` needs to reach a socket that only an
|
||||
in-container uid can open.
|
||||
|
||||
- **479: an unconfigured capability lets the question escape to web search.**
|
||||
Netscan off, asked `какие устройства в сети?`. She answered from the live web
|
||||
with a general article about network hardware. A question about his LAN went to
|
||||
an upstream engine. The crawler fails the same way.
|
||||
|
||||
Twenty-one defects were filed on 02-08-2026: 462 through 482. Six tasks this plan
|
||||
had written off as blocked turned out to be ready to check. All six ran. Every
|
||||
one of them is code-correct and stops at the deploy.
|
||||
|
||||
Three of the five config blockers in **472** were then cleared. The morning
|
||||
routine, ambient ingest, feeds, the crawler and netscan are all live. Two remain,
|
||||
and both are the owner's call: a token for each ecosystem sibling, and seed data
|
||||
in Nexus and Praxis.
|
||||
|
||||
---
|
||||
|
||||
## Before you start
|
||||
|
||||
Two things bite anyone running these checks on homesrv.
|
||||
@@ -36,45 +79,75 @@ Nothing here has been confirmed since the redeploy, and everything else assumes
|
||||
it works. Do this first.
|
||||
|
||||
Closes or advances: **44** (conversation), **45** (text chat), **287** (voice
|
||||
session quality), **321** steps 3-5 (quiet mode), **288** (STT fixtures).
|
||||
session quality), **321** steps 3-5 (quiet mode), **288** (STT golden audio).
|
||||
|
||||
**288 is not blocked.** The fixtures are committed under `cmd/mavsttd/testdata/`
|
||||
and `make test-stt-golden` runs today. This plan said otherwise until 02-08-2026.
|
||||
|
||||
Steps 1 and 3-6 were run on 02-08-2026 and pass. Steps 2 and 7-9 still need a
|
||||
person at the box, because they need a microphone or a nudge to arrive.
|
||||
|
||||
Steps 1 and 3-6 do not need a browser. `POST /api/chat` takes a form-encoded
|
||||
`text=` field and a cookie jar, and answers with the rendered `/chat` page:
|
||||
|
||||
```sh
|
||||
curl -s --noproxy '*' -c jar -b jar -L -X POST \
|
||||
http://127.0.0.1:9201/api/chat --data-urlencode 'text=привет'
|
||||
```
|
||||
|
||||
Parse the whole page, not the last text node. The page carries nav and footer
|
||||
text. A naive tail of the Cyrillic nodes returns the wrong string, which makes
|
||||
turns look misaligned when they are not.
|
||||
|
||||
1. Open `http://127.0.0.1:9201/chat` and hold a short conversation in Russian.
|
||||
Watch for three things: she answers in feminine forms (`рада`, `поняла`), she
|
||||
says `ты` and never `вы`, and no pet names appear.
|
||||
**Passes** (02-08-2026, five turns): `я рада`, `поняла`, `помогла`,
|
||||
`проверила`, `записала`, `грустна`, `ты` throughout, no pet names.
|
||||
2. Press push-to-talk on `/dash`. Say `привет`. Confirm a spoken reply comes
|
||||
back. This is the only check that covers mic to STT to core to TTS to
|
||||
speaker as one path. It is also the path the eleven-day outage most likely
|
||||
broke.
|
||||
3. Say `тихий режим`. Expect `тихий режим включён. буду реже напоминать.`
|
||||
4. Say `выключи тихий режим`. Expect `тихий режим выключен.` Negation must win.
|
||||
3. Say `тихий режим`. Expect `тихий режим включён. буду реже напоминать.` **Passes.**
|
||||
4. Say `выключи тихий режим`. Expect `тихий режим выключен.` Negation must win. **Passes.**
|
||||
5. Say `в комнате тихо`. Quiet mode must NOT flip. Confirm on `/history` that no
|
||||
`quiet_hours` fact was written.
|
||||
`quiet_hours` fact was written. **Passes**: no row written. She answers `пока
|
||||
не умею отвечать на этот вопрос.`, so it lands on `IntentSystem` with no arm.
|
||||
6. Say `включи режим тишины`, then `сделай потише`. Both must flip quiet mode
|
||||
on. These are the noun form and the comparative, added 01-08-2026.
|
||||
on. These are the noun form and the comparative, added 01-08-2026. **Both pass.**
|
||||
7. Wait for a nudge, then say `потом` within twenty minutes. Expect `хорошо,
|
||||
вернусь к этому позже.` and the nudge row on `/notifications` reading
|
||||
`snoozed`. Say `потом` again with nothing pending: it must route as an
|
||||
ordinary utterance, not be swallowed.
|
||||
8. Wait for the water nudge, then say `выпил воды`. Expect the ordinary fact
|
||||
reply and nothing extra — she must not congratulate you. Check
|
||||
reply and nothing extra. She must not congratulate you. Check
|
||||
`/notifications`: the row reads `acted`. Then trigger another nudge and say
|
||||
`готово`; expect `отлично, отметила.` and the same outcome.
|
||||
`готово`. Expect `отлично, отметила.` and the same outcome.
|
||||
9. Note anything where she is slow, cuts off, or talks over herself. That is
|
||||
287's whole content and it has no written acceptance criteria yet.
|
||||
**First evidence, in text** (02-08-2026): nothing breaks, but answers wander
|
||||
and stitch unrelated topics. Asked whether he should move flats, she opened
|
||||
with the weather. That is 287, and it is a phrasing problem, not a loop problem.
|
||||
|
||||
**319 is fixed** (01-08-2026). Single-word Russian utterances no longer come
|
||||
**The wake path cannot be checked as deployed.** `mavwaked` and `mavenclient`
|
||||
appear in no compose file and run as no host process. Step 2 covers only
|
||||
push-to-talk, from `/dash` through mavsttd and mavttsd. Wake word and VAD
|
||||
are untested by construction. Decide whether they belong in compose or on a
|
||||
client machine, and say which in the deploy docs. Tracked as **463**.
|
||||
|
||||
**319's single-token bug is fixed** (01-08-2026). Single-word Russian utterances no longer come
|
||||
back as `не совсем поняла — можешь переформулировать?`. `привет` and `поужинал`
|
||||
both pass now: `thinSingleToken` spares social singles and any token carrying a
|
||||
verb ending, and only thins a bare nominal like `вода`. If a one-word utterance
|
||||
still gets clarified during the smoke test, that is a new case for the lexicon,
|
||||
not the old bug.
|
||||
verb ending, and only thins a bare nominal like `вода`. A one-word utterance that
|
||||
still gets clarified in this session is a new case for the lexicon, not the old bug.
|
||||
|
||||
---
|
||||
|
||||
## Session 2: measurement (half a day, mostly waiting)
|
||||
|
||||
Closes or advances: **320** items 2-4, **278** (make the eval lab routine),
|
||||
**319** (gate recalibration).
|
||||
Closes or advances: **320** items 2-4, **278** (make the eval lab routine).
|
||||
Also **248** (memory evaluation), **319** (the margin gate) and **323** (the
|
||||
startup timeout arm).
|
||||
|
||||
The resident llama-server cannot be reached by the eval harness. It binds
|
||||
`--host 127.0.0.1 --port 0` inside the container, so the port is kernel-assigned
|
||||
@@ -99,14 +172,67 @@ make eval-recall
|
||||
|
||||
A large miss against 72.7% means the deploy differs from the bench harness.
|
||||
|
||||
Two things to decide while the numbers are in front of you:
|
||||
**Run on 02-08-2026 @ af9d213. The deploy matches the bench.** `eval-models`
|
||||
scored 56 of 77: 72.7% full, 77.9% intent-only, 2 false clarifies and 1 missed.
|
||||
That is the recorded figure to the decimal, and calendar sat at 2 of 2, so the
|
||||
stage 0 agenda rules hold. `eval-phrasing` scored 21 of 27 on the talk fixture
|
||||
against a recorded 20, and the 15 nudge templates passed every check.
|
||||
|
||||
- **319's gate recalibration.** The single-token rule needs narrowing or
|
||||
dropping. This needs your judgement, not a threshold sweep. The fixture and the
|
||||
daemon disagree about what is correct on two of the three false clarifies.
|
||||
Two numbers in this repo were wrong, and both flattered the resident model.
|
||||
|
||||
- **The classifier is not 36.8% and not 31ms.** `make eval-router` reports
|
||||
`classifier+onnx: 53/77 (68.8% full)` at p50 16.6µs. The figure repeated here
|
||||
and in `CLAUDE.md` predates the stage 0 rules and the seed additions. Both now
|
||||
score inside that baseline. The accuracy gap the router buys is
|
||||
roughly 4 points, not 36. Re-argue the trade on the real numbers: **464**.
|
||||
- **Router latency was measured under contention again.** p50 1.126s, p95 1.58s,
|
||||
max 3.24s, against a recorded p50 825ms. The resident model was serving the
|
||||
daemon on the same iGPU throughout. Do not record this as a regression, and do
|
||||
not record it as a measurement either. Stop the stack before timing the router.
|
||||
|
||||
`classifier+hash` scores 19.5%, which is the no-ONNX degraded path and is not the
|
||||
failure floor the deploy uses. Do not quote it as the classifier baseline.
|
||||
|
||||
Then three things to decide while the numbers are in front of you:
|
||||
|
||||
- **319 is done.** 359 gave the LLM path a real confidence signal.
|
||||
`thinSingleToken` was narrowed on 01-08-2026, and agenda questions moved to
|
||||
stage 0. Missed clarify sits at 1 of 6 and false clarifies at 2. Item 2 point 2
|
||||
closed on 02-08-2026: the `make eval-recall` margin sweep is the distribution
|
||||
that was asked for, and `0.008` sits at the knee.
|
||||
|
||||
| delta | answered | false recall |
|
||||
|---|---|---|
|
||||
| 0.005 | 18/27 | 2/5 |
|
||||
| **0.008** | **18/27** | **1/5** |
|
||||
| 0.010 | 16/27 | 1/5 |
|
||||
|
||||
It removes four of five false recalls at no cost in answers, and the next step
|
||||
costs two answers for nothing. The hand-picked value survives on evidence.
|
||||
- **278's real ask** is making the eval lab routine rather than building it. It
|
||||
is built. Decide whether it runs on a timer, on every merge, or on demand, and
|
||||
the task can close.
|
||||
- **248** is the memory evaluation loop. It ships, it writes notes, and it cannot
|
||||
speak. `make eval-recall` covers the retrieval half. The open question is whether
|
||||
a written evaluation nobody reads is worth the tick.
|
||||
|
||||
**323 is down to one check.** PR #90 covered the spawn path and took phraser
|
||||
coverage to 76.9%. Only the 60s startup timeout arm is untested, because testing it
|
||||
needs a `StartupTimeout` field on `Config` rather than a test-only hack. While you
|
||||
are on the box, time a cold 1.7B load off spinning disk. If it runs near 60s, the
|
||||
default is too tight and the field earns itself twice.
|
||||
|
||||
Warm, it is nowhere near. A second llama-server answered `/health` 1.8s after
|
||||
launch at `n_ctx 4096` on 02-08-2026. That is page cache, so it does not settle
|
||||
the question. A cold read needs a cache drop, which needs root.
|
||||
|
||||
**`CheckFeminine` has a false positive.** On 02-08-2026 it failed
|
||||
`query-notes-do-not-answer` for `ты заплатил`, calling it masculine
|
||||
self-reference. Masculine second person is correct, because the owner is male.
|
||||
The check matches a masculine
|
||||
past-tense verb before `за` without confirming the subject is `я`. Fix it in
|
||||
`internal/phraser/eval/checks.go` before trusting a phrasing score to the case.
|
||||
The real talk-fixture score on that run is 22 of 27, not 21. Tracked as **462**.
|
||||
|
||||
Item 4 of **320** needs a permission I do not have. Kill the `llama-server`
|
||||
pid under `maven-mavend-1`, post a turn, and confirm it still completes
|
||||
@@ -115,47 +241,269 @@ check that the failure floor catches a mid-session model death.
|
||||
|
||||
---
|
||||
|
||||
## Session 3: the interaction batch (a day, or three sittings)
|
||||
## Session 3: the interaction batch (a day, or five sittings)
|
||||
|
||||
These need real use rather than a command, grouped by what one sitting covers.
|
||||
|
||||
**Morning and delivery** (**280**, **281**, **128**, **282**): open `/morning`,
|
||||
walk the seven required behaviours, then check the four interruption outcomes
|
||||
and the digest gap. **282** needs the `desk_active` script enabled on the desk
|
||||
PC first, which is **15** and needs you at that machine.
|
||||
**Morning and delivery** (**280**, **281**, **128**, **282**, **283**, **285**):
|
||||
open `/morning`, walk the seven required behaviours, then check the four
|
||||
interruption outcomes and the digest gap. **282** needs the `desk_active` script
|
||||
enabled on the desk PC first, which is **15** and needs you at that machine.
|
||||
**283** is the event intake envelope every reach shares, so a delivery check
|
||||
exercises it whether you name it or not. **285** is not verification: the bridge
|
||||
framework works and the remaining ask is more adapters. Decide which reach comes
|
||||
next, or park it.
|
||||
|
||||
Run 02-08-2026. **280 is blocked.** No morning routine is configured (**472**).
|
||||
`morning.Item` also has no required-versus-optional field, so behaviour 1 cannot
|
||||
hold whatever you configure (**473**). **281's digest gap is closed**, and
|
||||
its presence rule passes on inspection. Three of its five items need traffic the
|
||||
box has not had. **283 is blocked**: nothing feeds the intake journal. **128
|
||||
found the worst defect of the whole session, see below.**
|
||||
|
||||
Three of 472's five blockers were cleared the same day, in `deploy/mavend.json`
|
||||
and `docker-compose.yml`.
|
||||
|
||||
- A `morning_routines` block, one routine `утро` 08:00-11:00 with medicine,
|
||||
water and pets. It is live: the dispatcher logged `dropped morning:утро (sev1,
|
||||
presence=away)`, so the plan builds and the nudge is proposed. 280's
|
||||
behaviours and 128 step 11 are checkable now. 473 still stands.
|
||||
- `-ambient-token` on mavweb, value in a gitignored `/.env` that docker compose
|
||||
reads for interpolation. `/api/ambient` answers 401 without the token and 201
|
||||
with it, storing `calendar_event_20260802_Standup`. 283 step 5 and 128 step 8
|
||||
are unblocked. The token is a flag, so it shows in `ps` inside that container.
|
||||
The zenmoney and IMAP secrets are read from files instead. Ingest also
|
||||
reads the notification's wall clock as UTC and stores a 14:30 meeting at 18:30
|
||||
(**482**).
|
||||
- `feeds` (two sources), `crawl.on_demand` and `netscan.enabled`. The intake
|
||||
journal now fills: `/events` holds `scan:lan` and `ambient:notif` rows.
|
||||
|
||||
Two are not mine to clear. No sibling has a `token` in `deploy/mavend.json`, so
|
||||
273 steps 6 and 8 need a credential decision. Nexus has no entities and Praxis no
|
||||
attention items, so 272 step 3 needs seed data whose content is the owner's call.
|
||||
|
||||
For **285**, two facts bear on the choice. Synapse is already running on this box
|
||||
and healthy, so a Matrix reach has a live target and needs no new service. And
|
||||
mavweb is already a PWA with a service worker, which 285 itself calls the highest
|
||||
value adapter left. Today's reaches are ntfy, telegram and voice.
|
||||
|
||||
**Query sources** (**258**, **286**): ask her something the RSS feeds answer and
|
||||
something only a ZIM answers, with the search block on. Live search leads and the
|
||||
ZIMs are the fallback since 02-08-2026. **286**'s remaining half is doc and
|
||||
git ingestion, which is build work, not a check.
|
||||
|
||||
**Do not read `/trace` for this.** `/trace` is the nudge-rule trace: rule,
|
||||
severity, predicate, gate, selected. No query-source field exists anywhere in the
|
||||
codebase. The only evidence of which query source claimed a turn is the
|
||||
`voice: search:` and `voice: kiwix:` lines in `docker compose logs mavend`
|
||||
(`actions_query.go:589` and `:660`).
|
||||
|
||||
Run 02-08-2026, 20 turns. **Search leads and the personal boundary holds.** Every
|
||||
world question that reached the boundary was claimed by search. All three
|
||||
personal questions produced no search and no kiwix line at all.
|
||||
|
||||
The rest of this sitting went badly. **Kiwix has zero live coverage.** SearXNG
|
||||
returns four results for everything, including two invented nonsense terms. So
|
||||
`querySearch` always claims, and Kiwix is unreachable code as deployed. The ZIM
|
||||
half of the 02-08-2026 decision is unverified. A ZIM answer cannot signal a
|
||||
silent search failure, because a ZIM answer cannot happen.
|
||||
**Ordering defects** in feeds and calendar, plus 258 step 1's utterance not
|
||||
working: **474**. And the sitting independently found stage 2 of **470**.
|
||||
|
||||
**Tasks and calendar** (**129**, **130**, **127**, **126**, **246**): capture a
|
||||
task by voice, confirm it lands, check prioritisation ordering is not nonsense.
|
||||
**246** (mail reader) also exercises the `IngestMail` rung that moved to
|
||||
`AuthWrite` this morning.
|
||||
|
||||
Run 02-08-2026. **129 passes.** The page and the spoken answer agree on ordering.
|
||||
The undistinguished task carries no invented reason on either surface, which is
|
||||
the thing 129 asks for. **130 fails outright** and **127 half fails**:
|
||||
**467**, **469**. **246 cannot be run**: `mavmaild` is commented out in
|
||||
`docker-compose.yml` and there is no `email` block, so nothing in steps 4-13 is
|
||||
reachable. The `IngestMail` rung does sit at `AuthWrite`
|
||||
(`internal/auth/policy.go:96`, asserted in `auth_test.go:421`), verified by
|
||||
reading only.
|
||||
|
||||
**Routines and patterns** (**43**, **46**, **247**, **254**): these need history
|
||||
to detect against. If the database is thin after the outage, they may have
|
||||
nothing to propose, which is not a failure. Check `/routines` before
|
||||
concluding anything.
|
||||
|
||||
Run 02-08-2026. The answer is the middle case: **the detector ran and found
|
||||
nothing.** The tick loop is live, and `detectPatterns` is called unconditionally
|
||||
at `cmd/mavend/tick.go:227`. It has run about 25 times since the restart. It
|
||||
finds nothing because the events table is empty upstream of it. Rows land there
|
||||
only from `pattern.Extract` at fact-write time, and `Extract` requires the fact
|
||||
value to match a closed 7-action lexicon. All 200 facts on `/history` are
|
||||
`page_heartbeat`, `netdata_alarm`, `quiet_hours`, `name`, `service_down` and
|
||||
`рост`. Not one lexicon hit, so no event can exist, let alone the four one pair
|
||||
needs. **46 step 5 passes**: `/routines` renders `noticed 0` with the empty state
|
||||
and the hint string.
|
||||
|
||||
Two things block this sitting, and both are build work. The seeding recipe on
|
||||
**43** goes through `sqlite3` and cannot work. And `pattern.Detect` has no
|
||||
minimum-interval floor, so seeding by hand mints a permanent false routine
|
||||
(**468**). Do not try to seed a pattern with four fast chat turns.
|
||||
|
||||
**Ecosystem** (**272**, **273**, **276**): nexus, hexis and praxis are wired and
|
||||
logged clean at boot. **276** is the degraded-mode suite, which means taking
|
||||
siblings down on purpose. Worth doing while you are already in there.
|
||||
logged clean at boot.
|
||||
|
||||
Run 02-08-2026, read-only half. All three answer `/health` 200 and `/ecosystem`
|
||||
lists 18 Hexis capabilities with correct read-only and mutating badges. **272 and
|
||||
273 are blocked on empty data**, not on code. Nexus holds no entities, Praxis
|
||||
holds no attention items, and the Calls panel has never recorded a call. See
|
||||
**472**, and read its warning first. 273's trace fix has never been validated
|
||||
here. An empty Calls panel is exactly what the old bug looked like. The page is
|
||||
`/ecosystem`, not `/siblings`.
|
||||
|
||||
**276 ran 02-08-2026 and the suite is sound.** 17 `TestEcosystem_` cases pass
|
||||
under `-race`, not the 10 the task describes. The mutation check bites: patching
|
||||
the Nexus-error branch of `handleHexisAct` to `return ""` fails
|
||||
`TestEcosystem_MalformedNexusResponseFailsClosed` on the expected line.
|
||||
|
||||
Steps 4 and 6 could not be checked through chat, because no utterance reaches
|
||||
Praxis (**475**). «что требует внимания» routes to `intent=query` and is answered
|
||||
by the search leg, identically whether `ecosystem-praxis-1` is up or stopped. The
|
||||
degraded string never appears because its branch is never entered. Step 5 is
|
||||
blocked the same way: `перезапусти muzick indexer` clarifies on
|
||||
`HasFn:false`, and the router had already rewritten the entity name to
|
||||
`музик индексер` (**476**).
|
||||
|
||||
Both steps were checked on `/ecosystem` instead, which reads Praxis directly.
|
||||
With Praxis stopped the card reads `praxis — unreachable` while Nexus and Hexis
|
||||
keep rendering. On `docker start` the card returns to `nothing needs attention.`
|
||||
with no mavend restart. Independent degradation and recovery both hold.
|
||||
|
||||
**Operations** (**249**, **250**): both ran 02-08-2026. The code is correct and
|
||||
neither lever can be pulled on this box. See **477**.
|
||||
|
||||
**250** passes steps 1, 2, 3, 9 and 10 on the deploy. The capability announces
|
||||
itself. `/models` names the model llama-server reports, not the config filename.
|
||||
Asking her to switch models does nothing. Removing `swap_models` renders `swap
|
||||
not configured`. Step 4's refusal half passes at HTTP 403, and the 403 comes from
|
||||
mavend rather than mavweb. WebAuthn is unconfigured, so the web gate fails open
|
||||
and the wire gate fails closed. Steps 5 to 8 need a passkey assertion nothing on
|
||||
this box can produce. They pass in test: 13 swap cases and 7 page cases covering
|
||||
drain, mid-swap refusal, rollback, failed rollback and the not-owned refusal.
|
||||
|
||||
**249** passes steps 1 and 2. Step 3 stops it. `mavupdate` health-checks over
|
||||
`/run/maven/mavend.sock`, which is `srw------- 1 10001 999` inside a docker
|
||||
volume. The host owner cannot traverse `/var/lib/docker/volumes` and cannot
|
||||
connect to a socket owned by an in-container uid. `mavupdate` assumes a
|
||||
host-installed daemon and the deploy is containers. Do not sudo around this.
|
||||
|
||||
---
|
||||
|
||||
## Housekeeping (one sitting, no box needed)
|
||||
## Housekeeping (done 02-08-2026, and this section was mostly wrong)
|
||||
|
||||
Four QA tasks will not close no matter how long they sit, because they are
|
||||
gated on something that does not exist:
|
||||
This section claimed eleven tasks were not verification work. **Three were not.
|
||||
The other eight are.** Every one of the eight has shipped, tested code behind it.
|
||||
The error ran one way: it wrote off work that is ready to check. Do not trust a
|
||||
"nothing is built" line in this plan without grepping for the package first.
|
||||
|
||||
- **125** zenmoney: needs a token you have not minted.
|
||||
- **256** Home Assistant: needs HA configured.
|
||||
- **257** Bluetooth: BLOCKED, no bluez on the box. Says so in the title.
|
||||
- **288** STT golden audio: needs fixtures generated.
|
||||
Relabelled to `Blocked:`, claim verified:
|
||||
|
||||
Relabel these so they stop reading as backlog. They are not verification work
|
||||
that is pending, they are work that has not started.
|
||||
- **125** zenmoney. `internal/zenmoney/` ships and is tested against a fixture.
|
||||
`deploy/zenmoney.token` does not exist and the compose mount is commented out.
|
||||
One token unblocks it.
|
||||
- **256** Home Assistant. `internal/smarthome/` ships, the `smarthome` block sits
|
||||
in `deploy/mavend.json` at `enabled: false`, and 8123 and 1883 are closed.
|
||||
- **14** cold-start unlock. The seam is real at `cmd/mavend/main.go:128` and
|
||||
`internal/webauthn/prf.go` is in place. `lockedAPI` is gone, replaced by
|
||||
`Server.Check` in `internal/ipc/server.go`. Gated on an authenticator that
|
||||
implements the WebAuthn PRF extension, which is hardware, not code.
|
||||
|
||||
Same treatment for the five plan-only tasks (**251** MCP, **252** vision,
|
||||
**253** hearing, **255** speaker recognition, **259** crawler). A `QA:` prefix on
|
||||
a plan is misleading.
|
||||
Left alone, because the claim here was false:
|
||||
|
||||
- **284** simulator. `cmd/mavend/simulator_test.go`, three scenarios under
|
||||
`cmd/mavend/testdata/scenarios/`, and a `simulate` target at `Makefile:98`.
|
||||
**Run 02-08-2026: all three scenarios pass**, plus the determinism and
|
||||
backwards-step guards. One defect found, see below.
|
||||
- **288** STT golden audio. Four WAVs and `golden_v1.json` are committed under
|
||||
`cmd/mavsttd/testdata/`, the make targets exist, and `models/stt/ggml-small.bin`
|
||||
is on the box. Session 1 lists 288 as blocked on fixtures, which is wrong.
|
||||
**Run 02-08-2026: all four pass**, WER at or under ceiling with no drift.
|
||||
|
||||
| fixture | transcript | WER | ceiling |
|
||||
|---|---|---|---|
|
||||
| ru_reminder | `Напомни мне через час позвонить маме.` | 0.00 | 0.10 |
|
||||
| ru_fact | `А отметь, что я выпил воды.` | 0.20 | 0.25 |
|
||||
| ru_query | `Что у меня сегодня по календарю?` | 0.00 | 0.10 |
|
||||
| en_act | `Restart the web server and check the disk space.` | 0.00 | 0.10 |
|
||||
|
||||
That also settles a session 1 worry indirectly: whisper.cpp works on Vulkan
|
||||
after the redeploy. Only the mic and the wake path remain unproven.
|
||||
|
||||
**The simulator routes with an empty seed set.** Every `make simulate` run logs
|
||||
`loaded 0 seed examples from models/seeds`, seven times per scenario. The test
|
||||
runs from `cmd/mavend`, and the seed path is relative to the repo root. The
|
||||
scenarios still pass, which means they pass without the classifier having any
|
||||
seeds to match against. Whatever 284 is proving, it is not proving the routing
|
||||
the deploy runs. Fix the path before trusting a green simulator.
|
||||
- **257** Bluetooth. The bluez half is genuinely absent. The LAN-scan half shipped
|
||||
(`internal/netscan/`), and steps 1-9 run today. Only step 10 is Bluetooth, so
|
||||
relabelling the whole task would bury real pending work.
|
||||
- **251** MCP, **253** hearing, **259** crawler. All three ship
|
||||
(`internal/mcp/`, `internal/capture/`, `internal/crawl/`) with no external gate.
|
||||
Fully checkable. `259`'s step 1 wants no `crawl` block in `deploy/mavend.json`,
|
||||
and there is none, so it is already set up correctly.
|
||||
- **252** vision and **255** speaker recognition. Both ship. Each is blocked only
|
||||
on a model download: a vision gguf with mmproj, and a speaker embedding model.
|
||||
Neither is present under `/mnt/hdd1`. Their refusal-path steps run today.
|
||||
|
||||
So the honest split is three blocked on a credential or hardware, two blocked on
|
||||
a download, and six ready to check. That is roughly a session of real QA this
|
||||
plan had written off as backlog.
|
||||
|
||||
**All six ran on 02-08-2026.** Every one of them is code-correct and stops at the
|
||||
deploy. The pattern repeats often enough to be the headline: the packages pass,
|
||||
and the box cannot reach them.
|
||||
|
||||
**251, MCP.** Steps 1, 2, 3, 4 and 13 pass. Package tests green under `-race`.
|
||||
Off-by-default is clean, and the SSRF refusal is exact: without `allow_private`
|
||||
the log reads `refusing to connect to a private address: 127.0.0.1` and `/tools`
|
||||
shows the server down with zero proposals. Steps 5 to 12 are blocked. `ss -lntp`
|
||||
shows the Vikunja MCP server on `127.0.0.1:9100` only, so no container reaches it
|
||||
at any address (**478**). `allow_private` does work, measured both ways.
|
||||
|
||||
**253, hearing.** Steps 1, 2 and 17 pass. `internal/capture` covers 90.3%. Steps
|
||||
7 to 16 are blocked on something nobody can work around: no shipped client calls
|
||||
`CaptureStart`. There is no `cmd/mavheard`, no mavweb route, and `mavenclient`
|
||||
never calls it (**480**). Two of its QA steps are also stale.
|
||||
|
||||
**257, netscan.** Steps 2, 3 and 9 pass at unit level. Step 1 fails. Steps 4 to 8
|
||||
need the block enabled. Step 10 is Bluetooth and stays skipped.
|
||||
|
||||
**259, crawler.** Steps 1 and 15 pass. Step 2 fails. Steps 3 to 14 need a `crawl`
|
||||
block that nobody has written.
|
||||
|
||||
Both were configured later the same day, and both work. `netscan.enabled: true`
|
||||
answers `какие устройства в сети?` with `нашла 3 устройства, из них 2 с вебом, 2 с
|
||||
ssh. список записала.` and the scan lands in the intake journal as `scan:lan`.
|
||||
`crawl.on_demand: true` answers `посмотри https://lwn.net — что там пишут?` from
|
||||
the real page. So **479** is one defect, not the routing defect it was filed as.
|
||||
An unconfigured capability declines its own turn instead of naming the gap.
|
||||
Nothing is wrong with the routing.
|
||||
|
||||
257 step 1 and 259 step 2 fail the same way and share a task (**479**). An
|
||||
unconfigured capability does not name the gap, so the question escapes to web
|
||||
search. `какие устройства в сети?` was answered with a general article about
|
||||
network hardware. That is his LAN going to an upstream engine.
|
||||
|
||||
**252 vision and 255 speaker.** Both confirmed blocked. The disk claim was
|
||||
re-verified rather than taken on trust: 16 text-only ggufs under `/mnt/hdd1`, no
|
||||
mmproj and no speaker embedding model. Everything not needing the model passes,
|
||||
including the two refusals that matter. `TestNewLocalRefusesNonPrivateEndpoints`
|
||||
rejects `https://api.openai.com`, and forget really deletes
|
||||
(`internal/store/memory.go:145` is a real `DELETE`, not a tombstone). Vision is
|
||||
19/19, speaker 22/22, media 16/16.
|
||||
|
||||
**470 got worse.** Both poisoned facts show `voided` on `/history`, and the
|
||||
defect survives. Re-measured at 15:42, after four restarts: `почему небо синее?`
|
||||
still answers `какая последняя версия языка Go?` with no `search:` line. What
|
||||
comes back is the question he typed, not the value the fact held. So the poison
|
||||
is a vector in the memory index, and `revert` does not remove it. There is
|
||||
currently no documented way to repair a poisoned box.
|
||||
|
||||
---
|
||||
|
||||
@@ -171,16 +519,39 @@ Not QA. These are blocked on a decision or a credential only you have.
|
||||
| 355 | Deploy the Hexis auth change. Was blocked on Maven being under construction, which it no longer is. The client half is vendored and wired. |
|
||||
| 357 | Decide whether entity-existence validation is the permanent target guard or whether blessing lands in Nexus. |
|
||||
| 275 | Hexis native API and MCP parity. |
|
||||
| — | Decide on `-require-stepup`. Making it the default needs WebAuthn configured first, or it locks you out of your own admin surfaces. See **317**. |
|
||||
| — | Three nginx sites bind wildcard `:80` (`acme.conf`, `matrix`, `panel`), so the ecosystem's bind-level protection is not in effect and `allow`/`deny` is carrying it alone. See **354**. |
|
||||
| — | Decide on `-require-stepup`. Making it the default needs WebAuthn configured first, or it locks you out of your own admin surfaces. |
|
||||
|
||||
317 and 354 closed on 01-08-2026. The step-up gate now covers `POST /api/chat` and
|
||||
`/routines`, and the nginx template is locked down with a `maven.<domain>` block for
|
||||
mavweb. The `-require-stepup` default is still your call.
|
||||
|
||||
---
|
||||
|
||||
## Not this repo
|
||||
|
||||
Two open tasks sit on the Maven board and are not Maven work. Move them or note
|
||||
where they land, so the board stops reading as 50 things Maven owes.
|
||||
|
||||
- **358** replace the rowid execution cursor with a real seq column. This is Hexis,
|
||||
and it must land before any execution retention or pruning does.
|
||||
- **362** mirror the router prompt reorder into the relabelling prompt. This is the
|
||||
training workspace, enforced by `llm/check_prompt_parity.py` there, not here.
|
||||
|
||||
---
|
||||
|
||||
## Suggested order
|
||||
|
||||
1. Session 1. If the voice loop is broken, nothing else matters.
|
||||
2. The `-require-stepup` and Kuma decisions. Five minutes, unblocks **317** fully
|
||||
and **16**.
|
||||
3. Session 2. The numbers tell you whether the router is worth its 90x latency.
|
||||
2. The `-require-stepup` and Kuma decisions. Five minutes, and it unblocks **16**.
|
||||
3. Session 2. **Run on 02-08-2026.** The numbers came back worse for the router
|
||||
than the docs claimed. The classifier is 68.8%, not 36.8%, and 16.6µs, not
|
||||
31ms. The router buys about 4 points of accuracy for four orders of magnitude
|
||||
of latency. Whether that still earns its place is now an open question.
|
||||
4. Housekeeping. Cheap, and it makes the remaining backlog honest.
|
||||
5. Session 3, split whichever way suits you.
|
||||
5. Session 3, split whichever way suits you. All five sittings ran on
|
||||
02-08-2026. Read the per-sitting notes before repeating any of them.
|
||||
|
||||
The next thing to fix is not in this plan. Four defects say the same sentence:
|
||||
a capability is built and no utterance reaches it. **466** (a clarify is global),
|
||||
**467** (capture is act-routed), **475** (attention is act-routed), **476** (the
|
||||
router rewrites entity names). Routing is where the work is.
|
||||
|
||||
Reference in New Issue
Block a user