Merge pull request 'QA: Voice session quality polish' (#171) from task/287-qa-voice-session-quality-polish into master

This commit was merged in pull request #171.
This commit is contained in:
2026-08-04 21:26:01 +02:00
4 changed files with 270 additions and 18 deletions
+116 -3
View File
@@ -1,6 +1,6 @@
# QA plan: checking Maven properly # QA plan: checking Maven properly
*Last verified: 2026-08-04 @ a4d5155. Living doc: correct it in place, do not append.* *Last verified: 2026-08-04 @ 8d816f4. Living doc: correct it in place, do not append.*
Written 2026-08-01, after the 35-PR stack landed and the box came back up. Written 2026-08-01, after the 35-PR stack landed and the box came back up.
Refreshed 2026-08-02 against the live list, after PRs #85-#90. Refreshed 2026-08-02 against the live list, after PRs #85-#90.
@@ -88,8 +88,71 @@ session quality), **321** steps 3-5 (quiet mode), **288** (STT golden audio).
**288 is not blocked.** The fixtures are committed under `cmd/mavsttd/testdata/` **288 is not blocked.** The fixtures are committed under `cmd/mavsttd/testdata/`
and `make test-stt-golden` runs today. This plan said otherwise until 02-08-2026. and `make test-stt-golden` runs today. This plan said otherwise until 02-08-2026.
Steps 1 and 3-6 were run on 02-08-2026 and pass. Steps 2 and 7-9 still need a Steps 1 and 3-6 were run on 02-08-2026 and pass.
person at the box, because they need a microphone or a nudge to arrive.
**Step 2 no longer needs a person, and step 9 has a number now** (04-08-2026).
`POST /api/ptt` takes raw PCM16 16kHz mono and answers with audio plus an
`X-Reply-Text` header, so the committed STT fixtures stand in for a microphone:
```sh
tail -c +45 cmd/mavsttd/testdata/ru_query.wav > /tmp/q.pcm
curl -s --noproxy '*' -D /tmp/h -o /tmp/reply.pcm -X POST \
http://127.0.0.1:9201/api/ptt --data-binary @/tmp/q.pcm \
-H 'Content-Type: application/octet-stream' -m 180
```
That covers audio in → STT → router → phrasing → TTS audio out. It leaves only
browser microphone capture needing a person, and the wake path needing a machine.
Do not post `en_act.wav` without deciding first: it is a mutating act.
**Steps 7 and 8 still cannot run, but 15 is no longer the reason** (04-08-2026).
The desk presence poster is installed on workpc. It is a `maven-desk` systemd
user timer on a 60s cadence, gated by hypridle at 120s idle. `desk_active` facts
now arrive, and the first landed at 18:43.
What blocks the two steps now is that no rule wants to fire. `/trace` shows all
five at `predicate`, none inert:
| rule | sev | why it is false |
|---|---|---|
| water | 1 | needs ≥3h since the last `water` fact; step 2's `ru_fact` wrote one |
| meal | 1 | needs ≥6h since a `meal` fact; none exists |
| break | 2 | needs both `desk_active` and a `break` fact; `break` has never been written |
| service_down | 4 | no kuma monitor is down |
| netdata_critical | 3 | nothing critical |
So the honest way to run step 8 is to wait three hours after the last `water`
fact, or to write one antedated. Do not read the water rule's silence as a defect.
**The sev4 telegram reach works** (04-08-2026). Resuming a paused kuma monitor
for paperless, which is genuinely down, put a real `service_down` through the
whole path with presence away:
```
23:03 voicesink: no live voice session for service_down, falling through to away channels
/notifications: 19:03 | service_down | telegram | pending | Сервис перестал отвечать.
04.08 23:03 | nudge | service_down | telegram | sent | 23:03
```
`ChannelsFor(Sev4, Away)` returned telegram, the send succeeded, and the row
holds at `pending` because sev4 repeats until acked. The 15:51 row shows the
same rule reaching `acted` earlier, so the ack path works too.
The body was `Сервис перестал отвечать.`, which names no service. That is a bug
and it is deterministic, filed as **534**. `nudgeValues` fills `{service}` from
`State.Fact("service_down")`, an exact key mavpoll stopped writing when
per-monitor facts landed. Nine of the ten templates carry `{service}`, so all
nine are rejected as unfillable. The one nameless variant is left as the only
usable one, every time. The stub and LLM phrasers both call `loop.DownServices`
and get it right. The template path is the one that runs.
**Presence itself has a real defect, filed as 532.** `SavePresenceState` has no
caller outside tests, so the singleton row is never written. The gate is fine,
because it reads the bucket `GatherState` computes in memory each tick. Two
things follow. Hysteresis is dead, because `lastBucket` is always cold-start `Away`
and the 0.30-0.55 hold band never applies. And every presence readout lies:
`/dash` shows `away — score 0.00 (never)` with fresh `desk_active` facts arriving
every 60s. Do not trust that number while checking anything else here.
Steps 1 and 3-6 do not need a browser. `POST /api/chat` takes a form-encoded Steps 1 and 3-6 do not need a browser. `POST /api/chat` takes a form-encoded
`text=` field and a cookie jar, and answers with the rendered `/chat` page: `text=` field and a cookie jar, and answers with the rendered `/chat` page:
@@ -112,6 +175,33 @@ turns look misaligned when they are not.
back. This covers browser mic to STT to core to TTS as one path. It does back. This covers browser mic to STT to core to TTS as one path. It does
**not** cover the wake word or the voice-activity gate, and no step here **not** cover the wake word or the voice-activity gate, and no step here
does — see below. does — see below.
**Passes below the browser** (04-08-2026, three fixtures through `/api/ptt`):
HTTP 200, `audio/l16;rate=16000;channels=1`, and real speech back. `ru_query`
answered `на 04.08.2026 ничего нет.` in 3.82s of audio at RMS 3865, `ru_fact`
answered `отметила: water = выпил`, `ru_reminder` answered `хорошо, напомню.`
at `intent=reminder`.
**Passes in the browser too** (04-08-2026), and it needed no person. Headless
Chrome takes a fake microphone, so the whole browser half runs unattended:
```sh
chrome --headless=new --remote-debugging-port=9333 --remote-allow-origins='*' \
--use-fake-device-for-media-stream --use-fake-ui-for-media-stream \
--use-file-for-fake-audio-capture=cmd/mavsttd/testdata/ru_query.wav%noloop
```
Then drive it over the debug protocol: click `#btn`, wait, click again, read
`#status` and `#log`. That covers `getUserMedia`, `MediaRecorder`, the webm
decode and the hand-written resample to 16k Int16. It logged
`sending 188160 bytes`, which is 5.88s at 16k mono, and got the reply back.
**The button is on `/`, not `/dash`.** `handleVoice` serves it at the root
(`main.go:332`). `/dash` is the presence and fact dashboard and carries no
`#btn`. This step said `/dash` until 04-08-2026.
One defect fell out, filed as **533**. The reply logged as
`на+04.08.2026+ничего+нет.` The header is escaped with `url.QueryEscape`,
which writes a space as `+`, then decoded with `decodeURIComponent`, which
leaves `+` alone. Transcript only, the audio is fine.
3. Say `тихий режим`. Expect `тихий режим включён. буду реже напоминать.` **Passes.** 3. Say `тихий режим`. Expect `тихий режим включён. буду реже напоминать.` **Passes.**
4. Say `выключи тихий режим`. Expect `тихий режим выключен.` Negation must win. **Passes.** 4. Say `выключи тихий режим`. Expect `тихий режим выключен.` Negation must win. **Passes.**
5. Say `в комнате тихо`. Quiet mode must NOT flip. Confirm on `/history` that no 5. Say `в комнате тихо`. Quiet mode must NOT flip. Confirm on `/history` that no
@@ -132,6 +222,29 @@ turns look misaligned when they are not.
**First evidence, in text** (02-08-2026): nothing breaks, but answers wander **First evidence, in text** (02-08-2026): nothing breaks, but answers wander
and stitch unrelated topics. Asked whether he should move flats, she opened and stitch unrelated topics. Asked whether he should move flats, she opened
with the weather. That is 287, and it is a phrasing problem, not a loop problem. with the weather. That is 287, and it is a phrasing problem, not a loop problem.
**The slowness now has a cause and a number** (04-08-2026). A spoken turn
takes 32 to 34 seconds. One phrasing call is 30.0s of that. STT is 1.0s
and routing is under 10ms. Both interactive calls decoded exactly 512 tokens,
which is the phrasing cap. Both were truncated, to produce a reply of
under 25 characters.
The cause is `responseGrammar`, not the model. Its last rule is
`ws ::= [ \t\n]*`, and `*` is unbounded, so the model emits `{` and then
satisfies `ws` with whitespace until `max_tokens` stops it. Reproduced on a
second server: at `repeat_penalty` 1.0 it runs to 512 and returns
`finish_reason=length`, at 1.3 it stops at 24. Bounding the rule to
`[ \t\n]{0,4}` gives a clean stop at 33 tokens three times out of three with
no penalty at all.
Only some callers are exposed. `internal/llm.Req` sends `repeat_penalty` and
the replier sets it to 1.3, so that path is protected by accident. `chatReq`
in the phraser sends no penalty, so `PhraseChat`, `PhraseQuery`,
`PhraseNudge` and `PhraseReminder` all run at the default 1.0. Filed as
**531**.
Two guesses were wrong on the way and are recorded so nobody repeats them.
It is not reasoning tokens: the probe returned `reasoning_content` of length
0, and the grammar constrains output from the first token. It is not the
`--cache-ram 512` limit either: that is MiB of prompt cache and the 512 that
was hit is a token count.
The wandering is a second thing and stays on 287.
**The wake path cannot be checked here, and that is now the decision rather **The wake path cannot be checked here, and that is now the decision rather
than a gap.** `mavwaked` and `mavenclient` appear in no compose file and run as than a gap.** `mavwaked` and `mavenclient` appear in no compose file and run as
+33 -6
View File
@@ -94,7 +94,7 @@ func (t *NudgeTemplates) PhraseNudge(_ context.Context, c loop.Candidate) (deliv
// template fits it uses the plain per-rule fallback. // template fits it uses the plain per-rule fallback.
func (t *NudgeTemplates) Nudge(c loop.Candidate) (body, mood string) { func (t *NudgeTemplates) Nudge(c loop.Candidate) (body, mood string) {
rule := c.Rule.Name rule := c.Rule.Name
family := t.family(rule) family := t.pluralFamily(t.family(rule), c)
set, ok := t.file.Rules[family] set, ok := t.file.Rules[family]
if !ok { if !ok {
return fallbackNudge(c), "neutral" return fallbackNudge(c), "neutral"
@@ -155,6 +155,25 @@ func (t *NudgeTemplates) family(rule string) string {
return "default" return "default"
} }
// pluralFamily swaps in the plural wording when {service} will hold a list.
// Russian agrees the verb with the subject, so one set of templates cannot
// serve both: "Сервис paperless не отвечает" and "Сервисы nginx, paperless не
// отвечают" differ in the noun, the verb and the adjective. Filling a list into
// the singular text is the kind of near-miss that reads as machine-written.
//
// Only service_down has a plural form today. A family with no "_many" set in
// the file is returned unchanged, so adding one is a data change.
func (t *NudgeTemplates) pluralFamily(family string, c loop.Candidate) string {
if len(loop.DownServices(c.State)) < 2 {
return family
}
many := family + "_many"
if _, ok := t.file.Rules[many]; ok {
return many
}
return family
}
// placeholderRE — the {name} slots a template may use. // placeholderRE — the {name} slots a template may use.
var placeholderRE = regexp.MustCompile(`\{([a-z]+)\}`) var placeholderRE = regexp.MustCompile(`\{([a-z]+)\}`)
@@ -165,17 +184,25 @@ func nudgeValues(c loop.Candidate) map[string]string {
vals := map[string]string{} vals := map[string]string{}
rule := c.Rule.Name rule := c.Rule.Name
// {service} — one fact per kuma monitor, keyed "service_down:<name>", so
// the name lives in the key SUFFIX and there is no fact called plain
// "service_down" to read. loop.DownServices is the same helper the rule
// fired on, which is what stops the message naming a service that is up.
// This used to read c.State.Fact(rule) — the pre-per-monitor aggregate —
// and so never filled, leaving the one nameless variant as the only
// fillable template every time (Vikunja #534).
if down := loop.DownServices(c.State); len(down) > 0 {
vals["service"] = strings.Join(down, ", ")
}
// {since} — only at hour scale. Below an hour the phrase would be minutes, // {since} — only at hour scale. Below an hour the phrase would be minutes,
// and none of the templates read well with "сорок минут". // and none of the templates read well with "сорок минут". service_down has
// no {since} to offer: its facts are keyed by monitor, and the rule is
// edge-triggered, so it fires on the transition rather than hours later.
if d, ok := c.State.Since(rule); ok && d >= time.Hour { if d, ok := c.State.Since(rule); ok && d >= time.Hour {
if s := ruSinceWords(d); s != "" { if s := ruSinceWords(d); s != "" {
vals["since"] = s vals["since"] = s
} }
} }
// {service} — the aggregate fact's key carries the service name.
if f, ok := c.State.Fact(rule); ok && f.Key != "" && f.Key != rule {
vals["service"] = f.Key
}
// {what} — the Russian suffix of "routine:таблетки" / "morning:утро". // {what} — the Russian suffix of "routine:таблетки" / "morning:утро".
if i := strings.IndexByte(rule, ':'); i > 0 && i+1 < len(rule) { if i := strings.IndexByte(rule, ':'); i > 0 && i+1 < len(rule) {
vals["what"] = rule[i+1:] vals["what"] = rule[i+1:]
+106 -6
View File
@@ -11,6 +11,102 @@ import (
"github.com/kami/maven/internal/store" "github.com/kami/maven/internal/store"
) )
// downCand builds a service_down candidate the way a tick actually does it:
// one fact per kuma monitor under the prefix, carrying the source and value
// loop.DownServices checks. The old cand() shape wrote a single fact keyed
// plain "service_down", which mavpoll stopped producing, and that is why the
// tests passed through the whole of #534.
func downCand(names ...string) loop.Candidate {
now := time.Date(2026, 7, 31, 21, 40, 0, 0, time.UTC)
st := loop.State{Now: now, Facts: map[string]store.Fact{}}
for _, n := range names {
key := loop.ServiceDownPrefix + n
st.Facts[key] = store.Fact{
Key: key, Ts: now.Add(-3 * time.Minute),
Source: loop.ServiceDownSource, Value: `"down"`,
}
}
return loop.Candidate{
Rule: loop.Rule{Name: "service_down", Severity: loop.Sev4},
Severity: loop.Sev4, State: st,
}
}
// The nudge he reads on telegram must name what broke. It is a sev4 that
// reaches him away from the box, so "a service is down" costs him a trip to
// kuma to learn anything at all.
func TestNudgeNamesTheDownService(t *testing.T) {
// Lowercased before matching: a name that opens the sentence is
// capitalized by capitalizeFirst, which is wanted.
nt := newTestTemplates(t, 5)
for i := 0; i < 40; i++ {
body, _ := nt.Nudge(downCand("paperless"))
if !strings.Contains(strings.ToLower(body), "paperless") {
t.Fatalf("body does not name the service: %q", body)
}
}
// Two down: both named, in the key order the rule itself uses.
for i := 0; i < 40; i++ {
body, _ := nt.Nudge(downCand("nginx", "paperless"))
low := strings.ToLower(body)
if !strings.Contains(low, "nginx") || !strings.Contains(low, "paperless") {
t.Fatalf("body drops a service: %q", body)
}
}
}
// Russian agrees the verb with the subject, so a list of services cannot go
// into the singular sentence. One down takes the singular set, two or more
// take service_down_many.
func TestNudgeAgreesWithTheServiceCount(t *testing.T) {
nt := newTestTemplates(t, 9)
// "упал " keeps its trailing space: "упали" starts with "упал", and the
// plural must not read as the singular by prefix.
singular := []string{"не отвечает", "недоступен", "лежит", "упал "}
plural := []string{"не отвечают", "недоступны", "лежат", "упали"}
for i := 0; i < 60; i++ {
body, _ := nt.Nudge(downCand("paperless"))
if !containsAny(body, singular) {
t.Fatalf("one down, no singular verb: %q", body)
}
if containsAny(body, plural) {
t.Fatalf("one down, plural wording: %q", body)
}
}
for i := 0; i < 60; i++ {
body, _ := nt.Nudge(downCand("nginx", "paperless"))
if !containsAny(body, plural) {
t.Fatalf("two down, no plural verb: %q", body)
}
if containsAny(body, singular) {
t.Fatalf("two down, singular wording: %q", body)
}
}
}
func containsAny(s string, subs []string) bool {
for _, sub := range subs {
if strings.Contains(s, sub) {
return true
}
}
return false
}
// Nothing down means no template fits, and the fallback answers rather than
// the picker inventing a name.
func TestNudgeServiceDownWithoutFacts(t *testing.T) {
nt := newTestTemplates(t, 5)
body, mood := nt.Nudge(downCand())
if body != "Сервис не отвечает." {
t.Fatalf("fallback body %q", body)
}
if mood != "neutral" {
t.Fatalf("mood %q", mood)
}
}
// cand builds a candidate the way a tick would. // cand builds a candidate the way a tick would.
func cand(rule string, sinceMin int, factKey string) loop.Candidate { func cand(rule string, sinceMin int, factKey string) loop.Candidate {
now := time.Date(2026, 7, 31, 21, 40, 0, 0, time.UTC) now := time.Date(2026, 7, 31, 21, 40, 0, 0, time.UTC)
@@ -36,7 +132,7 @@ func newTestTemplates(t *testing.T, seed int64) *NudgeTemplates {
func TestNudgeTemplatesLoad(t *testing.T) { func TestNudgeTemplatesLoad(t *testing.T) {
nt := newTestTemplates(t, 1) nt := newTestTemplates(t, 1)
for _, rule := range []string{"water", "meal", "break", "service_down", "netdata_critical", "routine", "morning", "default"} { for _, rule := range []string{"water", "meal", "break", "service_down", "service_down_many", "netdata_critical", "routine", "morning", "default"} {
set, ok := nt.file.Rules[rule] set, ok := nt.file.Rules[rule]
if !ok { if !ok {
t.Errorf("no templates for %q", rule) t.Errorf("no templates for %q", rule)
@@ -46,8 +142,12 @@ func TestNudgeTemplatesLoad(t *testing.T) {
t.Errorf("%s: only %d variants", rule, len(set.Variants)) t.Errorf("%s: only %d variants", rule, len(set.Variants))
} }
// Every rule needs one variant that needs no value, or a candidate // Every rule needs one variant that needs no value, or a candidate
// without context has nothing to say. routine and morning are exempt: // without context has nothing to say. routine, morning and
// they always carry a name and must always say it. // service_down are exempt: they always carry a name and must always
// say it. service_down's predicate cannot fire without a down fact,
// so loop.DownServices always has something to fill {service} with,
// and the nameless variant it used to carry was the bug (#534) —
// {service} never filled, so that variant was the only fillable one.
plain := 0 plain := 0
seen := map[string]bool{} seen := map[string]bool{}
for _, v := range set.Variants { for _, v := range set.Variants {
@@ -59,7 +159,7 @@ func TestNudgeTemplatesLoad(t *testing.T) {
} }
seen[v] = true seen[v] = true
} }
if plain == 0 && rule != "routine" && rule != "morning" { if plain == 0 && rule != "routine" && rule != "morning" && !strings.HasPrefix(rule, "service_down") {
t.Errorf("%s: every variant needs a placeholder value", rule) t.Errorf("%s: every variant needs a placeholder value", rule)
} }
} }
@@ -102,8 +202,8 @@ func TestNudgeNoLeftoverPlaceholders(t *testing.T) {
cand("water", 0, ""), // no duration cand("water", 0, ""), // no duration
cand("water", 30, ""), // under an hour cand("water", 30, ""), // under an hour
cand("water", 200, ""), // hours cand("water", 200, ""), // hours
cand("service_down", 3, "vaultwarden"), downCand("vaultwarden"),
cand("service_down", 3, ""), // no service name downCand(), // nothing down: the fallback answers
cand("routine:таблетки", 0, ""), cand("routine:таблетки", 0, ""),
cand("morning:утро", 0, ""), cand("morning:утро", 0, ""),
cand("unknown_rule", 0, ""), cand("unknown_rule", 0, ""),
+15 -3
View File
@@ -5,7 +5,8 @@
"Hand-written Russian nudges. Edit the wording here, no Go changes needed.", "Hand-written Russian nudges. Edit the wording here, no Go changes needed.",
"Rules: she is feminine about herself, he is a man addressed as ты. Never вы/вас/ваш, never plural imperatives (выпейте), never он/его about him.", "Rules: she is feminine about herself, he is a man addressed as ты. Never вы/вас/ваш, never plural imperatives (выпейте), never он/его about him.",
"One short sentence. No questions, no emoji, no pet names, no emotional support.", "One short sentence. No questions, no emoji, no pet names, no emotional support.",
"Placeholders: {since} how long it has been (only used when it is at least an hour), {service} the service name, {what} the routine name. A variant whose placeholder has no value is skipped, so every rule needs at least one variant with no placeholder. The exception is routine and morning: those only exist for rules like routine:таблетки that always carry a name, and a routine nudge that drops the name is useless.", "Placeholders: {since} how long it has been (only used when it is at least an hour), {service} the service name, {what} the routine name. A variant whose placeholder has no value is skipped, so every rule needs at least one variant with no placeholder. The exception is routine, morning and service_down: those only exist for rules that always carry a name, and one that drops the name is useless.",
"A rule may carry a second set named <rule>_many, used when {service} holds more than one name. Russian agrees the verb with the subject, so the plural needs its own wording rather than a list dropped into the singular sentence. Only service_down has one.",
"mood must be one of: neutral, happy, thinking, tired, confused." "mood must be one of: neutral, happy, thinking, tired, confused."
], ],
"rules": { "rules": {
@@ -62,13 +63,24 @@
"{service} не отвечает, сервис нужно поднимать.", "{service} не отвечает, сервис нужно поднимать.",
"Сервис {service} недоступен.", "Сервис {service} недоступен.",
"Проверь {service}: сервис не отвечает.", "Проверь {service}: сервис не отвечает.",
"Сервис перестал отвечать.",
"Сервис {service} лежит, нужно смотреть.", "Сервис {service} лежит, нужно смотреть.",
"{service} не отвечает уже {since}.",
"Мониторинг сообщает: {service} лежит.", "Мониторинг сообщает: {service} лежит.",
"Сервис {service} не отвечает, посмотри логи." "Сервис {service} не отвечает, посмотри логи."
] ]
}, },
"service_down_many": {
"mood": "neutral",
"variants": [
"Сервисы {service} не отвечают.",
"{service} упали — сервисы не отвечают.",
"{service} не отвечают, сервисы нужно поднимать.",
"Сервисы {service} недоступны.",
"Проверь {service}: сервисы не отвечают.",
"Сервисы {service} лежат, нужно смотреть.",
"Мониторинг сообщает: {service} лежат.",
"Сервисы {service} не отвечают, посмотри логи."
]
},
"netdata_critical": { "netdata_critical": {
"mood": "neutral", "mood": "neutral",
"variants": [ "variants": [