QA: Voice session quality polish #171

Merged
kami merged 6 commits from task/287-qa-voice-session-quality-polish into master 2026-08-04 21:26:02 +02:00
Showing only changes of commit 758fb6a3f0 - Show all commits
+19 -6
View File
@@ -161,12 +161,25 @@ turns look misaligned when they are not.
**The slowness now has a cause and a number** (04-08-2026). A spoken turn
takes 32 to 34 seconds. One phrasing call is 30.0s of that. STT is 1.0s
and routing is under 10ms. Both interactive calls decoded exactly 512 tokens,
which is the `chat` cap. So both ran to the cap and were truncated, to
produce a 16-character reply. The resident model is a Thinking variant and nothing
tells it to skip reasoning. `deploy/mavgpud.json` already passes
`--chat-template-kwargs {"enable_thinking":false}` for the workstation and
`llamaArgs` does not. Filed as **531**, which also asks whether that gap
inflates the workstation comparison.
which is the phrasing cap. Both were truncated, to produce a reply of
under 25 characters.
The cause is `responseGrammar`, not the model. Its last rule is
`ws ::= [ \t\n]*`, and `*` is unbounded, so the model emits `{` and then
satisfies `ws` with whitespace until `max_tokens` stops it. Reproduced on a
second server: at `repeat_penalty` 1.0 it runs to 512 and returns
`finish_reason=length`, at 1.3 it stops at 24. Bounding the rule to
`[ \t\n]{0,4}` gives a clean stop at 33 tokens three times out of three with
no penalty at all.
Only some callers are exposed. `internal/llm.Req` sends `repeat_penalty` and
the replier sets it to 1.3, so that path is protected by accident. `chatReq`
in the phraser sends no penalty, so `PhraseChat`, `PhraseQuery`,
`PhraseNudge` and `PhraseReminder` all run at the default 1.0. Filed as
**531**.
Two guesses were wrong on the way and are recorded so nobody repeats them.
It is not reasoning tokens: the probe returned `reasoning_content` of length
0, and the grammar constrains output from the first token. It is not the
`--cache-ram 512` limit either: that is MiB of prompt cache and the 512 that
was hit is a token count.
The wandering is a second thing and stays on 287.
**The wake path cannot be checked here, and that is now the decision rather