QA: Voice session quality polish #171
+19
-6
@@ -161,12 +161,25 @@ turns look misaligned when they are not.
|
||||
**The slowness now has a cause and a number** (04-08-2026). A spoken turn
|
||||
takes 32 to 34 seconds. One phrasing call is 30.0s of that. STT is 1.0s
|
||||
and routing is under 10ms. Both interactive calls decoded exactly 512 tokens,
|
||||
which is the `chat` cap. So both ran to the cap and were truncated, to
|
||||
produce a 16-character reply. The resident model is a Thinking variant and nothing
|
||||
tells it to skip reasoning. `deploy/mavgpud.json` already passes
|
||||
`--chat-template-kwargs {"enable_thinking":false}` for the workstation and
|
||||
`llamaArgs` does not. Filed as **531**, which also asks whether that gap
|
||||
inflates the workstation comparison.
|
||||
which is the phrasing cap. Both were truncated, to produce a reply of
|
||||
under 25 characters.
|
||||
The cause is `responseGrammar`, not the model. Its last rule is
|
||||
`ws ::= [ \t\n]*`, and `*` is unbounded, so the model emits `{` and then
|
||||
satisfies `ws` with whitespace until `max_tokens` stops it. Reproduced on a
|
||||
second server: at `repeat_penalty` 1.0 it runs to 512 and returns
|
||||
`finish_reason=length`, at 1.3 it stops at 24. Bounding the rule to
|
||||
`[ \t\n]{0,4}` gives a clean stop at 33 tokens three times out of three with
|
||||
no penalty at all.
|
||||
Only some callers are exposed. `internal/llm.Req` sends `repeat_penalty` and
|
||||
the replier sets it to 1.3, so that path is protected by accident. `chatReq`
|
||||
in the phraser sends no penalty, so `PhraseChat`, `PhraseQuery`,
|
||||
`PhraseNudge` and `PhraseReminder` all run at the default 1.0. Filed as
|
||||
**531**.
|
||||
Two guesses were wrong on the way and are recorded so nobody repeats them.
|
||||
It is not reasoning tokens: the probe returned `reasoning_content` of length
|
||||
0, and the grammar constrains output from the first token. It is not the
|
||||
`--cache-ram 512` limit either: that is MiB of prompt cache and the 512 that
|
||||
was hit is a token count.
|
||||
The wandering is a second thing and stays on 287.
|
||||
|
||||
**The wake path cannot be checked here, and that is now the decision rather
|
||||
|
||||
Reference in New Issue
Block a user