diff --git a/docs/qa.md b/docs/qa.md index d187b94..adcda56 100644 --- a/docs/qa.md +++ b/docs/qa.md @@ -161,12 +161,25 @@ turns look misaligned when they are not. **The slowness now has a cause and a number** (04-08-2026). A spoken turn takes 32 to 34 seconds. One phrasing call is 30.0s of that. STT is 1.0s and routing is under 10ms. Both interactive calls decoded exactly 512 tokens, - which is the `chat` cap. So both ran to the cap and were truncated, to - produce a 16-character reply. The resident model is a Thinking variant and nothing - tells it to skip reasoning. `deploy/mavgpud.json` already passes - `--chat-template-kwargs {"enable_thinking":false}` for the workstation and - `llamaArgs` does not. Filed as **531**, which also asks whether that gap - inflates the workstation comparison. + which is the phrasing cap. Both were truncated, to produce a reply of + under 25 characters. + The cause is `responseGrammar`, not the model. Its last rule is + `ws ::= [ \t\n]*`, and `*` is unbounded, so the model emits `{` and then + satisfies `ws` with whitespace until `max_tokens` stops it. Reproduced on a + second server: at `repeat_penalty` 1.0 it runs to 512 and returns + `finish_reason=length`, at 1.3 it stops at 24. Bounding the rule to + `[ \t\n]{0,4}` gives a clean stop at 33 tokens three times out of three with + no penalty at all. + Only some callers are exposed. `internal/llm.Req` sends `repeat_penalty` and + the replier sets it to 1.3, so that path is protected by accident. `chatReq` + in the phraser sends no penalty, so `PhraseChat`, `PhraseQuery`, + `PhraseNudge` and `PhraseReminder` all run at the default 1.0. Filed as + **531**. + Two guesses were wrong on the way and are recorded so nobody repeats them. + It is not reasoning tokens: the probe returned `reasoning_content` of length + 0, and the grammar constrains output from the first token. It is not the + `--cache-ram 512` limit either: that is MiB of prompt cache and the 512 that + was hit is a token count. The wandering is a second thing and stays on 287. **The wake path cannot be checked here, and that is now the decision rather