llm.Client carries a bearer credential and sets it on the completion, and
Pair signs the /health probe with it too. An unsigned probe would answer 401,
Pair would read that as a card that is busy, and every workstation turn would
fall back to the resident model with nothing naming why.
The token comes from workstation.token, expanded from MAVEN_GPU_TOKEN like
every other secret in that file. Missing, and voicewire says so at startup:
the fallback is silent by design and this failure would otherwise be
invisible.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ESv8hqNPseYt1CnotZpqDz
A spoken turn took 25-34 seconds and effectively all of it was one phrasing
call generating whitespace. Both interactive turns measured on 2026-08-04
decoded exactly 512 tokens, which is the phrasing MaxTokens, and both ran to
the cap. Background phrasing on the same server in the same window stopped at
32-36 tokens in 4.3s, so it was never the server and never contention.
`ws ::= [ \t\n]*` is a licence to emit whitespace until max_tokens. The model
opens the object, satisfies ws forever, and only the cap stops it. Bounding
the rule fixes it outright with no repeat penalty at all: three runs, three
clean stops at 33 tokens. routeGrammar carried the same rule and is bounded
too — it never ran away only because that path sends routeRepeatPenalty, which
is an accident rather than a defence.
chatReq had no repeat-penalty field at all, so every caller through
chatWithSystem ran at the server default of 1.0 while Replier.PhraseReply sent
1.3 through internal/llm and was protected by accident. Adding it is defence
in depth, not the fix. Two wire structs disagreeing about the sampler is not a
decision anybody made.
finish_reason is parsed on both transports now and a cap hit logs. Both replies
that ran away happened to parse — the grammar had already closed the JSON — so
a truncated generation was indistinguishable from a whole one at every layer
above the response struct.
The phraser test rejects unbounded repetition anywhere in responseGrammar
rather than checking ws by name. A grammar is a budget: every repetition in it
is something the model may do until the token cap, and the cap is not a design.
routeGrammar keeps one, `("," ws action)*`, because a compound utterance is any
number of actions and capping it would drop the last ask.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011x5DgnExQ5XZy8TZPs5bot
The phraser's own transport has always sampled at 0.7 and this client has
always been greedy. Routing a phrasing call through the client must not
change how it decodes, so Req carries the temperature and 0 — the zero
value, and what every existing caller wanted — is still greedy.
The drain counted only the phrasing paths in internal/phraser. The router, the
replier, the mail extractor and the memory evaluator reach llama-server through
llm.Client, so quiesce could report zero requests in flight while the router was
mid-generation, and the old server was killed under it. The turn then finished
on the new model, which is the split turn the swap exists to prevent. llm.Client
now enters an optional Gate before every completion and LLMPhraser implements
it, so one counter covers every holder of the base URL.
A total failure also reported itself as a rollback. Swap set RolledBack on the
path where the rollback failed too, so the page rendered "rolled back to — she
is still answering, with the old model" over an empty model name and a daemon
with no model at all. The total failure has its own flag now, LiveModel stops
naming a gguf that is not loaded, and the log says another attempt can recover
without a restart, which is true.
The swap also ran on the connection every other page shares. ipc.Client holds
its mutex for a whole roundtrip with no read deadline on either side, so a load
froze /dash, /history and /notifications for minutes. mavweb dials a second
connection for /models alone. POST /models joins the route table, and the load
settings no longer come off a form that renders no input for them.
Found in review of #68.
llama-server is started without -np, so it serves one request at a time and
everything else queues. Mail extraction is allowed two minutes on a Thinking
1.7B, and the reader hands core up to 25 messages back to back. A turn arriving
mid-extraction therefore waited for whatever was left of that budget: the router
timed out into the classifier cascade and its 36.8% floor, and the phraser, which
has no floor, simply waited. Memory evaluation had the same shape with a five
minute budget.
llm.Gate is the bound. Foreground requests never wait. Background requests run
one at a time and yield while a foreground request is in flight, plus a quiet
window after it that covers the gap between the router call and the phraser call
of one turn. Clients get their priority from llmClientFor or
llmBackgroundClientFor, so which side a caller is on is decided at wiring time.
It gates only what goes through those clients, which the comment on Gate says.
mail intake: the extraction timeout no longer wraps the capture writes. A model
answering at 119 seconds of a 120 second budget left the first CaptureTask one
second and the third none, so candidates the model had already produced were
dropped with a deadline error. The mailbox name is validated before it becomes
provenance, since "email:" is not a source and neither is an arbitrary string
posted at the socket. The enable log prints the normalised candidate bound
rather than the configured one, which said "max 0" and then wrote three.
Found in review of #64.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TrVSBKe3RFDF4fGYKWYQnX
Loading a different gguf was a one-line edit to phraser.model_path plus a
restart. It is now an owner-triggered IPC call, off unless configured.
internal/phraser/swap.go holds the safety properties as code:
- Never two models resident. The old llama-server is killed and reaped
before the new one is launched. One 1.7B fits the Vega iGPU; a
blue/green overlap would OOM the box, so it is not offered.
- Atomic from a turn's point of view. Swap drains the in-flight turns
(they finish on the old model), then refuses arrivals with ErrSwapping
until the new server has answered /v1/models. No turn ever sees half a
swap; refused turns fall back to the classifier cascade.
- A failed load rolls back. If the new model does not start or does not
probe, the previous one is reloaded and the call returns RolledBack
with the error. If the rollback also fails the daemon says so and
degrades to the classifier rather than pretending to serve.
Holders of the completion client are re-pointed, not rebuilt: llm.Client
guards its base URL and LLMPhraser.OnSwap re-points it, so the router, the
replier, the mail extractor and the memory evaluator follow the new port
without knowing a swap happened.
Reach is deliberately narrow. phraser.swap_models is an exact-match
allowlist of absolute paths a human wrote, rejected at startup otherwise,
so "swap the model" can never mean "load any file on my disk"; the running
model is always swappable back to. MethodSwapModel is AuthStepUp, the same
rung as mutating the tool allowlist, and /models gates POST through the
same stepUpOK the tools page uses. Nothing calls Swap on a timer and no
act, intent or utterance reaches it.
Vikunja #250
Daemon side of Decision B: parse {"response","mood"} across the 4 consumers
(replier, nudges, reminders, chat), fall back to legacy formats. Drop the
LLM router — the classifier handles routing; replier/phraser share one
llm.Client (timeout 20s->60s). llm.Client reads reasoning_content when
content is empty (thinking models).
Docs: TTS piper-student plan (OmniVoice teacher -> piper student, from
scratch, phoneme-first). CLAUDE.md training guide.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add llmphraser: LFM-based phraser implementing Phraser interface with
PhraseChat, PhraseNudge, PhraseReactive, and PhraseReminder methods.
- Add shared internal/llm/client: llama-server completion client used by
both the phraser (talking back) and router (routing), sharing one model.
- Add LLMReplier in mavend: replaces StubReplier for chat/nudge/reactive
replies, falls back to stub on model errors.
- Update Phraser interface: add PhraseChat method, update stub to match.
- Wire LLM phaser into mavend voice init, plumb LLM config from JSON.
claude
kami