The drain counted only the phrasing paths in internal/phraser. The router, the
replier, the mail extractor and the memory evaluator reach llama-server through
llm.Client, so quiesce could report zero requests in flight while the router was
mid-generation, and the old server was killed under it. The turn then finished
on the new model, which is the split turn the swap exists to prevent. llm.Client
now enters an optional Gate before every completion and LLMPhraser implements
it, so one counter covers every holder of the base URL.
A total failure also reported itself as a rollback. Swap set RolledBack on the
path where the rollback failed too, so the page rendered "rolled back to — she
is still answering, with the old model" over an empty model name and a daemon
with no model at all. The total failure has its own flag now, LiveModel stops
naming a gguf that is not loaded, and the log says another attempt can recover
without a restart, which is true.
The swap also ran on the connection every other page shares. ipc.Client holds
its mutex for a whole roundtrip with no read deadline on either side, so a load
froze /dash, /history and /notifications for minutes. mavweb dials a second
connection for /models alone. POST /models joins the route table, and the load
settings no longer come off a form that renders no input for them.
Found in review of #68.
Loading a different gguf was a one-line edit to phraser.model_path plus a
restart. It is now an owner-triggered IPC call, off unless configured.
internal/phraser/swap.go holds the safety properties as code:
- Never two models resident. The old llama-server is killed and reaped
before the new one is launched. One 1.7B fits the Vega iGPU; a
blue/green overlap would OOM the box, so it is not offered.
- Atomic from a turn's point of view. Swap drains the in-flight turns
(they finish on the old model), then refuses arrivals with ErrSwapping
until the new server has answered /v1/models. No turn ever sees half a
swap; refused turns fall back to the classifier cascade.
- A failed load rolls back. If the new model does not start or does not
probe, the previous one is reloaded and the call returns RolledBack
with the error. If the rollback also fails the daemon says so and
degrades to the classifier rather than pretending to serve.
Holders of the completion client are re-pointed, not rebuilt: llm.Client
guards its base URL and LLMPhraser.OnSwap re-points it, so the router, the
replier, the mail extractor and the memory evaluator follow the new port
without knowing a swap happened.
Reach is deliberately narrow. phraser.swap_models is an exact-match
allowlist of absolute paths a human wrote, rejected at startup otherwise,
so "swap the model" can never mean "load any file on my disk"; the running
model is always swappable back to. MethodSwapModel is AuthStepUp, the same
rung as mutating the tool allowlist, and /models gates POST through the
same stepUpOK the tools page uses. Nothing calls Swap on a timer and no
act, intent or utterance reaches it.
Vikunja #250
The phrasing eval printed "llm (0.8B, ...)" no matter which gguf
llama-server had loaded, so two runs of two different models came out
named the same and were easy to mix up when comparing.
It now asks llama-server over /v1/models, same as the router eval
already did. The helper moved to internal/llm so both share it, and it
now errors instead of returning a blank name when the id field is
missing — an unreachable server gets labelled "unknown-model", never a
plausible-looking guess.
Both eval paths stay opt-in behind MAVEN_LLM_URL; no server needed for
go test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CGeSZxh1DCtRxmFVSYVGvJ
Daemon side of Decision B: parse {"response","mood"} across the 4 consumers
(replier, nudges, reminders, chat), fall back to legacy formats. Drop the
LLM router — the classifier handles routing; replier/phraser share one
llm.Client (timeout 20s->60s). llm.Client reads reasoning_content when
content is empty (thinking models).
Docs: TTS piper-student plan (OmniVoice teacher -> piper student, from
scratch, phoneme-first). CLAUDE.md training guide.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add llmphraser: LFM-based phraser implementing Phraser interface with
PhraseChat, PhraseNudge, PhraseReactive, and PhraseReminder methods.
- Add shared internal/llm/client: llama-server completion client used by
both the phraser (talking back) and router (routing), sharing one model.
- Add LLMReplier in mavend: replaces StubReplier for chat/nudge/reactive
replies, falls back to stub on model errors.
- Update Phraser interface: add PhraseChat method, update stub to match.
- Wire LLM phaser into mavend voice init, plumb LLM config from JSON.
kami