phraser: gate every llm.Client call on the swap drain

The drain counted only the phrasing paths in internal/phraser. The router, the
replier, the mail extractor and the memory evaluator reach llama-server through
llm.Client, so quiesce could report zero requests in flight while the router was
mid-generation, and the old server was killed under it. The turn then finished
on the new model, which is the split turn the swap exists to prevent. llm.Client
now enters an optional Gate before every completion and LLMPhraser implements
it, so one counter covers every holder of the base URL.

A total failure also reported itself as a rollback. Swap set RolledBack on the
path where the rollback failed too, so the page rendered "rolled back to  — she
is still answering, with the old model" over an empty model name and a daemon
with no model at all. The total failure has its own flag now, LiveModel stops
naming a gguf that is not loaded, and the log says another attempt can recover
without a restart, which is true.

The swap also ran on the connection every other page shares. ipc.Client holds
its mutex for a whole roundtrip with no read deadline on either side, so a load
froze /dash, /history and /notifications for minutes. mavweb dials a second
connection for /models alone. POST /models joins the route table, and the load
settings no longer come off a form that renders no input for them.

Found in review of #68.
This commit is contained in:
kami
2026-08-01 14:15:58 +04:00
parent 810076451f
commit 3ff2a9340a
8 changed files with 312 additions and 17 deletions
+6
View File
@@ -58,6 +58,7 @@ func wireModelSwap(srv *ipc.Server, phr phraser.Phraser, cfg *config.Config) {
ModelPath: res.ModelPath,
BaseURL: res.BaseURL,
RolledBack: res.RolledBack,
NoBackend: res.NoBackend,
TookMs: res.Took.Milliseconds(),
}
if err != nil {
@@ -106,8 +107,13 @@ func wireModelSwap(srv *ipc.Server, phr phraser.Phraser, cfg *config.Config) {
// the port of a server that no longer exists, and the daemon would degrade to
// the classifier permanently after the first swap. The client is re-pointed, not
// rebuilt, so nothing that holds it has to know a swap happened.
// SetGate is the other half, and on the deploy shape it is the load-bearing one:
// llama-server is relaunched on the same fixed port, so SetBaseURL is usually a
// no-op, while the gate is what makes the swap's drain count these callers at
// all. Without it a swap can kill the server mid-routing-decision.
func llmClientFor(lp *phraser.LLMPhraser, timeout time.Duration) *llm.Client {
c := llm.New(lp.BaseURL(), timeout)
lp.OnSwap(func(base string) { c.SetBaseURL(base) })
c.SetGate(lp)
return c
}