phraser: gate every llm.Client call on the swap drain

The drain counted only the phrasing paths in internal/phraser. The router, the
replier, the mail extractor and the memory evaluator reach llama-server through
llm.Client, so quiesce could report zero requests in flight while the router was
mid-generation, and the old server was killed under it. The turn then finished
on the new model, which is the split turn the swap exists to prevent. llm.Client
now enters an optional Gate before every completion and LLMPhraser implements
it, so one counter covers every holder of the base URL.

A total failure also reported itself as a rollback. Swap set RolledBack on the
path where the rollback failed too, so the page rendered "rolled back to  — she
is still answering, with the old model" over an empty model name and a daemon
with no model at all. The total failure has its own flag now, LiveModel stops
naming a gguf that is not loaded, and the log says another attempt can recover
without a restart, which is true.

The swap also ran on the connection every other page shares. ipc.Client holds
its mutex for a whole roundtrip with no read deadline on either side, so a load
froze /dash, /history and /notifications for minutes. mavweb dials a second
connection for /models alone. POST /models joins the route table, and the load
settings no longer come off a form that renders no input for them.

Found in review of #68.
This commit is contained in:
kami
2026-08-01 14:15:58 +04:00
parent 810076451f
commit 3ff2a9340a
8 changed files with 312 additions and 17 deletions
+13 -1
View File
@@ -363,6 +363,11 @@ func main() {
flag.Parse()
var core ipc.CoreAPI
// swapConn — a second connection, for /models and nothing else. A model swap
// is a multi-minute IPC call and ipc.Client serialises everything on one
// mutex, so sharing the connection would freeze every other page for the
// length of the load. See handleModels.
var swapConn modelController
if *coreSock != "" {
c, err := ipc.DialWait(*coreSock, 60*time.Second)
if err != nil {
@@ -370,6 +375,12 @@ func main() {
}
defer c.Close()
core = c
if sc, err := ipc.Dial(*coreSock); err != nil {
log.Printf("models: second core connection failed (%v) — /models will share the main one and a swap will block the other pages", err)
} else {
defer sc.Close()
swapConn = sc
}
}
mux := http.NewServeMux()
@@ -499,13 +510,14 @@ func main() {
// /tools, and for a comparable reason: which model is loaded decides how every
// utterance is routed and how every reply is worded. GET is read-only.
mux.HandleFunc("/models", func(w http.ResponseWriter, r *http.Request) {
handleModels(w, r, core, stepUpSession, *requireStepUp)
handleModels(w, r, core, swapConn, stepUpSession, *requireStepUp)
})
// State-changing routes on this server, and their gate (Vikunja #317):
//
// POST /tools step-up — defines argv that internal/tool executes
// POST /routines step-up — accepting schedules recurring firing
// POST /models step-up — replaces the model that routes and phrases
// POST /api/revert step-up — voids the latest fact for a key
// POST /api/chat step-up — reaches the router, LLM and the act path
// POST /api/signal none — appends a presence fact, no argv, no act