llm: keep the priority gate and the swap drain apart

Two fix branches independently added a Gate to internal/llm. One is priority
between a voice turn and a background job, the other is admission control while
the resident model is swapped. They are orthogonal and both are needed, so the
swap one is now SwapGate, with SetSwapGate to install it.

Complete takes the priority gate first and the drain second. A background
request can wait a long time on priority, and counting it as in flight against
the drain that whole time would stall a swap on a request that has not started.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TrVSBKe3RFDF4fGYKWYQnX
This commit is contained in:
kami
2026-08-01 14:38:11 +04:00
parent 2bf11f052d
commit 724e90759e
2 changed files with 4 additions and 4 deletions
+2 -2
View File
@@ -30,7 +30,7 @@ import (
// model load. No turn ever gets half of one model and half of another.
// "Every path" means every path: the router, the replier, the mail
// extractor and the memory evaluator do not call acquire, they call
// llm.Client.Complete, so LLMPhraser implements llm.Gate and the client
// llm.Client.Complete, so LLMPhraser implements llm.SwapGate and the client
// enters through the same counter.
//
// 3. A failed load rolls back to the model that was working. The new server is
@@ -149,7 +149,7 @@ func (p *LLMPhraser) acquire() (string, func(), error) {
}, nil
}
// Enter implements llm.Gate so every holder of an *llm.Client is drained by a
// Enter implements llm.SwapGate so every holder of an *llm.Client is drained by a
// swap, not only the phrasing paths in this package.
//
// The router, the replier, the mail extractor and the memory evaluator do not
+2 -2
View File
@@ -40,7 +40,7 @@ func TestSwap_WaitsForARouterCallThatWentThroughLLMClient(t *testing.T) {
release := make(chan struct{})
c := llm.New(blockingLLM(t, release).URL, 5*time.Second)
c.SetGate(p)
c.SetSwapGate(p)
completed := make(chan error, 1)
go func() {
@@ -81,7 +81,7 @@ func TestSwap_RefusesARouterCallThatArrivesMidSwap(t *testing.T) {
open := make(chan struct{})
close(open)
c := llm.New(blockingLLM(t, open).URL, 5*time.Second)
c.SetGate(p)
c.SetSwapGate(p)
// Hold the door shut the way quiesce does.
_, held, err := p.acquire()