Swap the resident model without restarting mavend (#250)

Loading a different gguf was a one-line edit to phraser.model_path plus a
restart. It is now an owner-triggered IPC call, off unless configured.

internal/phraser/swap.go holds the safety properties as code:

  - Never two models resident. The old llama-server is killed and reaped
    before the new one is launched. One 1.7B fits the Vega iGPU; a
    blue/green overlap would OOM the box, so it is not offered.
  - Atomic from a turn's point of view. Swap drains the in-flight turns
    (they finish on the old model), then refuses arrivals with ErrSwapping
    until the new server has answered /v1/models. No turn ever sees half a
    swap; refused turns fall back to the classifier cascade.
  - A failed load rolls back. If the new model does not start or does not
    probe, the previous one is reloaded and the call returns RolledBack
    with the error. If the rollback also fails the daemon says so and
    degrades to the classifier rather than pretending to serve.

Holders of the completion client are re-pointed, not rebuilt: llm.Client
guards its base URL and LLMPhraser.OnSwap re-points it, so the router, the
replier, the mail extractor and the memory evaluator follow the new port
without knowing a swap happened.

Reach is deliberately narrow. phraser.swap_models is an exact-match
allowlist of absolute paths a human wrote, rejected at startup otherwise,
so "swap the model" can never mean "load any file on my disk"; the running
model is always swappable back to. MethodSwapModel is AuthStepUp, the same
rung as mutating the tool allowlist, and /models gates POST through the
same stepUpOK the tools page uses. Nothing calls Swap on a timer and no
act, intent or utterance reaches it.

Vikunja #250
This commit is contained in:
kami
2026-08-01 03:59:08 +04:00
parent 2c1b0eede0
commit ad074cea31
22 changed files with 1523 additions and 43 deletions
+46 -2
View File
@@ -432,6 +432,19 @@ type Server struct {
// every CoreAPI implementation has to carry.
IngestMailFn IngestMailFunc
// SwapModelFn / ModelStatusFn — the on-the-fly resident model swap (Vikunja
// #250) and its read side. Set by the daemon only when phraser.swap_models
// lists at least one model AND the phraser owns a llama-server; nil ⇒ both
// methods answer ErrUnknownMethod, which is what "off unless configured"
// looks like at the wire.
//
// They bypass CoreAPI for the same reason IngestMailFn does: this is not a
// store operation, it needs the daemon's llama-server, and no other CoreAPI
// implementation should have to carry it. MethodSwapModel is AuthStepUp in
// internal/auth — owner-triggered, never an act and never a timer.
SwapModelFn SwapModelFunc
ModelStatusFn ModelStatusFunc
// UnlockFn — unwraps the store encryption key from the wrapped blob using
// the passkey credential public key, opens the encrypted store, and wires
// the rest of the daemon (voice, loop, delivery). Set by the daemon when
@@ -450,6 +463,12 @@ type WrapKeyFunc func(ctx context.Context, publicKey []byte) error
// public key and completes daemon initialization.
type UnlockFunc func(ctx context.Context, publicKey []byte) error
// SwapModelFunc — loads another resident model in place of the live one.
type SwapModelFunc func(ctx context.Context, req SwapModelReq) (SwapModelResp, error)
// ModelStatusFunc — reports the resident model and the swap allowlist.
type ModelStatusFunc func(ctx context.Context) (ModelStatusResp, error)
// IngestMailFunc — core-side mail extraction. Returns what was captured.
type IngestMailFunc func(ctx context.Context, req IngestMailReq) (IngestMailResp, error)
@@ -620,8 +639,9 @@ func withoutParams[R any](fn func(ctx context.Context, api CoreAPI) (R, error))
// existed) as an argument — so SetAPI's runtime swap (the unlock transition)
// is still honored on the very next request with no extra plumbing here.
//
// MethodAssertStepUp, MethodStoreEncryptionKey, MethodUnlock and
// MethodIngestMail are NOT in this table: they bypass CoreAPI entirely
// MethodAssertStepUp, MethodStoreEncryptionKey, MethodUnlock,
// MethodIngestMail, MethodSwapModel and MethodModelStatus are NOT in this
// table: they bypass CoreAPI entirely
// (s.StepUp / s.WrapKeyFn / s.UnlockFn / s.IngestMailFn), so dispatch
// special-cases them before consulting the table.
var methodTable = map[Method]handlerFunc{
@@ -874,6 +894,30 @@ func (s *Server) dispatch(ctx context.Context, req Request) (json.RawMessage, er
return marshalResult(resp), nil
}
return nil, fmt.Errorf("%w: %s", ErrUnknownMethod, req.Method)
case MethodSwapModel:
if s.SwapModelFn != nil {
var p SwapModelReq
if err := unmarshalParams(req.Params, &p); err != nil {
return nil, err
}
resp, err := s.SwapModelFn(ctx, p)
if err != nil {
return nil, err
}
return marshalResult(resp), nil
}
return nil, fmt.Errorf("%w: %s", ErrUnknownMethod, req.Method)
case MethodModelStatus:
if s.ModelStatusFn != nil {
resp, err := s.ModelStatusFn(ctx)
if err != nil {
return nil, err
}
return marshalResult(resp), nil
}
return nil, fmt.Errorf("%w: %s", ErrUnknownMethod, req.Method)
}
h, ok := methodTable[req.Method]