kami
ad074cea31
Loading a different gguf was a one-line edit to phraser.model_path plus a
restart. It is now an owner-triggered IPC call, off unless configured.
internal/phraser/swap.go holds the safety properties as code:
- Never two models resident. The old llama-server is killed and reaped
before the new one is launched. One 1.7B fits the Vega iGPU; a
blue/green overlap would OOM the box, so it is not offered.
- Atomic from a turn's point of view. Swap drains the in-flight turns
(they finish on the old model), then refuses arrivals with ErrSwapping
until the new server has answered /v1/models. No turn ever sees half a
swap; refused turns fall back to the classifier cascade.
- A failed load rolls back. If the new model does not start or does not
probe, the previous one is reloaded and the call returns RolledBack
with the error. If the rollback also fails the daemon says so and
degrades to the classifier rather than pretending to serve.
Holders of the completion client are re-pointed, not rebuilt: llm.Client
guards its base URL and LLMPhraser.OnSwap re-points it, so the router, the
replier, the mail extractor and the memory evaluator follow the new port
without knowing a swap happened.
Reach is deliberately narrow. phraser.swap_models is an exact-match
allowlist of absolute paths a human wrote, rejected at startup otherwise,
so "swap the model" can never mean "load any file on my disk"; the running
model is always swappable back to. MethodSwapModel is AuthStepUp, the same
rung as mutating the tool allowlist, and /models gates POST through the
same stepUpOK the tools page uses. Nothing calls Swap on a timer and no
act, intent or utterance reaches it.
Vikunja #250
88 lines
3.1 KiB
Go
88 lines
3.1 KiB
Go
// mavend/memoryeval.go — the driver for background memory evaluation
|
|
// (Vikunja #248). The evaluator itself is pure-ish and lives in
|
|
// internal/memeval; this is the one impure part: a ticker, the store, and the
|
|
// resident model's base URL.
|
|
//
|
|
// It is its own goroutine and NOT a step on the main tick, deliberately. The
|
|
// tick runs every 60s and has a delivery deadline behind it; an evaluation is
|
|
// a multi-second LLM round-trip on the same llama-server that answers voice
|
|
// turns, and it happens hourly at most. Bolting it onto the tick would make
|
|
// every hour's tick the slow one for no benefit.
|
|
package main
|
|
|
|
import (
|
|
"context"
|
|
"log"
|
|
"time"
|
|
|
|
"github.com/kami/maven/internal/config"
|
|
"github.com/kami/maven/internal/memeval"
|
|
"github.com/kami/maven/internal/phraser"
|
|
"github.com/kami/maven/internal/store"
|
|
)
|
|
|
|
// memoryEvalWorker — ticker + evaluator.
|
|
type memoryEvalWorker struct {
|
|
eval *memeval.Evaluator
|
|
interval time.Duration
|
|
}
|
|
|
|
// newMemoryEvalWorker wires the evaluation loop, or returns nil when it should
|
|
// not run at all. nil is the normal case and every caller must handle it:
|
|
//
|
|
// - no memory_eval config block ⇒ off (a capability is off unless configured);
|
|
// - no LLM phraser ⇒ nothing to evaluate with. There is no template fallback
|
|
// here on purpose: a "memory evaluation" assembled from string templates
|
|
// would be a fixed sentence pretending to be an observation.
|
|
func newMemoryEvalWorker(st *store.Store, phr phraser.Phraser, cfg *config.Config) *memoryEvalWorker {
|
|
if cfg.MemoryEval == nil {
|
|
return nil
|
|
}
|
|
lp, ok := phr.(*phraser.LLMPhraser)
|
|
if !ok {
|
|
log.Printf("memory eval: configured but no llama-server phraser — evaluation disabled")
|
|
return nil
|
|
}
|
|
interval := time.Duration(cfg.MemoryEval.Interval)
|
|
if interval <= 0 {
|
|
interval = config.DefaultMemoryEvalInterval
|
|
}
|
|
// A generous per-request timeout: this is a long prompt to a Thinking model
|
|
// and nobody is waiting on the answer.
|
|
client := llmClientFor(lp, 5*time.Minute)
|
|
ev := memeval.NewEvaluator(st, st, client, memeval.Config{
|
|
MaxItems: cfg.MemoryEval.MaxItems,
|
|
MinConfidence: cfg.MemoryEval.MinConfidence,
|
|
ContextBlock: contextBlockFn(cfg, time.Now),
|
|
})
|
|
log.Printf("memory eval: enabled, every %s", interval)
|
|
return &memoryEvalWorker{eval: ev, interval: interval}
|
|
}
|
|
|
|
// run evaluates every interval until ctx is canceled.
|
|
//
|
|
// The first evaluation waits a full interval rather than firing at startup, the
|
|
// opposite of the tick loop's cold-start behaviour. A tick that fires late is a
|
|
// nudge that arrives late; an evaluation that fires late is nothing at all, and
|
|
// the alternative is a heavy LLM call competing with startup — including with
|
|
// the first voice turn after a restart.
|
|
func (w *memoryEvalWorker) run(ctx context.Context) {
|
|
ticker := time.NewTicker(w.interval)
|
|
defer ticker.Stop()
|
|
for {
|
|
select {
|
|
case <-ctx.Done():
|
|
return
|
|
case now := <-ticker.C:
|
|
obs, err := w.eval.Evaluate(ctx, now)
|
|
if err != nil {
|
|
log.Printf("memory eval: %v", err)
|
|
continue
|
|
}
|
|
for _, o := range obs {
|
|
log.Printf("memory eval: noted (%.2f, %s): %s", o.Conf, o.Action, o.Text)
|
|
}
|
|
}
|
|
}
|
|
}
|