kami
dc7c72a3d7
Ships the real, local, testable part of the memory-evaluation plan
(docs/plans/03-memory-evaluation.md): Maven reads back her own recent
memory on a slow ticker, asks the resident model what it notices, and
records the confident answers as notes.
internal/memeval — not internal/memory/eval.go as the plan says, because
internal/store imports internal/memory for the vector backend and an
evaluator has to read store.Fact/Note/Nudge, which would close the
cycle. Evaluate() gathers RecentFacts/RecentNotes/RecentNudges, prompts
under a GBNF grammar bounded to three {observation, confidence,
suggested_action} objects, drops anything under min_confidence,
deduplicates against what earlier runs wrote, and writes the rest as
notes with source infer:memory-eval. /dash already renders notes with
their source, so the output is visible with no UI change.
cmd/mavend/memoryeval.go drives it on its own goroutine and ticker, not
on the 60s tick: an evaluation is a multi-second round-trip on the same
llama-server that answers voice turns, and it runs hourly at most. The
memory_eval config block is absent by default and absence means the
goroutine does not exist. No llama-server phraser also means no loop —
there is no template fallback, because a "memory evaluation" assembled
from templates is a fixed sentence pretending to be an observation.
What it deliberately cannot do, since this is the feature most likely to
turn Maven into a nag:
- It cannot speak. No dispatcher reference, no channel, no nudge. An
observation is a thought she wrote down and he reads on /dash.
Announcing them is a separate decision with its own opt-in.
- It cannot act. suggested_action is recorded as text and interpreted
by nobody — no reminder, routine or fact is created from it.
- It says nothing about an empty store: no memory means no LLM call,
so there are no observations invented out of two facts.
- Its own notes are excluded from the next evaluation's input, and are
written with a nil embedding so they stay out of the recall pool.
The plan's remaining items (dispatching observations, an /eval IPC
method and trace view, RecentEvents) and the fact that output quality is
entirely unmeasured are written up at the bottom of the plan doc.
89 lines
3.1 KiB
Go
89 lines
3.1 KiB
Go
// mavend/memoryeval.go — the driver for background memory evaluation
|
|
// (Vikunja #248). The evaluator itself is pure-ish and lives in
|
|
// internal/memeval; this is the one impure part: a ticker, the store, and the
|
|
// resident model's base URL.
|
|
//
|
|
// It is its own goroutine and NOT a step on the main tick, deliberately. The
|
|
// tick runs every 60s and has a delivery deadline behind it; an evaluation is
|
|
// a multi-second LLM round-trip on the same llama-server that answers voice
|
|
// turns, and it happens hourly at most. Bolting it onto the tick would make
|
|
// every hour's tick the slow one for no benefit.
|
|
package main
|
|
|
|
import (
|
|
"context"
|
|
"log"
|
|
"time"
|
|
|
|
"github.com/kami/maven/internal/config"
|
|
"github.com/kami/maven/internal/llm"
|
|
"github.com/kami/maven/internal/memeval"
|
|
"github.com/kami/maven/internal/phraser"
|
|
"github.com/kami/maven/internal/store"
|
|
)
|
|
|
|
// memoryEvalWorker — ticker + evaluator.
|
|
type memoryEvalWorker struct {
|
|
eval *memeval.Evaluator
|
|
interval time.Duration
|
|
}
|
|
|
|
// newMemoryEvalWorker wires the evaluation loop, or returns nil when it should
|
|
// not run at all. nil is the normal case and every caller must handle it:
|
|
//
|
|
// - no memory_eval config block ⇒ off (a capability is off unless configured);
|
|
// - no LLM phraser ⇒ nothing to evaluate with. There is no template fallback
|
|
// here on purpose: a "memory evaluation" assembled from string templates
|
|
// would be a fixed sentence pretending to be an observation.
|
|
func newMemoryEvalWorker(st *store.Store, phr phraser.Phraser, cfg *config.Config) *memoryEvalWorker {
|
|
if cfg.MemoryEval == nil {
|
|
return nil
|
|
}
|
|
lp, ok := phr.(*phraser.LLMPhraser)
|
|
if !ok {
|
|
log.Printf("memory eval: configured but no llama-server phraser — evaluation disabled")
|
|
return nil
|
|
}
|
|
interval := time.Duration(cfg.MemoryEval.Interval)
|
|
if interval <= 0 {
|
|
interval = config.DefaultMemoryEvalInterval
|
|
}
|
|
// A generous per-request timeout: this is a long prompt to a Thinking model
|
|
// and nobody is waiting on the answer.
|
|
client := llm.New(lp.BaseURL(), 5*time.Minute)
|
|
ev := memeval.NewEvaluator(st, st, client, memeval.Config{
|
|
MaxItems: cfg.MemoryEval.MaxItems,
|
|
MinConfidence: cfg.MemoryEval.MinConfidence,
|
|
ContextBlock: contextBlockFn(cfg, time.Now),
|
|
})
|
|
log.Printf("memory eval: enabled, every %s", interval)
|
|
return &memoryEvalWorker{eval: ev, interval: interval}
|
|
}
|
|
|
|
// run evaluates every interval until ctx is canceled.
|
|
//
|
|
// The first evaluation waits a full interval rather than firing at startup, the
|
|
// opposite of the tick loop's cold-start behaviour. A tick that fires late is a
|
|
// nudge that arrives late; an evaluation that fires late is nothing at all, and
|
|
// the alternative is a heavy LLM call competing with startup — including with
|
|
// the first voice turn after a restart.
|
|
func (w *memoryEvalWorker) run(ctx context.Context) {
|
|
ticker := time.NewTicker(w.interval)
|
|
defer ticker.Stop()
|
|
for {
|
|
select {
|
|
case <-ctx.Done():
|
|
return
|
|
case now := <-ticker.C:
|
|
obs, err := w.eval.Evaluate(ctx, now)
|
|
if err != nil {
|
|
log.Printf("memory eval: %v", err)
|
|
continue
|
|
}
|
|
for _, o := range obs {
|
|
log.Printf("memory eval: noted (%.2f, %s): %s", o.Conf, o.Action, o.Text)
|
|
}
|
|
}
|
|
}
|
|
}
|