71 lines
4.6 KiB
Markdown
71 lines
4.6 KiB
Markdown
# Plan: Background Memory Evaluation & Idea Generation
|
|
|
|
**Goal:** Maven periodically reviews her own memory stores (facts, notes, events, nudges), evaluates coherence and gaps, and generates proactive proposals — new routines, configuration tweaks, observations she can share with the user.
|
|
|
|
**Done when:**
|
|
- `internal/memory/eval.go` — periodic evaluation loop runs on a slow cadence (1h)
|
|
- Evaluation reads `RecentFacts`, `RecentNotes`, `RecentNudges`, `RecentEvents` via `store.Store` or `ipc.CoreAPI`
|
|
- LLM summarizes state, detects anomalies (e.g. "you haven't recorded a meal in 3 days — is your routine broken?"), proposes new care rules
|
|
- Generated proposals are written as notes (kind `note`, source `infer:memory-eval`) and/or trigger nudges through the dispatcher
|
|
- Evaluation trace visible on `/history` page in mavweb
|
|
|
|
**Scope:**
|
|
- New `internal/memory/eval.go` — evaluator struct calling `internal/llm.Client` with a summarization prompt
|
|
- Reuses `internal/delivery.Dispatcher` for surfacing insights as care nudges (sev1)
|
|
- Reuses `internal/store` for reading memory state and writing evaluation notes
|
|
- Daemon wiring: new evaluation goroutine in `cmd/mavend/main.go`
|
|
- Config: `memory_eval_interval` in `config.Config` (default 1h, 0 to disable)
|
|
|
|
**Steps:**
|
|
1. Create `internal/memory/eval.go` — `Evaluator` struct holding `*store.Store`, `*llm.Client`, `*delivery.Dispatcher`
|
|
2. Implement `Evaluate(ctx)` — reads last N facts, notes, nudges, events, builds a prompt summarizing patterns, anomalies, gaps
|
|
3. LLM call returns structured observations: `{"observation":"...","confidence":0.8,"suggested_action":"remind|propose|notify"}`
|
|
4. High-confidence observations written as notes (`source:infer:memory-eval`) or dispatched as care nudges (sev1) through `dispatcher.DispatchNudge`
|
|
5. Wire evaluator goroutine in `cmd/mavend/main.go` — separate ticker, not on the main tick loop
|
|
6. Add `/eval` API method to `ipc.CoreAPI` (or reuse `Chat` with system context) so mavweb can show evaluation history
|
|
7. Add `memory_eval` block to `deploy/mavend.json`
|
|
8. Test with synthetic store state — verify observations match expected patterns
|
|
|
|
---
|
|
|
|
## Status 2026-08-01 — foundation shipped (V-248)
|
|
|
|
**Shipped:** `internal/memeval` (not `internal/memory/eval.go` — `internal/store`
|
|
imports `internal/memory` for the vector backend, so an evaluator that reads
|
|
`store.Fact` there would close an import cycle). `Evaluator.Evaluate` reads
|
|
`RecentFacts` / `RecentNotes` / `RecentNudges`, prompts the resident model under
|
|
a GBNF grammar for at most three `{observation, confidence, suggested_action}`
|
|
objects, drops anything under `min_confidence`, deduplicates against what earlier
|
|
evaluations wrote, and records the rest as notes with source `infer:memory-eval`.
|
|
Driver: `cmd/mavend/memoryeval.go`, its own goroutine on its own ticker. Config:
|
|
the `memory_eval` block — **absent ⇒ the loop does not run**. Visibility: `/dash`
|
|
already renders notes with their source, so evaluation output is visible with no
|
|
UI change.
|
|
|
|
**Deliberately not shipped — this is policy, not an unfinished edge:**
|
|
|
|
- *Dispatching observations as care nudges (plan step 4).* An hourly LLM loop
|
|
with permission to speak is a machine for generating interruptions, and the
|
|
content is model-generated text about his own life. The evaluator has no
|
|
dispatcher reference at all, so it cannot reach a channel by accident. Wiring
|
|
it to `delivery.Dispatcher` is a separate decision with its own opt-in.
|
|
- *Acting on `suggested_action`.* It is recorded inside the note text and
|
|
interpreted by nobody. No reminder, routine or fact is created.
|
|
- *Writing observation embeddings.* Notes are written with a nil embedding, so
|
|
they stay out of the RAG recall pool. Feeding generated text back into the pool
|
|
it came from is how a small model starts citing its own guesses as evidence.
|
|
|
|
**Deferred, wants a decision or another capability:**
|
|
|
|
- *Plan step 6, the `/eval` IPC method and an evaluation-history view.* `/dash`
|
|
covers reading the output; a dedicated trace surface is worth building once
|
|
there is real output to look at, and it should probably show the prompt too.
|
|
- *`RecentEvents`.* The plan lists it; the evaluator reads facts, notes and
|
|
nudges. Detected action/object events already drive pattern proposals (#43), and
|
|
duplicating them here would mostly re-derive that.
|
|
- *Output quality is unmeasured.* There is no fixture for "did she notice
|
|
something true". The tests cover the machinery — empty store, confidence floor,
|
|
dedupe, own-notes exclusion, error handling — not the observations. Until
|
|
someone reads a week of real output on `/dash`, treat the wording and the
|
|
`min_confidence` default as unvalidated.
|