kami
da60c14399
- Dockerfile: multi-stage build with CGO_ENABLED=0, embedder model copy, non-root user, healthcheck, and /data volume. - docker-compose.yml: mavend + mavweb services with shared volume, health checks, and restart policy. - .gitignore: ignore models/llm/*.gguf, deploy/telegram.env, tmp artifacts. - deploy/mavend.json: add LLM, phraser, voice sections (embedder, model paths, wake sensitivity). Add telegram token env-var expansion. - deploy/telegram.env.example: template for telegram bot token. - internal/config/config.go: add LLM config struct, voice config struct (embedder, llama, wake sensitivity), telegram token loading. - telegramsink: add chat intent delivery support alongside existing types. - voicesink: skip empty payloads in delivery. - dialogue/session: add chat intent to anaphora resolution, test coverage. - AGENTS.md: update with LLM embedder, LFM model download/configure steps, new UI conventions. - REARCH.md: architecture research document. - cmd/mavend/main.go: wire LLM config, phraser, embedder, telegram config, WebAuthn, IPC event/routine handlers, and reactive notes.
93 lines
4.2 KiB
Markdown
93 lines
4.2 KiB
Markdown
# Maven — Re-architecture (router-centric, 2026-07-10)
|
|
|
|
> Supersedes the classifier-first routing model. Agreed in a design session
|
|
> after diagnosing that homesrv deploys with a **stub phraser** (no LLM
|
|
> running) and an embedder-classifier that routes by nearest-neighbor between
|
|
> frozen seed phrases — the structural cause of "she messes up queries."
|
|
>
|
|
> Hardware reality: homesrv = Ryzen 5 5600U laptop, Vega iGPU, 14 GB shared
|
|
> RAM. Workstation (RX 7900 XT) is NOT the deploy target and is often busy.
|
|
> So: small models, on-demand where heavy, always-on where cheap.
|
|
|
|
## Principle
|
|
|
|
The LLM is **not** the center of everything. Deterministic tools handle the
|
|
bulk. The LLM is used for exactly two things: **routing/reasoning** and
|
|
**talking back**. A sub-1B agentic model (LFM 2.5) is enough for both.
|
|
|
|
**If the router is good, Maven feels good.** Routing is the linchpin.
|
|
|
|
## The spine
|
|
|
|
```
|
|
utterance
|
|
→ [world-state context] cheap: time, presence, calendar_busy, weather (no LLM)
|
|
→ ROUTER = LFM (always-on, agentic)
|
|
reads utterance + context + tool schema, emits a STRUCTURED action:
|
|
• call a tool (deterministic) • answer directly
|
|
• escalate → 4B reasoner (on-demand)
|
|
→ tools (deterministic, fast) / 4B reasoner (on-demand summon)
|
|
→ PHRASER = LFM (always-on, same process as router) → TTS / text
|
|
```
|
|
|
|
- **Router = Phraser = one resident sub-1B LFM llama-server**, two call-sites
|
|
(route-prompt, phrase-prompt). Always warm, no cold start. Cheap on 14 GB.
|
|
- **4B reasoner (Qwen3-4B, already on disk)** — summoned on-demand for
|
|
genuinely complex turns, torn down / idle-unloaded after. Never resident.
|
|
- **Embedder demoted from router to tool** — it now backs `memory.search`
|
|
(RAG) and gives the router a cheap "similar past notes/intents" hint. The
|
|
router no longer depends on it clearing a threshold. Upgrade MiniLM → bge-m3
|
|
for better RU retrieval later (model swap, not architecture).
|
|
|
|
### Router output
|
|
- Constrained structured JSON action `{tool, args, escalate}` — NOT free-form
|
|
multi-step function-calling. Sub-1B is far more reliable emitting a fixed
|
|
schema. Enforce with a **GBNF grammar** in llama.cpp (near-bulletproof).
|
|
- Keep the existing **stage-0 exact-match fast-path** for dead-obvious commands
|
|
(skips the router entirely) — cheap insurance, already built.
|
|
|
|
## The proactive / memory half — one background engine
|
|
|
|
"Take notes," "remember," "reflect," "suggest do you want to add X?", "remind"
|
|
are NOT request-path features. They are one **digestion worker**:
|
|
|
|
```
|
|
DIGESTION WORKER (periodic + event-driven, off the request path)
|
|
• reads new facts/notes since last pass
|
|
• RAG-consolidates: dedupe, link, summarize into durable memory
|
|
• reflects: detect patterns ("mentioned X three times")
|
|
• proposes: "want me to add X / remind you about Y?" → nudge dispatcher
|
|
• surfaces due reminders
|
|
runs LFM (cheap) or summons 4B (real synthesis) — never blocks a turn
|
|
```
|
|
|
|
Notes capture is a deterministic Tier-0 tool; making notes *mean something
|
|
later* is the worker + RAG.
|
|
|
|
## Layer table
|
|
|
|
| Layer | What | Runs |
|
|
|---|---|---|
|
|
| Context | world-state (time/presence/calendar/weather) | always, no LLM |
|
|
| **Router** | LFM agentic orchestrator — linchpin | **always-on** |
|
|
| Tools | note/reminder/memory/calendar/weather/act (deterministic) | always |
|
|
| Reasoner | Qwen3-4B for complex turns | **on-demand summon** |
|
|
| Phraser | LFM — final voice | **always-on (same proc as router)** |
|
|
| Digestion worker | reflection → suggestions/nudges/memory | **background** |
|
|
| Reach | telegram (+ existing ntfy/voice) | quick win |
|
|
| Voice quality | custom/better TTS | **deferred** (workstation GPU busy) |
|
|
|
|
## Build order
|
|
|
|
1. **Foundation + router** — router-as-LFM, turn the engine ON (resident
|
|
sub-1B), verify notes+reminders actually round-trip, date/number TTS
|
|
normalizer, wire telegram reach. After this she's a trustworthy plain
|
|
assistant.
|
|
2. **On-demand 4B reasoner** — summon/idle lifecycle + router escalation path.
|
|
3. **Digestion worker** — reflection, proactive suggestions, memory
|
|
consolidation, RAG read-back.
|
|
4. **Embodiment** — voice quality (deferred).
|
|
|
|
## Non-goals (unchanged)
|
|
Never phones home. Not a nag. Not autonomous. Feminine-gendered RU self-ref.
|