feat(mavweb): /ecosystem page consuming Nexus/Praxis/Hexis + shell fixes
Add a read-only /ecosystem page that consumes the sibling services' JSON APIs (Nexus entities, Praxis attention, Hexis capabilities), fetched concurrently with honest per-panel error states. Siblings stay headless — mavweb is their human surface (arch §16). Wired via mavweb -nexus/-praxis/-hexis flags; mavweb joins the ecosystem compose network. Fix mobile horizontal overflow across all pages: .content is a flex child with default min-width:auto, so it refused to shrink below the tables' intrinsic width. min-width:0 lets wide tables pan inside .scroll instead of dragging the page sideways. Verified via CDP geometry check (scrollWidth === clientWidth at 430px). Also includes in-progress Ethos UI redesign, ecosystem deploy compose, and planning docs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,29 @@
|
||||
# Plan: Background Email Reader
|
||||
|
||||
**Goal:** Maven periodically polls configured IMAP mailboxes, extracts structured items (tasks, reminders, calendar events, important facts) via LLM, and writes them into the store through `ipc.CoreAPI` — discarding spam, newsletters, and noise.
|
||||
|
||||
**Done when:**
|
||||
- `internal/email/` package exists with IMAP idle+poll loop
|
||||
- New `email` block in `config.Config` (mailboxes, poll interval, extraction model)
|
||||
- Extracted items land as `WriteFact`/`CreateReminder`/`WriteNote` via `ipc.Client`
|
||||
- Spam/promotions/social are silently dropped (Gmail `X-GM-LABELS` or IMAP keyword check)
|
||||
- Integration test with a throwaway mailbox verifies the round-trip
|
||||
|
||||
**Scope:**
|
||||
- New `internal/email/` package — IMAP client (stdlib `net/mail` + `github.com/emersion/go-imap` or raw `net/textproto`)
|
||||
- Config extension: `config.Config.Email` block with `PollInterval`, `Accounts[]` (host, username, password/app-password, folders)
|
||||
- LLM extraction prompt (shared `internal/llm.Client`) classifies each unseen message as `task|reminder|event|fact|junk` and fills slots — same GBNF-constrained pattern as `internal/router/llmrouter.go:routeGrammar`
|
||||
- Extracted items written through `ipc.CoreAPI` interface (same seam `internal/voice/reactiveHandler` uses)
|
||||
- Daemon lifecycle: wired in `cmd/mavend/main.go` alongside the tick loop, starts after unlock
|
||||
- `deploy/mavend.json` gets the email block
|
||||
|
||||
**Steps:**
|
||||
1. Add `EmailConfig` struct to `internal/config/config.go` (accounts, poll interval, folders to scan)
|
||||
2. Create `internal/email/` with `Poller` — IMAP connect, `IDLE` for push, fallback to periodic `SELECT INBOX` + `SEARCH UNSEEN`
|
||||
3. Implement message → plaintext conversion (`text/plain` body, strip HTML from `text/html` via `golang.org/x/net/html` or simple regex)
|
||||
4. Add LLM extraction via `internal/llm.Client.Complete()` with a GBNF grammar that outputs `{"type":"task|reminder|event|fact","key":"...","value":"...","when":"..."}` — mirroring `internal/router/llmrouter.go:routeGrammar`
|
||||
5. Map extracted items to `ipc.WriteFactReq` / `ipc.CreateReminder` / `ipc.WriteNote` calls
|
||||
6. Wire `Poller` into `cmd/mavend/main.go` — start on unlock, stop on context cancel
|
||||
7. Test with a throwaway mailbox (Gmail app password or a self-hosted Dovecot); verify extraction accuracy
|
||||
8. Add `models/seeds/email_*.txt` seed examples for the extraction classifier
|
||||
9. Log poll stats (seen, extracted, junk, errors) through `log.Printf` (consistent with existing pattern in `cmd/mavend/tick.go`)
|
||||
@@ -0,0 +1,28 @@
|
||||
# Plan: Background Pattern Detection & Proactive Proposal
|
||||
|
||||
**Goal:** Maven continuously monitors facts written to the store, detects recurring patterns (behavioral, operational, temporal), and proposes new routines, reminders, or config changes back to the user via the existing `ProposedRoutine` / tool-proposal mechanism.
|
||||
|
||||
**Done when:**
|
||||
- Background analyzer reads `RecentFacts` and `Events` on a slow cadence (every 10min, not every tick)
|
||||
- New patterns beyond the existing action-lexicon `internal/pattern/extractor.go` — e.g. time-of-day correlations, service-down + user-awake sequences, repeated tool invocations
|
||||
- Proposals land as `ProposedRoutine` rows in the store (reusing `store.CreateProposedRoutine`) or `ProposeTool` calls
|
||||
- User confirms via voice or mavweb — existing `pendingRoutineConfirm` / `resolveConfirm` path re-used
|
||||
- False-positive rate measured and acceptable (proposals are parked, never auto-enabled)
|
||||
|
||||
**Scope:**
|
||||
- New `internal/analyzer/` package — runs on a separate ticker (10-30min), reads from `ipc.CoreAPI` (or direct `store.Store` for performance)
|
||||
- Extends `internal/pattern/` detector with multi-signal patterns (not just action+object interval)
|
||||
- Reuses existing `store.Events`, `store.ProposedRoutine`, `store.Tools` tables
|
||||
- Daemon wiring: new `analyzer` goroutine in `cmd/mavend/main.go` alongside tick loop
|
||||
|
||||
**Steps:**
|
||||
1. Create `internal/analyzer/` with periodic scan — `Analyzer.Run(ctx)` goroutine
|
||||
2. Implement new detectors:
|
||||
- Time-of-day behavioral patterns ("you always water plants at 9pm" → remind at 8:45pm)
|
||||
- Service-down + user-present correlation ("service X died while you were away, you fixed it when you sat down" → propose auto-remediation tool)
|
||||
- Repeated tool use patterns ("you restart plex every Tuesday" → propose a routine)
|
||||
3. Register proposals via existing `store.CreateProposedRoutine` (reuses `internal/voice/reactiveHandler:detectPattern` pattern) and `ipc.ProposeTool`
|
||||
4. Wire analyzer goroutine in `cmd/mavend/main.go` — starts after unlock alongside `tl.run(ctx)`
|
||||
5. Add `analyzer_interval` to `config.Config` (default 10min)
|
||||
6. Add voice confirm path — proposals detected in background trigger a nudge with a yes/no prompt, reusing `pendingRoutineConfirm` from `cmd/mavend/voice.go:resolveConfirm`
|
||||
7. Test with seeded events in `internal/store/events_test.go` pattern
|
||||
@@ -0,0 +1,27 @@
|
||||
# Plan: Background Memory Evaluation & Idea Generation
|
||||
|
||||
**Goal:** Maven periodically reviews her own memory stores (facts, notes, events, nudges), evaluates coherence and gaps, and generates proactive proposals — new routines, configuration tweaks, observations she can share with the user.
|
||||
|
||||
**Done when:**
|
||||
- `internal/memory/eval.go` — periodic evaluation loop runs on a slow cadence (1h)
|
||||
- Evaluation reads `RecentFacts`, `RecentNotes`, `RecentNudges`, `RecentEvents` via `store.Store` or `ipc.CoreAPI`
|
||||
- LLM summarizes state, detects anomalies (e.g. "you haven't recorded a meal in 3 days — is your routine broken?"), proposes new care rules
|
||||
- Generated proposals are written as notes (kind `note`, source `infer:memory-eval`) and/or trigger nudges through the dispatcher
|
||||
- Evaluation trace visible on `/history` page in mavweb
|
||||
|
||||
**Scope:**
|
||||
- New `internal/memory/eval.go` — evaluator struct calling `internal/llm.Client` with a summarization prompt
|
||||
- Reuses `internal/delivery.Dispatcher` for surfacing insights as care nudges (sev1)
|
||||
- Reuses `internal/store` for reading memory state and writing evaluation notes
|
||||
- Daemon wiring: new evaluation goroutine in `cmd/mavend/main.go`
|
||||
- Config: `memory_eval_interval` in `config.Config` (default 1h, 0 to disable)
|
||||
|
||||
**Steps:**
|
||||
1. Create `internal/memory/eval.go` — `Evaluator` struct holding `*store.Store`, `*llm.Client`, `*delivery.Dispatcher`
|
||||
2. Implement `Evaluate(ctx)` — reads last N facts, notes, nudges, events, builds a prompt summarizing patterns, anomalies, gaps
|
||||
3. LLM call returns structured observations: `{"observation":"...","confidence":0.8,"suggested_action":"remind|propose|notify"}`
|
||||
4. High-confidence observations written as notes (`source:infer:memory-eval`) or dispatched as care nudges (sev1) through `dispatcher.DispatchNudge`
|
||||
5. Wire evaluator goroutine in `cmd/mavend/main.go` — separate ticker, not on the main tick loop
|
||||
6. Add `/eval` API method to `ipc.CoreAPI` (or reuse `Chat` with system context) so mavweb can show evaluation history
|
||||
7. Add `memory_eval` block to `deploy/mavend.json`
|
||||
8. Test with synthetic store state — verify observations match expected patterns
|
||||
@@ -0,0 +1,29 @@
|
||||
# Plan: Self-Update with Rollback
|
||||
|
||||
**Goal:** Maven can update her own code, config, skills (seed files), tool definitions, and integrations while running, with a rollback mechanism if the new state causes failures.
|
||||
|
||||
**Done when:**
|
||||
- `internal/update/` package manages versioned snapshots of the binary + config + models + seeds
|
||||
- Daemon can fetch a new release artifact (git pull + `go build`, or download a pre-built binary)
|
||||
- On success: atomically swaps binaries/symlinks and sends SIGHUP to itself for graceful reload
|
||||
- On failure (daemon crash within a grace window): init/systemd restarts the old binary automatically, or an in-process supervisor detects crash-loop and rolls back
|
||||
- Rollback is automatic on crash-loop detection (>2 crashes in 5min) — previous known-good snapshot is re-deployed
|
||||
- All state (sqlite store) is forward/backward compatible within the same schema version (`store.Migrate`)
|
||||
|
||||
**Scope:**
|
||||
- New `internal/update/` package — snapshot manager, downloader, binary swap, health check
|
||||
- New `cmd/mavend/updater.go` — the imperative orchestration (swap + SIGHUP + watch)
|
||||
- Reuses `internal/store.Migrate` for schema compatibility
|
||||
- Config: `update` block in `config.Config` (repo URL, auto-update channel, rollback max crashes)
|
||||
- New IPC methods: `MethodCheckUpdate`, `MethodApplyUpdate`, `MethodRollback`
|
||||
|
||||
**Steps:**
|
||||
1. Design the update data model: versioned snapshots under `state_dir/updates/v<N>/` — binary, config, models, seeds; current symlink at `state_dir/current`
|
||||
2. Create `internal/update/checker.go` — checks GitHub releases (or a custom update server) for newer version; compares semver
|
||||
3. Create `internal/update/downloader.go` — downloads artifact, verifies checksum, extracts to new snapshot dir
|
||||
4. Create `internal/update/swapper.go` — atomically swaps symlink, sends SIGHUP to self (`syscall.SIGUSR1` or `SIGHUP`)
|
||||
5. Wire SIGHUP handler in `cmd/mavend/main.go` (already has `signal.NotifyContext` with `SIGHUP`) — re-read config, re-open store, swap phraser/router/delivery without dropping IPC connections
|
||||
6. Create crash-loop detector in `internal/update/health.go` — watches process start time, counts crashes in window, triggers rollback
|
||||
7. Add IPC methods `MethodCheckUpdate`, `MethodApplyUpdate`, `MethodRollback` to `internal/ipc/api.go` and wire through `ipc.Server` dispatch
|
||||
8. Add `update` block to `config.Config` and `deploy/mavend.json`
|
||||
9. Test rollback: deploy a deliberately broken binary, verify crash-loop detection reverts to previous version
|
||||
@@ -0,0 +1,29 @@
|
||||
# Plan: On-the-Fly Model Swap (Multi-Model Routing)
|
||||
|
||||
**Goal:** Maven can switch between LLM models at runtime — including using a remote `llama-server` instance on workpc over LAN — without restarting the daemon. The phraser, router (LLM router), and replier all point at a dynamic backend that can be re-pointed via IPC.
|
||||
|
||||
**Done when:**
|
||||
- `internal/llm/client.go` supports a dynamic base URL that can be swapped at runtime
|
||||
- `internal/phraser/llmphraser.go` can hot-swap its backend (stop current `llama-server` subprocess, start new one, or point to a remote one)
|
||||
- Remote model config: `phraser.mode = "remote"` with `remote_url = "http://workpc:8080"` — connects without spawning a subprocess
|
||||
- Swap is triggered via IPC (`MethodSwapModel`) with a new config block — no daemon restart
|
||||
- Router's `LLMRouter` (in `internal/router/llmrouter.go`) follows the same swap
|
||||
- Fallback: if the new model fails to respond within timeout, the old model stays active (never leave the user with no model)
|
||||
|
||||
**Scope:**
|
||||
- `internal/llm/client.go` — add `SetBaseURL(string)` method for runtime re-pointing
|
||||
- `internal/phraser/llmphraser.go` — add `Swap(Config) error` method
|
||||
- `internal/router/llmrouter.go` — already holds a `Completer` interface; swap the underlying client
|
||||
- `cmd/mavend/voice.go` — re-creates `LLMReplier` when model changes
|
||||
- New IPC method `MethodSwapModel` in `internal/ipc/api.go`
|
||||
- Config: `phraser.mode` field (`local|remote`), `phraser.remote_url`
|
||||
|
||||
**Steps:**
|
||||
1. Add `SetBaseURL(url string)` to `internal/llm/client.go` — atomically swaps the `base` field under a mutex (add `sync.RWMutex` to `Client`)
|
||||
2. Add `Swap(cfg Config) error` to `internal/phraser/llmphraser.go` — stops current `llama-server` (via `Close()`), starts new one with new config, or connects to remote URL without spawning
|
||||
3. Extend `PhraserConfig` in `internal/config/config.go` with `Mode string` (`"local"` or `"remote"`) and `RemoteURL string`
|
||||
4. Create new `internal/llm/manager.go` — manages a set of named backends, allows `SwitchModel(name)` that re-wires phraser + LLM router + replier atomically
|
||||
5. Add `MethodSwapModel` to `internal/ipc/api.go` with request `{model_path, mode, remote_url, n_gpu_layers, n_ctx}`
|
||||
6. Wire swap handler in `cmd/mavend/main.go` — `srv.ModelSwapFn` called from IPC dispatch, re-wires phraser, rebuilds router with new LLMRouter, rebuilds replier
|
||||
7. Add `model` block to `config.Config` with named model definitions (local paths + remote URLs)
|
||||
8. Test: swap between local `StubPhraser` and remote llama-server on LAN; verify phraser + router + replier all use the new backend
|
||||
@@ -0,0 +1,29 @@
|
||||
# Plan: MCP (Model Context Protocol) Integration
|
||||
|
||||
**Goal:** Maven acts as an MCP host — she can connect to external MCP servers (tools, data sources, file systems) via the Model Context Protocol, and expose those capabilities as tool verbs in her allowlist, or use them as context sources for routing/phrasing.
|
||||
|
||||
**Done when:**
|
||||
- `internal/mcp/` package implements an MCP client — connects to MCP servers over stdio or TCP, negotiates protocol version, discovers tools/resources
|
||||
- MCP-discovered tools are registered as proposed tools (`ProposeTool`) in the store
|
||||
- User enables them on mavweb (existing `EnableTool` flow)
|
||||
- MCP resource contents are available as context for the router's `LLMRouter` or the phraser's prompts
|
||||
- Multiple MCP servers can be configured in `deploy/mavend.json`
|
||||
- MCP connections are managed (reconnect on drop, timeout, graceful shutdown)
|
||||
|
||||
**Scope:**
|
||||
- New `internal/mcp/` package — protocol client (`github.com/mark3labs/mcp-go` or raw JSON-RPC over stdio/TCP)
|
||||
- New `cmd/mcphost/` or built into `cmd/mavend/` — MCP server manager goroutine
|
||||
- Config extension: `mcp_servers` array in `config.Config` — `{name, command, args, env, enabled}`
|
||||
- IPC extension: `MethodListMCPTools`, `MethodListMCPResources`, `MethodCallMCPTool`
|
||||
- Reuses `internal/tool/Executor` and `internal/router` seams
|
||||
|
||||
**Steps:**
|
||||
1. Research MCP protocol spec and pick a Go client library (`github.com/mark3labs/mcp-go` exists as of 2025, or implement raw JSON-RPC 2.0 over stdio/TCP)
|
||||
2. Create `internal/mcp/client.go` — `Client` struct handling stdio subprocess lifecycle: spawn, negotiate protocol version (`initialize` handshake), `ListTools`, `CallTool`
|
||||
3. Create `internal/mcp/manager.go` — `Manager` that reads `mcp_servers` config, starts/stops per-server clients, exposes a consolidated tool list
|
||||
4. Wire MCP-discovered tools into `ProposeTool` (same flow as `cmd/mavend/voice.go:proposeGap`)
|
||||
5. Add resource content fetching — `ReadResource` returns text content that can be injected into LLM prompts
|
||||
6. Add IPC methods `MethodListMCPTools`, `MethodListMCPResources`, `MethodCallMCPTool` to `internal/ipc/api.go`
|
||||
7. Expose MCP tools in mavweb UI — new `/tools/mcp` page
|
||||
8. Add `mcp_servers` block to `config.Config` and `deploy/mavend.json`
|
||||
9. Test with a local MCP demo server (e.g., `mcp-server-sqlite` or a custom echo server)
|
||||
@@ -0,0 +1,27 @@
|
||||
# Plan: Vision — Image Understanding Capability
|
||||
|
||||
**Goal:** Maven can "see" — accept images (from mavweb upload, Telegram, or filesystem paths), run vision inference via a local or remote multimodal model, and answer questions about the image content or extract structured information.
|
||||
|
||||
**Done when:**
|
||||
- Vision model backend is configurable: local multimodal LLM (e.g., LLaVA, Qwen-VL via `llama-server` mmproj) or remote API
|
||||
- `internal/vision/` package handles image preprocessing, model inference, result parsing
|
||||
- Voice/text commands like "что на картинке?" or "прочитай текст с экрана" route to the vision handler
|
||||
- Extracted information can be written as facts/notes through `ipc.CoreAPI`
|
||||
- Telegram image messages are processed through the same pipeline
|
||||
|
||||
**Scope:**
|
||||
- New `internal/vision/` package — image loader (Go stdlib `image` + `golang.org/x/image`), inference client
|
||||
- New config block: `voice.vision` in `config.Config` — `{enabled, provider, model_path, mmproj_path, remote_url}`
|
||||
- Router intent extension: new `IntentVision` or reuse `IntentQuery` with a vision flag
|
||||
- Reuses `internal/llm.Client` for API-compatible backends (OpenAI-compatible vision API)
|
||||
- Reuses `internal/ipc.CoreAPI` for writing extracted data
|
||||
|
||||
**Steps:**
|
||||
1. Create `internal/vision/provider.go` — `Provider` interface with `Describe(image []byte, prompt string) (string, error)` and `ExtractText(image []byte) (string, error)`
|
||||
2. Implement `LocalProvider` — spawns `llama-server` with mmproj, sends multimodal chat completion requests
|
||||
3. Implement `RemoteProvider` — calls an OpenAI-compatible vision API endpoint, reuses `internal/llm.Client`
|
||||
4. Create `internal/vision/processor.go` — image preprocessing (resize, format conversion to JPEG/PNG, base64 encoding)
|
||||
5. Wire vision into `cmd/mavend/voice.go:reactiveHandler` — detect vision intent from router (new `IntentVision` or a `Slots.HasImage` flag)
|
||||
6. Add IPC method `MethodDescribeImage` for programmatic access (mavweb upload, telegram bot)
|
||||
7. Add vision config block to `config.Config` and wire in `cmd/mavend/main.go`
|
||||
8. Test with a local multimodal model: send an image via mavweb, verify description and text extraction
|
||||
@@ -0,0 +1,28 @@
|
||||
# Plan: Hearing — Audio Stream Monitoring & Meeting Summarization
|
||||
|
||||
**Goal:** Maven can "hear" ambient audio from workpc — microphone input during meetings, system audio — and on demand (or on trigger) produce transcripts, summaries, or extract action items. A typical use case: "Maven, запиши встречу" starts capture, "хватит" stops it, and Maven writes a summary note.
|
||||
|
||||
**Done when:**
|
||||
- `internal/audio/capture.go` — remote microphone capture client (receives PCM stream from workpc over WebSocket or the existing voice TCP protocol)
|
||||
- `internal/stt/` — streaming transcription (uses existing `stt.Transcriber` interface, extended with streaming support)
|
||||
- Meeting capture triggered by voice command (IntentCapture) or configurable keyword ("maven record")
|
||||
- Raw audio is either streamed to STT in real-time or saved to a WAV file and transcribed after capture ends
|
||||
- Transcription + LLM summary is written as a note (`source:capture:meeting`) through `ipc.CoreAPI`
|
||||
- New `mavheary` module (`cmd/mavheard/`) — the workpc-side agent that captures mic/speaker audio and streams it to mavend
|
||||
|
||||
**Scope:**
|
||||
- New `cmd/mavheard/` — workpc-side agent: captures microphone (PortAudio or ALSA `arecord`), streams over WebSocket to mavend
|
||||
- `internal/audio/` extended with capture types: `MicCapture`, `SystemCapture`, `FileCapture`
|
||||
- `internal/stt/stt.go` extended with `StreamingTranscriber` interface (or reuse existing with chunked input)
|
||||
- Router: new `IntentCapture` intent for start/stop commands
|
||||
- Reuses `internal/llm.Client` for summarization
|
||||
- Reuses `internal/voice/server.go` TCP protocol for streaming audio
|
||||
|
||||
**Steps:**
|
||||
1. Create `cmd/mavheard/main.go` — workpc-side daemon: captures microphone via `arecord` pipe or PortAudio, opens WebSocket or TCP connection to mavend, streams PCM frames
|
||||
2. Create `internal/audio/capture.go` — `Capture` interface: `Start()`, `Stop()`, `AudioCh <-chan Audio`; implement `MicCapture` (reads from `mavheard` stream) and `FileCapture` (reads WAV)
|
||||
3. Extend `internal/stt/stt.go` — add `TranscribeStream(ctx, audio <-chan Audio) (string, error)` to `Transcriber` interface; `Stub` returns empty; `Remote` forwards chunks to worker socket
|
||||
4. Add `IntentCapture` to `internal/router/intent.go` — slots: `Action` ("start"/"stop"/"status"), `Duration`
|
||||
5. Wire capture handler in `cmd/mavend/voice.go:reactiveHandler` — start = spawn goroutine receiving audio, stream to STT; stop = finalize, send to LLM for summarization, write note via `WriteNote`
|
||||
6. Add capture config to `voice` block in `config.Config` — `{capture_enabled, capture_timeout}`
|
||||
7. Test with a recorded WAV file — simulate a meeting, verify transcription + summary note is created
|
||||
@@ -0,0 +1,29 @@
|
||||
# Plan: Behavioral Memory — How I Do Stuff
|
||||
|
||||
**Goal:** Maven builds a rich behavioral model of the user over time: habits, routines, preferences, recurring tasks, deadlines, and commitments. She uses this model to proactively propose plans, surface reminders, and adjust her behavior — all grounded in the existing fact/event/note stores.
|
||||
|
||||
**Done when:**
|
||||
- `internal/memory/behavior.go` — behavioral model builder reads facts, events, notes, nudges, reminders, tools, calendar events
|
||||
- Model is exposed as a structured profile: `{"routines": [...], "preferences": {...}, "recurring_tasks": [...], "typical_schedule": {...}}`
|
||||
- LLM generates this profile periodically (daily) and stores it as a note/fact
|
||||
- Proactive loop uses the profile to propose daily plans: "сегодня ты обычно делаешь X, Y, Z. напомнить?"
|
||||
- Profile is queryable via voice: "что я обычно делаю по вторникам?"
|
||||
- Profile updates on fact write — not just periodic — so a new "walk" fact immediately adjusts the walking schedule
|
||||
|
||||
**Scope:**
|
||||
- `internal/memory/behavior.go` — behavior builder
|
||||
- `internal/memory/profile.go` — profile data structures (routines, preferences, schedule, commitments)
|
||||
- Reuses `internal/llm.Client` for profile generation
|
||||
- Reuses `internal/pattern/detector.go` for interval detection on behavioral data
|
||||
- Extends `internal/router/intent.go` — `IntentQuery` extended with behavioral sub-queries
|
||||
- Reuses `internal/delivery.Dispatcher` for surfacing proposals as nudges
|
||||
|
||||
**Steps:**
|
||||
1. Design profile schema: `BehaviorProfile` struct with `Routines []Routine`, `Preferences map[string]string`, `RecurringTasks []Task`, `WeeklySchedule map[string][]Activity`
|
||||
2. Create `internal/memory/behavior.go` — `Builder` that reads `store.RecentFacts(1000)`, `store.EventsFor` (all action+object combos), `store.RecentNotes(500)`, `store.ListReminders`
|
||||
3. Implement profile generation — prompt for `llm.Client` that takes raw facts and outputs a structured JSON profile; stores result as a fact (`kind=config, key=behavior_profile`)
|
||||
4. Create `internal/memory/planner.go` — reads the profile each morning (via routine cron `"0 8 * * *"`) and proposes a daily plan through `dispatcher.DispatchNudge`
|
||||
5. Add real-time updates — when a fact is written via `WriteFact`, the behavior builder incrementally updates the relevant profile section (append-only, no full rebuild)
|
||||
6. Wire voice query — `"что я обычно делаю?"` routes to `IntentQuery` → behavior profile lookup → LLM-phrased answer
|
||||
7. Add IPC read method `MethodGetBehaviorProfile` so mavweb can display it on `/dash`
|
||||
8. Test with synthetic fact history — verify weekly schedule is correctly inferred
|
||||
@@ -0,0 +1,27 @@
|
||||
# Plan: Speaker Recognition
|
||||
|
||||
**Goal:** Maven can distinguish between different speakers on the voice channel — recognize known voices (the user, family members) and tag facts/notes/transcripts with a speaker identity.
|
||||
|
||||
**Done when:**
|
||||
- Speaker embedding extractor (e.g., ECAPA-TDNN or a simple MFCC + GMM) runs on incoming voice PCM before STT
|
||||
- Embedding is compared against enrolled speaker profiles (stored as vectors in the `memory_vectors` table alongside semantic memory)
|
||||
- Unknown speakers are enrolled on first interaction (prompt: "кто это?")
|
||||
- All voice fact/note writes are tagged with `speaker:<id>` in the value/source metadata
|
||||
- Speaker identity is available as context to the router, phraser, and replier ("ok, <name>")
|
||||
|
||||
**Scope:**
|
||||
- New `internal/speaker/` package — enrollment, recognition, embedding extraction
|
||||
- Reuses `internal/store.MemoryStore` for speaker vector storage (same `memory_vectors` table, different `source` prefix)
|
||||
- Reuses `internal/audio` for PCM preprocessing
|
||||
- Integration point: `cmd/mavend/voice.go:HandlePushToTalk` — speaker ID extracted before STT, passed through context
|
||||
|
||||
**Steps:**
|
||||
1. Research speaker embedding approaches — simplest floor: MFCC + cosine similarity via `github.com/mjibson/go-dsp` or a pre-trained ONNX model (SpeechBrain ECAPA)
|
||||
2. Create `internal/speaker/recognizer.go` — `Recognizer` interface: `Identify(pcm []float32) (SpeakerID, confidence)`, `Enroll(id, pcm)`
|
||||
3. Create `internal/speaker/store.go` — speaker profile CRUD via `store.MemoryStore`: `Insert("speaker:<id>", embedding, meta)`, `Search(embedding, k)`
|
||||
4. Create `internal/speaker/enroll.go` — enrollment flow: capture N seconds of audio, extract embedding, prompt for name via TTS + STT round-trip
|
||||
5. Wire into `cmd/mavend/voice.go:HandlePushToTalk` — run speaker ID on the PCM before STT; pass speaker ID through `context.Context` to `applyAction`
|
||||
6. Tag all voice-written facts/notes with speaker ID — `Source` becomes `tap:voice:speaker:<id>` or metadata field
|
||||
7. Add IPC methods `MethodEnrollSpeaker`, `MethodListSpeakers`, `MethodRemoveSpeaker`
|
||||
8. Add speaker config block to `voice` in `config.Config` — `{speaker_recognition: true, model_path}`
|
||||
9. Test with 2+ recorded voice samples — verify correct identification and rejection of unknown speakers
|
||||
@@ -0,0 +1,28 @@
|
||||
# Plan: SmartHome Ecosystem Integration
|
||||
|
||||
**Goal:** Maven connects to the SmartHome ecosystem — Home Assistant, MQTT, Zigbee2MQTT, or direct HTTP APIs — to read sensor states, control devices, and trigger automations based on facts and presence.
|
||||
|
||||
**Done when:**
|
||||
- MQTT client in `internal/smarthome/mqtt.go` — connects to broker, subscribes to topic patterns, publishes control messages
|
||||
- Home Assistant API client in `internal/smarthome/ha.go` — REST API + WebSocket for state reads and service calls
|
||||
- SmartHome entities are writable as facts (sensor → fact write for loop predicates)
|
||||
- Tool verbs for device control: `turn_on`, `turn_off`, `set_temp`, `set_brightness` — mapped to existing `tool.Executor` or the SmartHome API directly
|
||||
- Presence integration: motion sensors, door sensors, WiFi presence feed into the existing `store.PresenceProbes` pipeline
|
||||
- Voice control: "maven, выключи свет в гостиной" routes through `IntentAct` → SmartHome tool
|
||||
|
||||
**Scope:**
|
||||
- New `internal/smarthome/` package — MQTT client, Home Assistant client, entity registry
|
||||
- New `cmd/mavpoll/` extension — existing polling infrastructure extended with SmartHome sensors
|
||||
- Config: `smarthome` block in `config.Config` — `{provider: "homeassistant|mqtt", url, token, mqtt_broker}`
|
||||
- Reuses `internal/tool.Executor` for device control tools
|
||||
- Reuses `internal/store.PresenceProbes` for presence input
|
||||
|
||||
**Steps:**
|
||||
1. Create `internal/smarthome/ha.go` — Home Assistant REST client: `GetStates()`, `CallService(domain, service, target, data)`, subscribe to WebSocket events
|
||||
2. Create `internal/smarthome/mqtt.go` — MQTT client via `github.com/eclipse/paho.mqtt.golang`: subscribe to `zigbee2mqtt/#`, `homeassistant/#`, publish to `cmnd/#`
|
||||
3. Create `internal/smarthome/entity.go` — entity registry: maps entity_id → fact key, device_class → fact kind
|
||||
4. Create programmatic tool registration — on startup, enumerate SmartHome entities and call `ProposeTool` for each controllable device (light, switch, climate, cover)
|
||||
5. Wire MQTT sensor updates into `store.WriteFact` — e.g., `zigbee2mqtt/temperature` → `WriteFact(kind="env", key="temp:living_room", value="22.5")`
|
||||
6. Wire SmartHome presence signals into `store.PresenceProbes` — e.g., WiFi presence or motion sensor → `presence_wifi` / `presence_motion` probes
|
||||
7. Add `smarthome` block to `config.Config` and wire into `cmd/mavend/main.go` — starts separate goroutine for MQTT/WebSocket event loop
|
||||
8. Test with a local MQTT broker and simulated sensor messages — verify facts are written and loop predicates can read them
|
||||
@@ -0,0 +1,27 @@
|
||||
# Plan: Bluetooth Control & Network Scanning
|
||||
|
||||
**Goal:** Maven can scan Bluetooth devices (discover, connect, read characteristics) and scan the local network (discover hosts, open ports, service fingerprints) — exposed as tools in her allowlist for query and automation.
|
||||
|
||||
**Done when:**
|
||||
- `internal/bluetooth/` package wraps `bluez` D-Bus API or `hcitool`/`bluetoothctl` CLI for device discovery, pairing, and RSSI reading
|
||||
- `internal/netscan/` package performs ARP scan, TCP port scan, service detection
|
||||
- Both are wired as tools (same `ProposeTool`/`EnableTool` flow, status/proposed files in `deploy/mavend.json` or generated at runtime)
|
||||
- Results are writable as facts/key-value observations for loop predicates
|
||||
- Voice commands: "maven, просканируй bluetooth", "какие устройства в сети?" route through `IntentAct` or `IntentQuery`
|
||||
|
||||
**Scope:**
|
||||
- New `internal/bluetooth/` — BlueZ D-Bus client (`github.com/godbus/dbus/v5` or exec wrappers)
|
||||
- New `internal/netscan/` — ARP scanner (`net/http` + arp table read), TCP connect scanner (`net.DialTimeout`), service probe
|
||||
- Config: `tools` block extended with auto-generated scan tool entries
|
||||
- Reuses `internal/tool.Executor` for running scan commands
|
||||
- Reuses `internal/store.WriteFact` for scan results as facts
|
||||
|
||||
**Steps:**
|
||||
1. Create `internal/bluetooth/scanner.go` — `Scan(duration) ([]Device, error)`: calls `bluetoothctl --timeout N scan on`, parses output; or uses D-Bus `org.bluez` API
|
||||
2. Create `internal/bluetooth/presence.go` — RSSI-based presence probe: scan for known MAC, write `presence_bt:<name>` fact with RSSI value for the presence pipeline (`internal/store/presence.go`)
|
||||
3. Create `internal/netscan/scanner.go` — `ScanLAN() ([]Host, error)`: reads ARP table (`/proc/net/arp`), TCP scans common ports (22, 80, 443, 8080, 9090, 9100) with `net.DialTimeout`
|
||||
4. Create `internal/netscan/service.go` — service probes: HTTP GET on port 80/8080, SSH banner grab on 22, ping
|
||||
5. Wire scan tools into `ProposeTool` at startup — `bluetooth_scan`, `wifi_scan`, `port_scan`, `network_map`
|
||||
6. Presence integration: periodic BT scan writes `presence_bt:<device>` facts; `internal/store/presence.go:PresenceSignals` reads them alongside WiFi probes
|
||||
7. Add `bluetooth` and `netscan` config blocks to `config.Config` — `{scan_interval, known_devices, scan_timeout}`
|
||||
8. Test with local network — verify host discovery matches `nmap` output; verify BT scan discovers known devices
|
||||
@@ -0,0 +1,29 @@
|
||||
# Plan: RSS / News Feed Reader
|
||||
|
||||
**Goal:** Maven periodically polls configured RSS/Atom feeds, extracts new items, classifies them by relevance using the embedder, and either stores interesting items as notes or surfaces breaking news via the dispatcher.
|
||||
|
||||
**Done when:**
|
||||
- `internal/rss/` package — feed parser (Go stdlib `encoding/xml` or `github.com/mmcdole/gofeed`), poll loop
|
||||
- Feed config in `config.Config` — `feeds: [{name, url, category, poll_interval}]`
|
||||
- New items are classified by the existing `router.Embedder` — items below a relevance threshold are silently dropped
|
||||
- Interesting items: written as `WriteNote` with `source="rss:<feed_name>"`; breaking/high-severity items dispatched as care nudges (sev2-3)
|
||||
- Queryable via voice: "что нового в лентах?", "что по технологиям?"
|
||||
|
||||
**Scope:**
|
||||
- New `internal/rss/` package — periodic poller, feed parser, item classifier
|
||||
- Config extension: `feeds` array in `config.Config`
|
||||
- Reuses `internal/router.Embedder` for relevance scoring (same ONNX model or HashEmbedder floor)
|
||||
- Reuses `internal/ipc.CoreAPI` for writing notes
|
||||
- Reuses `internal/delivery.Dispatcher` for breaking news nudges
|
||||
- Reuses `internal/llm.Client` for item summarization (optional, for long articles)
|
||||
|
||||
**Steps:**
|
||||
1. Add `gofeed` or raw XML parser — `internal/rss/feed.go`: `ParseFeed(url) ([]Item, error)` where `Item{Title, Link, Summary, Published, Content}`
|
||||
2. Create `internal/rss/poller.go` — `Poller` holds `[]FeedConfig`, polls each on its own interval, tracks `last_poll` per feed via a fact (`kind=config, key=rss:poll:<name>`)
|
||||
3. Implement relevance classifier — embed each item's title+summary with `router.Embedder`, compare against user interest profile (built from notes/facts), drop items below threshold
|
||||
4. Interesting items → `ipc.WriteNote(ctx, ts, title+"\n"+summary, embedding, "rss:<feed_name>")`
|
||||
5. Breaking items (keywords: "CVE", "outage", "critical") → `ipc.WriteFact(kind=env, key=news:breaking, value=title)` → loop rule `BreakingNewsRule` (sev3) dispatches nudge
|
||||
6. Wire poller into `cmd/mavend/main.go` — starts after unlock, separate goroutine
|
||||
7. Add voice query handler — `"что нового?"` queries `RecentNotes` filtered by source prefix `rss:` and phrases via `phraser.PhraseQuery`
|
||||
8. Add `feeds` block to `config.Config` and `deploy/mavend.json`
|
||||
9. Test with a live RSS feed (e.g., `https://news.ycombinator.com/rss`) — verify items appear in notes table
|
||||
@@ -0,0 +1,30 @@
|
||||
# Plan: Web Crawler
|
||||
|
||||
**Goal:** Maven can crawl web pages on demand or on a schedule — fetch page content, extract structured data (via LLM or CSS selectors), and store results as facts, notes, or reminders. Used for: price monitoring, documentation updates, recipe extraction, content summarization.
|
||||
|
||||
**Done when:**
|
||||
- `internal/crawl/` package — HTTP fetcher with polite defaults (rate limiting, robots.txt respect, user-agent)
|
||||
- Content extraction: HTML→plaintext (Go stdlib `golang.org/x/net/html`), or full-page LLM summarization
|
||||
- Crawl scheduler in config — `crawls: [{name, url, selector, schedule, store_as}]`
|
||||
- On-demand crawl via voice: "maven, посмотри страницу X и запиши цену"
|
||||
- Results are stored as facts/notes through `ipc.CoreAPI`
|
||||
- Crawl history visible on mavweb `/tools` page
|
||||
|
||||
**Scope:**
|
||||
- New `internal/crawl/` package — fetcher, parser, scheduler, extractor
|
||||
- Config extension: `crawls` array in `config.Config`
|
||||
- Reuses `internal/llm.Client` for intelligent extraction (e.g., "extract the price, description, and availability from this page")
|
||||
- Reuses `internal/ipc.CoreAPI` for storing results
|
||||
- Reuses `internal/router.Embedder` for deduplication (don't re-store identical content)
|
||||
- Reuses `internal/routine.Routine` mechanics for scheduled crawls
|
||||
|
||||
**Steps:**
|
||||
1. Create `internal/crawl/fetcher.go` — `Fetch(url) ([]byte, error)`: HTTP GET with timeout (30s), rate limiting (1 req/sec), `robots.txt` check via `github.com/temoto/robotstxt`
|
||||
2. Create `internal/crawl/extractor.go` — `Extract(html []byte, extraction_type string) (map[string]string, error)`: for simple extraction use CSS selector (`github.com/PuerkitoBio/goquery`); for complex extraction use `llm.Client` with a prompt
|
||||
3. Create `internal/crawl/scheduler.go` — `Scheduler` that reads `crawls` config, runs each on its cron schedule, tracks last-run via facts
|
||||
4. Create `internal/crawl/dedup.go` — compute content hash, skip if identical to last fetched (stored as fact `kind=config, key=crawl:hash:<name>`)
|
||||
5. Wire on-demand crawl into `IntentAct` — new tool verb `crawl` that accepts a URL argument
|
||||
6. Wire scheduled crawls into `cmd/mavend/main.go` — separate goroutine manages the crawl scheduler
|
||||
7. Add IPC methods `MethodTriggerCrawl(name)`, `MethodListCrawls`, `MethodGetCrawlResult(name)`
|
||||
8. Add `crawls` block to `config.Config` and `deploy/mavend.json`
|
||||
9. Test with a static HTML page — verify extraction matches expected values, verify scheduling fires correctly
|
||||
@@ -1,5 +1,11 @@
|
||||
# Plan — Sub-project 1: Router-as-LFM + Foundation
|
||||
|
||||
> **Historical/completed foundation.** The shared llama-server client, router,
|
||||
> fallback and replier described here were implemented. The resident-model
|
||||
> decision changed on 2026-07-18 from LFM to locally trained Qwen3-1.7B. Do not
|
||||
> use the embedded LFM model paths or old single-object examples as current ops
|
||||
> guidance; see `2026-07-18-qwen3-resident-training-eval.md`.
|
||||
|
||||
> Scope from `REARCH.md`. Make Maven trustworthy: the LFM becomes the router
|
||||
> (fixes "messes up queries" / "doesn't take notes"), the engine actually runs
|
||||
> (fixes stub replies), dates stop being read as "number dot number dot number",
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Route-training data plan (LLM-as-router)
|
||||
|
||||
Goal: training data that teaches the CPT'd Qwen3-1.7B to emit the **route
|
||||
contract** — `{"intent":<enum>, key?, value?, text?, verb?}`, GBNF-constrained —
|
||||
contract** — `[{"intent":<enum>, key?, value?, text?, verb?}, ...]`, GBNF-constrained —
|
||||
matching `internal/router/llmrouter.go` (`routeSystem` + `routeGrammar`) verbatim.
|
||||
Train=deploy parity: label with the EXACT prompt the daemon sends.
|
||||
|
||||
@@ -19,24 +19,27 @@ state vs static memo).
|
||||
3. **Validate** — intent ∈ enum, keys ⊆ {intent,key,value,text,verb}. Drop invalid.
|
||||
4. **Balance check** — after a run, count intents. `act`/`system`/`fact` likely thin
|
||||
(function_calling skews query/act). Author extra examples for the holes; re-run.
|
||||
5. **Better prompt first** — improve `routeSystem` for sub-1B disambiguation before a
|
||||
5. **Better prompt first** — improve `routeSystem` for 1.7B disambiguation before a
|
||||
big generation run (awaiting prompt-guy input). Re-labeling is cheap; regenerate.
|
||||
6. **Train** — route-LoRA on top of CPT base, OR fold into the persona SFT as a second
|
||||
contract (decide once volume known). Eval = route accuracy on a held-out REAL set.
|
||||
6. **Train** — locked decision: fold route and persona examples into one balanced
|
||||
Qwen3 SFT. The system prompt selects the contract. Evaluate the two tasks
|
||||
separately; a separate route adapter is the fallback if joint SFT interferes.
|
||||
|
||||
## Blocking
|
||||
|
||||
- Router at `inference.kvmx.ru` / `localhost:6446` must be up (currently down).
|
||||
- CPT must finish before the route-LoRA trains on it.
|
||||
- CPT must finish and pass the raw-vs-CPT decision gate before joint SFT.
|
||||
|
||||
## Files
|
||||
|
||||
- `esp32-whisper-fine-tune/llm/gen_route_data.py` — the relabeler (done, self-checks).
|
||||
`ROUTE_SYSTEM` const = verbatim copy of Go `routeSystem`; **keep in sync**.
|
||||
- Output: `llm/data/route_train.jsonl` (resumable append).
|
||||
- Held-out: `llm/data/route_eval.jsonl`, generated from human labels by
|
||||
`build_route_eval.py`; never include it in route training generation.
|
||||
|
||||
## Prompt-guy question (sent 2026-07-11)
|
||||
|
||||
How to structure the router system prompt for a sub-1B model doing 7-intent
|
||||
How to structure the router system prompt for a 1.7B model doing 7-intent
|
||||
classification + slot extraction, GBNF-constrained — example ordering/count,
|
||||
contrastive near-miss pairs (note vs reminder) vs more singles, rule placement.
|
||||
|
||||
@@ -1,5 +1,11 @@
|
||||
# Plan — DIY RU-native base via Continued Pretraining (CPT)
|
||||
|
||||
> **Execution update, 2026-07-18:** corpus packing and full-weight smoke are
|
||||
> complete; training reached step 1000/8077. The materialized corpus is 264.6M
|
||||
> tokens from CulturaX/Wikipedia/books only. Evaluation, the exact decision gate
|
||||
> and Qwen3 joint SFT are now canonical in
|
||||
> `2026-07-18-qwen3-resident-training-eval.md`.
|
||||
|
||||
> **Goal:** Build our *own* RU-native base model instead of using Vikhr. Take a
|
||||
> clean newer base (Qwen3-1.7B-Base), continue-pretrain it on a curated Russian
|
||||
> corpus so it spells Cyrillic natively, then run the existing persona-LoRA →
|
||||
@@ -256,10 +262,10 @@ plateaus (not NaN, not flat-from-step-0); checkpoint saved to
|
||||
|
||||
## 4. Eval — did CPT actually help?
|
||||
|
||||
Extend the existing `llama-eval-test.py` into `eval_cyrillic.py`. This is also an
|
||||
open item in `CLAUDE.md` and is the regression metric for the whole project.
|
||||
`eval_cyrillic.py` now writes deterministic machine-readable results using
|
||||
pinned human-written RU and EN Universal Dependencies test sets.
|
||||
|
||||
**Metrics (run on a held-out set of real RU prompts — NOT synthetic, NOT training data):**
|
||||
**Metrics (run on held-out human-written text — NOT synthetic, NOT training data):**
|
||||
1. **Cyrillic validity %** — generate on RU prompts; % of outputs with zero
|
||||
mixed-script words and no homoglyph swaps (reuse §2.3 rule 3 as the checker).
|
||||
2. **Perplexity** on a held-out clean RU text set (lower = better RU fit).
|
||||
@@ -291,10 +297,10 @@ open item in `CLAUDE.md` and is the regression metric for the whole project.
|
||||
|
||||
## 6. Persona LoRA on the CPT'd base (existing pipeline)
|
||||
|
||||
Now the CPT base is just a better base. Run the **existing** `train_rocm.py`
|
||||
persona/format LoRA on top, with two changes:
|
||||
- Point base model at `./Qwen3-1.7B-ru-cpt/` (the CPT checkpoint), not Vikhr/Qwen2.5.
|
||||
- Confirm ChatML template matches Qwen3 (`<|im_start|>assistant … <|im_end|>`).
|
||||
Now the CPT base is just a better base. The rewritten `train_rocm.py` trains a
|
||||
balanced joint persona/router adapter and uses Qwen3's own chat template. It
|
||||
masks the rendered assistant continuation instead of assuming literal ChatML
|
||||
boundary token IDs.
|
||||
- The persona **data** is the `{response,mood}` corpus per `CLAUDE.md` Decision B
|
||||
(separate from the CPT corpus): 45% persona chit-chat, 15% graceful failure
|
||||
(`tired`/`confused`), 20% tool calls, 10% real utterances, 10% EN→EN. Mood
|
||||
|
||||
@@ -0,0 +1,157 @@
|
||||
# Plan — Qwen3-1.7B resident model: audit, evaluation, SFT and gate
|
||||
|
||||
> **Canonical model plan as of 2026-07-18.** Qwen3-1.7B is the one resident
|
||||
> router/phraser. Larger reasoners and custom Piper training are deferred until
|
||||
> the main Maven features are complete.
|
||||
|
||||
## Locked architecture
|
||||
|
||||
```
|
||||
Qwen3-1.7B-Base
|
||||
→ full-weight Russian CPT
|
||||
→ raw-vs-CPT decision gate
|
||||
→ one balanced persona + route LoRA
|
||||
→ contract evaluation
|
||||
→ merge → GGUF Q8_0 → homesrv
|
||||
```
|
||||
|
||||
The route system prompt selects a JSON-array action contract. The persona
|
||||
system prompt selects `{response,mood}`. Metrics are reported independently so
|
||||
joint-training interference is visible. If one task regresses, use separate
|
||||
adapters before considering a merged multi-task checkpoint.
|
||||
|
||||
## Current evidence
|
||||
|
||||
- Packed corpus: 129,221 × 2,048 = **264,644,608 tokens**.
|
||||
- Materialized character mix: CulturaX 69.407%, Wikipedia 17.991%, books
|
||||
12.602%. Planned synthetic/log/conversational buckets are absent from this run.
|
||||
- All 64 sampled Arrow rows had the right length and valid token IDs.
|
||||
- The audit found 247 CulturaX documents with glued `<phone>` placeholders and
|
||||
Cyrillic text. This run continues; fix placeholder spacing before any rebuild.
|
||||
- Full-weight Adafactor training reached checkpoint 1,000/8,077 (12.38%). Loss
|
||||
and gradient norms are finite, so this checkpoint is the full-weight smoke
|
||||
test. Do not launch a competing smoke job.
|
||||
|
||||
Machine-readable audit:
|
||||
`/home/kami/Programs/esp32-whisper-fine-tune/llm/CPT_CORPUS_AUDIT.json`.
|
||||
|
||||
## Tools
|
||||
|
||||
All commands run from `/home/kami/Programs/esp32-whisper-fine-tune` using its
|
||||
`.venv`.
|
||||
|
||||
```bash
|
||||
# Reproduce the read-only corpus/checkpoint audit.
|
||||
./.venv/bin/python llm/audit_cpt_corpus.py \
|
||||
--output llm/CPT_CORPUS_AUDIT.json
|
||||
|
||||
# Fetch pinned, human-written UD RU/EN test sets and their hashes.
|
||||
./.venv/bin/python llm/prepare_eval_data.py
|
||||
|
||||
# Prompt parity and held-out route labels.
|
||||
./.venv/bin/python llm/check_prompt_parity.py
|
||||
./.venv/bin/python llm/build_route_eval.py
|
||||
```
|
||||
|
||||
## Baseline and post-CPT evaluation
|
||||
|
||||
The evaluation is deterministic and uses token-weighted perplexity. Preserve
|
||||
the raw result; do not regenerate it with different limits when comparing CPT.
|
||||
|
||||
```bash
|
||||
./.venv/bin/python llm/eval_cyrillic.py \
|
||||
--model Qwen/Qwen3-1.7B-Base --limit 128 --device cuda \
|
||||
--output llm/data/eval/raw_qwen3_1.7b.json
|
||||
|
||||
# After CPT finishes:
|
||||
./.venv/bin/python llm/eval_cyrillic.py \
|
||||
--model llm/Qwen3-1.7B-ru-cpt --limit 128 --device cuda \
|
||||
--output llm/data/eval/cpt_qwen3_1.7b.json
|
||||
```
|
||||
|
||||
For a publication-quality comparison, repeat both with `--limit 0`; the quick
|
||||
128-sentence pair is the operational gate and must use identical arguments.
|
||||
|
||||
## Exact CPT decision gate
|
||||
|
||||
```bash
|
||||
./.venv/bin/python llm/decision_gate.py \
|
||||
--raw llm/data/eval/raw_qwen3_1.7b.json \
|
||||
--cpt llm/data/eval/cpt_qwen3_1.7b.json
|
||||
```
|
||||
|
||||
Exit 0 means all conditions passed:
|
||||
|
||||
- RU perplexity improves by at least 2%;
|
||||
- EN perplexity regresses by no more than 10%;
|
||||
- RU generation validity does not decline;
|
||||
- deterministic English probes remain English.
|
||||
- mean repeated 4-gram rate stays below 15%, or within five percentage points
|
||||
of the raw baseline when the baseline itself is worse.
|
||||
|
||||
Exit 1 blocks SFT. Inspect the JSON check map before changing a threshold. A
|
||||
threshold change is a documented architecture decision, not a convenient rerun.
|
||||
|
||||
Checkpoint 1,000 is an informative interim result: RU PPL improved 4.82% and EN
|
||||
PPL regressed only 2.20%, but repeated 4-grams rose from 8.56% to 25.90%.
|
||||
Therefore it passes the language-loss checks but **fails the complete gate**.
|
||||
This is not a stop signal at 12.38% of training; it is a regression to watch at
|
||||
the final checkpoint and a reason the gate includes generation degeneration.
|
||||
|
||||
## Resume CPT
|
||||
|
||||
The latest complete checkpoint is selected by Transformers:
|
||||
|
||||
```bash
|
||||
cd /home/kami/Programs/esp32-whisper-fine-tune/llm
|
||||
HSA_OVERRIDE_GFX_VERSION=11.0.0 ../.venv/bin/python train_cpt.py --resume \
|
||||
2>&1 | tee -a cpt_run.log
|
||||
```
|
||||
|
||||
Confirm there is only one `train_cpt.py` process before resuming. Completion is
|
||||
step 8,077 and must produce the final tokenizer/model files at
|
||||
`llm/Qwen3-1.7B-ru-cpt/`, not only checkpoint directories.
|
||||
|
||||
## Joint Qwen3 SFT
|
||||
|
||||
Generate route training data first. `route_train.jsonl` is currently a blocker;
|
||||
the held-out `route_eval.jsonl` already exists and must never be merged into it.
|
||||
|
||||
```bash
|
||||
cd /home/kami/Programs/esp32-whisper-fine-tune
|
||||
./.venv/bin/python llm/gen_route_data.py
|
||||
./.venv/bin/python llm/train_rocm.py --check-data
|
||||
./.venv/bin/python llm/train_rocm.py
|
||||
```
|
||||
|
||||
The rewritten trainer:
|
||||
|
||||
- loads the completed CPT base;
|
||||
- consumes explicit persona and route train/eval files;
|
||||
- balances the two tasks by oversampling only within the training set;
|
||||
- renders Qwen3's own chat template with thinking disabled;
|
||||
- masks loss before the assistant continuation without hard-coded ChatML IDs;
|
||||
- supports deterministic seeds, resume, early stopping and best-checkpoint load;
|
||||
- writes a training manifest.
|
||||
|
||||
After training:
|
||||
|
||||
```bash
|
||||
./.venv/bin/python llm/eval_contracts.py \
|
||||
--base llm/Qwen3-1.7B-ru-cpt \
|
||||
--adapter llm/Qwen3-1.7B-maven-sft \
|
||||
--output llm/data/eval/qwen3_contracts.json
|
||||
```
|
||||
|
||||
Initial deploy gates: persona JSON validity ≥99%, route JSON validity ≥99%,
|
||||
route intent sequence exact ≥90%, action intent accuracy ≥95%, slot exact ≥85%.
|
||||
Mood accuracy is diagnostic until the persona evaluation set is manually
|
||||
quality-reviewed; the existing examples contain stale technical answers.
|
||||
|
||||
## Deferred
|
||||
|
||||
- Piper/phoneme training.
|
||||
- On-demand larger reasoner.
|
||||
- Model hot-swap and the broad capability plans.
|
||||
|
||||
These resume after the main Maven feature set is complete.
|
||||
Reference in New Issue
Block a user