Maven/docs/plans/05-model-swap.md

Plan: On-the-Fly Model Swap (Multi-Model Routing)

Goal: Maven can switch between LLM models at runtime — including using a remote llama-server instance on workpc over LAN — without restarting the daemon. The phraser, router (LLM router), and replier all point at a dynamic backend that can be re-pointed via IPC.

Done when:

• internal/llm/client.go supports a dynamic base URL that can be swapped at runtime
• internal/phraser/llmphraser.go can hot-swap its backend (stop current llama-server subprocess, start new one, or point to a remote one)
• Remote model config: phraser.mode = "remote" with remote_url = "http://workpc:8080" — connects without spawning a subprocess
• Swap is triggered via IPC (MethodSwapModel) with a new config block — no daemon restart
• Router's LLMRouter (in internal/router/llmrouter.go) follows the same swap
• Fallback: if the new model fails to respond within timeout, the old model stays active (never leave the user with no model)

Scope:

• internal/llm/client.go — add SetBaseURL(string) method for runtime re-pointing
• internal/phraser/llmphraser.go — add Swap(Config) error method
• internal/router/llmrouter.go — already holds a Completer interface; swap the underlying client
• cmd/mavend/voice.go — re-creates LLMReplier when model changes
• New IPC method MethodSwapModel in internal/ipc/api.go
• Config: phraser.mode field (local|remote), phraser.remote_url

Steps:

1. Add SetBaseURL(url string) to internal/llm/client.go — atomically swaps the base field under a mutex (add sync.RWMutex to Client)
2. Add Swap(cfg Config) error to internal/phraser/llmphraser.go — stops current llama-server (via Close()), starts new one with new config, or connects to remote URL without spawning
3. Extend PhraserConfig in internal/config/config.go with Mode string ("local" or "remote") and RemoteURL string
4. Create new internal/llm/manager.go — manages a set of named backends, allows SwitchModel(name) that re-wires phraser + LLM router + replier atomically
5. Add MethodSwapModel to internal/ipc/api.go with request {model_path, mode, remote_url, n_gpu_layers, n_ctx}
6. Wire swap handler in cmd/mavend/main.go — srv.ModelSwapFn called from IPC dispatch, re-wires phraser, rebuilds router with new LLMRouter, rebuilds replier
7. Add model block to config.Config with named model definitions (local paths + remote URLs)
8. Test: swap between local StubPhraser and remote llama-server on LAN; verify phraser + router + replier all use the new backend