Maven/docs/plans/07-vision.md

Plan: Vision — Image Understanding Capability

Goal: Maven can "see" — accept images (from mavweb upload, Telegram, or filesystem paths), run vision inference via a local or remote multimodal model, and answer questions about the image content or extract structured information.

Done when:

• Vision model backend is configurable: local multimodal LLM (e.g., LLaVA, Qwen-VL via llama-server mmproj) or remote API
• internal/vision/ package handles image preprocessing, model inference, result parsing
• Voice/text commands like "что на картинке?" or "прочитай текст с экрана" route to the vision handler
• Extracted information can be written as facts/notes through ipc.CoreAPI
• Telegram image messages are processed through the same pipeline

Scope:

• New internal/vision/ package — image loader (Go stdlib image + golang.org/x/image), inference client
• New config block: voice.vision in config.Config — {enabled, provider, model_path, mmproj_path, remote_url}
• Router intent extension: new IntentVision or reuse IntentQuery with a vision flag
• Reuses internal/llm.Client for API-compatible backends (OpenAI-compatible vision API)
• Reuses internal/ipc.CoreAPI for writing extracted data

Steps:

1. Create internal/vision/provider.go — Provider interface with Describe(image []byte, prompt string) (string, error) and ExtractText(image []byte) (string, error)
2. Implement LocalProvider — spawns llama-server with mmproj, sends multimodal chat completion requests
3. Implement RemoteProvider — calls an OpenAI-compatible vision API endpoint, reuses internal/llm.Client
4. Create internal/vision/processor.go — image preprocessing (resize, format conversion to JPEG/PNG, base64 encoding)
5. Wire vision into cmd/mavend/voice.go:reactiveHandler — detect vision intent from router (new IntentVision or a Slots.HasImage flag)
6. Add IPC method MethodDescribeImage for programmatic access (mavweb upload, telegram bot)
7. Add vision config block to config.Config and wire in cmd/mavend/main.go
8. Test with a local multimodal model: send an image via mavweb, verify description and text extraction