kami
cb3641e7bb
internal/rss parses RSS 2.0 and Atom, and polls each configured feed on its own interval; internal/webfetch is the one door either of them uses to touch the network. The poller writes items as notes with source "rss:<feed>" and nothing else: the answer path reads them back when he asks "что нового в лентах?", and nothing is announced on arrival. A feed that dispatched would be a nag, which is why the plan's breaking-news rule was left out rather than built. webfetch is where the limits live, as code rather than a paragraph: http(s) only, an allowlist (the configured feeds' hosts) and a denylist, a 2 MiB body cap, a 3-redirect cap, one request per host per second, and a refusal to connect to any private address — checked in the dialer's Control hook so it holds for every resolved address and every redirect hop, not just for a literal IP. Off unless configured: no "feeds" block, no poller, no outbound request. How far a feed was read is a config fact (rss:latest:<name>), so a restart does not re-note yesterday's headlines.
55 lines
3.7 KiB
Markdown
55 lines
3.7 KiB
Markdown
# Plan: RSS / News Feed Reader
|
|
|
|
**Goal:** Maven periodically polls configured RSS/Atom feeds, extracts new items, classifies them by relevance using the embedder, and either stores interesting items as notes or surfaces breaking news via the dispatcher.
|
|
|
|
**Done when:**
|
|
- `internal/rss/` package — feed parser (Go stdlib `encoding/xml` or `github.com/mmcdole/gofeed`), poll loop
|
|
- Feed config in `config.Config` — `feeds: [{name, url, category, poll_interval}]`
|
|
- New items are classified by the existing `router.Embedder` — items below a relevance threshold are silently dropped
|
|
- Interesting items: written as `WriteNote` with `source="rss:<feed_name>"`; breaking/high-severity items dispatched as care nudges (sev2-3)
|
|
- Queryable via voice: "что нового в лентах?", "что по технологиям?"
|
|
|
|
**Scope:**
|
|
- New `internal/rss/` package — periodic poller, feed parser, item classifier
|
|
- Config extension: `feeds` array in `config.Config`
|
|
- Reuses `internal/router.Embedder` for relevance scoring (same ONNX model or HashEmbedder floor)
|
|
- Reuses `internal/ipc.CoreAPI` for writing notes
|
|
- Reuses `internal/delivery.Dispatcher` for breaking news nudges
|
|
- Reuses `internal/llm.Client` for item summarization (optional, for long articles)
|
|
|
|
**Steps:**
|
|
1. Add `gofeed` or raw XML parser — `internal/rss/feed.go`: `ParseFeed(url) ([]Item, error)` where `Item{Title, Link, Summary, Published, Content}`
|
|
2. Create `internal/rss/poller.go` — `Poller` holds `[]FeedConfig`, polls each on its own interval, tracks `last_poll` per feed via a fact (`kind=config, key=rss:poll:<name>`)
|
|
3. Implement relevance classifier — embed each item's title+summary with `router.Embedder`, compare against user interest profile (built from notes/facts), drop items below threshold
|
|
4. Interesting items → `ipc.WriteNote(ctx, ts, title+"\n"+summary, embedding, "rss:<feed_name>")`
|
|
5. Breaking items (keywords: "CVE", "outage", "critical") → `ipc.WriteFact(kind=env, key=news:breaking, value=title)` → loop rule `BreakingNewsRule` (sev3) dispatches nudge
|
|
6. Wire poller into `cmd/mavend/main.go` — starts after unlock, separate goroutine
|
|
7. Add voice query handler — `"что нового?"` queries `RecentNotes` filtered by source prefix `rss:` and phrases via `phraser.PhraseQuery`
|
|
8. Add `feeds` block to `config.Config` and `deploy/mavend.json`
|
|
9. Test with a live RSS feed (e.g., `https://news.ycombinator.com/rss`) — verify items appear in notes table
|
|
|
|
---
|
|
|
|
## Shipped 2026-08-01 (#258)
|
|
|
|
`internal/webfetch` (the guarded HTTP door: scheme, allow/deny hosts, private-address
|
|
refusal in the dialer, size cap, redirect cap, per-host rate limit), `internal/rss`
|
|
(RSS 2.0 + Atom parser, poller with durable marks and a keyword filter),
|
|
`cmd/mavend/feeds.go` (ticker, fetcher adapter, `rss:latest:<feed>` fact marks),
|
|
config block `feeds`, and the `feeds` query source with `router.ParseFeedQuery`.
|
|
|
|
Deviations from the plan above, both deliberate:
|
|
|
|
- **Step 5 (breaking-news nudges) was not built.** A feed that dispatches is a nag,
|
|
and the one thing Maven is not is a nag. Items are read when asked and nowhere else.
|
|
If breaking news is ever wanted, it belongs behind the existing delivery policy
|
|
(severity, quiet hours, digest), not in the poller.
|
|
- **Step 3 (embedder relevance) is a seam, not an implementation.** `rss.Ranker`
|
|
exists and is wired nil. Scoring items against an "interest profile" needs a
|
|
profile, and there is none yet; a threshold with nothing to compare against is a
|
|
random filter with a confident name. The filter that runs is the per-feed
|
|
include/exclude keyword list, which he can read and predict.
|
|
|
|
No new dependency: stdlib `encoding/xml`, no gofeed. Stock deploy config has no
|
|
`feeds` block, so the capability is off.
|