kami/Maven

Files

T

kami cb3641e7bb Read RSS and Atom feeds, and speak about them only when asked (#258 )

internal/rss parses RSS 2.0 and Atom, and polls each configured feed on its own
interval; internal/webfetch is the one door either of them uses to touch the
network. The poller writes items as notes with source "rss:<feed>" and nothing
else: the answer path reads them back when he asks "что нового в лентах?", and
nothing is announced on arrival. A feed that dispatched would be a nag, which is
why the plan's breaking-news rule was left out rather than built.

webfetch is where the limits live, as code rather than a paragraph: http(s)
only, an allowlist (the configured feeds' hosts) and a denylist, a 2 MiB body
cap, a 3-redirect cap, one request per host per second, and a refusal to connect
to any private address — checked in the dialer's Control hook so it holds for
every resolved address and every redirect hop, not just for a literal IP.

Off unless configured: no "feeds" block, no poller, no outbound request. How far
a feed was read is a config fact (rss:latest:<name>), so a restart does not
re-note yesterday's headlines.

2026-08-01 03:27:45 +04:00

3.7 KiB

Raw Blame History

Plan: RSS / News Feed Reader

Goal: Maven periodically polls configured RSS/Atom feeds, extracts new items, classifies them by relevance using the embedder, and either stores interesting items as notes or surfaces breaking news via the dispatcher.

Done when:

internal/rss/ package — feed parser (Go stdlib encoding/xml or github.com/mmcdole/gofeed), poll loop
Feed config in config.Config — feeds: [{name, url, category, poll_interval}]
New items are classified by the existing router.Embedder — items below a relevance threshold are silently dropped
Interesting items: written as WriteNote with source="rss:<feed_name>"; breaking/high-severity items dispatched as care nudges (sev2-3)
Queryable via voice: "что нового в лентах?", "что по технологиям?"

Scope:

New internal/rss/ package — periodic poller, feed parser, item classifier
Config extension: feeds array in config.Config
Reuses internal/router.Embedder for relevance scoring (same ONNX model or HashEmbedder floor)
Reuses internal/ipc.CoreAPI for writing notes
Reuses internal/delivery.Dispatcher for breaking news nudges
Reuses internal/llm.Client for item summarization (optional, for long articles)

Steps:

Add gofeed or raw XML parser — internal/rss/feed.go: ParseFeed(url) ([]Item, error) where Item{Title, Link, Summary, Published, Content}
Create internal/rss/poller.go — Poller holds []FeedConfig, polls each on its own interval, tracks last_poll per feed via a fact (kind=config, key=rss:poll:<name>)
Implement relevance classifier — embed each item's title+summary with router.Embedder, compare against user interest profile (built from notes/facts), drop items below threshold
Interesting items → ipc.WriteNote(ctx, ts, title+"\n"+summary, embedding, "rss:<feed_name>")
Breaking items (keywords: "CVE", "outage", "critical") → ipc.WriteFact(kind=env, key=news:breaking, value=title) → loop rule BreakingNewsRule (sev3) dispatches nudge
Wire poller into cmd/mavend/main.go — starts after unlock, separate goroutine
Add voice query handler — "что нового?" queries RecentNotes filtered by source prefix rss: and phrases via phraser.PhraseQuery
Add feeds block to config.Config and deploy/mavend.json
Test with a live RSS feed (e.g., https://news.ycombinator.com/rss) — verify items appear in notes table

Shipped 2026-08-01 (#258)

internal/webfetch (the guarded HTTP door: scheme, allow/deny hosts, private-address refusal in the dialer, size cap, redirect cap, per-host rate limit), internal/rss (RSS 2.0 + Atom parser, poller with durable marks and a keyword filter), cmd/mavend/feeds.go (ticker, fetcher adapter, rss:latest:<feed> fact marks), config block feeds, and the feeds query source with router.ParseFeedQuery.

Deviations from the plan above, both deliberate:

Step 5 (breaking-news nudges) was not built. A feed that dispatches is a nag, and the one thing Maven is not is a nag. Items are read when asked and nowhere else. If breaking news is ever wanted, it belongs behind the existing delivery policy (severity, quiet hours, digest), not in the poller.
Step 3 (embedder relevance) is a seam, not an implementation. rss.Ranker exists and is wired nil. Scoring items against an "interest profile" needs a profile, and there is none yet; a threshold with nothing to compare against is a random filter with a confident name. The filter that runs is the per-feed include/exclude keyword list, which he can read and predict.

No new dependency: stdlib encoding/xml, no gofeed. Stock deploy config has no feeds block, so the capability is off.

3.7 KiB Raw Blame History

Plan: RSS / News Feed Reader

Shipped 2026-08-01 (#258)

3.7 KiB

Raw Blame History