rss: stop claiming "что нового" and stop re-noting the same items
"что нового?" is a greeting, and the feed matcher claimed it: "нового" was a feed noun and "что" an ask. With no feeds block, which is what ships, the answer to hello was "я пока не читаю ленты — они не настроены". A newness word now needs a named topic or a real feed noun beside it. The topic prepositions lose "о" for the same class of reason: one rune of filler produced a category of whatever followed it, and then "по этой теме в лентах пока ничего". An undated feed was re-noted in full on every boot. Dated items are deduped against the durable mark, undated ones against a map that dies with the process, so five items became five more on the next start, stamped now, at the top of the recent-notes window. A crash loop made that a flood. The mark is now set for an undated feed too, and its existence marks the first poll after a restart as a resync: those items are recorded as seen rather than written. A burst larger than max_items lost its middle. The poll walked the feed newest-first, stopped at the cap, and marked the newest item written, which put everything below the cap behind the mark forever. The cap now applies to the oldest candidates and the mark follows what was written, so max_items paces instead of dropping. The category tag was read out loud: "Заголовок [технологии]" went through piper brackets and all, because the answer path took the whole first line. The tag is parsed off for reading and is now the only thing a topic is matched against. Matching the whole note meant "что нового про погоду" hit any tech headline whose link contained "pogod". Also: the charset comment on dec.Strict described something Strict does not do, and a skipped feed is named in the log. Found in review of #66.
This commit is contained in:
@@ -72,9 +72,13 @@ type feedDoc struct {
|
||||
func Parse(r io.Reader) (Feed, error) {
|
||||
var doc feedDoc
|
||||
dec := xml.NewDecoder(r)
|
||||
// Feeds in the wild declare windows-1251 and worse. We only ever read
|
||||
// UTF-8; a charset we cannot decode is a feed we do not read, which is
|
||||
// better than mojibake in his notes.
|
||||
// Strict=false buys tolerance of the malformed markup feeds are full of:
|
||||
// unclosed tags, stray entities. It has nothing to do with charsets.
|
||||
//
|
||||
// Charsets are handled by not handling them: CharsetReader stays nil, so a
|
||||
// feed declaring windows-1251 fails to parse rather than being read as
|
||||
// UTF-8. That is the behaviour we want — a charset we cannot decode is a
|
||||
// feed we do not read, which beats mojibake in his notes.
|
||||
dec.Strict = false
|
||||
if err := dec.Decode(&doc); err != nil {
|
||||
return Feed{}, fmt.Errorf("rss: bad xml: %w", err)
|
||||
|
||||
Reference in New Issue
Block a user