kami
6c81df17ec
The Gmail category rule matched X-GM-LABELS and X-Gmail-Labels against the parsed header block. Neither is a header. X-GM-LABELS is a Gmail FETCH data item and never appears in the message source, and X-Gmail-Labels only exists in a Takeout export, so the branch could not fire against a real mailbox while its doc comment promised a Promotions filter. Its test built the header by hand and therefore asserted the matcher rather than the plumbing. The rule is removed and the comment says what bringing it back would take. multipartText folded a nested multipart's answer into one string, so HTML derived text landed in the plain bucket and a real text/plain sibling later in the message was discarded by the guard on plain being set. The two buckets now stay separate through the recursion. windows-1251 returned an unsupported-charset error and the message degraded to subject only. That is the charset older Russian senders still use, so those mails could never produce a task candidate. It is decoded from a 128 entry table here rather than by vendoring x/text, for the body and for encoded words in the subject. Every other unknown charset still degrades to subject only. Found in review of #63. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TrVSBKe3RFDF4fGYKWYQnX
72 lines
3.2 KiB
Go
72 lines
3.2 KiB
Go
package email
|
|
|
|
import (
|
|
"net/mail"
|
|
"strings"
|
|
)
|
|
|
|
// The junk filter — the cheapest and most important half of reading mail.
|
|
//
|
|
// A mailbox is mostly machine-generated: newsletters, receipts nobody acts on,
|
|
// social notifications, marketing. Sending all of it to a 1.7B and asking "is
|
|
// there a task here" produces confident nonsense at a rate proportional to the
|
|
// volume, so junk is decided by HEADERS, before any model sees the message.
|
|
//
|
|
// The rules are all bulk-mail markers that senders set on themselves, never
|
|
// guesses about content:
|
|
//
|
|
// - List-Unsubscribe / List-Id — by definition a mailing list. If he can
|
|
// unsubscribe from it, it is not asking him to do anything.
|
|
// - Precedence: bulk|junk|list — the sender declaring itself bulk.
|
|
// - Auto-Submitted other than "no" (RFC 3834) — generated by a machine.
|
|
// - X-Spam-Flag: YES, X-Spam-Status: Yes — the spam filter upstream already
|
|
// decided; we do not second-guess it in the other direction.
|
|
//
|
|
// There is deliberately NO Gmail-category rule. One was written and removed:
|
|
// it matched X-GM-LABELS and X-Gmail-Labels against the parsed header block,
|
|
// and neither is a header. X-GM-LABELS is a Gmail FETCH data item, requested as
|
|
// "UID FETCH n (X-GM-LABELS)" and never present in the message source;
|
|
// X-Gmail-Labels only exists in a Takeout mbox export. This client asks for
|
|
// BODY.PEEK[] and nothing else, so the rule could not fire against a real
|
|
// mailbox while its doc comment promised a Promotions filter. Gmail's promotion
|
|
// mail carries List-Unsubscribe in practice and is caught by the rule above.
|
|
// Bringing the category rule back means adding the FETCH item and carrying the
|
|
// labels into classifyJunk out of band, not matching a header that never
|
|
// arrives.
|
|
//
|
|
// Deliberately NOT here: sender allow/deny lists and subject keyword matching.
|
|
// Both are configuration that ages badly and both would be a place for his
|
|
// contacts to end up in a config file. If a real correspondent's mail is being
|
|
// dropped, the fix is a rule about a header, not a list of names.
|
|
//
|
|
// A junk verdict never deletes anything and never touches a flag on the server.
|
|
// It means "do not spend the model on this", nothing more.
|
|
|
|
// junkHeaders — headers whose mere presence marks bulk mail.
|
|
var junkPresence = []string{"List-Unsubscribe", "List-Id", "List-Post"}
|
|
|
|
// classifyJunk returns whether the message is bulk/automated and why. The
|
|
// reason is a short header name, safe to log — it names the marker, never the
|
|
// sender or the subject.
|
|
func classifyJunk(h mail.Header) (bool, string) {
|
|
for _, name := range junkPresence {
|
|
if strings.TrimSpace(h.Get(name)) != "" {
|
|
return true, strings.ToLower(name)
|
|
}
|
|
}
|
|
switch strings.ToLower(strings.TrimSpace(h.Get("Precedence"))) {
|
|
case "bulk", "junk", "list":
|
|
return true, "precedence"
|
|
}
|
|
if v := strings.ToLower(strings.TrimSpace(h.Get("Auto-Submitted"))); v != "" && v != "no" {
|
|
return true, "auto-submitted"
|
|
}
|
|
if strings.EqualFold(strings.TrimSpace(h.Get("X-Spam-Flag")), "yes") {
|
|
return true, "x-spam-flag"
|
|
}
|
|
if v := strings.ToLower(strings.TrimSpace(h.Get("X-Spam-Status"))); strings.HasPrefix(v, "yes") {
|
|
return true, "x-spam-status"
|
|
}
|
|
return false, ""
|
|
}
|