package email import ( "net/mail" "strings" ) // The junk filter — the cheapest and most important half of reading mail. // // A mailbox is mostly machine-generated: newsletters, receipts nobody acts on, // social notifications, marketing. Sending all of it to a 1.7B and asking "is // there a task here" produces confident nonsense at a rate proportional to the // volume, so junk is decided by HEADERS, before any model sees the message. // // The rules are all bulk-mail markers that senders set on themselves, never // guesses about content: // // - List-Unsubscribe / List-Id — by definition a mailing list. If he can // unsubscribe from it, it is not asking him to do anything. // - Precedence: bulk|junk|list — the sender declaring itself bulk. // - Auto-Submitted other than "no" (RFC 3834) — generated by a machine. // - X-Spam-Flag: YES, X-Spam-Status: Yes — the spam filter upstream already // decided; we do not second-guess it in the other direction. // // There is deliberately NO Gmail-category rule. One was written and removed: // it matched X-GM-LABELS and X-Gmail-Labels against the parsed header block, // and neither is a header. X-GM-LABELS is a Gmail FETCH data item, requested as // "UID FETCH n (X-GM-LABELS)" and never present in the message source; // X-Gmail-Labels only exists in a Takeout mbox export. This client asks for // BODY.PEEK[] and nothing else, so the rule could not fire against a real // mailbox while its doc comment promised a Promotions filter. Gmail's promotion // mail carries List-Unsubscribe in practice and is caught by the rule above. // Bringing the category rule back means adding the FETCH item and carrying the // labels into classifyJunk out of band, not matching a header that never // arrives. // // Deliberately NOT here: sender allow/deny lists and subject keyword matching. // Both are configuration that ages badly and both would be a place for his // contacts to end up in a config file. If a real correspondent's mail is being // dropped, the fix is a rule about a header, not a list of names. // // A junk verdict never deletes anything and never touches a flag on the server. // It means "do not spend the model on this", nothing more. // junkHeaders — headers whose mere presence marks bulk mail. var junkPresence = []string{"List-Unsubscribe", "List-Id", "List-Post"} // classifyJunk returns whether the message is bulk/automated and why. The // reason is a short header name, safe to log — it names the marker, never the // sender or the subject. func classifyJunk(h mail.Header) (bool, string) { for _, name := range junkPresence { if strings.TrimSpace(h.Get(name)) != "" { return true, strings.ToLower(name) } } switch strings.ToLower(strings.TrimSpace(h.Get("Precedence"))) { case "bulk", "junk", "list": return true, "precedence" } if v := strings.ToLower(strings.TrimSpace(h.Get("Auto-Submitted"))); v != "" && v != "no" { return true, "auto-submitted" } if strings.EqualFold(strings.TrimSpace(h.Get("X-Spam-Flag")), "yes") { return true, "x-spam-flag" } if v := strings.ToLower(strings.TrimSpace(h.Get("X-Spam-Status"))); strings.HasPrefix(v, "yes") { return true, "x-spam-status" } return false, "" }