email: drop the dead Gmail rule, fix nested MIME, decode windows-1251

The Gmail category rule matched X-GM-LABELS and X-Gmail-Labels against the
parsed header block. Neither is a header. X-GM-LABELS is a Gmail FETCH data
item and never appears in the message source, and X-Gmail-Labels only exists in
a Takeout export, so the branch could not fire against a real mailbox while its
doc comment promised a Promotions filter. Its test built the header by hand and
therefore asserted the matcher rather than the plumbing. The rule is removed and
the comment says what bringing it back would take.

multipartText folded a nested multipart's answer into one string, so HTML
derived text landed in the plain bucket and a real text/plain sibling later in
the message was discarded by the guard on plain being set. The two buckets now
stay separate through the recursion.

windows-1251 returned an unsupported-charset error and the message degraded to
subject only. That is the charset older Russian senders still use, so those
mails could never produce a task candidate. It is decoded from a 128 entry table
here rather than by vendoring x/text, for the body and for encoded words in the
subject. Every other unknown charset still degrades to subject only.
Found in review of #63.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TrVSBKe3RFDF4fGYKWYQnX
This commit is contained in:
kami
2026-08-01 14:01:04 +04:00
parent d69a1f8076
commit 6c81df17ec
7 changed files with 182 additions and 55 deletions
+43 -3
View File
@@ -70,13 +70,28 @@ func TestParsePrefersPlainAndSkipsAttachments(t *testing.T) {
}
}
// An unsupported charset must degrade to headers-only rather than to mojibake
// the model would then extract a task from.
func TestParseUnsupportedCharsetKeepsHeaders(t *testing.T) {
// windows-1251 is what older Russian senders still emit. Subject-only for those
// mails meant they could never produce a task candidate.
func TestParseCP1251(t *testing.T) {
msg, err := ParseMessage(3, fixture(t, "cp1251.eml"))
if err != nil {
t.Fatalf("parse: %v", err)
}
if want := "Счёт за интернет"; msg.Subject != want {
t.Errorf("subject = %q, want %q", msg.Subject, want)
}
if want := "Оплати счёт до пятницы."; !strings.Contains(msg.Body, want) {
t.Errorf("body = %q, want it to contain %q", msg.Body, want)
}
}
// A charset with no table here must degrade to headers-only rather than to
// mojibake the model would then extract a task from.
func TestParseUnsupportedCharsetKeepsHeaders(t *testing.T) {
msg, err := ParseMessage(3, fixture(t, "koi8r.eml"))
if err != nil {
t.Fatalf("parse: %v", err)
}
if msg.Subject != "Legacy" {
t.Errorf("subject = %q, want Legacy", msg.Subject)
}
@@ -85,6 +100,31 @@ func TestParseUnsupportedCharsetKeepsHeaders(t *testing.T) {
}
}
// A nested multipart/alternative that only had HTML must not fill the plain
// bucket: a real text/plain sibling later in the message is the better text and
// used to be discarded.
func TestParseNestedHTMLDoesNotShadowLaterPlain(t *testing.T) {
raw := "Subject: nested\r\n" +
"Content-Type: multipart/mixed; boundary=OUT\r\n\r\n" +
"--OUT\r\n" +
"Content-Type: multipart/alternative; boundary=IN\r\n\r\n" +
"--IN\r\n" +
"Content-Type: text/html; charset=utf-8\r\n\r\n" +
"<p>from the html part</p>\r\n" +
"--IN--\r\n" +
"--OUT\r\n" +
"Content-Type: text/plain; charset=utf-8\r\n\r\n" +
"the real plain text\r\n" +
"--OUT--\r\n"
msg, err := ParseMessage(5, []byte(raw))
if err != nil {
t.Fatalf("parse: %v", err)
}
if got := strings.TrimSpace(msg.Body); got != "the real plain text" {
t.Errorf("body = %q, want the text/plain part to win", got)
}
}
func TestParseTruncatesLongBody(t *testing.T) {
var b strings.Builder
b.WriteString("Subject: long\r\nContent-Type: text/plain; charset=utf-8\r\n\r\n")