Files
Maven/internal/email/fetch.go
kami 8c6332f95c mavmaild: own its state volume, retire aged-out UIDs, stop restart-looping
The commented compose service mounted dbdata, the encrypted database volume,
read-write, for one JSON file of UIDs. The header of that same file says only
mavend holds the key and the db volume, and the whole argument for a separate
reader is that a compromise on either side does not reach the other. It gets its
own volume now, at its own path, so neither can be restored from a backup of the
other.

The high-water mark only advances through a contiguous run, and a failed ingest
is deliberately not marked. One message that never ingested therefore pinned the
mark forever: after the lookback window passed it could never be fetched again,
so the gap never closed, every UID above it stayed in the explicit set, and save
rewrote all of them every poll. FetchSince now reports the SEARCH window and the
poller retires everything below it, since a UID that can no longer be searched
for can never be read.

On ErrUnknownMethod the daemon logged "stopping" and then exited at the next
tick with status 0. The compose service inherits restart: unless-stopped, which
restarts a clean exit, so the real behaviour was a loop of four IMAP logins an
hour against a mailbox core would not accept anything from. It now stays up and
polls nothing.

The reader also sends the Junk verdict instead of counting bulk locally, which
is what the wire doc says it does. The verdict carries no mail content, since
nothing on the other side will read it. RunWith is gone, so the tests fake the
read rather than the transport.
Found in review of #65.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TrVSBKe3RFDF4fGYKWYQnX
2026-08-01 14:08:32 +04:00

140 lines
5.0 KiB
Go

package email
import (
"errors"
"fmt"
"time"
)
// FetchSince is the whole read path in one call: connect, log in, examine the
// mailbox read-only, list what arrived since a date, fetch and parse the ones
// the caller has not seen, log out.
//
// It is a function rather than a long-lived object because a mail poller should
// not hold an authenticated session (and therefore his credential in a live TLS
// state) between polls. Connect, read, drop.
//
// skip decides which UIDs are already known — the poller's seen-set. max bounds
// one poll: a mailbox that received 400 messages overnight must not turn into
// 400 LLM calls, and the newest max are the ones a task could still be hiding
// in. Junk messages are returned too, flagged, so the caller can mark them seen
// without a second protocol round.
type FetchSince struct {
Addr string // host or host:993
User string
Mailbox string // e.g. "INBOX"
Timeout time.Duration
Since time.Time
Max int
Skip func(uid uint32) bool
// OnSearch, when set, is handed the whole SEARCH result before anything is
// fetched, ascending, seen UIDs included. It is how the poller learns which
// UIDs are still inside the lookback window: anything below the lowest one
// can never be searched for again, and therefore can never be read again.
// Without that the poller cannot tell a UID it has not got to yet from one
// that has aged out of the window.
OnSearch func(uids []uint32)
// dial — the connection seam, unexported on purpose: see dialer(). Tests
// inside this package set it through export_test.go; nothing outside can.
dial func(addr string, timeout time.Duration) (*Conn, error)
}
// dial is the connection seam. nil means Dial (implicit TLS); the tests set it
// through export_test.go. It is unexported and there is no exported wrapper
// that takes a dialer, so no code outside this package can point the reader at
// a non-TLS transport and hand it the password. That is a property the compiler
// enforces, not a claim about the callers that happen to exist today.
func (f FetchSince) dialer() func(addr string, timeout time.Duration) (*Conn, error) {
if f.dial != nil {
return f.dial
}
return Dial
}
// Run performs one read. password is passed here, not stored in the struct, so
// the configuration of a mailbox and the secret for it are never the same value
// sitting in the same place.
//
// One message that cannot be read does not abandon the poll: the rest of the
// mailbox is still worth reading, and returning early meant one oversized or
// unreadable message permanently hid every older message behind it, poll after
// poll. The returned error joins whatever failed, and the messages that did
// come back come back with it.
func (f FetchSince) Run(password string) ([]Message, error) {
if f.Addr == "" || f.User == "" || f.Mailbox == "" {
return nil, fmt.Errorf("email: mailbox not configured (addr/user/mailbox)")
}
// Timeout is validated like the other three fields. Zero disables every
// deadline in the path — net.Dialer{Timeout: 0} and a Conn that never calls
// SetDeadline — so a dead server parks the poller forever on a socket read,
// with his credential live in a TLS state. That is the exact thing the
// connect-read-drop shape exists to avoid.
if f.Timeout <= 0 {
return nil, fmt.Errorf("email: timeout must be positive")
}
dial := f.dialer()
c, err := dial(f.Addr, f.Timeout)
if err != nil {
return nil, err
}
defer c.Close()
if err := c.Login(f.User, password); err != nil {
return nil, err
}
defer c.Logout()
if err := c.Select(f.Mailbox); err != nil {
return nil, err
}
uids, err := c.SearchSince(f.Since)
if err != nil {
return nil, err
}
if f.OnSearch != nil {
f.OnSearch(uids)
}
// Newest UIDs first — IMAP hands them back ascending, and when Max clips the
// list the recent mail is what matters.
wanted := make([]uint32, 0, len(uids))
for i := len(uids) - 1; i >= 0; i-- {
if f.Skip != nil && f.Skip(uids[i]) {
continue
}
wanted = append(wanted, uids[i])
if f.Max > 0 && len(wanted) >= f.Max {
break
}
}
out := make([]Message, 0, len(wanted))
var failed []error
for _, uid := range wanted {
raw, err := c.Fetch(uid)
if errors.Is(err, ErrMessageTooLarge) {
// Too big to read is a permanent verdict, not a failure to retry:
// the message will be the same size next poll. Carried as bulk so
// the poller marks it seen and stops fetching it, exactly like a
// newsletter. Nothing is sent to the model.
out = append(out, Message{UID: uid, Junk: true, JunkReason: "oversize"})
continue
}
if err != nil {
// The error names the UID, never the message. Collected rather than
// returned, so the messages behind this one are still read.
failed = append(failed, fmt.Errorf("email: fetch uid %d: %w", uid, err))
continue
}
if len(raw) == 0 {
continue // vanished between SEARCH and FETCH
}
msg, err := ParseMessage(uid, raw)
if err != nil {
continue // unparsable headers — nothing to review, skip silently
}
out = append(out, msg)
}
return out, errors.Join(failed...)
}