Compare commits

...

4 Commits

Author SHA1 Message Date
kami ee7bec11e3 Add mavmaild, the read-only IMAP poller that feeds mail intake (#246)
The extraction seam landed on the previous branch but nothing fed it. This
adds the daemon that does: every interval it opens one mailbox read-only
(EXAMINE + BODY.PEEK, so reading leaves no \Seen behind), fetches the UIDs
it has not handed over yet, and posts each message to core over
ingest_mail. Core runs the model and writes task candidates; this daemon
writes nothing and cannot create a reminder.

It is a separate daemon because of the credential. mavpoll set the
precedent with the zenmoney token (#125): the module talking to the third
party holds the secret, reads it from a file so it never lands in argv, in
docker-compose.yml or in shell history, and core never sees it. There is
deliberately no -password flag, and a test asserts that.

Off unless configured at both ends: without -password-file the daemon
refuses to start, and if core has no email block the first ingest returns
ErrUnknownMethod, which disables the reader instead of hammering a socket
that will keep refusing. A seen-UID state file (0600, atomic write) keeps a
restart from re-extracting the whole lookback window; correctness does not
depend on it, since capture dedupes on normalised text. Logs are counts and
UIDs — no subject, sender or body.

Verified with an in-process IMAP server and a fake core: bulk mail is
filtered before core is asked, seen UIDs are not re-fetched, a failed
ingest is retried next poll, ErrUnknownMethod stops at the first message,
and state survives a restart. The live half is untested by design — no IMAP
credential exists on this box; setup is written up as QA steps.

Vikunja #246
2026-08-01 03:13:18 +04:00
kami f42d1594ef Turn a mail into task candidates, and into nothing else (#246)
The extraction half. internal/email.Extractor asks the resident Qwen3-1.7B,
under a GBNF grammar, what one message requires of him, and returns at most
three short candidates with an optional date.

Everything it can produce is a row in `tasks` with status "candidate",
written through the intake seam #130 built for exactly this (Source
"email:<mailbox>", Evidence = the subject line). No reminder, no fact, no
note, no calendar event. That bound is the design: a reminder FIRES, so a
1.7B misreading "встреча была в четверг" as a future appointment would wake
him up about it, whereas a wrong candidate is a line he dismisses in one
click. A due date the model read out of the mail is stored on the candidate,
where no scheduler reads it — the review page sorts by it. Relative wording
("до пятницы") is deliberately left in the text rather than resolved to a
date the model would get wrong.

The prompt is written against the two things a small model does here: it
summarises when asked to extract, and it invents an obligation out of a
polite closing line. Hence the demand for a verb phrase, and an explicit
empty array — most mail contains no task, and a model with no way to say
"nothing" says something.

Wiring: core owns extraction because llama-server lives in core's process,
so the reader hands messages over a new ipc.MethodIngestMail. It is a Server
hook (like StepUp/UnlockFn), not a CoreAPI method — not a store operation,
and no CoreAPI implementation should have to carry it. The hook stays nil
without an `email` config block or without a llama-server phraser, so the
method answers ErrUnknownMethod: off unless configured, twice over. There is
no keyword fallback on purpose — "the subject became a task" is a mailbox
rendered as a to-do list, not extraction.

Privacy: junk is refused before the model is called, mail text is never
search input, extraction errors carry byte counts rather than the reply, the
stored evidence is a truncated subject, and the log line names the mailbox
and the UID only.
2026-08-01 03:06:55 +04:00
kami b4646155b4 Read a mailbox read-only, in a client small enough to audit (#246)
internal/email is the reading half of the email reader: a ~200-line IMAP
client (LOGIN, EXAMINE, UID SEARCH SINCE, UID FETCH BODY.PEEK, LOGOUT), a
MIME-to-plaintext converter, and a header-only junk filter.

Two protocol choices are the design, not shortcuts. EXAMINE instead of
SELECT means the session is read-only at the protocol level, so no command
in it can flip a flag or expunge anything by mistake. BODY.PEEK instead of
BODY means reading a message does not mark it \Seen — Maven reads his mail
and leaves no trace of having done so, and the unread state in his own
client stays his.

Hand-rolled rather than go-imap because this is the one path that holds his
mailbox credential and reads his private mail: five commands with no
dependencies is auditable in a sitting. No IDLE and no cleartext/STARTTLS
either — an option to send his password over a plain socket is an option to
get it wrong once.

Junk is decided by headers alone, before any model is involved:
List-Unsubscribe/List-Id, Precedence: bulk, Auto-Submitted, the spam
headers, and Gmail's own category labels. Sender lists and subject keywords
are deliberately absent — they age badly and they would put his contacts in
a config file. A junk verdict only means "do not spend the model on this";
nothing is deleted and no server flag is touched.

Nothing here logs a body, a subject or an address, the junk reason names a
header rather than content, and an undecodable charset degrades to
headers-only instead of feeding the model mojibake. Verified against
recorded .eml fixtures and an in-process fake IMAP server.
2026-08-01 02:59:24 +04:00
kami da647e87d0 Read spending from zenmoney in the poller, answer it from facts (#125)
The trust boundary is zenmoney, not maven — they already hold his bank
sessions. So the poller reads /v8/diff/ and writes totals as
facts(kind=env, source=poll:zenmoney); core reads those back when he asks
and never sees the token.

internal/zenmoney sums transactions per currency over a window, skipping
tombstoned rows and transfers between his own accounts, and refuses to
encode a summary built from zero transactions. That refusal is the whole
design: a failed or empty read writes nothing and leaves the last good
total alone, because a zero recited as fact is worse than silence. No
currency conversion either — a figure he can check against his bank beats
one he cannot.

Off unless configured, and the token is read from a FILE rather than a
flag so it never lands in `ps`, in docker-compose.yml, or in shell
history. Nothing about the money is search input, no tick rule reads the
keys, and the log lines name keys, never figures.

The live-credential half is BLOCKED: there is no zenmoney account or token
here, so everything is verified against a recorded diff fixture.
2026-08-01 02:50:27 +04:00
43 changed files with 3823 additions and 12 deletions
+5
View File
@@ -7,6 +7,7 @@
/mavpoll
/mavcaldav
/mavwaked
/mavmaild
# Certs (private keys, don't commit)
certs/
@@ -34,6 +35,10 @@ deps
deploy/db_key.env
# Deploy secret (telegram bot token + chat id) — never commit
deploy/telegram.env
# zenmoney API token, read by mavpoll (never in argv, never committed)
deploy/zenmoney.token
# IMAP password, read by mavmaild (never in argv, never committed)
deploy/imap.password
# Temp files
/tmp/
+2 -1
View File
@@ -34,7 +34,7 @@ CGO daemons (`mavend`, `mavsttd`, `mavttsd`, `mavenclient`) need the vendored to
and libs wired through the Makefile — **do not** call `go build` on them bare, use `make`:
```sh
make build # all 8 binaries
make build # all 9 binaries
make build-web # single daemon (pure-Go ones: web/waked/poll/caldav build without CGO)
make test # go test -race across ./internal/... ./cmd/... with CGO env set
```
@@ -62,6 +62,7 @@ Pure-Go packages (`router`, `memory`, `mavweb`, …) run under a plain `go test
| `mavenclient` | Voice loop client (mic → stt → core → tts). |
| `mavpoll` | Telegram long-poll reach. |
| `mavcaldav` | CalDAV calendar sync. |
| `mavmaild` | Mail reader (IMAP, read-only). Holds the IMAP password; core never sees it. |
Daemons are wired socket-to-socket, not linked. `internal/ipc` is the client/server wire
protocol; the config in `deploy/mavend.json` (with `${VAR}` env expansion from gitignored
+2 -1
View File
@@ -51,7 +51,8 @@ RUN go build -o /out/mavend ./cmd/mavend && \
go build -o /out/mavttsd ./cmd/mavttsd && \
go build -o /out/mavweb ./cmd/mavweb && \
go build -o /out/mavpoll ./cmd/mavpoll && \
go build -o /out/mavcaldav ./cmd/mavcaldav
go build -o /out/mavcaldav ./cmd/mavcaldav && \
go build -o /out/mavmaild ./cmd/mavmaild
# llama.cpp Vulkan build — the phraser/router LFM engine (llama-server). Built
# from source (not a prebuilt vendored blob) so the binary's glibc/GLIBCXX match
+5 -2
View File
@@ -20,7 +20,7 @@ PIPER_ESPEAK := $(shell pwd)/deps/piper/espeak-ng-data
all: build
build: build-stt build-tts build-daemon build-client build-waked build-web build-poll build-caldav
build: build-stt build-tts build-daemon build-client build-waked build-web build-poll build-caldav build-mail
build-stt:
CGO_CFLAGS="$(CGO_CFLAGS)" CGO_LDFLAGS="$(CGO_LDFLAGS)" LD_LIBRARY_PATH="$(shell pwd)/deps/lib" \
@@ -50,6 +50,9 @@ build-poll:
build-caldav:
$(GO) build $(GOFLAGS) -o mavcaldav ./cmd/mavcaldav/
build-mail:
$(GO) build $(GOFLAGS) -o mavmaild ./cmd/mavmaild/
run-web: build-web
./mavweb -addr :9200 -voice 127.0.0.1:9100
@@ -185,4 +188,4 @@ download-embedder:
@echo ' sudo cp onnxruntime-linux-x64-1.15.1/lib/libonnxruntime.so* /usr/local/lib/'
clean:
rm -f mavend mavenclient mavsttd mavttsd mavweb mavpoll mavcaldav mavwaked
rm -f mavend mavenclient mavsttd mavttsd mavweb mavpoll mavcaldav mavwaked mavmaild
+78
View File
@@ -0,0 +1,78 @@
package main
import (
"context"
"log"
"github.com/kami/maven/internal/ipc"
"github.com/kami/maven/internal/router"
"github.com/kami/maven/internal/zenmoney"
)
// Money questions (Vikunja #125).
//
// This is the whole read side: mavpoll holds the zenmoney token and writes
// facts(kind=env, source=poll:zenmoney); core reads them back when he asks.
// Core never sees the token, never calls zenmoney, and has no rule on these
// keys — a total is never a reason for Maven to speak first. Maven is not a
// nag, least of all about his money.
//
// Nothing here can reach the external search capability: the figures are read
// from the store and rendered locally, and his financial data is never search
// input.
// queryMoney — "сколько я потратил сегодня?", "покажи мои траты".
//
// Answers only from the latest fact the poller wrote. Three honest outcomes and
// no fourth: the figure, "the fact is old and here is its date", or "money
// tracking is not connected". It never computes, estimates or rounds a total of
// its own — an invented number about his money is the worst thing this could do.
func (h *reactiveHandler) queryMoney(ctx context.Context, t *queryTurn) (string, bool) {
window, ok := router.ParseMoneyQuery(t.dec.Utterance)
if !ok {
return "", false
}
key, phrase := zenmoney.KeySpentMonth, "в этом месяце"
if window == router.MoneyToday {
key, phrase = zenmoney.KeySpentToday, "сегодня"
}
fact, err := h.api.LatestFactBySource(ctx, key, zenmoney.Source)
if err != nil {
// No fact at all is the normal state when the capability is off. Claim
// the turn anyway: falling through to recall would answer a question
// about money with whatever note happens to be nearest.
if !isNoFactErr(err) {
log.Printf("voice: money fact: %v", err)
}
return "я не отслеживаю траты — не подключено.", true
}
val, err := zenmoney.ParseFactValue(fact.Value)
if err != nil {
log.Printf("voice: money fact: decode: %v", err)
return "не получилось прочитать траты.", true
}
reply := val.FormatRU(phrase)
if reply == "" {
return "по тратам пока нечего сказать.", true
}
// A stale fact is reported as stale rather than spoken as today's number.
if h.now().Sub(fact.Ts) > zenmoney.StaleAfter {
return "данные от " + fact.Ts.Local().Format("02.01") + ": " + reply, true
}
return reply, true
}
// isNoFactErr — ErrNoFact survives the wire wrapped, so unwrap for it.
func isNoFactErr(err error) bool {
for e := err; e != nil; {
if e == ipc.ErrNoFact {
return true
}
u, ok := e.(interface{ Unwrap() error })
if !ok {
return false
}
e = u.Unwrap()
}
return false
}
+139
View File
@@ -0,0 +1,139 @@
package main
import (
"context"
"strings"
"testing"
"time"
"github.com/kami/maven/internal/ipc"
"github.com/kami/maven/internal/router"
"github.com/kami/maven/internal/zenmoney"
)
// moneyAPI answers only LatestFactBySource; everything else is unimplemented,
// which is the assertion that answering a money question costs no model call
// and reaches no network.
type moneyAPI struct {
ipc.UnimplementedCoreAPI
fact ipc.Fact
err error
gotKey string
gotSrc string
callCnt int
}
func (a *moneyAPI) LatestFactBySource(_ context.Context, key, source string) (ipc.Fact, error) {
a.gotKey, a.gotSrc = key, source
a.callCnt++
return a.fact, a.err
}
func moneyNow() time.Time { return time.Date(2026, 8, 15, 20, 0, 0, 0, time.UTC) }
func moneyFact(ts time.Time, val string) ipc.Fact {
return ipc.Fact{Kind: "env", Key: zenmoney.KeySpentMonth, Value: val, Source: zenmoney.Source, Ts: ts}
}
func TestQueryMoneyAnswersFromTheFact(t *testing.T) {
api := &moneyAPI{fact: moneyFact(moneyNow(), `{"spent":[{"currency":"RUB","amount":1749.5}],"count":3}`)}
h := &reactiveHandler{api: api, now: moneyNow}
reply, ok := h.queryMoney(context.Background(), &queryTurn{
dec: router.Decision{Utterance: "сколько я потратил в этом месяце?"},
})
if !ok {
t.Fatal("the money source must claim a money question")
}
if api.gotKey != zenmoney.KeySpentMonth || api.gotSrc != zenmoney.Source {
t.Errorf("read %q/%q, want the month key from the poller's source", api.gotKey, api.gotSrc)
}
if !strings.Contains(reply, "1749.5") {
t.Errorf("reply = %q, want the exact figure", reply)
}
if !strings.Contains(reply, "в этом месяце") {
t.Errorf("reply = %q, want the window named", reply)
}
}
func TestQueryMoneyPicksTodaysKey(t *testing.T) {
api := &moneyAPI{fact: moneyFact(moneyNow(), `{"spent":[{"currency":"RUB","amount":250}],"count":1}`)}
h := &reactiveHandler{api: api, now: moneyNow}
if _, ok := h.queryMoney(context.Background(), &queryTurn{
dec: router.Decision{Utterance: "сколько я потратил сегодня?"},
}); !ok {
t.Fatal("expected the source to claim it")
}
if api.gotKey != zenmoney.KeySpentToday {
t.Errorf("key = %q, want today's", api.gotKey)
}
}
// The capability is off unless configured, and then there is no fact. She says
// so instead of letting the recall pass answer a money question from a note.
func TestQueryMoneySaysNotConnected(t *testing.T) {
h := &reactiveHandler{api: &moneyAPI{err: ipc.ErrNoFact}, now: moneyNow}
reply, ok := h.queryMoney(context.Background(), &queryTurn{
dec: router.Decision{Utterance: "сколько я потратил?"},
})
if !ok {
t.Fatal("expected the source to claim it")
}
if !strings.Contains(reply, "не подключено") {
t.Errorf("reply = %q, want an honest 'not connected'", reply)
}
// No number of any kind in that answer.
for _, d := range []string{"0", "1", "2", "3", "4", "5", "6", "7", "8", "9"} {
if strings.Contains(reply, d) {
t.Errorf("reply %q contains a digit — nothing was read, so there is no figure", reply)
}
}
}
// A fact older than the staleness bound is dated rather than spoken as if it
// were current: the poller can be down, and last week's total presented as
// today's is a lie by omission.
func TestQueryMoneyDatesAStaleFact(t *testing.T) {
old := moneyNow().Add(-72 * time.Hour)
api := &moneyAPI{fact: moneyFact(old, `{"spent":[{"currency":"RUB","amount":100}],"count":1}`)}
h := &reactiveHandler{api: api, now: moneyNow}
reply, _ := h.queryMoney(context.Background(), &queryTurn{
dec: router.Decision{Utterance: "сколько я потратил?"},
})
if !strings.Contains(reply, "данные от") {
t.Errorf("reply = %q, want the stale fact dated", reply)
}
}
func TestQueryMoneyPassesOtherQuestions(t *testing.T) {
api := &moneyAPI{}
h := &reactiveHandler{api: api, now: moneyNow}
for _, u := range []string{"какая погода?", "я потратил весь день на это", "какие у меня задачи?"} {
if _, ok := h.queryMoney(context.Background(), &queryTurn{dec: router.Decision{Utterance: u}}); ok {
t.Errorf("the money source claimed %q", u)
}
}
if api.callCnt != 0 {
t.Error("a non-money question must not read the money facts")
}
}
// Money must be answered before the recall sources, or a question about
// spending gets answered by the nearest note.
func TestQuerySourcesOrderMoneyBeforeRecall(t *testing.T) {
moneyAt, notesAt := -1, -1
for i, src := range querySources {
switch src.name {
case "money":
moneyAt = i
case "notes":
notesAt = i
}
}
if moneyAt < 0 || notesAt < 0 {
t.Fatalf("sources missing: money=%d notes=%d", moneyAt, notesAt)
}
if moneyAt > notesAt {
t.Errorf("money source at %d, after notes at %d", moneyAt, notesAt)
}
}
+5
View File
@@ -62,6 +62,11 @@ var querySources = []querySource{
// matcher requires a task noun or an explicit "что … сделать", so a
// date-bearing question still reaches the calendar.
{"tasks", (*reactiveHandler).queryTasks},
// Before the recall sources too: "сколько я потратил?" is a question about
// the money facts the poller wrote, and the notes pass would otherwise
// answer it from whatever he once said about spending. Its matcher needs a
// money noun plus an actual ask, so "я потратил весь день" is untouched.
{"money", (*reactiveHandler).queryMoney},
{"calendar", (*reactiveHandler).queryCalendar},
{"weather", (*reactiveHandler).queryWeather},
{"embed", (*reactiveHandler).queryEmbed},
+158
View File
@@ -0,0 +1,158 @@
// mavend/mail.go — core's half of the email reader (Vikunja #246,
// docs/plans/01-email-reader.md).
//
// The split: cmd/mavmaild holds the IMAP credential, connects to the mailbox
// and converts messages to plaintext; it hands each message to core over
// ipc.MethodIngestMail. Core runs the extraction on the resident model —
// llama-server lives in this process, spawned by the phraser — and writes what
// comes back through the one task intake seam.
//
// What this file may produce is exactly one thing: rows in `tasks` with status
// "candidate". No fact, no reminder, no note, no nudge, no calendar event. A
// 1.7B misreading a mail can therefore put a wrong line on a review page and
// nothing else; it can never make Maven speak, and it can never make her
// recite something out of an advert as true.
//
// Off unless configured twice over: no `email` block in mavend.json ⇒ the IPC
// method does not exist; no llama-server phraser ⇒ same. A reader pointed at a
// core that is not set up for mail gets ErrUnknownMethod rather than silence.
package main
import (
"context"
"fmt"
"log"
"time"
"github.com/kami/maven/internal/config"
"github.com/kami/maven/internal/email"
"github.com/kami/maven/internal/ipc"
"github.com/kami/maven/internal/llm"
"github.com/kami/maven/internal/phraser"
"github.com/kami/maven/internal/store"
)
// evidenceMaxChars — how much of the subject line is kept as a candidate's
// evidence. Enough to recognise the mail on /tasks, not enough to turn the task
// list into a copy of his mailbox.
const evidenceMaxChars = 160
// mailIntake — extraction + capture for one message at a time.
type mailIntake struct {
st *store.Store
ex *email.Extractor
timeout time.Duration
now func() time.Time
}
// newMailIntake returns nil when mail ingestion must not be available, which is
// the default. Both preconditions are real:
//
// - no cfg.Email ⇒ not configured, and a capability is off unless configured;
// - no llama-server phraser ⇒ nothing to extract with. There is deliberately
// no keyword fallback: "the subject line became a task" is not extraction,
// it is a mailbox rendered as a to-do list, and it would fill the review
// page faster than he could clear it.
func newMailIntake(st *store.Store, phr phraser.Phraser, cfg *config.Config) *mailIntake {
if cfg.Email == nil {
return nil
}
lp, ok := phr.(*phraser.LLMPhraser)
if !ok {
log.Printf("mail intake: configured but no llama-server phraser — mail ingestion disabled")
return nil
}
timeout := time.Duration(cfg.Email.Timeout)
if timeout <= 0 {
timeout = config.DefaultEmailTimeout
}
ex := email.NewExtractor(llm.New(lp.BaseURL(), timeout), cfg.Email.MaxTasks, contextBlockFn(cfg, time.Now))
log.Printf("mail intake: enabled (max %d candidates per message, timeout %s)", cfg.Email.MaxTasks, timeout)
return &mailIntake{st: st, ex: ex, timeout: timeout, now: time.Now}
}
// ingest handles one ipc.MethodIngestMail call.
//
// Junk and empty messages are answered Skipped without touching the model — the
// reader's header filter is what keeps the resident model off newsletters.
//
// Every candidate is captured with Status "candidate", Source "email:<mailbox>"
// and the subject as Evidence. CaptureTask dedupes on normalised text among
// live rows, so a mailbox re-read after a restart produces Created=0 rather
// than a second copy of every task.
func (m *mailIntake) ingest(ctx context.Context, req ipc.IngestMailReq) (ipc.IngestMailResp, error) {
msg := email.Message{
UID: req.UID,
From: req.From,
Subject: req.Subject,
Date: req.Date,
Body: req.Body,
Junk: req.Junk,
}
if msg.Junk || (msg.Subject == "" && msg.Body == "") {
return ipc.IngestMailResp{Skipped: true}, nil
}
ctx, cancel := context.WithTimeout(ctx, m.timeout)
defer cancel()
cands, err := m.ex.Extract(ctx, msg)
if err != nil {
// The error from internal/email never carries mail text; keep it that way
// by not adding the subject here.
return ipc.IngestMailResp{}, fmt.Errorf("mail intake: uid %d: %w", req.UID, err)
}
if len(cands) == 0 {
return ipc.IngestMailResp{}, nil
}
source := email.SourcePrefix + req.Mailbox
evidence := truncateRunes(req.Subject, evidenceMaxChars)
now := m.now()
var resp ipc.IngestMailResp
for _, c := range cands {
t := store.Task{
CreatedTs: now,
Text: c.Text,
Source: source,
Evidence: evidence,
// The one status this path may ever write. Anything Maven derived from
// something she read is a suggestion until he confirms it on /tasks.
Status: store.TaskCandidate,
}
if due, ok := email.ParseDue(c.Due); ok {
t.Due = &due
}
id, created, err := m.st.CaptureTask(ctx, t)
if err != nil {
return resp, fmt.Errorf("mail intake: capture: %w", err)
}
resp.TaskIDs = append(resp.TaskIDs, id)
if created {
resp.Created++
}
}
// Counts only: the log line names the mailbox and the UID, never the subject,
// the sender or the task text. Reviewing a candidate is what /tasks is for.
log.Printf("mail intake: %s uid %d → %d candidate(s), %d new", source, req.UID, len(cands), resp.Created)
return resp, nil
}
// wireMailIntake installs the IPC hook, or leaves it nil so the method reports
// ErrUnknownMethod. Called on both startup paths (unlocked boot and passkey
// unlock) so mail behaves the same either way.
func wireMailIntake(srv *ipc.Server, st *store.Store, phr phraser.Phraser, cfg *config.Config) {
mi := newMailIntake(st, phr, cfg)
if mi == nil {
return
}
srv.IngestMailFn = mi.ingest
}
// truncateRunes cuts a string to n runes, marking the cut.
func truncateRunes(s string, n int) string {
r := []rune(s)
if len(r) <= n {
return s
}
return string(r[:n]) + "…"
}
+182
View File
@@ -0,0 +1,182 @@
package main
import (
"context"
"strings"
"testing"
"time"
"github.com/kami/maven/internal/config"
"github.com/kami/maven/internal/email"
"github.com/kami/maven/internal/ipc"
"github.com/kami/maven/internal/llm"
"github.com/kami/maven/internal/store"
)
// mailLLM — a canned extraction reply.
type mailLLM struct {
reply string
calls int
}
func (m *mailLLM) Complete(_ context.Context, _ llm.Req) (string, error) {
m.calls++
return m.reply, nil
}
func newTestIntake(t *testing.T, reply string) (*mailIntake, *store.Store, *mailLLM) {
t.Helper()
st := newTestStore(t)
fake := &mailLLM{reply: reply}
return &mailIntake{
st: st,
ex: email.NewExtractor(fake, 0, nil),
timeout: 5 * time.Second,
now: func() time.Time { return time.Date(2026, 8, 1, 10, 0, 0, 0, time.UTC) },
}, st, fake
}
func ingestReq() ipc.IngestMailReq {
return ipc.IngestMailReq{
Mailbox: "INBOX", UID: 42,
From: "billing@isp.example",
Subject: "Счёт за интернет",
Body: "Оплатите счёт до 5 августа.",
}
}
// The one property that matters: a mail-derived task is a candidate, attributed
// to the mailbox, with the subject as reviewable evidence — and nothing else is
// written.
func TestIngestCapturesCandidates(t *testing.T) {
mi, st, _ := newTestIntake(t, `[{"text":"оплатить счёт за интернет","due":"2026-08-05"}]`)
resp, err := mi.ingest(context.Background(), ingestReq())
if err != nil {
t.Fatalf("ingest: %v", err)
}
if resp.Created != 1 || len(resp.TaskIDs) != 1 {
t.Fatalf("resp = %+v, want one created task", resp)
}
tasks, err := st.ListTasks(context.Background(), "")
if err != nil {
t.Fatalf("list: %v", err)
}
if len(tasks) != 1 {
t.Fatalf("got %d tasks, want 1", len(tasks))
}
got := tasks[0]
if got.Status != store.TaskCandidate {
t.Errorf("status = %q, want %q — mail may only produce candidates", got.Status, store.TaskCandidate)
}
if got.Source != "email:INBOX" {
t.Errorf("source = %q, want email:INBOX", got.Source)
}
if got.Evidence != "Счёт за интернет" {
t.Errorf("evidence = %q, want the subject line", got.Evidence)
}
if got.Due == nil || got.Due.Format("2006-01-02") != "2026-08-05" {
t.Errorf("due = %v, want 2026-08-05", got.Due)
}
// Nothing else may have been written: no reminder, no fact.
rem, err := st.ListReminders(context.Background(), 10)
if err != nil {
t.Fatalf("list reminders: %v", err)
}
if len(rem) != 0 {
t.Errorf("mail created %d reminders; a misread mail must never be able to fire", len(rem))
}
}
// Re-reading a mailbox must not grow the list — CaptureTask dedupes among live
// rows, and the intake relies on exactly that.
func TestIngestSameMailTwiceIsIdempotent(t *testing.T) {
mi, st, _ := newTestIntake(t, `[{"text":"оплатить счёт","due":""}]`)
if _, err := mi.ingest(context.Background(), ingestReq()); err != nil {
t.Fatalf("first ingest: %v", err)
}
resp, err := mi.ingest(context.Background(), ingestReq())
if err != nil {
t.Fatalf("second ingest: %v", err)
}
if resp.Created != 0 || len(resp.TaskIDs) != 1 {
t.Errorf("resp = %+v, want the existing row and Created=0", resp)
}
tasks, _ := st.ListTasks(context.Background(), "")
if len(tasks) != 1 {
t.Errorf("got %d tasks after two reads, want 1", len(tasks))
}
}
func TestIngestJunkSkipsTheModel(t *testing.T) {
mi, st, fake := newTestIntake(t, `[{"text":"купить со скидкой","due":""}]`)
req := ingestReq()
req.Junk = true
resp, err := mi.ingest(context.Background(), req)
if err != nil {
t.Fatalf("ingest: %v", err)
}
if !resp.Skipped || resp.Created != 0 {
t.Errorf("resp = %+v, want skipped", resp)
}
if fake.calls != 0 {
t.Errorf("model called %d times for junk, want 0", fake.calls)
}
if tasks, _ := st.ListTasks(context.Background(), ""); len(tasks) != 0 {
t.Errorf("junk produced %d tasks, want 0", len(tasks))
}
}
func TestIngestEmptyMessageSkipped(t *testing.T) {
mi, _, fake := newTestIntake(t, "[]")
resp, err := mi.ingest(context.Background(), ipc.IngestMailReq{Mailbox: "INBOX", UID: 1})
if err != nil || !resp.Skipped {
t.Fatalf("resp = %+v, err = %v; want skipped", resp, err)
}
if fake.calls != 0 {
t.Errorf("model called %d times for an empty message, want 0", fake.calls)
}
}
func TestIngestNoTasksWritesNothing(t *testing.T) {
mi, st, _ := newTestIntake(t, "[]")
resp, err := mi.ingest(context.Background(), ingestReq())
if err != nil {
t.Fatalf("ingest: %v", err)
}
if resp.Created != 0 || len(resp.TaskIDs) != 0 || resp.Skipped {
t.Errorf("resp = %+v, want nothing captured and not skipped", resp)
}
if tasks, _ := st.ListTasks(context.Background(), ""); len(tasks) != 0 {
t.Errorf("got %d tasks, want 0", len(tasks))
}
}
func TestIngestTruncatesEvidence(t *testing.T) {
mi, st, _ := newTestIntake(t, `[{"text":"дело","due":""}]`)
req := ingestReq()
req.Subject = strings.Repeat("щ", 400)
if _, err := mi.ingest(context.Background(), req); err != nil {
t.Fatalf("ingest: %v", err)
}
tasks, _ := st.ListTasks(context.Background(), "")
if len(tasks) != 1 {
t.Fatalf("got %d tasks, want 1", len(tasks))
}
if n := len([]rune(tasks[0].Evidence)); n > evidenceMaxChars+1 {
t.Errorf("evidence kept %d runes, want ≤ %d", n, evidenceMaxChars)
}
}
// Off unless configured: no email block ⇒ no intake, so the IPC method does not
// exist at all.
func TestNewMailIntakeOffWithoutConfig(t *testing.T) {
st := newTestStore(t)
if mi := newMailIntake(st, nil, &config.Config{}); mi != nil {
t.Error("no email block must mean no mail intake")
}
// Configured but with a non-LLM phraser: still off — there is no fallback
// extraction, by design.
if mi := newMailIntake(st, nil, &config.Config{Email: &config.EmailConfig{}}); mi != nil {
t.Error("without a llama-server phraser there is nothing to extract with")
}
}
+8
View File
@@ -320,6 +320,13 @@ func run(args []string) error {
srv.StepUp = func(ctx context.Context) error { return passkeySess.Assert(ctx, auth.Scope{}) }
// Mail ingestion (Vikunja #246): the hook stays nil unless an email block is
// configured and there is a llama-server to extract with, in which case
// ipc.MethodIngestMail reports ErrUnknownMethod.
if !locked {
wireMailIntake(srv, st, phr, cfg)
}
// WrapKeyFn — wraps the env key with a passkey credential public key and
// persists the wrapped blob. Only wired when the daemon has the key in
// memory (env key mode). Called by mavweb after passkey enrollment.
@@ -457,6 +464,7 @@ func run(args []string) error {
}
srv.SetAPI(newAPI)
srv.Check = (&auth.Gate{Enrollment: auth.NewFloorEnrollment(), Session: passkeySess}).Check
wireMailIntake(srv, st, phr, cfg)
// Start voice server.
if voiceW != nil {
+332
View File
@@ -0,0 +1,332 @@
// mavmaild — the mail reader module (Vikunja #246,
// docs/plans/01-email-reader.md).
//
// Every so often it opens one IMAP mailbox read-only, fetches the messages it
// has not read yet, and hands each one to core over ipc.MethodIngestMail. Core
// runs the extraction on the resident model and writes what comes back as task
// CANDIDATES he reviews on /tasks. Nothing here writes to the store, nothing
// here can create a reminder, and nothing here speaks.
//
// Why a separate daemon rather than a loop inside mavend, when extraction has
// to happen in mavend anyway: the credential. mavpoll set the precedent with the
// zenmoney token (#125) — the module that talks to a third party holds the
// secret, reads it from a FILE so it never appears in `ps`, in
// docker-compose.yml or in shell history, and core never sees it. Core learns
// that mail exists only as message text on one IPC method; it cannot connect to
// the mailbox even if it wanted to, and a compromised core yields no mail
// password.
//
// Off unless configured: without -password-file there is nothing to run, and
// the daemon says so and exits. If core has no `email` block the very first
// ingest comes back ErrUnknownMethod and this daemon stops polling instead of
// hammering a socket that will keep refusing.
//
// Mail is personal, so the log is counts and UIDs: how many messages were
// fetched, how many were bulk, how many candidates came back. No subject, no
// sender, no body, ever — reviewing a candidate is what /tasks is for.
package main
import (
"context"
"encoding/json"
"errors"
"flag"
"fmt"
"log"
"os"
"os/signal"
"path/filepath"
"sort"
"strings"
"syscall"
"time"
"github.com/kami/maven/internal/email"
"github.com/kami/maven/internal/ipc"
)
func main() {
if err := run(os.Args[1:]); err != nil {
fmt.Fprintln(os.Stderr, "mavmaild:", err)
os.Exit(1)
}
}
func run(args []string) error {
fs := flag.NewFlagSet("mavmaild", flag.ContinueOnError)
socket := fs.String("socket", "", "core IPC socket path (required)")
server := fs.String("imap", "", "IMAP server, host or host:993 (required)")
user := fs.String("user", "", "IMAP username (required)")
passFile := fs.String("password-file", "", "file holding the IMAP password (required — never passed as a flag value)")
mailbox := fs.String("mailbox", "INBOX", "mailbox to read, read-only")
interval := fs.Duration("interval", 15*time.Minute, "how often to read the mailbox")
lookback := fs.Duration("lookback", 72*time.Hour, "how far back to search on each poll")
max := fs.Int("max", 25, "most messages to fetch in one poll")
timeout := fs.Duration("timeout", 30*time.Second, "IMAP network timeout")
statePath := fs.String("state", "", "file remembering which UIDs were read (default: none — every poll re-reads the window)")
if err := fs.Parse(args); err != nil {
return err
}
if *socket == "" {
return fmt.Errorf("-socket is required")
}
if *server == "" || *user == "" || *passFile == "" {
return fmt.Errorf("mail reading is off unless configured: set -imap, -user and -password-file")
}
// The password is read from a file, never taken as a flag value: an argv
// secret is visible in `ps` to every user on the box and lands in the compose
// file and the shell history. Read once at start — a rotated password means a
// restart, which is cheaper than re-reading his credential every quarter hour.
raw, err := os.ReadFile(*passFile)
if err != nil {
return fmt.Errorf("read password file: %w", err)
}
password := strings.TrimSpace(string(raw))
if password == "" {
return fmt.Errorf("password file %s is empty", *passFile)
}
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
defer stop()
core, err := ipc.DialWait(*socket, 60*time.Second)
if err != nil {
return err
}
defer core.Close()
r := &reader{
core: core,
addr: *server,
user: *user,
mailbox: *mailbox,
lookback: *lookback,
max: *max,
timeout: *timeout,
state: newSeenState(*statePath),
}
if err := r.state.load(); err != nil {
// A missing or corrupt state file must not stop mail from being read: the
// worst case is re-reading the window, and capture dedupes on text.
log.Printf("mavmaild: state: %v (starting from an empty seen-set)", err)
}
// The password is never logged, not even its length.
log.Printf("mavmaild: reading %s on %s every %s (lookback %s, max %d/poll)",
*mailbox, *server, *interval, *lookback, *max)
r.pollOnce(ctx, password) // don't idle a full interval on start
t := time.NewTicker(*interval)
defer t.Stop()
for {
select {
case <-ctx.Done():
log.Printf("mavmaild: bye")
return nil
case <-t.C:
if r.disabled {
// Core told us mail ingestion is not configured. Nothing will change
// without a core restart, and a restart restarts us too.
log.Printf("mavmaild: core does not accept mail — idling")
return nil
}
r.pollOnce(ctx, password)
}
}
}
// mailIngester — the slice of core this daemon uses. One method: hand over a
// message. It cannot write a fact, create a reminder or read the store, and the
// interface says so.
type mailIngester interface {
IngestMail(ctx context.Context, req ipc.IngestMailReq) (ipc.IngestMailResp, error)
}
type reader struct {
core mailIngester
addr string
user string
mailbox string
lookback time.Duration
max int
timeout time.Duration
state *seenState
// dial — connection seam for the tests; nil ⇒ implicit TLS.
dial func(addr string, timeout time.Duration) (*email.Conn, error)
// disabled — core answered ErrUnknownMethod, i.e. it has no email block.
disabled bool
}
// pollOnce — one read of the mailbox, then one ingest per message.
//
// A fetch error aborts this poll and nothing else; the next tick tries again.
// An ingest error for one message does not skip the rest — one mail the model
// choked on should not hide the four behind it.
func (r *reader) pollOnce(ctx context.Context, password string) {
msgs, err := r.fetch(password)
if err != nil {
// The error may name a UID; it never names a subject or a sender.
log.Printf("mavmaild: fetch: %v", err)
if len(msgs) == 0 {
return
}
}
var junk, candidates, created int
for _, m := range msgs {
if ctx.Err() != nil {
return
}
if m.Junk {
junk++
// Marked seen without a model call: the header filter already decided,
// and re-classifying it every quarter hour would be pure waste.
r.state.mark(m.UID)
continue
}
resp, err := r.core.IngestMail(ctx, ipc.IngestMailReq{
Mailbox: r.mailbox,
UID: m.UID,
From: m.From,
Subject: m.Subject,
Date: m.Date,
Body: m.Body,
})
if errors.Is(err, ipc.ErrUnknownMethod) {
log.Printf("mavmaild: core has no email block configured — mail ingestion is off; stopping")
r.disabled = true
return
}
if err != nil {
// Not marked seen: an ingest that failed should be retried next poll.
log.Printf("mavmaild: ingest uid %d: %v", m.UID, err)
continue
}
r.state.mark(m.UID)
candidates += len(resp.TaskIDs)
created += resp.Created
}
if err := r.state.save(); err != nil {
log.Printf("mavmaild: state: %v", err)
}
log.Printf("mavmaild: %s: %d read, %d bulk, %d candidate(s), %d new", r.mailbox, len(msgs), junk, candidates, created)
}
// fetch reads the mailbox. Messages already in the seen-set are not fetched at
// all, so a steady mailbox costs one SEARCH per poll and nothing else.
func (r *reader) fetch(password string) ([]email.Message, error) {
f := email.FetchSince{
Addr: r.addr,
User: r.user,
Mailbox: r.mailbox,
Timeout: r.timeout,
Since: time.Now().Add(-r.lookback),
Max: r.max,
Skip: r.state.seen,
}
return f.RunWith(password, r.dial)
}
// ---- seen state ------------------------------------------------------------
// seenState — the UIDs already handed to core, persisted so a restart does not
// re-read (and re-extract, at multi-second LLM cost) the whole lookback window.
//
// Correctness does not depend on it: ipc.CaptureTask dedupes on normalised text
// among live tasks, so a re-read produces no duplicate rows. This exists to save
// the model's time, which is why a broken state file is a log line rather than a
// failure.
//
// UIDs are per-mailbox and monotonic, so the set is kept as a high-water mark
// plus the stragglers above it. If the server ever changes UIDVALIDITY, UIDs
// reset and the window is simply re-read once — dedupe absorbs it.
type seenState struct {
path string
high uint32
set map[uint32]bool
dirty bool
}
func newSeenState(path string) *seenState {
return &seenState{path: path, set: map[uint32]bool{}}
}
type seenFile struct {
High uint32 `json:"high"`
UIDs []uint32 `json:"uids,omitempty"`
}
func (s *seenState) seen(uid uint32) bool {
return uid <= s.high || s.set[uid]
}
func (s *seenState) mark(uid uint32) {
if s.seen(uid) {
return
}
s.set[uid] = true
s.dirty = true
// Advance the high-water mark through any contiguous run, so the explicit set
// stays small on a mailbox read in order.
for {
next := s.high + 1
if !s.set[next] {
break
}
delete(s.set, next)
s.high = next
}
}
func (s *seenState) load() error {
if s.path == "" {
return nil
}
b, err := os.ReadFile(s.path)
if errors.Is(err, os.ErrNotExist) {
return nil // first run
}
if err != nil {
return err
}
var f seenFile
if err := json.Unmarshal(b, &f); err != nil {
return fmt.Errorf("parse %s: %w", s.path, err)
}
s.high = f.High
for _, u := range f.UIDs {
s.set[u] = true
}
return nil
}
// save writes the state atomically (temp file + rename), 0600: it is a list of
// message ids from his mailbox, which is metadata about his mail.
func (s *seenState) save() error {
if s.path == "" || !s.dirty {
return nil
}
uids := make([]uint32, 0, len(s.set))
for u := range s.set {
uids = append(uids, u)
}
sort.Slice(uids, func(i, j int) bool { return uids[i] < uids[j] })
b, err := json.Marshal(seenFile{High: s.high, UIDs: uids})
if err != nil {
return err
}
tmp := s.path + ".tmp"
if err := os.MkdirAll(filepath.Dir(s.path), 0o700); err != nil {
return err
}
if err := os.WriteFile(tmp, b, 0o600); err != nil {
return err
}
if err := os.Rename(tmp, s.path); err != nil {
return err
}
s.dirty = false
return nil
}
+254
View File
@@ -0,0 +1,254 @@
package main
import (
"bufio"
"context"
"fmt"
"net"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
"time"
"github.com/kami/maven/internal/email"
"github.com/kami/maven/internal/ipc"
)
// ---- a scripted IMAP server, same shape internal/email's tests use ---------
type fakeIMAP struct {
msgs map[uint32]string
uids []uint32
cmds []string
}
func (f *fakeIMAP) serve(c net.Conn) {
defer c.Close()
fmt.Fprint(c, "* OK fake ready\r\n")
r := bufio.NewReader(c)
for {
line, err := r.ReadString('\n')
if err != nil {
return
}
parts := strings.SplitN(strings.TrimRight(line, "\r\n"), " ", 2)
if len(parts) != 2 {
return
}
tag, cmd := parts[0], parts[1]
f.cmds = append(f.cmds, cmd)
upper := strings.ToUpper(cmd)
switch {
case strings.HasPrefix(upper, "LOGIN"), strings.HasPrefix(upper, "EXAMINE"):
fmt.Fprintf(c, "%s OK\r\n", tag)
case strings.HasPrefix(upper, "UID SEARCH"):
var ids []string
for _, u := range f.uids {
ids = append(ids, strconv.FormatUint(uint64(u), 10))
}
fmt.Fprintf(c, "* SEARCH %s\r\n%s OK\r\n", strings.Join(ids, " "), tag)
case strings.HasPrefix(upper, "UID FETCH"):
uid, _ := strconv.ParseUint(strings.Fields(cmd)[2], 10, 32)
if raw, ok := f.msgs[uint32(uid)]; ok {
fmt.Fprintf(c, "* 1 FETCH (UID %d BODY[] {%d}\r\n%s)\r\n", uid, len(raw), raw)
}
fmt.Fprintf(c, "%s OK\r\n", tag)
case strings.HasPrefix(upper, "LOGOUT"):
fmt.Fprintf(c, "* BYE\r\n%s OK\r\n", tag)
return
default:
fmt.Fprintf(c, "%s BAD\r\n", tag)
}
}
}
func (f *fakeIMAP) dial(_ string, timeout time.Duration) (*email.Conn, error) {
cli, srv := net.Pipe()
go f.serve(srv)
return email.NewConn(cli, timeout)
}
// ---- a fake core -----------------------------------------------------------
type fakeCore struct {
got []ipc.IngestMailReq
resp ipc.IngestMailResp
err error
}
func (c *fakeCore) IngestMail(_ context.Context, req ipc.IngestMailReq) (ipc.IngestMailResp, error) {
c.got = append(c.got, req)
if c.err != nil {
return ipc.IngestMailResp{}, c.err
}
return c.resp, nil
}
func mail(subject, body string, extraHeaders ...string) string {
h := "Subject: " + subject + "\r\nFrom: a@b.c\r\nContent-Type: text/plain; charset=utf-8\r\n"
for _, e := range extraHeaders {
h += e + "\r\n"
}
return h + "\r\n" + body + "\r\n"
}
func newTestReader(t *testing.T, f *fakeIMAP, core *fakeCore, statePath string) *reader {
t.Helper()
return &reader{
core: core, addr: "mail.example:993", user: "kami", mailbox: "INBOX",
lookback: 72 * time.Hour, max: 25, timeout: 5 * time.Second,
state: newSeenState(statePath),
dial: f.dial,
}
}
func TestPollHandsMessagesToCore(t *testing.T) {
f := &fakeIMAP{
uids: []uint32{1, 2},
msgs: map[uint32]string{
1: mail("Счёт", "Оплатить до 5 августа."),
2: mail("Скидки", "Sale!", "List-Unsubscribe: <mailto:u@x>"),
},
}
core := &fakeCore{resp: ipc.IngestMailResp{TaskIDs: []int64{1}, Created: 1}}
r := newTestReader(t, f, core, "")
r.pollOnce(context.Background(), "secret")
// The newsletter is filtered before core is asked: only the real mail crosses.
if len(core.got) != 1 {
t.Fatalf("core saw %d messages, want 1 (the bulk one must not cross): %+v", len(core.got), core.got)
}
got := core.got[0]
if got.UID != 1 || got.Mailbox != "INBOX" || got.Subject != "Счёт" {
t.Errorf("ingest req = %+v", got)
}
if !strings.Contains(got.Body, "Оплатить") {
t.Errorf("body = %q", got.Body)
}
}
// A second poll must not re-send what core already saw — extraction is a
// multi-second LLM call per message.
func TestPollSkipsSeenUIDs(t *testing.T) {
f := &fakeIMAP{uids: []uint32{5}, msgs: map[uint32]string{5: mail("Счёт", "текст")}}
core := &fakeCore{}
r := newTestReader(t, f, core, "")
r.pollOnce(context.Background(), "secret")
r.pollOnce(context.Background(), "secret")
if len(core.got) != 1 {
t.Errorf("core saw %d messages over two polls, want 1", len(core.got))
}
}
// An ingest that failed is NOT marked seen: the next poll retries it.
func TestPollRetriesFailedIngest(t *testing.T) {
f := &fakeIMAP{uids: []uint32{5}, msgs: map[uint32]string{5: mail("Счёт", "текст")}}
core := &fakeCore{err: fmt.Errorf("llama-server is warming up")}
r := newTestReader(t, f, core, "")
r.pollOnce(context.Background(), "secret")
core.err = nil
r.pollOnce(context.Background(), "secret")
if len(core.got) != 2 {
t.Errorf("core saw %d attempts, want 2 (a failed ingest is retried)", len(core.got))
}
}
// Core without an email block ⇒ stop, don't hammer the socket.
func TestPollStopsWhenCoreRefusesMail(t *testing.T) {
f := &fakeIMAP{uids: []uint32{1, 2}, msgs: map[uint32]string{1: mail("a", "b"), 2: mail("c", "d")}}
core := &fakeCore{err: fmt.Errorf("call: %w", ipc.ErrUnknownMethod)}
r := newTestReader(t, f, core, "")
r.pollOnce(context.Background(), "secret")
if !r.disabled {
t.Error("ErrUnknownMethod must disable the reader")
}
if len(core.got) != 1 {
t.Errorf("core saw %d messages, want 1 — stop at the first refusal", len(core.got))
}
}
func TestSeenStatePersists(t *testing.T) {
path := filepath.Join(t.TempDir(), "state", "seen.json")
f := &fakeIMAP{uids: []uint32{9}, msgs: map[uint32]string{9: mail("Счёт", "текст")}}
core := &fakeCore{}
r := newTestReader(t, f, core, path)
r.pollOnce(context.Background(), "secret")
fi, err := os.Stat(path)
if err != nil {
t.Fatalf("state file: %v", err)
}
// A list of message ids from his mailbox is metadata about his mail.
if perm := fi.Mode().Perm(); perm != 0o600 {
t.Errorf("state file mode = %v, want 0600", perm)
}
// A fresh reader with the same state file must not re-read the message.
core2 := &fakeCore{}
r2 := newTestReader(t, f, core2, path)
if err := r2.state.load(); err != nil {
t.Fatalf("load: %v", err)
}
r2.pollOnce(context.Background(), "secret")
if len(core2.got) != 0 {
t.Errorf("after a restart core saw %d messages, want 0", len(core2.got))
}
}
func TestSeenStateHighWaterMark(t *testing.T) {
s := newSeenState("")
s.mark(1)
s.mark(3)
s.mark(2)
if s.high != 3 {
t.Errorf("high = %d, want 3 (contiguous run collapses)", s.high)
}
if len(s.set) != 0 {
t.Errorf("explicit set = %v, want empty", s.set)
}
if !s.seen(2) || s.seen(4) {
t.Errorf("seen(2)=%v seen(4)=%v", s.seen(2), s.seen(4))
}
}
func TestSeenStateCorruptFileIsNotFatal(t *testing.T) {
path := filepath.Join(t.TempDir(), "seen.json")
if err := os.WriteFile(path, []byte("{not json"), 0o600); err != nil {
t.Fatal(err)
}
s := newSeenState(path)
if err := s.load(); err == nil {
t.Error("a corrupt state file should report an error the caller logs")
}
if s.seen(1) {
t.Error("a corrupt state file must leave an empty seen-set, not a poisoned one")
}
}
// Off unless configured, and the credential is never a flag value.
func TestRunRequiresConfig(t *testing.T) {
if err := run([]string{}); err == nil {
t.Error("no -socket must be an error")
}
if err := run([]string{"-socket", "/tmp/nope.sock"}); err == nil {
t.Error("no mailbox configuration must be an error, not a default mailbox")
}
// There is no -password flag at all: only -password-file.
if err := run([]string{"-socket", "/x", "-imap", "h", "-user", "u", "-password", "p"}); err == nil ||
!strings.Contains(err.Error(), "flag provided but not defined") {
t.Errorf("a -password flag must not exist; err = %v", err)
}
}
func TestRunRejectsEmptyPasswordFile(t *testing.T) {
path := filepath.Join(t.TempDir(), "pass")
if err := os.WriteFile(path, []byte(" \n"), 0o600); err != nil {
t.Fatal(err)
}
err := run([]string{"-socket", "/x/y.sock", "-imap", "h", "-user", "u", "-password-file", path})
if err == nil || !strings.Contains(err.Error(), "empty") {
t.Errorf("an empty password file must be refused before dialling; err = %v", err)
}
}
+124 -3
View File
@@ -9,6 +9,12 @@
// Two sources, each its own provenance (the loop's rules trust source):
// - netdata → poll:netdata resource alarms (disk/mem/cert/temp)
// - kuma → poll:uptimekuma service up/down (the source of truth for it)
// - zenmoney → poll:zenmoney spending/income totals (Vikunja #125)
//
// The zenmoney source is why the token lives HERE and not in core: the poller
// already owns every other third-party credential, it holds no store key, and
// core never needs to know an account exists to answer a question about a fact
// the poller wrote. It is off unless -zenmoney-token-file is given.
//
// Netdata needs no auth over the wg-fronted net. Kuma's /metrics needs an API
// key (basic-auth); without -kuma the whole kuma path is skipped (netdata-only
@@ -37,6 +43,7 @@ import (
"time"
"github.com/kami/maven/internal/ipc"
"github.com/kami/maven/internal/zenmoney"
)
func main() {
@@ -52,6 +59,9 @@ func run(args []string) error {
netdataURL := fs.String("netdata", "http://127.0.0.1:19999", "netdata base URL ('' to disable)")
kumaURL := fs.String("kuma", "", "uptime-kuma metrics URL, e.g. http://127.0.0.1:3001/metrics ('' to disable)")
kumaKey := fs.String("kuma-key", "", "uptime-kuma API key (basic-auth username)")
zenTokenFile := fs.String("zenmoney-token-file", "", "file holding the zenmoney API token ('' disables money tracking)")
zenURL := fs.String("zenmoney-url", zenmoney.DefaultBaseURL, "zenmoney API base URL (tests/self-hosted proxies)")
zenInterval := fs.Duration("zenmoney-interval", time.Hour, "how often to read zenmoney (money does not move every minute)")
wgIface := fs.String("wg", "", "wireguard interface for the presence signal, e.g. wg0 or 'all' ('' to disable)")
wgCmd := fs.String("wg-cmd", "wg", "wg binary (use e.g. 'sudo wg' if the poller lacks CAP_NET_ADMIN)")
interval := fs.Duration("interval", 60*time.Second, "poll cadence")
@@ -62,8 +72,24 @@ func run(args []string) error {
if *socket == "" {
return fmt.Errorf("-socket is required")
}
if *netdataURL == "" && *kumaURL == "" && *wgIface == "" {
return fmt.Errorf("nothing to poll: set -netdata, -kuma and/or -wg")
if *netdataURL == "" && *kumaURL == "" && *wgIface == "" && *zenTokenFile == "" {
return fmt.Errorf("nothing to poll: set -netdata, -kuma, -wg and/or -zenmoney-token-file")
}
// The token is read from a file, never taken as a flag value: an argv token
// is visible in `ps` to every user on the box and lands in the compose file
// and the shell history. Read once at start — a rotated token means a
// restart, which is cheaper than re-reading his credential every hour.
var zen *zenmoney.Client
if *zenTokenFile != "" {
raw, err := os.ReadFile(*zenTokenFile)
if err != nil {
return fmt.Errorf("read zenmoney token: %w", err)
}
zen, err = zenmoney.New(strings.TrimSpace(string(raw)), *zenURL, *timeout*3)
if err != nil {
return err
}
}
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
@@ -83,9 +109,13 @@ func run(args []string) error {
kumaKey: *kumaKey,
wgIface: *wgIface,
wgCmd: *wgCmd,
zen: zen,
zenEvery: *zenInterval,
}
log.Printf("mavpoll: polling every %s (netdata=%q kuma=%q wg=%q)", *interval, *netdataURL, *kumaURL, *wgIface)
// The token is never logged, not even its length.
log.Printf("mavpoll: polling every %s (netdata=%q kuma=%q wg=%q zenmoney=%v every %s)",
*interval, *netdataURL, *kumaURL, *wgIface, zen != nil, *zenInterval)
p.pollOnce(ctx) // fire immediately; don't idle a full interval on start
t := time.NewTicker(*interval)
defer t.Stop()
@@ -108,6 +138,12 @@ type poller struct {
kumaKey string
wgIface string
wgCmd string
// zen is nil unless a token file was configured — money tracking is a
// capability, off by default like weather and telegram.
zen *zenmoney.Client
zenEvery time.Duration
zenLast time.Time
}
// pollOnce — one sweep of both sources. A failure in one source logs and does
@@ -129,6 +165,66 @@ func (p *poller) pollOnce(ctx context.Context) {
log.Printf("mavpoll: wg: %v", err)
}
}
// Money on its own, much slower cadence: a bank feed that updates hourly
// polled every minute is 60 pointless reads of his financial history.
if p.zen != nil && now.Sub(p.zenLast) >= p.zenEvery {
p.zenLast = now
if err := p.pollZenmoney(ctx, now); err != nil {
log.Printf("mavpoll: zenmoney: %v", err)
}
}
}
// ---- zenmoney: spending/income totals → money facts ------------------------
// pollZenmoney reads today's and this month's totals and writes them as
// facts(kind=env, source=poll:zenmoney) (Vikunja #125).
//
// Two properties this function exists to hold:
//
// - An empty or failed read writes NOTHING. zenmoney.Summary.Value() refuses
// to encode a summary built from zero transactions, so a poller that cannot
// reach the API leaves the last good fact in place rather than overwriting
// it with a zero Maven would then recite as fact.
// - Nothing about the money leaves the box except the diff request itself, to
// the service that already holds his bank sessions. The totals are written
// to the store and read back only when he asks; they are never search input
// and no tick rule fires on them.
//
// Both windows are read from one diff call each. Two calls an hour against an
// API whose whole job is this is not worth caching.
// moneyWindow — one fact key and the period it covers.
type moneyWindow struct {
key string
from, to time.Time
}
func (p *poller) pollZenmoney(ctx context.Context, now time.Time) error {
dFrom, dTo := zenmoney.DayWindow(now)
mFrom, mTo := zenmoney.MonthWindow(now)
windows := []moneyWindow{
{zenmoney.KeySpentToday, dFrom, dTo},
{zenmoney.KeySpentMonth, mFrom, mTo},
}
var firstErr error
for _, w := range windows {
sum, err := p.zen.Since(ctx, w.from, w.to)
if err != nil {
if firstErr == nil {
firstErr = err
}
continue
}
val, ok := sum.Value()
if !ok {
// Nothing read. Silence, not a zero.
continue
}
if err := p.writeIfChangedRaw(ctx, w.key, zenmoney.Source, val, now); err != nil && firstErr == nil {
firstErr = err
}
}
return firstErr
}
// ---- wireguard: latest handshake → presence signal -------------------------
@@ -301,6 +397,31 @@ func (p *poller) writeIfChanged(ctx context.Context, key, source, val string, no
return nil
}
// writeIfChangedRaw is writeIfChanged for values that are already JSON (the
// money facts store an object, not a string). Kept separate rather than
// generalising writeIfChanged, because the string-valued env facts encoding
// their own value is the convention the rules rely on.
//
// The log line names the key and the source, never the figures: mavpoll's log
// is not the place his spending ends up.
func (p *poller) writeIfChangedRaw(ctx context.Context, key, source, jsonVal string, now time.Time) error {
prev, err := p.core.LatestFactBySource(ctx, key, source)
switch {
case err == nil && prev.Value == jsonVal:
return nil
case err != nil && err != ipc.ErrNoFact && !isNoFact(err):
return fmt.Errorf("read %s: %w", key, err)
}
if _, err := p.core.WriteFact(ctx, ipc.WriteFactReq{
Ts: now, Kind: "env", Key: key, Value: jsonVal,
Source: source, Confidence: 1.0,
}); err != nil {
return fmt.Errorf("write %s: %w", key, err)
}
log.Printf("mavpoll: %s updated (%s)", key, source)
return nil
}
// isNoFact — ErrNoFact rehydrated over the wire is wrapped (fmt.Errorf %w), so
// errors.Is is the right check; keep a helper so the switch above reads clean.
func isNoFact(err error) bool {
+126
View File
@@ -1,8 +1,17 @@
package main
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
"time"
"github.com/kami/maven/internal/ipc"
"github.com/kami/maven/internal/zenmoney"
)
func TestMaxSeverity(t *testing.T) {
@@ -62,3 +71,120 @@ func TestParseMaxHandshake(t *testing.T) {
}
}
}
// ---- zenmoney (Vikunja #125) ----------------------------------------------
// factCore records the facts the poller wrote and answers "no fact yet".
type factCore struct {
ipc.UnimplementedCoreAPI
written []ipc.WriteFactReq
prev map[string]string
}
func (c *factCore) LatestFactBySource(_ context.Context, key, source string) (ipc.Fact, error) {
if v, ok := c.prev[key+"|"+source]; ok {
return ipc.Fact{Key: key, Source: source, Value: v}, nil
}
return ipc.Fact{}, ipc.ErrNoFact
}
func (c *factCore) WriteFact(_ context.Context, req ipc.WriteFactReq) (int64, error) {
c.written = append(c.written, req)
return int64(len(c.written)), nil
}
func zenFixtureServer(t *testing.T, body []byte, status int) *httptest.Server {
t.Helper()
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if status != http.StatusOK {
w.WriteHeader(status)
return
}
w.Write(body)
}))
}
func TestPollZenmoneyWritesMoneyFacts(t *testing.T) {
body, err := os.ReadFile("../../internal/zenmoney/testdata/diff.json")
if err != nil {
t.Fatal(err)
}
srv := zenFixtureServer(t, body, http.StatusOK)
defer srv.Close()
zen, err := zenmoney.New("tok", srv.URL, time.Second)
if err != nil {
t.Fatal(err)
}
core := &factCore{}
p := &poller{core: core, zen: zen}
now := time.Date(2026, 8, 1, 21, 0, 0, 0, time.UTC)
if err := p.pollZenmoney(context.Background(), now); err != nil {
t.Fatal(err)
}
if len(core.written) != 2 {
t.Fatalf("wrote %d facts, want today + month", len(core.written))
}
for _, f := range core.written {
if f.Kind != "env" || f.Source != zenmoney.Source {
t.Errorf("fact = %+v, want kind=env source=%s", f, zenmoney.Source)
}
if _, err := zenmoney.ParseFactValue(f.Value); err != nil {
t.Errorf("fact value %q does not decode: %v", f.Value, err)
}
}
}
// A read that returns nothing for the window writes NOTHING. Silence, not a
// zero: an invented 0 would be recited back to him as fact.
func TestPollZenmoneyWritesNothingWhenEmpty(t *testing.T) {
srv := zenFixtureServer(t, []byte(`{"serverTimestamp":1,"instrument":[],"transaction":[]}`), http.StatusOK)
defer srv.Close()
zen, _ := zenmoney.New("tok", srv.URL, time.Second)
core := &factCore{}
p := &poller{core: core, zen: zen}
if err := p.pollZenmoney(context.Background(), time.Now()); err != nil {
t.Fatal(err)
}
if len(core.written) != 0 {
t.Errorf("wrote %+v, want no fact at all", core.written)
}
}
// An API failure must not overwrite the last good total either.
func TestPollZenmoneyFailureWritesNothing(t *testing.T) {
srv := zenFixtureServer(t, nil, http.StatusUnauthorized)
defer srv.Close()
zen, _ := zenmoney.New("bad", srv.URL, time.Second)
core := &factCore{}
p := &poller{core: core, zen: zen}
if err := p.pollZenmoney(context.Background(), time.Now()); err == nil {
t.Error("want the 401 reported")
}
if len(core.written) != 0 {
t.Errorf("wrote %+v on a failed read", core.written)
}
}
// Unchanged totals do not churn the facts table.
func TestWriteIfChangedRawSkipsUnchanged(t *testing.T) {
core := &factCore{prev: map[string]string{
zenmoney.KeySpentToday + "|" + zenmoney.Source: `{"count":1}`,
}}
p := &poller{core: core}
if err := p.writeIfChangedRaw(context.Background(), zenmoney.KeySpentToday, zenmoney.Source, `{"count":1}`, time.Now()); err != nil {
t.Fatal(err)
}
if len(core.written) != 0 {
t.Errorf("wrote %+v for an unchanged value", core.written)
}
}
// Money tracking is off unless configured: no token file, no zenmoney client,
// and the poller still refuses to start with nothing at all to poll.
func TestRunRequiresSomethingToPoll(t *testing.T) {
err := run([]string{"-socket", "/tmp/nope.sock", "-netdata", "", "-kuma", "", "-wg", ""})
if err == nil || !strings.Contains(err.Error(), "nothing to poll") {
t.Errorf("err = %v, want a 'nothing to poll' refusal", err)
}
}
+33
View File
@@ -101,9 +101,42 @@ services:
"-netdata", "http://127.0.0.1:19999",
"-kuma", "http://127.0.0.1:3001/metrics",
"-kuma-key", "uk5_mavpoll-key"]
# Money tracking (Vikunja #125) is OFF: it needs a zenmoney token,
# which mavpoll reads from a FILE so it never appears in `ps`, in
# this file, or in shell history. To enable, mount the token and
# append: "-zenmoney-token-file", "/run/secrets/zenmoney.token"
# (optionally "-zenmoney-interval", "1h"). Core never sees the
# token — the poller writes facts(kind=env, source=poll:zenmoney)
# and mavend only reads those back when he asks.
depends_on: [mavend]
volumes:
- sockets:/run/maven
# - ./deploy/zenmoney.token:/run/secrets/zenmoney.token:ro
# The mail reader (Vikunja #246) is OFF and commented out: it needs an IMAP
# account, and there is none on this box. mavmaild reads the password from a
# FILE so it never appears in `ps`, in this file, or in shell history — the
# same rule mavpoll follows for the zenmoney token. Core never sees the
# password: the reader hands core message text on one IPC method, and core
# writes what the model extracts as task CANDIDATES he reviews on /tasks.
# Nothing here can create a reminder, so a misread mail cannot fire.
#
# To enable: write the password to deploy/imap.password (0600, gitignored),
# add an "email": {} block to deploy/mavend.json, and uncomment this service.
# mavmaild:
# <<: *image
# command: ["mavmaild", "-socket", "/run/maven/mavend.sock",
# "-imap", "imap.example.org:993",
# "-user", "kami@example.org",
# "-password-file", "/run/secrets/imap.password",
# "-mailbox", "INBOX",
# "-interval", "15m",
# "-state", "/var/lib/maven/mail-seen.json"]
# depends_on: [mavend]
# volumes:
# - sockets:/run/maven
# - dbdata:/var/lib/maven
# - ./deploy/imap.password:/run/secrets/imap.password:ro
volumes:
dbdata:
+6 -1
View File
@@ -74,7 +74,12 @@ func Requirement(m ipc.Method) Authority {
// existing analogue.
ipc.MethodCaptureTask,
ipc.MethodListTasks,
ipc.MethodSetTaskStatus:
ipc.MethodSetTaskStatus,
// Mail ingestion (Vikunja #246). AuthRead because of what the method can
// produce: candidate tasks and nothing else. It cannot write a fact, set a
// reminder, or touch the tool allowlist, so a compromised mail reader can
// at worst put junk on a review page he clears in one click.
ipc.MethodIngestMail:
return AuthRead
}
// Unknown method ⇒ AuthRead, but ipc.dispatch returns ErrUnknownMethod
+32
View File
@@ -150,6 +150,12 @@ type Config struct {
// absent ⇒ no evaluation loop at all. See MemoryEvalConfig.
MemoryEval *MemoryEvalConfig `json:"memory_eval,omitempty"`
// Email — mail ingestion (Vikunja #246). nil / absent ⇒ core refuses
// ipc.MethodIngestMail outright, so a mail reader cannot make Maven read a
// mailbox by merely existing. See EmailConfig; the IMAP host and credential
// live in the reader (cmd/mavmaild), never here.
Email *EmailConfig `json:"email,omitempty"`
// Praxis — the ecosystem attention-state service. When configured, maven
// calls the Praxis HTTP tools API for attention listing and item lifecycle.
// Maven never touches Praxis's database directly (ecosystem invariant: no
@@ -418,6 +424,26 @@ type MemoryEvalConfig struct {
MinConfidence float64 `json:"min_confidence,omitempty"`
}
// EmailConfig — core's half of the email reader: how many task candidates one
// message may produce, and how long the extraction call may take.
//
// There is deliberately nothing about a mailbox here. Core does not connect to
// IMAP, does not know an account exists, and holds no mail credential — the
// reader daemon does, the same split mavpoll uses for the zenmoney token. This
// block only says "extraction is allowed, with these bounds".
type EmailConfig struct {
// MaxTasks — candidates per message. 0 ⇒ email.MaxCandidates (3).
MaxTasks int `json:"max_tasks,omitempty"`
// Timeout — per-message extraction budget. 0 ⇒ DefaultEmailTimeout. This is
// a Thinking model reading a mail; nobody is waiting on the answer, but a
// hung llama-server must not pin the reader's connection forever.
Timeout Duration `json:"timeout,omitempty"`
}
// DefaultEmailTimeout — extraction budget per message.
const DefaultEmailTimeout = 2 * time.Minute
// PhraserConfig — the LLM-backed phraser seam. The daemon spawns llama-server
// as a managed subprocess and sends chat-completion requests to phrase nudge
// and reminder messages. nil ⇒ the template-based Stub is used instead.
@@ -607,6 +633,12 @@ func (c *Config) applyDefaults() {
c.MemoryEval.Interval = Duration(DefaultMemoryEvalInterval)
}
// Same rule again: absent stays nil (⇒ mail ingestion refused), present gets
// the timeout default so `{}` is a valid "on with the defaults".
if c.Email != nil && c.Email.Timeout <= 0 {
c.Email.Timeout = Duration(DefaultEmailTimeout)
}
if c.Voice != nil {
if c.Voice.RouterThreshold <= 0 {
c.Voice.RouterThreshold = DefaultRouterThreshold
+226
View File
@@ -0,0 +1,226 @@
package email
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
"github.com/kami/maven/internal/llm"
"github.com/kami/maven/internal/persona"
)
// Extraction — turning one mail into task CANDIDATES, and nothing else.
//
// The output of this file can only ever become rows in `tasks` with status
// "candidate" (store.TaskCandidate), written through the one intake seam
// (ipc.CaptureTaskReq, Vikunja #130). That bound is the whole design:
//
// - No reminder. A reminder FIRES; it speaks to him unprompted. A 1.7B that
// misreads "встреча была в четверг" as a future appointment would then wake
// him up about it. A candidate that is wrong is a line on a review page he
// dismisses in one click, which is the correct cost of a model being wrong
// about someone's mail.
// - No fact. A fact is a claim Maven will later recite as true. Nothing read
// out of a marketing mail deserves that standing.
// - No calendar event, no note, no action. Extraction writes candidates or
// writes nothing.
//
// The due date the model may return is stored on the candidate (tasks.due_ts),
// which no scheduler reads — it is there so the review page can sort by it.
//
// Privacy: the mail text goes to the resident model on this box and nowhere
// else. It is never search input (CLAUDE.md: "his notes and facts are never
// search input" — mail is the same class), and Evidence keeps only the subject
// line, so the review page shows him where a candidate came from without the
// store growing a copy of his mailbox.
// MaxCandidates — at most this many candidates per message, enforced by the
// grammar. A mail with four tasks in it is a mail he has to read himself; a
// model allowed ten will produce ten.
const MaxCandidates = 3
// SourcePrefix — provenance for everything this package captures. The mailbox
// name is appended: "email:INBOX". Same vocabulary as tap:voice / poll:netdata.
const SourcePrefix = "email:"
// Candidate — one piece of work the model thinks the mail is asking for.
type Candidate struct {
Text string `json:"text"`
// Due — "YYYY-MM-DD" or empty. A date the model read out of the text, not a
// date it computed: relative wording ("до пятницы") is left in Text, because
// a small model resolving "пятница" against today's date gets it wrong often
// enough that a stored wrong date is worse than no date.
Due string `json:"due"`
}
// Completer — the llama-server seam, same shape memeval and the router use, so
// the one resident model serves this caller too.
type Completer interface {
Complete(ctx context.Context, r llm.Req) (string, error)
}
// Extractor reads a message and returns candidates. It holds no store and no
// writer on purpose: this type cannot persist anything, so "extraction never
// acts" is a property of the code, not of a review.
type Extractor struct {
llm Completer
// MaxCandidates — 0 ⇒ MaxCandidates.
max int
// ContextBlock — the shared persona block, optional. Extraction output is
// not spoken, so the persona matters less here than in the phraser; it is
// wired anyway so a candidate reads in her voice on the review page.
contextBlock func() string
}
func NewExtractor(c Completer, max int, contextBlock func() string) *Extractor {
if max <= 0 || max > MaxCandidates {
max = MaxCandidates
}
return &Extractor{llm: c, max: max, contextBlock: contextBlock}
}
// extractGrammar — GBNF pinning the answer to a bounded array of fixed-shape
// candidates. Same reasoning as memeval's evalGrammar and the router's
// routeGrammar: the shape and the length bound are what keep a small model from
// drifting into prose or spending the token budget repeating one field.
//
// The empty array is reachable, deliberately: most mail contains no task, and a
// model with no way to say "nothing" invents something.
const extractGrammar = `
root ::= "[" ws (item ("," ws item){0,2})? ws "]"
item ::= "{" ws "\"text\"" ws ":" ws text "," ws "\"due\"" ws ":" ws due ws "}"
text ::= "\"" ([^"\\] | "\\" .){1,120} "\""
due ::= "\"\"" | "\"" [0-9]{4} "-" [0-9]{2} "-" [0-9]{2} "\""
ws ::= [ \t\n]*
`
// extractSystem — the extraction prompt.
//
// Written around the two failure modes a small model has on this task: it
// summarises when asked to extract (turning a mail into "письмо от Антона"),
// and it invents an obligation from any polite closing sentence. Hence the
// insistence on a verb phrase, and the explicit permission to return [].
const extractSystem = `Ты читаешь одно письмо из его почты и достаёшь из него дела, которые письмо от него требует.
Правила:
- Отвечай ТОЛЬКО массивом JSON. Каждый элемент: {"text": "...", "due": "ГГГГ-ММ-ДД" или ""}.
- text — короткая формулировка дела по-русски, с глаголом: "оплатить счёт за интернет", "отправить акт". Не пересказывай письмо и не описывай его.
- Дело — это то, что должен сделать ОН. Рассылка, реклама, уведомление, отчёт, письмо «просто к сведению» — дел не содержат.
- Если письмо ничего от него не требует, верни пустой массив []. Это нормальный ответ, так бывает чаще всего.
- Ничего не придумывай. Если срока в письме нет — "".
- due заполняй только когда в письме стоит конкретная дата. Слова вроде «до пятницы» оставь в text, дату не вычисляй.
- Максимум три дела. Лучше одно точное, чем три общих.`
// Extract returns the candidates in one message.
//
// Junk is refused without an LLM call — cheapest possible defence, and the
// reason the header filter exists. An empty message (no subject, no body) is
// likewise not worth a round trip.
//
// A parse failure is an error the caller logs and moves past. It is never
// silently turned into zero candidates, because "the model went off the rails"
// and "the mail contains no task" want different reactions from a human reading
// the log.
func (e *Extractor) Extract(ctx context.Context, msg Message) ([]Candidate, error) {
if msg.Junk {
return nil, nil
}
user := renderForModel(msg)
if user == "" {
return nil, nil
}
raw, err := e.llm.Complete(ctx, llm.Req{
System: persona.Prepend(e.contextBlock, extractSystem),
User: user,
Grammar: extractGrammar,
MaxTokens: 512,
RepeatPenalty: 1.1,
})
if err != nil {
return nil, fmt.Errorf("email: extract: %w", err)
}
items, err := parseCandidates(raw)
if err != nil {
// The raw reply is NOT in the error: it is a transformation of his mail,
// and this error reaches the daemon log.
return nil, fmt.Errorf("email: extract: unparsable reply (%d bytes)", len(raw))
}
out := make([]Candidate, 0, len(items))
seen := map[string]bool{}
for _, it := range items {
it.Text = strings.TrimSpace(it.Text)
if it.Text == "" {
continue
}
key := strings.ToLower(strings.Join(strings.Fields(it.Text), " "))
if seen[key] {
continue // the model repeating itself is not two tasks
}
seen[key] = true
if _, ok := ParseDue(it.Due); !ok {
it.Due = "" // a date the grammar allowed but the calendar does not
}
out = append(out, it)
if len(out) >= e.max {
break
}
}
return out, nil
}
// renderForModel is the user turn: subject, sender and body, labelled. Only
// these three fields — no headers, no recipient list, no message-id, nothing
// that would let the model start reasoning about routing metadata.
func renderForModel(msg Message) string {
var b strings.Builder
if msg.From != "" {
fmt.Fprintf(&b, "От: %s\n", msg.From)
}
if msg.Subject != "" {
fmt.Fprintf(&b, "Тема: %s\n", msg.Subject)
}
if msg.Body != "" {
fmt.Fprintf(&b, "\n%s\n", msg.Body)
}
if msg.Subject == "" && msg.Body == "" {
return ""
}
return b.String()
}
// parseCandidates decodes the grammar-constrained reply, tolerating the
// wrappers a Thinking model sometimes leaves around it (a fenced block, or
// leading reasoning before the array).
func parseCandidates(raw string) ([]Candidate, error) {
s := strings.TrimSpace(raw)
if i := strings.Index(s, "["); i > 0 {
s = s[i:]
}
if j := strings.LastIndex(s, "]"); j >= 0 {
s = s[:j+1]
}
var out []Candidate
if err := json.Unmarshal([]byte(s), &out); err != nil {
return nil, err
}
return out, nil
}
// ParseDue turns the model's "YYYY-MM-DD" into a time in UTC. Exported because
// the daemon-side intake stores it on the candidate.
//
// The zero-value/empty case returns ok=false rather than an error: no date is
// the common answer, not a failure.
func ParseDue(s string) (time.Time, bool) {
s = strings.TrimSpace(s)
if s == "" {
return time.Time{}, false
}
t, err := time.Parse("2006-01-02", s)
if err != nil {
return time.Time{}, false
}
return t, true
}
+143
View File
@@ -0,0 +1,143 @@
package email
import (
"context"
"strings"
"testing"
"github.com/kami/maven/internal/llm"
)
// fakeLLM returns a canned reply and records the request, so a test can assert
// on the grammar and on what of the mail was sent.
type fakeLLM struct {
reply string
err error
got llm.Req
calls int
}
func (f *fakeLLM) Complete(_ context.Context, r llm.Req) (string, error) {
f.calls++
f.got = r
return f.reply, f.err
}
func msgFor(subject, body string) Message {
return Message{UID: 1, From: "anton@example.org", Subject: subject, Body: body}
}
func TestExtractCandidates(t *testing.T) {
f := &fakeLLM{reply: `[{"text":"отправить акт","due":""},{"text":"оплатить счёт","due":"2026-08-05"}]`}
e := NewExtractor(f, 0, nil)
got, err := e.Extract(context.Background(), msgFor("Акт и счёт", "Надо отправить акт и оплатить счёт до 5 августа."))
if err != nil {
t.Fatalf("extract: %v", err)
}
if len(got) != 2 {
t.Fatalf("got %d candidates, want 2: %+v", len(got), got)
}
if got[0].Text != "отправить акт" || got[1].Due != "2026-08-05" {
t.Errorf("candidates = %+v", got)
}
if f.got.Grammar == "" {
t.Error("extraction must be grammar-constrained")
}
// The subject and body go to the model; nothing else about the message does.
if !strings.Contains(f.got.User, "Акт и счёт") || !strings.Contains(f.got.User, "оплатить счёт") {
t.Errorf("user turn = %q", f.got.User)
}
}
func TestExtractEmptyArrayIsNotAnError(t *testing.T) {
f := &fakeLLM{reply: "[]"}
got, err := NewExtractor(f, 0, nil).Extract(context.Background(), msgFor("FYI", "Просто к сведению."))
if err != nil || len(got) != 0 {
t.Fatalf("got (%v, %v), want (empty, nil) — no task is the normal answer", got, err)
}
}
// Junk must never reach the model: the header filter exists so the resident
// model is not spent on newsletters.
func TestExtractSkipsJunkWithoutCallingModel(t *testing.T) {
f := &fakeLLM{reply: `[{"text":"купить всё со скидкой","due":""}]`}
msg := msgFor("Скидки", "Sale!")
msg.Junk = true
got, err := NewExtractor(f, 0, nil).Extract(context.Background(), msg)
if err != nil || got != nil {
t.Fatalf("got (%v, %v), want (nil, nil)", got, err)
}
if f.calls != 0 {
t.Errorf("model called %d times for junk, want 0", f.calls)
}
}
func TestExtractEmptyMessageIsNotSent(t *testing.T) {
f := &fakeLLM{reply: "[]"}
if _, err := NewExtractor(f, 0, nil).Extract(context.Background(), Message{UID: 3}); err != nil {
t.Fatalf("extract: %v", err)
}
if f.calls != 0 {
t.Errorf("model called %d times for an empty message, want 0", f.calls)
}
}
func TestExtractCaps(t *testing.T) {
f := &fakeLLM{reply: `[{"text":"a","due":""},{"text":"b","due":""},{"text":"c","due":""}]`}
got, err := NewExtractor(f, 2, nil).Extract(context.Background(), msgFor("s", "b"))
if err != nil {
t.Fatalf("extract: %v", err)
}
if len(got) != 2 {
t.Errorf("got %d, want the configured cap of 2", len(got))
}
}
func TestExtractDropsRepeatsAndBadDates(t *testing.T) {
f := &fakeLLM{reply: `[{"text":"Отправить акт","due":"2026-02-31"},{"text":"отправить акт","due":""},{"text":" ","due":""}]`}
got, err := NewExtractor(f, 0, nil).Extract(context.Background(), msgFor("s", "b"))
if err != nil {
t.Fatalf("extract: %v", err)
}
if len(got) != 1 {
t.Fatalf("got %d candidates, want 1 (repeat and blank dropped): %+v", len(got), got)
}
if got[0].Due != "" {
t.Errorf("due = %q, want empty — 2026-02-31 is not a date", got[0].Due)
}
}
// A Thinking model sometimes wraps the array; and when it emits something
// unparsable the caller must hear about it rather than see "no tasks".
func TestParseCandidatesTolerance(t *testing.T) {
got, err := parseCandidates("думаю... [{\"text\":\"x\",\"due\":\"\"}] всё")
if err != nil || len(got) != 1 || got[0].Text != "x" {
t.Fatalf("got (%+v, %v)", got, err)
}
if _, err := parseCandidates("нет никакого JSON"); err == nil {
t.Error("unparsable output must be an error")
}
}
func TestExtractParseErrorHidesMailText(t *testing.T) {
f := &fakeLLM{reply: "он просил отправить акт, вот такой ответ"}
_, err := NewExtractor(f, 0, nil).Extract(context.Background(), msgFor("Акт", "секретный текст"))
if err == nil {
t.Fatal("want an error")
}
if strings.Contains(err.Error(), "акт") || strings.Contains(err.Error(), "секретный") {
t.Errorf("error text leaks mail content: %v", err)
}
}
func TestParseDue(t *testing.T) {
if _, ok := ParseDue(""); ok {
t.Error("empty due must be (zero, false)")
}
if got, ok := ParseDue("2026-08-05"); !ok || got.Year() != 2026 || got.Month() != 8 || got.Day() != 5 {
t.Errorf("ParseDue = (%v, %v)", got, ok)
}
if _, ok := ParseDue("05.08.2026"); ok {
t.Error("a non-ISO date must not parse")
}
}
+98
View File
@@ -0,0 +1,98 @@
package email
import (
"fmt"
"time"
)
// FetchSince is the whole read path in one call: connect, log in, examine the
// mailbox read-only, list what arrived since a date, fetch and parse the ones
// the caller has not seen, log out.
//
// It is a function rather than a long-lived object because a mail poller should
// not hold an authenticated session (and therefore his credential in a live TLS
// state) between polls. Connect, read, drop.
//
// skip decides which UIDs are already known — the poller's seen-set. max bounds
// one poll: a mailbox that received 400 messages overnight must not turn into
// 400 LLM calls, and the newest max are the ones a task could still be hiding
// in. Junk messages are returned too, flagged, so the caller can mark them seen
// without a second protocol round.
type FetchSince struct {
Addr string // host or host:993
User string
Mailbox string // e.g. "INBOX"
Timeout time.Duration
Since time.Time
Max int
Skip func(uid uint32) bool
}
// Run performs one read. password is passed here, not stored in the struct, so
// the configuration of a mailbox and the secret for it are never the same value
// sitting in the same place.
func (f FetchSince) Run(password string) ([]Message, error) {
return f.RunWith(password, nil)
}
// RunWith is Run with an explicit connection function, which is how the reader
// daemon and the tests substitute an in-process server. nil ⇒ Dial, i.e.
// implicit TLS with certificate verification; there is no configuration path
// that reaches this, so no deployment can end up talking cleartext IMAP.
func (f FetchSince) RunWith(password string, dial func(addr string, timeout time.Duration) (*Conn, error)) ([]Message, error) {
if f.Addr == "" || f.User == "" || f.Mailbox == "" {
return nil, fmt.Errorf("email: mailbox not configured (addr/user/mailbox)")
}
if dial == nil {
dial = Dial
}
c, err := dial(f.Addr, f.Timeout)
if err != nil {
return nil, err
}
defer c.Close()
if err := c.Login(f.User, password); err != nil {
return nil, err
}
defer c.Logout()
if err := c.Select(f.Mailbox); err != nil {
return nil, err
}
uids, err := c.SearchSince(f.Since)
if err != nil {
return nil, err
}
// Newest UIDs first — IMAP hands them back ascending, and when Max clips the
// list the recent mail is what matters.
wanted := make([]uint32, 0, len(uids))
for i := len(uids) - 1; i >= 0; i-- {
if f.Skip != nil && f.Skip(uids[i]) {
continue
}
wanted = append(wanted, uids[i])
if f.Max > 0 && len(wanted) >= f.Max {
break
}
}
out := make([]Message, 0, len(wanted))
for _, uid := range wanted {
raw, err := c.Fetch(uid)
if err != nil {
// One unreadable message does not abandon the poll; the rest of the
// mailbox is still worth reading. The error names the UID, not the
// message.
return out, fmt.Errorf("email: fetch uid %d: %w", uid, err)
}
if len(raw) == 0 {
continue // vanished between SEARCH and FETCH
}
msg, err := ParseMessage(uid, raw)
if err != nil {
continue // unparsable headers — nothing to review, skip silently
}
out = append(out, msg)
}
return out, nil
}
+49
View File
@@ -0,0 +1,49 @@
package email
import (
"net"
"strings"
"testing"
"time"
)
func TestFetchSinceRun(t *testing.T) {
mk := func(subject string) string {
return "Subject: " + subject + "\r\nContent-Type: text/plain; charset=utf-8\r\n\r\nbody\r\n"
}
f := &fakeIMAP{
uids: []uint32{1, 2, 3},
msgs: map[uint32]string{1: mk("one"), 2: mk("two"), 3: mk("three")},
}
fs := FetchSince{
Addr: "mail.example:993", User: "kami", Mailbox: "INBOX",
Timeout: 5 * time.Second,
Since: time.Date(2026, 7, 30, 0, 0, 0, 0, time.UTC),
Max: 2,
Skip: func(uid uint32) bool { return uid == 3 },
}
msgs, err := fs.RunWith("secret", func(addr string, timeout time.Duration) (*Conn, error) {
cli, srv := net.Pipe()
go f.serve(t, srv)
return NewConn(cli, timeout)
})
if err != nil {
t.Fatalf("run: %v", err)
}
// Newest first, the already-seen UID skipped, Max respected.
if len(msgs) != 2 {
t.Fatalf("got %d messages, want 2: %+v", len(msgs), msgs)
}
if msgs[0].Subject != "two" || msgs[1].Subject != "one" {
t.Errorf("subjects = %q,%q, want two,one (newest first)", msgs[0].Subject, msgs[1].Subject)
}
if strings.Contains(strings.Join(f.cmds, " "), "UID FETCH 3") {
t.Error("a skipped UID must not be fetched again")
}
}
func TestFetchSinceRequiresConfig(t *testing.T) {
if _, err := (FetchSince{}).Run("secret"); err == nil {
t.Fatal("an unconfigured mailbox must not be read")
}
}
+280
View File
@@ -0,0 +1,280 @@
package email
import (
"bufio"
"crypto/tls"
"fmt"
"io"
"net"
"regexp"
"strconv"
"strings"
"time"
)
// A minimal IMAP4rev1 client — LOGIN, SELECT, UID SEARCH, UID FETCH with
// BODY.PEEK, LOGOUT, and nothing else.
//
// Why hand-rolled instead of go-imap: the whole surface Maven needs is five
// commands, and this is the one code path that holds his mailbox credential and
// reads his private mail. A ~200-line client with no dependencies is auditable
// in one sitting; a general-purpose IMAP library is a much larger amount of
// code doing much more than we asked, in the most sensitive place in the tree.
// If IDLE, CONDSTORE or server-side threading ever become worth having, that
// trade should be re-made deliberately.
//
// BODY.PEEK[] rather than BODY[] is load-bearing: Maven reads his mail and must
// leave no trace of having done so. Reading a message here does not mark it
// \Seen, so the unread state in his own mail client stays his.
// DefaultIMAPPort — implicit-TLS IMAP. There is no cleartext and no STARTTLS
// path in this client: an option to send his password over a plain socket is an
// option to get it wrong once.
const DefaultIMAPPort = "993"
// Conn — one authenticated IMAP connection. Not safe for concurrent use; the
// poller drives one connection at a time.
type Conn struct {
rwc io.ReadWriteCloser
r *bufio.Reader
tag int
timeout time.Duration
}
// Dial opens an implicit-TLS connection and reads the server greeting.
func Dial(addr string, timeout time.Duration) (*Conn, error) {
host, _, err := net.SplitHostPort(addr)
if err != nil {
host, addr = addr, net.JoinHostPort(addr, DefaultIMAPPort)
}
d := &net.Dialer{Timeout: timeout}
// ServerName is set from the host we asked for: certificate verification is
// the only thing standing between his password and a MITM on the way out.
c, err := tls.DialWithDialer(d, "tcp", addr, &tls.Config{ServerName: host, MinVersion: tls.VersionTLS12})
if err != nil {
return nil, fmt.Errorf("email: dial %s: %w", addr, err)
}
return NewConn(c, timeout)
}
// NewConn wraps an already-open stream (the tests speak IMAP over a pipe) and
// consumes the greeting.
func NewConn(rwc io.ReadWriteCloser, timeout time.Duration) (*Conn, error) {
c := &Conn{rwc: rwc, r: bufio.NewReaderSize(rwc, 64<<10), timeout: timeout}
line, err := c.readLine()
if err != nil {
return nil, fmt.Errorf("email: greeting: %w", err)
}
if !strings.HasPrefix(line, "* OK") && !strings.HasPrefix(line, "* PREAUTH") {
c.rwc.Close()
return nil, fmt.Errorf("email: server refused connection: %s", line)
}
return c, nil
}
func (c *Conn) Close() error { return c.rwc.Close() }
// Login authenticates with LOGIN. The password is passed as an argument and
// never stored on the Conn: nothing in this package keeps a credential alive
// past the command that uses it, so no struct dump or panic trace can carry it.
func (c *Conn) Login(user, pass string) error {
// The command line itself is never logged (see exec) — a LOGIN line IS the
// credential.
if _, err := c.exec(fmt.Sprintf("LOGIN %s %s", quote(user), quote(pass))); err != nil {
return fmt.Errorf("email: login: %w", err)
}
return nil
}
// Select opens a mailbox read-only. EXAMINE, not SELECT: read-only at the
// protocol level means no command in this session can change a flag, expunge a
// message, or move anything, even by mistake.
func (c *Conn) Select(mailbox string) error {
if _, err := c.exec(fmt.Sprintf("EXAMINE %s", quote(mailbox))); err != nil {
return fmt.Errorf("email: examine %s: %w", mailbox, err)
}
return nil
}
// SearchSince returns the UIDs of messages received on or after since. An
// unlimited search is not offered: the first poll against a years-old mailbox
// would otherwise fetch everything and hand a decade of mail to the model.
//
// The IMAP SINCE key has date granularity (and compares the server's internal
// date), so the result can include messages slightly older than since. The
// caller dedupes by UID anyway, so a wider window costs one extra fetch.
func (c *Conn) SearchSince(since time.Time) ([]uint32, error) {
cmd := fmt.Sprintf("UID SEARCH SINCE %s", since.Format("2-Jan-2006"))
lines, err := c.exec(cmd)
if err != nil {
return nil, fmt.Errorf("email: search: %w", err)
}
var uids []uint32
for _, l := range lines {
rest, ok := untagged(l, "SEARCH")
if !ok {
continue
}
for _, f := range strings.Fields(rest) {
n, err := strconv.ParseUint(f, 10, 32)
if err == nil {
uids = append(uids, uint32(n))
}
}
}
return uids, nil
}
var literalSize = regexp.MustCompile(`\{(\d+)\}$`)
// Fetch returns the raw RFC 5322 bytes of one message, by UID.
//
// Returns (nil, nil) when the UID no longer exists — a message he deleted
// between SEARCH and FETCH is normal, not an error.
func (c *Conn) Fetch(uid uint32) ([]byte, error) {
tag := c.nextTag()
if err := c.send(fmt.Sprintf("%s UID FETCH %d (BODY.PEEK[])", tag, uid)); err != nil {
return nil, err
}
var raw []byte
for {
line, err := c.readLine()
if err != nil {
return nil, fmt.Errorf("email: fetch %d: %w", uid, err)
}
if done, err := c.tagged(tag, line); done {
if err != nil {
return nil, fmt.Errorf("email: fetch %d: %w", uid, err)
}
return raw, nil
}
m := literalSize.FindStringSubmatch(strings.TrimSpace(line))
if m == nil {
continue
}
n, err := strconv.Atoi(m[1])
if err != nil {
continue
}
buf := make([]byte, n)
if _, err := io.ReadFull(c.r, buf); err != nil {
return nil, fmt.Errorf("email: fetch %d: literal: %w", uid, err)
}
if raw == nil {
raw = buf
}
}
}
// Logout ends the session politely. A failure is not worth reporting — the
// connection is being closed either way.
func (c *Conn) Logout() {
_, _ = c.exec("LOGOUT")
}
// ---- protocol plumbing -----------------------------------------------------
func (c *Conn) nextTag() string {
c.tag++
return fmt.Sprintf("a%03d", c.tag)
}
// exec sends one command and returns the untagged response lines.
//
// Neither the command nor the response is ever logged here. LOGIN goes through
// this function, and a debug line "sent: a001 LOGIN ..." is how a credential
// ends up in a log file forever.
func (c *Conn) exec(cmd string) ([]string, error) {
tag := c.nextTag()
if err := c.send(tag + " " + cmd); err != nil {
return nil, err
}
var lines []string
for {
line, err := c.readLine()
if err != nil {
return nil, err
}
if done, err := c.tagged(tag, line); done {
return lines, err
}
lines = append(lines, line)
// A response line may carry a literal (e.g. a header FETCH). Nothing we
// send asks for one outside Fetch, but skip it if it appears so the
// stream stays aligned.
if m := literalSize.FindStringSubmatch(strings.TrimSpace(line)); m != nil {
if n, err := strconv.Atoi(m[1]); err == nil {
if _, err := io.CopyN(io.Discard, c.r, int64(n)); err != nil {
return nil, err
}
}
}
}
}
// tagged reports whether line completes the command with this tag, and turns a
// NO/BAD completion into an error. The error text is the server's, which never
// echoes a password.
func (c *Conn) tagged(tag, line string) (bool, error) {
if !strings.HasPrefix(line, tag+" ") {
return false, nil
}
rest := strings.TrimSpace(line[len(tag):])
switch {
case strings.HasPrefix(rest, "OK"):
return true, nil
case strings.HasPrefix(rest, "NO"), strings.HasPrefix(rest, "BAD"):
return true, fmt.Errorf("server said: %s", rest)
default:
return true, fmt.Errorf("unexpected completion: %s", rest)
}
}
func (c *Conn) send(line string) error {
c.setDeadline()
if _, err := io.WriteString(c.rwc, line+"\r\n"); err != nil {
return fmt.Errorf("email: write: %w", err)
}
return nil
}
func (c *Conn) readLine() (string, error) {
c.setDeadline()
line, err := c.r.ReadString('\n')
if err != nil {
return "", err
}
return strings.TrimRight(line, "\r\n"), nil
}
// setDeadline applies the per-connection timeout when the transport supports
// one. A hung IMAP server must not park the poller forever.
func (c *Conn) setDeadline() {
if c.timeout <= 0 {
return
}
if d, ok := c.rwc.(interface{ SetDeadline(time.Time) error }); ok {
_ = d.SetDeadline(time.Now().Add(c.timeout))
}
}
// untagged splits "* SEARCH 1 2 3" into its payload when the key matches.
func untagged(line, key string) (string, bool) {
if !strings.HasPrefix(line, "* ") {
return "", false
}
rest := strings.TrimSpace(line[2:])
if !strings.HasPrefix(rest, key) {
return "", false
}
return strings.TrimSpace(rest[len(key):]), true
}
// quote renders an IMAP quoted string. Passwords routinely contain characters
// that would otherwise end the argument early, and CR/LF are stripped rather
// than escaped because there is no legal way to send them — a credential file
// with a stray newline must not become a second command.
func quote(s string) string {
s = strings.NewReplacer("\r", "", "\n", "").Replace(s)
return `"` + strings.NewReplacer(`\`, `\\`, `"`, `\"`).Replace(s) + `"`
}
+162
View File
@@ -0,0 +1,162 @@
package email
import (
"bufio"
"fmt"
"net"
"strconv"
"strings"
"testing"
"time"
)
// fakeIMAP is a scripted server: enough of IMAP to exercise the client, and
// nothing more. It records the commands it received so a test can assert on the
// protocol (BODY.PEEK rather than BODY, EXAMINE rather than SELECT).
type fakeIMAP struct {
msgs map[uint32]string
uids []uint32
cmds []string
failOn string // substring of a command to answer NO
}
func (f *fakeIMAP) serve(t *testing.T, c net.Conn) {
t.Helper()
defer c.Close()
fmt.Fprint(c, "* OK fake IMAP ready\r\n")
r := bufio.NewReader(c)
for {
line, err := r.ReadString('\n')
if err != nil {
return
}
line = strings.TrimRight(line, "\r\n")
parts := strings.SplitN(line, " ", 2)
if len(parts) != 2 {
return
}
tag, cmd := parts[0], parts[1]
f.cmds = append(f.cmds, cmd)
if f.failOn != "" && strings.Contains(cmd, f.failOn) {
fmt.Fprintf(c, "%s NO computer says no\r\n", tag)
continue
}
upper := strings.ToUpper(cmd)
switch {
case strings.HasPrefix(upper, "LOGIN"), strings.HasPrefix(upper, "EXAMINE"):
fmt.Fprintf(c, "%s OK done\r\n", tag)
case strings.HasPrefix(upper, "UID SEARCH"):
var ids []string
for _, u := range f.uids {
ids = append(ids, strconv.FormatUint(uint64(u), 10))
}
fmt.Fprintf(c, "* SEARCH %s\r\n", strings.Join(ids, " "))
fmt.Fprintf(c, "%s OK search done\r\n", tag)
case strings.HasPrefix(upper, "UID FETCH"):
uid64, _ := strconv.ParseUint(strings.Fields(cmd)[2], 10, 32)
raw, ok := f.msgs[uint32(uid64)]
if ok {
fmt.Fprintf(c, "* 1 FETCH (UID %d BODY[] {%d}\r\n", uid64, len(raw))
fmt.Fprint(c, raw)
fmt.Fprint(c, ")\r\n")
}
fmt.Fprintf(c, "%s OK fetch done\r\n", tag)
case strings.HasPrefix(upper, "LOGOUT"):
fmt.Fprint(c, "* BYE\r\n")
fmt.Fprintf(c, "%s OK bye\r\n", tag)
return
default:
fmt.Fprintf(c, "%s BAD unknown\r\n", tag)
}
}
}
// dialFake wires a client Conn to an in-process server over net.Pipe.
func dialFake(t *testing.T, f *fakeIMAP) *Conn {
t.Helper()
cli, srv := net.Pipe()
go f.serve(t, srv)
c, err := NewConn(cli, 5*time.Second)
if err != nil {
t.Fatalf("greeting: %v", err)
}
t.Cleanup(func() { c.Close() })
return c
}
func TestIMAPRoundTrip(t *testing.T) {
body := "Subject: hello\r\nContent-Type: text/plain; charset=utf-8\r\n\r\nCall the bank.\r\n"
f := &fakeIMAP{uids: []uint32{4, 9}, msgs: map[uint32]string{4: body, 9: body}}
c := dialFake(t, f)
if err := c.Login("kami", `pa"ss\word`); err != nil {
t.Fatalf("login: %v", err)
}
if err := c.Select("INBOX"); err != nil {
t.Fatalf("select: %v", err)
}
uids, err := c.SearchSince(time.Date(2026, 8, 1, 0, 0, 0, 0, time.UTC))
if err != nil {
t.Fatalf("search: %v", err)
}
if len(uids) != 2 || uids[0] != 4 || uids[1] != 9 {
t.Fatalf("uids = %v, want [4 9]", uids)
}
raw, err := c.Fetch(9)
if err != nil {
t.Fatalf("fetch: %v", err)
}
if string(raw) != body {
t.Errorf("fetched %q, want the literal verbatim", raw)
}
c.Logout()
joined := strings.Join(f.cmds, "\n")
// Read-only at the protocol level, and peeking — Maven must leave no trace
// of having read his mail.
if !strings.Contains(joined, "EXAMINE") || strings.Contains(joined, "SELECT ") {
t.Errorf("want EXAMINE (read-only), got:\n%s", joined)
}
if !strings.Contains(joined, "BODY.PEEK[]") {
t.Errorf("want BODY.PEEK, got:\n%s", joined)
}
// The password must have been quoted and escaped, not truncated at the quote.
if !strings.Contains(joined, `"pa\"ss\\word"`) {
t.Errorf("password not quoted correctly:\n%s", joined)
}
// SINCE must carry the IMAP date form.
if !strings.Contains(joined, "SINCE 1-Aug-2026") {
t.Errorf("want a SINCE date, got:\n%s", joined)
}
}
func TestIMAPServerNoIsAnError(t *testing.T) {
f := &fakeIMAP{failOn: "LOGIN"}
c := dialFake(t, f)
err := c.Login("kami", "wrong")
if err == nil {
t.Fatal("a NO completion must be an error")
}
// The error is the server's text; it must not echo the credential.
if strings.Contains(err.Error(), "wrong") {
t.Errorf("error leaks the password: %v", err)
}
}
func TestIMAPFetchMissingUID(t *testing.T) {
f := &fakeIMAP{uids: []uint32{1}, msgs: map[uint32]string{}}
c := dialFake(t, f)
raw, err := c.Fetch(1)
if err != nil {
t.Fatalf("fetch: %v", err)
}
if raw != nil {
t.Errorf("a vanished UID should give nil, got %q", raw)
}
}
func TestQuoteStripsNewlines(t *testing.T) {
if got := quote("pass\r\nA1 LOGOUT"); strings.ContainsAny(got, "\r\n") {
t.Errorf("quote kept a line break: %q", got)
}
}
+80
View File
@@ -0,0 +1,80 @@
package email
import (
"net/mail"
"strings"
)
// The junk filter — the cheapest and most important half of reading mail.
//
// A mailbox is mostly machine-generated: newsletters, receipts nobody acts on,
// social notifications, marketing. Sending all of it to a 1.7B and asking "is
// there a task here" produces confident nonsense at a rate proportional to the
// volume, so junk is decided by HEADERS, before any model sees the message.
//
// The rules are all bulk-mail markers that senders set on themselves, never
// guesses about content:
//
// - List-Unsubscribe / List-Id — by definition a mailing list. If he can
// unsubscribe from it, it is not asking him to do anything.
// - Precedence: bulk|junk|list — the sender declaring itself bulk.
// - Auto-Submitted other than "no" (RFC 3834) — generated by a machine.
// - X-Spam-Flag: YES, X-Spam-Status: Yes — the spam filter upstream already
// decided; we do not second-guess it in the other direction.
// - X-GM-LABELS / X-Gmail-Labels containing a Gmail category — Gmail's own
// Promotions/Social/Forums/Spam classification, when the server sends it.
//
// Deliberately NOT here: sender allow/deny lists and subject keyword matching.
// Both are configuration that ages badly and both would be a place for his
// contacts to end up in a config file. If a real correspondent's mail is being
// dropped, the fix is a rule about a header, not a list of names.
//
// A junk verdict never deletes anything and never touches a flag on the server.
// It means "do not spend the model on this", nothing more.
// junkHeaders — headers whose mere presence marks bulk mail.
var junkPresence = []string{"List-Unsubscribe", "List-Id", "List-Post"}
// gmailCategories — Gmail's category labels, lowercased as they appear in
// X-GM-LABELS. "important" and "inbox" are labels too, and are NOT categories.
// Matching is by these exact tokens (substring is fine — they are namespaced
// and cannot appear in a hand-made label by accident), so a user label named
// "Social Club" is not mistaken for Gmail's Social category.
var gmailCategories = []string{
"category_promotions", "category_social", "category_forums", "category_updates",
`\spam`, `\junk`,
}
// classifyJunk returns whether the message is bulk/automated and why. The
// reason is a short header name, safe to log — it names the marker, never the
// sender or the subject.
func classifyJunk(h mail.Header) (bool, string) {
for _, name := range junkPresence {
if strings.TrimSpace(h.Get(name)) != "" {
return true, strings.ToLower(name)
}
}
switch strings.ToLower(strings.TrimSpace(h.Get("Precedence"))) {
case "bulk", "junk", "list":
return true, "precedence"
}
if v := strings.ToLower(strings.TrimSpace(h.Get("Auto-Submitted"))); v != "" && v != "no" {
return true, "auto-submitted"
}
if strings.EqualFold(strings.TrimSpace(h.Get("X-Spam-Flag")), "yes") {
return true, "x-spam-flag"
}
if v := strings.ToLower(strings.TrimSpace(h.Get("X-Spam-Status"))); strings.HasPrefix(v, "yes") {
return true, "x-spam-status"
}
labels := strings.ToLower(h.Get("X-GM-LABELS") + " " + h.Get("X-Gmail-Labels"))
for _, c := range gmailCategories {
if c == "" {
continue
}
if strings.Contains(labels, c) {
return true, "gmail-category"
}
}
return false, ""
}
+59
View File
@@ -0,0 +1,59 @@
package email
import (
"net/mail"
"strings"
"testing"
)
func headers(t *testing.T, raw string) mail.Header {
t.Helper()
m, err := mail.ReadMessage(strings.NewReader(strings.ReplaceAll(raw, "\n", "\r\n") + "\r\n\r\nbody\r\n"))
if err != nil {
t.Fatalf("read headers: %v", err)
}
return m.Header
}
func TestClassifyJunk(t *testing.T) {
cases := []struct {
name string
raw string
junk bool
reason string
}{
{"personal", "From: a@b.c\nSubject: привет", false, ""},
{"list-unsubscribe", "From: a@b.c\nList-Unsubscribe: <mailto:u@b.c>", true, "list-unsubscribe"},
{"list-id", "From: a@b.c\nList-Id: <golang-nuts.example>", true, "list-id"},
{"precedence bulk", "From: a@b.c\nPrecedence: bulk", true, "precedence"},
{"auto-submitted", "From: a@b.c\nAuto-Submitted: auto-generated", true, "auto-submitted"},
{"auto-submitted no", "From: a@b.c\nAuto-Submitted: no", false, ""},
{"spam flag", "From: a@b.c\nX-Spam-Flag: YES", true, "x-spam-flag"},
{"spam status", "From: a@b.c\nX-Spam-Status: Yes, score=9.1", true, "x-spam-status"},
{"spam status no", "From: a@b.c\nX-Spam-Status: No, score=0.1", false, ""},
{"gmail promo", "From: a@b.c\nX-Gmail-Labels: Inbox,CATEGORY_PROMOTIONS", true, "gmail-category"},
{"user label", "From: a@b.c\nX-Gmail-Labels: Social Club,Important", false, ""},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
junk, reason := classifyJunk(headers(t, c.raw))
if junk != c.junk || reason != c.reason {
t.Errorf("classifyJunk = (%v, %q), want (%v, %q)", junk, reason, c.junk, c.reason)
}
})
}
}
func TestNewsletterFixtureIsJunk(t *testing.T) {
msg, err := ParseMessage(9, fixture(t, "newsletter.eml"))
if err != nil {
t.Fatalf("parse: %v", err)
}
if !msg.Junk {
t.Fatal("a newsletter with List-Unsubscribe + Precedence: bulk must be junk")
}
// The reason is what gets logged, so it must never carry mail content.
if strings.Contains(msg.JunkReason, "@") || strings.Contains(msg.JunkReason, "Скидки") {
t.Errorf("junk reason leaks content: %q", msg.JunkReason)
}
}
+258
View File
@@ -0,0 +1,258 @@
// Package email is the reading half of the email reader (Vikunja #246,
// docs/plans/01-email-reader.md): a small IMAP client, a MIME-to-plaintext
// converter, and the junk filter that decides a message is not worth reading at
// all. Extraction lives in extract.go and writes nothing itself.
//
// Two constraints shape everything here, both from CLAUDE.md:
//
// - Mail is personal. Nothing in this package logs a body, a subject, or an
// address; callers get the text and decide. Mail text is never search input
// — no function here reaches the network except the IMAP connection itself.
// - Off unless configured. There is no default host, no default account, and
// no fallback that would make a mailbox get read because a field was empty.
//
// The IMAP subset is deliberately tiny (LOGIN, SELECT, UID SEARCH, UID FETCH
// with BODY.PEEK, LOGOUT). No IDLE: a poll every few minutes is what a task
// candidate needs, and IDLE would mean holding a connection and a credential
// open forever for latency nobody is waiting on.
package email
import (
"encoding/base64"
"fmt"
"io"
"mime"
"mime/multipart"
"mime/quotedprintable"
"net/mail"
"regexp"
"strings"
)
// MaxBodyBytes — how much of one message body is kept. A task hides in the
// first screenful; the rest is signature, quoted history and legal boilerplate,
// and it would only spend the resident model's 4096-token context.
const MaxBodyBytes = 4000
// Message — one mail, reduced to the fields extraction and review need.
//
// Raw is deliberately absent: once a message is parsed the original bytes are
// dropped, so no caller can accidentally log or forward the whole mail.
type Message struct {
UID uint32
From string
Subject string
Date string // as sent, unparsed — display only
Body string // plaintext, decoded, HTML-stripped, truncated
// Junk is set by the junk filter (see junk.go). A junk message is carried
// rather than dropped so the poller can count it and still mark it seen.
Junk bool
JunkReason string
}
// ParseMessage turns one RFC 5322 message into a Message.
//
// It never fails on a body it cannot understand: an unparsable or
// unsupported-charset body yields an empty Body and the headers still come
// through, because a subject line alone is often the whole task ("Счёт за
// интернет"). Only a message whose headers cannot be read at all is an error.
func ParseMessage(uid uint32, raw []byte) (Message, error) {
m, err := mail.ReadMessage(strings.NewReader(string(raw)))
if err != nil {
return Message{}, fmt.Errorf("email: parse message: %w", err)
}
msg := Message{
UID: uid,
From: decodeHeader(m.Header.Get("From")),
Subject: decodeHeader(m.Header.Get("Subject")),
Date: m.Header.Get("Date"),
}
msg.Junk, msg.JunkReason = classifyJunk(m.Header)
body, err := plaintextBody(m.Header.Get("Content-Type"), m.Header.Get("Content-Transfer-Encoding"), m.Body)
if err == nil {
msg.Body = truncate(collapse(body), MaxBodyBytes)
}
return msg, nil
}
// plaintextBody walks the MIME tree and returns the best plaintext it can.
//
// Preference order inside a multipart: text/plain first, text/html stripped
// only when there is no plain part. multipart/mixed attachments are skipped
// wholesale — an attachment is a file, not a sentence, and reading one would
// mean parsing arbitrary formats from the network.
func plaintextBody(contentType, encoding string, body io.Reader) (string, error) {
mediaType, params, err := mime.ParseMediaType(contentType)
if contentType == "" || err != nil {
// No Content-Type at all is legal and means text/plain; a broken one is
// treated the same rather than dropping the message.
mediaType, params = "text/plain", nil
}
switch {
case strings.HasPrefix(mediaType, "multipart/"):
boundary := params["boundary"]
if boundary == "" {
return "", fmt.Errorf("email: multipart without boundary")
}
return multipartText(multipart.NewReader(body, boundary))
case mediaType == "text/html":
raw, err := decodeBody(body, encoding, params["charset"])
if err != nil {
return "", err
}
return stripHTML(raw), nil
case mediaType == "text/plain":
return decodeBody(body, encoding, params["charset"])
default:
// A single-part non-text message (a bare PDF, say). No body, headers only.
return "", nil
}
}
// multipartText reads one multipart level, recursing into nested multiparts.
// Returns the plain part if any part yielded one, else the stripped HTML.
func multipartText(mr *multipart.Reader) (string, error) {
var plain, html string
for {
part, err := mr.NextPart()
if err == io.EOF {
break
}
if err != nil {
// A truncated multipart still gives up whatever came before it.
break
}
if part.FileName() != "" {
part.Close()
continue // attachment
}
ct := part.Header.Get("Content-Type")
mediaType, _, _ := mime.ParseMediaType(ct)
text, err := plaintextBody(ct, part.Header.Get("Content-Transfer-Encoding"), part)
part.Close()
if err != nil || strings.TrimSpace(text) == "" {
continue
}
if mediaType == "text/html" && !strings.HasPrefix(mediaType, "multipart/") {
if html == "" {
html = text
}
continue
}
if plain == "" {
plain = text
}
}
if strings.TrimSpace(plain) != "" {
return plain, nil
}
return html, nil
}
// decodeBody applies the transfer encoding, then the charset.
//
// Charset support is UTF-8 (and ASCII, its subset) only, on purpose: x/text's
// encoding tables are not vendored here, and guessing at windows-1251 bytes
// would feed the model mojibake it would happily extract a task from. An
// unsupported charset returns an error, which ParseMessage turns into an empty
// body — subject-only, which is honest.
func decodeBody(r io.Reader, encoding, charset string) (string, error) {
switch strings.ToLower(strings.TrimSpace(encoding)) {
case "quoted-printable":
r = quotedprintable.NewReader(r)
case "base64":
r = newBase64Reader(r)
}
b, err := io.ReadAll(io.LimitReader(r, 1<<20))
if err != nil && len(b) == 0 {
return "", fmt.Errorf("email: read body: %w", err)
}
switch cs := strings.ToLower(strings.TrimSpace(charset)); cs {
case "", "utf-8", "utf8", "us-ascii", "ascii":
return string(b), nil
default:
return "", fmt.Errorf("email: unsupported charset %q", cs)
}
}
// decodeHeader decodes RFC 2047 encoded words ("=?utf-8?B?...?="), which is how
// every Russian subject line arrives. Undecodable headers come back as-is
// rather than empty: a mangled subject is still a hint, and it is only ever
// shown to him as evidence.
func decodeHeader(v string) string {
dec := new(mime.WordDecoder)
out, err := dec.DecodeHeader(v)
if err != nil {
return collapse(v)
}
return collapse(out)
}
var (
scriptStyle = regexp.MustCompile(`(?is)<(script|style)\b[^>]*>.*?</\s*(script|style)\s*>`)
htmlBreak = regexp.MustCompile(`(?i)<\s*(br\s*/?|/p|/div|/tr|/li|/h[1-6])\s*>`)
htmlTag = regexp.MustCompile(`(?s)<[^>]*>`)
htmlComment = regexp.MustCompile(`(?s)<!--.*?-->`)
)
// stripHTML reduces an HTML part to text. A regex stripper, not a parser:
// x/net/html is not vendored, and the consumer is a model reading prose — a
// stray angle bracket costs nothing, whereas a new dependency for the privacy-
// sensitive path costs review.
func stripHTML(s string) string {
s = scriptStyle.ReplaceAllString(s, " ")
s = htmlComment.ReplaceAllString(s, " ")
s = htmlBreak.ReplaceAllString(s, "\n")
s = htmlTag.ReplaceAllString(s, " ")
return unescapeEntities(s)
}
var entities = strings.NewReplacer(
"&nbsp;", " ", "&amp;", "&", "&lt;", "<", "&gt;", ">",
"&quot;", `"`, "&#39;", "'", "&apos;", "'", "&mdash;", "—", "&ndash;", "",
)
func unescapeEntities(s string) string { return entities.Replace(s) }
// collapse squeezes runs of whitespace, keeping single newlines. Mail bodies
// arrive with hard-wrapped lines and blocks of blank space; the model does not
// need them and they are pure context budget.
func collapse(s string) string {
lines := strings.Split(strings.ReplaceAll(s, "\r\n", "\n"), "\n")
var out []string
blank := 0
for _, l := range lines {
l = strings.TrimSpace(strings.Join(strings.Fields(l), " "))
if l == "" {
blank++
if blank > 1 {
continue
}
out = append(out, "")
continue
}
blank = 0
out = append(out, l)
}
return strings.TrimSpace(strings.Join(out, "\n"))
}
// truncate cuts to n bytes on a rune boundary.
func truncate(s string, n int) string {
if len(s) <= n {
return s
}
cut := s[:n]
for len(cut) > 0 && !isRuneStart(cut[len(cut)-1]) {
cut = cut[:len(cut)-1]
}
return strings.TrimSpace(cut) + "…"
}
func isRuneStart(b byte) bool { return b&0xC0 != 0x80 }
// newBase64Reader — base64.NewDecoder already skips the CRLFs mail bodies wrap
// with, so this is only a named seam for decodeBody to read cleanly.
func newBase64Reader(r io.Reader) io.Reader {
return base64.NewDecoder(base64.StdEncoding, r)
}
+111
View File
@@ -0,0 +1,111 @@
package email
import (
"os"
"path/filepath"
"strings"
"testing"
)
func fixture(t *testing.T, name string) []byte {
t.Helper()
b, err := os.ReadFile(filepath.Join("testdata", name))
if err != nil {
t.Fatalf("read fixture %s: %v", name, err)
}
return b
}
func TestParsePlainRussian(t *testing.T) {
msg, err := ParseMessage(7, fixture(t, "plain_ru.eml"))
if err != nil {
t.Fatalf("parse: %v", err)
}
if msg.UID != 7 {
t.Errorf("uid = %d, want 7", msg.UID)
}
if want := "Нужно закрыть задачу"; msg.Subject != want {
t.Errorf("subject = %q, want %q", msg.Subject, want)
}
if !strings.Contains(msg.From, "Антон") {
t.Errorf("from = %q, want the decoded display name", msg.From)
}
if !strings.Contains(msg.Body, "Надо отправить акт до пятницы.") {
t.Errorf("body = %q, want the quoted-printable text decoded", msg.Body)
}
if msg.Junk {
t.Errorf("a personal mail must not be junk (%s)", msg.JunkReason)
}
}
func TestParseHTMLOnlyIsStripped(t *testing.T) {
msg, err := ParseMessage(1, fixture(t, "html_only.eml"))
if err != nil {
t.Fatalf("parse: %v", err)
}
if strings.Contains(msg.Body, "<") || strings.Contains(msg.Body, "color:red") || strings.Contains(msg.Body, "x()") {
t.Errorf("body still has markup/script/style: %q", msg.Body)
}
for _, want := range []string{"Счёт за интернет: 700", "Оплатить до 5 августа."} {
if !strings.Contains(msg.Body, want) {
t.Errorf("body = %q, want it to contain %q", msg.Body, want)
}
}
// &nbsp; must have become a real space, not vanished into the number.
if strings.Contains(msg.Body, "&nbsp;") {
t.Errorf("entity left unescaped: %q", msg.Body)
}
}
func TestParsePrefersPlainAndSkipsAttachments(t *testing.T) {
msg, err := ParseMessage(2, fixture(t, "mixed_attachment.eml"))
if err != nil {
t.Fatalf("parse: %v", err)
}
if got := strings.TrimSpace(msg.Body); got != "Sign the contract before Monday." {
t.Errorf("body = %q, want the text/plain alternative only", got)
}
if strings.Contains(msg.Body, "PDF") {
t.Errorf("attachment bytes leaked into the body: %q", msg.Body)
}
}
// An unsupported charset must degrade to headers-only rather than to mojibake
// the model would then extract a task from.
func TestParseUnsupportedCharsetKeepsHeaders(t *testing.T) {
msg, err := ParseMessage(3, fixture(t, "cp1251.eml"))
if err != nil {
t.Fatalf("parse: %v", err)
}
if msg.Subject != "Legacy" {
t.Errorf("subject = %q, want Legacy", msg.Subject)
}
if msg.Body != "" {
t.Errorf("body = %q, want empty for an undecodable charset", msg.Body)
}
}
func TestParseTruncatesLongBody(t *testing.T) {
var b strings.Builder
b.WriteString("Subject: long\r\nContent-Type: text/plain; charset=utf-8\r\n\r\n")
for i := 0; i < 2000; i++ {
b.WriteString("длинная строка ")
}
msg, err := ParseMessage(4, []byte(b.String()))
if err != nil {
t.Fatalf("parse: %v", err)
}
if len(msg.Body) > MaxBodyBytes+8 {
t.Errorf("body kept %d bytes, want ≤ %d", len(msg.Body), MaxBodyBytes)
}
if !strings.HasSuffix(msg.Body, "…") {
t.Errorf("truncated body should be marked: %q", msg.Body[len(msg.Body)-20:])
}
}
func TestCollapseSqueezesBlankLines(t *testing.T) {
got := collapse(" a b \r\n\r\n\r\n\r\n c \r\n")
if got != "a b\n\nc" {
t.Errorf("collapse = %q, want %q", got, "a b\n\nc")
}
}
+7
View File
@@ -0,0 +1,7 @@
From: legacy@example.org
To: kami@example.org
Subject: Legacy
Date: Fri, 01 Aug 2026 05:00:00 +0400
Content-Type: text/plain; charset="windows-1251"
Ï
+13
View File
@@ -0,0 +1,13 @@
From: billing@isp.example
To: kami@example.org
Subject: =?utf-8?B?0KHRh9GR0YIg0LfQsCDQuNC90YLQtdGA0L3QtdGC?=
Date: Fri, 01 Aug 2026 08:00:00 +0400
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="B1"
--B1
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: base64
PGh0bWw+PGhlYWQ+PHN0eWxlPnB7Y29sb3I6cmVkfTwvc3R5bGU+PC9oZWFkPjxib2R5PjxwPtCh0YfRkdGCINC30LAg0LjQvdGC0LXRgNC90LXRgjogNzAwJm5ic3A74oK9PC9wPjxwPtCe0L/Qu9Cw0YLQuNGC0Ywg0LTQviA1INCw0LLQs9GD0YHRgtCwLjwvcD48c2NyaXB0PngoKTwvc2NyaXB0PjwvYm9keT48L2h0bWw+
--B1--
+26
View File
@@ -0,0 +1,26 @@
From: hr@work.example
To: kami@example.org
Subject: Contract
Date: Fri, 01 Aug 2026 07:00:00 +0400
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="M1"
--M1
Content-Type: multipart/alternative; boundary="A1"
--A1
Content-Type: text/plain; charset="utf-8"
Sign the contract before Monday.
--A1
Content-Type: text/html; charset="utf-8"
<p>Sign the contract before Monday.</p>
--A1--
--M1
Content-Type: application/pdf; name="contract.pdf"
Content-Disposition: attachment; filename="contract.pdf"
Content-Transfer-Encoding: base64
JVBERi0xLjQgbm90IHJlYWxseSBhIHBkZg==
--M1--
+9
View File
@@ -0,0 +1,9 @@
From: news@shop.example
To: kami@example.org
Subject: =?utf-8?B?0KHQutC40LTQutC4INGC0L7Qu9GM0LrQviDRgdC10LPQvtC00L3Rjw==?=
Date: Fri, 01 Aug 2026 06:00:00 +0400
List-Unsubscribe: <mailto:unsub@shop.example>
Precedence: bulk
Content-Type: text/plain; charset="utf-8"
Sale!
+14
View File
@@ -0,0 +1,14 @@
From: =?utf-8?B?0JDQvdGC0L7QvQ==?= <anton@example.org>
To: kami@example.org
Subject: =?utf-8?B?0J3Rg9C20L3QviDQt9Cw0LrRgNGL0YLRjCDQt9Cw0LTQsNGH0YM=?=
Date: Fri, 01 Aug 2026 09:12:00 +0400
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Message-ID: <plain-ru@example.org>
=D0=9F=D1=80=D0=B8=D0=B2=D0=B5=D1=82! =D0=9D=D0=B0=D0=B4=D0=BE =D0=BE=D1=82=
=D0=BF=D1=80=D0=B0=D0=B2=D0=B8=D1=82=D1=8C =D0=B0=D0=BA=D1=82 =D0=B4=D0=BE =
=D0=BF=D1=8F=D1=82=D0=BD=D0=B8=D1=86=D1=8B.
--
Anton
+38
View File
@@ -138,6 +138,44 @@ type CaptureTaskResp struct {
Created bool `json:"created"`
}
// IngestMailReq — one message a mail reader has fetched, handed to core for
// extraction (Vikunja #246).
//
// The mail reader (cmd/mavmaild) holds the IMAP credential and core never sees
// it, the same split mavpoll uses for the zenmoney token. What crosses this
// boundary is only the message text, because extraction runs on the resident
// model and llama-server lives inside core's process.
//
// Body is already plaintext and truncated by internal/email; core does not
// re-parse MIME and never stores the body. Junk means the reader's header
// filter already classified the message as bulk — core is told rather than
// asked, so a junk message can be counted without a model call.
//
// This method is available only when core has an email block configured AND a
// llama-server phraser; otherwise it answers ErrUnknownMethod, which is what
// "off unless configured" looks like at the wire.
type IngestMailReq struct {
Mailbox string `json:"mailbox"`
UID uint32 `json:"uid"`
From string `json:"from,omitempty"`
Subject string `json:"subject,omitempty"`
Date string `json:"date,omitempty"`
Body string `json:"body,omitempty"`
Junk bool `json:"junk,omitempty"`
}
// IngestMailResp — what core did with the message. TaskIDs are the rows
// CaptureTask returned; Created counts the ones that were new (a re-read
// mailbox dedupes to Created=0). Skipped is set when nothing was asked of the
// model at all — junk, or an empty message.
//
// Nothing here echoes the mail back. The reader logs counts.
type IngestMailResp struct {
TaskIDs []int64 `json:"task_ids,omitempty"`
Created int `json:"created"`
Skipped bool `json:"skipped,omitempty"`
}
type listTasksReq struct {
Status string `json:"status"` // "" all | "live" | candidate|open|done|dropped
}
+11
View File
@@ -448,6 +448,17 @@ func (c *Client) SetTaskStatus(ctx context.Context, id int64, status string, ts
return c.call(ctx, MethodSetTaskStatus, setTaskStatusReq{ID: id, Status: status, Ts: ts}, nil)
}
// IngestMail hands one fetched message to core for extraction. ErrUnknownMethod
// means core has no email block configured — the caller should stop asking, not
// retry.
func (c *Client) IngestMail(ctx context.Context, req IngestMailReq) (IngestMailResp, error) {
var r IngestMailResp
if err := c.call(ctx, MethodIngestMail, req, &r); err != nil {
return IngestMailResp{}, err
}
return r, nil
}
func (c *Client) DismissProposedRoutine(ctx context.Context, id int64) error {
return c.call(ctx, MethodDismissProposedRoutine, dismissProposedRoutineReq{ID: id}, nil)
}
+33
View File
@@ -565,3 +565,36 @@ func mustJSON(v any) []byte {
}
return b
}
// TestIngestMail_OffUnlessConfigured — with no IngestMailFn set (the default,
// and what an unconfigured core looks like) the method does not exist. A mail
// reader gets a refusal it can act on rather than a silent success.
func TestIngestMail_OffUnlessConfigured(t *testing.T) {
_, _, cli, _ := newServerWithStore(t)
if _, err := cli.IngestMail(context.Background(), IngestMailReq{Mailbox: "INBOX", UID: 1}); !errors.Is(err, ErrUnknownMethod) {
t.Fatalf("IngestMail error = %v, want ErrUnknownMethod", err)
}
}
// TestIngestMail_Hook — when the daemon wires the hook, the message crosses the
// boundary intact and the response comes back.
func TestIngestMail_Hook(t *testing.T) {
_, srv, cli, _ := newServerWithStore(t)
var got IngestMailReq
srv.IngestMailFn = func(_ context.Context, req IngestMailReq) (IngestMailResp, error) {
got = req
return IngestMailResp{TaskIDs: []int64{7}, Created: 1}, nil
}
resp, err := cli.IngestMail(context.Background(), IngestMailReq{
Mailbox: "INBOX", UID: 12, Subject: "Счёт", Body: "Оплатить.", Junk: false,
})
if err != nil {
t.Fatalf("IngestMail: %v", err)
}
if resp.Created != 1 || len(resp.TaskIDs) != 1 || resp.TaskIDs[0] != 7 {
t.Errorf("resp = %+v", resp)
}
if got.UID != 12 || got.Subject != "Счёт" || got.Body != "Оплатить." {
t.Errorf("req across the wire = %+v", got)
}
}
+33 -4
View File
@@ -421,6 +421,17 @@ type Server struct {
// Set by the daemon; nil ⇒ MethodStoreEncryptionKey returns ErrUnknownMethod.
WrapKeyFn WrapKeyFunc
// IngestMailFn — extracts task candidates from one fetched message. Set by
// the daemon only when an email block is configured AND there is a
// llama-server to extract with; nil ⇒ MethodIngestMail returns
// ErrUnknownMethod, so a mail reader pointed at a core that is not
// configured for mail is refused rather than silently ignored.
//
// Like StepUp/WrapKeyFn/UnlockFn this bypasses CoreAPI: it is not a store
// operation, it needs the resident model, and it must not become a method
// every CoreAPI implementation has to carry.
IngestMailFn IngestMailFunc
// UnlockFn — unwraps the store encryption key from the wrapped blob using
// the passkey credential public key, opens the encrypted store, and wires
// the rest of the daemon (voice, loop, delivery). Set by the daemon when
@@ -439,6 +450,9 @@ type WrapKeyFunc func(ctx context.Context, publicKey []byte) error
// public key and completes daemon initialization.
type UnlockFunc func(ctx context.Context, publicKey []byte) error
// IngestMailFunc — core-side mail extraction. Returns what was captured.
type IngestMailFunc func(ctx context.Context, req IngestMailReq) (IngestMailResp, error)
// CheckFunc — the auth hook signature. Wired by the daemon (auth.Gate.Check
// satisfies this); dispatch calls it once per request after param-unmarshal
// independence (it gets the raw params, may unmarshal what it needs — ipc
@@ -606,9 +620,10 @@ func withoutParams[R any](fn func(ctx context.Context, api CoreAPI) (R, error))
// existed) as an argument — so SetAPI's runtime swap (the unlock transition)
// is still honored on the very next request with no extra plumbing here.
//
// MethodAssertStepUp, MethodStoreEncryptionKey and MethodUnlock are NOT in
// this table: they bypass CoreAPI entirely (s.StepUp / s.WrapKeyFn /
// s.UnlockFn), so dispatch special-cases them before consulting the table.
// MethodAssertStepUp, MethodStoreEncryptionKey, MethodUnlock and
// MethodIngestMail are NOT in this table: they bypass CoreAPI entirely
// (s.StepUp / s.WrapKeyFn / s.UnlockFn / s.IngestMailFn), so dispatch
// special-cases them before consulting the table.
var methodTable = map[Method]handlerFunc{
MethodWriteFact: withParams(func(ctx context.Context, api CoreAPI, p WriteFactReq) (idResp, error) {
id, err := api.WriteFact(ctx, p)
@@ -816,7 +831,7 @@ func (s *Server) dispatch(ctx context.Context, req Request) (json.RawMessage, er
}
}
// These three bypass CoreAPI entirely — they drive Server fields set
// These bypass CoreAPI entirely — they drive Server fields set
// directly by the daemon (StepUp / WrapKeyFn / UnlockFn), not store
// state, so they can never be table entries keyed on a CoreAPI method.
switch req.Method {
@@ -845,6 +860,20 @@ func (s *Server) dispatch(ctx context.Context, req Request) (json.RawMessage, er
return marshalResult(nil), s.UnlockFn(ctx, p.PublicKey)
}
return nil, fmt.Errorf("%w: %s", ErrUnknownMethod, req.Method)
case MethodIngestMail:
if s.IngestMailFn != nil {
var p IngestMailReq
if err := unmarshalParams(req.Params, &p); err != nil {
return nil, err
}
resp, err := s.IngestMailFn(ctx, p)
if err != nil {
return nil, err
}
return marshalResult(resp), nil
}
return nil, fmt.Errorf("%w: %s", ErrUnknownMethod, req.Method)
}
h, ok := methodTable[req.Method]
+1
View File
@@ -50,6 +50,7 @@ const (
MethodCaptureTask Method = "capture_task"
MethodListTasks Method = "list_tasks"
MethodSetTaskStatus Method = "set_task_status"
MethodIngestMail Method = "ingest_mail"
)
// Request — one frame from module to core. Params is the JSON-encoded argument
+71
View File
@@ -0,0 +1,71 @@
package router
import "strings"
// Money questions, matched deterministically (Vikunja #125).
//
// No new intent, for the same reason as tasks: the intent enum is a contract
// with the relabelling prompt. "сколько я потратил?" is a query; which figure
// it asks for is a lookup, not something to ask a 1.7B — and a model asked to
// invent a spending total will happily do it.
// MoneyWindow — which period a money question asks about.
type MoneyWindow int
const (
MoneyNone MoneyWindow = iota
MoneyToday
MoneyMonth
)
// moneyNouns — the words that make a question be about his money.
var moneyNouns = []string{
"потратил", "потратила", "тратил", "траты", "трат", "расходы", "расходов",
"заработал", "потрачено", "денег", "spend", "spent", "expenses",
}
// ParseMoneyQuery reports whether an utterance asks about spending or income,
// and over which window. Defaults to the month: "сколько я потратил?" without a
// period is the month-to-date question, which is the one worth answering.
//
// Narrow on purpose. A money noun alone is not enough — "я потратил весь день
// на это" is him talking about his day, so an amount word or an explicit
// question word has to be there too.
func ParseMoneyQuery(text string) (MoneyWindow, bool) {
toks := planTokens(text)
if len(toks) == 0 {
return MoneyNone, false
}
hasNoun := false
for _, t := range toks {
for _, n := range moneyNouns {
if t == n {
hasNoun = true
}
}
}
if !hasNoun {
return MoneyNone, false
}
// "весь день", "время", "силы" — spending that is not money.
for _, t := range toks {
switch t {
case "день", "дня", "время", "времени", "силы", "сил", "нервы":
return MoneyNone, false
}
}
asking := hasTok(toks, "сколько") || hasTok(toks, "какие") || hasTok(toks, "покажи") ||
hasTok(toks, "how") || hasTok(toks, "much") || hasTok(toks, "my") ||
hasTok(toks, "мои") || hasTok(toks, "траты") || hasTok(toks, "расходы")
if !asking {
return MoneyNone, false
}
lower := strings.ToLower(text)
switch {
case hasTok(toks, "сегодня") || strings.Contains(lower, "today"):
return MoneyToday, true
case hasTok(toks, "месяц") || hasTok(toks, "месяце") || strings.Contains(lower, "month"):
return MoneyMonth, true
}
return MoneyMonth, true
}
+31
View File
@@ -0,0 +1,31 @@
package router
import "testing"
func TestParseMoneyQuery(t *testing.T) {
cases := []struct {
in string
window MoneyWindow
ok bool
}{
{"сколько я потратил сегодня?", MoneyToday, true},
{"сколько я потратил в этом месяце?", MoneyMonth, true},
{"сколько я потратил?", MoneyMonth, true}, // month-to-date by default
{"покажи мои траты", MoneyMonth, true},
{"какие у меня расходы за месяц", MoneyMonth, true},
{"how much did I spend today", MoneyToday, true},
{"сколько я заработал в этом месяце", MoneyMonth, true},
// Not about money.
{"я потратил весь день на это", MoneyNone, false},
{"потратил много сил", MoneyNone, false},
{"какая погода?", MoneyNone, false},
{"я купил молоко", MoneyNone, false},
{"", MoneyNone, false},
}
for _, c := range cases {
w, ok := ParseMoneyQuery(c.in)
if ok != c.ok || w != c.window {
t.Errorf("ParseMoneyQuery(%q) = (%v, %v), want (%v, %v)", c.in, w, ok, c.window, c.ok)
}
}
}
+267
View File
@@ -0,0 +1,267 @@
// Package zenmoney reads spending and income from ZenMoney's /v8/diff/ API
// (Vikunja #125).
//
// Trust boundary: ZenMoney, not Maven. They already hold his bank sessions —
// this package only reads back what they have, over a token that lives in the
// poller module and is never handed to core. Nothing here writes to ZenMoney;
// diff is called read-only (an empty change set in, a change set out).
//
// Two rules the code exists to enforce:
//
// - NEVER invent a number. Every figure in a Summary is a sum of amounts the
// API returned. A request that fails, or returns nothing, produces no
// summary and therefore no fact — silence, not a zero. A confidently wrong
// "ты потратил 0" is worse than no answer.
// - His money is never search input. This package holds no notes, no
// utterances and no persona text, and it has no path to the external search
// capability. The only thing that leaves the box here is the diff request
// itself, to the service that already has the data.
package zenmoney
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"sort"
"strings"
"time"
)
// DefaultBaseURL — ZenMoney's API root. Overridable so the tests can point at
// an httptest server replaying a recorded response.
const DefaultBaseURL = "https://api.zenmoney.ru"
// Client is a ZenMoney diff reader. The token is held here, in the poller's
// address space; core never receives it and never learns it exists.
type Client struct {
BaseURL string
Token string
HTTP *http.Client
}
// New returns a client with a bounded HTTP timeout. An empty token is a
// programming error the caller must catch — the capability is off unless
// configured, so a client is only ever built when a token was supplied.
func New(token, baseURL string, timeout time.Duration) (*Client, error) {
if strings.TrimSpace(token) == "" {
return nil, fmt.Errorf("zenmoney: empty token")
}
if baseURL == "" {
baseURL = DefaultBaseURL
}
if timeout <= 0 {
timeout = 20 * time.Second
}
return &Client{
BaseURL: strings.TrimRight(baseURL, "/"),
Token: token,
HTTP: &http.Client{Timeout: timeout},
}, nil
}
// diffRequest — the smallest body /v8/diff/ accepts. serverTimestamp is the
// incremental cursor: the server returns objects changed at or after it.
type diffRequest struct {
CurrentClientTimestamp int64 `json:"currentClientTimestamp"`
ServerTimestamp int64 `json:"serverTimestamp"`
}
// diffResponse — only the fields spending needs. ZenMoney returns a dozen more
// object types (tags, merchants, budgets, reminders); decoding them would mean
// holding more of his financial life in memory than the question needs.
type diffResponse struct {
ServerTimestamp int64 `json:"serverTimestamp"`
Instrument []instrument `json:"instrument"`
Transaction []transaction `json:"transaction"`
}
type instrument struct {
ID int64 `json:"id"`
ShortTitle string `json:"shortTitle"`
}
type transaction struct {
ID string `json:"id"`
Date string `json:"date"` // "2026-07-15"
Deleted bool `json:"deleted"`
Income float64 `json:"income"`
Outcome float64 `json:"outcome"`
IncomeInstrument int64 `json:"incomeInstrument"`
OutcomeInstrmnt int64 `json:"outcomeInstrument"`
IncomeAccount string `json:"incomeAccount"`
OutcomeAccount string `json:"outcomeAccount"`
}
// Money — an amount in one currency. Kept as the currency's own short title
// ("RUB", "EUR") rather than converted: ZenMoney's rates are a snapshot, and
// converting would turn a figure he can check against his bank into one he
// cannot.
type Money struct {
Currency string `json:"currency"`
Amount float64 `json:"amount"`
}
// Summary — what was spent and earned over a window, per currency, plus how
// many transactions it was computed from. Count is the honesty check: a
// summary built from zero transactions is not "you spent nothing", it is "there
// was nothing to read", and callers treat it as no answer.
type Summary struct {
From, To time.Time
Spent []Money `json:"spent"`
Earned []Money `json:"earned"`
Count int `json:"count"`
// ServerTimestamp — the cursor the API returned, for the caller to log or
// carry. Not used as an incremental cursor for summaries; see Since.
ServerTimestamp int64 `json:"-"`
}
// Since returns the summary of transactions dated in [from, to).
//
// The diff cursor is set to `from` so the server only sends objects changed
// since then, which for a "this month" window is everything filed this month.
// The caveat, deliberately accepted: a transaction he EDITED this month but
// dated last month arrives too, and is then excluded by date — so editing old
// records cannot inflate this month's total. The reverse case (a transaction
// dated this month, filed and last changed before `from`) cannot exist.
func (c *Client) Since(ctx context.Context, from, to time.Time) (Summary, error) {
resp, err := c.diff(ctx, from.Unix())
if err != nil {
return Summary{}, err
}
return summarize(resp, from, to), nil
}
func (c *Client) diff(ctx context.Context, serverTimestamp int64) (diffResponse, error) {
body, err := json.Marshal(diffRequest{
CurrentClientTimestamp: time.Now().Unix(),
ServerTimestamp: serverTimestamp,
})
if err != nil {
return diffResponse{}, err
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.BaseURL+"/v8/diff/", bytes.NewReader(body))
if err != nil {
return diffResponse{}, err
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", "Bearer "+c.Token)
hc := c.HTTP
if hc == nil {
hc = &http.Client{Timeout: 20 * time.Second}
}
res, err := hc.Do(req)
if err != nil {
return diffResponse{}, err
}
defer res.Body.Close()
raw, err := io.ReadAll(io.LimitReader(res.Body, 32<<20))
if err != nil {
return diffResponse{}, err
}
if res.StatusCode != http.StatusOK {
// The status only. The body of a failed diff can echo account data, and
// this string reaches the log.
return diffResponse{}, fmt.Errorf("zenmoney diff: %s", res.Status)
}
var out diffResponse
if err := json.Unmarshal(raw, &out); err != nil {
return diffResponse{}, fmt.Errorf("zenmoney diff: decode: %w", err)
}
return out, nil
}
// summarize sums the transactions dated inside the window.
//
// Excluded, in order: deleted rows (ZenMoney tombstones rather than removes),
// transfers and currency exchanges (income and outcome both non-zero — moving
// his own money between his own accounts is not spending), and anything dated
// outside the window.
func summarize(resp diffResponse, from, to time.Time) Summary {
cur := map[int64]string{}
for _, in := range resp.Instrument {
cur[in.ID] = in.ShortTitle
}
spent := map[string]float64{}
earned := map[string]float64{}
count := 0
for _, t := range resp.Transaction {
if t.Deleted {
continue
}
d, err := time.ParseInLocation("2006-01-02", t.Date, from.Location())
if err != nil {
continue // an undated row is not a number we can place
}
if d.Before(from) || !d.Before(to) {
continue
}
if t.Income > 0 && t.Outcome > 0 {
continue // transfer / exchange
}
switch {
case t.Outcome > 0:
spent[currency(cur, t.OutcomeInstrmnt)] += t.Outcome
count++
case t.Income > 0:
earned[currency(cur, t.IncomeInstrument)] += t.Income
count++
}
}
return Summary{
From: from, To: to,
Spent: sortMoney(spent), Earned: sortMoney(earned),
Count: count, ServerTimestamp: resp.ServerTimestamp,
}
}
// currency names the instrument, or says it does not know. An unknown id keeps
// the amount rather than dropping it: a sum without a currency label is still
// his money, and silently discarding it would understate the total.
func currency(names map[int64]string, id int64) string {
if s := names[id]; s != "" {
return s
}
return "?"
}
// sortMoney gives the amounts a stable order (largest first) so the rendered
// string and the written fact do not churn between polls.
func sortMoney(m map[string]float64) []Money {
out := make([]Money, 0, len(m))
for c, a := range m {
out = append(out, Money{Currency: c, Amount: a})
}
sort.Slice(out, func(i, j int) bool {
if out[i].Amount != out[j].Amount {
return out[i].Amount > out[j].Amount
}
return out[i].Currency < out[j].Currency
})
return out
}
// Empty reports whether the summary rests on no transactions at all. Callers
// must treat an empty summary as "nothing to say", never as a zero: the
// difference between "he spent nothing" and "the read returned nothing" is the
// difference between an answer and an invented one.
func (s Summary) Empty() bool { return s.Count == 0 }
// MonthWindow — the first instant of now's month, and now's own day-end
// exclusive bound, in now's location. The window a "сколько я потратил в этом
// месяце?" question means.
func MonthWindow(now time.Time) (from, to time.Time) {
loc := now.Location()
from = time.Date(now.Year(), now.Month(), 1, 0, 0, 0, 0, loc)
to = time.Date(now.Year(), now.Month(), now.Day(), 0, 0, 0, 0, loc).AddDate(0, 0, 1)
return from, to
}
// DayWindow — today, in now's location.
func DayWindow(now time.Time) (from, to time.Time) {
loc := now.Location()
from = time.Date(now.Year(), now.Month(), now.Day(), 0, 0, 0, 0, loc)
return from, from.AddDate(0, 0, 1)
}
+178
View File
@@ -0,0 +1,178 @@
package zenmoney
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
"time"
)
// fixtureServer replays testdata/diff.json and records the request, so the
// tests can assert the wire contract (Bearer token, POST, /v8/diff/) without a
// ZenMoney account.
func fixtureServer(t *testing.T, got *diffRequest, auth *string) *httptest.Server {
t.Helper()
body, err := os.ReadFile("testdata/diff.json")
if err != nil {
t.Fatal(err)
}
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
t.Errorf("method = %s, want POST", r.Method)
}
if r.URL.Path != "/v8/diff/" {
t.Errorf("path = %s, want /v8/diff/", r.URL.Path)
}
if auth != nil {
*auth = r.Header.Get("Authorization")
}
if got != nil {
if err := json.NewDecoder(r.Body).Decode(got); err != nil {
t.Errorf("decode request: %v", err)
}
}
w.Header().Set("Content-Type", "application/json")
w.Write(body)
}))
}
func aug(day int) time.Time { return time.Date(2026, 8, day, 0, 0, 0, 0, time.UTC) }
func TestSinceSumsSpendingPerCurrency(t *testing.T) {
var req diffRequest
var auth string
srv := fixtureServer(t, &req, &auth)
defer srv.Close()
c, err := New("tok", srv.URL, time.Second)
if err != nil {
t.Fatal(err)
}
s, err := c.Since(context.Background(), aug(1), aug(6))
if err != nil {
t.Fatal(err)
}
if auth != "Bearer tok" {
t.Errorf("Authorization = %q", auth)
}
if req.ServerTimestamp != aug(1).Unix() {
t.Errorf("serverTimestamp = %d, want the window start", req.ServerTimestamp)
}
// 1500 + 249.5 RUB spent, 12 EUR spent, 3000 RUB in. The transfer (t4), the
// deleted row (t6) and July's salary (t3) are all excluded.
want := map[string]float64{"RUB": 1749.5, "EUR": 12}
if len(s.Spent) != 2 {
t.Fatalf("spent = %+v, want two currencies", s.Spent)
}
for _, m := range s.Spent {
if want[m.Currency] != m.Amount {
t.Errorf("spent %s = %v, want %v", m.Currency, m.Amount, want[m.Currency])
}
}
if len(s.Earned) != 1 || s.Earned[0].Amount != 3000 || s.Earned[0].Currency != "RUB" {
t.Errorf("earned = %+v, want 3000 RUB (July's salary is outside the window)", s.Earned)
}
if s.Count != 4 {
t.Errorf("count = %d, want 4 counted transactions", s.Count)
}
// Largest first, so the fact value does not churn between polls.
if s.Spent[0].Currency != "RUB" {
t.Errorf("spent order = %+v, want the largest amount first", s.Spent)
}
}
// A window with nothing in it is NOT a zero. No transactions means no answer,
// and the caller must be able to tell the difference.
func TestSinceEmptyWindowIsNotAZero(t *testing.T) {
srv := fixtureServer(t, nil, nil)
defer srv.Close()
c, _ := New("tok", srv.URL, time.Second)
s, err := c.Since(context.Background(), time.Date(2026, 9, 1, 0, 0, 0, 0, time.UTC), time.Date(2026, 9, 30, 0, 0, 0, 0, time.UTC))
if err != nil {
t.Fatal(err)
}
if !s.Empty() {
t.Fatalf("summary = %+v, want empty", s)
}
if _, ok := s.Value(); ok {
t.Error("an empty summary must not produce a fact value")
}
}
func TestSinceReportsHTTPFailureWithoutTheBody(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Error(w, `{"account":"acc-card","secret":"leaky"}`, http.StatusUnauthorized)
}))
defer srv.Close()
c, _ := New("tok", srv.URL, time.Second)
_, err := c.Since(context.Background(), aug(1), aug(6))
if err == nil {
t.Fatal("want an error on 401")
}
if strings.Contains(err.Error(), "acc-card") || strings.Contains(err.Error(), "leaky") {
t.Errorf("error %q echoes the response body — it reaches the log", err)
}
}
func TestNewRequiresAToken(t *testing.T) {
if _, err := New(" ", "", 0); err == nil {
t.Error("want an error for an empty token — the capability is off unless configured")
}
}
func TestMonthAndDayWindows(t *testing.T) {
now := time.Date(2026, 8, 15, 21, 30, 0, 0, time.UTC)
from, to := MonthWindow(now)
if from != time.Date(2026, 8, 1, 0, 0, 0, 0, time.UTC) || to != time.Date(2026, 8, 16, 0, 0, 0, 0, time.UTC) {
t.Errorf("month window = %v..%v", from, to)
}
from, to = DayWindow(now)
if from != time.Date(2026, 8, 15, 0, 0, 0, 0, time.UTC) || to != time.Date(2026, 8, 16, 0, 0, 0, 0, time.UTC) {
t.Errorf("day window = %v..%v", from, to)
}
}
func TestFactValueRoundTripAndFormat(t *testing.T) {
s := Summary{Spent: []Money{{"RUB", 1749.5}}, Earned: []Money{{"RUB", 3000}}, Count: 3}
raw, ok := s.Value()
if !ok {
t.Fatal("want a fact value")
}
v, err := ParseFactValue(raw)
if err != nil {
t.Fatal(err)
}
got := v.FormatRU("в этом месяце")
if !strings.Contains(got, "1749.5 RUB") || !strings.Contains(got, "3000 RUB") {
t.Errorf("reply = %q, want the exact figures", got)
}
// Persona: informal, feminine, no commentary on his spending.
for _, bad := range []string{"вы", "ваш", "милый", "дорогой", "рад ", "слишком", "много"} {
if strings.Contains(got, bad) {
t.Errorf("reply %q contains %q", got, bad)
}
}
if strings.Contains(got, "он ") {
t.Errorf("reply %q talks about him in the third person", got)
}
}
// An empty fact value renders to nothing, so a caller cannot accidentally
// speak a zero.
func TestFormatRUEmptyRendersNothing(t *testing.T) {
if got := (FactValue{}).FormatRU("сегодня"); got != "" {
t.Errorf("reply = %q, want empty", got)
}
}
func TestFormatAmountKeepsTheTruth(t *testing.T) {
for in, want := range map[float64]string{1500: "1500", 249.5: "249.5", 0.99: "0.99", 1749.55: "1749.55"} {
if got := formatAmount(in); got != want {
t.Errorf("formatAmount(%v) = %q, want %q", in, got, want)
}
}
}
+99
View File
@@ -0,0 +1,99 @@
package zenmoney
import (
"encoding/json"
"fmt"
"strings"
"time"
)
// Fact keys the poller writes, all under source "poll:zenmoney". Two windows,
// because they are the two questions he actually asks; a per-category
// breakdown would mean storing what he bought, and the store is not a ledger.
const (
KeySpentToday = "money_today"
KeySpentMonth = "money_month"
)
// Source — the provenance every money fact carries. The loop's rules trust
// source, and nothing in Maven has a rule on these keys: they are read when he
// asks, never a reason to speak. Maven is not a nag, least of all about money.
const Source = "poll:zenmoney"
// FactValue — the JSON stored in a money fact. A wire shape of its own rather
// than the Summary struct so From/To (which carry a timezone and a clock) stay
// out of the store; the key already says which window it is.
type FactValue struct {
Spent []Money `json:"spent"`
Earned []Money `json:"earned"`
Count int `json:"count"`
}
// Value encodes the summary for the facts table. Returns ok=false for an empty
// summary: no transactions read means no fact written, so that a failed or
// empty poll can never be recited back to him as a zero.
func (s Summary) Value() (string, bool) {
if s.Empty() {
return "", false
}
b, err := json.Marshal(FactValue{Spent: s.Spent, Earned: s.Earned, Count: s.Count})
if err != nil {
return "", false
}
return string(b), true
}
// ParseFactValue decodes a stored money fact.
func ParseFactValue(raw string) (FactValue, error) {
var v FactValue
if err := json.Unmarshal([]byte(raw), &v); err != nil {
return FactValue{}, err
}
return v, nil
}
// FormatRU renders a money fact the way Maven says it — feminine, informal,
// and only about numbers that came from ZenMoney. window is the Russian phrase
// for the period ("сегодня", "в этом месяце").
//
// No commentary. She reports the figure and stops: an opinion about his
// spending is exactly the nagging Maven is not for.
func (v FactValue) FormatRU(window string) string {
if v.Count == 0 {
return ""
}
var parts []string
if len(v.Spent) > 0 {
parts = append(parts, "потратил "+joinMoney(v.Spent))
}
if len(v.Earned) > 0 {
parts = append(parts, "получил "+joinMoney(v.Earned))
}
if len(parts) == 0 {
return ""
}
return window + " ты " + strings.Join(parts, ", ") + "."
}
func joinMoney(ms []Money) string {
parts := make([]string, 0, len(ms))
for _, m := range ms {
parts = append(parts, fmt.Sprintf("%s %s", formatAmount(m.Amount), m.Currency))
}
return strings.Join(parts, " и ")
}
// formatAmount — whole units when the amount is whole, two decimals otherwise.
// Never rounded to something prettier than the truth.
func formatAmount(a float64) string {
if a == float64(int64(a)) {
return fmt.Sprintf("%d", int64(a))
}
return strings.TrimRight(strings.TrimRight(fmt.Sprintf("%.2f", a), "0"), ".")
}
// StaleAfter — how old a money fact may be and still be worth reciting. The
// poller is off unless configured and can be down; answering with last week's
// total as if it were today's would be a lie by omission, so a stale fact is
// reported as stale.
const StaleAfter = 26 * time.Hour
+35
View File
@@ -0,0 +1,35 @@
{
"serverTimestamp": 1785312000,
"instrument": [
{"id": 2, "title": "Российский рубль", "shortTitle": "RUB", "symbol": "₽", "rate": 1},
{"id": 3, "title": "Евро", "shortTitle": "EUR", "symbol": "€", "rate": 100}
],
"account": [
{"id": "acc-card", "title": "карта", "instrument": 2},
{"id": "acc-cash", "title": "наличные", "instrument": 2},
{"id": "acc-eur", "title": "евро", "instrument": 3}
],
"transaction": [
{"id": "t1", "date": "2026-08-01", "changed": 1785300000, "income": 0, "outcome": 1500,
"incomeInstrument": 2, "outcomeInstrument": 2, "incomeAccount": "acc-card", "outcomeAccount": "acc-card",
"payee": "пятёрочка", "deleted": false},
{"id": "t2", "date": "2026-08-01", "changed": 1785300001, "income": 0, "outcome": 249.5,
"incomeInstrument": 2, "outcomeInstrument": 2, "incomeAccount": "acc-card", "outcomeAccount": "acc-card",
"payee": "метро", "deleted": false},
{"id": "t3", "date": "2026-07-20", "changed": 1785300002, "income": 120000, "outcome": 0,
"incomeInstrument": 2, "outcomeInstrument": 2, "incomeAccount": "acc-card", "outcomeAccount": "acc-card",
"payee": "зарплата", "deleted": false},
{"id": "t4", "date": "2026-08-02", "changed": 1785300003, "income": 5000, "outcome": 5000,
"incomeInstrument": 2, "outcomeInstrument": 2, "incomeAccount": "acc-cash", "outcomeAccount": "acc-card",
"payee": "", "comment": "снял наличные", "deleted": false},
{"id": "t5", "date": "2026-08-03", "changed": 1785300004, "income": 0, "outcome": 12,
"incomeInstrument": 3, "outcomeInstrument": 3, "incomeAccount": "acc-eur", "outcomeAccount": "acc-eur",
"payee": "hosting", "deleted": false},
{"id": "t6", "date": "2026-08-04", "changed": 1785300005, "income": 0, "outcome": 999,
"incomeInstrument": 2, "outcomeInstrument": 2, "incomeAccount": "acc-card", "outcomeAccount": "acc-card",
"payee": "удалённая", "deleted": true},
{"id": "t7", "date": "2026-08-05", "changed": 1785300006, "income": 3000, "outcome": 0,
"incomeInstrument": 2, "outcomeInstrument": 2, "incomeAccount": "acc-card", "outcomeAccount": "acc-card",
"payee": "возврат", "deleted": false}
]
}