Turn a mail into task candidates, and into nothing else (#246)

The extraction half. internal/email.Extractor asks the resident Qwen3-1.7B,
under a GBNF grammar, what one message requires of him, and returns at most
three short candidates with an optional date.

Everything it can produce is a row in `tasks` with status "candidate",
written through the intake seam #130 built for exactly this (Source
"email:<mailbox>", Evidence = the subject line). No reminder, no fact, no
note, no calendar event. That bound is the design: a reminder FIRES, so a
1.7B misreading "встреча была в четверг" as a future appointment would wake
him up about it, whereas a wrong candidate is a line he dismisses in one
click. A due date the model read out of the mail is stored on the candidate,
where no scheduler reads it — the review page sorts by it. Relative wording
("до пятницы") is deliberately left in the text rather than resolved to a
date the model would get wrong.

The prompt is written against the two things a small model does here: it
summarises when asked to extract, and it invents an obligation out of a
polite closing line. Hence the demand for a verb phrase, and an explicit
empty array — most mail contains no task, and a model with no way to say
"nothing" says something.

Wiring: core owns extraction because llama-server lives in core's process,
so the reader hands messages over a new ipc.MethodIngestMail. It is a Server
hook (like StepUp/UnlockFn), not a CoreAPI method — not a store operation,
and no CoreAPI implementation should have to carry it. The hook stays nil
without an `email` config block or without a llama-server phraser, so the
method answers ErrUnknownMethod: off unless configured, twice over. There is
no keyword fallback on purpose — "the subject became a task" is a mailbox
rendered as a to-do list, not extraction.

Privacy: junk is refused before the model is called, mail text is never
search input, extraction errors carry byte counts rather than the reply, the
stored evidence is a truncated subject, and the log line names the mailbox
and the UID only.

internal/email/extract.go

@@ -0,0 +1,226 @@
+package email
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"strings"
+	"time"
+
+	"github.com/kami/maven/internal/llm"
+	"github.com/kami/maven/internal/persona"
+)
+
+// Extraction — turning one mail into task CANDIDATES, and nothing else.
+//
+// The output of this file can only ever become rows in `tasks` with status
+// "candidate" (store.TaskCandidate), written through the one intake seam
+// (ipc.CaptureTaskReq, Vikunja #130). That bound is the whole design:
+//
+//   - No reminder. A reminder FIRES; it speaks to him unprompted. A 1.7B that
+//     misreads "встреча была в четверг" as a future appointment would then wake
+//     him up about it. A candidate that is wrong is a line on a review page he
+//     dismisses in one click, which is the correct cost of a model being wrong
+//     about someone's mail.
+//   - No fact. A fact is a claim Maven will later recite as true. Nothing read
+//     out of a marketing mail deserves that standing.
+//   - No calendar event, no note, no action. Extraction writes candidates or
+//     writes nothing.
+//
+// The due date the model may return is stored on the candidate (tasks.due_ts),
+// which no scheduler reads — it is there so the review page can sort by it.
+//
+// Privacy: the mail text goes to the resident model on this box and nowhere
+// else. It is never search input (CLAUDE.md: "his notes and facts are never
+// search input" — mail is the same class), and Evidence keeps only the subject
+// line, so the review page shows him where a candidate came from without the
+// store growing a copy of his mailbox.
+
+// MaxCandidates — at most this many candidates per message, enforced by the
+// grammar. A mail with four tasks in it is a mail he has to read himself; a
+// model allowed ten will produce ten.
+const MaxCandidates = 3
+
+// SourcePrefix — provenance for everything this package captures. The mailbox
+// name is appended: "email:INBOX". Same vocabulary as tap:voice / poll:netdata.
+const SourcePrefix = "email:"
+
+// Candidate — one piece of work the model thinks the mail is asking for.
+type Candidate struct {
+	Text string `json:"text"`
+	// Due — "YYYY-MM-DD" or empty. A date the model read out of the text, not a
+	// date it computed: relative wording ("до пятницы") is left in Text, because
+	// a small model resolving "пятница" against today's date gets it wrong often
+	// enough that a stored wrong date is worse than no date.
+	Due string `json:"due"`
+}
+
+// Completer — the llama-server seam, same shape memeval and the router use, so
+// the one resident model serves this caller too.
+type Completer interface {
+	Complete(ctx context.Context, r llm.Req) (string, error)
+}
+
+// Extractor reads a message and returns candidates. It holds no store and no
+// writer on purpose: this type cannot persist anything, so "extraction never
+// acts" is a property of the code, not of a review.
+type Extractor struct {
+	llm Completer
+	// MaxCandidates — 0 ⇒ MaxCandidates.
+	max int
+	// ContextBlock — the shared persona block, optional. Extraction output is
+	// not spoken, so the persona matters less here than in the phraser; it is
+	// wired anyway so a candidate reads in her voice on the review page.
+	contextBlock func() string
+}
+
+func NewExtractor(c Completer, max int, contextBlock func() string) *Extractor {
+	if max <= 0 || max > MaxCandidates {
+		max = MaxCandidates
+	}
+	return &Extractor{llm: c, max: max, contextBlock: contextBlock}
+}
+
+// extractGrammar — GBNF pinning the answer to a bounded array of fixed-shape
+// candidates. Same reasoning as memeval's evalGrammar and the router's
+// routeGrammar: the shape and the length bound are what keep a small model from
+// drifting into prose or spending the token budget repeating one field.
+//
+// The empty array is reachable, deliberately: most mail contains no task, and a
+// model with no way to say "nothing" invents something.
+const extractGrammar = `
+root  ::= "[" ws (item ("," ws item){0,2})? ws "]"
+item  ::= "{" ws "\"text\"" ws ":" ws text "," ws "\"due\"" ws ":" ws due ws "}"
+text  ::= "\"" ([^"\\] | "\\" .){1,120} "\""
+due   ::= "\"\"" | "\"" [0-9]{4} "-" [0-9]{2} "-" [0-9]{2} "\""
+ws    ::= [ \t\n]*
+`
+
+// extractSystem — the extraction prompt.
+//
+// Written around the two failure modes a small model has on this task: it
+// summarises when asked to extract (turning a mail into "письмо от Антона"),
+// and it invents an obligation from any polite closing sentence. Hence the
+// insistence on a verb phrase, and the explicit permission to return [].
+const extractSystem = `Ты читаешь одно письмо из его почты и достаёшь из него дела, которые письмо от него требует.
+
+Правила:
+- Отвечай ТОЛЬКО массивом JSON. Каждый элемент: {"text": "...", "due": "ГГГГ-ММ-ДД" или ""}.
+- text — короткая формулировка дела по-русски, с глаголом: "оплатить счёт за интернет", "отправить акт". Не пересказывай письмо и не описывай его.
+- Дело — это то, что должен сделать ОН. Рассылка, реклама, уведомление, отчёт, письмо «просто к сведению» — дел не содержат.
+- Если письмо ничего от него не требует, верни пустой массив []. Это нормальный ответ, так бывает чаще всего.
+- Ничего не придумывай. Если срока в письме нет — "".
+- due заполняй только когда в письме стоит конкретная дата. Слова вроде «до пятницы» оставь в text, дату не вычисляй.
+- Максимум три дела. Лучше одно точное, чем три общих.`
+
+// Extract returns the candidates in one message.
+//
+// Junk is refused without an LLM call — cheapest possible defence, and the
+// reason the header filter exists. An empty message (no subject, no body) is
+// likewise not worth a round trip.
+//
+// A parse failure is an error the caller logs and moves past. It is never
+// silently turned into zero candidates, because "the model went off the rails"
+// and "the mail contains no task" want different reactions from a human reading
+// the log.
+func (e *Extractor) Extract(ctx context.Context, msg Message) ([]Candidate, error) {
+	if msg.Junk {
+		return nil, nil
+	}
+	user := renderForModel(msg)
+	if user == "" {
+		return nil, nil
+	}
+	raw, err := e.llm.Complete(ctx, llm.Req{
+		System:        persona.Prepend(e.contextBlock, extractSystem),
+		User:          user,
+		Grammar:       extractGrammar,
+		MaxTokens:     512,
+		RepeatPenalty: 1.1,
+	})
+	if err != nil {
+		return nil, fmt.Errorf("email: extract: %w", err)
+	}
+	items, err := parseCandidates(raw)
+	if err != nil {
+		// The raw reply is NOT in the error: it is a transformation of his mail,
+		// and this error reaches the daemon log.
+		return nil, fmt.Errorf("email: extract: unparsable reply (%d bytes)", len(raw))
+	}
+	out := make([]Candidate, 0, len(items))
+	seen := map[string]bool{}
+	for _, it := range items {
+		it.Text = strings.TrimSpace(it.Text)
+		if it.Text == "" {
+			continue
+		}
+		key := strings.ToLower(strings.Join(strings.Fields(it.Text), " "))
+		if seen[key] {
+			continue // the model repeating itself is not two tasks
+		}
+		seen[key] = true
+		if _, ok := ParseDue(it.Due); !ok {
+			it.Due = "" // a date the grammar allowed but the calendar does not
+		}
+		out = append(out, it)
+		if len(out) >= e.max {
+			break
+		}
+	}
+	return out, nil
+}
+
+// renderForModel is the user turn: subject, sender and body, labelled. Only
+// these three fields — no headers, no recipient list, no message-id, nothing
+// that would let the model start reasoning about routing metadata.
+func renderForModel(msg Message) string {
+	var b strings.Builder
+	if msg.From != "" {
+		fmt.Fprintf(&b, "От: %s\n", msg.From)
+	}
+	if msg.Subject != "" {
+		fmt.Fprintf(&b, "Тема: %s\n", msg.Subject)
+	}
+	if msg.Body != "" {
+		fmt.Fprintf(&b, "\n%s\n", msg.Body)
+	}
+	if msg.Subject == "" && msg.Body == "" {
+		return ""
+	}
+	return b.String()
+}
+
+// parseCandidates decodes the grammar-constrained reply, tolerating the
+// wrappers a Thinking model sometimes leaves around it (a fenced block, or
+// leading reasoning before the array).
+func parseCandidates(raw string) ([]Candidate, error) {
+	s := strings.TrimSpace(raw)
+	if i := strings.Index(s, "["); i > 0 {
+		s = s[i:]
+	}
+	if j := strings.LastIndex(s, "]"); j >= 0 {
+		s = s[:j+1]
+	}
+	var out []Candidate
+	if err := json.Unmarshal([]byte(s), &out); err != nil {
+		return nil, err
+	}
+	return out, nil
+}
+
+// ParseDue turns the model's "YYYY-MM-DD" into a time in UTC. Exported because
+// the daemon-side intake stores it on the candidate.
+//
+// The zero-value/empty case returns ok=false rather than an error: no date is
+// the common answer, not a failure.
+func ParseDue(s string) (time.Time, bool) {
+	s = strings.TrimSpace(s)
+	if s == "" {
+		return time.Time{}, false
+	}
+	t, err := time.Parse("2006-01-02", s)
+	if err != nil {
+		return time.Time{}, false
+	}
+	return t, true
+}

internal/email/extract_test.go

@@ -0,0 +1,143 @@
+package email
+
+import (
+	"context"
+	"strings"
+	"testing"
+
+	"github.com/kami/maven/internal/llm"
+)
+
+// fakeLLM returns a canned reply and records the request, so a test can assert
+// on the grammar and on what of the mail was sent.
+type fakeLLM struct {
+	reply string
+	err   error
+	got   llm.Req
+	calls int
+}
+
+func (f *fakeLLM) Complete(_ context.Context, r llm.Req) (string, error) {
+	f.calls++
+	f.got = r
+	return f.reply, f.err
+}
+
+func msgFor(subject, body string) Message {
+	return Message{UID: 1, From: "anton@example.org", Subject: subject, Body: body}
+}
+
+func TestExtractCandidates(t *testing.T) {
+	f := &fakeLLM{reply: `[{"text":"отправить акт","due":""},{"text":"оплатить счёт","due":"2026-08-05"}]`}
+	e := NewExtractor(f, 0, nil)
+	got, err := e.Extract(context.Background(), msgFor("Акт и счёт", "Надо отправить акт и оплатить счёт до 5 августа."))
+	if err != nil {
+		t.Fatalf("extract: %v", err)
+	}
+	if len(got) != 2 {
+		t.Fatalf("got %d candidates, want 2: %+v", len(got), got)
+	}
+	if got[0].Text != "отправить акт" || got[1].Due != "2026-08-05" {
+		t.Errorf("candidates = %+v", got)
+	}
+	if f.got.Grammar == "" {
+		t.Error("extraction must be grammar-constrained")
+	}
+	// The subject and body go to the model; nothing else about the message does.
+	if !strings.Contains(f.got.User, "Акт и счёт") || !strings.Contains(f.got.User, "оплатить счёт") {
+		t.Errorf("user turn = %q", f.got.User)
+	}
+}
+
+func TestExtractEmptyArrayIsNotAnError(t *testing.T) {
+	f := &fakeLLM{reply: "[]"}
+	got, err := NewExtractor(f, 0, nil).Extract(context.Background(), msgFor("FYI", "Просто к сведению."))
+	if err != nil || len(got) != 0 {
+		t.Fatalf("got (%v, %v), want (empty, nil) — no task is the normal answer", got, err)
+	}
+}
+
+// Junk must never reach the model: the header filter exists so the resident
+// model is not spent on newsletters.
+func TestExtractSkipsJunkWithoutCallingModel(t *testing.T) {
+	f := &fakeLLM{reply: `[{"text":"купить всё со скидкой","due":""}]`}
+	msg := msgFor("Скидки", "Sale!")
+	msg.Junk = true
+	got, err := NewExtractor(f, 0, nil).Extract(context.Background(), msg)
+	if err != nil || got != nil {
+		t.Fatalf("got (%v, %v), want (nil, nil)", got, err)
+	}
+	if f.calls != 0 {
+		t.Errorf("model called %d times for junk, want 0", f.calls)
+	}
+}
+
+func TestExtractEmptyMessageIsNotSent(t *testing.T) {
+	f := &fakeLLM{reply: "[]"}
+	if _, err := NewExtractor(f, 0, nil).Extract(context.Background(), Message{UID: 3}); err != nil {
+		t.Fatalf("extract: %v", err)
+	}
+	if f.calls != 0 {
+		t.Errorf("model called %d times for an empty message, want 0", f.calls)
+	}
+}
+
+func TestExtractCaps(t *testing.T) {
+	f := &fakeLLM{reply: `[{"text":"a","due":""},{"text":"b","due":""},{"text":"c","due":""}]`}
+	got, err := NewExtractor(f, 2, nil).Extract(context.Background(), msgFor("s", "b"))
+	if err != nil {
+		t.Fatalf("extract: %v", err)
+	}
+	if len(got) != 2 {
+		t.Errorf("got %d, want the configured cap of 2", len(got))
+	}
+}
+
+func TestExtractDropsRepeatsAndBadDates(t *testing.T) {
+	f := &fakeLLM{reply: `[{"text":"Отправить акт","due":"2026-02-31"},{"text":"отправить  акт","due":""},{"text":"  ","due":""}]`}
+	got, err := NewExtractor(f, 0, nil).Extract(context.Background(), msgFor("s", "b"))
+	if err != nil {
+		t.Fatalf("extract: %v", err)
+	}
+	if len(got) != 1 {
+		t.Fatalf("got %d candidates, want 1 (repeat and blank dropped): %+v", len(got), got)
+	}
+	if got[0].Due != "" {
+		t.Errorf("due = %q, want empty — 2026-02-31 is not a date", got[0].Due)
+	}
+}
+
+// A Thinking model sometimes wraps the array; and when it emits something
+// unparsable the caller must hear about it rather than see "no tasks".
+func TestParseCandidatesTolerance(t *testing.T) {
+	got, err := parseCandidates("думаю... [{\"text\":\"x\",\"due\":\"\"}] всё")
+	if err != nil || len(got) != 1 || got[0].Text != "x" {
+		t.Fatalf("got (%+v, %v)", got, err)
+	}
+	if _, err := parseCandidates("нет никакого JSON"); err == nil {
+		t.Error("unparsable output must be an error")
+	}
+}
+
+func TestExtractParseErrorHidesMailText(t *testing.T) {
+	f := &fakeLLM{reply: "он просил отправить акт, вот такой ответ"}
+	_, err := NewExtractor(f, 0, nil).Extract(context.Background(), msgFor("Акт", "секретный текст"))
+	if err == nil {
+		t.Fatal("want an error")
+	}
+	if strings.Contains(err.Error(), "акт") || strings.Contains(err.Error(), "секретный") {
+		t.Errorf("error text leaks mail content: %v", err)
+	}
+}
+
+func TestParseDue(t *testing.T) {
+	if _, ok := ParseDue(""); ok {
+		t.Error("empty due must be (zero, false)")
+	}
+	if got, ok := ParseDue("2026-08-05"); !ok || got.Year() != 2026 || got.Month() != 8 || got.Day() != 5 {
+		t.Errorf("ParseDue = (%v, %v)", got, ok)
+	}
+	if _, ok := ParseDue("05.08.2026"); ok {
+		t.Error("a non-ISO date must not parse")
+	}
+}