Compare commits

...

14 Commits

Author SHA1 Message Date
claude 569991bb15 pattern: a burst of taps is not a routine (V-468)
Detect had no floor on the interval. Four events minutes apart give gaps
near 0.002 days, every one of them inside the ±50% band, so it proposed a
routine and PhraseRoutine called it "каждый день".

UNIQUE(action, object) makes that unrecoverable: dismissing the bogus
proposal burns the pair, and the real routine behind it can never be
proposed again. It also made hand-QA unsafe — seeding a pattern with four
chat turns poisoned the pair being tested.

The floor is two hours against the median, not a day, because meals, water
and breaks are genuine several-times-a-day habits.
2026-08-04 03:00:11 +04:00
claude c915115096 eval: a verb governed by "ты" is his, not her drift (V-462)
CheckFeminine flagged "ты заплатил за домен" as a masculine self-reference.
The second pass reads a masculine past-tense verb before "тебе", "тебя" or
"за" as her speaking with the pronoun dropped, and it checked neither the
subject nor what "за" pointed at. He is male, so a verb governed by "ты"
must be masculine, and "за домен" is a price rather than a favour.

The talk fixture was under-reporting by a point whenever a reply addressed
him in the past tense, which is common.
2026-08-04 02:58:19 +04:00
claude 908d92a7e8 calendar: a Russian summary keeps its letters in the fact key (V-443)
safeKey kept ASCII only, so "Встреча с Аней" and "Обед с мамой" both
reduced to "--" and shared one key on one day. The second event of the
day overwrote the first, silently, and his calendar is Russian.

Letters and digits in any script now pass. Migration #18 deletes the rows
written under the old rule instead of rewriting them: a calendar fact is
derived, the next poll writes the day again, and a stale row reads as an
extra meeting.
2026-08-04 02:56:09 +04:00
claude 43f2c37538 router: stage 0 claims the other days and the named event (V-471)
"какие планы на сегодня" worked and "какие планы на завтра" answered
"пока не умею": the agenda rule needs "у меня" or a calendar noun, and
that phrasing carries neither. "когда планёрка?" had the same shape.

Two rules. One takes a plan noun aimed at a named day, one takes a closed
list of event nouns after "когда"/"во сколько". Both route intent only,
so the query chain still decides which source answers.

classifier+onnx over the fixture: 55/79, 69.6% full, with the two new
cases passing and no case moving the other way.
2026-08-04 02:52:51 +04:00
claude 6d3f5b5b01 router: a reminder with no subject asks instead of guessing (V-383)
Slots.Text was the raw utterance for every intent, so a reminder could not
have an empty subject. StillMissing never reported SlotText, the question
"О чём напомнить?" was unaskable, and the branch in PendingQuestion.Answer
that fills a text slot could only overwrite the whole request.

The LLM path now keeps the model's own text, empty included, and the gate
turns a subjectless reminder into a question. The classifier path is
unchanged: it has no subject parser, so the utterance is the only signal it
has.
2026-08-04 02:49:02 +04:00
claude eda1112f3b mavgpud: a yield stops writing a core and reads as a yield (V-491)
llama-server aborts inside its own static teardown on SIGTERM — the
handler calls exit(), stream_session_manager's destructor throws, and the
process dies "signal: aborted (core dumped)". mavgpud sends that signal on
every eviction, so a routine yield wrote a multi-gigabyte core into
systemd-coredump and logged the same line a real crash would.

LimitCORE=0 in the unit stops the disk cost. A yielding flag, set by stop
and cleared by start, makes the log distinguish the two: only an exit we
did not ask for is still reported as an exit.

Not filed upstream. Searched ggml-org/llama.cpp for
"ggml_uncaught_exception" with SIGTERM and for stream_session_manager and
found nothing matching, so the issue still wants writing — by someone with
an account on that tracker, which is why it is not in this commit.
2026-08-04 02:01:17 +04:00
kami 71041029e2 Merge pull request 'The reply path can't be tested — llmReplier is stuck in package main' (#107) from task/396-the-reply-path-can-t-be-tested-llmreplie into master
Reviewed-on: #107
2026-08-03 22:35:56 +02:00
claude 35018226ef eval: score the reply path, the fourth phrasing path (V-396)
Nine reply cases and a fourth column in the talk report. The reply path is a
separate object from the phraser in the daemon, so Pair joins a Talker and a
Confirmer for a run that covers everything Maven says.

Cases carry intent/key/value because the replier is phrased from the decision the
router resolved, not from the raw utterance. Three of them are baits the other
paths cannot produce: a masculine verb about himself that she must not copy onto
herself, a polite plural input that must still come back на ты, and an unresolved
note that invites a question a confirmation is not allowed to ask.

Not scored against a model here — this box has no llama-server, and the baseline
test is opt-in on MAVEN_LLM_URL.
2026-08-04 00:33:41 +04:00
claude 8833a9c76b mavend: keep only the stub floor in llmReplier (V-396)
The prompt, the call and the output parsing now live in internal/phraser. What is
left here is the one thing the daemon adds: a clarify, a model error and an
unusable generation all answer from voice.StubReplier, so a turn never breaks on
the model. The duplicated stripThink and parseResponseMood copies are gone;
capture.go uses phraser.StripThink.
2026-08-04 00:33:30 +04:00
claude 6c07409452 phraser: add Replier, the reply path lifted out of package main (V-396)
llmReplier lived in cmd/mavend, so the confirmation he hears after every fact,
note and reminder was the one phrasing path nothing could import or score.

Replier owns the prompt, the call and the parsing, and returns its errors instead
of hiding them — a dead model shows up as an error rather than as bad phrasing.
It has no stub fallback of its own; the daemon keeps that. StripThink is exported
for the daemon's own model callers.
2026-08-04 00:33:30 +04:00
kami 1c2541f7d6 Merge pull request 'llama-server holds 7.9GB RSS for a 1.1GB model, and its startup log goes nowhere' (#105) from task/496-recall-a-cross-language-question-loses-i into master
Reviewed-on: #105
2026-08-03 22:04:29 +02:00
claude 9e25f18a3e memory: record the recall topic veto's real price (V-496)
#496 asked to skip the veto when the question and the hit are in
different scripts, so an English question stops losing a Russian note.
Measured first: the fixture has no cross-language case, and en-hard-024
is an English question against an English note. Both proposed fixes are
no-ops.

What the veto actually does on the fixture, with the real embedder: it
costs en-hard-024 and buys ru-silent-029. Pass count is 22/32 either
way; false recall is 0/5 with it and 1/5 without. The two cases are one
lexical class, so no rule cheap enough for RecallAllowed separates them.

Accepts the loss and pins both sides in a test.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 00:01:49 +04:00
kami 197897516e Merge pull request 'Task/495 bug x escapes the personal boundary and' (#104) from task/495-bug-x-escapes-the-personal-boundary-and into master
Reviewed-on: #104
2026-08-03 21:30:56 +02:00
kami 6e5364a0ed Merge pull request 'Bug: "что я говорил про X" escapes the personal boundary and reaches web search' (#102) from task/495-bug-x-escapes-the-personal-boundary-and into master
Reviewed-on: #102
2026-08-03 20:59:34 +02:00
29 changed files with 890 additions and 170 deletions
+1 -1
View File
@@ -100,7 +100,7 @@ func (l llmCompleter) Complete(ctx context.Context, system, user string) (string
// grammar, or a llama-server too old to honour one, gets the plain text it used
// to get rather than an empty meeting summary.
func unwrapSummary(raw string) string {
s := stripThink(strings.TrimSpace(raw))
s := phraser.StripThink(strings.TrimSpace(raw))
start := strings.Index(s, "{")
end := strings.LastIndex(s, "}")
if start < 0 || end <= start {
+36
View File
@@ -465,3 +465,39 @@ func TestExpiryNoticeSurvivesAConfirmTurn(t *testing.T) {
t.Fatal("the expired question must be gone")
}
}
// The other half of the subject question: his answer must fill the empty slot,
// not replace the request. Slots.Text used to be the whole raw utterance for
// every intent, so the branch that fills a text slot could only ever overwrite
// (Vikunja #383). Here the parked request holds the hour and the answer holds
// what to say at it, and the reminder that lands has both.
func TestClarifySubjectAnswerFillsRatherThanClobbers(t *testing.T) {
ctx := context.Background()
h, st, _ := newClarifyHandler(t)
at := h.now().Add(2 * time.Hour)
question, asked := h.askClarify(clarifyDec(router.IntentReminder,
router.Slots{Time: at, HasTime: true}, "напомни в 11"))
if !asked || question != "О чём напомнить?" {
t.Fatalf("expected the subject question, got %q asked=%v", question, asked)
}
reply, handled := h.resolveClarifyAnswer(ctx, "позвонить маме")
if !handled {
t.Fatal("the answer to an open question must be consumed as an answer")
}
if reply == clarifyGaveUp {
t.Fatalf("a good answer must not drop the request: %q", reply)
}
reminders, err := st.DueReminders(ctx, h.now().Add(48*time.Hour))
if err != nil || len(reminders) != 1 {
t.Fatalf("clarified reminder was not created: reminders=%v err=%v", reminders, err)
}
if !strings.Contains(reminders[0].Payload, "маме") {
t.Fatalf("the answer never reached the reminder: %q", reminders[0].Payload)
}
if !strings.Contains(reminders[0].Payload, "11") {
t.Fatalf("the answer clobbered the original request: %q", reminders[0].Payload)
}
}
+11 -100
View File
@@ -2,122 +2,33 @@ package main
import (
"context"
"encoding/json"
"strings"
"time"
"github.com/kami/maven/internal/llm"
"github.com/kami/maven/internal/persona"
"github.com/kami/maven/internal/phraser"
"github.com/kami/maven/internal/router"
"github.com/kami/maven/internal/voice"
)
// completer is the LLM seam for the replier (subset of router.Completer).
// *llm.Client satisfies it.
type completer interface {
Complete(ctx context.Context, r llm.Req) (string, error)
}
// llmReplier phrases reactive confirmations with the resident model
// (Qwen3-1.7B). Stub is the
// floor on any error (offline-safe). Maven speaks as "she", feminine RU.
// llmReplier is the daemon-side wiring around phraser.Replier: it owns the
// deterministic floor, and nothing else. The phrasing itself, the prompt and the
// output parsing live in internal/phraser so the eval can score them (#396).
type llmReplier struct {
c completer
p *phraser.Replier
stub *voice.StubReplier
// block renders the shared context block per turn (who he is, the time).
// nil ⇒ the prompt stands alone.
block func() string
}
func newLLMReplier(c completer, block func() string) *llmReplier {
return &llmReplier{c: c, stub: voice.NewStubReplier(), block: block}
func newLLMReplier(c phraser.Completer, block func() string) *llmReplier {
return &llmReplier{p: phraser.NewReplier(c, block), stub: voice.NewStubReplier()}
}
const replySystem = `Ты — Maven, домашняя ассистентка (о себе — в женском роде). Владелец — мужчина, говоришь с ним на "ты", в единственном числе; никогда не "вы"/"ваш" и не "он"/"его". Подтверди действие РОВНО ОДНИМ коротким предложением (≤120 символов), по-русски, спокойно и без официальных формулировок. Не задавай вопросов, не повторяй слова, не добавляй ничего после точки. Отвечай ТОЛЬКО одним объектом JSON с полями "response" (текст) и "mood" (ровно одно из: neutral, happy, thinking, tired, confused).
Пример: {"response": "Записала, что ты выпил стакан воды.", "mood": "neutral"}
Никогда не пиши "..." в поле response.`
// Reply never fails: a clarify, a model error and an unusable generation all
// answer from the stub, which is what keeps a turn from breaking on the model.
func (r *llmReplier) Reply(d router.Decision) string {
if d.Clarify {
return r.stub.Reply(d)
}
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
out, err := r.c.Complete(ctx, llm.Req{
System: persona.Prepend(r.block, replySystem),
User: replyContext(d),
Grammar: phraser.ResponseGrammar,
MaxTokens: 512,
RepeatPenalty: 1.3,
})
if err != nil {
out, err := r.p.PhraseReply(context.Background(), d)
if err != nil || out == "" {
return r.stub.Reply(d)
}
out = stripThink(out)
if response, _ := parseResponseMood(out); response != "" {
return response
}
// fallback: try plain-text parsing
if out = firstSentence(out); out != "" {
return out
}
return r.stub.Reply(d)
}
// firstSentence trims the model's output to a single clean confirmation: first
// line, first sentence, whitespace-normalized — the last-line defense against a
// small model that rambles past the first period despite the prompt + stop.
// stripThink removes the <think> block that Thinking-variant models emit.
func stripThink(s string) string {
if i := strings.LastIndex(s, "</think>"); i >= 0 {
s = strings.TrimSpace(s[i+8:])
}
return s
}
func firstSentence(s string) string {
s = strings.TrimSpace(s)
if i := strings.IndexByte(s, '\n'); i >= 0 {
s = s[:i]
}
// keep up to and including the first sentence-ending punctuation.
if i := strings.IndexAny(s, ".!?"); i >= 0 {
s = s[:i+1]
}
return strings.TrimSpace(s)
}
// parseResponseMood extracts {"response","mood"} from LLM output, tolerant
// of thinking tokens and extra text before/after the JSON block.
func parseResponseMood(raw string) (response, mood string) {
cleaned := strings.TrimSpace(raw)
start := strings.Index(cleaned, "{")
end := strings.LastIndex(cleaned, "}")
if start < 0 || end < 0 || end <= start {
return "", ""
}
var parsed struct {
Response string `json:"response"`
Mood string `json:"mood"`
}
if err := json.Unmarshal([]byte(cleaned[start:end+1]), &parsed); err != nil {
return "", ""
}
return parsed.Response, parsed.Mood
}
// replyContext renders the decision into a compact RU description for the model.
func replyContext(d router.Decision) string {
switch d.Intent {
case router.IntentFact:
return "записала факт: " + d.Slots.Key + " " + d.Slots.Value
case router.IntentNote:
return "сохранила заметку: " + d.Slots.Text
case router.IntentReminder:
return "поставила напоминание: " + d.Slots.Text
default:
return string(d.Intent) + ": " + d.Slots.Text
}
return out
}
+20 -50
View File
@@ -5,28 +5,22 @@ import (
"testing"
"github.com/kami/maven/internal/llm"
"github.com/kami/maven/internal/phraser"
"github.com/kami/maven/internal/router"
"github.com/kami/maven/internal/voice"
)
type mockCompleter struct {
// The phrasing itself is tested in internal/phraser. What is left here is the
// only thing the daemon adds: the stub floor, on the three ways a reply can
// fail to arrive.
type stubCompleter struct {
out string
err error
}
func (m mockCompleter) Complete(_ context.Context, _ llm.Req) (string, error) { return m.out, m.err }
func (s stubCompleter) Complete(_ context.Context, _ llm.Req) (string, error) { return s.out, s.err }
func TestLLMReplierReturnsLLMReply(t *testing.T) {
r := newLLMReplier(mockCompleter{out: `{"response":"записала, кофе закончился","mood":"neutral"}`}, nil)
got := r.Reply(router.Decision{Intent: router.IntentNote, Slots: router.Slots{Text: "кофе закончился"}})
if got != "записала, кофе закончился" {
t.Errorf("got %q, want %q", got, "записала, кофе закончился")
}
}
func TestLLMReplierFallsBackToPlainText(t *testing.T) {
r := newLLMReplier(mockCompleter{out: "записала, кофе закончился"}, nil)
func TestLLMReplierPassesTheModelReplyThrough(t *testing.T) {
r := newLLMReplier(stubCompleter{out: `{"response":"записала, кофе закончился","mood":"neutral"}`}, nil)
got := r.Reply(router.Decision{Intent: router.IntentNote, Slots: router.Slots{Text: "кофе закончился"}})
if got != "записала, кофе закончился" {
t.Errorf("got %q, want %q", got, "записала, кофе закончился")
@@ -34,54 +28,30 @@ func TestLLMReplierFallsBackToPlainText(t *testing.T) {
}
func TestLLMReplierFallsBackToStubOnError(t *testing.T) {
r := newLLMReplier(mockCompleter{err: errTestLLMDown}, nil)
noteDec := router.Decision{Intent: router.IntentNote}
got := r.Reply(noteDec)
want := voice.NewStubReplier().Reply(noteDec)
if got != want {
t.Errorf("on llm error: got %q, want stub %q", got, want)
}
r := newLLMReplier(stubCompleter{err: errReplierTest}, nil)
assertStub(t, r, router.Decision{Intent: router.IntentNote}, "llm error")
}
func TestLLMReplierFallsBackToStubOnEmpty(t *testing.T) {
r := newLLMReplier(mockCompleter{out: ""}, nil)
noteDec := router.Decision{Intent: router.IntentNote}
got := r.Reply(noteDec)
want := voice.NewStubReplier().Reply(noteDec)
if got != want {
t.Errorf("on empty llm: got %q, want stub %q", got, want)
}
r := newLLMReplier(stubCompleter{out: ""}, nil)
assertStub(t, r, router.Decision{Intent: router.IntentNote}, "empty llm")
}
func TestLLMReplierClarifyUsesStub(t *testing.T) {
r := newLLMReplier(mockCompleter{out: "я всё поняла"}, nil)
clarifyDec := router.Decision{Clarify: true}
got := r.Reply(clarifyDec)
want := voice.NewStubReplier().Reply(clarifyDec)
r := newLLMReplier(stubCompleter{out: "я всё поняла"}, nil)
assertStub(t, r, router.Decision{Clarify: true}, "clarify")
}
func assertStub(t *testing.T, r *llmReplier, d router.Decision, what string) {
t.Helper()
got, want := r.Reply(d), voice.NewStubReplier().Reply(d)
if got != want {
t.Errorf("on clarify: got %q, want stub %q", got, want)
t.Errorf("on %s: got %q, want stub %q", what, got, want)
}
}
var errTestLLMDown = errTest("llm down")
var errReplierTest = errTest("llm down")
type errTest string
func (e errTest) Error() string { return string(e) }
// grammarRecorder captures the request so the grammar can be asserted on.
type grammarRecorder struct{ req llm.Req }
func (g *grammarRecorder) Complete(_ context.Context, r llm.Req) (string, error) {
g.req = r
return `{"response":"записала","mood":"neutral"}`, nil
}
func TestLLMReplierCarriesTheResponseGrammar(t *testing.T) {
rec := &grammarRecorder{}
r := newLLMReplier(rec, nil)
r.Reply(router.Decision{Intent: router.IntentNote, Slots: router.Slots{Text: "кофе закончился"}})
if rec.req.Grammar != phraser.ResponseGrammar {
t.Errorf("grammar = %q, want phraser.ResponseGrammar", rec.req.Grammar)
}
}
+17 -3
View File
@@ -24,7 +24,12 @@ type runner struct {
mu sync.Mutex
cmd *exec.Cmd
ready bool
http *http.Client
// yielding — stop() has sent the signal and the exit that follows is ours.
// llama-server aborts on SIGTERM (its static teardown throws, upstream
// ggml-org/llama.cpp), so a routine yield and a real crash produce the same
// "signal: aborted" and used to log identically (Vikunja #491).
yielding bool
http *http.Client
}
func newRunner(bin string, args []string, readyURL string) *runner {
@@ -70,13 +75,18 @@ func (r *runner) start() error {
if err := cmd.Start(); err != nil {
return err
}
r.cmd, r.ready = cmd, false
r.cmd, r.ready, r.yielding = cmd, false, false
log.Printf("mavgpud: started llama-server pid=%d", cmd.Process.Pid)
go func() {
err := cmd.Wait()
r.mu.Lock()
r.cmd, r.ready = nil, false
yielded := r.yielding
r.cmd, r.ready, r.yielding = nil, false, false
r.mu.Unlock()
if yielded {
log.Printf("mavgpud: llama-server stopped, card yielded (%v)", err)
return
}
log.Printf("mavgpud: llama-server exited: %v", err)
}()
return nil
@@ -90,6 +100,10 @@ func (r *runner) stop(grace time.Duration) {
r.mu.Lock()
cmd := r.cmd
r.ready = false
if cmd != nil && cmd.Process != nil {
// The exit that follows is ours, not a crash.
r.yielding = true
}
r.mu.Unlock()
if cmd == nil || cmd.Process == nil {
return
+59
View File
@@ -0,0 +1,59 @@
package main
import (
"os"
"path/filepath"
"testing"
"time"
)
// fakeServer writes an executable standing in for llama-server: it ignores
// SIGTERM the way the real one effectively does — by dying messily rather than
// cleanly — and reports a non-zero status.
func fakeServer(t *testing.T, body string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "fake-llama-server")
if err := os.WriteFile(path, []byte("#!/bin/sh\n"+body+"\n"), 0o755); err != nil {
t.Fatal(err)
}
return path
}
// A deliberate stop is a yield, and the log has to say so.
//
// llama-server aborts inside its own static teardown on SIGTERM, so the exit
// status of a routine yield is identical to that of a real crash. Reading the
// mavgpud log, the two were indistinguishable (Vikunja #491).
func TestStopMarksTheExitAsAYield(t *testing.T) {
r := newRunner(fakeServer(t, "while : ; do sleep 1 ; done"), nil, "")
if err := r.start(); err != nil {
t.Fatalf("start: %v", err)
}
r.mu.Lock()
if r.yielding {
t.Error("a freshly started server is already marked as yielding")
}
r.mu.Unlock()
r.stop(2 * time.Second)
deadline := time.Now().Add(2 * time.Second)
for time.Now().Before(deadline) {
if !r.running() {
return
}
time.Sleep(10 * time.Millisecond)
}
t.Fatal("the child outlived stop")
}
// Stopping when nothing is running must not arm the flag for the next child.
// The next exit after that would be a real crash logged as a yield.
func TestStopWithNoChildDoesNotArmTheFlag(t *testing.T) {
r := newRunner("/nonexistent", nil, "")
r.stop(10 * time.Millisecond)
r.mu.Lock()
defer r.mu.Unlock()
if r.yielding {
t.Error("stop armed the yield flag with no child running")
}
}
+4
View File
@@ -19,6 +19,10 @@ RestartSec=5
# llama-server on SIGTERM, so give it longer than stop_grace to do that.
KillSignal=SIGTERM
TimeoutStopSec=60
# llama-server aborts inside its own static teardown on SIGTERM, so every
# routine yield used to write a multi-gigabyte core into systemd-coredump
# (Vikunja #491). Yielding is meant to happen several times a day.
LimitCORE=0
[Install]
WantedBy=default.target
@@ -0,0 +1,65 @@
# Recall topic veto, what it costs and what it buys, 2026-08-03
Vikunja #496. The task asked for a cross-language fix. Skip the topic veto in
`memory.RecallAllowed` when the question and the hit are in different scripts.
An English question would then stop losing a Russian note.
No such case exists. No fixture case puts the question and its wanted note in
different scripts. The case the task named is not one either.
en-hard-024
query "what fixed the screen problem"
note "the flicker went away once i swapped the display cable"
Both are English. It is a paraphrase failure, not a language failure. A script
test would not have changed a single case, and neither would a bilingual stem
map.
## What the veto is worth today
Measured with the real embedder, multilingual-e5-small int8, gate 0.55, margin
0.008. The first row is the veto as it ships. The second is `RecallAllowed`
forced to true.
| | cases passing | answered | false recall | silenced by gate |
|---|---|---|---|---|
| veto on | 22/32 | 17/27 | 0/5 | 2 |
| veto off | 22/32 | 18/27 | 1/5 | 1 |
The pass count does not move. The veto trades one true recall for one false one.
It costs `en-hard-024` and it buys `ru-silent-029`:
ru-silent-029
query "во сколько отходит поезд"
note "погулял вдоль реки" 0.835, margin 0.019
The second case counted as silenced by the gate is `ru-home-026` at margin
0.001, which the margin gate stops. The veto has nothing to do with it.
## Why no lexical rule separates the two
`en-hard-024` and `ru-silent-029` are in the same lexical class. Both questions
share zero content words with their hit, and neither carries a first-person
marker. The scores sit on top of each other, 0.826 against 0.835, and so do the
margins, 0.023 against 0.019. Only one thing separates them. A screen problem
and a swapped display cable are the same event. A train and a river walk are
not. The embedder scores that difference at nine thousandths.
So the signal is semantic and the gate is lexical. Any rule cheap enough to sit
in `RecallAllowed` and strong enough to recover `en-hard-024` also re-admits
`ru-silent-029`, which puts false recall back to 1/5.
One near-miss rule was tried on paper and rejected: let the veto pass when the
hit itself is first person. It works on these two, because the English note says
"i swapped" and the Russian note says only "погулял". It is backwards as a
principle. A first-person note is exactly the personal note the veto keeps away
from a world question. The rule would weaken the veto where it was designed to
bite. It survives here only because Russian drops the pronoun.
## Decision
Accept the loss. `en-hard-024` stays silenced and false recall stays 0/5.
The way out is a reranker, not a longer word list. Recall@3 is 85.2% against
recall@1 at 70.4%, so the right note is usually in the returned set and ranked
wrong. That is where the remaining points are, and it is not this task.
+11 -4
View File
@@ -18,6 +18,7 @@ import (
"sort"
"strings"
"time"
"unicode"
)
// Fact sources. A calendar event reaches the store as a
@@ -153,14 +154,20 @@ func Overlapping(events []Event, from, to time.Time) []Event {
return out
}
// safeKey makes a summary safe to use inside a fact key (ASCII alphanumerics
// and dashes). Non-Latin summaries collapse to their punctuation, which is why
// the day prefix carries the identity and this only disambiguates within a day.
// safeKey makes a summary safe to use inside a fact key: letters and digits in
// any script, plus dashes, with space and underscore folded to a dash.
//
// It kept ASCII only until 04-08-2026, and dropped everything else. His
// calendar is Russian, so "Встреча с Аней" and "Обед с мамой" both reduced to
// "--" and produced the same key on the same day — the second event of the day
// silently overwrote the first (Vikunja #443). Letting the letters through is
// what makes the key identify the event. Migration #18 drops the keys written
// under the old rule; they are re-derived on the next poll.
func safeKey(s string) string {
var b strings.Builder
for _, r := range s {
switch {
case (r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') || (r >= '0' && r <= '9') || r == '-':
case unicode.IsLetter(r) || unicode.IsDigit(r) || r == '-':
b.WriteRune(r)
case r == ' ' || r == '_':
b.WriteRune('-')
+19
View File
@@ -139,6 +139,9 @@ func TestSafeKey(t *testing.T) {
{"Hello_World", "Hello-World"},
{"special@#$chars!!", "specialchars"},
{"ALL_CAPS_123", "ALL-CAPS-123"},
// His calendar is Russian. These reduced to "--" and "--" (Vikunja #443).
{"Встреча с Аней", "Встреча-с-Аней"},
{"Обед с мамой", "Обед-с-мамой"},
}
for _, tt := range tests {
if got := safeKey(tt.in); got != tt.want {
@@ -263,3 +266,19 @@ func TestSourceTrust(t *testing.T) {
t.Errorf("Sources() = %v", Sources())
}
}
// Two Russian events on one day must not share a key. They did: safeKey kept
// ASCII only, so both summaries collapsed to their spaces and the second event
// overwrote the first in the store (Vikunja #443).
func TestFactKeyDistinguishesRussianEventsOnOneDay(t *testing.T) {
day := time.Date(2026, 8, 4, 0, 0, 0, 0, time.UTC)
a := Event{Summary: "Встреча с Аней", Start: day.Add(10 * time.Hour), End: day.Add(11 * time.Hour)}
b := Event{Summary: "Обед с мамой", Start: day.Add(13 * time.Hour), End: day.Add(14 * time.Hour)}
if FactKeyIn(a, time.UTC) == FactKeyIn(b, time.UTC) {
t.Fatalf("both events keyed as %q", FactKeyIn(a, time.UTC))
}
// The day prefix still has to survive, because the store range-scans on it.
if !strings.HasPrefix(FactKeyIn(a, time.UTC), KeyPrefixForDay(day)) {
t.Fatalf("key %q lost the day prefix %q", FactKeyIn(a, time.UTC), KeyPrefixForDay(day))
}
}
+8
View File
@@ -60,6 +60,14 @@ var firstPerson = map[string]bool{
// kill one false one. A question about his own life keeps the embedder alone
// as its judge. A question about the world has to name something the memory
// actually mentions.
//
// The veto's price was re-measured on 2026-08-03 (#496,
// docs/evals/2026-08-03-recall-topic-veto.md). It costs one true recall and
// buys one false one, and the fixture pass count is the same either way. The
// lost case is an English paraphrase, not the cross-language loss it was
// reported as, and the fixture has no cross-language case at all. Do not add a
// script test or a bilingual stem map for it — both are no-ops here. The
// separating signal is semantic and belongs in a reranker, not in this file.
func RecallAllowed(query, text string) bool {
if mentionsHim(query) {
return true
+16
View File
@@ -31,6 +31,22 @@ func TestRecallAllowed(t *testing.T) {
}
}
// The known cost of the veto and the thing that pays for it, both measured on
// the held-out fixture with the real embedder (#496,
// docs/evals/2026-08-03-recall-topic-veto.md). The two are one lexical class:
// zero shared content words, no first-person marker, scores 0.826 against 0.835
// and margins 0.023 against 0.019. Recovering the first re-admits the second,
// which puts false recall back to 1/5. Anyone loosening the veto has to move
// the first line without moving the second.
func TestRecallVetoTradeIsPinned(t *testing.T) {
if RecallAllowed("what fixed the screen problem", "the flicker went away once i swapped the display cable") {
t.Error("en-hard-024 is expected to stay vetoed — if this passes now, re-measure false recall before celebrating")
}
if RecallAllowed("во сколько отходит поезд", "погулял вдоль реки") {
t.Error("ru-silent-029 must stay vetoed — this is the false recall the veto exists to stop")
}
}
// A question made only of filler has no topic word to match on, and the score
// gate is then the only judge it can have.
func TestRecallAllowedFallsBackWhenNothingToCompare(t *testing.T) {
+24 -2
View File
@@ -53,9 +53,31 @@ const MinOnPatternFraction = 0.7
// a repeat. False negatives cost one more observation and nothing else.
const MinEvents = 4
// MinIntervalDays — the fastest rhythm that may be called a routine. Two
// hours.
//
// Without a floor, four taps of the same key minutes apart give intervals near
// 0.002 days. They all sit inside the ±50% band by construction, so the
// detector proposed a routine and PhraseRoutine worded it as "каждый день"
// (Vikunja #468). The damage outlives the mistake: UNIQUE(action, object)
// means dismissing the bogus proposal burns that pair permanently, so the real
// routine behind it can never be proposed again.
//
// Two hours rather than a day, because a genuine habit can run several times a
// day — meals, water, a break. Anything faster than that is not a habit she
// should be proposing to remind him about; the loop rules already cover that
// range, and they are rules, not guesses. It is checked against the median, so
// one quick repeat inside a real rhythm still counts.
//
// The other half of this is that hand-QA of the detector was unsafe: seeding a
// pattern the obvious way, four chat turns in a row, poisoned the very pair
// being tested.
const MinIntervalDays = 2.0 / 24.0
// Detect checks whether a sequence of events for the same action+object
// forms a stable recurring pattern. Returns a ProposedRoutine when:
// - At least MinEvents events exist (≥3 intervals)
// - The median interval is at least MinIntervalDays
// - At least MinOnPatternFraction of the intervals sit within
// MaxIntervalRatio of the median interval
//
@@ -88,8 +110,8 @@ func Detect(events []Event) (*ProposedRoutine, error) {
}
center := medianFloat(intervals)
if center <= 0 {
return nil, nil
if center <= 0 || center < MinIntervalDays {
return nil, nil // a burst, not a rhythm — see MinIntervalDays
}
// Keep the intervals that sit inside the band around the median. The
+43
View File
@@ -216,3 +216,46 @@ func TestDetectMedianBandNotExtremes(t *testing.T) {
})
}
}
// A burst is not a habit. Four taps of the same key minutes apart give
// intervals near 0.002 days, all inside the ±50% band by construction, so the
// detector called it a daily routine (Vikunja #468). Dismissing that proposal
// burns the action+object pair permanently, which also made hand-QA of the
// detector unsafe.
func TestDetectRejectsABurst(t *testing.T) {
base := time.Date(2026, 8, 4, 9, 0, 0, 0, time.UTC)
var events []Event
for i := 0; i < 4; i++ {
events = append(events, Event{
Action: "refill", Object: "cat_water",
Ts: base.Add(time.Duration(i) * 7 * time.Minute),
})
}
r, err := Detect(events)
if err != nil {
t.Fatalf("Detect: %v", err)
}
if r != nil {
t.Fatalf("four taps minutes apart proposed a routine every %.3f days", r.IntervalDays)
}
}
// The floor is two hours, not a day: a habit that runs several times a day is
// still a habit.
func TestDetectKeepsASeveralTimesADayHabit(t *testing.T) {
base := time.Date(2026, 8, 4, 8, 0, 0, 0, time.UTC)
var events []Event
for i := 0; i < 5; i++ {
events = append(events, Event{
Action: "drink", Object: "water",
Ts: base.Add(time.Duration(i) * 4 * time.Hour),
})
}
r, err := Detect(events)
if err != nil {
t.Fatalf("Detect: %v", err)
}
if r == nil {
t.Fatal("a four-hour rhythm over five events is a habit, got nil")
}
}
+29 -2
View File
@@ -173,12 +173,23 @@ func checkFeminine(body string) Result {
// Second pass: self-reference with the pronoun dropped — "напомнил тебе",
// "проверил за тебя". A masculine past-tense verb whose object is HIM can
// only be her speaking about herself.
//
// Two guards, both from a false positive on the talk fixture: "ты заплатил
// за домен до марта" scored as her drift and cost the run a point it had
// earned (Vikunja #462). He is male, so a past-tense verb governed by "ты"
// must be masculine. And a bare "за" is not evidence of anything — "за
// домен" is a price, "за тебя" is her doing something on his behalf — so it
// only counts when he is the one it points at.
for i, w := range words {
if !masculinePast(w) || i+1 >= len(words) {
if !masculinePast(w) || i+1 >= len(words) || governedByYou(words, i) {
continue
}
next := words[i+1]
if next == "тебе" || next == "тебя" || next == "за" {
aboutHim := next == "тебе" || next == "тебя"
if next == "за" && i+2 < len(words) && (words[i+2] == "тебя" || words[i+2] == "тебе") {
aboutHim = true
}
if aboutHim {
return Result{CheckFeminine, false,
fmt.Sprintf("masculine self-reference %q before %q", w, next)}
}
@@ -652,3 +663,19 @@ func checkEllipsis(body string) Result {
}
return Result{CheckEllipsis, true, ""}
}
// governedByYou reports whether "ты" stands close enough in front of the verb
// at index i to be its subject. Three words, the same window checkFeminine's
// first pass uses after "я", and it stops at a first-person pronoun so "ты
// просил, я напомнил" still trips.
func governedByYou(words []string, i int) bool {
for j := i - 1; j >= 0 && j >= i-3; j-- {
switch words[j] {
case "ты":
return true
case "я":
return false
}
}
return false
}
+6
View File
@@ -106,6 +106,12 @@ func TestChecksCatchWhatTheyClaim(t *testing.T) {
{"masculine predicative", "я должен сказать: попей воды.", CheckFeminine},
// The other direction: HE is male, so second-person masculine is right.
{"second person masculine ok", "ты не пил воду четыре часа.", ""},
// The recorded false positive: "заплатил" sits before "за", and the
// second pass read that as her dropping the pronoun. The subject is
// "ты" and he is male, so the reply is right (Vikunja #462).
{"second person masculine before за", "ты заплатил за домен до марта, а воду пить всё равно надо.", ""},
// The same shape she really does get wrong still trips.
{"masculine on his behalf", "проверил за тебя — воды не было четыре часа.", CheckFeminine},
// The real observed failure: she addressed him as a woman.
{"feminine second person", "ты давно не отдыхала — попей воды.", CheckHisGender},
{"feminine second person no dash", "ты пила воду четыре часа назад.", CheckHisGender},
+47 -4
View File
@@ -26,20 +26,22 @@ import (
"time"
"github.com/kami/maven/internal/dialogue"
"github.com/kami/maven/internal/router"
)
//go:embed talk_v1.json
var talkFixtureJSON []byte
// The three phrasing paths under test. Values match the fixture's "path" field.
// The phrasing paths under test. Values match the fixture's "path" field.
const (
PathChat = "chat" // PhraseChat
PathQuery = "query" // PhraseQuery with notes
PathKnowledge = "knowledge" // PhraseQuery with no notes
PathReply = "reply" // PhraseReply, the reactive confirmation
)
// TalkPaths — report order.
var TalkPaths = []string{PathChat, PathQuery, PathKnowledge}
var TalkPaths = []string{PathChat, PathQuery, PathKnowledge, PathReply}
// TalkCheckNames — the checks that apply to a free-form reply, in report order.
// Deliberately a subset of CheckNames: length, mood and "no questions" are nudge
@@ -58,12 +60,19 @@ var TalkCheckNames = []string{
// WantAny is the on-topic contract: at least one lowercased fragment must appear
// in the reply. Fragments are stems ("пароль" → "парол") so declension does not
// defeat them.
//
// Intent, Key and Value carry the reply path's decision: that path is phrased
// from what the router already resolved, not from the raw utterance. Utterance
// stays filled anyway, because it is what a human reads in the report.
type TalkCase struct {
ID string `json:"id"`
Path string `json:"path"`
Utterance string `json:"utterance"`
History []string `json:"history,omitempty"`
Notes []string `json:"notes,omitempty"`
Intent string `json:"intent,omitempty"`
Key string `json:"key,omitempty"`
Value string `json:"value,omitempty"`
WantAny []string `json:"want_any"`
Tags []string `json:"tags,omitempty"`
Note string `json:"note,omitempty"`
@@ -92,13 +101,27 @@ func LoadTalk() (TalkFixture, error) {
return f, nil
}
// Talker — the two methods a conversational path must have to be scorable.
// *phraser.LLMPhraser satisfies it; same trick as Nudger.
// Talker — the methods a conversational path must have to be scorable.
// *phraser.LLMPhraser satisfies the first two; *phraser.Replier satisfies the
// third, so a run that scores all four paths passes a Pair.
type Talker interface {
PhraseChat(ctx context.Context, utterance string, history []dialogue.Turn) (string, error)
PhraseQuery(ctx context.Context, utterance string, notes []string) (string, error)
}
// Confirmer — the reply path. *phraser.Replier satisfies it.
type Confirmer interface {
PhraseReply(ctx context.Context, d router.Decision) (string, error)
}
// Pair joins the two objects the daemon wires separately — the phraser and the
// replier — so one ScoreTalk call covers every path Maven speaks through. A bare
// Talker still works; its reply cases score as errors, which is honest.
type Pair struct {
Talker
Confirmer
}
// TalkOutcome — one scored case.
type TalkOutcome struct {
Case TalkCase
@@ -194,10 +217,30 @@ func (c TalkCase) run(ctx context.Context, t Talker) (string, error) {
return t.PhraseQuery(ctx, c.Utterance, c.Notes)
case PathKnowledge:
return t.PhraseQuery(ctx, c.Utterance, nil)
case PathReply:
conf, ok := t.(Confirmer)
if !ok {
return "", fmt.Errorf("target cannot phrase replies — pass a Pair")
}
return conf.PhraseReply(ctx, c.decision())
}
return "", fmt.Errorf("unknown path %q", c.Path)
}
// decision rebuilds what the router would have handed the replier. Text is the
// utterance for a note or a reminder, which is what the router puts there.
func (c TalkCase) decision() router.Decision {
return router.Decision{
Intent: router.Intent(c.Intent),
Slots: router.Slots{
Key: c.Key,
Value: c.Value,
Text: c.Utterance,
HasKey: c.Key != "",
},
}
}
func (c TalkCase) turns() []dialogue.Turn {
turns := make([]dialogue.Turn, 0, len(c.History))
for _, h := range c.History {
+17 -2
View File
@@ -11,6 +11,7 @@ import (
"github.com/kami/maven/internal/llm"
"github.com/kami/maven/internal/persona"
"github.com/kami/maven/internal/phraser"
"github.com/kami/maven/internal/router"
)
// perPathMinimum — the resolution floor. A per-path score built on a handful of
@@ -36,6 +37,10 @@ func TestTalkFixture(t *testing.T) {
switch c.Path {
case PathChat, PathQuery, PathKnowledge:
case PathReply:
if c.Intent == "" {
t.Errorf("%s: reply case has no intent — the replier is phrased from the decision", c.ID)
}
default:
t.Errorf("%s: unknown path %q", c.ID, c.Path)
}
@@ -69,10 +74,15 @@ type fakeTalker struct{ reply string }
func (f fakeTalker) PhraseChat(context.Context, string, []dialogue.Turn) (string, error) {
return f.reply, nil
}
func (f fakeTalker) PhraseQuery(context.Context, string, []string) (string, error) {
return f.reply, nil
}
func (f fakeTalker) PhraseReply(context.Context, router.Decision) (string, error) {
return f.reply, nil
}
// TestScoreTalkCounts — a reply that fails on purpose must be counted on every
// path, so a real run cannot report a hidden zero.
func TestScoreTalkCounts(t *testing.T) {
@@ -104,7 +114,7 @@ func TestScoreTalkCounts(t *testing.T) {
}
}
// TestLLMTalkBaseline — the resident model on the three conversational paths.
// TestLLMTalkBaseline — the resident model on all four phrasing paths.
// Opt-in exactly like TestLLMPhrasingBaseline: CI has no model and a run costs
// minutes on the CPU target.
//
@@ -148,7 +158,12 @@ func TestLLMTalkBaseline(t *testing.T) {
}
t.Logf("scoring model %s at %s", model, base)
rep, err := ScoreTalk(ctx, "llm ("+model+", built-in persona)", p, f)
// The reply path is a separate object in the daemon too: the phraser owns its
// own llama-server, the replier is handed an llm.Client. Pair scores both.
block := func() string { return persona.Facts{}.Block(time.Now()) }
target := Pair{Talker: p, Confirmer: phraser.NewReplier(llm.New(base, cfg.Timeout), block)}
rep, err := ScoreTalk(ctx, "llm ("+model+", built-in persona)", target, f)
if err != nil {
t.Fatalf("ScoreTalk: %v", err)
}
+84
View File
@@ -222,6 +222,90 @@
"utterance": "почему гром слышно позже молнии?",
"want_any": ["звук", "све", "быстр", "гром", "молни"],
"tags": ["general"]
},
{
"id": "reply-fact-coffee",
"path": "reply",
"intent": "fact",
"key": "кофе",
"value": "закончился",
"utterance": "кофе закончился",
"want_any": ["коф"],
"tags": ["fact"],
"note": "The plainest confirmation there is, and the sentence he hears most often."
},
{
"id": "reply-fact-weight",
"path": "reply",
"intent": "fact",
"key": "вес",
"value": "82",
"utterance": "мой вес 82",
"want_any": ["вес", "82"],
"tags": ["fact", "number"],
"note": "A number must survive into the confirmation; a paraphrase that drops it is useless."
},
{
"id": "reply-fact-pill",
"path": "reply",
"intent": "fact",
"key": "таблетки",
"value": "выпил",
"utterance": "таблетки выпил",
"want_any": ["таблетк"],
"tags": ["fact", "feminine"],
"note": "He says 'выпил', masculine and about himself. She must not copy the form onto herself."
},
{
"id": "reply-note-router",
"path": "reply",
"intent": "note",
"utterance": "роутер перезагружается сам по ночам",
"want_any": ["роутер"],
"tags": ["note"]
},
{
"id": "reply-note-long",
"path": "reply",
"intent": "note",
"utterance": "если диск снова отвалится, посмотреть кабель, а не контроллер, в прошлый раз был кабель",
"want_any": ["диск", "кабел"],
"tags": ["note", "length"],
"note": "A long note baits a long confirmation. One sentence is the contract."
},
{
"id": "reply-reminder-evening",
"path": "reply",
"intent": "reminder",
"utterance": "напомни вечером полить цветы",
"want_any": ["цвет", "полит", "вечер"],
"tags": ["reminder"]
},
{
"id": "reply-reminder-tomorrow",
"path": "reply",
"intent": "reminder",
"utterance": "напомни завтра позвонить в поликлинику",
"want_any": ["поликлиник", "позвон", "звон"],
"tags": ["reminder"]
},
{
"id": "reply-formality-bait",
"path": "reply",
"intent": "note",
"utterance": "запишите пожалуйста что счётчики я сдал",
"want_any": ["счётчик", "счетчик"],
"tags": ["note", "persona-bait", "address"],
"note": "Polite plural in the input. The confirmation must still be на ты."
},
{
"id": "reply-question-bait",
"path": "reply",
"intent": "note",
"utterance": "надо купить фильтр для воды, не помню какой",
"want_any": ["фильтр"],
"tags": ["note", "no-question"],
"note": "An unresolved note invites her to ask which filter. A confirmation does not ask."
}
]
}
+112
View File
@@ -0,0 +1,112 @@
// phraser/replier.go — reactive reply phrasing, the confirmation he hears
// after every fact, note and reminder.
//
// It lived in cmd/mavend as package main until Vikunja #396, which meant the
// most frequently heard sentence Maven says was the one path the phrasing eval
// could not import, let alone score. Nothing here talks to the daemon: the
// caller supplies the completer and the context block, and cmd/mavend keeps the
// stub fallback so a model error still answers.
package phraser
import (
"context"
"strings"
"time"
"github.com/kami/maven/internal/llm"
"github.com/kami/maven/internal/persona"
"github.com/kami/maven/internal/router"
)
// Completer is the model seam for the replier, a subset of router.Completer.
// *llm.Client satisfies it.
type Completer interface {
Complete(ctx context.Context, r llm.Req) (string, error)
}
// replyTimeout bounds one reply. Generous because the resident model on the CPU
// floor is slow and the caller has a deterministic fallback anyway.
const replyTimeout = 60 * time.Second
// ReplySystemPrompt — the reactive confirmation contract: one short Russian
// sentence, feminine self-reference, informal address, no question.
const ReplySystemPrompt = `Ты — Maven, домашняя ассистентка (о себе — в женском роде). Владелец — мужчина, говоришь с ним на "ты", в единственном числе; никогда не "вы"/"ваш" и не "он"/"его". Подтверди действие РОВНО ОДНИМ коротким предложением (≤120 символов), по-русски, спокойно и без официальных формулировок. Не задавай вопросов, не повторяй слова, не добавляй ничего после точки. Отвечай ТОЛЬКО одним объектом JSON с полями "response" (текст) и "mood" (ровно одно из: neutral, happy, thinking, tired, confused).
Пример: {"response": "Записала, что ты выпил стакан воды.", "mood": "neutral"}
Никогда не пиши "..." в поле response.`
// Replier phrases reactive confirmations with the resident model. It has no
// fallback of its own: an error is returned, and the daemon answers from the
// deterministic stub. That is also what makes it scorable — a dead server shows
// up as an error rather than as bad phrasing.
type Replier struct {
c Completer
// block renders the shared context block per turn (who he is, the time).
// nil ⇒ the prompt stands alone.
block func() string
}
// NewReplier builds a replier over c. block may be nil.
func NewReplier(c Completer, block func() string) *Replier {
return &Replier{c: c, block: block}
}
// PhraseReply returns the confirmation for one decision. An empty string with a
// nil error means the model produced nothing usable, which the caller must
// treat exactly like an error.
func (r *Replier) PhraseReply(ctx context.Context, d router.Decision) (string, error) {
ctx, cancel := context.WithTimeout(ctx, replyTimeout)
defer cancel()
out, err := r.c.Complete(ctx, llm.Req{
System: persona.Prepend(r.block, ReplySystemPrompt),
User: replyContext(d),
Grammar: ResponseGrammar,
MaxTokens: 512,
RepeatPenalty: 1.3,
})
if err != nil {
return "", err
}
out = stripThink(out)
if response, _, perr := parseResponseMood(out); perr != nil {
return "", perr
} else if response != "" {
return response, nil
}
// fallback: the model answered in bare prose, which is fine here.
return firstSentence(out), nil
}
// firstSentence trims the model's output to a single clean confirmation: first
// line, first sentence, whitespace-normalized — the last-line defense against a
// small model that rambles past the first period despite the prompt + stop.
func firstSentence(s string) string {
s = strings.TrimSpace(s)
if i := strings.IndexByte(s, '\n'); i >= 0 {
s = s[:i]
}
// keep up to and including the first sentence-ending punctuation.
if i := strings.IndexAny(s, ".!?"); i >= 0 {
s = s[:i+1]
}
return strings.TrimSpace(s)
}
// replyContext renders the decision into a compact RU description for the model.
func replyContext(d router.Decision) string {
switch d.Intent {
case router.IntentFact:
return "записала факт: " + d.Slots.Key + " " + d.Slots.Value
case router.IntentNote:
return "сохранила заметку: " + d.Slots.Text
case router.IntentReminder:
return "поставила напоминание: " + d.Slots.Text
default:
return string(d.Intent) + ": " + d.Slots.Text
}
}
// StripThink removes the <think> block a Thinking-variant model emits before its
// answer. Exported for the daemon's own model callers, which parse output that
// never passes through a phraser method.
func StripThink(s string) string { return stripThink(s) }
+90
View File
@@ -0,0 +1,90 @@
package phraser
import (
"context"
"testing"
"github.com/kami/maven/internal/llm"
"github.com/kami/maven/internal/router"
)
type mockCompleter struct {
out string
err error
}
func (m mockCompleter) Complete(_ context.Context, _ llm.Req) (string, error) { return m.out, m.err }
func TestReplierReturnsLLMReply(t *testing.T) {
r := NewReplier(mockCompleter{out: `{"response":"записала, кофе закончился","mood":"neutral"}`}, nil)
got, err := r.PhraseReply(context.Background(), noteDecision())
if err != nil || got != "записала, кофе закончился" {
t.Errorf("got %q, %v, want %q, nil", got, err, "записала, кофе закончился")
}
}
func TestReplierFallsBackToPlainText(t *testing.T) {
r := NewReplier(mockCompleter{out: "записала, кофе закончился"}, nil)
got, err := r.PhraseReply(context.Background(), noteDecision())
if err != nil || got != "записала, кофе закончился" {
t.Errorf("got %q, %v, want %q, nil", got, err, "записала, кофе закончился")
}
}
func TestReplierReportsTheModelError(t *testing.T) {
r := NewReplier(mockCompleter{err: errTestLLMDown}, nil)
got, err := r.PhraseReply(context.Background(), noteDecision())
if err == nil {
t.Errorf("got %q, nil error — a dead model must be reported, not phrased around", got)
}
}
// A fragment the grammar left half-open is a failed generation. It must come
// back as an error so the daemon reaches its stub, not as a reply.
func TestReplierRejectsBrokenJSON(t *testing.T) {
r := NewReplier(mockCompleter{out: `{"response":"запис`}, nil)
got, err := r.PhraseReply(context.Background(), noteDecision())
if err == nil || got != "" {
t.Errorf("got %q, %v, want empty and an error", got, err)
}
}
func TestReplierEmptyOutputIsEmpty(t *testing.T) {
r := NewReplier(mockCompleter{out: ""}, nil)
got, err := r.PhraseReply(context.Background(), noteDecision())
if err != nil || got != "" {
t.Errorf("got %q, %v, want empty and no error", got, err)
}
}
// grammarRecorder captures the request so the grammar can be asserted on.
type grammarRecorder struct{ req llm.Req }
func (g *grammarRecorder) Complete(_ context.Context, r llm.Req) (string, error) {
g.req = r
return `{"response":"записала","mood":"neutral"}`, nil
}
func TestReplierCarriesTheResponseGrammar(t *testing.T) {
rec := &grammarRecorder{}
r := NewReplier(rec, nil)
if _, err := r.PhraseReply(context.Background(), noteDecision()); err != nil {
t.Fatalf("PhraseReply: %v", err)
}
if rec.req.Grammar != ResponseGrammar {
t.Errorf("grammar = %q, want ResponseGrammar", rec.req.Grammar)
}
if rec.req.System != ReplySystemPrompt {
t.Errorf("system prompt = %q, want ReplySystemPrompt", rec.req.System)
}
}
func noteDecision() router.Decision {
return router.Decision{Intent: router.IntentNote, Slots: router.Slots{Text: "кофе закончился"}}
}
var errTestLLMDown = errTest("llm down")
type errTest string
func (e errTest) Error() string { return string(e) }
+44
View File
@@ -75,3 +75,47 @@ func TestAgendaGrammarSparesStatements(t *testing.T) {
}
}
}
// The tomorrow form and the bare event noun. Both were measured answering
// "пока не умею" on the deployed daemon, 02-08-2026, while the same question
// about today worked — the first rule set needed "у меня" or a calendar noun
// and these phrasings carry neither (Vikunja #471).
func TestAgendaCoversOtherDaysAndNamedEvents(t *testing.T) {
r := agendaRouter(t)
for _, u := range []string{
"какие планы на завтра?",
"какие планы на послезавтра",
"что по делам в среду",
"какие планы на выходные",
"когда планёрка?",
"во сколько созвон",
"когда будет совещание",
} {
d, err := r.Route(context.Background(), u, refNow())
if err != nil {
t.Fatalf("route(%q): %v", u, err)
}
if d.Intent != IntentQuery {
t.Errorf("route(%q) = %s, want query", u, d.Intent)
}
}
}
// The two new rules are narrow on purpose. A world question that opens with
// "когда" is not an agenda question, and telling her about a plan is not
// asking about one.
func TestAgendaGrammarsLeaveTheWorldAlone(t *testing.T) {
r := agendaRouter(t)
for _, u := range []string{
"когда была битва при ватерлоо",
"когда изобрели телефон",
} {
d, err := r.Route(context.Background(), u, refNow())
if err != nil {
t.Fatalf("route(%q): %v", u, err)
}
if d.Stage == 0 {
t.Errorf("route(%q) was claimed at stage 0 as %s", u, d.Intent)
}
}
}
+2
View File
@@ -23,6 +23,8 @@
{ "id": "ru-query-012", "utterance": "какие заметки я оставил про полив", "lang": "ru", "intent": "query", "tags": ["recall"] },
{ "id": "ru-query-013", "utterance": "во сколько у меня встреча", "lang": "ru", "intent": "query", "tags": ["calendar"] },
{ "id": "ru-query-019", "utterance": "что у меня стоит в календаре на послезавтра", "lang": "ru", "intent": "query", "tags": ["calendar", "hard"], "note": "agenda, not the clock: the daemon answers this from CalendarEvents inside the query branch, so the clock/date system rule must not swallow it" },
{ "id": "ru-query-022", "utterance": "какие планы на завтра?", "lang": "ru", "intent": "query", "tags": ["calendar"], "note": "the same agenda question as ru-query-019 aimed at another day; it answered \u043f\u043e\u043a\u0430 \u043d\u0435 \u0443\u043c\u0435\u044e on the deployed daemon while the today form worked (Vikunja #471)" },
{ "id": "ru-query-023", "utterance": "\u043a\u043e\u0433\u0434\u0430 \u043f\u043b\u0430\u043d\u0451\u0440\u043a\u0430?", "lang": "ru", "intent": "query", "tags": ["calendar", "hard"], "note": "a named event with no calendar word — the noun is the only signal that this is a question about his day" },
{ "id": "ru-query-014", "utterance": "я успеваю до дедлайна", "lang": "ru", "intent": "query", "tags": ["hard", "no-question-word"] },
{ "id": "ru-query-015", "utterance": "сколько я прошёл шагов", "lang": "ru", "intent": "query", "tags": ["aggregate"] },
{ "id": "ru-query-016", "utterance": "покажи давление за неделю", "lang": "ru", "intent": "query", "tags": ["hard", "imperative"], "note": "imperative form but a read — must not route to act" },
+5 -1
View File
@@ -210,7 +210,11 @@ func (lr *LLMRouter) Route(ctx context.Context, utterance string, now time.Time)
d.Slots.HasKey = a.Key != ""
case IntentReminder:
d.Intent = IntentReminder
d.Slots.Text = firstNonEmpty(a.Text, utterance)
// No utterance fallback here, unlike every other intent below. The
// model returning no text for a reminder means it found no subject,
// and "напомни в 11" is not a subject. Leaving Text empty is what
// lets the gate turn that into a question (Vikunja #383).
d.Slots.Text = a.Text
case IntentNote:
d.Intent = IntentNote
d.Slots.Text = firstNonEmpty(a.Text, utterance)
+32
View File
@@ -356,3 +356,35 @@ func TestRouterLLMFactWithResolvedKeyStaysConfident(t *testing.T) {
t.Fatalf("a fact the parser could key must not clarify: %+v", d)
}
}
// A reminder with a time and no subject must come back empty and gated, not
// backfilled with the raw words. "напомни в 11" carries an hour and nothing to
// say at that hour; parking the utterance in Text made the request look
// complete, so the daemon set a reminder that fires saying "напомни в 11"
// (Vikunja #383).
func TestLLMReminderWithoutSubjectAsksInsteadOfGuessing(t *testing.T) {
r := newLLMTestRouter(t, `{"intent":"reminder"}`)
d, err := r.Route(context.Background(), "напомни в 11", refNow())
if err != nil {
t.Fatalf("route: %v", err)
}
if d.Slots.Text != "" {
t.Fatalf("subject backfilled from the utterance: %q", d.Slots.Text)
}
if !d.Clarify {
t.Fatalf("a subjectless reminder was accepted, confidence %v", d.Confidence)
}
}
// The gate is about the subject, not about reminders in general: one that has
// both halves still runs without a question.
func TestLLMReminderWithSubjectIsNotGated(t *testing.T) {
r := newLLMTestRouter(t, `{"intent":"reminder","text":"позвонить маме"}`)
d, err := r.Route(context.Background(), "напомни в 11 позвонить маме", refNow())
if err != nil {
t.Fatalf("route: %v", err)
}
if d.Clarify {
t.Fatalf("a complete reminder was sent back as a question: %+v", d.Slots)
}
}
+15 -1
View File
@@ -147,7 +147,15 @@ func (r *Router) fillSlots(ctx context.Context, d *Decision, now time.Time) {
d.Slots.Fn, d.Slots.Args, d.Slots.HasFn = fn, args, true
}
}
if d.Slots.Text == "" {
// The extractor's Text is the raw utterance, which is the payload for a
// note, a query or a chat turn but not for a reminder — there Text is the
// subject, what she says at the hour. Backfilling it made Text impossible
// to be empty, so StillMissing never reported SlotText and "О чём
// напомнить?" was unaskable; the answer to a question she did manage to
// ask then overwrote the whole request instead of filling one gap
// (Vikunja #383). A reminder with no subject stays empty and is gated
// below into a question.
if d.Slots.Text == "" && d.Intent != IntentReminder {
d.Slots.Text = ex.Text
}
// Stage stays 1: it says who decided the route, and that was the LLM.
@@ -177,6 +185,12 @@ func (r *Router) gateLLMDecision(d *Decision) {
if d.Intent == IntentAct && !d.Slots.HasFn && d.Confidence > llmThinConfidence {
d.Confidence = llmThinConfidence
}
// A reminder with no subject: she knows when but not what to say then.
// Setting it anyway fires an empty reminder at the hour, which reads as a
// bug to him and cannot be repaired after the fact. Ask (Vikunja #383).
if d.Intent == IntentReminder && d.Slots.Text == "" && d.Confidence > llmThinConfidence {
d.Confidence = llmThinConfidence
}
if d.Confidence < r.threshold {
d.Clarify = true
}
+29
View File
@@ -182,9 +182,38 @@ func AgendaQueryGrammars() []Grammar {
Pattern: regexp.MustCompile(`(?i)^\s*(что|чего|какие|сколько|во\s+сколько|когда)\s+у\s+меня(\s|[?!.]|$)`),
Build: agendaQueryBuild,
},
{
// A plan noun aimed at a named day, with no possessive to anchor
// on: "какие планы на завтра", "что по делам в среду". The rule
// above wants "у меня" and this phrasing never has it, so
// "какие планы на завтра" answered "пока не умею" while "какие
// планы на сегодня" worked (Vikunja #471). The day word is what
// makes it an agenda question rather than a topic.
Name: "plan-day-query",
// Only "план" and "дел". A verb stem like "встреч" would take
// "встречаемся в среду", which is him telling her something, not
// asking.
Pattern: regexp.MustCompile(`(?i)(^|\s)(план|дел)[а-я]*\s+(на|в|во|по)\s+` + dayWordPattern + `(\s|[?!.]|$)`),
Build: agendaQueryBuild,
},
{
// A named event with no calendar word at all: "когда планёрка?",
// "во сколько созвон". He is asking when something on his calendar
// happens, and the noun is the only signal. Closed list, so "когда
// битва при Ватерлоо" is still a world question.
Name: "event-time-query",
Pattern: regexp.MustCompile(`(?i)^\s*(когда|во\s+сколько|в\s+котором\s+часу)\s+(будет\s+|у\s+нас\s+)?(планёрк|планерк|встреч|созвон|митинг|совещани|звонок|созвон|приём|прием|интервью|собеседовани|тренировк|урок|занятие|пара)[а-я]*(\s|[?!.]|$)`),
Build: agendaQueryBuild,
},
}
}
// dayWordPattern — the day words an agenda question can name. Weekdays appear
// in the accusative and prepositional forms the questions actually use ("в
// среду", "на среде"), which is why the stems carry an inflection tail rather
// than a fixed ending.
const dayWordPattern = `(сегодня|завтра|послезавтра|выходн[а-я]+|недел[а-я]+|понедельник[а-я]*|вторник[а-я]*|сред[ауые][а-я]*|четверг[а-я]*|пятниц[ауые][а-я]*|суббот[ауые][а-я]*|воскресень[ея][а-я]*)`
// agendaQueryBuild — shared Build for the agenda grammars. Confidence 1.0 on
// the intent only: the utterance travels intact and the query chain's own
// matchers decide the rest.
+11
View File
@@ -208,6 +208,17 @@ ALTER TABLE reminders ADD COLUMN next_fire_ts INTEGER;`, // #2
// list_tasks into something that writes without the row changing by one
// byte. The fingerprint is the declared shape at approval time, so a
// redefinition is a re-approval instead of a silent upgrade.
`DELETE FROM facts
WHERE key LIKE 'calendar_event_%'
AND replace(substr(key, 25), '-', '') = '';`,
// #18 — drop the calendar keys written while safeKey dropped Cyrillic
// (Vikunja #443). Everything after the date prefix was punctuation, so
// every Russian event on one day shared one key and only the last one
// survived. Deleting rather than rewriting: a calendar fact is derived
// data, the next poll writes the day again under keys that identify the
// event, and the old rows would otherwise be recited as extra meetings.
// The filter is exact — it keeps any key whose summary part still has a
// letter or a digit in it.
}
// migrate applies every migration with a number greater than the DB's current
+33
View File
@@ -47,3 +47,36 @@ func TestMigrateAppliesOnceAndIsIdempotent(t *testing.T) {
t.Fatalf("after re-migrate user_version = %d, want %d", v, want)
}
}
// Migration #18 clears the calendar keys written while safeKey dropped
// Cyrillic. Those rows are indistinguishable from real events on read, so
// leaving them would recite one meeting as several (Vikunja #443).
func TestCollapsedCalendarKeysAreDropped(t *testing.T) {
ctx := context.Background()
s := newTestStore(t)
rows := []string{
"calendar_event_20260804_--", // "Встреча с Аней" under the old rule
"calendar_event_20260804_", // a one-word Russian summary
"calendar_event_20260804_Встреча-с-Аней", // the new format
"calendar_event_20260804_Standup", // an ASCII summary, always fine
}
for _, key := range rows {
if _, err := s.db.ExecContext(ctx,
`INSERT INTO facts (ts, kind, key, value, source, confidence) VALUES (0, 'env', ?, 'x', 'poll:caldav', 1.0)`,
key); err != nil {
t.Fatalf("seed %q: %v", key, err)
}
}
if _, err := s.db.ExecContext(ctx, migrations[17]); err != nil {
t.Fatalf("migration 18: %v", err)
}
var got int
if err := s.db.QueryRowContext(ctx, `SELECT count(*) FROM facts WHERE key LIKE 'calendar_event_%'`).Scan(&got); err != nil {
t.Fatal(err)
}
if got != 2 {
t.Fatalf("%d calendar rows left, want the 2 that identify their event", got)
}
}