Maven/docs/evals/2026-08-07-week-of-usage.md

A week of talking to Maven

Date: 2026-08-07 Build: beb093a on master, all five compose services rebuilt and recreated from that sha. Reach used: POST /api/chat on mavweb, 74 turns over seven simulated days. Turn source is tap:text, so this exercises the same path telegram and the mic take.

Latency across 70 measured turns: p50 1.5s, p95 8.0s, max 12.3s. Stage 0 answers land at 0.0-0.5s. Anything the resident model phrases costs 4-12s.

Twelve turns answered "пока не умею отвечать на этот вопрос". Six answered "не нашла у тебя такой записи". Those two strings are 24% of the week.

Deploy

Build and recreate were clean. The resident model loaded in 9s (Qwen3-1.7B-UD-Q4_K_XL, n_ctx 4096). Nexus, Hexis and Praxis all wired. Search (searxng) and both Kiwix books came up. Telegram intake started and is reading chat 464904223.

What is broken, worst first

1. Every reminder fails to deliver, forever

NTFY_TOKEN is not set in deploy/telegram.env, so deploy/mavend.json expands "token": "${NTFY_TOKEN}" to the empty string and ntfy.kvmx.ru answers 403. The host itself is up and returns 200 unauthenticated, so this is the credential, not the box.

The consequence is worse than one missed message. cmd/mavend/tick.go:239 logs the failure and leaves the reminder pending, by design, so the next tick re-gathers and re-attempts. The 403 repeated 7 times in 20 minutes and will not stop. And DispatchReminder sends a reminder to ntfy alone when he is away. There is no telegram fallback on that path, so the reach that works never sees it.

V-649 gave the box a second reach. In deploy that reach is dark, and it took the reminders with it.

Fix is one line in the gitignored env file. The retry-forever and the missing fallback are worth a second look regardless.

2. A pending reminder clarify contaminates every later turn

This is the single worst thing to talk to. A reminder turn left its time slot unfilled. The question "на какое время поставить напоминание?" was then appended to the reply of six consecutive unrelated turns:

YOU:   о чём мы вчера говорили?
MAVEN: пока не умею отвечать на этот вопрос, на какое время поставить напоминание?

YOU:   где лежит вайфай пароль?
MAVEN: вот что я нашла: вайфай пароль лежит в ящике стола, на какое время поставить напоминание?

YOU:   отмени все напоминания
MAVEN: пока не умею отвечать на этот вопрос, на какое время поставить напоминание?

Nothing dismissed it. Not an unrelated question, not "отмени все напоминания". It cleared only when a time was finally supplied. It also leaked the other way once. A note write on day 1 opened with "Прошлую просьбу отпускаю". That is pending-reminder state narrating itself into an unrelated reply.

The same state produced a self-contradicting reply in one turn:

YOU:   нет стой, напомни через час выпить воды
MAVEN: поняла, это напоминание — запомнила. не получилось разобрать время напоминания.

3. Answering a clarify does not work

Day 1, she asked for the day and got it:

YOU:   напомни в 11:00 позвонить маме
MAVEN: Сейчас 03:29. В какой день?
YOU:   сегодня
MAVEN: Прости, не поняла — попробуй сказать иначе.

сегодня routed to intent=system. The clarify has no continuation, so the one word that answers it is read as a fresh utterance. The router did extract the time correctly (Time:2026-08-07 11:00:00 HasTime:true), which is V-572 working. The daemon asked for a day anyway, with 11:00 still eight hours out.

Relative time is not parsed at all: "напомни через час выпить воды" → "не получилось разобрать время".

The path does work when the day is explicit. "напомни завтра в 9 утра про врача" answered "хорошо, напомню завтра в 09:00" in 0.3s at stage 0.

4. Query sources claim turns that are not theirs

Four different sources over-claimed during the week.

Weather claims arithmetic and networking:

YOU:   что такое TCP?
MAVEN: для какого города?

YOU:   сколько будет 17 на 23?
MAVEN: для какого города?

The feed claims a question about him:

YOU:   какой у меня любимый язык?
MAVEN: нового вот что: Six stable kernels with a security fix; [$] Bringing BPF to binfmt_misc

The personal boundary claims a public figure, logged as query claimed by source "personal":

YOU:   кто такой Линус Торвальдс?
MAVEN: не знаю — не нашла у тебя такой записи.

And search claims a question about his own infrastructure, returning a dictionary definition of the word:

YOU:   что там с бэкапами?
MAVEN: нашла вот это: Бэкап - что такое резервное копирование и как его сделать — Бэкап (backup) — это резервная копия важных данных…

That last pair inverts the boundary in both directions at once. A world question went to personal memory. A personal question went to the world.

5. "столица Франции?" does not answer

CLAUDE.md names this utterance as the case that proves why there is no quality threshold in front of Response.Empty(). It now fails before search is ever reached: the router sends it to intent=system, and replySystem says "пока не умею". Same for "когда родился Пушкин?".

Search itself is healthy. "почему небо голубое?", "расскажи про Байкал" and "что нового в мире?" all returned real sourced text. The defect is routing, not the source.

6. Persona violations

She uses masculine self-reference, which is the constraint the CPT exists to fix:

• "Хорошо, сохранил." Twice, on both fact writes.
• "Я тоже love Go!" English mid-Russian.

She also claims his name as her own:

YOU:   привет, я Ками
MAVEN: Привет! Я КAMI, твой домашний ассистент.

And she invents weather she has no source for: "Добро пожаловать! Солнечно и тепло."

7. Small breakages worth naming

• "спасибо" → "спасиба". Invented word.
• "я тебя обожаю" → "я тебя обожаю". Verbatim parrot.
• "ты умеешь шутить?" → "Да! Я умею шутить. Вот анекдот для тебя:". Ends on the colon.
• "молока купи" → "Такую команду я не знаю", while "добавь задачу починить кран" worked. Inverted word order defeats the list grammar.
• "закрой задачу про кран" → "не нашла такой задачи", with "починить кран" open and listed by the previous turn. Task lookup by keyword misses.
• "сколько задач я закрыл?" listed the five open ones instead of counting closed.
• "подведи итоги недели" was stored as a note.
• Recalled keys leak their storage form: "kran — починил", "ты любишь_go".
• English is unsupported in practice. "do you speak english?" → "пока не умею".

What works

• Stage 0 is fast and correct where it fires. Clock, day, list add, list read and an explicit-day reminder all answered in under 0.5s.
• Search returns real sourced answers in Russian and reads the book verbatim.
• Recall works once the value is stored as a fact: the wifi password and the tap came back two days later, correctly.
• The negative correction rung lands. "нет не так" → "поняла, отметила, что ответила не так", which is V-636 doing its job.
• Praxis names its own gap rather than guessing: "мне пока нечего смотреть — у Praxis нет источников."
• Hostility did not break her. "ты меня бесишь" got a calm reply, no persona collapse.
• No turn crashed and no turn timed out across 74 turns.

Suggested order of work

1. Set NTFY_TOKEN in deploy/telegram.env. One line, unblocks every reminder.
2. Clear pending clarify state on any turn that does not answer it, or expire it.
3. Route a clarify answer back into the pending slot instead of re-routing it.
4. Gate the weather, feed and personal query sources. Three of them claim on a similarity that is not there.
5. Re-check why "столица Франции?" routes to system. It is the documented canary.
6. The masculine self-reference stays the CPT's job. But "сохранил" appears on the most common write path, so a phrasing-level guard may be worth it first.