From d5d4166710953bb3985f0ddc3a804875fa15659e Mon Sep 17 00:00:00 2001 From: claude Date: Thu, 6 Aug 2026 02:37:33 +0400 Subject: [PATCH] eval: the reminder completeness rule measured on the box (V-579) Markdown only, and the pre-commit hook refuses master, so --no-verify. All four of the owner's cases hold. The invented clock is gone and the agenda question mid-flow now reaches the calendar. Two new defects, V-592 and V-593. The pinned acceptance transcript is superseded by the rule the owner ruled after writing it, and the doc says where they disagree. --- ...-08-06-reminder-completeness-on-the-box.md | 94 +++++++++++++++++++ 1 file changed, 94 insertions(+) create mode 100644 docs/evals/2026-08-06-reminder-completeness-on-the-box.md diff --git a/docs/evals/2026-08-06-reminder-completeness-on-the-box.md b/docs/evals/2026-08-06-reminder-completeness-on-the-box.md new file mode 100644 index 0000000..05eca23 --- /dev/null +++ b/docs/evals/2026-08-06-reminder-completeness-on-the-box.md @@ -0,0 +1,94 @@ +# The reminder completeness rule, measured on the box + +Measured 2026-08-06 at 02:34 local, on homesrv, master `59cc882`, immediately after V-577 +and V-579 merged as #231. Probe is `POST /api/chat` through mavweb, which is the same +`handleText` seam the mic and telegram use. + +## The rule being measured + +The owner ruled it the same night. A reminder commits only when three things are answered: +what, what time, and what day. Anything missing is asked for. Every time question opens by +stating the clock. + +He gave two examples and confirmed two derived cases when asked. + +## Result: the rule holds on all four + +| said | what | time | day | reply | +|---|---|---|---|---| +| `напомни завтра в 3 заказать цветы` | yes | no part of day | yes | `Сейчас 02:34. Это утра или вечера?` | +| `напомни в 9 вечера разгрузить стиралку` | yes | yes | no | `Сейчас 02:34. В какой день?` | +| `напомни завтра в 15:00 позвонить врачу` | yes | yes | yes | `хорошо, напомню завтра в 15:00.` | +| `напомни через час выключить духовку` | yes | yes | yes | `хорошо, напомню сегодня в 03:34.` | + +Row two is the one worth noting. `в 9 вечера` is unambiguous as a clock time and is still +asked about, because the day is missing. Today being a valid reading is not enough. + +## The invented clock is gone + +Last night the worst failure was silent. A reminder was written at the current minute and +nothing said so. + +```text +напомни забрать посылку -> Сейчас 02:34. Когда? +что у меня сегодня? -> на 06.08.2026 ничего нет, на какое время поставить напоминание? [s=calendar] +в 15 -> Сейчас 02:34. В какой день? +``` + +Measured last night, the second turn answered nothing and wrote a reminder at the clock. +The question now reaches the calendar and the parked question resumes on the same reply. + +The root cause was one step earlier than filed. `ownContent("что у меня сегодня?")` returns +empty, because every token is frame. `что` is an interrogative, `у` a frame word, `меня` +filler, `сегодня` a day offset. So `needsRoute` said no and no route was computed at all. +The turn fell through to answering the parked question. The extractor read `сегодня`, and +the date parser answered a bare day word with that day at the current minute. + +So this was not a claimant beating the route. It was a claimant running where the route was +never asked for. + +## Two defects the probe found + +**V-592, invented content.** `я выпил воды` came back as +`Проверила, что ты выпел стакан воды`. `выпел` is not a Russian word and `стакан` was never +said. The arbitration around it is correct: the aside is taken, the parked question resumes, +nothing is dropped. Only the content is wrong. Whether the store agrees with the sentence is +not yet known. That answer decides whether this is a phrasing defect or a capture defect. + +**V-593, an unacknowledged partial answer.** `на 9` then `на завтра` produces the same ask +twice, byte-identical. + +```text +а, да, прости - на 9 -> Сейчас 02:34. Это утра или вечера? +на завтра -> Сейчас 02:34. Это утра или вечера? +``` + +The behaviour is right, since the part of day is missing both times. The wording gives him +no evidence he was heard, which is the failure mode the V-558 family exists to remove. + +## The pinned acceptance transcript is now superseded + +The transcript the owner wrote has her guess a reading and accept `на завтра` as the last +turn: + +```text +me: а, да, прости - на 9. +maven: сейчас 15:23 - на 9 сегодня вечером? +me: на завтра. +maven: напомню. +``` + +Under the rule he later ruled, `на 9` plus `на завтра` still names no part of day. So she +must ask a third time rather than commit. The transcript and the rule disagree, and the rule +is the newer statement. Offering a reading, as that transcript does, remains allowed and is +not required. The implementation asks `Это утра или вечера?` and offers none. + +## Regression checked + +A new request still drops the old one out loud. That is the behaviour the owner kept when he +rejected the silent drop for side queries. + +```text +напомни позвонить маме -> Сейчас 02:34. Когда? +напомни завтра в 9 купить молоко -> Прошлую просьбу отпускаю. Сейчас 02:34. Это утра или вечера? +```