claude
c259ed6c73
Step 4 of the board build (docs/plans/15-board-surface.md). Naming a task instead of its position reached nothing: "закрой задачу купить молоко" routed act, found no allowlisted fn, and the gate asked "Что сделать?". The position path already worked through resolveCandidate, but only in the two turns after she read the list out. TaskStatusGrammar is the same shape TaskCaptureGrammar uses — matches broadly, decides in Build, no eighth intent — and fills the fn slot with task_status, which is neither a Hexis capability nor a Praxis one. Three conditions, all required: the board noun, so no ordinary sentence claims a turn; exactly one status class, since "готово, убери" names two and asking beats picking; and a status word matched as an imperative exactly or a stative by lemma. So a bare "готово" and a bare "закрой" are not this rule's, and the second belongs to Praxis, which claims it already. Two lexicon sets rather than one with a value. The store records which of the two transitions happened and /tasks shows it: work he chose to stop is not work he did. Measured on the fixture, two new cases (ru-act-020, ru-act-021). Classifier + ONNX 62/89 (69.7%) → 64/91 (70.3%); cascade+llm 67/89 (75.3%) → 69/91 (75.8%, 80.2% intent-only) at p50 1.225s. Both new cases claimed at stage 0, no case regressed, clarify counts unchanged at 3 false / 1 missed. The task's own warning stands: every such grammar runs its parser ahead of the resident model on every turn, so this is the last one that is free. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SoL7EBdYC5Mhz3DJd49GJy
112 lines
17 KiB
JSON
112 lines
17 KiB
JSON
{
|
|
"schema_version": 1,
|
|
"name": "ru_routing_v1",
|
|
"reference_now": "2026-07-30T12:00:00Z",
|
|
"notes": [
|
|
"Held-out routing contract. Every utterance here is absent from models/seeds/*.txt (TestFixtureIsHeldOut enforces it verbatim) — scoring a classifier on its own seed phrases measures memorisation, not routing.",
|
|
"This is a CONTRACT, not a snapshot of current behaviour. Cases the classifier cascade fails today are expected to stay in the file and fail loudly; that failure count is the number Vikunja #319 compares against the LLM router before #320 flips the default.",
|
|
"Slot expectations are deployment-independent on purpose. want_fn is a boolean (the act must resolve to SOME allowlisted fn) because the allowlist lives in deploy config, not here. want_fact_key names the loop's rule keys (water/meal/sleep/break/shower) — a fact that lands under the wrong key silently starves the predicate that reads it.",
|
|
"want_clarify cases carry intent \"\": the contract is that the router refuses rather than guesses. A confident answer there is a worse failure than a miss."
|
|
],
|
|
"cases": [
|
|
{ "id": "ru-query-001", "utterance": "сколько воды я выпил с утра", "lang": "ru", "intent": "query", "tags": ["aggregate"] },
|
|
{ "id": "ru-query-002", "utterance": "я сегодня вообще пил воду", "lang": "ru", "intent": "query", "tags": ["hard", "fact-shaped"], "note": "past-tense fact lexicon in a question — the classifier's fact centroid pulls this hard" },
|
|
{ "id": "ru-query-003", "utterance": "во сколько я лёг вчера", "lang": "ru", "intent": "query", "tags": ["temporal"] },
|
|
{ "id": "ru-query-004", "utterance": "давно я не тренировался", "lang": "ru", "intent": "query", "tags": ["hard", "no-question-word"] },
|
|
{ "id": "ru-query-005", "utterance": "напоминания на завтра есть", "lang": "ru", "intent": "query", "tags": ["hard", "reminder-shaped"], "note": "asks about reminders, does not create one" },
|
|
{ "id": "ru-query-006", "utterance": "что я записывал про кота", "lang": "ru", "intent": "query", "tags": ["recall"] },
|
|
{ "id": "ru-query-007", "utterance": "сколько раз я ел вчера", "lang": "ru", "intent": "query", "tags": ["aggregate", "hard"] },
|
|
{ "id": "ru-query-008", "utterance": "мой вес за последний месяц", "lang": "ru", "intent": "query", "tags": ["no-verb"] },
|
|
{ "id": "ru-query-009", "utterance": "когда я в последний раз принимал витамины", "lang": "ru", "intent": "query", "tags": ["temporal"] },
|
|
{ "id": "ru-query-010", "utterance": "есть новости по бэкапу базы", "lang": "ru", "intent": "query", "tags": ["homelab"] },
|
|
{ "id": "ru-query-011", "utterance": "почему сервер тормозит", "lang": "ru", "intent": "query", "tags": ["homelab", "hard"], "note": "diagnostic question, not a chat opener" },
|
|
{ "id": "ru-query-012", "utterance": "какие заметки я оставил про полив", "lang": "ru", "intent": "query", "tags": ["recall"] },
|
|
{ "id": "ru-query-013", "utterance": "во сколько у меня встреча", "lang": "ru", "intent": "query", "tags": ["calendar"] },
|
|
{ "id": "ru-query-019", "utterance": "что у меня стоит в календаре на послезавтра", "lang": "ru", "intent": "query", "tags": ["calendar", "hard"], "note": "agenda, not the clock: the daemon answers this from CalendarEvents inside the query branch, so the clock/date system rule must not swallow it" },
|
|
{ "id": "ru-query-022", "utterance": "какие планы на завтра?", "lang": "ru", "intent": "query", "tags": ["calendar"], "note": "the same agenda question as ru-query-019 aimed at another day; it answered \u043f\u043e\u043a\u0430 \u043d\u0435 \u0443\u043c\u0435\u044e on the deployed daemon while the today form worked (Vikunja #471)" },
|
|
{ "id": "ru-query-023", "utterance": "\u043a\u043e\u0433\u0434\u0430 \u043f\u043b\u0430\u043d\u0451\u0440\u043a\u0430?", "lang": "ru", "intent": "query", "tags": ["calendar", "hard"], "note": "a named event with no calendar word — the noun is the only signal that this is a question about his day" },
|
|
{ "id": "ru-query-024", "utterance": "что дальше?", "lang": "ru", "intent": "query", "tags": ["calendar", "no-question-word"], "note": "the rest of the day, with no possessive and no plan word to anchor on; the model called it a fact and the write had to be caught downstream (Vikunja #498)" },
|
|
{ "id": "ru-query-025", "utterance": "расскажи про битву при Ватерлоо", "lang": "ru", "intent": "query", "tags": ["world", "no-question-word"], "note": "a narrative request carries no question mark and no interrogative, so it routed fact; contrast ru-chat-003, where the same verb asks for a joke" },
|
|
{ "id": "ru-query-014", "utterance": "я успеваю до дедлайна", "lang": "ru", "intent": "query", "tags": ["hard", "no-question-word"] },
|
|
{ "id": "ru-query-015", "utterance": "сколько я прошёл шагов", "lang": "ru", "intent": "query", "tags": ["aggregate"] },
|
|
{ "id": "ru-query-016", "utterance": "покажи давление за неделю", "lang": "ru", "intent": "query", "tags": ["hard", "imperative"], "note": "imperative form but a read — must not route to act" },
|
|
{ "id": "ru-query-017", "utterance": "чем я занимался в среду", "lang": "ru", "intent": "query", "tags": ["hard", "chat-shaped"] },
|
|
{ "id": "ru-query-018", "utterance": "хватает ли места под новые бэкапы", "lang": "ru", "intent": "query", "tags": ["homelab"] },
|
|
{ "id": "ru-query-020", "utterance": "что дальше?", "lang": "ru", "intent": "query", "tags": ["agenda", "hard"], "note": "the rest of the day, with no interrogative the model can read as a question — it routed fact until a stage 0 rule claimed it (V-498)" },
|
|
{ "id": "ru-query-021", "utterance": "расскажи про битву при Ватерлоо", "lang": "ru", "intent": "query", "tags": ["world", "hard"], "note": "a world question phrased as an instruction. It routed fact, and the fact gate had to catch the write (V-498)" },
|
|
{ "id": "en-query-001", "utterance": "did I take my vitamins today", "lang": "en", "intent": "query", "tags": ["fact-shaped"] },
|
|
{ "id": "en-query-002", "utterance": "how long since the last backup finished", "lang": "en", "intent": "query", "tags": ["temporal"] },
|
|
{ "id": "en-query-003", "utterance": "show me this week's weight", "lang": "en", "intent": "query", "tags": ["imperative"] },
|
|
|
|
{ "id": "ru-fact-001", "utterance": "только что выпил кружку воды", "lang": "ru", "intent": "fact", "want_fact_key": "water" },
|
|
{ "id": "ru-fact-002", "utterance": "воды попил наконец", "lang": "ru", "intent": "fact", "want_fact_key": "water", "tags": ["inverted"] },
|
|
{ "id": "ru-fact-003", "utterance": "поужинал", "lang": "ru", "intent": "fact", "want_fact_key": "meal", "tags": ["single-word"] },
|
|
{ "id": "ru-fact-004", "utterance": "отметь что я позавтракал овсянкой", "lang": "ru", "intent": "fact", "want_fact_key": "meal" },
|
|
{ "id": "ru-fact-005", "utterance": "поспал часов пять", "lang": "ru", "intent": "fact", "want_fact_key": "sleep" },
|
|
{ "id": "ru-fact-006", "utterance": "сходил в душ", "lang": "ru", "intent": "fact", "want_fact_key": "shower" },
|
|
{ "id": "ru-fact-007", "utterance": "отдохнул минут двадцать", "lang": "ru", "intent": "fact", "want_fact_key": "break" },
|
|
{ "id": "ru-fact-008", "utterance": "запиши вес 74 килограмма", "lang": "ru", "intent": "fact", "tags": ["unparsed-key"], "note": "no recognizer for weight yet — intent must still be fact; HasKey false is allowed" },
|
|
{ "id": "ru-fact-009", "utterance": "отметь что я выпил таблетки утром", "lang": "ru", "intent": "fact", "tags": ["unparsed-key"] },
|
|
{ "id": "ru-fact-010", "utterance": "сделал зарядку двадцать минут", "lang": "ru", "intent": "fact", "tags": ["unparsed-key"] },
|
|
{ "id": "en-fact-001", "utterance": "just drank a glass of water", "lang": "en", "intent": "fact", "want_fact_key": "water" },
|
|
{ "id": "en-fact-002", "utterance": "slept about seven hours", "lang": "en", "intent": "fact", "want_fact_key": "sleep" },
|
|
|
|
{ "id": "ru-rem-001", "utterance": "напомни через сорок минут выключить свет", "lang": "ru", "intent": "reminder", "want_time": true },
|
|
{ "id": "ru-rem-002", "utterance": "напомни завтра в 7 позвонить в клинику", "lang": "ru", "intent": "reminder", "want_time": true },
|
|
{ "id": "ru-rem-003", "utterance": "не забудь напомнить мне про счёт за свет", "lang": "ru", "intent": "reminder", "tags": ["no-time"], "note": "no datetime — HasTime false is correct; the daemon asks for a time" },
|
|
{ "id": "ru-rem-004", "utterance": "поставь напоминание через полчаса", "lang": "ru", "intent": "reminder", "want_time": true },
|
|
{ "id": "ru-rem-005", "utterance": "разбуди меня в 6:30", "lang": "ru", "intent": "reminder", "want_time": true, "tags": ["hard"], "note": "wake-me phrasing, no напомни stem" },
|
|
{ "id": "ru-rem-006", "utterance": "напомни послезавтра в 12 забрать заказ", "lang": "ru", "intent": "reminder", "want_time": true },
|
|
{ "id": "ru-rem-007", "utterance": "через два часа напомни проверить бэкап", "lang": "ru", "intent": "reminder", "want_time": true, "tags": ["inverted"] },
|
|
{ "id": "ru-rem-008", "utterance": "напомни в половине восьмого выпить таблетку", "lang": "ru", "intent": "reminder", "want_time": true, "tags": ["hard"], "note": "half past, spoken as the ordinal of the hour being entered: 07:30, not 08:30 (V-538)" },
|
|
{ "id": "ru-rem-009", "utterance": "разбуди меня полвосьмого", "lang": "ru", "intent": "reminder", "want_time": true, "tags": ["hard"], "note": "the contracted half hour, and no preposition to hang the clock on" },
|
|
{ "id": "ru-rem-010", "utterance": "напомни без четверти восемь выходить", "lang": "ru", "intent": "reminder", "want_time": true, "tags": ["hard"], "note": "quarter to eight is 07:45, counted from a cardinal" },
|
|
{ "id": "en-rem-001", "utterance": "remind me in 45 minutes to stretch", "lang": "en", "intent": "reminder", "want_time": true },
|
|
{ "id": "en-rem-002", "utterance": "wake me at 6:15", "lang": "en", "intent": "reminder", "want_time": true, "tags": ["hard"] },
|
|
|
|
{ "id": "ru-act-001", "utterance": "перезапусти докер", "lang": "ru", "intent": "act", "want_fn": true, "tags": ["homelab", "destructive"] },
|
|
{ "id": "ru-act-002", "utterance": "выключи свет в спальне", "lang": "ru", "intent": "act", "want_fn": true },
|
|
{ "id": "ru-act-003", "utterance": "maven, останови бэкап", "lang": "ru", "intent": "act", "want_fn": true, "tags": ["wake-token"] },
|
|
{ "id": "ru-act-004", "utterance": "включи вытяжку", "lang": "ru", "intent": "act", "want_fn": true },
|
|
{ "id": "ru-act-005", "utterance": "запусти бэкап сейчас", "lang": "ru", "intent": "act", "want_fn": true },
|
|
{ "id": "ru-act-006", "utterance": "закрой жалюзи", "lang": "ru", "intent": "act", "want_fn": true },
|
|
{ "id": "ru-act-020", "utterance": "закрой задачу купить молоко", "lang": "ru", "intent": "act", "want_fn": true, "tags": ["task", "status"], "note": "a spoken status change over the board. Routed act with no allowlisted fn until V-512, so the gate asked \"Что сделать?\"; TaskStatusGrammar fills the fn slot with task_status and actionAct answers it from Maven's own store" },
|
|
{ "id": "ru-act-021", "utterance": "убери из задач оплатить интернет", "lang": "ru", "intent": "act", "want_fn": true, "tags": ["task", "status"], "note": "the drop half of the same rule. Dropped work he chose to stop is not work he did, so the two status sets are separate lexicons" },
|
|
{ "id": "en-act-001", "utterance": "maven, restart the media server", "lang": "en", "intent": "act", "want_fn": true, "tags": ["wake-token"] },
|
|
{ "id": "en-act-002", "utterance": "turn off the kitchen light", "lang": "en", "intent": "act", "want_fn": true },
|
|
|
|
{ "id": "ru-note-001", "utterance": "заметка: продлить домен в августе", "lang": "ru", "intent": "note" },
|
|
{ "id": "ru-note-002", "utterance": "запиши что кран на кухне снова капает", "lang": "ru", "intent": "note" },
|
|
{ "id": "ru-note-003", "utterance": "заметка про настройку vlan на свитче", "lang": "ru", "intent": "note", "tags": ["homelab"] },
|
|
{ "id": "ru-note-004", "utterance": "запиши идею: гидропоника на балконе", "lang": "ru", "intent": "note" },
|
|
{ "id": "ru-note-005", "utterance": "запиши что сосед просил номер электрика", "lang": "ru", "intent": "note" },
|
|
{ "id": "ru-note-006", "utterance": "добавь в задачи купить молоко", "lang": "ru", "intent": "note", "tags": ["capture"], "note": "an explicit capture marker — the model called it an act and rewrote the payload (Vikunja #467), stage 0 claims it" },
|
|
{ "id": "en-note-001", "utterance": "note: rotate the kuma api key", "lang": "en", "intent": "note", "tags": ["homelab"] },
|
|
|
|
{ "id": "ru-sys-001", "utterance": "сколько сейчас времени в киеве", "lang": "ru", "intent": "system", "tags": ["time"] },
|
|
{ "id": "ru-sys-002", "utterance": "какое число завтра", "lang": "ru", "intent": "system", "tags": ["date"] },
|
|
{ "id": "ru-sys-003", "utterance": "переходи в тихий режим", "lang": "ru", "intent": "system", "tags": ["quiet"] },
|
|
{ "id": "ru-sys-004", "utterance": "хватит тихого режима", "lang": "ru", "intent": "system", "tags": ["quiet", "hard"] },
|
|
{ "id": "ru-sys-005", "utterance": "какой день недели послезавтра", "lang": "ru", "intent": "system", "tags": ["date"] },
|
|
{ "id": "en-sys-001", "utterance": "what time is it now", "lang": "en", "intent": "system", "tags": ["time"] },
|
|
{ "id": "en-sys-002", "utterance": "turn quiet mode back on", "lang": "en", "intent": "system", "tags": ["quiet"] },
|
|
|
|
{ "id": "ru-chat-001", "utterance": "мне немного грустно сегодня", "lang": "ru", "intent": "chat" },
|
|
{ "id": "ru-chat-002", "utterance": "что думаешь про переезд на другую квартиру", "lang": "ru", "intent": "chat" },
|
|
{ "id": "ru-chat-003", "utterance": "расскажи анекдот про программистов", "lang": "ru", "intent": "chat" },
|
|
{ "id": "ru-chat-004", "utterance": "доброе утро", "lang": "ru", "intent": "chat", "tags": ["greeting"] },
|
|
{ "id": "ru-chat-005", "utterance": "спасибо тебе", "lang": "ru", "intent": "chat" },
|
|
{ "id": "en-chat-001", "utterance": "good morning", "lang": "en", "intent": "chat", "tags": ["greeting"] },
|
|
{ "id": "en-chat-002", "utterance": "i had a rough day", "lang": "en", "intent": "chat" },
|
|
|
|
{ "id": "amb-001", "utterance": "вода", "lang": "ru", "want_clarify": true, "tags": ["ambiguous"], "note": "fact write or a query? one noun decides nothing — ask" },
|
|
{ "id": "amb-002", "utterance": "бэкап", "lang": "ru", "want_clarify": true, "tags": ["ambiguous"], "note": "run it, or report on it? a confident act here is destructive" },
|
|
{ "id": "amb-003", "utterance": "ну это", "lang": "ru", "want_clarify": true, "tags": ["ambiguous", "filler"] },
|
|
{ "id": "amb-004", "utterance": "сделай это", "lang": "ru", "want_clarify": true, "tags": ["ambiguous", "anaphora"], "note": "unresolved anaphora with an imperative — must not guess an fn" },
|
|
{ "id": "amb-005", "utterance": "потом", "lang": "ru", "want_clarify": true, "tags": ["ambiguous", "filler"] },
|
|
{ "id": "amb-006", "utterance": "the thing from earlier", "lang": "en", "want_clarify": true, "tags": ["ambiguous", "anaphora"] },
|
|
{ "id": "amb-007", "utterance": "напомни", "lang": "ru", "want_clarify": true, "tags": ["ambiguous", "reminder"], "note": "the reminder verb and nothing else — she knows the shape of the request and not one thing about it. Answered 'не получилось разобрать время напоминания' on the box until V-548: the subjectless-reminder gate tested Slots.Text == \"\", and fillSlots had put the verb in that slot" },
|
|
{ "id": "amb-008", "utterance": "ну напомни же", "lang": "ru", "want_clarify": true, "tags": ["ambiguous", "reminder", "filler"], "note": "the same request wrapped in particles, which is why filler_particles is a lexicon set — without it the particles read as the subject" }
|
|
]
|
|
}
|