claude
8d2c1b6f99
Item 4. actionChat calls h.phraser.PhraseChat, and LLMPhraser posts raw
HTTP to /v1/chat/completions rather than going through the llm client
scriptedLLM stands in for. The simulator wired phraser.NewStub() anyway,
so no scenario could assert what she says on a chat turn: every reply came
back as a pick from fallbacks_ru_v1.json, four variants deep, and the same
scenario returned "тут я пас." one run and "не знаю, честно." the next.
scriptedPhraser embeds the Stub and overrides PhraseChat only, reading the
same script entries the router reads. A reply is accepted in either shape
the phrasing contract allows, the {"response","mood"} object or plain text,
so a scenario writes one thing for both paths.
An unscripted chat turn returns an error rather than a fallback, matching
scriptedLLM: actionChat logs it and uses ChatFallback(), so scenarios that
never meant to assert a chat reply behave as before.
conversation_anaphora turn 4 now pins its text — the reply that asks which
device he means, which is the recorded defect in the box's own words.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
80 lines
6.5 KiB
JSON
80 lines
6.5 KiB
JSON
{
|
||
"schema_version": 1,
|
||
"name": "conversation_anaphora",
|
||
"description": "Five consecutive Russian turns about one object, replayed from the run that found V-542 on the box on 05-08-2026. He names a monitor, then asks four questions that all say \"он\" and never name it again.\n\nThis scenario exists because the shape had nowhere to fail. The routing fixture scores one utterance at a time, so a conversation that breaks on its second turn cannot lose a point there, and V-44 step 2 could only be verified by hand. That is item 3 of V-542.\n\nFour of the five replies below are WRONG, and the assertions pin them anyway. Read them as the recorded defect rather than the contract: she has the last four turns in front of her and never once names the thing he is asking about. Every wrong assertion is marked in its step note with what it must become. When V-542 lands, those flip and the ones marked correct do not move.\n\nWhat the four assert is that the reply LACKS \"монитор\". Absence is the defect itself: she is answering a question about a thing she wrote down two minutes ago and cannot name it. It also survives the fallback picker, which matters on the three query turns — they refuse from internal/phraser/fallbacks_ru_v1.json, four variants deep, and the same scenario returned \"тут я пас.\" one run and \"не знаю, честно.\" the next, so a string assertion there would pin the picker rather than the daemon.\n\nTurn 4 asserts its text as well, because that turn goes through the chat path and the chat path is now scriptable. scriptedPhraser in simulator_test.go answers PhraseChat from the same script entries the router reads (V-542 item 4); before it, the simulator wired phraser.NewStub() and no scenario could say what she SAYS on a chat turn at all.\n\nThe routes are scripted exactly as the box produced them, because the failure is not the model's. Turn 1 went to fact despite \"давай поболтаем\", every question after it went to query, and turn 4 went to chat. A scripted route is what lets this scenario pin the daemon's half without a llama-server in the loop.",
|
||
"start": "2026-08-05T14:00:00+03:00",
|
||
"script": [
|
||
{
|
||
"match": "купил новый монитор",
|
||
"route": "[{\"intent\":\"fact\",\"key\":\"purchase\",\"value\":\"новый монитор\"}]",
|
||
"reply": "{\"response\":\"записала: новый монитор.\",\"mood\":\"neutral\"}"
|
||
},
|
||
{
|
||
"match": "он большой",
|
||
"route": "[{\"intent\":\"query\",\"text\":\"а он большой?\"}]"
|
||
},
|
||
{
|
||
"match": "сколько он примерно стоит",
|
||
"route": "[{\"intent\":\"query\",\"text\":\"сколько он примерно стоит по-твоему?\"}]"
|
||
},
|
||
{
|
||
"match": "переплатил",
|
||
"route": "[{\"intent\":\"chat\",\"text\":\"мне кажется я переплатил\"}]",
|
||
"reply": "{\"response\":\"я не знаю, о каком именно устройстве ты говоришь.\",\"mood\":\"neutral\"}"
|
||
},
|
||
{
|
||
"match": "стоит его вернуть",
|
||
"route": "[{\"intent\":\"query\",\"text\":\"стоит его вернуть?\"}]"
|
||
},
|
||
{
|
||
"match": "",
|
||
"route": "[{\"intent\":\"chat\",\"text\":\"\"}]",
|
||
"reply": "{\"response\":\"я рада тебя слышать.\",\"mood\":\"happy\"}"
|
||
}
|
||
],
|
||
"steps": [
|
||
{
|
||
"at": "14:00",
|
||
"note": "CORRECT, and it is the first half of the defect. \"давай поболтаем\" is an explicit request to converse and the turn is filed as a fact anyway. Storing what he said is not wrong on its own — he did buy a monitor — but the object then lives in the fact store and never enters the transcript PhraseChat reads. That is V-542 decision 2: either the marker claims the turn at stage 0, or it means nothing and comes out of the fixture.",
|
||
"say": "давай поболтаем: я вчера купил новый монитор",
|
||
"expect_events": ["purchase"],
|
||
"expect_no_send": true
|
||
},
|
||
{
|
||
"at": "14:01",
|
||
"note": "WRONG. \"он\" is the monitor from one turn ago, and she says she has no record of it. followUpMerge inherits prev.Slots.Key, and a query turn asking about a pronoun has no key to merge, so the question reaches the query sources naked and the notes source answers the only way it can. Must become: an answer about the monitor, or a route to chat where the transcript is.",
|
||
"say": "а он большой?",
|
||
"expect_reply_lacks": ["монитор"],
|
||
"expect_no_send": true
|
||
},
|
||
{
|
||
"at": "14:02",
|
||
"note": "WRONG, and it rules out one explanation. This is not the previous turn failing to stick — it is the same wall a second time, two turns from where the monitor was named. Nothing accumulates across query turns.",
|
||
"say": "сколько он примерно стоит по-твоему?",
|
||
"expect_reply_lacks": ["монитор"],
|
||
"expect_no_send": true
|
||
},
|
||
{
|
||
"at": "14:03",
|
||
"note": "WRONG, and it is the same wall from the other side. This turn routed chat, so it HAD the history that Session.History holds, and it asks which device he means anyway — because turn 1's object went to the fact store rather than the transcript. So a source reading the conversation is not sufficient on its own; decision 1 has to say which store the referent comes from. This is the one step whose text is pinned: the reply is scripted and reaches PhraseChat, so it is the box's own words rather than a fallback pick. Must become: a reply that names the monitor.",
|
||
"say": "мне кажется я переплатил",
|
||
"expect_reply_contains": ["о каком именно устройстве"],
|
||
"expect_reply_lacks": ["монитор"],
|
||
"expect_no_send": true
|
||
},
|
||
{
|
||
"at": "14:04",
|
||
"note": "WRONG. The fifth turn is the one that shows the cost. A returns question about a purchase two minutes old is answered with \"не нашла у тебя такой записи\", which is wrong in kind rather than merely unhelpful: the record exists, she wrote it herself at 14:00 under the key purchase.",
|
||
"say": "стоит его вернуть?",
|
||
"expect_reply_lacks": ["монитор"],
|
||
"expect_no_send": true
|
||
},
|
||
{
|
||
"at": "14:05",
|
||
"note": "CORRECT, and it is the control. Nothing in five conversational turns was sent at him unprompted, and a tick with him mid-conversation stays silent. Whatever V-542 changes must not change this.",
|
||
"tick": true,
|
||
"expect_no_send": true
|
||
}
|
||
]
|
||
}
|