dialogue contract tests: the six traces that do not (V-563)
Each carries the correct expectation and is skipped with the task that will unskip it, because a weakened expectation would pin the bug as the contract. MAVEN_DIALOGUE_NO_SKIP=1 runs them. V-561: the owner's transcript, and the same shape in words the offline date parser reads — a side query drops the parked question instead of suspending it, so Rome is never answered and the reminder is never set. V-560: a cancel is scored as a failed answer and spends a retry; clarify pre-empts the repair marker, so no correction can be spoken mid-flow. V-562: a stage-0 reminder never meets the extractor, so a reminder said whole with its hour in it is still asked about; and finishClarified goes straight to applyAction, so a repaired decision that lands short answers with a parse error instead of asking.
This commit is contained in:
@@ -437,5 +437,120 @@ func dialogueTraces() []trace {
|
||||
},
|
||||
end: endState{tasks: []string{"купить молоко"}},
|
||||
},
|
||||
|
||||
// ---- rows below carry the CORRECT expectation and fail today ----
|
||||
|
||||
// The owner's target transcript, V-561. He asks for a reminder, she asks
|
||||
// when, he asks something else entirely, and then comes back to her
|
||||
// question. On the box this created a reminder at 00:12 and never
|
||||
// answered Rome; on the offline floor the side question is recognised as
|
||||
// its own request and the flow is dropped instead, so the wrong reminder
|
||||
// is not made and the right one is not either.
|
||||
//
|
||||
// Both are the same defect: there is no suspend and resume. The correct
|
||||
// shape is the middle turn answered on its own and the parked question
|
||||
// still standing, on the same attempt — a side query is not a failed
|
||||
// answer and must not spend a retry.
|
||||
//
|
||||
// Unskipping this needs more than V-561. "на 9" and "на завтра" are not
|
||||
// read by StubDateTimeParser, which is what the offline floor runs, so
|
||||
// the row below it is the same shape in words the floor can parse and is
|
||||
// the one to watch first.
|
||||
{
|
||||
name: "the owner's transcript from V-561",
|
||||
skip: "V-561: a parked question is not suspended for a side query and never resumes",
|
||||
turns: []turn{
|
||||
{say: "напомни позвонить маме", question: dialogue.SlotTime, attempt: 1,
|
||||
parked: &parkedWant{slot: dialogue.SlotTime, attempt: 1}},
|
||||
{say: "какая сейчас погода в Риме?",
|
||||
parked: &parkedWant{slot: dialogue.SlotTime, attempt: 1, carries: "маме"}},
|
||||
{say: "а, да, прости - на 9.",
|
||||
parked: &parkedWant{slot: dialogue.SlotTime, attempt: 1, carries: "маме"}},
|
||||
{say: "на завтра."},
|
||||
},
|
||||
end: endState{reminders: []reminderWant{{payload: "позвонить маме", fireAt: "2026-08-01 09:00"}}},
|
||||
},
|
||||
// The same shape said in words StubDateTimeParser reads, so this row
|
||||
// turns green on V-561 alone. Same three claims: Rome is answered, the
|
||||
// question survives the side query on the same attempt, and the answer
|
||||
// after it completes the reminder he actually asked for.
|
||||
{
|
||||
name: "nested question: a parked question, then one of his own",
|
||||
skip: "V-561: a side query drops the parked question instead of suspending it",
|
||||
turns: []turn{
|
||||
{say: "напомни позвонить маме", question: dialogue.SlotTime, attempt: 1,
|
||||
parked: &parkedWant{slot: dialogue.SlotTime, attempt: 1}},
|
||||
{say: "какая сейчас погода в Риме?",
|
||||
parked: &parkedWant{slot: dialogue.SlotTime, attempt: 1, carries: "маме"}},
|
||||
{say: "в 11:00", contains: []string{"11:00"}},
|
||||
},
|
||||
end: endState{reminders: []reminderWant{{payload: "позвонить маме", fireAt: "2026-07-31 11:00"}}},
|
||||
},
|
||||
// A cancel is one of the five turn roles V-560 names, and today it is
|
||||
// none of them: "неважно" fills no slot and carries no request of its
|
||||
// own, so it reads as a failed answer and spends a retry. Two turns
|
||||
// later she is still asking about a reminder he called off.
|
||||
//
|
||||
// The row asserts what is knowable — nothing armed, nothing written, and
|
||||
// not another question — rather than her wording for it, which is not
|
||||
// written yet and is not this task's to invent.
|
||||
{
|
||||
name: "cancel: a parked question, then never mind",
|
||||
skip: "V-560: a cancel is scored as a failed answer, not as a cancel",
|
||||
turns: []turn{
|
||||
{say: "напомни позвонить маме", question: dialogue.SlotTime, attempt: 1,
|
||||
parked: &parkedWant{slot: dialogue.SlotTime, attempt: 1}},
|
||||
{say: "неважно", noQuestion: true},
|
||||
},
|
||||
end: endState{},
|
||||
},
|
||||
// Order in runTurn is the whole arbitration (V-558), and this is what it
|
||||
// costs: the clarify answer is checked at step 3 and the repair marker at
|
||||
// step 4d, so while a question is parked no correction can be made. She
|
||||
// scores "нет, это была заметка" as a bad time answer and asks again.
|
||||
{
|
||||
name: "correction while a question is parked",
|
||||
skip: "V-560: clarify pre-empts the repair marker, so a correction cannot be spoken mid-flow",
|
||||
turns: []turn{
|
||||
{say: "добавь в задачи купить молоко", contains: []string{"купить молоко"}},
|
||||
{say: "напомни позвонить маме", question: dialogue.SlotTime, attempt: 1,
|
||||
parked: &parkedWant{slot: dialogue.SlotTime, attempt: 1}},
|
||||
{say: "нет, это был вопрос", contains: []string{"поняла, это вопрос"},
|
||||
parked: &parkedWant{slot: dialogue.SlotTime, attempt: 1, carries: "маме"}},
|
||||
},
|
||||
end: endState{tasks: []string{"купить молоко"}},
|
||||
},
|
||||
// A reminder said whole, in one breath, with the hour in it — and she
|
||||
// asks when. ReminderGrammar (stage0.go) builds its slots by hand and
|
||||
// never runs the extractor, so a stage-0 reminder carries no time
|
||||
// whatever the sentence says, and the clarify gate reads the gap as
|
||||
// real. It costs a turn on the commonest reminder shape there is.
|
||||
//
|
||||
// Hermetic despite the date parser: stage 0 calls no parser at all, so
|
||||
// this fails the same way with or without python dateparser installed.
|
||||
{
|
||||
name: "a reminder said whole is not asked about",
|
||||
skip: "V-562: a stage-0 decision never meets the extractor, so its slots are never validated",
|
||||
turns: []turn{
|
||||
{say: "напомни в 11:00 позвонить маме", contains: []string{"11:00"}, noQuestion: true},
|
||||
},
|
||||
end: endState{reminders: []reminderWant{{payload: "позвонить маме", fireAt: "2026-07-31 11:00"}}},
|
||||
},
|
||||
// The same gap on the repair path. A correction redoes the request
|
||||
// through finishClarified, which goes straight to applyAction — it never
|
||||
// passes the clarify gate — so a redo that lands short answers with the
|
||||
// parse error V-557 removed from the routing path: "не поняла, на когда
|
||||
// напомнить." She should ask, exactly as she does for a fresh reminder
|
||||
// with no time.
|
||||
{
|
||||
name: "a correction that lands short asks rather than failing",
|
||||
skip: "V-562: finishClarified skips the clarify gate, so a repaired decision is never checked for gaps",
|
||||
turns: []turn{
|
||||
{say: "добавь в задачи купить молоко", contains: []string{"купить молоко"}},
|
||||
{say: "нет, это было напоминание", contains: []string{"поняла, это напоминание"},
|
||||
parked: &parkedWant{slot: dialogue.SlotTime, attempt: 1}},
|
||||
},
|
||||
end: endState{tasks: []string{"купить молоко"}},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user