A scenario is a JSON file under cmd/mavend/testdata/scenarios: a start
instant, a script of canned model answers, and a list of steps at "HH:MM".
Each step does one thing — say, audio, signal, arrive, tick, fault — and
then asserts on what she said, what was sent, which ecosystem services were
called, and what landed in the intake journal.
Between those boundaries the real components run: the real router cascade
(stage0, the LLM router over a scripted completer, the classifier
underneath it), the real store, the real reactive handler, the real tick
loop, and the same intake-decorated ipc.CoreAPI the daemon wires. What is
faked is only what a test cannot have: the model, the microphone, the
speaker, the delivery sink, and the ecosystem HTTP services.
Time is a single fakeClock threaded into every reader — the handler, the
intake publish stamp and tick(ctx, now) — so there is no time.Now() on the
replay path and a scenario is reproducible. TestSimulatorIsDeterministic
enforces that by replaying twice and diffing the transcripts byte for byte;
advanceTo refuses a step that goes backwards.
Two scenarios ship. morning_missed replays #284's own description: he
appears at the desk, a feed item, a mail candidate and a relayed
notification arrive through the morning, two ticks pass, and the assertions
are as much about nothing being sent at him unprompted as about what she
said. evening_degraded picks up the tier-2 pipeline case #288 deferred
here — a golden WAV through the STT seam to a written fact — and then puts
the ecosystem into 503 and checks that the proactive loop stays quiet and
that intake keeps working without it.
This is test-only code. Nothing in the production binaries changed, so the
daemon behaves identically when no scenario is running.
`make simulate` runs them verbose so the transcript is readable; `make
test` runs them with everything else.
Vikunja #284
The review's verification plan needs an instrument to settle whether the
classifier or the LLM router handles RU queries better, but that comparison
only means something if the daemon's own safety invariants are pinned
independently first.
These scenarios are deliberately narrow. They consume already-normalized
router decisions and assert what the post-router daemon owns: that a decision
requiring confirmation cannot execute before it is confirmed, that an
unresolved entity is never guessed at, and that named capabilities stay
unexecuted. Model routing quality is a separate question, evaluated against a
held-out contract fixture — mixing the two would produce a suite that fails for
two unrelated reasons.
The fixture is versioned (schema_version) so scenarios can be added without
rewriting the loader.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5JApcrCRVGmqrxnhynSik
kami