Add a replayable full-system simulator on a fake clock (#284)
A scenario is a JSON file under cmd/mavend/testdata/scenarios: a start instant, a script of canned model answers, and a list of steps at "HH:MM". Each step does one thing — say, audio, signal, arrive, tick, fault — and then asserts on what she said, what was sent, which ecosystem services were called, and what landed in the intake journal. Between those boundaries the real components run: the real router cascade (stage0, the LLM router over a scripted completer, the classifier underneath it), the real store, the real reactive handler, the real tick loop, and the same intake-decorated ipc.CoreAPI the daemon wires. What is faked is only what a test cannot have: the model, the microphone, the speaker, the delivery sink, and the ecosystem HTTP services. Time is a single fakeClock threaded into every reader — the handler, the intake publish stamp and tick(ctx, now) — so there is no time.Now() on the replay path and a scenario is reproducible. TestSimulatorIsDeterministic enforces that by replaying twice and diffing the transcripts byte for byte; advanceTo refuses a step that goes backwards. Two scenarios ship. morning_missed replays #284's own description: he appears at the desk, a feed item, a mail candidate and a relayed notification arrive through the morning, two ticks pass, and the assertions are as much about nothing being sent at him unprompted as about what she said. evening_degraded picks up the tier-2 pipeline case #288 deferred here — a golden WAV through the STT seam to a written fact — and then puts the ecosystem into 503 and checks that the proactive loop stays quiet and that intake keeps working without it. This is test-only code. Nothing in the production binaries changed, so the daemon behaves identically when no scenario is running. `make simulate` runs them verbose so the transcript is readable; `make test` runs them with everything else. Vikunja #284
This commit is contained in:
@@ -16,7 +16,7 @@ PIPER_BIN := $(shell pwd)/deps/piper/piper
|
||||
PIPER_MODEL := $(shell pwd)/models/tts/ru_RU-irina-medium.onnx
|
||||
PIPER_ESPEAK := $(shell pwd)/deps/piper/espeak-ng-data
|
||||
|
||||
.PHONY: stt-fixtures test-stt-golden all build build-stt build-tts build-daemon build-client build-waked build-web build-poll build-caldav clean test fmt-check vet run-stt run-tts run-web download-embedder deps-go eval-router eval-recall eval-phrasing eval-models
|
||||
.PHONY: simulate stt-fixtures test-stt-golden all build build-stt build-tts build-daemon build-client build-waked build-web build-poll build-caldav clean test fmt-check vet run-stt run-tts run-web download-embedder deps-go eval-router eval-recall eval-phrasing eval-models
|
||||
|
||||
all: build
|
||||
|
||||
@@ -91,6 +91,14 @@ vet:
|
||||
CGO_CFLAGS="$(CGO_CFLAGS)" CGO_LDFLAGS="$(CGO_LDFLAGS)" LD_LIBRARY_PATH="$(shell pwd)/deps/lib" \
|
||||
$(GO) vet ./internal/... ./cmd/...
|
||||
|
||||
# simulate — replay every scripted day under cmd/mavend/testdata/scenarios
|
||||
# through the real router, store, tick loop and intake journal, on a fake clock
|
||||
# (Vikunja #284). Verbose so the transcript of each scenario lands in the
|
||||
# terminal. Also runs as part of `make test`; this target is for reading it.
|
||||
simulate:
|
||||
CGO_CFLAGS="$(CGO_CFLAGS)" CGO_LDFLAGS="$(CGO_LDFLAGS)" LD_LIBRARY_PATH="$(shell pwd)/deps/lib" \
|
||||
$(GO) test -v -count=1 -run TestSimulator ./cmd/mavend/
|
||||
|
||||
test: fmt-check vet
|
||||
CGO_CFLAGS="$(CGO_CFLAGS)" CGO_LDFLAGS="$(CGO_LDFLAGS)" LD_LIBRARY_PATH="$(shell pwd)/deps/lib" \
|
||||
$(GO) test -race -coverprofile=coverage.out ./internal/... ./cmd/...
|
||||
|
||||
Reference in New Issue
Block a user