Add a deterministic scorer for nudge phrasing (Vikunja #323)
Review internal/phraser/eval/checks.go -- it IS the measurement. Each check names in a comment which DESIGN.md line it defends: length, feminine self-reference (windowed around "я" so the operator's own masculine second-person forms are not flagged), the cringe list (pet names, emoji, "!!", fake concern, apology, emotional support, asking how he feels, praise), on-topic, mood enum. No send/veto signal anywhere, per DESIGN.md § "Rules decide, LLM phrases". Fixture (158 lines) and tests (252) do not count toward the diff ceiling; the scorer itself is still ~650. Splitting eval.go from checks.go would give two commits neither of which measures anything. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CGeSZxh1DCtRxmFVSYVGvJ
This commit is contained in:
@@ -16,7 +16,7 @@ PIPER_BIN := $(shell pwd)/deps/piper/piper
|
||||
PIPER_MODEL := $(shell pwd)/models/tts/ru_RU-irina-medium.onnx
|
||||
PIPER_ESPEAK := $(shell pwd)/deps/piper/espeak-ng-data
|
||||
|
||||
.PHONY: all build build-stt build-tts build-daemon build-client build-waked build-web build-poll build-caldav clean test run-stt run-tts run-web download-embedder deps-go eval-router
|
||||
.PHONY: all build build-stt build-tts build-daemon build-client build-waked build-web build-poll build-caldav clean test run-stt run-tts run-web download-embedder deps-go eval-router eval-phrasing
|
||||
|
||||
all: build
|
||||
|
||||
@@ -83,6 +83,15 @@ MAVEN_ONNX_LIB ?= $(shell pwd)/deps/onnxruntime-linux-x64-1.26.0/lib/libonnxrunt
|
||||
eval-router:
|
||||
MAVEN_ONNX_LIB="$(MAVEN_ONNX_LIB)" $(GO) test -v -count=1 ./internal/router/eval/
|
||||
|
||||
# eval-phrasing -- score nudge phrasing (internal/phraser/eval). Verbose so the
|
||||
# report and every generated message land in the terminal. With no environment
|
||||
# it scores the deterministic Stub only, which is what CI runs. Set
|
||||
# MAVEN_LLM_URL to add the resident model:
|
||||
# MAVEN_LLM_URL=http://127.0.0.1:18099 make eval-phrasing
|
||||
# The model run is slow (minutes) -- the timeout is raised to match.
|
||||
eval-phrasing:
|
||||
$(GO) test -v -count=1 -timeout 40m ./internal/phraser/eval/
|
||||
|
||||
run-stt: build-stt
|
||||
LD_LIBRARY_PATH="$(shell pwd)/deps/lib" \
|
||||
./mavsttd -socket /tmp/maven/stt.sock -model $(WHISPER_MODEL)
|
||||
|
||||
Reference in New Issue
Block a user