kami/Maven
1
0
Fork 0
Code Issues 1 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Salvage two eval harnesses: nudge phrasing and per-model routing #13

Closed
claude wants to merge 4 commits from overnight/eval-harnesses into overnight/fmt-gate
pull from: overnight/eval-harnesses
merge into: kami:overnight/fmt-gate
Conversation 2 Commits 4 Files Changed 10
+1144 -6

4 Commits

Author SHA1 Message Date
kami 1c4eab2107 Merge commit '94eb92f' into overnight-jul31 
# Conflicts:
#	Makefile
 2026-07-31 10:11:56 +04:00
kami 0914e0a3d5 Merge commit '4ba9a6f' into overnight-jul31 
# Conflicts:
#	Makefile
 2026-07-31 10:11:29 +04:00
kami 4ba9a6f422 Add a deterministic scorer for nudge phrasing (Vikunja #323) 
Review internal/phraser/eval/checks.go -- it IS the measurement. Each check
names in a comment which DESIGN.md line it defends: length, feminine
self-reference (windowed around "я" so the operator's own masculine
second-person forms are not flagged), the cringe list (pet names, emoji,
"!!", fake concern, apology, emotional support, asking how he feels,
praise), on-topic, mood enum. No send/veto signal anywhere, per
DESIGN.md § "Rules decide, LLM phrases".
Fixture (158 lines) and tests (252) do not count toward the diff ceiling;
the scorer itself is still ~650. Splitting eval.go from checks.go would
give two commits neither of which measures anything.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CGeSZxh1DCtRxmFVSYVGvJ
 2026-07-31 02:30:52 +04:00
kami 94eb92fb15 Label LLM eval runs with the model llama-server has loaded 
The bake-off in #278/#250 needs two models' scores side by side, and the
report names only carried the config, so the rows were indistinguishable.
ModelID reads /v1/models instead of taking a string that goes stale.
New target: make eval-models MAVEN_LLM_URL=...

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CGeSZxh1DCtRxmFVSYVGvJ
 2026-07-31 02:17:20 +04:00