Records all four configurations on the 27-case talk fixture, three runs
each: no grammar, plus grammar, plus Russian prompts, plus the truncation
fix. Composite, per-path and per-check, with the reproduce command.
The short version is that the plumbing got fixed and the score barely
moved. Grammar was the real win. Russian prompts helped a little and cut
latency by 5x. The truncation fix was necessary and bought nothing.
Also writes down three things that are easy to lose:
- The truncation cause was the grammar's 400-character bound, not the
token cap. Measured at three caps, same 400 characters every time.
- Then I set the bound to 1000 against a 768-token cap and made it worse.
The two limits have to agree.
- One run is contaminated and marked void: I ran an agent against the same
llama-server, and the report still claimed zero errors while a third of
the fixture silently answered "не знаю.". That is #397 and it is worse
than filed — a busy server is indistinguishable from bad phrasing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CGeSZxh1DCtRxmFVSYVGvJ
kami