phrasing and routing: a raw newline is not JSON (V-537)

Sixty of the failures in the 2026-08-05 temperature sweep were one error,
`phraser: model output starts as JSON but does not parse`, all of them in the
reply family and two of them in all twelve runs. The write-up read that as
truncation. It is not: no run hit the token cap.

The string rule in both grammars was `[^"\\]`, which admits a literal
newline. A model that wants two lines writes one, the generation satisfies the
grammar, and json.Unmarshal then rejects it with "invalid character '\n' in
string literal". The object starts with "{", so it came back as errBrokenJSON
and the reply was an empty string. The router's rule also admitted `"\\" .`,
so \q satisfied it and failed to parse the same way.

Both string rules are now llama.cpp's own json.gbnf class: the control range is
out and the escape alternatives are exact. Verified against the resident model
on 8899 — llama-server accepts both grammars and both still emit what they did.

escapeRawControls is the second line, for NoGrammar and for a remote server that
ignores a grammar: a reply whose only fault is a raw newline is readable, so it
is read rather than dropped.
This commit is contained in:
2026-08-05 11:24:35 +04:00
parent fdc18edd87
commit 32d5f68710
5 changed files with 124 additions and 10 deletions
+9 -1
View File
@@ -38,9 +38,17 @@ func TestLLMRouterSetsRepeatPenalty(t *testing.T) {
// An unbounded string rule lets one field eat the whole token budget.
func TestRouteGrammarBoundsStrings(t *testing.T) {
if !strings.Contains(routeGrammar, `string ::= "\"" ([^"\\] | "\\" .){0,120} "\""`) {
if !strings.Contains(routeGrammar, `{0,120} "\""`) {
t.Fatal("grammar string rule lost its length bound")
}
// And it must not admit a raw newline or a made-up escape, either of which
// makes the route unparseable and costs the turn its router (Vikunja #537).
if !strings.Contains(routeGrammar, `[^"\\\x00-\x1F]`) {
t.Error("string rule admits raw control characters")
}
if strings.Contains(routeGrammar, `"\\" .`) {
t.Error(`string rule still admits "\\" . — \q satisfies the grammar and fails to parse`)
}
}
// A question naming a fact key used to be stored as a fact because the fact rule