Merge pull request 'The router prompt has no destination, so the model arm of V-655 names nothing' (#201) from task/660-router-prompt-destination into master

CLAUDE.md

@@ -433,9 +433,25 @@ not a gap in the labelling.
 so the fixture scored a grammar set nobody runs. Fixed by V-659, worth 3 points of
 destination and nothing else. Check that function when adding a grammar.
 
-The model arm is still the follow-up. It lands on V-546. Intent, mood and BIO slot
-tags were already three heads on one forward pass of the resident e5-small.
-Destination is a fourth head on the same pass.
+**The model arm landed the same day** (V-660,
+`docs/evals/2026-08-08-destination-model-arm.md`). `routeGrammar` carries a
+`source` rule closed over `router.Sources` plus the empty floor, so the model
+cannot emit a destination that does not exist. The prompt lists the twelve in
+Russian and says `""` is a normal answer to give often. `LLMRouter.Route` reads it
+back through `ValidSource` and on `IntentQuery` alone. Against gemma-4-12b on the
+workstation the cascade scores destination **24/33 (72.7%)** with intent unmoved
+at 84.4%, and **recall goes 0/15 to 14/15**. The resident Qwen3-1.7B is
+unmeasured, because it binds `--port 0` inside the container.
+
+**Stage 0 now costs four destination points.** It did not before. The four cases
+the cascade loses and the model alone wins are all calendar. The possessive
+agenda rules claim them first and name nothing on purpose. That caution was free
+while nothing downstream could name anything either. It is not free now, and the
+fix is the owner's call rather than a quiet edit.
+
+The last arm is V-546. Intent, mood and BIO slot tags were already three heads on
+one forward pass of the resident e5-small. Destination is a fourth head on the
+same pass, and 72.7% from a 12B teacher is the label source for training it.
 
 ## LLM output contract
 

docs/evals/2026-08-08-destination-model-arm.md

@@ -0,0 +1,66 @@
+# The destination, with a model that can name one
+
+Measured 2026-08-08 against gemma-4-12b on the workstation, the same 96-case
+fixture V-659 built. Covers V-660.
+
+```sh
+no_proxy='*' MAVEN_LLM_URL=http://192.168.1.105:8080 \
+  make t PKG=./internal/router/eval/ RUN=TestLLMRouterBaseline V=1
+```
+
+## The gap was structural
+
+V-659 measured the destination at 12/33 on the classifier cascade, with recall
+at 0/15. Nothing in `routeSystem` named a `Source` and `routeGrammar` could not
+emit one, so the resident model had no string to write. That is the shape V-517
+measured for Praxis reach at 0/12: not a weak model, an absent contract.
+
+`routeGrammar` now carries a `source` rule closed over `router.Sources` plus the
+empty floor. The prompt lists the twelve destinations in Russian and says that
+`""` is a normal answer to give often.
+
+## Result
+
+| run | intent | destination |
+|---|---|---|
+| classifier + ONNX (V-659) | 73/96 (76.0%) | 12/33 (36.4%) |
+| gemma-4-12b alone | 79/96 intent-only (82.3%) | 26/33 (78.8%) |
+| cascade + gemma-4-12b + hash fallback | 81/96 (84.4%) | 24/33 (72.7%) |
+
+Recall is the move: 0/15 to 14/15. Intent did not shift, which was the
+constraint. The prompt is shared, so a destination rule that costs routing
+points is not a win.
+
+The eight llm-only errors are the eight `want_clarify` cases. The model returned
+`unknown` on every one, which is correct, and the llm-only harness surfaces a
+decline as an error by design.
+
+## Stage 0 now costs four destination points
+
+The four cases the cascade loses and the model alone wins are all calendar. The
+possessive agenda rules claim them at stage 0 and deliberately name nothing.
+"что у меня в списке покупок" matches the same rule. Naming the calendar there
+would take the list source off the turn (V-655).
+
+So a rule written to be careful about the list now blocks a model that would
+have named the calendar correctly. Before V-660 that caution was free, because
+nothing downstream of stage 0 could name anything either.
+
+Three ways out, and each costs something. Split the possessive rule so the
+calendar-shaped half names its destination. Let a later stage overwrite an empty
+destination a grammar left behind, which reverses "a matched value always wins".
+Or leave it, on the argument that four points is cheap next to a wrong
+destination on a shopping list. This wants the owner's call rather than a quiet
+edit.
+
+## What this does not measure
+
+The resident Qwen3-1.7B, which is what homesrv runs. It binds `--port 0` inside
+the container and no host process can reach it. Scoring it needs a second
+llama-server on a fixed port. The workstation is never assumed
+up, so the homesrv number is the one that decides whether this ships on by
+default.
+
+The fixture is 33 labelled destinations over twelve values. Recall carries 15 of
+them and five destinations carry none at all. A per-destination number below
+world, recall, calendar and the floor is not supported by this fixture.

internal/router/llmrouter.go

@@ -45,12 +45,19 @@ const routeGrammar = `
 root   ::= "[" ws action ("," ws action)* ws "]"
 action ::= "{" ws "\"intent\"" ws ":" ws intent ("," ws field)* ws "}"
 intent ::= "\"fact\"" | "\"reminder\"" | "\"note\"" | "\"query\"" | "\"act\"" | "\"chat\"" | "\"system\"" | "\"unknown\""
-field  ::= key ws ":" ws string
+field  ::= (key ws ":" ws string) | ("\"source\"" ws ":" ws source)
 key    ::= "\"key\"" | "\"value\"" | "\"text\"" | "\"verb\""
+source ::= "\"recall\"" | "\"calendar\"" | "\"tasks\"" | "\"list\"" | "\"money\"" | "\"weather\"" | "\"home\"" | "\"network\"" | "\"feeds\"" | "\"attention\"" | "\"self\"" | "\"world\"" | "\"\""
 string ::= "\"" ([^"\\\x00-\x1F] | "\\" ["\\/bfnrt] | "\\u" [0-9a-fA-F]{4}){0,120} "\""
 ws     ::= [ \t\n]{0,4}
 `
 
+// TestRouteGrammarCoversSources holds the source rule above to router.Sources.
+// The enum is the point: a grammar cannot emit a destination that does not
+// exist, which is the guarantee V-546 wants from a softmax and gets here for
+// free. Empty is the thirteenth alternative and it is not an oversight — it is
+// the SourceUnknown floor, and the model must be able to decline.
+
 // routeSystem — the router prompt. Changed 31-07-2026: the query test now sits
 // above the fact test and there is an explicit question test. Before that, a
 // question naming a fact key ("сколько воды я выпил с утра") matched the fact
@@ -121,6 +128,31 @@ const routeSystem = `Классифицируй ровно одно сообще
 "что такое кватернион?" → {"intent":"query","text":"что такое кватернион"}
 "ага" → {"intent":"chat","text":"ага"}
 
+Только для query добавь поле source — где лежит ответ:
+- recall — его заметки, факты и то, что он раньше говорил
+- calendar — встречи и события
+- tasks — список задач
+- list — списки покупок и другие именованные списки
+- money — траты
+- weather — погода
+- home — свет, устройства, дом
+- network — локальная сеть, сервер, диски
+- feeds — новостные ленты
+- attention — что требует внимания сейчас
+- self — вопрос про самого ассистента
+- world — всё остальное: определения, счёт, люди, факты о мире
+
+Пустое значение "" — нормальный ответ и его надо ставить часто. Ставь "", если ответ могут дать сразу два источника или если не уверен: тогда проверяются все по порядку, и это правильно. Никогда не угадывай.
+
+"сколько воды я выпил с утра" → {"intent":"query","text":"сколько воды я выпил с утра","source":"recall"}
+"что я записывал про кота" → {"intent":"query","text":"что я записывал про кота","source":"recall"}
+"во сколько у меня встреча" → {"intent":"query","text":"во сколько у меня встреча","source":"calendar"}
+"что такое docker?" → {"intent":"query","text":"что такое docker","source":"world"}
+"кто такой Линус Торвальдс?" → {"intent":"query","text":"кто такой Линус Торвальдс","source":"world"}
+"сколько будет 17 на 23?" → {"intent":"query","text":"сколько будет 17 на 23","source":"world"}
+"почему сервер тормозит" → {"intent":"query","text":"почему сервер тормозит","source":""}
+"есть новости по бэкапу базы" → {"intent":"query","text":"есть новости по бэкапу базы","source":""}
+
 Ответ — JSON-массив: по одному объекту на каждую просьбу. Обычно один. Если в реплике несколько просьб — по объекту на каждую. "напомни купить молоко, и запиши что кофе кончился" → [{"intent":"reminder","text":"купить молоко"},{"intent":"note","text":"кофе кончился"}]. Только JSON, без пояснений.`
 
 // routeRepeatPenalty — the sub-1B model loops one sentence inside the text field
@@ -172,6 +204,7 @@ type routeAction struct {
 	Value  string `json:"value"`
 	Text   string `json:"text"`
 	Verb   string `json:"verb"`
+	Source string `json:"source"`
 }
 
 // Route asks the model for one decision. The bool is false when there is no
@@ -240,6 +273,14 @@ func (lr *LLMRouter) Route(ctx context.Context, utterance string, now time.Time)
 	case IntentQuery:
 		d.Intent = IntentQuery
 		d.Slots.Text = firstNonEmpty(a.Text, utterance)
+		// Through ValidSource, and on query alone. The grammar already bounds
+		// the enum, but the grammar is a request to a server that may be
+		// running a different build, and a destination this binary does not
+		// know would take real query sources off the turn. Anything unknown
+		// drops to SourceUnknown, which is the floor and costs nothing.
+		if ValidSource(Source(a.Source)) {
+			d.Source = Source(a.Source)
+		}
 	case IntentAct:
 		d.Intent = IntentAct
 		d.Slots.Text = firstNonEmpty(a.Verb, utterance)

internal/router/llmrouter_test.go

@@ -398,3 +398,70 @@ func TestLLMReminderWithSubjectIsNotGated(t *testing.T) {
 		t.Fatalf("a complete reminder was sent back as a question: %+v", d.Slots)
 	}
 }
+
+// TestRouteGrammarCoversSources — the grammar enum and router.Sources are two
+// hand-written lists of the same twelve destinations, and nothing else notices
+// when one grows. A destination missing from the grammar is a destination the
+// model is structurally unable to name, which is the exact defect V-517
+// measured for Praxis: not a weak model, an absent string.
+func TestRouteGrammarCoversSources(t *testing.T) {
+	for _, s := range Sources {
+		if !strings.Contains(routeGrammar, `"\"`+string(s)+`\""`) {
+			t.Errorf("routeGrammar cannot emit %q — the model can never name it", s)
+		}
+	}
+	// The floor has to be reachable too, or the model is forced to pick one.
+	if !strings.Contains(routeGrammar, `"\"\""`) {
+		t.Error(`routeGrammar cannot emit "" — the model cannot decline a destination`)
+	}
+	// Count the alternatives on the source rule: an extra one is a destination
+	// the daemon would drop to SourceUnknown after the model spent tokens on it.
+	for _, line := range strings.Split(routeGrammar, "\n") {
+		if !strings.HasPrefix(line, "source ") {
+			continue
+		}
+		if got, want := strings.Count(line, "|")+1, len(Sources)+1; got != want {
+			t.Errorf("source rule has %d alternatives, want %d (Sources plus the floor)", got, want)
+		}
+	}
+}
+
+// The destination is read back only through ValidSource. A model on an older or
+// newer build can write a string this binary does not know, and trusting it
+// would take real query sources off the turn for a name nothing answers.
+func TestLLMUnknownSourceFallsToTheFloor(t *testing.T) {
+	r := newLLMTestRouter(t, `{"intent":"query","text":"что там с бэкапами","source":"praxis"}`)
+	d, err := r.Route(context.Background(), "что там с бэкапами", refNow())
+	if err != nil {
+		t.Fatalf("route: %v", err)
+	}
+	if d.Source != SourceUnknown {
+		t.Fatalf("invented destination %q was trusted, want the floor", d.Source)
+	}
+}
+
+// And a known one survives, or the read-back is just a filter.
+func TestLLMNamedSourceSurvives(t *testing.T) {
+	r := newLLMTestRouter(t, `{"intent":"query","text":"кто такой Линус Торвальдс","source":"world"}`)
+	d, err := r.Route(context.Background(), "кто такой Линус Торвальдс?", refNow())
+	if err != nil {
+		t.Fatalf("route: %v", err)
+	}
+	if d.Source != SourceWorld {
+		t.Fatalf("source %q, want %q", d.Source, SourceWorld)
+	}
+}
+
+// A destination on anything but a query is dropped. Only IntentQuery reaches
+// queryWalk, so a source elsewhere is a field nobody reads and a claim nobody
+// checks.
+func TestLLMSourceIsQueryOnly(t *testing.T) {
+	r := newLLMTestRouter(t, `{"intent":"note","text":"кофе кончился","source":"recall"}`)
+	d, err := r.Route(context.Background(), "запиши что кофе кончился", refNow())
+	if err != nil {
+		t.Fatalf("route: %v", err)
+	}
+	if d.Source != SourceUnknown {
+		t.Fatalf("a note carried destination %q", d.Source)
+	}
+}