kami
46259b4571
Completes #319's comparison. Three configurations, because "the LLM router" was ambiguous: the model alone, the cascade #320 would actually ship (stage-0 grammar → model → classifier floor), and a thinking-off diagnostic. intent-only full RU missed-clarify p50 classifier+onnx 36.8% 36.8% 25/61 5/6 31ms llm-only (0.8B) 48.7% 23.7% 13/61 6/6 850ms cascade+llm (0.8B) 50.0% 32.9% 18/61 6/6 825ms On the question asked — does the resident model route better? — yes, 50.0% vs 36.8% intent accuracy. REARCH.md's premise holds. It costs 27x the latency (p50 825ms vs 31ms, max 3.1s), on the same llama-server the phraser needs, so it is a trade rather than a free win. Three things the numbers surface that the headline hides: query→fact x15 is the dominant failure, four times the classifier's x4 on the same axis. routeSystem's decision order puts "сообщает или обновляет состояние" (rule 3) above "хочет получить информацию" (rule 4), so any utterance naming a fact key matches the earlier rule and a question about past state reads as an assertion of it. A prompt fix, not a model limit. The LLM router cannot clarify: llmrouter.go hardcodes Confidence 1.0, so stage 3's gate can never fire on its decisions — 6/6 missed. With #359's finding that the classifier's gate is miscalibrated under ONNX, neither path currently refuses. Flipping #320 as-is removes the refusal lane. The gap between 50.0% intent and 32.9% full accuracy is entirely slots: the LLM path fills neither Fn nor Time (it returns Slots.Text for acts, and Extract never runs on an LLM decision). Also settles a hypothesis rather than leaving it in the air: thinking mode is a non-issue under a grammar (identical score), and the grammar's unbounded ("," ws action)* repetition that ran away in an isolated smoke test does not reproduce under the real prompt — 2 errors in 76, not 76. internal/llm deliberately does not grow a chat_template_kwargs field. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X5JApcrCRVGmqrxnhynSik