router: a Russian verb is a whole sentence, not thin evidence

The clarify gate thinned any one-word utterance to 0.3 confidence, which
trips the stage-3 gate and comes back as "не совсем поняла". That is an
English intuition. Russian packs subject, tense and gender into one word,
so "поужинал" is a complete report and "привет" a complete greeting, and
both got clarified.

thinSingleToken keeps the rule for bare nominals, where it is real ("вода"
is a fact-or-query coin flip), and spares two classes: a closed lexicon of
social and control singles, and any token carrying a verb ending. Both
tests are offline.

Fixture: false clarifies 3 → 2, intent-only 74.0% → 75.3%, full accuracy
unchanged at 70.1%, missed clarify still 1.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TrVSBKe3RFDF4fGYKWYQnX
This commit is contained in:
kami
2026-08-01 21:29:21 +04:00
parent 47dda97226
commit 29329b5f0e
4 changed files with 138 additions and 6 deletions
+10 -2
View File
@@ -102,8 +102,16 @@ Re-measured on the fixture after the fix: **missed clarify 6/6 → 1**, at the c
clarifies and 2.6pt of full accuracy (72.7% → 70.1%, intent-only 67.5% → 74.0%). Two of the clarifies and 2.6pt of full accuracy (72.7% → 70.1%, intent-only 67.5% → 74.0%). Two of the
three false clarifies are acts the model mis-routed and the gate caught — asking beats wrongly three false clarifies are acts the model mis-routed and the gate caught — asking beats wrongly
executing, so the fixture and the daemon disagree about what is correct there. The third, executing, so the fixture and the daemon disagree about what is correct there. The third,
`"поужинал"`, is a real defect: **the single-token rule is an English intuition and does not `"поужинал"`, was a real defect: the single-token rule was an English intuition and does not
transfer to Russian**, where one word is routinely a whole sentence. Narrow or drop it. transfer to Russian, where one word is routinely a whole sentence.
Narrowed 01-08-2026. `thinSingleToken` (`internal/router/singletoken.go`) still thins a bare
one-word nominal — "вода", "бэкап" — but spares two classes: a closed lexicon of social and
control singles ("привет", "спасибо", "стоп", "yes"), and any token carrying a Russian verb
ending (past tense, 2nd person, reflexive), because a verb already contains its subject. Both
tests are offline and cost nothing. Re-measured: **false clarifies 3 → 2, intent-only 74.0% →
75.3%, full accuracy unchanged at 70.1%, missed clarify still 1.** The two remaining false
clarifies are the act-with-no-allowlisted-fn arm of the gate, not this rule.
## LLM output contract ## LLM output contract
+5 -4
View File
@@ -194,12 +194,13 @@ func (lr *LLMRouter) Route(ctx context.Context, utterance string, now time.Time)
return Decision{}, false, nil return Decision{}, false, nil
} }
d := Decision{Utterance: utterance, Stage: 1, Confidence: llmFullConfidence} d := Decision{Utterance: utterance, Stage: 1, Confidence: llmFullConfidence}
// A single-token utterance is thin evidence: the model had nothing to // A bare one-word nominal is thin evidence: the model had nothing to
// disambiguate on ("вода" is a fact-or-query coin flip, "бэкап" an // disambiguate on ("вода" is a fact-or-query coin flip, "бэкап" an
// act-or-report one) and stage 0 would already have won on anything // act-or-report one) and stage 0 would already have won on anything
// that pattern-matches cleanly. Flag it now; router.go's stage-3 gate // that pattern-matches cleanly. A greeting or an inflected verb is NOT
// (Router.Route) decides whether that trips Clarify. // thin, however short — see thinSingleToken. Flag it now; router.go's
if len(strings.Fields(utterance)) <= 1 { // stage-3 gate (Router.Route) decides whether that trips Clarify.
if thinSingleToken(utterance) {
d.Confidence = llmThinConfidence d.Confidence = llmThinConfidence
} }
switch Intent(a.Intent) { switch Intent(a.Intent) {
+90
View File
@@ -0,0 +1,90 @@
package router
import "strings"
// thinSingleToken — is a one-word utterance thin evidence, or is it a whole
// sentence?
//
// The rule this replaces was `len(strings.Fields(u)) <= 1`, an English
// intuition. It does not transfer: Russian packs a subject, a tense and a
// gender into one word, so "поужинал" is a complete report and "привет" a
// complete greeting, yet both got thinned and came back as "не совсем поняла".
// Meanwhile the case the rule exists for is real — a bare noun like "вода" or
// "бэкап" genuinely does not say fact-vs-query or act-vs-report.
//
// So: still one token, but only thin it when the token is a bare nominal.
// Two escapes, both cheap and both offline:
//
// - a closed lexicon of social and command singles, which are complete by
// definition ("привет", "спасибо", "стоп", "yes");
// - a suffix test for an inflected predicate — past tense, 2nd person,
// reflexive. Verbs carry their own subject, so a verb IS a sentence.
//
// The suffix test is deliberately loose about nouns that happen to end the
// same way ("канал" reads as past tense here). That direction of error only
// costs a clarify we would not have asked for; the other direction — treating
// a real report as thin — is the bug being fixed.
func thinSingleToken(utterance string) bool {
f := strings.Fields(utterance)
if len(f) != 1 {
return false
}
w := strings.ToLower(strings.Trim(f[0], ".,!?;:—-\"'«»()"))
if w == "" {
return false
}
if completeSingles[w] {
return false
}
return !looksInflected(w)
}
// completeSingles — one-word utterances that need no second half. Greetings,
// acknowledgements and the control words a voice loop has to honour instantly.
var completeSingles = map[string]bool{
// ru: social
"привет": true, "здравствуй": true, "здравствуйте": true, "здорово": true,
"пока": true, "прощай": true, "спокойной": true, "спасибо": true,
"благодарю": true, "извини": true, "прости": true, "пожалуйста": true,
"да": true, "нет": true, "ага": true, "угу": true, "ок": true, "окей": true,
"хорошо": true, "ладно": true, "конечно": true, "верно": true, "точно": true,
// ru: control
"стоп": true, "отмена": true, "отбой": true, "хватит": true, "тихо": true,
"повтори": true, "продолжай": true, "помоги": true, "помощь": true,
// en
"hi": true, "hello": true, "hey": true, "bye": true, "goodbye": true,
"thanks": true, "thank": true, "sorry": true, "please": true,
"yes": true, "no": true, "yep": true, "nope": true, "ok": true, "okay": true,
"sure": true, "right": true, "stop": true, "cancel": true, "help": true,
"repeat": true, "continue": true,
}
// inflectedSuffixes — endings that mark a finite or past-tense Russian verb.
// Ordered longest-first is unnecessary (any match wins), but each entry is
// chosen to be long enough that common nouns rarely collide.
var inflectedSuffixes = []string{
// reflexive — strongly verbal whatever precedes it
"ся", "сь",
// past tense
"ал", "ял", "ил", "ел", "ыл", "ул", "ёл", "ала", "яла", "ила", "ела",
"ыла", "ула", "али", "яли", "или", "ели",
// 2nd person singular
"ешь", "ишь", "ёшь",
// 1st/2nd person plural, 3rd person plural
"аем", "яем", "уем", "аете", "ите", "ают", "яют", "уют", "ат", "ят",
}
// looksInflected — does the word carry a verb ending? Short words are exempt:
// a three-letter token is not enough stem to trust a two-letter suffix on
// ("газ" would otherwise never match, but "нос" and "лес" would).
func looksInflected(w string) bool {
if len([]rune(w)) < 5 {
return false
}
for _, s := range inflectedSuffixes {
if strings.HasSuffix(w, s) {
return true
}
}
return false
}
+33
View File
@@ -0,0 +1,33 @@
package router
import "testing"
func TestThinSingleTokenThinsBareNominals(t *testing.T) {
// The case the rule exists for: one noun, no way to tell what was asked.
for _, w := range []string{"вода", "бэкап", "нексус", "почта", "backup"} {
if !thinSingleToken(w) {
t.Errorf("thinSingleToken(%q) = false, want true", w)
}
}
}
func TestThinSingleTokenSparesCompleteUtterances(t *testing.T) {
// Regression: every one of these used to be answered with
// "не совсем поняла — можешь переформулировать?".
for _, w := range []string{
"привет", "Привет!", "спасибо", "да", "нет", "стоп", "hello", "yes",
"поужинал", "проснулась", "устал", "выспался", "договорились",
} {
if thinSingleToken(w) {
t.Errorf("thinSingleToken(%q) = true, want false", w)
}
}
}
func TestThinSingleTokenIgnoresMultiWord(t *testing.T) {
for _, s := range []string{"выпил воды", "что там с бэкапом", ""} {
if thinSingleToken(s) {
t.Errorf("thinSingleToken(%q) = true, want false", s)
}
}
}