router: a Russian verb is a whole sentence, not thin evidence
The clarify gate thinned any one-word utterance to 0.3 confidence, which
trips the stage-3 gate and comes back as "не совсем поняла". That is an
English intuition. Russian packs subject, tense and gender into one word,
so "поужинал" is a complete report and "привет" a complete greeting, and
both got clarified.
thinSingleToken keeps the rule for bare nominals, where it is real ("вода"
is a fact-or-query coin flip), and spares two classes: a closed lexicon of
social and control singles, and any token carrying a verb ending. Both
tests are offline.
Fixture: false clarifies 3 → 2, intent-only 74.0% → 75.3%, full accuracy
unchanged at 70.1%, missed clarify still 1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TrVSBKe3RFDF4fGYKWYQnX
This commit is contained in:
@@ -102,8 +102,16 @@ Re-measured on the fixture after the fix: **missed clarify 6/6 → 1**, at the c
|
|||||||
clarifies and 2.6pt of full accuracy (72.7% → 70.1%, intent-only 67.5% → 74.0%). Two of the
|
clarifies and 2.6pt of full accuracy (72.7% → 70.1%, intent-only 67.5% → 74.0%). Two of the
|
||||||
three false clarifies are acts the model mis-routed and the gate caught — asking beats wrongly
|
three false clarifies are acts the model mis-routed and the gate caught — asking beats wrongly
|
||||||
executing, so the fixture and the daemon disagree about what is correct there. The third,
|
executing, so the fixture and the daemon disagree about what is correct there. The third,
|
||||||
`"поужинал"`, is a real defect: **the single-token rule is an English intuition and does not
|
`"поужинал"`, was a real defect: the single-token rule was an English intuition and does not
|
||||||
transfer to Russian**, where one word is routinely a whole sentence. Narrow or drop it.
|
transfer to Russian, where one word is routinely a whole sentence.
|
||||||
|
|
||||||
|
Narrowed 01-08-2026. `thinSingleToken` (`internal/router/singletoken.go`) still thins a bare
|
||||||
|
one-word nominal — "вода", "бэкап" — but spares two classes: a closed lexicon of social and
|
||||||
|
control singles ("привет", "спасибо", "стоп", "yes"), and any token carrying a Russian verb
|
||||||
|
ending (past tense, 2nd person, reflexive), because a verb already contains its subject. Both
|
||||||
|
tests are offline and cost nothing. Re-measured: **false clarifies 3 → 2, intent-only 74.0% →
|
||||||
|
75.3%, full accuracy unchanged at 70.1%, missed clarify still 1.** The two remaining false
|
||||||
|
clarifies are the act-with-no-allowlisted-fn arm of the gate, not this rule.
|
||||||
|
|
||||||
## LLM output contract
|
## LLM output contract
|
||||||
|
|
||||||
|
|||||||
@@ -194,12 +194,13 @@ func (lr *LLMRouter) Route(ctx context.Context, utterance string, now time.Time)
|
|||||||
return Decision{}, false, nil
|
return Decision{}, false, nil
|
||||||
}
|
}
|
||||||
d := Decision{Utterance: utterance, Stage: 1, Confidence: llmFullConfidence}
|
d := Decision{Utterance: utterance, Stage: 1, Confidence: llmFullConfidence}
|
||||||
// A single-token utterance is thin evidence: the model had nothing to
|
// A bare one-word nominal is thin evidence: the model had nothing to
|
||||||
// disambiguate on ("вода" is a fact-or-query coin flip, "бэкап" an
|
// disambiguate on ("вода" is a fact-or-query coin flip, "бэкап" an
|
||||||
// act-or-report one) and stage 0 would already have won on anything
|
// act-or-report one) and stage 0 would already have won on anything
|
||||||
// that pattern-matches cleanly. Flag it now; router.go's stage-3 gate
|
// that pattern-matches cleanly. A greeting or an inflected verb is NOT
|
||||||
// (Router.Route) decides whether that trips Clarify.
|
// thin, however short — see thinSingleToken. Flag it now; router.go's
|
||||||
if len(strings.Fields(utterance)) <= 1 {
|
// stage-3 gate (Router.Route) decides whether that trips Clarify.
|
||||||
|
if thinSingleToken(utterance) {
|
||||||
d.Confidence = llmThinConfidence
|
d.Confidence = llmThinConfidence
|
||||||
}
|
}
|
||||||
switch Intent(a.Intent) {
|
switch Intent(a.Intent) {
|
||||||
|
|||||||
@@ -0,0 +1,90 @@
|
|||||||
|
package router
|
||||||
|
|
||||||
|
import "strings"
|
||||||
|
|
||||||
|
// thinSingleToken — is a one-word utterance thin evidence, or is it a whole
|
||||||
|
// sentence?
|
||||||
|
//
|
||||||
|
// The rule this replaces was `len(strings.Fields(u)) <= 1`, an English
|
||||||
|
// intuition. It does not transfer: Russian packs a subject, a tense and a
|
||||||
|
// gender into one word, so "поужинал" is a complete report and "привет" a
|
||||||
|
// complete greeting, yet both got thinned and came back as "не совсем поняла".
|
||||||
|
// Meanwhile the case the rule exists for is real — a bare noun like "вода" or
|
||||||
|
// "бэкап" genuinely does not say fact-vs-query or act-vs-report.
|
||||||
|
//
|
||||||
|
// So: still one token, but only thin it when the token is a bare nominal.
|
||||||
|
// Two escapes, both cheap and both offline:
|
||||||
|
//
|
||||||
|
// - a closed lexicon of social and command singles, which are complete by
|
||||||
|
// definition ("привет", "спасибо", "стоп", "yes");
|
||||||
|
// - a suffix test for an inflected predicate — past tense, 2nd person,
|
||||||
|
// reflexive. Verbs carry their own subject, so a verb IS a sentence.
|
||||||
|
//
|
||||||
|
// The suffix test is deliberately loose about nouns that happen to end the
|
||||||
|
// same way ("канал" reads as past tense here). That direction of error only
|
||||||
|
// costs a clarify we would not have asked for; the other direction — treating
|
||||||
|
// a real report as thin — is the bug being fixed.
|
||||||
|
func thinSingleToken(utterance string) bool {
|
||||||
|
f := strings.Fields(utterance)
|
||||||
|
if len(f) != 1 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
w := strings.ToLower(strings.Trim(f[0], ".,!?;:—-\"'«»()"))
|
||||||
|
if w == "" {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if completeSingles[w] {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return !looksInflected(w)
|
||||||
|
}
|
||||||
|
|
||||||
|
// completeSingles — one-word utterances that need no second half. Greetings,
|
||||||
|
// acknowledgements and the control words a voice loop has to honour instantly.
|
||||||
|
var completeSingles = map[string]bool{
|
||||||
|
// ru: social
|
||||||
|
"привет": true, "здравствуй": true, "здравствуйте": true, "здорово": true,
|
||||||
|
"пока": true, "прощай": true, "спокойной": true, "спасибо": true,
|
||||||
|
"благодарю": true, "извини": true, "прости": true, "пожалуйста": true,
|
||||||
|
"да": true, "нет": true, "ага": true, "угу": true, "ок": true, "окей": true,
|
||||||
|
"хорошо": true, "ладно": true, "конечно": true, "верно": true, "точно": true,
|
||||||
|
// ru: control
|
||||||
|
"стоп": true, "отмена": true, "отбой": true, "хватит": true, "тихо": true,
|
||||||
|
"повтори": true, "продолжай": true, "помоги": true, "помощь": true,
|
||||||
|
// en
|
||||||
|
"hi": true, "hello": true, "hey": true, "bye": true, "goodbye": true,
|
||||||
|
"thanks": true, "thank": true, "sorry": true, "please": true,
|
||||||
|
"yes": true, "no": true, "yep": true, "nope": true, "ok": true, "okay": true,
|
||||||
|
"sure": true, "right": true, "stop": true, "cancel": true, "help": true,
|
||||||
|
"repeat": true, "continue": true,
|
||||||
|
}
|
||||||
|
|
||||||
|
// inflectedSuffixes — endings that mark a finite or past-tense Russian verb.
|
||||||
|
// Ordered longest-first is unnecessary (any match wins), but each entry is
|
||||||
|
// chosen to be long enough that common nouns rarely collide.
|
||||||
|
var inflectedSuffixes = []string{
|
||||||
|
// reflexive — strongly verbal whatever precedes it
|
||||||
|
"ся", "сь",
|
||||||
|
// past tense
|
||||||
|
"ал", "ял", "ил", "ел", "ыл", "ул", "ёл", "ала", "яла", "ила", "ела",
|
||||||
|
"ыла", "ула", "али", "яли", "или", "ели",
|
||||||
|
// 2nd person singular
|
||||||
|
"ешь", "ишь", "ёшь",
|
||||||
|
// 1st/2nd person plural, 3rd person plural
|
||||||
|
"аем", "яем", "уем", "аете", "ите", "ают", "яют", "уют", "ат", "ят",
|
||||||
|
}
|
||||||
|
|
||||||
|
// looksInflected — does the word carry a verb ending? Short words are exempt:
|
||||||
|
// a three-letter token is not enough stem to trust a two-letter suffix on
|
||||||
|
// ("газ" would otherwise never match, but "нос" and "лес" would).
|
||||||
|
func looksInflected(w string) bool {
|
||||||
|
if len([]rune(w)) < 5 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, s := range inflectedSuffixes {
|
||||||
|
if strings.HasSuffix(w, s) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
package router
|
||||||
|
|
||||||
|
import "testing"
|
||||||
|
|
||||||
|
func TestThinSingleTokenThinsBareNominals(t *testing.T) {
|
||||||
|
// The case the rule exists for: one noun, no way to tell what was asked.
|
||||||
|
for _, w := range []string{"вода", "бэкап", "нексус", "почта", "backup"} {
|
||||||
|
if !thinSingleToken(w) {
|
||||||
|
t.Errorf("thinSingleToken(%q) = false, want true", w)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestThinSingleTokenSparesCompleteUtterances(t *testing.T) {
|
||||||
|
// Regression: every one of these used to be answered with
|
||||||
|
// "не совсем поняла — можешь переформулировать?".
|
||||||
|
for _, w := range []string{
|
||||||
|
"привет", "Привет!", "спасибо", "да", "нет", "стоп", "hello", "yes",
|
||||||
|
"поужинал", "проснулась", "устал", "выспался", "договорились",
|
||||||
|
} {
|
||||||
|
if thinSingleToken(w) {
|
||||||
|
t.Errorf("thinSingleToken(%q) = true, want false", w)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestThinSingleTokenIgnoresMultiWord(t *testing.T) {
|
||||||
|
for _, s := range []string{"выпил воды", "что там с бэкапом", ""} {
|
||||||
|
if thinSingleToken(s) {
|
||||||
|
t.Errorf("thinSingleToken(%q) = true, want false", s)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user