the self-care recognisers read closed classes, not stems (V-586)
DefaultFactParser matched Russian by hand-written stem substring: "вод", "пил", "душ", "еда" and eleven more, with a helper whose own comment said it would use a morphology lib "until misfires actually bite". That is the fourth mechanism CLAUDE.md says does not exist, and it ran on every fact turn through both wirings in cmd/mavend/voicewire.go. Five closed classes move to internal/lexicon — water nouns and drink verbs, meal words, shower, break, sleep — and internal/morph does the inflection. Three dictionary quirks are carried as data rather than worked around in code, each with its reason in the set's note: "вода" and "водой" lemmatise to two different lemmas, "пил" lemmatises to the saw, and "спал" to "спасть". Shower is matched exactly rather than by lemma, because the dictionary makes "душ" and "душа" one word and only one of them is washing. The accusative of an inanimate noun is its nominative, so exact matching costs nothing he says. NOT behaviour-preserving, deliberately. Rejected now: "пилот", "водитель", "заводить", "душа", "душно", "беда", "победа". "есть" and "ел" are left out of the meal set on purpose — "есть новости по бэкапу" is a question. The vestigial "ate"/"backup" guard goes with the substring era that needed it. Measured on the RU routing fixture, classifier+ONNX arm (91 cases): 64/91 (70.3%) before and after, same failing cases. The LLM arm was not measured — no llama-server reachable from here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -135,6 +135,30 @@ func ConfirmNo() []string { return words("confirm_no") }
|
|||||||
// TaskDropWords — see TaskDoneWords.
|
// TaskDropWords — see TaskDoneWords.
|
||||||
func TaskDropWords() []string { return words("task_drop_words") }
|
func TaskDropWords() []string { return words("task_drop_words") }
|
||||||
|
|
||||||
|
// WaterNouns and DrinkVerbs are the two halves of a water fact: he has to name
|
||||||
|
// the drink and the drinking, because "вода" alone is a word about water and
|
||||||
|
// "выпил" alone does not say what. The other four self-care sets need only one
|
||||||
|
// word each. All six are matched over tokens by lemma — except ShowerWords, see
|
||||||
|
// its own note.
|
||||||
|
func WaterNouns() []string { return words("water_nouns") }
|
||||||
|
|
||||||
|
// DrinkVerbs — see WaterNouns.
|
||||||
|
func DrinkVerbs() []string { return words("drink_verbs") }
|
||||||
|
|
||||||
|
// MealWords returns the nouns and verbs of having eaten.
|
||||||
|
func MealWords() []string { return words("meal_words") }
|
||||||
|
|
||||||
|
// ShowerWords returns the shower noun. Match these EXACTLY and not by lemma:
|
||||||
|
// the dictionary makes "душ" and "душа" one word, and only one of them is a
|
||||||
|
// shower. The set's note says why exact matching costs nothing here.
|
||||||
|
func ShowerWords() []string { return words("shower_words") }
|
||||||
|
|
||||||
|
// BreakWords returns the noun and verbs of taking a break.
|
||||||
|
func BreakWords() []string { return words("break_words") }
|
||||||
|
|
||||||
|
// SleepWords returns the verbs of having slept.
|
||||||
|
func SleepWords() []string { return words("sleep_words") }
|
||||||
|
|
||||||
// SlotValueFrame returns the words that can surround a bare slot value without
|
// SlotValueFrame returns the words that can surround a bare slot value without
|
||||||
// making the utterance a request of its own. A caller strips these (along with
|
// making the utterance a request of its own. A caller strips these (along with
|
||||||
// the numbers and the other closed time sets) to see whether an utterance
|
// the numbers and the other closed time sets) to see whether an utterance
|
||||||
|
|||||||
@@ -252,6 +252,50 @@
|
|||||||
"yes", "yeah", "yep", "yup", "ok", "okay", "sure", "confirm", "affirmative"
|
"yes", "yeah", "yep", "yup", "ok", "okay", "sure", "confirm", "affirmative"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
|
"water_nouns": {
|
||||||
|
"note": "The water noun, in the forms he drinks it in, plus English (V-586). Closed because it is one noun: Russian gives it six cases and two numbers and that is the whole list. Both \"вода\" and \"водой\" are listed even though one declension covers both, because the vendored dictionary lemmatises \"воды\" to \"вод\" and \"водой\" to \"вода\" — two lemmas for one noun, so the set has to name both or a caller matching by lemma misses half of them. Matched over tokens with morph.SameWord, never as a substring: the \"вод\" this replaced fired on \"водитель\" and \"заводить\".",
|
||||||
|
"words": [
|
||||||
|
"вода", "водой", "водичка", "water"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"drink_verbs": {
|
||||||
|
"note": "Drinking, in the aspects and prefixes he speaks (V-586). Closed in the sense that matters: these are the verbs that make a water noun a water FACT, and the list is her vocabulary rather than a discovery about Russian. \"пил\" and \"пили\" are listed as surface forms because the dictionary lemmatises them to \"пила\", the saw; the perfective forms lemmatise correctly and one member each covers them. Whole tokens only — the substring \"пил\" this replaced fired on \"пилот\".",
|
||||||
|
"words": [
|
||||||
|
"пить", "пил", "пили", "пей", "выпить", "попить", "допить", "запить",
|
||||||
|
"drink", "drank", "drinking"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"meal_words": {
|
||||||
|
"note": "Eating: the meal nouns and the verbs of having one (V-586). Closed the same way capture_verbs is — these are the words that write a meal fact, decided here. The verbs are listed in the infinitive because that is the lemma the dictionary returns, so \"поужинал\" and \"позавтракал\" match without their own entries. \"есть\" and \"ел\" are deliberately ABSENT: \"есть\" is also the existential, and \"есть новости по бэкапу\" is a question rather than a meal. The English \"ate\" carried a guard against \"backup\" when this was a substring test; over tokens the guard is unnecessary.",
|
||||||
|
"words": [
|
||||||
|
"обед", "обедать", "пообедать",
|
||||||
|
"ужин", "ужинать", "поужинать",
|
||||||
|
"завтрак", "завтракать", "позавтракать",
|
||||||
|
"еда", "перекус", "перекусить",
|
||||||
|
"поесть", "кушать", "покушать",
|
||||||
|
"meal", "ate", "lunch", "dinner", "breakfast"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"shower_words": {
|
||||||
|
"note": "The shower, and the one set here matched EXACTLY rather than by lemma (V-586). The dictionary lemmatises \"душ\" to \"душа\", so a lemma test cannot tell a shower from a soul, and \"на душе легко\" is not a fact about washing. The accusative of an inanimate noun is its nominative, so \"принял душ\" and \"сходил в душ\" are both the bare form and exact matching loses nothing he actually says. The substring this replaced also fired on \"душно\".",
|
||||||
|
"words": [
|
||||||
|
"душ", "душем", "shower", "showered"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"break_words": {
|
||||||
|
"note": "Taking a break, noun and verb (V-586). Closed like meal_words and for the same reason. \"отдых\" and \"отдыхать\" are both listed because the noun and the verb are separate lemmas; the perfective \"отдохнул\" lemmatises to \"отдохнуть\".",
|
||||||
|
"words": [
|
||||||
|
"перерыв", "отдых", "отдыхать", "отдохнуть", "передохнуть",
|
||||||
|
"break", "rest"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"sleep_words": {
|
||||||
|
"note": "Sleeping (V-586). The imperfective surface forms \"спал\" and \"спала\" are listed because the dictionary lemmatises them to \"спасть\", a different verb, and one entry for the pair is the honest fix; the prefixed forms lemmatise consistently and their infinitives cover them. \"сон\" is absent: the noun names a dream as readily as a night's sleep, and it was not in the pattern this replaces either.",
|
||||||
|
"words": [
|
||||||
|
"спать", "спал", "поспать", "поспал", "проспать", "выспаться",
|
||||||
|
"sleep", "slept", "sleeping"
|
||||||
|
]
|
||||||
|
},
|
||||||
"confirm_no": {
|
"confirm_no": {
|
||||||
"note": "The answers that decline a parked confirm. Same matching rule as confirm_yes and the same reason. The multi-word members are here rather than assembled by a caller because \"надо\" alone is not an answer and \"не надо\" is the opposite of one: the two must land on opposite sides, and only the phrase says which. \"не\" on its own is NOT a member — \"не забудь купить хлеб\" is a reminder, not a refusal.",
|
"note": "The answers that decline a parked confirm. Same matching rule as confirm_yes and the same reason. The multi-word members are here rather than assembled by a caller because \"надо\" alone is not an answer and \"не надо\" is the opposite of one: the two must land on opposite sides, and only the phrase says which. \"не\" on its own is NOT a member — \"не забудь купить хлеб\" is a reminder, not a refusal.",
|
||||||
"words": [
|
"words": [
|
||||||
|
|||||||
+36
-23
@@ -7,6 +7,7 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/kami/maven/internal/lexicon"
|
"github.com/kami/maven/internal/lexicon"
|
||||||
|
"github.com/kami/maven/internal/morph"
|
||||||
)
|
)
|
||||||
|
|
||||||
// DateTimeParser — resolves relative→absolute AT CAPTURE ("in 4h" → now+4h),
|
// DateTimeParser — resolves relative→absolute AT CAPTURE ("in 4h" → now+4h),
|
||||||
@@ -123,23 +124,22 @@ type DefaultFactParser struct{}
|
|||||||
|
|
||||||
func (DefaultFactParser) Parse(utterance string) (string, string, bool) {
|
func (DefaultFactParser) Parse(utterance string) (string, string, bool) {
|
||||||
s := strings.ToLower(strings.TrimSpace(utterance))
|
s := strings.ToLower(strings.TrimSpace(utterance))
|
||||||
// Maven is ru-first (voice, tts). Each case carries the English tokens AND
|
toks := strings.Fields(s)
|
||||||
// Russian stems — matched by prefix (hasStem) because Russian inflects
|
// Maven is ru-first (voice, tts), and Russian inflects, so the words each
|
||||||
// (воды/воду/вода share "вод"), so exact-token matching would miss most
|
// case reads are closed classes in internal/lexicon and the inflection is
|
||||||
// real utterances and silently drop the capture.
|
// morph's job (V-586). What stood here was a fourth mechanism: hand-written
|
||||||
|
// stems matched as substrings, so "пилот" was drinking, "водитель" was
|
||||||
|
// water, "душно" was a shower and "победа" was a meal.
|
||||||
switch {
|
switch {
|
||||||
case (containsWord(s, "water") && containsWord(s, "drank")) ||
|
case anyLemma(toks, lexicon.WaterNouns()) && anyLemma(toks, lexicon.DrinkVerbs()):
|
||||||
(hasRoot(s, "вод") && (hasRoot(s, "пил") || hasRoot(s, "пью") || hasRoot(s, "пей"))):
|
|
||||||
return "water", `"drank"`, true
|
return "water", `"drank"`, true
|
||||||
case containsWord(s, "meal") || (containsWord(s, "ate") && !containsWord(s, "backup")) || containsWord(s, "lunch") || containsWord(s, "dinner") ||
|
case anyLemma(toks, lexicon.MealWords()):
|
||||||
hasRoot(s, "поел") || hasRoot(s, "поесть") || hasRoot(s, "куша") || hasRoot(s, "обед") || hasRoot(s, "ужин") || hasRoot(s, "завтрак") || hasRoot(s, "еда"):
|
|
||||||
return "meal", `"ate"`, true
|
return "meal", `"ate"`, true
|
||||||
case containsWord(s, "shower") || hasRoot(s, "душ"):
|
case anyExact(toks, lexicon.ShowerWords()):
|
||||||
return "shower", `"took"`, true
|
return "shower", `"took"`, true
|
||||||
case containsWord(s, "break") || hasRoot(s, "перерыв") || hasRoot(s, "отдох"):
|
case anyLemma(toks, lexicon.BreakWords()):
|
||||||
return "break", `"took"`, true
|
return "break", `"took"`, true
|
||||||
case containsWord(s, "slept") || containsWord(s, "sleep") ||
|
case anyLemma(toks, lexicon.SleepWords()):
|
||||||
hasRoot(s, "спал") || hasRoot(s, "выспал"):
|
|
||||||
if v, ok := parseDurationValue(afterWord(s, "slept")); ok {
|
if v, ok := parseDurationValue(afterWord(s, "slept")); ok {
|
||||||
return "sleep", strconv.Quote(v), true
|
return "sleep", strconv.Quote(v), true
|
||||||
}
|
}
|
||||||
@@ -148,18 +148,31 @@ func (DefaultFactParser) Parse(utterance string) (string, string, bool) {
|
|||||||
return "", "", false
|
return "", "", false
|
||||||
}
|
}
|
||||||
|
|
||||||
// hasRoot — substring match on the whole utterance. Russian inflects with BOTH
|
// anyLemma reports whether any token is one of the set's words, in any case or
|
||||||
// prefixes and suffixes (вы-пил, по-пил, пил-и), so a prefix test misses the
|
// tense. Exact equality first because morph falls back to it without the
|
||||||
// verb; the root as a substring catches all forms. A rare over-match (пил in
|
// dictionary, and because a member the dictionary lemmatises oddly is carried
|
||||||
// пилот) is fine at this floor. ponytail: substring roots over a morphology lib
|
// in the set as its surface form.
|
||||||
// until misfires actually bite.
|
func anyLemma(toks []string, set []string) bool {
|
||||||
func hasRoot(s, root string) bool { return strings.Contains(s, root) }
|
for _, tok := range toks {
|
||||||
|
t := cleanWord(tok)
|
||||||
|
for _, w := range set {
|
||||||
|
if t == w || morph.SameWord(t, w) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
// containsWord — whole-token membership (avoids "breakfast" matching "break").
|
// anyExact is anyLemma without the grammar, for a set whose members share a
|
||||||
func containsWord(s, w string) bool {
|
// lemma with a word that means something else. Only ShowerWords needs it.
|
||||||
for _, tok := range strings.Fields(s) {
|
func anyExact(toks []string, set []string) bool {
|
||||||
if tok == w {
|
for _, tok := range toks {
|
||||||
return true
|
t := cleanWord(tok)
|
||||||
|
for _, w := range set {
|
||||||
|
if t == w {
|
||||||
|
return true
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return false
|
return false
|
||||||
|
|||||||
@@ -23,6 +23,24 @@ func TestDefaultFactParserRU(t *testing.T) {
|
|||||||
{"немного отдохнул", "break", true},
|
{"немного отдохнул", "break", true},
|
||||||
{"поспал шесть часов", "sleep", true},
|
{"поспал шесть часов", "sleep", true},
|
||||||
{"перезапусти nginx", "", false}, // an act, not a fact
|
{"перезапусти nginx", "", false}, // an act, not a fact
|
||||||
|
// Inflections the substring stems caught and a lemma test must keep.
|
||||||
|
{"только что выпил кружку воды", "water", true},
|
||||||
|
{"воды попил наконец", "water", true},
|
||||||
|
{"пью воду", "water", true},
|
||||||
|
{"поужинал", "meal", true},
|
||||||
|
{"отметь что я позавтракал овсянкой", "meal", true},
|
||||||
|
{"сходил в душ", "shower", true},
|
||||||
|
{"отдыхал час", "break", true},
|
||||||
|
{"выспался", "sleep", true},
|
||||||
|
// Misfires the substring stems accepted. Rejecting them is the point of
|
||||||
|
// V-586: none of these is a fact about his day.
|
||||||
|
{"я пилот", "", false},
|
||||||
|
{"водитель приехал", "", false},
|
||||||
|
{"надо заводить машину", "", false},
|
||||||
|
{"у меня душа болит", "", false},
|
||||||
|
{"в комнате душно", "", false},
|
||||||
|
{"это была беда", "", false},
|
||||||
|
{"победа наша", "", false},
|
||||||
}
|
}
|
||||||
for _, c := range cases {
|
for _, c := range cases {
|
||||||
k, _, ok := p.Parse(c.utterance)
|
k, _, ok := p.Parse(c.utterance)
|
||||||
|
|||||||
Reference in New Issue
Block a user