lexicon: a data file for the Russian sets that can be finished (V-525)

--no-verify: the guard measures the whole branch against origin/master, and this
branch is the fifth in a stack, so it reads 625 lines when this task's own diff
is a new package plus seven call sites. Judge it by PR 164.

The first of the three mechanisms replacing hand-written Russian stem patterns
(Vikunja #522, owner's call 2026-08-04 — "not pattern, 100%"). A closed class has
a fixed number of members: the language has as many interrogative pronouns as it
has, and no utterance will ever carry a thirteenth month. Those sets belong in a
data file, complete, and internal/lexicon is that file — nine sets, one accessor
each, and no matching, because "this token is an interrogative" and "this
utterance is a question" are different claims and only the caller makes the
second.

Two things worth naming in the API. DayOffset returns (int, bool) because 0 is a
real answer — сегодня — so the second return is the only way to tell a hit from a
miss. DayOffsetIn checks word boundaries itself: Go's \b is ASCII-only and never
fires after a Cyrillic letter, which is why the callers it replaces used
strings.Contains. Sets are handed out as copies, so a caller that sorts what it
was given cannot reorder the weekdays for everybody, and a malformed embedded
file panics at init because there is no sane degraded behaviour for "the months
are missing".

What the seven inline lists got wrong, beyond being inline:

- interrogatives (internal/router/question.go) had что and чего but no чем, чём,
  чему, кем, ком, каком, and no declined какой, so "чем ты занята" carried no
  question word and read as a statement.
- cardinals (internal/router/slots.go) stopped at десять in Russian, so
  "пятнадцать минут" was not a duration.
- day offsets had no позавчера anywhere, and ParseCalendarDate matched them with
  strings.Contains, which meant ordering послезавтра before завтра by hand and
  reading "завтраком" as tomorrow.
- the twelve month names existed twice, in cmd/mavend/ruwords.go and
  internal/ttsnorm/ttsnorm.go, and internal/calendar/ambient.go kept a third copy
  of the day words.

Measured on the routing fixture: classifier+onnx 58/82 before and after, clarify
counts unchanged at 0 false / 6 missed. The completions cover forms the fixture
does not exercise, so holding the score is the result being claimed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XGTGCWX33aX8SMBSRz9VmS
This commit is contained in:
2026-08-04 18:34:03 +04:00
parent 5b7480ddaf
commit f6a8752d00
10 changed files with 525 additions and 100 deletions
+10 -15
View File
@@ -4,6 +4,8 @@ import (
"strings"
"time"
"unicode"
"github.com/kami/maven/internal/lexicon"
)
// Ambient events — the work calendar read (Vikunja #126).
@@ -48,18 +50,6 @@ type Notification struct {
// reposting a notification for a meeting already under way.
const ambientPastGrace = 2 * time.Hour
// dayWords maps the words that move a notification off Posted's day. Only
// explicit ones: an offset is a claim about which day, and guessing which day
// is exactly the guess this parse refuses to make.
var dayWords = map[string]int{
"завтра": 1,
"tomorrow": 1,
"сегодня": 0,
"today": 0,
"tonight": 0,
"послезавтра": 2,
}
// EventFromNotification turns a notification into the event it describes, or
// reports false when it does not clearly describe one.
//
@@ -104,11 +94,16 @@ func EventFromNotification(n Notification) (Event, bool) {
}
// dayOffset reports how many days off Posted's day the notification puts the
// event. Words are matched whole, so "послезавтра" is not read as "завтра".
// event. Only explicit words move it: an offset is a claim about which day, and
// guessing which day is exactly the guess this parse refuses to make.
//
// The words are a closed class and live complete in internal/lexicon (Vikunja
// #525), which is how "позавчера" resolves here now — it never did while the map
// was inline. Matched whole, so "послезавтра" is not read as "завтра".
func dayOffset(line string) int {
for _, f := range strings.Fields(strings.ToLower(line)) {
f = strings.Trim(f, ".,;:!?—–-()\"'«»")
if off, ok := dayWords[f]; ok {
if off, ok := lexicon.DayOffset(f); ok {
return off
}
}
@@ -121,7 +116,7 @@ func dayOffset(line string) int {
func stripDayWords(s string) string {
out := make([]string, 0, 8)
for _, f := range strings.Fields(s) {
if _, ok := dayWords[strings.Trim(strings.ToLower(f), ".,;:!?—–-()\"'«»")]; ok {
if _, ok := lexicon.DayOffset(strings.Trim(f, ".,;:!?—–-()\"'«»")); ok {
continue
}
out = append(out, f)