Compare commits
23 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 70fb7c030b | |||
| fe489dff6d | |||
| d8da529be0 | |||
| 0258a40b0d | |||
| f6a8752d00 | |||
| c5e993fc55 | |||
| a2081d8227 | |||
| 82bd160c0d | |||
| 1fe03f7a51 | |||
| 77206f298e | |||
| a1f811d4c8 | |||
| 7ab38cd7f7 | |||
| 6a85e71077 | |||
| bf6c2bf1a6 | |||
| 1558233665 | |||
| 5afa2dfb38 | |||
| 0f361d2034 | |||
| 19242ef73b | |||
| 26d6d71588 | |||
| df58710141 | |||
| 442d3ec08e | |||
| e481ad4930 | |||
| 3d8224fb04 |
@@ -189,6 +189,18 @@ fixture had said `query` since ru-query-019 was written. Measured: **full accura
|
||||
Go's `\b` is ASCII-only and never fires after a Cyrillic letter; the pattern needs an
|
||||
explicit `(\s|[?!.]|$)`.
|
||||
|
||||
Two more shapes taken off the model, 04-08-2026 (V-498). `rest-of-day-query` inside
|
||||
`AgendaQueryGrammars` claims "что дальше?" / "what's next", and `NarrativeQueryGrammar`
|
||||
(`stage0.go`, wired **last** in `buildRouter`, after the capture marker) claims "расскажи про
|
||||
X", "объясни X", "опиши X". Neither carries a question mark or an interrogative, so the model
|
||||
called both `IntentFact`; the write was caught downstream by `IsQuestionShaped`, so this was a
|
||||
latency and fixture defect, not a correctness one. The narrative rule reads the same
|
||||
`narrativeRequests` lexicon `IsQuestionShaped` reads, and declines `chatNarrativeTopics` — a
|
||||
joke, a bedtime story, herself — because the query chain has no source that answers those.
|
||||
New fixture cases ru-query-024 and ru-query-025. Classifier + ONNX baseline **56/80 (70.0%) →
|
||||
58/82 (70.7%)**, no case regressed, no new false clarify. The LLM arm was not measured (no
|
||||
llama-server in that run), so judge it again before quoting a cascade number.
|
||||
|
||||
## LLM output contract
|
||||
|
||||
All phrasing paths emit `{"response":"...","mood":"..."}` (parsed in `replier_llm.go` and
|
||||
@@ -198,6 +210,32 @@ All phrasing paths emit `{"response":"...","mood":"..."}` (parsed in `replier_ll
|
||||
note, query, act, chat, system`). `llm/check_prompt_parity.py` in the training
|
||||
workspace enforces that the Go and relabelling prompts remain identical.
|
||||
|
||||
## Russian patterns — three mechanisms, no fourth
|
||||
|
||||
Hand-written Russian stem patterns were swept out on 2026-08-04 (owner's call: not a
|
||||
pattern, and the resident model cannot be asked per turn either). A regex whose output is a
|
||||
fact or a route is the defect; a regex over structured input — HTML, MIME, JSON, a URL, an
|
||||
argv list — is not. Before writing a Russian word list, pick one of these:
|
||||
|
||||
- **`internal/lexicon`** — closed classes, in `lexicon_ru_v1.json`. Interrogatives,
|
||||
capture verbs, cardinals, day offsets, weekdays, months, spoken hours. Editing a word is
|
||||
a data change, and there is exactly one copy: months used to live in three files.
|
||||
- **`internal/morph`** — grammar, from the vendored golem Russian dictionary. `IsVerbForm`
|
||||
and `SameWord`. Note that lemma matching is BROADER than stem-plus-one-ending, so a verb
|
||||
slot that means the imperative must be matched exactly — `говори` and `говорил` are one
|
||||
lemma and only one of them is a command (`cmd/mavend/quiet_toggle.go`).
|
||||
- **`cmd/mavend/topics.go` and the embedder** — open sets, where the question is what a
|
||||
turn is ABOUT. Frozen seeds per subject plus a real `other` class, scored against the
|
||||
turn's own query vector. Same shape as the personal boundary in `personalboundary.go`,
|
||||
with one difference: a topic must clear the runner-up by `topicMargin`, because a false
|
||||
claim here spends a network scan rather than one honest "не знаю". The old keyword tests
|
||||
stay as the offline floor and may remain narrow, since they are no longer the only answer.
|
||||
- **The ecosystem trio** — when the answer is not in the utterance at all. Identity is
|
||||
Nexus's, never a local pattern.
|
||||
|
||||
Seeds are scoring data. Editing one moves a recogniser and must be re-measured against the
|
||||
`TestONNX*` tests, not eyeballed.
|
||||
|
||||
## Non-goals (hard constraints)
|
||||
|
||||
Not a nag, not autonomous. Maven's persona is **feminine** — Russian
|
||||
|
||||
@@ -16,7 +16,7 @@ PIPER_BIN := $(shell pwd)/deps/piper/piper
|
||||
PIPER_MODEL := $(shell pwd)/models/tts/ru_RU-irina-medium.onnx
|
||||
PIPER_ESPEAK := $(shell pwd)/deps/piper/espeak-ng-data
|
||||
|
||||
.PHONY: simulate stt-fixtures test-stt-golden all build build-stt build-tts build-daemon build-client build-waked build-web build-poll build-caldav clean test fmt-check vet run-stt run-tts run-web download-embedder deps-go eval-router eval-recall eval-phrasing eval-models build-gpud
|
||||
.PHONY: simulate stt-fixtures test-stt-golden all build build-stt build-tts build-daemon build-client build-waked build-web build-poll build-caldav clean test fmt-check vet run-stt run-tts run-web download-embedder deps-go deps-sentinel tidy eval-router eval-recall eval-phrasing eval-models build-gpud
|
||||
|
||||
all: build
|
||||
|
||||
@@ -74,7 +74,7 @@ run-web: build-web
|
||||
# base.Tool(), which only stats pkg/tool and exits. So build them in once here.
|
||||
GO_TARBALL := go$(GO_VERSION).linux-amd64.tar.gz
|
||||
GO_SHA256 := 9e9b755d63b36acf30c12a9a3fc379243714c1c6d3dd72861da637f336ebb35b
|
||||
deps-go:
|
||||
deps-go: deps-sentinel
|
||||
@mkdir -p deps/go
|
||||
cd deps/go && curl -fLO 'https://go.dev/dl/$(GO_TARBALL)'
|
||||
cd deps/go && echo '$(GO_SHA256) $(GO_TARBALL)' | sha256sum -c -
|
||||
@@ -84,6 +84,25 @@ deps-go:
|
||||
done
|
||||
$(GO) version
|
||||
|
||||
# deps/go.mod — the sentinel that stops the module walk at deps/ (Vikunja #454).
|
||||
# The vendored toolchain lives inside the module tree, so `go mod tidy` walked
|
||||
# Go's own compiler-error fixtures and died on files that are malformed on
|
||||
# purpose ("unicode//utf8": double slash). A nested module is not part of the
|
||||
# parent, so one three-line file ends the walk. deps/ is gitignored, so it is
|
||||
# generated here rather than committed, and every target that populates deps/
|
||||
# writes it.
|
||||
deps-sentinel:
|
||||
@mkdir -p deps
|
||||
@printf 'module github.com/kami/maven/deps\n\ngo 1.21\n' > deps/go.mod
|
||||
|
||||
# Run the tidy the sentinel makes possible. Not part of `test`: it rewrites
|
||||
# go.mod, and a build target that edits the module file is a surprise.
|
||||
# vendor/ is committed, so a tidy that drops a requirement must be followed by
|
||||
# a re-vendor or the next build fails on "inconsistent vendoring".
|
||||
tidy: deps-sentinel
|
||||
GOTOOLCHAIN=local GOFLAGS=-mod=mod $(GO) mod tidy
|
||||
GOTOOLCHAIN=local GOFLAGS=-mod=mod $(GO) mod vendor
|
||||
|
||||
# fmt-check fails if any file needs gofmt. docs/design.md has always said `make
|
||||
# test` gates on gofmt and vet; it did not, so nine files quietly drifted.
|
||||
# Run `gofmt -w` on whatever this prints.
|
||||
@@ -178,7 +197,7 @@ run-tts: build-tts
|
||||
./mavttsd -socket /tmp/maven/tts.sock \
|
||||
-piper $(PIPER_BIN) -model $(PIPER_MODEL) -espeak_data $(PIPER_ESPEAK)
|
||||
|
||||
deps: deps-whisper deps-piper
|
||||
deps: deps-sentinel deps-whisper deps-piper
|
||||
|
||||
deps-whisper:
|
||||
cd deps/whisper.cpp && cmake -B build -DCMAKE_BUILD_TYPE=Release \
|
||||
|
||||
@@ -58,6 +58,9 @@ var actionHandlers = map[router.Intent]func(*reactiveHandler, context.Context, r
|
||||
func (h *reactiveHandler) actionChat(ctx context.Context, dec router.Decision) string {
|
||||
// Conversational: build history from dialogue session (prior user turns)
|
||||
// and let the LLM respond from general knowledge + context.
|
||||
if h.phraser == nil {
|
||||
return "поговорили."
|
||||
}
|
||||
history := h.chatHistory()
|
||||
// The phraser hands back its own fallback text alongside the error, so the
|
||||
// turn survives a dead server and the failure still reaches the log.
|
||||
|
||||
@@ -39,6 +39,21 @@ func (h *reactiveHandler) actionFact(ctx context.Context, dec router.Decision) s
|
||||
q.Slots.Value = ""
|
||||
return h.actionQuery(ctx, q)
|
||||
}
|
||||
// A complaint is not a fact either (#481). "сеть какая-то медленная" and
|
||||
// "интернет не работает" were stored as `self` rows at confidence 1.00, and
|
||||
// recall reads a self row back later as if it were still true — the same
|
||||
// class of row that outranked live search in #470. The sentence describes a
|
||||
// moment, so she answers it and stores nothing. An explicit "запомни ..."
|
||||
// and anything about him are both left alone by the test.
|
||||
if router.IsTransientComplaint(dec.Utterance) {
|
||||
log.Printf("voice: fact write refused, utterance is a passing complaint: %q (key %q) — answering as chat",
|
||||
dec.Utterance, dec.Slots.Key)
|
||||
c := dec
|
||||
c.Intent = router.IntentChat
|
||||
c.Slots.Key, c.Slots.HasKey = "", false
|
||||
c.Slots.Value = ""
|
||||
return h.actionChat(ctx, c)
|
||||
}
|
||||
now := h.now()
|
||||
req := ipc.WriteFactReq{
|
||||
Ts: now,
|
||||
|
||||
@@ -81,6 +81,11 @@ var querySources = []querySource{
|
||||
// matcher requires a task noun or an explicit "что … сделать", so a
|
||||
// date-bearing question still reaches the calendar.
|
||||
{name: "tasks", answer: (*reactiveHandler).queryTasks},
|
||||
// Next to "tasks" and for the same reason: "что требует внимания?" is a
|
||||
// question about the operational state Praxis holds, and it used to fall
|
||||
// through every source to the web search (Vikunja #475). Its matcher needs
|
||||
// an attention marker, and it falls through when Praxis is not configured.
|
||||
{name: "attention", answer: (*reactiveHandler).queryAttention},
|
||||
// Before the recall sources too: "сколько я потратил?" is a question about
|
||||
// the money facts the poller wrote, and the notes pass would otherwise
|
||||
// answer it from whatever he once said about spending. Its matcher needs a
|
||||
@@ -91,6 +96,12 @@ var querySources = []querySource{
|
||||
// marker, so "надо бы съездить в магазин" is untouched.
|
||||
{name: "list", answer: (*reactiveHandler).queryList},
|
||||
{name: "money", answer: (*reactiveHandler).queryMoney},
|
||||
// Also above the recall sources: "что я тебе говорил?" is a question about
|
||||
// the facts he tapped in, and the notes pass would answer it with whatever
|
||||
// note is nearest (Vikunja #456). Its matcher needs both halves of a
|
||||
// history phrase and bails out when he names a topic, so "что я говорил
|
||||
// про сервер" is still recall.
|
||||
{name: "history", answer: (*reactiveHandler).queryHistory},
|
||||
// Before the recall sources and before general knowledge: "что нового?" is
|
||||
// a question about the feeds she reads, and general knowledge would answer
|
||||
// it by inventing news. Its matcher needs a feed noun plus an ask, so
|
||||
@@ -145,6 +156,13 @@ func (h *reactiveHandler) actionQuery(ctx context.Context, dec router.Decision)
|
||||
continue
|
||||
}
|
||||
if reply, ok := src.answer(h, ctx, t); ok {
|
||||
// Which source claimed is the one thing about a query turn that was
|
||||
// invisible from outside: /trace is the nudge-rule trace and carries
|
||||
// no query-source field, so a wrong answer could not be told from a
|
||||
// wrongly-ordered chain (Vikunja #474). Only the name is logged —
|
||||
// the utterance and the answer are already on the voice lines above
|
||||
// and below this one.
|
||||
log.Printf("voice: query claimed by source %q", src.name)
|
||||
return reply
|
||||
}
|
||||
}
|
||||
@@ -283,9 +301,17 @@ func (h *reactiveHandler) queryFeeds(ctx context.Context, t *queryTurn) (string,
|
||||
return "", false
|
||||
}
|
||||
if !h.feedsOn {
|
||||
// Claim the turn rather than fall through: "не читаю ленты" is true, and
|
||||
// letting general knowledge answer "что нового?" would be an invented
|
||||
// news bulletin.
|
||||
// Claim only when nothing below can read the world. The reason this
|
||||
// source used to claim unconditionally was that general knowledge would
|
||||
// answer "что нового?" with an invented news bulletin — true, and it
|
||||
// stopped being the only alternative on 2026-08-02, when live search
|
||||
// took the lead. With SearXNG or the ZIMs configured, "что происходит
|
||||
// в новостях про искусственный интеллект?" has a real answer below,
|
||||
// and a configuration status is the wrong thing to say instead
|
||||
// (Vikunja #474).
|
||||
if h.search != nil || h.kiwix != nil {
|
||||
return "", false
|
||||
}
|
||||
return phraser.Q(phraser.QueryFeedsOff, nil), true
|
||||
}
|
||||
// By source, not the last 200 notes of any kind: a busy day of voice notes
|
||||
@@ -322,6 +348,15 @@ func (h *reactiveHandler) queryFeeds(ctx context.Context, t *queryTurn) (string,
|
||||
// h.now(), not time.Now(): the handler's clock is the injected one, so this
|
||||
// source can be tested at a fixed time like the rest.
|
||||
func (h *reactiveHandler) queryCalendar(ctx context.Context, t *queryTurn) (string, bool) {
|
||||
// A day word is all this source matches on, so any question that merely
|
||||
// names a day reached it first. "какая сегодня погода в Москве?" answered
|
||||
// "на 02.08.2026 ничего нет." (Vikunja #474). Weather is asked about a day
|
||||
// far more often than the calendar is, and the weather source sits right
|
||||
// below, so the calendar steps aside on weather wording — the same bail-out
|
||||
// queryHome already does for the same reason.
|
||||
if isWeatherQuery(t.dec.Utterance) {
|
||||
return "", false
|
||||
}
|
||||
date, ok := router.ParseCalendarDate(t.dec.Utterance, h.now())
|
||||
if !ok {
|
||||
return "", false
|
||||
@@ -346,7 +381,7 @@ func (h *reactiveHandler) queryCalendar(ctx context.Context, t *queryTurn) (stri
|
||||
// calls States and nothing else, so there is no confirm turn here — the only
|
||||
// way to CHANGE something is an enabled allowlist row through tool.Executor.
|
||||
func (h *reactiveHandler) queryHome(ctx context.Context, t *queryTurn) (string, bool) {
|
||||
if !isHomeQuery(t.dec.Utterance) {
|
||||
if !h.turnIsAbout(ctx, t, topicHome, isHomeQuery) {
|
||||
return "", false
|
||||
}
|
||||
if h.home == nil {
|
||||
@@ -368,7 +403,7 @@ func (h *reactiveHandler) queryHome(ctx context.Context, t *queryTurn) (string,
|
||||
// because Scan takes no target — the utterance selects the question, never the
|
||||
// subnet.
|
||||
func (h *reactiveHandler) queryNetwork(ctx context.Context, t *queryTurn) (string, bool) {
|
||||
if !isNetworkQuery(t.dec.Utterance) {
|
||||
if !h.turnIsAbout(ctx, t, topicNetwork, isNetworkQuery) {
|
||||
return "", false
|
||||
}
|
||||
if h.netscan == nil {
|
||||
@@ -383,7 +418,7 @@ func (h *reactiveHandler) queryNetwork(ctx context.Context, t *queryTurn) (strin
|
||||
}
|
||||
|
||||
func (h *reactiveHandler) queryWeather(ctx context.Context, t *queryTurn) (string, bool) {
|
||||
if !isWeatherQuery(t.dec.Utterance) {
|
||||
if !h.turnIsAbout(ctx, t, topicWeather, isWeatherQuery) {
|
||||
return "", false
|
||||
}
|
||||
loc := extractWeatherLocation(t.dec.Utterance, h.weatherLocation)
|
||||
|
||||
@@ -25,7 +25,10 @@ func (h *reactiveHandler) actionReminder(ctx context.Context, dec router.Decisio
|
||||
return phraser.Ack(phraser.FailReminderTime, nil)
|
||||
}
|
||||
}
|
||||
payload := `{"text":` + jsonString(dec.Utterance) + `}`
|
||||
// The body is what she says at the hour, so the marker and the time come
|
||||
// out of it: the fire time is already a column, and "напомни" is an
|
||||
// instruction that has been carried out (Vikunja #469).
|
||||
payload := `{"text":` + jsonString(reminderBody(dec.Utterance, dec.Slots.Text)) + `}`
|
||||
if _, err := h.api.CreateReminder(ctx, dec.Slots.Time, payload, ""); err != nil {
|
||||
log.Printf("voice: create reminder: %v", err)
|
||||
return phraser.Ack(phraser.FailReminder, nil)
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"context"
|
||||
"log"
|
||||
|
||||
"github.com/kami/maven/internal/dialogue"
|
||||
"github.com/kami/maven/internal/ipc"
|
||||
"github.com/kami/maven/internal/phraser"
|
||||
"github.com/kami/maven/internal/router"
|
||||
@@ -71,7 +72,16 @@ func (h *reactiveHandler) queryTasks(ctx context.Context, t *queryTurn) (string,
|
||||
log.Printf("voice: list tasks: %v", err)
|
||||
return "не получилось посмотреть задачи.", true
|
||||
}
|
||||
return tasks.FormatRU(tasks.Rank(taskItems(live), h.now())), true
|
||||
ranked := tasks.Rank(taskItems(live), h.now())
|
||||
// Bind what she is about to say, in the order she says it, so "второй"
|
||||
// means the second task he heard (ordinal.go).
|
||||
spoken := tasks.Spoken(ranked)
|
||||
cands := make([]dialogue.Candidate, 0, len(spoken))
|
||||
for _, r := range spoken {
|
||||
cands = append(cands, dialogue.Candidate{Kind: "task", Ref: r.ID, Label: r.Text})
|
||||
}
|
||||
h.offerCandidates(cands)
|
||||
return tasks.FormatRU(ranked), true
|
||||
}
|
||||
|
||||
// taskItems maps wire rows onto the ranker's input. Written here rather than in
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
|
||||
"github.com/kami/maven/internal/router"
|
||||
)
|
||||
|
||||
// attentionMarkers — the offline floor under the attention topic (topics.go).
|
||||
// The seeds decide when the embedder is there; this answers when it is not, and
|
||||
// it stays a substring list on purpose for the reason the other floors do: a
|
||||
// narrow test made blind beats a broad guess made blind.
|
||||
//
|
||||
// "что нового" is deliberately absent: the feeds source claims it, and it
|
||||
// still should — a question about news is a question about the feeds she
|
||||
// reads. This list is about the operational state of his things.
|
||||
var attentionMarkers = []string{
|
||||
"внимани", "что требует", "что не так", "что важн", "что срочн",
|
||||
"needs attention", "what needs looking",
|
||||
}
|
||||
|
||||
// isAttentionQuery reports whether the utterance asks what needs looking at.
|
||||
// Called through turnIsAbout, never directly.
|
||||
func isAttentionQuery(u string) bool {
|
||||
s := strings.ToLower(strings.TrimSpace(u))
|
||||
if s == "" {
|
||||
return false
|
||||
}
|
||||
for _, m := range attentionMarkers {
|
||||
if strings.Contains(s, m) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// queryAttention answers "что требует внимания?" from Praxis.
|
||||
//
|
||||
// The capability was already built and already degraded correctly, and no
|
||||
// utterance could reach it (Vikunja #475). Its aliases live on the act
|
||||
// dispatch, and the question routes to IntentQuery, so it fell through every
|
||||
// source to the web search and came back with an encyclopedia article about
|
||||
// the concept of attention — worse than silence, because it reads as an
|
||||
// answer.
|
||||
//
|
||||
// Placed above the recall sources and well above the personal boundary: this
|
||||
// is operational state about his things, and a notes pass would otherwise
|
||||
// answer it from whatever he once wrote about a server. An unconfigured or
|
||||
// absent Praxis falls through rather than claiming the turn, the same
|
||||
// convention queryHome and queryNetwork follow. A Praxis that is configured
|
||||
// and down does claim it, and says it cannot reach the service — that is the
|
||||
// degradation the ecosystem contract asks for, and it comes from the same
|
||||
// handler the act path uses.
|
||||
func (h *reactiveHandler) queryAttention(ctx context.Context, t *queryTurn) (string, bool) {
|
||||
if !h.turnIsAbout(ctx, t, topicAttend, isAttentionQuery) {
|
||||
return "", false
|
||||
}
|
||||
if h.ecosystem == nil || h.ecosystem.praxis == nil {
|
||||
return "", false
|
||||
}
|
||||
reply := h.handlePraxisAct(ctx, router.Decision{
|
||||
Utterance: t.dec.Utterance,
|
||||
Intent: router.IntentAct,
|
||||
Slots: router.Slots{Fn: "list_attention", HasFn: true},
|
||||
})
|
||||
if reply == "" {
|
||||
return "", false
|
||||
}
|
||||
return reply, true
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/kami/maven/internal/router"
|
||||
)
|
||||
|
||||
func TestIsAttentionQuery(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
text string
|
||||
want bool
|
||||
}{
|
||||
{"что требует внимания", true},
|
||||
{"на что обратить внимание?", true},
|
||||
{"что не так?", true},
|
||||
{"что важного?", true},
|
||||
// The feeds source owns this one, and should keep owning it.
|
||||
{"что нового?", false},
|
||||
{"какая погода?", false},
|
||||
{"", false},
|
||||
} {
|
||||
if got := isAttentionQuery(tc.text); got != tc.want {
|
||||
t.Errorf("isAttentionQuery(%q) = %v, want %v", tc.text, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestAttentionQuestionReachesPraxis — the defect (Vikunja #475). The question
|
||||
// routes to IntentQuery, and every source used to pass, so a web search about
|
||||
// the concept of attention answered it.
|
||||
func TestAttentionQuestionReachesPraxis(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
praxis := newFakePraxis(t, fixturePraxisAttentionItems(map[string]any{
|
||||
"id": "item_1", "title": "disk almost full", "importance": 3.0,
|
||||
}))
|
||||
h := ecoHandler(t, nil, praxis, nil)
|
||||
|
||||
reply, ok := h.queryAttention(ctx, &queryTurn{dec: router.Decision{
|
||||
Intent: router.IntentQuery, Utterance: "что требует внимания",
|
||||
}})
|
||||
if !ok {
|
||||
t.Fatal("the attention question must be claimed before the world sources")
|
||||
}
|
||||
if !strings.Contains(reply, "disk almost full") {
|
||||
t.Fatalf("reply = %q, want the praxis item", reply)
|
||||
}
|
||||
}
|
||||
|
||||
// A configured Praxis that is down claims the turn and says so. Falling
|
||||
// through here would answer an outage with an encyclopedia article.
|
||||
func TestAttentionQuestionSaysWhenPraxisIsDown(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
praxis := newFakePraxis(t, fixturePraxisAttentionItems())
|
||||
h := ecoHandler(t, nil, praxis, nil)
|
||||
praxis.SetFault(503)
|
||||
|
||||
reply, ok := h.queryAttention(ctx, &queryTurn{dec: router.Decision{
|
||||
Intent: router.IntentQuery, Utterance: "что требует внимания",
|
||||
}})
|
||||
if !ok || !strings.Contains(reply, "не могу") {
|
||||
t.Fatalf("an outage must name the gap, got ok=%v reply=%q", ok, reply)
|
||||
}
|
||||
}
|
||||
|
||||
// No Praxis configured means no claim: the rest of the chain still runs.
|
||||
func TestAttentionQuestionFallsThroughWithoutPraxis(t *testing.T) {
|
||||
h, _ := newFactGateHandler(t, time.Now())
|
||||
if _, ok := h.queryAttention(context.Background(), &queryTurn{dec: router.Decision{
|
||||
Intent: router.IntentQuery, Utterance: "что требует внимания",
|
||||
}}); ok {
|
||||
t.Fatal("an unconfigured praxis must not claim the turn")
|
||||
}
|
||||
}
|
||||
+9
-16
@@ -33,19 +33,10 @@ var wantedSlots = map[router.Intent][]dialogue.Slot{
|
||||
router.IntentAct: {dialogue.SlotFn},
|
||||
}
|
||||
|
||||
// clarifyQuestions — one short question per missing slot.
|
||||
//
|
||||
// These are fixed templates, not model output. The resident model is a 0.8B; it
|
||||
// would wander, and a question whose wording changes every time is harder to
|
||||
// answer than a blunt one that always reads the same. They are infinitive
|
||||
// questions, so there is no gender agreement to get wrong; the feminine
|
||||
// self-reference lives in the reply she gives when she drops the request.
|
||||
var clarifyQuestions = map[dialogue.Slot]string{
|
||||
dialogue.SlotTime: "Когда?",
|
||||
dialogue.SlotText: "О чём напомнить?",
|
||||
dialogue.SlotKey: "Что записать?",
|
||||
dialogue.SlotFn: "Что сделать?",
|
||||
}
|
||||
// The questions themselves live in clarifytemplates.go, one list per slot,
|
||||
// picked by attempt (Vikunja #457). The first ask is the short one this map
|
||||
// used to hold; a re-ask says it differently, because a question he already
|
||||
// failed to answer is the worst one to repeat unchanged.
|
||||
|
||||
// clarifyGaveUp — she is out of questions and still does not have the slot. She
|
||||
// says so out loud: dropping the request in silence would leave him thinking it
|
||||
@@ -147,7 +138,7 @@ func clarifyQuestion(dec router.Decision) (dialogue.Slot, string, bool) {
|
||||
if len(missing) == 0 {
|
||||
return "", "", false
|
||||
}
|
||||
q, ok := clarifyQuestions[missing[0]]
|
||||
q, ok := clarifyQuestionFor(missing[0], 1)
|
||||
if !ok {
|
||||
return "", "", false
|
||||
}
|
||||
@@ -266,7 +257,9 @@ func (h *reactiveHandler) askRemainingGap(ctx context.Context, q *dialogue.Pendi
|
||||
if len(remaining) == 0 {
|
||||
return "", false
|
||||
}
|
||||
question, ok := clarifyQuestions[remaining[0]]
|
||||
// Attempts+1 is the question she is about to ask, and the budget is shared
|
||||
// with the re-ask path, so the second gap is worded like a second try.
|
||||
question, ok := clarifyQuestionFor(remaining[0], q.Attempts+1)
|
||||
if !ok || !q.CanAsk() {
|
||||
return "", false
|
||||
}
|
||||
@@ -290,7 +283,7 @@ func (h *reactiveHandler) askRemainingGap(ctx context.Context, q *dialogue.Pendi
|
||||
func (h *reactiveHandler) reaskOrGiveUp(ctx context.Context, q *dialogue.PendingQuestion, merged dialogue.Slots, text string) string {
|
||||
question := ""
|
||||
if len(q.Missing) > 0 {
|
||||
question = clarifyQuestions[q.Missing[0]]
|
||||
question, _ = clarifyQuestionFor(q.Missing[0], q.Attempts+1)
|
||||
}
|
||||
if question == "" || !q.CanAsk() {
|
||||
h.clarifyStore.Delete(dialogueIDOf(ctx))
|
||||
|
||||
@@ -156,8 +156,14 @@ func TestClarifyAsksThreeTimesThenSaysSo(t *testing.T) {
|
||||
if !handled {
|
||||
t.Fatalf("answer %d must be consumed as an answer", i)
|
||||
}
|
||||
if reply != "Когда?" {
|
||||
t.Fatalf("attempt %d should ask again, got %q", i, reply)
|
||||
// The wording changes with the attempt (Vikunja #457): repeating a
|
||||
// question he already failed to answer is the worst way to ask it.
|
||||
want, _ := clarifyQuestionFor(dialogue.SlotTime, i)
|
||||
if reply != want {
|
||||
t.Fatalf("attempt %d should ask again as %q, got %q", i, want, reply)
|
||||
}
|
||||
if first, _ := clarifyQuestionFor(dialogue.SlotTime, 1); reply == first {
|
||||
t.Fatalf("attempt %d repeated the first wording: %q", i, reply)
|
||||
}
|
||||
if h.clarifyStore.Get(voiceDialogueID, h.now()) == nil {
|
||||
t.Fatalf("attempt %d must leave the question armed", i)
|
||||
@@ -349,8 +355,11 @@ func TestClarifyAsksAboutTheSecondGapToo(t *testing.T) {
|
||||
if !handled {
|
||||
t.Fatal("the answer must be consumed as an answer")
|
||||
}
|
||||
if reply != "Когда?" {
|
||||
t.Fatalf("a filled subject with no time must ask about the time, got %q", reply)
|
||||
// Second gap, second attempt, so it is the second wording of the time
|
||||
// question — the attempt budget is shared between the two paths.
|
||||
want, _ := clarifyQuestionFor(dialogue.SlotTime, 2)
|
||||
if reply != want {
|
||||
t.Fatalf("a filled subject with no time must ask about the time as %q, got %q", want, reply)
|
||||
}
|
||||
q := h.clarifyStore.Get(voiceDialogueID, h.now())
|
||||
if q == nil {
|
||||
@@ -411,8 +420,9 @@ func TestClarifyProseHoldsThePersona(t *testing.T) {
|
||||
eval.CheckCringe: true,
|
||||
}
|
||||
lines := append([]string{clarifyGaveUp}, clarifyExpiredVariants...)
|
||||
for _, q := range clarifyQuestions {
|
||||
lines = append(lines, q)
|
||||
lines = append(lines, clarifyMissedVariants...)
|
||||
for _, variants := range clarifyQuestionVariants {
|
||||
lines = append(lines, variants...)
|
||||
}
|
||||
for _, line := range lines {
|
||||
for _, r := range eval.RunChecks(eval.Case{}, line, "neutral") {
|
||||
@@ -497,8 +507,12 @@ func TestClarifySubjectAnswerFillsRatherThanClobbers(t *testing.T) {
|
||||
if !strings.Contains(reminders[0].Payload, "маме") {
|
||||
t.Fatalf("the answer never reached the reminder: %q", reminders[0].Payload)
|
||||
}
|
||||
if !strings.Contains(reminders[0].Payload, "11") {
|
||||
t.Fatalf("the answer clobbered the original request: %q", reminders[0].Payload)
|
||||
// The hour is the fire time, not a word in the body: the body is what she
|
||||
// says at the hour, and the time expression is stripped out of it
|
||||
// (Vikunja #469). Clobbering the parked request would show up here as a
|
||||
// reminder that fires at some other time than the one he asked for.
|
||||
if got := reminders[0].FireTs.UTC(); !got.Equal(at.UTC()) {
|
||||
t.Fatalf("the answer clobbered the original request: fires at %v, want %v", got, at.UTC())
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"github.com/kami/maven/internal/dialogue"
|
||||
"github.com/kami/maven/internal/router"
|
||||
)
|
||||
|
||||
// The clarify copy deck (Vikunja #457).
|
||||
//
|
||||
// Every clarify turn used to say one sentence per gap, and a re-ask repeated
|
||||
// that sentence word for word. A question he already failed to answer is the
|
||||
// worst one to ask again unchanged: the second wording is the one that tells
|
||||
// him which part she missed.
|
||||
//
|
||||
// Fixed templates, not model output, for the reason clarifyQuestions has always
|
||||
// given: the resident model would wander, and a question whose wording changes
|
||||
// at random is harder to answer than a blunt one. What changes here is that the
|
||||
// wording varies with the attempt rather than with a die roll — the first ask is
|
||||
// short, the second names the gap, the third spells it out.
|
||||
//
|
||||
// No schema_version, unlike internal/phraser/nudge_templates.go. These are Go
|
||||
// constants compiled into the daemon, so there is no file that can drift out of
|
||||
// step with the code that reads it.
|
||||
//
|
||||
// Persona holds: infinitive and imperative questions, so there is no gender
|
||||
// agreement to get wrong, "ты" throughout, and no pet names.
|
||||
var clarifyQuestionVariants = map[dialogue.Slot][]string{
|
||||
dialogue.SlotTime: {
|
||||
"Когда?",
|
||||
"Во сколько напомнить?",
|
||||
"Скажи время — например, «в семь вечера» или «через час».",
|
||||
},
|
||||
dialogue.SlotText: {
|
||||
"О чём напомнить?",
|
||||
"Что сказать тебе в это время?",
|
||||
"Скажи одной фразой, о чём напомнить.",
|
||||
},
|
||||
dialogue.SlotKey: {
|
||||
"Что записать?",
|
||||
"Что именно отметить?",
|
||||
"Назови, что записать — например, «выпил воды».",
|
||||
},
|
||||
dialogue.SlotFn: {
|
||||
"Что сделать?",
|
||||
"Какое действие выполнить?",
|
||||
"Назови действие — я умею только то, что ты мне разрешил.",
|
||||
},
|
||||
}
|
||||
|
||||
// clarifyQuestionFor picks the wording for this attempt. attempt is 1-based, as
|
||||
// PendingQuestion.Attempts counts it; anything past the list uses the last and
|
||||
// most explicit phrasing rather than wrapping round to the short one, because
|
||||
// wrapping would ask the same short question he has already not answered.
|
||||
//
|
||||
// Deterministic on purpose, unlike clarifyExpiredLine: an expiry notice is the
|
||||
// same statement however it is worded, and a re-ask is not.
|
||||
func clarifyQuestionFor(slot dialogue.Slot, attempt int) (string, bool) {
|
||||
variants, ok := clarifyQuestionVariants[slot]
|
||||
if !ok || len(variants) == 0 {
|
||||
return "", false
|
||||
}
|
||||
i := attempt - 1
|
||||
if i < 0 {
|
||||
i = 0
|
||||
}
|
||||
if i >= len(variants) {
|
||||
i = len(variants) - 1
|
||||
}
|
||||
return variants[i], true
|
||||
}
|
||||
|
||||
// clarifyMissedVariants — she is asking for the whole utterance again, because
|
||||
// the gate fired on an intent with nothing identifiable to ask about (note,
|
||||
// query, chat, system are not in wantedSlots).
|
||||
//
|
||||
// Rotated like the expiry lines and for the same reason: this is the line he
|
||||
// hears whenever she misses him completely, so it is a line that repeats, and
|
||||
// the same sentence every time is what makes a house assistant sound like a
|
||||
// kiosk. All of them say the same two things — she did not catch it, and he
|
||||
// should say it again — because the wording may vary and the meaning may not.
|
||||
var clarifyMissedVariants = []string{
|
||||
"Не совсем поняла — скажи, пожалуйста, ещё раз.",
|
||||
"Я тебя не разобрала. Повтори, пожалуйста.",
|
||||
"Не уловила. Скажи это по-другому?",
|
||||
"Прости, не поняла — попробуй сказать иначе.",
|
||||
}
|
||||
|
||||
// clarifyMissedFor picks a wording by the utterance itself, so the same words
|
||||
// asked twice get the same answer and two different misses sound different.
|
||||
//
|
||||
// A hash, not rand: a test that drives an utterance twice must not depend on a
|
||||
// die roll, and the point of rotating is only that consecutive misses differ.
|
||||
func clarifyMissedFor(utterance string) string {
|
||||
var sum int
|
||||
for _, r := range utterance {
|
||||
sum += int(r)
|
||||
}
|
||||
return clarifyMissedVariants[sum%len(clarifyMissedVariants)]
|
||||
}
|
||||
|
||||
// clarifyMissedLine is the canned reply for a clarify decision she cannot turn
|
||||
// into a question. Returns "" for a decision that is not a clarify, so the
|
||||
// caller keeps its own reply.
|
||||
func clarifyMissedLine(dec router.Decision) string {
|
||||
if !dec.Clarify {
|
||||
return ""
|
||||
}
|
||||
return clarifyMissedFor(dec.Utterance)
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/kami/maven/internal/dialogue"
|
||||
"github.com/kami/maven/internal/router"
|
||||
)
|
||||
|
||||
// Every slot she can ask about has a wording for every attempt she is allowed,
|
||||
// and no two attempts on one slot read the same. A deck with a repeated line is
|
||||
// the defect this deck exists to fix (Vikunja #457).
|
||||
func TestClarifyQuestionsVaryByAttempt(t *testing.T) {
|
||||
for slot, variants := range clarifyQuestionVariants {
|
||||
seen := map[string]bool{}
|
||||
for _, v := range variants {
|
||||
if v == "" {
|
||||
t.Errorf("%s: empty wording in the deck", slot)
|
||||
}
|
||||
if seen[v] {
|
||||
t.Errorf("%s: repeated wording %q", slot, v)
|
||||
}
|
||||
seen[v] = true
|
||||
}
|
||||
for attempt := 1; attempt <= len(variants); attempt++ {
|
||||
got, ok := clarifyQuestionFor(slot, attempt)
|
||||
if !ok || got != variants[attempt-1] {
|
||||
t.Errorf("%s attempt %d = %q ok=%v, want %q", slot, attempt, got, ok, variants[attempt-1])
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Past the end she keeps the most explicit wording. Wrapping round would ask
|
||||
// the short question he has already not answered twice.
|
||||
func TestClarifyQuestionPastTheEndKeepsTheLastWording(t *testing.T) {
|
||||
last := clarifyQuestionVariants[dialogue.SlotTime][len(clarifyQuestionVariants[dialogue.SlotTime])-1]
|
||||
for _, attempt := range []int{0, 4, 9} {
|
||||
if got, _ := clarifyQuestionFor(dialogue.SlotTime, attempt); attempt > 1 && got != last {
|
||||
t.Errorf("attempt %d = %q, want the last wording %q", attempt, got, last)
|
||||
}
|
||||
}
|
||||
if _, ok := clarifyQuestionFor("nonesuch", 1); ok {
|
||||
t.Error("an unknown slot must have no question")
|
||||
}
|
||||
}
|
||||
|
||||
// The missed line is stable for one utterance and absent for a decision that is
|
||||
// not a clarify.
|
||||
func TestClarifyMissedLine(t *testing.T) {
|
||||
d := router.Decision{Clarify: true, Utterance: "мгм"}
|
||||
first := clarifyMissedLine(d)
|
||||
if first == "" || first != clarifyMissedLine(d) {
|
||||
t.Fatalf("the missed line must be stable for one utterance, got %q", first)
|
||||
}
|
||||
if got := clarifyMissedLine(router.Decision{Intent: router.IntentNote}); got != "" {
|
||||
t.Errorf("a decision that is not a clarify got %q", got)
|
||||
}
|
||||
// The empty utterance still gets a line: she has to say something.
|
||||
if got := clarifyMissedLine(router.Decision{Clarify: true}); got == "" {
|
||||
t.Error("an empty utterance must still be answered out loud")
|
||||
}
|
||||
}
|
||||
@@ -123,3 +123,48 @@ func mustEmbedPassage(t *testing.T, h *reactiveHandler, text string) []float32 {
|
||||
}
|
||||
return vec
|
||||
}
|
||||
|
||||
// The write half of #481: a complaint about a thing is a state of the
|
||||
// afternoon, not a fact about him. Stored as a `self` row at confidence 1.00
|
||||
// it comes back on recall as if the network were still down.
|
||||
func TestActionFact_ComplaintIsNotWritten(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
h, api := newFactGateHandler(t, time.Now())
|
||||
|
||||
reply := h.actionFact(ctx, router.Decision{
|
||||
Intent: router.IntentFact,
|
||||
Utterance: "сеть какая-то медленная",
|
||||
Slots: router.Slots{Key: "network_speed", HasKey: true, Value: "медленная"},
|
||||
})
|
||||
|
||||
if _, err := api.LatestFact(ctx, "network_speed"); err == nil {
|
||||
t.Fatal("a passing complaint was stored as a fact about him")
|
||||
}
|
||||
hits, err := h.memStore.Search(ctx, mustEmbedPassage(t, h, "сеть какая-то медленная"), 3)
|
||||
if err != nil {
|
||||
t.Fatalf("memory search: %v", err)
|
||||
}
|
||||
if len(hits) != 0 {
|
||||
t.Fatalf("the complaint was indexed for recall: %+v", hits)
|
||||
}
|
||||
if reply == "" {
|
||||
t.Fatal("the turn was neither stored nor answered")
|
||||
}
|
||||
}
|
||||
|
||||
// And the complaint he asked her to keep: the capture verb wins, as it does
|
||||
// over the question gate.
|
||||
func TestActionFact_AskedToRememberAComplaintStillWrites(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
h, api := newFactGateHandler(t, time.Now())
|
||||
|
||||
h.actionFact(ctx, router.Decision{
|
||||
Intent: router.IntentFact,
|
||||
Utterance: "запомни что интернет не работает",
|
||||
Slots: router.Slots{Key: "internet", HasKey: true, Value: "не работает"},
|
||||
})
|
||||
|
||||
if _, err := api.LatestFact(ctx, "internet"); err != nil {
|
||||
t.Fatalf("an explicit capture was refused: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -200,3 +200,22 @@ func TestFeedWorkerFetcherIsAllowlisted(t *testing.T) {
|
||||
t.Fatal("the poller fetched a private address")
|
||||
}
|
||||
}
|
||||
|
||||
// TestQueryFeedsPassesWhenTheWorldCanAnswer — the defect (Vikunja #474). The
|
||||
// deployed box has no feeds block and does have SearXNG, and "что происходит
|
||||
// сейчас в новостях про искусственный интеллект?" got a configuration status
|
||||
// instead of the live answer sitting one source below.
|
||||
func TestQueryFeedsPassesWhenTheWorldCanAnswer(t *testing.T) {
|
||||
h := buildFeedHandler(t, false)
|
||||
h.search = &searchWiring{max: 3, runes: 1500}
|
||||
|
||||
if reply, ok := askFeeds(t, h, "что нового в лентах?"); ok {
|
||||
t.Fatalf("feeds off with a search configured must fall through, got %q", reply)
|
||||
}
|
||||
// With nothing below that reads the world, the honest status is still said:
|
||||
// general knowledge would otherwise answer with an invented bulletin.
|
||||
h.search = nil
|
||||
if reply, ok := askFeeds(t, h, "что нового в лентах?"); !ok || !strings.Contains(reply, "не настроены") {
|
||||
t.Fatalf("no search and no ZIMs: reply = %q, ok = %v", reply, ok)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,117 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Command history — "что я тебе говорил?", "что ты записала сегодня?"
|
||||
// (Vikunja #456).
|
||||
//
|
||||
// Read-only over the facts that already exist. No new mechanism and no new
|
||||
// storage: everything he tapped in is already a row with a source and a
|
||||
// timestamp, and this only reads them back.
|
||||
|
||||
// historyMarkers — the ways he asks what he told her. Each entry is a pair of
|
||||
// substrings that must BOTH appear, because either half alone is a different
|
||||
// question: "что я говорил про сервер" is a recall question the notes pass
|
||||
// answers better, and "что ты записала" with no "что" is not a question at all.
|
||||
var historyMarkers = [][2]string{
|
||||
{"что я", "говорил"},
|
||||
{"что я", "сказал"},
|
||||
{"что я", "рассказ"},
|
||||
{"что ты", "записал"},
|
||||
{"что ты", "запомнил"},
|
||||
{"что я", "отмечал"},
|
||||
{"что я", "отметил"},
|
||||
{"what did i", "tell"},
|
||||
{"what did you", "record"},
|
||||
}
|
||||
|
||||
// historyRecall — the word that turns a history question into a recall
|
||||
// question. "что я говорил про сервер" names a topic, and the notes pass
|
||||
// answers a topic far better than a list of the last five facts does.
|
||||
var historyRecall = []string{" про ", " об ", " о ", " about "}
|
||||
|
||||
// isHistoryQuery reports whether he is asking what he told her.
|
||||
func isHistoryQuery(u string) bool {
|
||||
s := " " + strings.ToLower(strings.TrimSpace(u)) + " "
|
||||
if s == " " {
|
||||
return false
|
||||
}
|
||||
for _, r := range historyRecall {
|
||||
if strings.Contains(s, r) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
for _, pair := range historyMarkers {
|
||||
if strings.Contains(s, pair[0]) && strings.Contains(s, pair[1]) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// historyScan — how many recent facts are read before filtering. Deliberately
|
||||
// larger than historyReadOut: a poller writing every few minutes would
|
||||
// otherwise push everything he said out of the window, the same way his own
|
||||
// notes used to crowd out the feed headlines.
|
||||
const historyScan = 100
|
||||
|
||||
// historyReadOut — how many she says out loud. Five is what fits in one spoken
|
||||
// breath; the rest are on /history, which is the surface for reading a list.
|
||||
const historyReadOut = 5
|
||||
|
||||
// historyWindow — how far back "recently" reaches. A day, because the question
|
||||
// is about this conversation and not about the archive.
|
||||
const historyWindow = 24 * time.Hour
|
||||
|
||||
// queryHistory answers what he told her, from the facts he tapped in.
|
||||
//
|
||||
// Only "tap:" sources. A fact written by a poller, an inference or the ambient
|
||||
// relay is a thing she learned, not a thing he said, and reading those back
|
||||
// under "что я тебе говорил?" would put words in his mouth.
|
||||
//
|
||||
// Placed with the other sources that read his own rows and above the recall
|
||||
// pass: the notes pass would otherwise answer this from whatever note happens
|
||||
// to be nearest, which reads as an answer and is not one.
|
||||
func (h *reactiveHandler) queryHistory(ctx context.Context, t *queryTurn) (string, bool) {
|
||||
if !isHistoryQuery(t.dec.Utterance) {
|
||||
return "", false
|
||||
}
|
||||
facts, err := h.api.RecentFacts(ctx, historyScan)
|
||||
if err != nil {
|
||||
log.Printf("voice: history: recent facts: %v", err)
|
||||
return "не получилось посмотреть, что ты говорил.", true
|
||||
}
|
||||
cutoff := h.now().Add(-historyWindow)
|
||||
var said []string
|
||||
for _, f := range facts {
|
||||
if !strings.HasPrefix(f.Source, "tap:") || f.Ts.Before(cutoff) {
|
||||
continue
|
||||
}
|
||||
said = append(said, historyLine(f.Key, f.Value, f.Ts))
|
||||
if len(said) == historyReadOut {
|
||||
break
|
||||
}
|
||||
}
|
||||
if len(said) == 0 {
|
||||
// Claim the turn rather than fall through. "ничего не говорил" is the
|
||||
// true answer, and recall would answer it with an old note instead.
|
||||
return "за последние сутки ты мне ничего такого не говорил.", true
|
||||
}
|
||||
return "ты говорил: " + strings.Join(said, "; "), true
|
||||
}
|
||||
|
||||
// historyLine — one fact as she says it. The hour and minute, because the day
|
||||
// is already bounded by historyWindow and a date would be noise.
|
||||
func historyLine(key, value string, ts time.Time) string {
|
||||
what := key
|
||||
if value != "" {
|
||||
what = key + " — " + value
|
||||
}
|
||||
return fmt.Sprintf("%s (%s)", what, ts.Local().Format("15:04"))
|
||||
}
|
||||
@@ -0,0 +1,104 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/kami/maven/internal/ipc"
|
||||
"github.com/kami/maven/internal/router"
|
||||
)
|
||||
|
||||
// historyAPI serves a fixed set of recent facts.
|
||||
type historyAPI struct {
|
||||
ipc.UnimplementedCoreAPI
|
||||
facts []ipc.Fact
|
||||
calls int
|
||||
}
|
||||
|
||||
func (a *historyAPI) RecentFacts(context.Context, int) ([]ipc.Fact, error) {
|
||||
a.calls++
|
||||
return a.facts, nil
|
||||
}
|
||||
|
||||
func historyHandler(now time.Time, facts ...ipc.Fact) (*reactiveHandler, *historyAPI) {
|
||||
api := &historyAPI{facts: facts}
|
||||
return &reactiveHandler{api: api, now: func() time.Time { return now }}, api
|
||||
}
|
||||
|
||||
func askHistory(h *reactiveHandler, u string) (string, bool) {
|
||||
return h.queryHistory(context.Background(), &queryTurn{
|
||||
dec: router.Decision{Intent: router.IntentQuery, Utterance: u},
|
||||
})
|
||||
}
|
||||
|
||||
func TestIsHistoryQuery(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
text string
|
||||
want bool
|
||||
}{
|
||||
{"что я тебе говорил?", true},
|
||||
{"что ты записала сегодня?", true},
|
||||
{"что я отмечал?", true},
|
||||
// A named topic is a recall question, and the notes pass answers it
|
||||
// better than a list of the last five facts does.
|
||||
{"что я говорил про сервер?", false},
|
||||
{"что у меня сегодня?", false},
|
||||
{"", false},
|
||||
} {
|
||||
if got := isHistoryQuery(tc.text); got != tc.want {
|
||||
t.Errorf("isHistoryQuery(%q) = %v, want %v", tc.text, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestHistoryReadsOnlyWhatHeSaid(t *testing.T) {
|
||||
now := time.Date(2026, 8, 4, 20, 0, 0, 0, time.UTC)
|
||||
h, api := historyHandler(now,
|
||||
ipc.Fact{Key: "water", Value: "выпил", Source: "tap:voice", Ts: now.Add(-time.Hour)},
|
||||
// Learned, not said: a poller writing this back under "что я тебе
|
||||
// говорил?" would put words in his mouth.
|
||||
ipc.Fact{Key: "spent_today", Value: "1200", Source: "poll:zenmoney", Ts: now.Add(-time.Hour)},
|
||||
// Older than the window.
|
||||
ipc.Fact{Key: "shower", Value: "принял", Source: "tap:voice", Ts: now.Add(-30 * time.Hour)},
|
||||
)
|
||||
reply, ok := askHistory(h, "что я тебе говорил?")
|
||||
if !ok {
|
||||
t.Fatal("the history question must be claimed before the recall sources")
|
||||
}
|
||||
if !strings.Contains(reply, "water") {
|
||||
t.Errorf("reply = %q, want the fact he tapped in", reply)
|
||||
}
|
||||
if strings.Contains(reply, "spent_today") || strings.Contains(reply, "shower") {
|
||||
t.Errorf("reply = %q, want only what he said inside the window", reply)
|
||||
}
|
||||
if api.calls != 1 {
|
||||
t.Errorf("RecentFacts called %d times, want 1", api.calls)
|
||||
}
|
||||
}
|
||||
|
||||
// Nothing said is an answer of its own. Falling through would hand the question
|
||||
// to recall, which answers it with an old note.
|
||||
func TestHistorySaysWhenThereIsNothing(t *testing.T) {
|
||||
now := time.Date(2026, 8, 4, 20, 0, 0, 0, time.UTC)
|
||||
h, _ := historyHandler(now)
|
||||
reply, ok := askHistory(h, "что я тебе говорил?")
|
||||
if !ok || !strings.Contains(reply, "ничего") {
|
||||
t.Fatalf("reply = %q, ok = %v", reply, ok)
|
||||
}
|
||||
}
|
||||
|
||||
// Five is what fits in one spoken breath; the rest are on /history.
|
||||
func TestHistoryStopsAtFive(t *testing.T) {
|
||||
now := time.Date(2026, 8, 4, 20, 0, 0, 0, time.UTC)
|
||||
var facts []ipc.Fact
|
||||
for i := 0; i < 12; i++ {
|
||||
facts = append(facts, ipc.Fact{Key: "k", Value: "v", Source: "tap:voice", Ts: now.Add(-time.Minute)})
|
||||
}
|
||||
h, _ := historyHandler(now, facts...)
|
||||
reply, _ := askHistory(h, "что ты записала?")
|
||||
if got := strings.Count(reply, ";"); got != historyReadOut-1 {
|
||||
t.Fatalf("reply = %q has %d separators, want %d", reply, got, historyReadOut-1)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,145 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log"
|
||||
"strings"
|
||||
"unicode"
|
||||
|
||||
"github.com/kami/maven/internal/dialogue"
|
||||
"github.com/kami/maven/internal/store"
|
||||
)
|
||||
|
||||
// Ordinal selection over a list she just read (Vikunja #448).
|
||||
//
|
||||
// The dialogue session already carried the intent, the slots and the history.
|
||||
// What it did not carry was the list: she recited five tasks, he said "второй",
|
||||
// and there was nothing for that word to point at, so it routed as a fresh
|
||||
// utterance and meant nothing.
|
||||
//
|
||||
// Candidates are bound when she speaks the list, in the order she spoke it (see
|
||||
// tasks.Spoken). Binding afterwards would resolve "второй" against a fresh
|
||||
// query, and the list can change between two turns.
|
||||
//
|
||||
// An ordinal with no verb is read back, not acted on: "второй" names a task, it
|
||||
// does not say what to do with it. Acting on the bare word would guess, and a
|
||||
// wrong guess here closes work he never finished.
|
||||
|
||||
// candidateOrdinals — the words that pick a position, by index. Prefix match,
|
||||
// because Russian declines them: "первый", "первую", "первое".
|
||||
var candidateOrdinals = []struct {
|
||||
word string
|
||||
nth int
|
||||
}{
|
||||
{"перв", 1}, {"втор", 2}, {"трет", 3}, {"четв", 4}, {"пят", 5},
|
||||
{"first", 1}, {"second", 2}, {"third", 3},
|
||||
}
|
||||
|
||||
// candidateDigits — "второй" said as a number. Matched whole, never by prefix:
|
||||
// "15" starts with "1" and is a time, not a position.
|
||||
var candidateDigits = map[string]int{"1": 1, "2": 2, "3": 3, "4": 4, "5": 5}
|
||||
|
||||
// candidateLast — "последний" picks the end of the list whatever its length.
|
||||
var candidateLast = []string{"последн", "last"}
|
||||
|
||||
// parseOrdinal reads which position he named. 0 and false when he named none.
|
||||
// A negative result means the last one.
|
||||
func parseOrdinal(text string) (int, bool) {
|
||||
// Token by token, not substring: " 1" would otherwise match inside
|
||||
// "напомни в 15:00" and turn a reminder into a selection.
|
||||
for _, tok := range strings.FieldsFunc(strings.ToLower(text), func(r rune) bool {
|
||||
return !unicode.IsLetter(r) && !unicode.IsDigit(r)
|
||||
}) {
|
||||
for _, w := range candidateLast {
|
||||
if strings.HasPrefix(tok, w) {
|
||||
return -1, true
|
||||
}
|
||||
}
|
||||
if n, ok := candidateDigits[tok]; ok {
|
||||
return n, true
|
||||
}
|
||||
for _, o := range candidateOrdinals {
|
||||
// Prefix, because Russian declines them: "первый", "первую".
|
||||
if strings.HasPrefix(tok, o.word) {
|
||||
return o.nth, true
|
||||
}
|
||||
}
|
||||
}
|
||||
return 0, false
|
||||
}
|
||||
|
||||
// candidateVerbs — what he wants done with the one he picked. Nothing here is
|
||||
// destructive: a task moves forward or is dropped, and both are recorded with a
|
||||
// provenance the /tasks page shows.
|
||||
var candidateVerbs = []struct {
|
||||
words []string
|
||||
status string
|
||||
say string
|
||||
}{
|
||||
{[]string{"готов", "сделал", "выполнил", "закрыл", "done"}, store.TaskDone, "закрыла"},
|
||||
{[]string{"не надо", "убери", "отмени", "не буду", "drop"}, store.TaskDropped, "убрала"},
|
||||
{[]string{"подтвержда", "беру", "да,", "буду делать"}, store.TaskOpen, "взяла в работу"},
|
||||
}
|
||||
|
||||
func parseCandidateVerb(text string) (status, say string, ok bool) {
|
||||
s := strings.ToLower(strings.TrimSpace(text))
|
||||
for _, v := range candidateVerbs {
|
||||
for _, w := range v.words {
|
||||
if strings.Contains(s, w) {
|
||||
return v.status, v.say, true
|
||||
}
|
||||
}
|
||||
}
|
||||
return "", "", false
|
||||
}
|
||||
|
||||
// offerCandidates records the list she just read, so his next words can pick
|
||||
// from it. Best effort: no session store, or a session that expired between the
|
||||
// question and the answer, means the words route normally.
|
||||
func (h *reactiveHandler) offerCandidates(cands []dialogue.Candidate) {
|
||||
if h.dialogueSessions == nil || len(cands) == 0 {
|
||||
return
|
||||
}
|
||||
h.dialogueSessions.SetCandidates(voiceDialogueID, h.now(), cands)
|
||||
}
|
||||
|
||||
// resolveCandidate handles "второй", "первую сделал", "последнюю убери" against
|
||||
// the list she just read.
|
||||
func (h *reactiveHandler) resolveCandidate(ctx context.Context, text string) (string, bool) {
|
||||
if h.dialogueSessions == nil {
|
||||
return "", false
|
||||
}
|
||||
sess := h.dialogueSessions.Get(voiceDialogueID, h.now())
|
||||
if sess == nil || len(sess.Candidates) == 0 {
|
||||
return "", false
|
||||
}
|
||||
nth, ok := parseOrdinal(text)
|
||||
if !ok {
|
||||
return "", false
|
||||
}
|
||||
if nth < 0 {
|
||||
nth = len(sess.Candidates)
|
||||
}
|
||||
if nth > len(sess.Candidates) {
|
||||
// Claim the turn: he is picking from her list and named a position she
|
||||
// did not read. Routing it fresh would answer something else entirely.
|
||||
return fmt.Sprintf("я назвала только %d.", len(sess.Candidates)), true
|
||||
}
|
||||
pick := sess.Candidates[nth-1]
|
||||
status, say, hasVerb := parseCandidateVerb(text)
|
||||
if !hasVerb || pick.Kind != "task" {
|
||||
// Read it back and keep the list: naming one is often the first half of
|
||||
// a sentence, and the second half is the next turn.
|
||||
return pick.Label, true
|
||||
}
|
||||
if err := h.api.SetTaskStatus(ctx, pick.Ref, status, h.now(), "tap:voice"); err != nil {
|
||||
log.Printf("voice: candidate %d → %s: %v", pick.Ref, status, err)
|
||||
return "не получилось изменить задачу.", true
|
||||
}
|
||||
// Spent: the list she read is no longer the list, and a second ordinal
|
||||
// against it would close the wrong task.
|
||||
h.dialogueSessions.SetCandidates(voiceDialogueID, h.now(), nil)
|
||||
log.Printf("voice: candidate %d (%q) → %s", pick.Ref, pick.Label, status)
|
||||
return say + ": " + pick.Label, true
|
||||
}
|
||||
@@ -0,0 +1,128 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/kami/maven/internal/dialogue"
|
||||
"github.com/kami/maven/internal/store"
|
||||
)
|
||||
|
||||
func TestParseOrdinalReadsThePosition(t *testing.T) {
|
||||
cases := []struct {
|
||||
text string
|
||||
want int
|
||||
ok bool
|
||||
}{
|
||||
{"второй", 2, true},
|
||||
{"вторую сделал", 2, true},
|
||||
{"первую убери", 1, true},
|
||||
{"последнюю не надо", -1, true},
|
||||
{"3", 3, true},
|
||||
{"the second one", 2, true},
|
||||
// No position named.
|
||||
{"какие у меня задачи", 0, false},
|
||||
{"", 0, false},
|
||||
// A digit inside a time is not a position.
|
||||
{"напомни в 15:00", 0, false},
|
||||
}
|
||||
for _, c := range cases {
|
||||
got, ok := parseOrdinal(c.text)
|
||||
if ok != c.ok || (ok && got != c.want) {
|
||||
t.Errorf("parseOrdinal(%q) = %d,%v; want %d,%v", c.text, got, ok, c.want, c.ok)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestOrdinalPassesWithNothingOffered(t *testing.T) {
|
||||
h, _, _ := newClarifyHandler(t)
|
||||
if _, handled := h.resolveCandidate(context.Background(), "второй"); handled {
|
||||
t.Error("an ordinal with no list behind it was claimed")
|
||||
}
|
||||
}
|
||||
|
||||
func TestOrdinalReadsBackWithoutAVerb(t *testing.T) {
|
||||
h, st, _ := newClarifyHandler(t)
|
||||
ctx := context.Background()
|
||||
ids := seedTasks(t, st, "купить хлеб", "позвонить маме")
|
||||
putCandidates(h, ids, "купить хлеб", "позвонить маме")
|
||||
|
||||
reply, handled := h.resolveCandidate(ctx, "второй")
|
||||
if !handled || !strings.Contains(reply, "позвонить маме") {
|
||||
t.Fatalf("a bare ordinal did not read the task back: %q handled=%v", reply, handled)
|
||||
}
|
||||
// Still live: naming one is often the first half of a sentence.
|
||||
if _, handled := h.resolveCandidate(ctx, "первый"); !handled {
|
||||
t.Error("the list was spent by a read-back")
|
||||
}
|
||||
}
|
||||
|
||||
func TestOrdinalWithAVerbMovesTheTask(t *testing.T) {
|
||||
h, st, _ := newClarifyHandler(t)
|
||||
ctx := context.Background()
|
||||
ids := seedTasks(t, st, "купить хлеб", "позвонить маме")
|
||||
putCandidates(h, ids, "купить хлеб", "позвонить маме")
|
||||
|
||||
reply, handled := h.resolveCandidate(ctx, "первую сделал")
|
||||
if !handled || !strings.Contains(reply, "купить хлеб") {
|
||||
t.Fatalf("the pick was not acted on: %q handled=%v", reply, handled)
|
||||
}
|
||||
live, err := st.ListTasks(ctx, "live")
|
||||
if err != nil {
|
||||
t.Fatalf("list tasks: %v", err)
|
||||
}
|
||||
for _, task := range live {
|
||||
if task.ID == ids[0] {
|
||||
t.Fatalf("task %d is still live after he closed it", task.ID)
|
||||
}
|
||||
}
|
||||
// Spent: a second ordinal against a list that no longer holds would close
|
||||
// the wrong task.
|
||||
if _, handled := h.resolveCandidate(ctx, "второй"); handled {
|
||||
t.Error("the list survived the pick it was spent on")
|
||||
}
|
||||
}
|
||||
|
||||
func TestOrdinalPastTheEndSaysHowMany(t *testing.T) {
|
||||
h, st, _ := newClarifyHandler(t)
|
||||
ids := seedTasks(t, st, "купить хлеб")
|
||||
putCandidates(h, ids, "купить хлеб")
|
||||
|
||||
reply, handled := h.resolveCandidate(context.Background(), "третий")
|
||||
if !handled || !strings.Contains(reply, "1") {
|
||||
t.Fatalf("a position she never read was not answered: %q handled=%v", reply, handled)
|
||||
}
|
||||
}
|
||||
|
||||
// ordinalNow — a fixed capture time; the ranker only needs the rows to exist.
|
||||
var ordinalNow = time.Date(2026, 8, 4, 9, 0, 0, 0, time.UTC)
|
||||
|
||||
func seedTasks(t *testing.T, st *store.Store, texts ...string) []int64 {
|
||||
t.Helper()
|
||||
ctx := context.Background()
|
||||
var ids []int64
|
||||
for _, text := range texts {
|
||||
res, err := st.CaptureTask(ctx, store.Task{
|
||||
Text: text,
|
||||
Source: "tap:voice",
|
||||
Status: store.TaskOpen,
|
||||
CreatedTs: ordinalNow,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("capture task: %v", err)
|
||||
}
|
||||
ids = append(ids, res.ID)
|
||||
}
|
||||
return ids
|
||||
}
|
||||
|
||||
func putCandidates(h *reactiveHandler, ids []int64, labels ...string) {
|
||||
cands := make([]dialogue.Candidate, 0, len(ids))
|
||||
for i, id := range ids {
|
||||
cands = append(cands, dialogue.Candidate{Kind: "task", Ref: id, Label: labels[i]})
|
||||
}
|
||||
h.dialogueSessions.Put(voiceDialogueID, &dialogue.Session{Timestamp: h.now()})
|
||||
h.offerCandidates(cands)
|
||||
}
|
||||
+56
-36
@@ -11,6 +11,7 @@ import (
|
||||
"unicode"
|
||||
|
||||
"github.com/kami/maven/internal/ipc"
|
||||
"github.com/kami/maven/internal/morph"
|
||||
"github.com/kami/maven/internal/phraser"
|
||||
)
|
||||
|
||||
@@ -52,29 +53,39 @@ func (h *reactiveHandler) resolveQuietToggle(ctx context.Context, text string, s
|
||||
return reply, true
|
||||
}
|
||||
|
||||
// quietInflections — the inflectional endings a stem may carry and still be
|
||||
// the same word. Adjective/adverb/noun/verb endings, all ≤3 letters. This is
|
||||
// what separates "тихий"/"тихом"/"тихо" (stem "тих" + a real ending) from
|
||||
// "тихонько"/"потихоньку", which are different words: "онько" is not an
|
||||
// ending, and "потихоньку" doesn't start with the stem at all.
|
||||
var quietInflections = []string{
|
||||
"", "а", "е", "и", "й", "о", "у", "ы", "ю", "я",
|
||||
"ая", "ее", "ей", "ем", "ие", "ий", "им", "их", "ия", "ию", "ое", "ой", "ом", "ую", "ые", "ый", "ым", "ых", "ья",
|
||||
"ами", "ого", "ому", "ыми", "ать", "ить", "ять",
|
||||
}
|
||||
|
||||
// quietStem reports whether tok is the given stem carrying at most one
|
||||
// inflectional ending. Word boundaries come from tokenisation (see
|
||||
// quietTokens), not from a regexp — Go's \b is ASCII-oriented and treats every
|
||||
// Cyrillic letter as a non-word character, so `\bтих\b` would happily match
|
||||
// inside "тихонько". Comparing whole tokens sidesteps that entirely.
|
||||
func quietStem(tok, stem string) bool {
|
||||
if !strings.HasPrefix(tok, stem) {
|
||||
return false
|
||||
}
|
||||
suffix := tok[len(stem):]
|
||||
for _, e := range quietInflections {
|
||||
if suffix == e {
|
||||
// quietStem reports whether tok is one of the words a vocabulary slot accepts.
|
||||
// A slot is written as alternatives joined by "|", and an alternative comes in
|
||||
// two flavours:
|
||||
//
|
||||
// - a dictionary form, matched through the dictionary, so every case and
|
||||
// gender of it counts. This is what the nouns and adjectives want: "тихий",
|
||||
// "тихом", "тихо" and "тише" are one word.
|
||||
// - a form prefixed with "=", matched as the exact token. This is what the
|
||||
// VERBS want, and it is not a shortcut. A command is an imperative, and the
|
||||
// dictionary quite correctly files "говори" and "говорил" under one lemma —
|
||||
// so lemma-matching a verb slot read "он говорил тихим голосом весь вечер",
|
||||
// a remark about his evening, as an order to go quiet. Aspect pairs are two
|
||||
// separate verbs, which is why several imperatives are listed by hand.
|
||||
//
|
||||
// Word boundaries come from tokenisation (see quietTokens), not from a regexp —
|
||||
// Go's \b is ASCII-oriented and treats every Cyrillic letter as a non-word
|
||||
// character, so `\bтих\b` would happily match inside "тихонько". Comparing whole
|
||||
// tokens sidesteps that entirely.
|
||||
//
|
||||
// The comparison is a dictionary lookup, not a stem plus a list of 36 endings
|
||||
// (Vikunja #526). The distinction the old comment described is exactly the one a
|
||||
// dictionary makes: "тихий", "тихом", "тихо" and "тише" are one word inflected,
|
||||
// while "тихонько" and "потихоньку" are different words — and the dictionary
|
||||
// knows that without anybody deciding that "онько" is not an ending.
|
||||
func quietStem(tok, slot string) bool {
|
||||
for _, form := range strings.Split(slot, "|") {
|
||||
if exact, ok := strings.CutPrefix(form, "="); ok {
|
||||
if tok == exact {
|
||||
return true
|
||||
}
|
||||
continue
|
||||
}
|
||||
if morph.SameWord(tok, form) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
@@ -117,7 +128,10 @@ func quietPhrase(tokens, pattern []string) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
// quietOffPhrases / quietOnPhrases — the toggle vocabulary, as stem sequences.
|
||||
// quietOffPhrases / quietOnPhrases — the toggle vocabulary, as sequences of
|
||||
// dictionary forms. They used to be truncated stems ("тих", "выключ"), which is
|
||||
// what the ending list existed to complete; a dictionary form needs no
|
||||
// completing (Vikunja #526).
|
||||
//
|
||||
// Note what is NOT here any more: the OFF list used to carry {"не", "тих"} and
|
||||
// the ON list {"не", "шум"} / {"не", "беспоко"}. Both were adjacency patterns,
|
||||
@@ -128,36 +142,42 @@ func quietPhrase(tokens, pattern []string) bool {
|
||||
var (
|
||||
quietOffPhrases = [][]string{
|
||||
{"quiet", "off"}, {"quiet", "end"},
|
||||
{"громк", "режим"}, {"шумн", "режим"},
|
||||
{"отмен", "тих"}, {"выключ", "тих"},
|
||||
{"громкий", "режим"}, {"шумный", "режим"},
|
||||
{"=отмени|=отменяй|=отменить", "тихий"},
|
||||
{"=выключи|=выключай|=выключить", "тихий"},
|
||||
}
|
||||
quietOnPhrases = [][]string{
|
||||
{"quiet", "on"}, {"quiet", "mode"},
|
||||
{"тих", "режим"}, {"не", "шум"}, {"не", "беспоко"},
|
||||
{"тихий", "режим"}, {"не", "=шуми|=шумите"}, {"не", "=беспокой|=беспокоить"},
|
||||
// The noun form and the comparative. "режим тишины" is how the
|
||||
// setting is named half the time, and "сделай потише" is how it is
|
||||
// actually asked for out loud. Both used to fall through to the
|
||||
// router, which has no quiet intent, so the command did nothing.
|
||||
{"режим", "тишин"}, {"сделай", "тише"}, {"сделай", "потише"},
|
||||
{"говори", "тише"}, {"будь", "потише"},
|
||||
{"тих"}, {"потише"},
|
||||
{"режим", "тишина"}, {"=сделай", "тихий"}, {"=сделай", "потише"},
|
||||
{"=говори", "тихий"}, {"=будь", "потише"},
|
||||
{"тихий"}, {"потише"},
|
||||
}
|
||||
)
|
||||
|
||||
// quietWordStems — every stem that names the setting. Used by the
|
||||
// quietWordStems — every word that names the setting. Used by the
|
||||
// negated-but-unmatched fallback in classifyQuietToggle, which has to
|
||||
// recognise "хватит тишины" without an ON phrase having matched.
|
||||
var quietWordStems = []string{"тих", "тишин", "потише"}
|
||||
var quietWordStems = []string{"тихий", "тишина", "потише"}
|
||||
|
||||
// quietNegatorWords — negators that are whole words with no useful stem.
|
||||
var quietNegatorWords = map[string]bool{
|
||||
"не": true, "нет": true, "хватит": true, "no": true, "not": true, "off": true,
|
||||
}
|
||||
|
||||
// quietNegatorStems — negators that inflect. Matched through quietStem, the
|
||||
// same one-ending rule the toggle vocabulary uses, so "выключи", "выключить"
|
||||
// and "выключай" all count and "выключатель" does not.
|
||||
var quietNegatorStems = []string{"выключ", "отмен", "прекрат", "убер", "stop", "cancel", "disable"}
|
||||
// quietNegatorStems — negators that inflect. Imperatives, matched exactly for
|
||||
// the reason quietStem gives: "выключи" is a command and "выключил" is a report
|
||||
// about earlier, and one lemma covers both. "выключатель" was never a negator
|
||||
// and is not one now.
|
||||
var quietNegatorStems = []string{
|
||||
"=выключи|=выключай|=выключить", "=отмени|=отменяй|=отменить",
|
||||
"=прекрати|=прекращай|=прекратить", "=убери|=убирай|=убрать",
|
||||
"stop", "cancel", "disable",
|
||||
}
|
||||
|
||||
// quietNegated reports whether the utterance carries a negator. Two ON phrases
|
||||
// are themselves built on "не" — "не шуми", "не беспокой" — and those are
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// reminderMarker — the words that open a reminder. Stripped because they are
|
||||
// the instruction, not the thing to say at the hour.
|
||||
var reminderMarker = regexp.MustCompile(`(?i)^\s*(?:напомни(?:те)?|напомнить|remind)\s*(?:мне|me)?[\s,:—-]*`)
|
||||
|
||||
// reminderTimeWords — the time expressions a reminder carries, removed from
|
||||
// the body because the fire time is already a column. Ordered longest-first
|
||||
// where two could match the same words, so "через полтора часа" does not leave
|
||||
// "полтора" behind.
|
||||
//
|
||||
// Go's \b is ASCII-only and never fires next to a Cyrillic letter, so the word
|
||||
// boundaries here are written out as whitespace or an end of string — the same
|
||||
// trap the agenda grammars hit.
|
||||
var reminderTimeWords = []*regexp.Regexp{
|
||||
regexp.MustCompile(`(?i)(^|\s)через\s+\S+(\s+(часа?|часов|минут[уы]?|секунд[уы]?|дня|дней|недел[юи]))?(\s|$)`),
|
||||
regexp.MustCompile(`(?i)(^|\s)(в|во)\s+\d{1,2}(:\d{2})?(\s*(часа?|часов))?(\s*(утра|вечера|дня|ночи))?(\s|$)`),
|
||||
regexp.MustCompile(`(?i)(^|\s)(завтра|послезавтра|сегодня|вечером|утром|днём|днем|ночью)(\s|$)`),
|
||||
regexp.MustCompile(`(?i)(^|\s)(at|in)\s+\d{1,2}(:\d{2})?\s*(am|pm)?(\s|$)`),
|
||||
regexp.MustCompile(`(?i)(^|\s)(tomorrow|today|tonight)(\s|$)`),
|
||||
}
|
||||
|
||||
// reminderBody is what she says at the hour.
|
||||
//
|
||||
// The whole utterance used to be stored, so /reminders read "напомни завтра в
|
||||
// 9 утра выпить таблетки" where it should read "выпить таблетки", and the
|
||||
// agenda recited the marker back at him (Vikunja #469). The fire time is
|
||||
// already a column, and the marker is an instruction that was carried out.
|
||||
//
|
||||
// Falls back to the fuller text whenever stripping would leave nothing: an
|
||||
// empty body is a reminder that fires and says nothing, which is worse than a
|
||||
// wordy one.
|
||||
func reminderBody(utterance, text string) string {
|
||||
body := strings.TrimSpace(text)
|
||||
if body == "" {
|
||||
body = strings.TrimSpace(utterance)
|
||||
}
|
||||
stripped := reminderMarker.ReplaceAllString(body, "")
|
||||
for _, re := range reminderTimeWords {
|
||||
stripped = re.ReplaceAllString(stripped, " ")
|
||||
}
|
||||
stripped = strings.TrimSpace(strings.Join(strings.Fields(stripped), " "))
|
||||
stripped = strings.Trim(stripped, " ,;:—-")
|
||||
if stripped == "" {
|
||||
return body
|
||||
}
|
||||
return stripped
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
package main
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestReminderBody(t *testing.T) {
|
||||
for _, tc := range []struct{ utterance, text, want string }{
|
||||
// The row from the QA sitting: the whole utterance was the body.
|
||||
{"напомни завтра в 9 утра выпить таблетки", "завтра в 9 утра выпить таблетки", "выпить таблетки"},
|
||||
{"напомни мне позвонить маме в семь вечера", "позвонить маме в 7 вечера", "позвонить маме"},
|
||||
{"напомни через полчаса проверить бэкап", "через полчаса проверить бэкап", "проверить бэкап"},
|
||||
{"remind me to call mom at 7pm", "to call mom at 7pm", "to call mom"},
|
||||
// Nothing left after stripping ⇒ keep what there was. A reminder that
|
||||
// fires and says nothing is worse than a wordy one.
|
||||
{"напомни завтра", "завтра", "завтра"},
|
||||
{"", "", ""},
|
||||
} {
|
||||
if got := reminderBody(tc.utterance, tc.text); got != tc.want {
|
||||
t.Errorf("reminderBody(%q, %q) = %q, want %q", tc.utterance, tc.text, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,209 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
|
||||
"github.com/kami/maven/internal/lexicon"
|
||||
"github.com/kami/maven/internal/morph"
|
||||
"github.com/kami/maven/internal/router"
|
||||
)
|
||||
|
||||
// Conversation repair (Vikunja #455).
|
||||
//
|
||||
// The classifier has been able to learn from a correction since it was
|
||||
// written — CorrectMisroute appends the utterance as a new example for the
|
||||
// intent he names, append-only, no retrain. Nothing in the daemon could reach
|
||||
// it: the only caller was a test. So the mechanism existed and the behaviour
|
||||
// did not.
|
||||
//
|
||||
// This is the reachable half. He says she got it wrong and names what it
|
||||
// should have been, she redoes the previous utterance under that intent, and
|
||||
// she says out loud that the correction landed — because a correction he
|
||||
// cannot see is indistinguishable from one that was dropped.
|
||||
//
|
||||
// Taken before routing, like the confirm and clarify turns: "нет, это была
|
||||
// заметка" is an answer to the previous turn, not a fresh command, and routing
|
||||
// it as one files the correction itself as a note.
|
||||
|
||||
// routedTurn — the previous utterance and where it went, which is all a
|
||||
// correction needs to point at.
|
||||
type routedTurn struct {
|
||||
utterance string
|
||||
intent router.Intent
|
||||
at time.Time
|
||||
}
|
||||
|
||||
// repairWindow — how long a turn stays correctable. Long enough that he can
|
||||
// hear the wrong answer, think, and say so; short enough that "это заметка"
|
||||
// half an hour later is a fresh sentence and not a verdict on something he has
|
||||
// forgotten.
|
||||
const repairWindow = 5 * time.Minute
|
||||
|
||||
// repairMarkers — the ways he says she got it wrong. One of these must appear:
|
||||
// naming an intent alone is an ordinary sentence ("напиши заметку"), and
|
||||
// treating it as a correction would rewrite the last turn every time he used
|
||||
// the word.
|
||||
//
|
||||
// From the lexicon, and staying a list rather than becoming seeds (Vikunja
|
||||
// #528). This runs pre-route, before the turn vector exists, and a correction
|
||||
// redoes the previous request — so a near-miss would act on something he never
|
||||
// said. The set's note in lexicon_ru_v1.json carries the same reasoning.
|
||||
var repairMarkers = lexicon.RepairMarkers()
|
||||
|
||||
// repairIntents — the words he uses for each intent, as dictionary forms. They
|
||||
// used to be prefixes ("заметк"), which is what a prefix list costs: "команд"
|
||||
// also matched "командировка", and "факт" matched "фактически". morph.SameWord
|
||||
// compares the words themselves (Vikunja #528).
|
||||
var repairIntents = []struct {
|
||||
word string
|
||||
intent router.Intent
|
||||
say string
|
||||
}{
|
||||
{"заметка", router.IntentNote, "заметка"},
|
||||
{"напоминание", router.IntentReminder, "напоминание"},
|
||||
{"напомнить", router.IntentReminder, "напоминание"},
|
||||
{"факт", router.IntentFact, "факт"},
|
||||
{"вопрос", router.IntentQuery, "вопрос"},
|
||||
{"команда", router.IntentAct, "команда"},
|
||||
{"note", router.IntentNote, "заметка"},
|
||||
{"reminder", router.IntentReminder, "напоминание"},
|
||||
{"fact", router.IntentFact, "факт"},
|
||||
{"question", router.IntentQuery, "вопрос"},
|
||||
}
|
||||
|
||||
// parseRepair reads a spoken correction: a marker saying she was wrong, plus
|
||||
// the intent it should have been.
|
||||
//
|
||||
// The negated half is skipped. "это заметка, а не напоминание" names both
|
||||
// intents, and the one he is correcting TO is the one he did not put "не" in
|
||||
// front of.
|
||||
func parseRepair(utterance string) (router.Intent, string, bool) {
|
||||
s := strings.ToLower(strings.TrimSpace(utterance))
|
||||
if s == "" {
|
||||
return "", "", false
|
||||
}
|
||||
// A leading "нет" is a marker on its own — "нет, это заметка" is the
|
||||
// shortest correction he actually says. Only leading: "нет" in the middle
|
||||
// of a sentence is an ordinary word.
|
||||
marked := strings.HasPrefix(s, "нет") || strings.HasPrefix(s, "no,")
|
||||
for _, m := range repairMarkers {
|
||||
if marked || strings.Contains(s, m) {
|
||||
marked = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !marked {
|
||||
return "", "", false
|
||||
}
|
||||
// Over tokens, not byte offsets. The negation test used to read the string
|
||||
// immediately before a match, which meant it could only see "не" spelled
|
||||
// exactly there; a token list makes the previous word plain to read.
|
||||
toks := repairTokens(s)
|
||||
best, say, at := router.Intent(""), "", -1
|
||||
for i, tok := range toks {
|
||||
if at >= 0 && i > at {
|
||||
break
|
||||
}
|
||||
for _, w := range repairIntents {
|
||||
if !morph.SameWord(tok, w.word) {
|
||||
continue
|
||||
}
|
||||
if i > 0 && (toks[i-1] == "не" || toks[i-1] == "not") {
|
||||
// The one he is ruling out: "не напоминание, а заметка".
|
||||
continue
|
||||
}
|
||||
// Leftmost wins: "это заметка, а не напоминание" corrects to the
|
||||
// first.
|
||||
if at < 0 || i < at {
|
||||
best, say, at = w.intent, w.say, i
|
||||
}
|
||||
}
|
||||
}
|
||||
if at < 0 {
|
||||
return "", "", false
|
||||
}
|
||||
return best, say, true
|
||||
}
|
||||
|
||||
// repairTokens splits a correction into lowercase word tokens. Punctuation goes,
|
||||
// because "это заметка, а не напоминание" glues a comma to the word the negation
|
||||
// test has to look past.
|
||||
func repairTokens(s string) []string {
|
||||
return strings.FieldsFunc(s, func(r rune) bool {
|
||||
return !unicode.IsLetter(r) && !unicode.IsDigit(r)
|
||||
})
|
||||
}
|
||||
|
||||
// recordTurn keeps the utterance a correction would point at. Only turns she
|
||||
// acted on: a clarify asked instead of acting, so there is nothing yet to be
|
||||
// wrong about.
|
||||
func (h *reactiveHandler) recordTurn(utterance string, intent router.Intent) {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
h.lastRouted = &routedTurn{utterance: utterance, intent: intent, at: h.now()}
|
||||
}
|
||||
|
||||
func (h *reactiveHandler) takeLastTurn() *routedTurn {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
last := h.lastRouted
|
||||
// Taken, not read: one utterance is corrected once. Saying "нет, не так"
|
||||
// twice would otherwise redo the same request twice.
|
||||
h.lastRouted = nil
|
||||
return last
|
||||
}
|
||||
|
||||
// resolveRepair handles a spoken correction of the previous turn: teach the
|
||||
// classifier, redo the request under the corrected intent, and say so.
|
||||
func (h *reactiveHandler) resolveRepair(ctx context.Context, text string) (string, bool) {
|
||||
corrected, say, ok := parseRepair(text)
|
||||
if !ok || h.router == nil {
|
||||
return "", false
|
||||
}
|
||||
last := h.takeLastTurn()
|
||||
if last == nil || h.now().Sub(last.at) > repairWindow {
|
||||
return "", false
|
||||
}
|
||||
if last.intent == corrected {
|
||||
// She already did what he is asking for. Correcting the classifier
|
||||
// here would teach it the label it produced, and redoing the request
|
||||
// would file it twice.
|
||||
return "", false
|
||||
}
|
||||
learned := true
|
||||
if err := h.router.CorrectMisroute(ctx, last.utterance, corrected); err != nil {
|
||||
// The redo is still worth doing: he asked for something and it did not
|
||||
// happen. Only the learning half is lost, and he is told so.
|
||||
log.Printf("voice: repair: could not learn %q as %s: %v", last.utterance, corrected, err)
|
||||
learned = false
|
||||
}
|
||||
log.Printf("voice: repair — %q was %s, corrected to %s (learned=%v)", last.utterance, last.intent, corrected, learned)
|
||||
|
||||
dec := router.Decision{
|
||||
Utterance: last.utterance,
|
||||
Stage: 2,
|
||||
Intent: corrected,
|
||||
Slots: h.extractor.Extract(ctx, corrected, last.utterance, h.now()),
|
||||
}
|
||||
// A reminder's Text is what she says at the hour and stays empty when it
|
||||
// was not spoken, so the gap is asked about rather than filled with the
|
||||
// whole sentence. Everywhere else the utterance IS the payload.
|
||||
if dec.Slots.Text == "" && corrected != router.IntentReminder {
|
||||
dec.Slots.Text = last.utterance
|
||||
}
|
||||
return repairLine(say, learned) + " " + h.finishClarified(ctx, dec), true
|
||||
}
|
||||
|
||||
// repairLine — what she says before redoing it, so the correction is visible
|
||||
// and not just filed. Feminine, informal, no apology: he corrected a routing
|
||||
// call, he did not complain about her.
|
||||
func repairLine(say string, learned bool) string {
|
||||
if !learned {
|
||||
return "поняла, это " + say + " — переделываю, но запомнить поправку не вышло."
|
||||
}
|
||||
return "поняла, это " + say + " — запомнила."
|
||||
}
|
||||
@@ -0,0 +1,151 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/kami/maven/internal/router"
|
||||
)
|
||||
|
||||
func TestParseRepairReadsTheCorrectedIntent(t *testing.T) {
|
||||
cases := []struct {
|
||||
utterance string
|
||||
want router.Intent
|
||||
ok bool
|
||||
}{
|
||||
{"нет, ты не поняла, это заметка", router.IntentNote, true},
|
||||
{"нет, это заметка", router.IntentNote, true},
|
||||
{"это не напоминание, а заметка", router.IntentNote, true},
|
||||
{"это заметка, а не напоминание", router.IntentNote, true},
|
||||
{"ты не так поняла — это факт", router.IntentFact, true},
|
||||
{"неправильно поняла, это был вопрос", router.IntentQuery, true},
|
||||
{"you got it wrong, that was a note", router.IntentNote, true},
|
||||
// No marker: an ordinary request that happens to name an intent.
|
||||
{"запиши заметку купить хлеб", "", false},
|
||||
{"напомни мне про заметку", "", false},
|
||||
// A marker with no intent named: nothing to correct to.
|
||||
{"ты не так поняла", "", false},
|
||||
{"", "", false},
|
||||
}
|
||||
for _, c := range cases {
|
||||
got, _, ok := parseRepair(c.utterance)
|
||||
if ok != c.ok || (ok && got != c.want) {
|
||||
t.Errorf("parseRepair(%q) = %q,%v; want %q,%v", c.utterance, got, ok, c.want, c.ok)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestRepairTeachesTheClassifierAndRedoesTheTurn is the whole feature: the
|
||||
// previous utterance is filed under the intent he named, the classifier keeps
|
||||
// it as an example, and he hears that it landed.
|
||||
func TestRepairTeachesTheClassifierAndRedoesTheTurn(t *testing.T) {
|
||||
h, st, now := newClarifyHandler(t)
|
||||
emb := router.NewHashEmbedder(256)
|
||||
cls := router.NewClassifier(emb)
|
||||
h.embedder = emb
|
||||
h.router = router.New(router.Config{Classifier: cls, Extractor: h.extractor})
|
||||
ctx := context.Background()
|
||||
|
||||
h.recordTurn("купить хлеб", router.IntentFact)
|
||||
reply, handled := h.resolveRepair(ctx, "нет, ты не поняла, это заметка")
|
||||
if !handled {
|
||||
t.Fatal("a spoken correction was not handled")
|
||||
}
|
||||
if !strings.Contains(reply, "заметка") {
|
||||
t.Errorf("the correction is not named out loud: %q", reply)
|
||||
}
|
||||
if strings.Contains(reply, "не вышло") {
|
||||
t.Errorf("learning failed unexpectedly: %q", reply)
|
||||
}
|
||||
|
||||
ex := cls.Examples(router.IntentNote)
|
||||
if len(ex) != 1 || ex[0].Text != "купить хлеб" {
|
||||
t.Fatalf("the classifier did not learn the correction: %+v", ex)
|
||||
}
|
||||
notes, err := st.RecentNotes(ctx, 5)
|
||||
if err != nil {
|
||||
t.Fatalf("recent notes: %v", err)
|
||||
}
|
||||
if len(notes) != 1 || !strings.Contains(notes[0].Text, "купить хлеб") {
|
||||
t.Fatalf("the request was not redone as a note: %+v", notes)
|
||||
}
|
||||
_ = now
|
||||
}
|
||||
|
||||
func TestRepairNeedsARecentTurnToPointAt(t *testing.T) {
|
||||
h, _, now := newClarifyHandler(t)
|
||||
h.router = router.New(router.Config{Classifier: router.NewClassifier(router.NewHashEmbedder(256))})
|
||||
ctx := context.Background()
|
||||
|
||||
// Nothing said yet.
|
||||
if _, handled := h.resolveRepair(ctx, "нет, это заметка"); handled {
|
||||
t.Error("a correction with no previous turn was handled")
|
||||
}
|
||||
// Said, but long ago.
|
||||
h.recordTurn("купить хлеб", router.IntentFact)
|
||||
*now = now.Add(repairWindow + time.Minute)
|
||||
if _, handled := h.resolveRepair(ctx, "нет, это заметка"); handled {
|
||||
t.Error("a correction outside the window was handled")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRepairIsSpentOnce(t *testing.T) {
|
||||
h, _, _ := newClarifyHandler(t)
|
||||
emb := router.NewHashEmbedder(256)
|
||||
h.embedder = emb
|
||||
h.router = router.New(router.Config{Classifier: router.NewClassifier(emb), Extractor: h.extractor})
|
||||
ctx := context.Background()
|
||||
|
||||
h.recordTurn("купить хлеб", router.IntentFact)
|
||||
if _, handled := h.resolveRepair(ctx, "нет, это заметка"); !handled {
|
||||
t.Fatal("the first correction was not handled")
|
||||
}
|
||||
if _, handled := h.resolveRepair(ctx, "нет, это заметка"); handled {
|
||||
t.Error("the same turn was corrected twice")
|
||||
}
|
||||
}
|
||||
|
||||
// TestRepairPassesWhenSheAlreadyDidThat — he names the intent she used. There
|
||||
// is nothing to teach and redoing it would file the request a second time.
|
||||
func TestRepairPassesWhenSheAlreadyDidThat(t *testing.T) {
|
||||
h, _, _ := newClarifyHandler(t)
|
||||
h.router = router.New(router.Config{Classifier: router.NewClassifier(router.NewHashEmbedder(256))})
|
||||
h.recordTurn("купить хлеб", router.IntentNote)
|
||||
if _, handled := h.resolveRepair(context.Background(), "нет, это заметка"); handled {
|
||||
t.Error("a correction to the intent she already used was handled")
|
||||
}
|
||||
}
|
||||
|
||||
// TestRepairIntentWordCollisions — the prefix list matched more than the word
|
||||
// (Vikunja #528). "команд" is inside "командировка" and "факт" inside
|
||||
// "фактически", and either one used to name an intent she would redo the turn
|
||||
// under.
|
||||
func TestRepairIntentWordCollisions(t *testing.T) {
|
||||
for _, s := range []string{
|
||||
"нет, это про командировку",
|
||||
"нет, фактически всё нормально",
|
||||
} {
|
||||
if _, _, ok := parseRepair(s); ok {
|
||||
t.Errorf("parseRepair(%q) claimed a correction", s)
|
||||
}
|
||||
}
|
||||
// The declined forms the prefixes existed to cover still work, and the
|
||||
// negated half is still skipped.
|
||||
for _, tc := range []struct {
|
||||
utterance string
|
||||
want router.Intent
|
||||
}{
|
||||
{"нет, это заметка", router.IntentNote},
|
||||
{"ты не так поняла, это заметку надо было", router.IntentNote},
|
||||
{"нет, это напоминание, а не заметка", router.IntentReminder},
|
||||
{"нет, не напоминание, а заметка", router.IntentNote},
|
||||
{"нет, это командой было", router.IntentAct},
|
||||
} {
|
||||
got, _, ok := parseRepair(tc.utterance)
|
||||
if !ok || got != tc.want {
|
||||
t.Errorf("parseRepair(%q) = %q, %v; want %q, true", tc.utterance, got, ok, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -24,7 +24,12 @@ func newLLMReplier(c phraser.Completer, block func() string) *llmReplier {
|
||||
// answer from the stub, which is what keeps a turn from breaking on the model.
|
||||
func (r *llmReplier) Reply(d router.Decision) string {
|
||||
if d.Clarify {
|
||||
return r.stub.Reply(d)
|
||||
// The deck, not the stub's single sentence: a clarify she cannot turn
|
||||
// into a question is the line he hears most often when she misses him,
|
||||
// and it used to be the same words every time (Vikunja #457). Still no
|
||||
// model call — this text has to be right every time, and it is not worth
|
||||
// a generation to say something this small.
|
||||
return clarifyMissedLine(d)
|
||||
}
|
||||
out, err := r.p.PhraseReply(context.Background(), d)
|
||||
if err != nil || out == "" {
|
||||
|
||||
@@ -38,9 +38,21 @@ func TestLLMReplierFallsBackToStubOnEmpty(t *testing.T) {
|
||||
assertAck(t, r, router.Decision{Intent: router.IntentNote}, phraser.AckNote, "empty llm")
|
||||
}
|
||||
|
||||
func TestLLMReplierClarifyUsesStub(t *testing.T) {
|
||||
// A clarify never reaches the model, and since Vikunja #457 it is answered from
|
||||
// the clarify deck rather than the stub's single sentence.
|
||||
func TestLLMReplierClarifyReadsTheDeck(t *testing.T) {
|
||||
r := newLLMReplier(stubCompleter{out: "я всё поняла"}, nil)
|
||||
assertStub(t, r, router.Decision{Clarify: true}, "clarify")
|
||||
got := r.Reply(router.Decision{Clarify: true, Utterance: "мгм"})
|
||||
if got == "я всё поняла" {
|
||||
t.Fatal("a clarify must not be phrased by the model")
|
||||
}
|
||||
if want := clarifyMissedFor("мгм"); got != want {
|
||||
t.Errorf("on clarify: got %q, want %q", got, want)
|
||||
}
|
||||
// Two different misses do not sound identical.
|
||||
if same := r.Reply(router.Decision{Clarify: true, Utterance: "а"}); same == got {
|
||||
t.Log("two utterances hashed to the same line, which is allowed but should be rare")
|
||||
}
|
||||
}
|
||||
|
||||
// assertAck — the stub picks between variants now, so two calls to it are not
|
||||
|
||||
+14
-16
@@ -16,18 +16,14 @@ import (
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/kami/maven/internal/lexicon"
|
||||
"github.com/kami/maven/internal/say"
|
||||
)
|
||||
|
||||
var ruWeekdays = []string{
|
||||
"воскресенье", "понедельник", "вторник", "среда",
|
||||
"четверг", "пятница", "суббота",
|
||||
}
|
||||
|
||||
var ruMonths = []string{
|
||||
"января", "февраля", "марта", "апреля", "мая", "июня",
|
||||
"июля", "августа", "сентября", "октября", "ноября", "декабря",
|
||||
}
|
||||
// Weekday and month names are a closed class — the language has seven and
|
||||
// twelve — so they live complete in internal/lexicon, where internal/ttsnorm
|
||||
// reads the same twelve month names instead of keeping a second copy
|
||||
// (Vikunja #525).
|
||||
|
||||
// onlyLocalTimeReply — the honest answer when the user asks the time somewhere
|
||||
// other than here. She only keeps one clock, and saying so is better than
|
||||
@@ -39,13 +35,15 @@ var ruMonths = []string{
|
||||
const onlyLocalTimeReply = "я знаю только местное время, про другие города пока не скажу."
|
||||
|
||||
// notPlaceAfterV — words that follow "в" without naming a place, so
|
||||
// mentionsUnknownPlace does not mistake them for a city.
|
||||
var notPlaceAfterV = map[string]bool{
|
||||
"данный": true, "данную": true, "этот": true, "эту": true,
|
||||
"котором": true, "какое": true, "какой": true, "который": true,
|
||||
"общем": true, "точности": true, "курсе": true, "сутках": true,
|
||||
"часах": true, "минутах": true, "секундах": true, "неделе": true,
|
||||
}
|
||||
// mentionsUnknownPlace does not mistake them for a city. Closed set, kept
|
||||
// complete in internal/lexicon.
|
||||
var notPlaceAfterV = func() map[string]bool {
|
||||
m := map[string]bool{}
|
||||
for _, w := range lexicon.NotPlaceAfterV() {
|
||||
m[w] = true
|
||||
}
|
||||
return m
|
||||
}()
|
||||
|
||||
// mentionsUnknownPlace reports whether the question has a "в <слово>" phrase
|
||||
// that looks like a place we do not know ("который час в киеве"). Used only to
|
||||
|
||||
+12
-4
@@ -82,15 +82,23 @@ func (h *reactiveHandler) pendingNudge(ctx context.Context, now time.Time) (ipc.
|
||||
return ipc.Nudge{}, false
|
||||
}
|
||||
|
||||
// snoozePhrases — the deferral vocabulary, as stem sequences. Matched by
|
||||
// quietPhrase (quiet_toggle.go), which carries the rule that matters here:
|
||||
// snoozePhrases — the deferral vocabulary. Matched by quietPhrase and quietStem
|
||||
// (quiet_toggle.go), so an adverb is matched through the dictionary and a verb
|
||||
// exactly, prefixed with "=": "напомни" is a request and "напомнил" is a report
|
||||
// about earlier, and the dictionary files both under напомнить (Vikunja #526).
|
||||
// The verbs used to be truncated stems — "напомн", "отлож" — which is what the
|
||||
// deleted ending list existed to complete.
|
||||
//
|
||||
// quietPhrase carries the rule that matters here:
|
||||
// a single-word pattern matches only a single-word utterance. Bare "потом" is
|
||||
// an answer; "потом схожу за водой" is a plan, and reporting a plan must not
|
||||
// silence the rule that prompted it.
|
||||
var snoozePhrases = [][]string{
|
||||
{"не", "сейчас"}, {"не", "могу", "сейчас"}, {"не", "до", "этого"},
|
||||
{"напомн", "позже"}, {"напомн", "потом"}, {"спрос", "позже"},
|
||||
{"отлож"}, {"позже"}, {"потом"}, {"попозже"}, {"погоди"},
|
||||
{"=напомни|=напоминай", "позже"}, {"=напомни|=напоминай", "потом"},
|
||||
{"=спроси|=спрашивай", "позже"},
|
||||
{"=отложи|=отложим|=откладывай"}, {"позже"}, {"потом"}, {"попозже"},
|
||||
{"=погоди|=погодите"},
|
||||
{"not", "now"}, {"later"}, {"snooze"}, {"remind", "me", "later"},
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,229 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"sync"
|
||||
|
||||
"github.com/kami/maven/internal/router"
|
||||
)
|
||||
|
||||
// Which subject is this question about — the weather, the house, the LAN, what
|
||||
// needs looking at, or none of them. Third of the three mechanisms replacing hand-written Russian
|
||||
// patterns (Vikunja #522, owner's call 2026-08-04). internal/lexicon holds the
|
||||
// sets that can be finished and internal/morph answers the grammar questions;
|
||||
// this is for the sets that can never be finished, because "is this about the
|
||||
// house" is a question about meaning and no word list closes it.
|
||||
//
|
||||
// The recognisers this replaces were each built the same way: a stem list,
|
||||
// an ask test, a device-noun list, and a bail-out list for the neighbouring
|
||||
// topic. Every one of their own comments admits the shape. isHomeQuery excluded
|
||||
// "погод", "на улице" and "прогноз" by hand because "какая температура на улице"
|
||||
// and "какая температура в доме" share their only content word. isNetworkQuery
|
||||
// matched "сети" as a whole token because the substring lives inside "посетил",
|
||||
// so "сколько машин я посетил" read as a request to scan the LAN. Those are not
|
||||
// bugs in the lists, they are the lists being asked to do semantics.
|
||||
//
|
||||
// So the seeds decide, the same way the personal boundary does
|
||||
// (personalboundary.go), against the same embedder and the same query vector the
|
||||
// turn already carries. One difference in the gate, and it is deliberate. The
|
||||
// boundary claims on the sign of the difference, because there a false claim
|
||||
// costs one honest "не знаю". Here a false claim runs a network scan, or names a
|
||||
// capability as off on a box where it is simply not the subject — so a topic has
|
||||
// to win by a margin, and the losing side of a thin call falls through to the
|
||||
// next query source, which is what the narrow regexes were achieving.
|
||||
//
|
||||
// A fourth subject joined on the same day: isAttentionQuery, "что требует
|
||||
// внимания", which is Praxis's operational state and reached the web search
|
||||
// before the source existed (Vikunja #475).
|
||||
//
|
||||
// The regexes stay as the offline floor, unchanged, for a handler with no
|
||||
// embedder or a turn whose vector never got computed. They are allowed to remain
|
||||
// narrow now precisely because they are no longer the only answer.
|
||||
|
||||
// topicLabel — the subjects worth telling apart, plus the one that means none of
|
||||
// them. topicOther is a real class and not a threshold: a question needs
|
||||
// somewhere to lose TO, and "интернет не работает" losing to a set that contains
|
||||
// complaints is a better statement than it failing a number.
|
||||
type topicLabel string
|
||||
|
||||
const (
|
||||
topicWeather topicLabel = "weather"
|
||||
topicHome topicLabel = "home"
|
||||
topicNetwork topicLabel = "network"
|
||||
topicAttend topicLabel = "attention"
|
||||
topicOther topicLabel = "other"
|
||||
)
|
||||
|
||||
// topicMargin — how far a topic must clear the runner-up. Small, because the
|
||||
// margins between neighbouring topics are small: measured on held-out
|
||||
// utterances, a true weather question clears the home set by roughly 0.02 to
|
||||
// 0.09 and the nearest wrong call sits under 0.01. It exists at all for the
|
||||
// asymmetry named above — this gate spends a scan, so a coin-flip falls
|
||||
// through rather than acts.
|
||||
const topicMargin = 0.01
|
||||
|
||||
// topicSeedSets — frozen scoring data, like personalSeeds. Editing one moves a
|
||||
// recogniser and has to be re-measured against TestONNXTopics, not eyeballed.
|
||||
//
|
||||
// Each set covers the phrasings its old regex covered, INCLUDING the ones it
|
||||
// needed a bail-out list for: the weather set carries "какая температура на
|
||||
// улице" and the home set "какая температура в доме", so the pair that forced
|
||||
// isHomeQuery to exclude weather words by hand is now just two seeds sitting on
|
||||
// their own sides.
|
||||
var topicSeedSets = map[topicLabel][]string{
|
||||
topicWeather: {
|
||||
"какая сегодня погода",
|
||||
"какая температура на улице",
|
||||
"будет дождь сегодня",
|
||||
"на улице холодно",
|
||||
"прогноз погоды на завтра",
|
||||
"сколько градусов сейчас",
|
||||
"what is the weather like",
|
||||
"is it going to rain today",
|
||||
},
|
||||
topicHome: {
|
||||
"что включено в доме",
|
||||
"какая температура в доме",
|
||||
"свет в квартире горит",
|
||||
"сколько лампочек включено дома",
|
||||
"что у меня дома с датчиками",
|
||||
"умный дом что сейчас работает",
|
||||
"розетки в доме включены",
|
||||
"what is on in the house",
|
||||
},
|
||||
topicNetwork: {
|
||||
"какие устройства в сети",
|
||||
"кто в сети сейчас",
|
||||
"просканируй локальную сеть",
|
||||
"сколько машин в сетке",
|
||||
"покажи хосты в сети",
|
||||
"какие адреса заняты в локальной сети",
|
||||
"кто подключён к вайфаю",
|
||||
"what devices are on the network",
|
||||
},
|
||||
topicAttend: {
|
||||
"что требует внимания",
|
||||
"что не так сейчас",
|
||||
"на что мне посмотреть",
|
||||
"что важное я пропустил",
|
||||
"есть что-то срочное",
|
||||
"что там висит нерешённое",
|
||||
// With a thing named. Every other seed asks in the abstract, and the
|
||||
// attention question he actually asks names his services or his
|
||||
// projects.
|
||||
"что не так с сервисами",
|
||||
"что там с моими проектами",
|
||||
"what needs attention",
|
||||
"what needs looking at right now",
|
||||
},
|
||||
topicOther: {
|
||||
// Complaints, which are not requests to scan or to read the house.
|
||||
// isNetworkQuery's comment names this one: a scan she runs unasked is
|
||||
// the noisy behaviour the bounds exist to prevent.
|
||||
"интернет не работает",
|
||||
"вайфай тормозит",
|
||||
"свет погас",
|
||||
// Statements. "я дома" was the reason isHomeQuery needed an ask test.
|
||||
"я дома",
|
||||
"я уже дома",
|
||||
// The collision that made "сети" a whole-token match.
|
||||
"сколько машин я посетил",
|
||||
"сколько домов мы посмотрели",
|
||||
// Ordinary questions, his and the world's, so a topic has something
|
||||
// real to lose to rather than an arbitrary floor.
|
||||
"почему небо синее",
|
||||
"какая столица франции",
|
||||
"что я говорил про бэкапы",
|
||||
"что у меня сегодня по календарю",
|
||||
"напомни мне позвонить маме",
|
||||
// An attention question is about the state of his things; this is not.
|
||||
"что ты умеешь",
|
||||
"what did i say about backups",
|
||||
},
|
||||
}
|
||||
|
||||
// topicIndex holds the embedded seeds. Zero value is usable and means "not
|
||||
// loaded yet"; a handler built without an embedder never loads and every caller
|
||||
// uses its own floor instead.
|
||||
type topicIndex struct {
|
||||
once sync.Once
|
||||
vecs map[topicLabel][][]float32
|
||||
loaded bool
|
||||
}
|
||||
|
||||
// load embeds every set once per process, on the QUERY side — a question
|
||||
// compared with a question, for the reason personalBoundary.load gives.
|
||||
func (x *topicIndex) load(ctx context.Context, emb router.Embedder) {
|
||||
x.once.Do(func() {
|
||||
if emb == nil {
|
||||
return
|
||||
}
|
||||
vecs := make(map[topicLabel][][]float32, len(topicSeedSets))
|
||||
for label, seeds := range topicSeedSets {
|
||||
out := make([][]float32, 0, len(seeds))
|
||||
for _, s := range seeds {
|
||||
v, err := router.EmbedQuery(ctx, emb, s)
|
||||
if err != nil {
|
||||
log.Printf("voice: topic seeds unavailable (%v); falling back to keyword matching", err)
|
||||
return
|
||||
}
|
||||
out = append(out, v)
|
||||
}
|
||||
vecs[label] = out
|
||||
}
|
||||
x.vecs, x.loaded = vecs, true
|
||||
})
|
||||
}
|
||||
|
||||
// best returns the nearest label, how far it cleared the runner-up, and whether
|
||||
// the seeds answered at all. ok is false when they are not loaded, which is the
|
||||
// caller's signal to use its floor.
|
||||
func (x *topicIndex) best(vec []float32) (label topicLabel, margin float64, ok bool) {
|
||||
if !x.loaded || len(vec) == 0 {
|
||||
return "", 0, false
|
||||
}
|
||||
first, second := -1.0, -1.0
|
||||
for l, seeds := range x.vecs {
|
||||
top := -1.0
|
||||
for _, s := range seeds {
|
||||
if c := cosine(vec, s); c > top {
|
||||
top = c
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case top > first:
|
||||
label, first, second = l, top, first
|
||||
case top > second:
|
||||
second = top
|
||||
}
|
||||
}
|
||||
return label, first - second, true
|
||||
}
|
||||
|
||||
// turnIsAbout — the recogniser every topic source calls. The seeds decide when
|
||||
// the embedder is there, which is every deployed box; floor is the source's own
|
||||
// keyword test, which answers when they are not.
|
||||
//
|
||||
// A topic that wins WITHOUT the margin is handed to the floor rather than
|
||||
// claimed or dropped, and the near-miss is logged. That is the cascade shape
|
||||
// again: the better test leads, the offline one always answers, and a thin call
|
||||
// is exactly where the cheap high-precision test earns its place. Measured, the
|
||||
// one held-out case that lands there is "вайфай опять отвалился", which reads as
|
||||
// network by 0.0055; isNetworkQuery says no, so it stays the complaint it is.
|
||||
func (h *reactiveHandler) turnIsAbout(ctx context.Context, t *queryTurn, want topicLabel, floor func(string) bool) bool {
|
||||
h.topics.load(ctx, h.embedder)
|
||||
label, margin, ok := h.topics.best(t.vec)
|
||||
if !ok {
|
||||
return floor(t.dec.Utterance)
|
||||
}
|
||||
if label != want {
|
||||
return false
|
||||
}
|
||||
if margin < topicMargin {
|
||||
claimed := floor(t.dec.Utterance)
|
||||
log.Printf("voice: %q reads as %s by only %.4f; the keyword floor says %v", t.dec.Utterance, want, margin, claimed)
|
||||
return claimed
|
||||
}
|
||||
return true
|
||||
}
|
||||
@@ -0,0 +1,124 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/kami/maven/internal/router"
|
||||
)
|
||||
|
||||
// TestTopicFloorAnswersWithoutSeeds — a handler with no embedder never loads the
|
||||
// seeds, and every topic source has to keep working. This is the case that used
|
||||
// to be the only one, so a regression here is all four recognisers going silent
|
||||
// on a box with no embedder at all.
|
||||
func TestTopicFloorAnswersWithoutSeeds(t *testing.T) {
|
||||
h := &reactiveHandler{}
|
||||
for _, tc := range []struct {
|
||||
utterance string
|
||||
label topicLabel
|
||||
floor func(string) bool
|
||||
want bool
|
||||
}{
|
||||
{"какая сегодня погода", topicWeather, isWeatherQuery, true},
|
||||
{"что включено в доме?", topicHome, isHomeQuery, true},
|
||||
{"какие устройства в сети?", topicNetwork, isNetworkQuery, true},
|
||||
{"что требует внимания?", topicAttend, isAttentionQuery, true},
|
||||
{"почему небо синее", topicWeather, isWeatherQuery, false},
|
||||
{"я дома", topicHome, isHomeQuery, false},
|
||||
{"интернет не работает", topicNetwork, isNetworkQuery, false},
|
||||
} {
|
||||
turn := &queryTurn{dec: router.Decision{Utterance: tc.utterance}}
|
||||
if got := h.turnIsAbout(context.Background(), turn, tc.label, tc.floor); got != tc.want {
|
||||
t.Errorf("turnIsAbout(%q, %s) = %v, want %v", tc.utterance, tc.label, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestONNXTopics — the number that matters, scored against the embedder homesrv
|
||||
// actually runs. Opt-in via MAVEN_ONNX_LIB, like TestONNXPersonalBoundary.
|
||||
//
|
||||
// Every case is held out: none of these strings is a seed. It asserts what the
|
||||
// gate does, not what the raw scorer says — a label under topicMargin is handed
|
||||
// to the keyword floor, and one case turns on exactly that.
|
||||
//
|
||||
// The first three rows are the collisions the old regexes needed hand-written
|
||||
// bail-outs for: the temperature pair that made isHomeQuery exclude weather
|
||||
// words, and the "посетил" substring that made isNetworkQuery match "сети" as a
|
||||
// whole token.
|
||||
func TestONNXTopics(t *testing.T) {
|
||||
lib := os.Getenv("MAVEN_ONNX_LIB")
|
||||
if lib == "" {
|
||||
t.Skip("MAVEN_ONNX_LIB unset — see AGENTS.md § Embedder model for intent routing")
|
||||
}
|
||||
dir := filepath.Join("../..", "models/embedder/multilingual-e5-small")
|
||||
emb, err := router.NewONNXEmbedder(filepath.Join(dir, "model_quantized.onnx"), filepath.Join(dir, "tokenizer.json"), lib)
|
||||
if err != nil {
|
||||
t.Skipf("onnx embedder unavailable: %v", err)
|
||||
}
|
||||
defer emb.Close()
|
||||
|
||||
cases := []struct {
|
||||
utterance string
|
||||
want topicLabel
|
||||
floor func(string) bool
|
||||
}{
|
||||
{"какая температура на улице?", topicWeather, isWeatherQuery},
|
||||
{"какая температура в доме?", topicHome, isHomeQuery},
|
||||
{"сколько машин я посетил?", topicOther, nil},
|
||||
{"сколько сейчас градусов", topicWeather, isWeatherQuery},
|
||||
{"дождь будет вечером?", topicWeather, isWeatherQuery},
|
||||
{"тепло сегодня на улице?", topicWeather, isWeatherQuery},
|
||||
{"свет на кухне включен?", topicHome, isHomeQuery},
|
||||
{"что сейчас включено дома", topicHome, isHomeQuery},
|
||||
{"датчики в квартире что показывают", topicHome, isHomeQuery},
|
||||
{"просканируй сеть", topicNetwork, isNetworkQuery},
|
||||
{"сколько устройств в локальной сети", topicNetwork, isNetworkQuery},
|
||||
{"кто сейчас в сетке", topicNetwork, isNetworkQuery},
|
||||
// The case the margin exists for. It reads as network by 0.0055, and
|
||||
// isNetworkQuery says no, so it stays the complaint it is — a scan she
|
||||
// runs unasked is the behaviour the bounds prevent.
|
||||
{"вайфай опять отвалился", topicOther, isNetworkQuery},
|
||||
{"я уже приехал домой", topicOther, nil},
|
||||
{"что я говорил про погоду в москве", topicOther, nil},
|
||||
{"напомни полить цветы", topicOther, nil},
|
||||
{"что требует моего внимания сейчас", topicAttend, isAttentionQuery},
|
||||
{"что не так с базой данных", topicAttend, isAttentionQuery},
|
||||
{"есть что-то срочное на сегодня", topicAttend, isAttentionQuery},
|
||||
}
|
||||
|
||||
h := &reactiveHandler{embedder: emb}
|
||||
ctx := context.Background()
|
||||
h.topics.load(ctx, emb)
|
||||
if !h.topics.loaded {
|
||||
t.Fatal("topic seeds did not load")
|
||||
}
|
||||
right := 0
|
||||
for _, tc := range cases {
|
||||
vec, err := router.EmbedQuery(ctx, emb, tc.utterance)
|
||||
if err != nil {
|
||||
t.Fatalf("embed %q: %v", tc.utterance, err)
|
||||
}
|
||||
label, margin, ok := h.topics.best(vec)
|
||||
if !ok {
|
||||
t.Fatalf("best(%q) not ok", tc.utterance)
|
||||
}
|
||||
// What the gate would do, which is the thing under test: a label that
|
||||
// does not clear the margin is handed to that source's keyword floor.
|
||||
got := label
|
||||
if margin < topicMargin {
|
||||
got = topicOther
|
||||
if tc.floor != nil && tc.floor(tc.utterance) {
|
||||
got = label
|
||||
}
|
||||
}
|
||||
if got == tc.want {
|
||||
right++
|
||||
} else {
|
||||
t.Errorf("%q: %s by %.4f, want %s", tc.utterance, label, margin, tc.want)
|
||||
}
|
||||
t.Logf(" %-40s -> %-8s margin %.4f", tc.utterance, label, margin)
|
||||
}
|
||||
t.Logf("topics: %d/%d", right, len(cases))
|
||||
}
|
||||
+32
-2
@@ -55,6 +55,7 @@ import (
|
||||
"github.com/kami/maven/internal/crawl"
|
||||
"github.com/kami/maven/internal/dialogue"
|
||||
"github.com/kami/maven/internal/ipc"
|
||||
"github.com/kami/maven/internal/lexicon"
|
||||
"github.com/kami/maven/internal/memory"
|
||||
"github.com/kami/maven/internal/phraser"
|
||||
"github.com/kami/maven/internal/router"
|
||||
@@ -76,6 +77,11 @@ type reactiveHandler struct {
|
||||
tts tts.Synthesizer
|
||||
router *router.Router
|
||||
embedder router.Embedder // reused for note write/query (same model as the classifier)
|
||||
// topics — the embedded seed sets behind the weather, house and LAN
|
||||
// recognisers (topics.go). Same lifecycle as boundary below: zero value is
|
||||
// usable, loads on first query, and with no embedder it never loads and
|
||||
// each source falls back to its own keyword test.
|
||||
topics topicIndex
|
||||
// boundary — the embedded seed sets behind the personal boundary
|
||||
// (personalboundary.go). Zero value is usable and loads on first query;
|
||||
// with no embedder it never loads and the boundary uses personalMarkers.
|
||||
@@ -161,6 +167,7 @@ type reactiveHandler struct {
|
||||
// the y/n answer. ponytail: single slot, single-user box — a second act
|
||||
// while one waits overwrites it (last-asked wins); expires after confirmTTL.
|
||||
mu sync.Mutex
|
||||
lastRouted *routedTurn // the previous acted turn, for a spoken correction (repair.go)
|
||||
pending *pendingAct
|
||||
pendingRoutine *pendingRoutineConfirm // routine proposal awaiting y/n
|
||||
pendingHexis *pendingHexisExec // mutating Hexis capability awaiting y/n
|
||||
@@ -299,6 +306,22 @@ func (h *reactiveHandler) runTurn(ctx context.Context, text string, src turnSour
|
||||
return withNotice(expiredNotice, reply)
|
||||
}
|
||||
|
||||
// 4d. spoken correction — "нет, это была заметка" points at the previous
|
||||
// turn and names what it should have been (repair.go). Before routing,
|
||||
// like the confirm and clarify turns: routing the correction as a fresh
|
||||
// utterance files the correction itself instead of fixing anything.
|
||||
if reply, handled := h.resolveRepair(ctx, text); handled {
|
||||
return withNotice(expiredNotice, reply)
|
||||
}
|
||||
|
||||
// 4e. ordinal selection — "второй", "первую сделал" pick from the list she
|
||||
// just read (ordinal.go). Before routing, and only when a list is actually
|
||||
// bound to the session: with nothing offered, "второй" is an ordinary word
|
||||
// and keeps routing.
|
||||
if reply, handled := h.resolveCandidate(ctx, text); handled {
|
||||
return withNotice(expiredNotice, reply)
|
||||
}
|
||||
|
||||
// 5. route. An elliptical follow-up — "а завтра?" — is answered from the
|
||||
// previous turn instead (continuation.go): the intent is the part it is
|
||||
// missing, so no amount of routing recovers it, and the model's guess
|
||||
@@ -359,6 +382,13 @@ func (h *reactiveHandler) runTurn(ctx context.Context, text string, src turnSour
|
||||
}
|
||||
}
|
||||
|
||||
// Remember what this turn was routed as, so the next utterance can correct
|
||||
// it. Only turns she acts on: a clarify asked instead of acting, so there
|
||||
// is nothing yet to be wrong about.
|
||||
if !dec.Clarify {
|
||||
h.recordTurn(text, dec.Intent)
|
||||
}
|
||||
|
||||
// 8. action — execute the decision's intent. errors here surface as
|
||||
// short reply text (the user wants to know the action didn't land);
|
||||
// the round-trip stays alive.
|
||||
@@ -452,8 +482,8 @@ func (h *reactiveHandler) replySystem(ctx context.Context, dec router.Decision)
|
||||
// this arm was fixed for, so say what she can do instead.
|
||||
return onlyNearDaysReply
|
||||
}
|
||||
dow := ruWeekdays[day.Weekday()]
|
||||
month := ruMonths[day.Month()-1]
|
||||
dow := lexicon.Weekday(int(day.Weekday()))
|
||||
month := lexicon.MonthGenitive(int(day.Month()))
|
||||
return fmt.Sprintf("%s %s, %d %s %d года", prefix, dow, day.Day(), month, day.Year())
|
||||
case strings.Contains(u, "кто дома") || strings.Contains(u, "человек дома"):
|
||||
return "присутствие пока не подключено к голосовому запросу."
|
||||
|
||||
+10
-3
@@ -382,9 +382,12 @@ func buildRouter(emb router.Embedder, acts router.ActMatcher, threshold float64,
|
||||
// question and must keep reaching replySystem, while "что у меня сегодня"
|
||||
// is an agenda question and must not.
|
||||
grammars = append(grammars, router.AgendaQueryGrammars()...)
|
||||
// After the agenda rules: "расскажи, что у меня сегодня" is an agenda
|
||||
// question first and a narrative request second (Vikunja #498).
|
||||
grammars = append(grammars, router.NarrativeQueryGrammars()...)
|
||||
// Same reason as the agenda rules, for the feeds: "что нового в лентах?"
|
||||
// routed system and answered "пока не умею" (Vikunja #474).
|
||||
grammars = append(grammars, router.FeedQueryGrammar())
|
||||
// The list side of the same exposure: a phrasing with no possessive in it
|
||||
// ("список дел") routed system and never reached queryTasks (Vikunja #467).
|
||||
grammars = append(grammars, router.TaskListGrammar())
|
||||
grammars = append(grammars, router.ListGrammars()...)
|
||||
grammars = append(grammars, router.ReminderGrammar())
|
||||
// Last, and it matches any utterance shape — its Build is the filter. An
|
||||
@@ -392,6 +395,10 @@ func buildRouter(emb router.Embedder, acts router.ActMatcher, threshold float64,
|
||||
// rewrote the task text (Vikunja #467). After the rules above because a
|
||||
// marker never collides with a clock or agenda question.
|
||||
grammars = append(grammars, router.TaskCaptureGrammar())
|
||||
// After the capture marker, so "запиши" still wins over "расскажи", and
|
||||
// last overall because it matches on the first word alone: "расскажи про
|
||||
// X" is a world question the model called a fact (Vikunja #498).
|
||||
grammars = append(grammars, router.NarrativeQueryGrammars()...)
|
||||
return router.New(router.Config{
|
||||
Grammars: grammars,
|
||||
Classifier: cls,
|
||||
|
||||
@@ -1,6 +1,11 @@
|
||||
package main
|
||||
|
||||
import "testing"
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/kami/maven/internal/router"
|
||||
)
|
||||
|
||||
// TestExtractWeatherLocation — any place he names comes through, not just the
|
||||
// six that used to be in a table (Vikunja #421).
|
||||
@@ -31,3 +36,30 @@ func TestExtractWeatherLocation(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestCalendarStepsAsideForWeather — the defect (Vikunja #474). "какая сегодня
|
||||
// погода в Москве?" answered "на 02.08.2026 ничего нет.": the calendar matches
|
||||
// on a day word alone, and it sits above the weather source.
|
||||
func TestCalendarStepsAsideForWeather(t *testing.T) {
|
||||
h, api := contQueryHandler()
|
||||
for _, u := range []string{
|
||||
"какая сегодня погода в Москве?",
|
||||
"будет дождь завтра?",
|
||||
"сколько градусов сегодня?",
|
||||
} {
|
||||
if reply, ok := h.queryCalendar(context.Background(), &queryTurn{
|
||||
dec: router.Decision{Intent: router.IntentQuery, Utterance: u},
|
||||
}); ok {
|
||||
t.Errorf("the calendar claimed %q with %q", u, reply)
|
||||
}
|
||||
}
|
||||
if api.events != 0 {
|
||||
t.Errorf("CalendarEvents called %d times for weather questions, want 0", api.events)
|
||||
}
|
||||
// The agenda question it exists for still reaches it.
|
||||
if _, ok := h.queryCalendar(context.Background(), &queryTurn{
|
||||
dec: router.Decision{Intent: router.IntentQuery, Utterance: "что у меня сегодня?"},
|
||||
}); !ok {
|
||||
t.Fatal("the calendar stopped answering the agenda question")
|
||||
}
|
||||
}
|
||||
|
||||
+14
-1
@@ -23,6 +23,7 @@ import (
|
||||
"syscall"
|
||||
|
||||
"github.com/kami/maven/internal/audio"
|
||||
"github.com/kami/maven/internal/tts"
|
||||
"github.com/kami/maven/internal/worker"
|
||||
)
|
||||
|
||||
@@ -39,15 +40,27 @@ func run(args []string) error {
|
||||
model := flag.String("model", "", "path to piper onnx model file")
|
||||
espeakData := flag.String("espeak_data", "", "path to espeak-ng data directory")
|
||||
tashkeelModel := flag.String("tashkeel_model", "", "path to libtashkeel onnx model")
|
||||
lexiconPath := flag.String("lexicon", "", "path to the pronunciation dictionary (json, name to spelling)")
|
||||
|
||||
flag.CommandLine.Parse(args)
|
||||
|
||||
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM, syscall.SIGHUP)
|
||||
defer stop()
|
||||
|
||||
// Read before the handler is built: a dictionary he asked for and that
|
||||
// cannot be read is a startup failure, not a warning. Saying names wrong
|
||||
// in silence is the thing it exists to stop.
|
||||
lex, err := tts.LoadLexicon(*lexiconPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if lex.Size() > 0 {
|
||||
log.Printf("mavttsd: pronunciation dictionary: %d names from %s", lex.Size(), *lexiconPath)
|
||||
}
|
||||
|
||||
var s worker.Synthesizer
|
||||
if *piperBin != "" && *model != "" {
|
||||
s = newPiperHandler(*piperBin, *model, *espeakData, *tashkeelModel)
|
||||
s = newPiperHandler(*piperBin, *model, *espeakData, *tashkeelModel, lex)
|
||||
log.Printf("mavttsd: using piper tts (%s, model=%s)", *piperBin, *model)
|
||||
} else {
|
||||
log.Printf("mavttsd: no piper/model specified, using stub handler")
|
||||
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"testing"
|
||||
|
||||
"github.com/kami/maven/internal/audio"
|
||||
"github.com/kami/maven/internal/tts"
|
||||
"github.com/kami/maven/internal/worker"
|
||||
)
|
||||
|
||||
@@ -117,3 +118,18 @@ func abs(n int) int {
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// The dictionary that ships in deploy/ must parse and must be non-empty. It is
|
||||
// data, so nothing else would catch a trailing comma before the voice did.
|
||||
func TestShippedLexiconLoads(t *testing.T) {
|
||||
lex, err := tts.LoadLexicon(filepath.Join("..", "..", "deploy", "tts-lexicon.json"))
|
||||
if err != nil {
|
||||
t.Fatalf("deploy/tts-lexicon.json: %v", err)
|
||||
}
|
||||
if lex.Size() < 10 {
|
||||
t.Errorf("shipped dictionary holds %d names, want the full list", lex.Size())
|
||||
}
|
||||
if got := lex.Apply("задача в Vikunja"); got == "задача в Vikunja" {
|
||||
t.Error("the shipped dictionary did not rewrite a name it lists")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -9,6 +9,7 @@ import (
|
||||
"os/exec"
|
||||
|
||||
"github.com/kami/maven/internal/audio"
|
||||
"github.com/kami/maven/internal/tts"
|
||||
"github.com/kami/maven/internal/worker"
|
||||
)
|
||||
|
||||
@@ -18,15 +19,20 @@ type piperHandler struct {
|
||||
configPath string
|
||||
espeakData string
|
||||
tashkeelModel string
|
||||
// lexicon rewrites service ids and Latin names into the spelling the
|
||||
// Russian voice reads correctly (Vikunja #458). Nil-safe: an unconfigured
|
||||
// dictionary rewrites nothing.
|
||||
lexicon *tts.Lexicon
|
||||
}
|
||||
|
||||
func newPiperHandler(piperPath, modelPath, espeakData, tashkeelModel string) *piperHandler {
|
||||
func newPiperHandler(piperPath, modelPath, espeakData, tashkeelModel string, lexicon *tts.Lexicon) *piperHandler {
|
||||
return &piperHandler{
|
||||
piperPath: piperPath,
|
||||
modelPath: modelPath,
|
||||
configPath: modelPath + ".json",
|
||||
espeakData: espeakData,
|
||||
tashkeelModel: tashkeelModel,
|
||||
lexicon: lexicon,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -66,7 +72,14 @@ func (h *piperHandler) Synthesize(ctx context.Context, req worker.SynthesizeReq)
|
||||
return worker.SynthesizeResp{}, fmt.Errorf("piper: start: %w", err)
|
||||
}
|
||||
|
||||
if _, err := io.WriteString(stdin, req.Text); err != nil {
|
||||
// The dictionary is applied here, at the last edge before the voice: every
|
||||
// caller's text passes through this one point, and nothing upstream has to
|
||||
// know how a name is spelled out loud.
|
||||
text := req.Text
|
||||
if h.lexicon != nil {
|
||||
text = h.lexicon.Apply(text)
|
||||
}
|
||||
if _, err := io.WriteString(stdin, text); err != nil {
|
||||
stdin.Close()
|
||||
stdout.Close()
|
||||
_ = cmd.Wait()
|
||||
|
||||
@@ -139,7 +139,13 @@ func TestHandleAmbientIgnoresNonMeetings(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestHandleAmbientAuth(t *testing.T) {
|
||||
body := `{"title":"Планёрка 10:00","posted_at":"2026-08-03T09:40:00Z"}`
|
||||
// posted_at carries the local offset, and the clock reading inside the text
|
||||
// sits twenty minutes after it. A bare "Z" here would make the reading
|
||||
// stale by the test machine's own offset and the handler would answer 202
|
||||
// no-meeting, which says nothing about the auth this test is checking
|
||||
// (Vikunja #482).
|
||||
posted := time.Date(2026, 8, 3, 9, 40, 0, 0, time.Local)
|
||||
body := fmt.Sprintf(`{"title":"Планёрка 10:00","posted_at":%q}`, posted.Format(time.RFC3339))
|
||||
|
||||
newReq := func(hdr, val string) *http.Request {
|
||||
r := httptest.NewRequest(http.MethodPost, "/api/ambient", strings.NewReader(body))
|
||||
|
||||
+46
-1
@@ -946,6 +946,51 @@ func deliveryRows(as []ipc.DeliveryAttempt) []deliveryRow {
|
||||
return out
|
||||
}
|
||||
|
||||
// reminderRow is one line on /reminders, with the payload unwrapped and both
|
||||
// timestamps already in his clock.
|
||||
//
|
||||
// The page rendered `{{.Payload}}` and the UTC instant, so a reminder read
|
||||
// `{"text":"выпить таблетки"}` and fired an hour off what he was told
|
||||
// (Vikunja #469). Neither is a formatting nicety: the envelope is an internal
|
||||
// shape he never chose, and a time on a page he reads is the time on his wall.
|
||||
type reminderRow struct {
|
||||
Created string
|
||||
Fires string
|
||||
Status string
|
||||
Text string
|
||||
}
|
||||
|
||||
// reminderText unwraps the {"text":...} payload the router writes.
|
||||
//
|
||||
// A copy of store.ReminderText rather than a call to it, because mavweb is one
|
||||
// of the pure-Go daemons and internal/store carries the CGO sqlite driver. The
|
||||
// ipc DTO is decoupled from the store on purpose, so the unwrap belongs to
|
||||
// whoever renders it. Payload that is not that shape is shown as he said it.
|
||||
func reminderText(payload string) string {
|
||||
var m map[string]any
|
||||
if err := json.Unmarshal([]byte(payload), &m); err == nil {
|
||||
if t, ok := m["text"]; ok {
|
||||
if s, isStr := t.(string); isStr && s != "" {
|
||||
return s
|
||||
}
|
||||
}
|
||||
}
|
||||
return strings.TrimSpace(payload)
|
||||
}
|
||||
|
||||
func reminderRows(rs []ipc.Reminder) []reminderRow {
|
||||
out := make([]reminderRow, 0, len(rs))
|
||||
for _, r := range rs {
|
||||
out = append(out, reminderRow{
|
||||
Created: r.CreatedTs.Local().Format("02 Jan 15:04"),
|
||||
Fires: r.FireTs.Local().Format("02 Jan 15:04"),
|
||||
Status: r.Status,
|
||||
Text: reminderText(r.Payload),
|
||||
})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func handleReminders(w http.ResponseWriter, r *http.Request, core ipc.CoreAPI) {
|
||||
if core == nil {
|
||||
http.Error(w, "reminders disabled (no -core)", http.StatusServiceUnavailable)
|
||||
@@ -959,7 +1004,7 @@ func handleReminders(w http.ResponseWriter, r *http.Request, core ipc.CoreAPI) {
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
if err := remindersTmpl.Execute(w, map[string]any{"Reminders": reminders}); err != nil {
|
||||
if err := remindersTmpl.Execute(w, map[string]any{"Reminders": reminderRows(reminders)}); err != nil {
|
||||
log.Printf("reminders template: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3,10 +3,10 @@
|
||||
{{if .Reminders}}<div class=scroll><table>
|
||||
<tr><th>created</th><th>fires</th><th>status</th><th>what</th></tr>
|
||||
{{range .Reminders}}<tr>
|
||||
<td class=hint>{{.CreatedTs.Format "02 Jan 15:04"}}</td>
|
||||
<td>{{.FireTs.Format "02 Jan 15:04"}}</td>
|
||||
<td class=hint>{{.Created}}</td>
|
||||
<td>{{.Fires}}</td>
|
||||
<td><span class="badge {{.Status}}">{{.Status}}</span></td>
|
||||
<td class=text-max>{{.Payload}}</td>
|
||||
<td class=text-max>{{.Text}}</td>
|
||||
</tr>{{end}}</table></div>
|
||||
{{else}}<div class=empty>
|
||||
<svg class=icon width="24" height="24"><use href="/ethos-icons.svg#i-calendar"/></svg>
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/kami/maven/internal/ipc"
|
||||
)
|
||||
|
||||
// The page showed the storage envelope and the UTC instant (Vikunja #469).
|
||||
func TestReminderRowsUnwrapAndLocalise(t *testing.T) {
|
||||
fire := time.Date(2026, 8, 4, 18, 30, 0, 0, time.UTC)
|
||||
rows := reminderRows([]ipc.Reminder{{
|
||||
CreatedTs: fire.Add(-time.Hour),
|
||||
FireTs: fire,
|
||||
Status: "pending",
|
||||
Payload: `{"text":"выпить таблетки"}`,
|
||||
}})
|
||||
if len(rows) != 1 {
|
||||
t.Fatalf("rows = %d, want 1", len(rows))
|
||||
}
|
||||
if rows[0].Text != "выпить таблетки" {
|
||||
t.Errorf("Text = %q, want the words without the envelope", rows[0].Text)
|
||||
}
|
||||
if want := fire.Local().Format("02 Jan 15:04"); rows[0].Fires != want {
|
||||
t.Errorf("Fires = %q, want %q", rows[0].Fires, want)
|
||||
}
|
||||
if strings.Contains(rows[0].Text, "{") {
|
||||
t.Errorf("Text still carries JSON: %q", rows[0].Text)
|
||||
}
|
||||
}
|
||||
|
||||
// A payload that is not the envelope is his own words, so it is shown as it is.
|
||||
func TestReminderTextKeepsPlainPayload(t *testing.T) {
|
||||
for _, tc := range []struct{ in, want string }{
|
||||
{`{"text":"позвонить маме"}`, "позвонить маме"},
|
||||
{" полить цветы ", "полить цветы"},
|
||||
{`{"body":"nope"}`, `{"body":"nope"}`},
|
||||
{"", ""},
|
||||
} {
|
||||
if got := reminderText(tc.in); got != tc.want {
|
||||
t.Errorf("reminderText(%q) = %q, want %q", tc.in, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"Maven": "Мэйвен",
|
||||
"Nexus": "Нексус",
|
||||
"Praxis": "Праксис",
|
||||
"Hexis": "Хексис",
|
||||
"Vikunja": "Викунья",
|
||||
"SearXNG": "сёрчиксэнджи",
|
||||
"Kiwix": "Кивикс",
|
||||
"Gitea": "Гитея",
|
||||
"Home Assistant": "Хоум Ассистент",
|
||||
"Docker": "Докер",
|
||||
"Telegram": "Телеграм",
|
||||
"ntfy": "энтифай",
|
||||
"homesrv": "хоумсёрв",
|
||||
"workpc": "воркписи",
|
||||
"whisper": "виспер",
|
||||
"piper": "пайпер",
|
||||
"llama-server": "лама сервер",
|
||||
"Qwen": "Квен",
|
||||
"CalDAV": "калдав",
|
||||
"IMAP": "аймап",
|
||||
"API": "эй-пи-ай",
|
||||
"CPU": "си-пи-ю",
|
||||
"GPU": "джи-пи-ю",
|
||||
"RAM": "оперативная память",
|
||||
"SSD": "эс-эс-ди",
|
||||
"uptime": "аптайм",
|
||||
"backup": "бэкап",
|
||||
"deploy": "деплой"
|
||||
}
|
||||
@@ -223,6 +223,41 @@ Not alternatives — layers:
|
||||
Router contract: `[{"intent":<enum>, key?, value?, text?, verb?}, ...]` over
|
||||
7 intents (`fact, reminder, note, query, act, chat, system`).
|
||||
|
||||
#### "второй" points at the list she just read
|
||||
|
||||
Landed 2026-08-04 (Vikunja #448). The dialogue session carried the intent, the
|
||||
slots and the history, and not the list. She recited five tasks, he said
|
||||
"второй", and the word had nothing to point at.
|
||||
|
||||
`Session.Candidates` holds what she just offered, bound at the moment she speaks
|
||||
it and in the order she speaks it (`tasks.Spoken`). Binding afterwards would
|
||||
resolve the word against a fresh query, and the list changes between two turns.
|
||||
`cmd/mavend/ordinal.go` reads the position before routing and dispatches on the
|
||||
candidate's kind.
|
||||
|
||||
An ordinal with no verb is read back, not acted on — "второй" names a task, it
|
||||
does not say what to do with it. With a verb ("первую сделал", "последнюю
|
||||
убери") the task moves and the list is spent, because a second ordinal against a
|
||||
list that no longer holds closes the wrong work. A position she never read is
|
||||
answered with how many she did read, not routed as a fresh sentence.
|
||||
|
||||
#### Saying she got it wrong is a feature
|
||||
|
||||
Landed 2026-08-04 (Vikunja #455). `Router.CorrectMisroute` could always append a
|
||||
corrected utterance as a new classifier example, and until now nothing in the
|
||||
daemon called it, so the mechanism existed and the behaviour did not.
|
||||
|
||||
`cmd/mavend/repair.go` reaches it. He says she got it wrong and names what it
|
||||
should have been — "нет, это заметка", "это не напоминание, а факт" — and three
|
||||
things happen in one turn: the classifier learns the utterance under the named
|
||||
intent, the request is redone under it, and she says the correction landed. The
|
||||
utterance he is correcting TO is the one with no "не" in front of it.
|
||||
|
||||
Read before routing, next to the confirm and clarify turns, because a correction
|
||||
routed as a fresh utterance files the correction itself. One turn is correctable
|
||||
once, inside five minutes, and only turns she acted on — a clarify asked instead
|
||||
of acting, so there is nothing yet to be wrong about.
|
||||
|
||||
#### A restart expires a parked question
|
||||
|
||||
Decided 2026-08-04 (Vikunja #385). The follow-up dialogue session survives a
|
||||
|
||||
@@ -90,6 +90,13 @@ export LD_LIBRARY_PATH="$ROOT/deps/piper"
|
||||
|
||||
Without `-piper` it runs as a stub.
|
||||
|
||||
`-lexicon deploy/tts-lexicon.json` adds the pronunciation dictionary: a flat
|
||||
JSON object of name to Russian spelling, applied to the text just before piper
|
||||
reads it. It is how `Vikunja` is said as a word rather than spelled out, and how
|
||||
`SearXNG` and `homesrv` are said at all. Off unless the flag is set; a path that
|
||||
is set and unreadable stops mavttsd rather than letting it say names wrong in
|
||||
silence. Adding a name needs a restart of mavttsd and nothing else.
|
||||
|
||||
## mavweb — PWA voice bridge (WebSocket ↔ TCP)
|
||||
|
||||
No CGo, no deps; builds with stock Go.
|
||||
|
||||
@@ -10,12 +10,15 @@ require (
|
||||
modernc.org/sqlite v1.54.0
|
||||
)
|
||||
|
||||
require github.com/kami/hexis v0.0.0
|
||||
require (
|
||||
github.com/aaaton/golem/v4 v4.0.2
|
||||
github.com/aaaton/golem/v4/dicts/ru v0.0.0-20250408131944-3488790fc110
|
||||
github.com/kami/hexis v0.0.0
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/dustin/go-humanize v1.0.1 // indirect
|
||||
github.com/google/uuid v1.6.0 // indirect
|
||||
github.com/kami/praxis v0.0.0
|
||||
github.com/mattn/go-isatty v0.0.20 // indirect
|
||||
github.com/ncruces/go-strftime v1.0.0 // indirect
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
|
||||
@@ -25,8 +28,4 @@ require (
|
||||
modernc.org/memory v1.11.0 // indirect
|
||||
)
|
||||
|
||||
replace github.com/kami/praxis v0.0.0 => /home/kami/apps/praxis
|
||||
|
||||
replace github.com/kami/nexus v0.0.0 => /home/kami/apps/nexus
|
||||
|
||||
replace github.com/kami/hexis v0.0.0 => /home/kami/apps/hexis
|
||||
|
||||
@@ -1,3 +1,8 @@
|
||||
github.com/aaaton/golem/v4 v4.0.0/go.mod h1:OfK/S5v9Exsx1yO21WorREuIVV+Y5K2hygP0A9oJCCI=
|
||||
github.com/aaaton/golem/v4 v4.0.2 h1:m4FvpSL8Zcv7XjmrKiBP7dp5FzhPCji9FcQRcH6T23k=
|
||||
github.com/aaaton/golem/v4 v4.0.2/go.mod h1:OfK/S5v9Exsx1yO21WorREuIVV+Y5K2hygP0A9oJCCI=
|
||||
github.com/aaaton/golem/v4/dicts/ru v0.0.0-20250408131944-3488790fc110 h1:aRLhKltXUyZg/7DoZE5PcNhETfTjMBXBEzD/JVRVVN4=
|
||||
github.com/aaaton/golem/v4/dicts/ru v0.0.0-20250408131944-3488790fc110/go.mod h1:n14MqOgbLBidXRIvLw9H3/vFyE4+PcjVxYOu05f55R4=
|
||||
github.com/coder/websocket v1.8.12 h1:5bUXkEPPIbewrnkU8LTCLVaxi4N4J8ahufH2vlo4NAo=
|
||||
github.com/coder/websocket v1.8.12/go.mod h1:LNVeNrXQZfe5qhS9ALED3uA+l5pPqvwXg3CKoDBB2gs=
|
||||
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
|
||||
|
||||
@@ -4,6 +4,8 @@ import (
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
|
||||
"github.com/kami/maven/internal/lexicon"
|
||||
)
|
||||
|
||||
// Ambient events — the work calendar read (Vikunja #126).
|
||||
@@ -48,18 +50,6 @@ type Notification struct {
|
||||
// reposting a notification for a meeting already under way.
|
||||
const ambientPastGrace = 2 * time.Hour
|
||||
|
||||
// dayWords maps the words that move a notification off Posted's day. Only
|
||||
// explicit ones: an offset is a claim about which day, and guessing which day
|
||||
// is exactly the guess this parse refuses to make.
|
||||
var dayWords = map[string]int{
|
||||
"завтра": 1,
|
||||
"tomorrow": 1,
|
||||
"сегодня": 0,
|
||||
"today": 0,
|
||||
"tonight": 0,
|
||||
"послезавтра": 2,
|
||||
}
|
||||
|
||||
// EventFromNotification turns a notification into the event it describes, or
|
||||
// reports false when it does not clearly describe one.
|
||||
//
|
||||
@@ -68,10 +58,19 @@ var dayWords = map[string]int{
|
||||
// word ("завтра", "tomorrow") when the notification carries one, and the result
|
||||
// is refused if it lands more than ambientPastGrace in the past. A bare start
|
||||
// time gets DefaultReminderDuration.
|
||||
//
|
||||
// The clock reading is read in the daemon's zone (Vikunja #482). Posted is an
|
||||
// instant and carries an offset; "созвон в 14:30" is a wall clock and carries
|
||||
// none, so the zone has to come from somewhere else. A relay that posts
|
||||
// "2026-08-02T09:00:00Z" used to make that 14:30 UTC, which stored an 18:30
|
||||
// meeting on a UTC+4 box — wrong by the deploy's own offset, and invisible on a
|
||||
// UTC box. The owner's phone and the box share a zone, so the box's zone is the
|
||||
// honest reading of a bare wall clock.
|
||||
func EventFromNotification(n Notification) (Event, bool) {
|
||||
if n.Posted.IsZero() {
|
||||
return Event{}, false
|
||||
}
|
||||
n.Posted = n.Posted.In(time.Local)
|
||||
line := strings.TrimSpace(n.Title + " " + n.Text)
|
||||
start, end, ok := parseTimeRange(line)
|
||||
if !ok {
|
||||
@@ -104,11 +103,16 @@ func EventFromNotification(n Notification) (Event, bool) {
|
||||
}
|
||||
|
||||
// dayOffset reports how many days off Posted's day the notification puts the
|
||||
// event. Words are matched whole, so "послезавтра" is not read as "завтра".
|
||||
// event. Only explicit words move it: an offset is a claim about which day, and
|
||||
// guessing which day is exactly the guess this parse refuses to make.
|
||||
//
|
||||
// The words are a closed class and live complete in internal/lexicon (Vikunja
|
||||
// #525), which is how "позавчера" resolves here now — it never did while the map
|
||||
// was inline. Matched whole, so "послезавтра" is not read as "завтра".
|
||||
func dayOffset(line string) int {
|
||||
for _, f := range strings.Fields(strings.ToLower(line)) {
|
||||
f = strings.Trim(f, ".,;:!?—–-()\"'«»")
|
||||
if off, ok := dayWords[f]; ok {
|
||||
if off, ok := lexicon.DayOffset(f); ok {
|
||||
return off
|
||||
}
|
||||
}
|
||||
@@ -121,7 +125,7 @@ func dayOffset(line string) int {
|
||||
func stripDayWords(s string) string {
|
||||
out := make([]string, 0, 8)
|
||||
for _, f := range strings.Fields(s) {
|
||||
if _, ok := dayWords[strings.Trim(strings.ToLower(f), ".,;:!?—–-()\"'«»")]; ok {
|
||||
if _, ok := lexicon.DayOffset(strings.Trim(f, ".,;:!?—–-()\"'«»")); ok {
|
||||
continue
|
||||
}
|
||||
out = append(out, f)
|
||||
|
||||
@@ -1,12 +1,22 @@
|
||||
package calendar
|
||||
|
||||
import (
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// A bare clock reading in a notification is read in the daemon's zone, so every
|
||||
// test here needs a known one. UTC+4 is the deploy's (Europe/Samara) and it is
|
||||
// the offset the 18:30 bug was measured at, so a regression shows up as four
|
||||
// hours rather than as nothing at all on a UTC runner.
|
||||
func TestMain(m *testing.M) {
|
||||
time.Local = time.FixedZone("+04", 4*3600)
|
||||
os.Exit(m.Run())
|
||||
}
|
||||
|
||||
func TestEventFromNotification(t *testing.T) {
|
||||
posted := time.Date(2026, 8, 3, 9, 40, 0, 0, time.FixedZone("+04", 4*3600))
|
||||
posted := time.Date(2026, 8, 3, 9, 40, 0, 0, time.Local)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
@@ -98,10 +108,10 @@ func TestEventFromNotification(t *testing.T) {
|
||||
if !ev.End.After(ev.Start) {
|
||||
t.Errorf("end %v must be after start %v", ev.End, ev.Start)
|
||||
}
|
||||
// The event lands on the day the phone showed it, in the phone's
|
||||
// location — not shifted into UTC.
|
||||
if ev.Start.Location() != posted.Location() {
|
||||
t.Errorf("location = %v, want %v", ev.Start.Location(), posted.Location())
|
||||
// The event lands on the day the phone showed it, in the daemon's
|
||||
// zone — the clock reading is a wall clock, not an instant.
|
||||
if ev.Start.Location() != time.Local {
|
||||
t.Errorf("location = %v, want %v", ev.Start.Location(), time.Local)
|
||||
}
|
||||
if y, m, d := ev.Start.Date(); y != 2026 || m != time.August || d != 3 {
|
||||
t.Errorf("date = %d-%02d-%02d, want 2026-08-03", y, m, d)
|
||||
@@ -115,8 +125,7 @@ func TestEventFromNotification(t *testing.T) {
|
||||
// the meeting twelve hours in the past and filed it under today in FactKey. A
|
||||
// wrong meeting stored is worse than nothing stored.
|
||||
func TestEventFromNotificationDayWords(t *testing.T) {
|
||||
loc := time.FixedZone("+04", 4*3600)
|
||||
evening := time.Date(2026, 8, 3, 21, 0, 0, 0, loc)
|
||||
evening := time.Date(2026, 8, 3, 21, 0, 0, 0, time.Local)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
@@ -191,7 +200,7 @@ func TestEventFromNotificationDayWords(t *testing.T) {
|
||||
func TestEventFromNotificationDropsDayWordFromSummary(t *testing.T) {
|
||||
ev, ok := EventFromNotification(Notification{
|
||||
Title: "Завтра Планёрка 09:00",
|
||||
Posted: time.Date(2026, 8, 3, 21, 0, 0, 0, time.UTC),
|
||||
Posted: time.Date(2026, 8, 3, 21, 0, 0, 0, time.Local),
|
||||
})
|
||||
if !ok {
|
||||
t.Fatal("expected an event")
|
||||
@@ -201,6 +210,31 @@ func TestEventFromNotificationDropsDayWordFromSummary(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// Vikunja #482. A relay that posts its instant as UTC used to hand the wall
|
||||
// clock inside the text the same zone, so "созвон в 14:30" was stored as 14:30Z
|
||||
// and read back as 18:30 on a UTC+4 box — late by exactly the deploy's offset,
|
||||
// and correct-looking on a UTC one. Nobody writes a notification meaning 14:30Z.
|
||||
func TestEventFromNotificationReadsTheClockAsLocalTime(t *testing.T) {
|
||||
ev, ok := EventFromNotification(Notification{
|
||||
Package: "com.slack",
|
||||
Title: "Standup",
|
||||
Text: "созвон в 14:30",
|
||||
Posted: time.Date(2026, 8, 2, 9, 0, 0, 0, time.UTC), // 13:00 local
|
||||
})
|
||||
if !ok {
|
||||
t.Fatal("expected an event")
|
||||
}
|
||||
if got := ev.Start.Format("15:04"); got != "14:30" {
|
||||
t.Errorf("start = %s, want 14:30 local", got)
|
||||
}
|
||||
if ev.Start.Location() != time.Local {
|
||||
t.Errorf("location = %v, want %v", ev.Start.Location(), time.Local)
|
||||
}
|
||||
if got, want := FactKey(ev), "calendar_event_20260802_Standup"; got != want {
|
||||
t.Errorf("fact key = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestEventFromNotificationNeedsPostedAt(t *testing.T) {
|
||||
if _, ok := EventFromNotification(Notification{Title: "Планёрка 10:00"}); ok {
|
||||
t.Error("a notification with no posted_at has no date to sit on")
|
||||
@@ -211,7 +245,7 @@ func TestEventFromNotificationNeedsPostedAt(t *testing.T) {
|
||||
func TestAmbientEventsAreStoredAtReducedConfidence(t *testing.T) {
|
||||
ev, ok := EventFromNotification(Notification{
|
||||
Title: "Планёрка 10:00-10:30",
|
||||
Posted: time.Date(2026, 8, 3, 9, 0, 0, 0, time.UTC),
|
||||
Posted: time.Date(2026, 8, 3, 9, 0, 0, 0, time.Local),
|
||||
})
|
||||
if !ok {
|
||||
t.Fatal("expected an event")
|
||||
|
||||
@@ -42,12 +42,27 @@ type Turn struct {
|
||||
Text string // raw utterance
|
||||
}
|
||||
|
||||
// Candidate — one item she just read out loud, kept so his next words can
|
||||
// pick it ("второй", "первую сделал"). Vikunja #448.
|
||||
//
|
||||
// Bound at the moment she speaks the list, not resolved afterwards: the list
|
||||
// can change between two turns, and "второй" means the second thing she said,
|
||||
// not the second row of a fresh query.
|
||||
type Candidate struct {
|
||||
Kind string // what it is, e.g. "task" — the resolver dispatches on this
|
||||
Ref int64 // the row it points at
|
||||
Label string // what she called it, so she can repeat it back
|
||||
}
|
||||
|
||||
type Session struct {
|
||||
Intent Intent
|
||||
Slots Slots
|
||||
Timestamp time.Time
|
||||
TTL time.Duration
|
||||
History []Turn // most recent turns, newest last; used for anaphora + cross-intent
|
||||
// Candidates — the list she just offered, in the order she said it. Empty
|
||||
// on every turn that offered no choice, which is most of them.
|
||||
Candidates []Candidate
|
||||
}
|
||||
|
||||
func (s *Session) IsExpired(now time.Time) bool {
|
||||
@@ -143,6 +158,26 @@ func (s *SessionStore) Put(id string, sess *Session) {
|
||||
s.save(id, sess)
|
||||
}
|
||||
|
||||
// SetCandidates attaches a just-spoken list to the live session.
|
||||
//
|
||||
// In place rather than through Put, because the turn was already remembered by
|
||||
// the time the answer was built: replacing the session here would drop the
|
||||
// slots the next follow-up inherits. No session, no candidates — a choice with
|
||||
// no turn behind it has nothing to be a choice about.
|
||||
func (s *SessionStore) SetCandidates(id string, now time.Time, cands []Candidate) {
|
||||
s.mu.Lock()
|
||||
sess, ok := s.sessions[id]
|
||||
if ok && !sess.IsExpired(now) {
|
||||
sess.Candidates = cands
|
||||
} else {
|
||||
ok = false
|
||||
}
|
||||
s.mu.Unlock()
|
||||
if ok {
|
||||
s.save(id, sess)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *SessionStore) Delete(id string) {
|
||||
s.mu.Lock()
|
||||
delete(s.sessions, id)
|
||||
|
||||
@@ -0,0 +1,229 @@
|
||||
// Package lexicon holds the Russian word sets that can be finished.
|
||||
//
|
||||
// A closed class has a fixed number of members: the language has as many
|
||||
// interrogative pronouns as it has, and no utterance will ever contain a
|
||||
// thirteenth month. Those sets belong in a data file, complete, and that is what
|
||||
// this package is (Vikunja #522, owner's call 2026-08-04 — "not pattern, 100%").
|
||||
//
|
||||
// It is the first of three mechanisms that replaced hand-written Russian
|
||||
// patterns, and the only one that answers with certainty. The other two are the
|
||||
// embedder, for recognising an open set of phrasings, and a morphological
|
||||
// dictionary, for questions about grammar. A list that can never be finished is
|
||||
// a guess dressed as a rule and does not go here.
|
||||
//
|
||||
// What this package does NOT do is match. It hands out sets and lookups; the
|
||||
// caller decides what to do with a hit, because "this token is an interrogative"
|
||||
// and "this utterance is a question" are different claims.
|
||||
package lexicon
|
||||
|
||||
import (
|
||||
"embed"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
"unicode"
|
||||
"unicode/utf8"
|
||||
)
|
||||
|
||||
//go:embed lexicon_ru_v1.json
|
||||
var files embed.FS
|
||||
|
||||
// ruFile is the versioned file this package reads. A new version is a new file,
|
||||
// not an edit to this one, so a caller pinned to v1 keeps the words it was
|
||||
// measured against.
|
||||
const ruFile = "lexicon_ru_v1.json"
|
||||
|
||||
type lexiconFile struct {
|
||||
SchemaVersion int `json:"schema_version"`
|
||||
Name string `json:"name"`
|
||||
Sets map[string]lexiSet `json:"sets"`
|
||||
}
|
||||
|
||||
type lexiSet struct {
|
||||
Note string `json:"note"`
|
||||
Words []string `json:"words"`
|
||||
Values map[string]int `json:"values"`
|
||||
}
|
||||
|
||||
// ru is parsed once at init. A malformed embedded file is a build-time mistake
|
||||
// that survived to runtime, and there is no sane degraded behaviour for "the
|
||||
// months are missing", so it panics rather than answering with an empty set.
|
||||
var ru = mustLoad()
|
||||
|
||||
func mustLoad() lexiconFile {
|
||||
data, err := files.ReadFile(ruFile)
|
||||
if err != nil {
|
||||
panic(fmt.Sprintf("lexicon: read %s: %v", ruFile, err))
|
||||
}
|
||||
var f lexiconFile
|
||||
if err := json.Unmarshal(data, &f); err != nil {
|
||||
panic(fmt.Sprintf("lexicon: parse %s: %v", ruFile, err))
|
||||
}
|
||||
for _, name := range []string{
|
||||
"interrogatives", "capture_verbs", "narrative_requests", "cardinals",
|
||||
"day_offsets", "weekdays", "months_genitive", "hours_spoken",
|
||||
"not_place_after_v",
|
||||
} {
|
||||
s, ok := f.Sets[name]
|
||||
if !ok || (len(s.Words) == 0 && len(s.Values) == 0) {
|
||||
panic(fmt.Sprintf("lexicon: %s has no set %q", ruFile, name))
|
||||
}
|
||||
}
|
||||
return f
|
||||
}
|
||||
|
||||
// words returns a copy of a word set, so a caller cannot edit the lexicon by
|
||||
// holding onto what it was given.
|
||||
func words(set string) []string {
|
||||
src := ru.Sets[set].Words
|
||||
out := make([]string, len(src))
|
||||
copy(out, src)
|
||||
return out
|
||||
}
|
||||
|
||||
// Interrogatives returns the question words, Russian and English.
|
||||
func Interrogatives() []string { return words("interrogatives") }
|
||||
|
||||
// CaptureVerbs returns the imperatives that mean "record this".
|
||||
func CaptureVerbs() []string { return words("capture_verbs") }
|
||||
|
||||
// NarrativeRequests returns the imperatives that mean "tell me about".
|
||||
func NarrativeRequests() []string { return words("narrative_requests") }
|
||||
|
||||
// RepairMarkers lists the ways he says the previous turn was routed wrong. See
|
||||
// the set's own note for why this one is a list and not a seed set.
|
||||
func RepairMarkers() []string { return words("repair_markers") }
|
||||
|
||||
// FirstPerson lists every form of the first-person pronoun. Callers use it to
|
||||
// decide that a sentence is about him: internal/router/complaint.go keeps a
|
||||
// complaint out of the fact store unless one of these appears, because losing a
|
||||
// fact he meant to store is the worse mistake.
|
||||
func FirstPerson() []string { return words("first_person") }
|
||||
|
||||
// NotPlaceAfterV returns the words that follow "в" without naming a place.
|
||||
func NotPlaceAfterV() []string { return words("not_place_after_v") }
|
||||
|
||||
// Cardinal reports the value of a spoken number word. The word is compared
|
||||
// lowercased and trimmed, because it arrives from a tokenizer that may not have
|
||||
// done either.
|
||||
func Cardinal(word string) (int, bool) {
|
||||
n, ok := ru.Sets["cardinals"].Values[norm(word)]
|
||||
return n, ok
|
||||
}
|
||||
|
||||
// DayOffset reports how many days a relative day word moves from today.
|
||||
//
|
||||
// The zero value is a real answer here — "сегодня" is offset 0 — so the second
|
||||
// return is the only way to tell a hit from a miss. Callers that used to switch
|
||||
// on strings.Contains had to order "послезавтра" before "завтра" by hand,
|
||||
// because one contains the other; a lookup has no such trap.
|
||||
func DayOffset(word string) (int, bool) {
|
||||
n, ok := ru.Sets["day_offsets"].Values[norm(word)]
|
||||
return n, ok
|
||||
}
|
||||
|
||||
// DayOffsetWords lists the relative day words themselves, sorted so the order is
|
||||
// stable across builds — a caller that folds them into a regexp alternation would
|
||||
// otherwise produce a different pattern every run. Map iteration order is why
|
||||
// this sorts rather than the caller.
|
||||
func DayOffsetWords() []string {
|
||||
vals := ru.Sets["day_offsets"].Values
|
||||
out := make([]string, 0, len(vals))
|
||||
for w := range vals {
|
||||
out = append(out, w)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
// DayOffsetIn finds a relative day word anywhere in a phrase and reports its
|
||||
// offset. Where two words appear, the one that moves furthest from today wins in
|
||||
// absolute terms: "не сегодня, а послезавтра" is about the day after tomorrow,
|
||||
// and the longest-match rule that picking the first word would need is exactly
|
||||
// what the old Contains switch got wrong.
|
||||
func DayOffsetIn(text string) (int, bool) {
|
||||
lower := norm(text)
|
||||
best, found := 0, false
|
||||
for word, n := range ru.Sets["day_offsets"].Values {
|
||||
if !containsWord(lower, word) {
|
||||
continue
|
||||
}
|
||||
if !found || abs(n) > abs(best) {
|
||||
best, found = n, true
|
||||
}
|
||||
}
|
||||
return best, found
|
||||
}
|
||||
|
||||
// Weekday returns the Russian name of a weekday index, Sunday first, matching
|
||||
// Go's time.Weekday. An index off the end returns "".
|
||||
func Weekday(i int) string { return at("weekdays", i) }
|
||||
|
||||
// MonthGenitive returns the month name a date takes — "10 июля", not "июль".
|
||||
// The set is 1-indexed, so MonthGenitive(int(t.Month())) is the whole call.
|
||||
func MonthGenitive(m int) string { return at("months_genitive", m) }
|
||||
|
||||
// HourSpoken returns an hour spelled out for the voice. 0 to 23.
|
||||
func HourSpoken(h int) string { return at("hours_spoken", h) }
|
||||
|
||||
func at(set string, i int) string {
|
||||
w := ru.Sets[set].Words
|
||||
if i < 0 || i >= len(w) {
|
||||
return ""
|
||||
}
|
||||
return w[i]
|
||||
}
|
||||
|
||||
func norm(s string) string { return strings.ToLower(strings.TrimSpace(s)) }
|
||||
|
||||
func abs(n int) int {
|
||||
if n < 0 {
|
||||
return -n
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// containsWord reports whether haystack holds needle on word boundaries. Go's
|
||||
// \b is ASCII-only and never fires after a Cyrillic letter, so the boundary is
|
||||
// checked here instead: a rune on either side must not be a letter or a digit.
|
||||
func containsWord(haystack, needle string) bool {
|
||||
if needle == "" {
|
||||
return false
|
||||
}
|
||||
from := 0
|
||||
for {
|
||||
i := strings.Index(haystack[from:], needle)
|
||||
if i < 0 {
|
||||
return false
|
||||
}
|
||||
i += from
|
||||
if boundaryBefore(haystack, i) && boundaryAfter(haystack, i+len(needle)) {
|
||||
return true
|
||||
}
|
||||
from = i + len(needle)
|
||||
if from >= len(haystack) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func boundaryBefore(s string, i int) bool {
|
||||
if i == 0 {
|
||||
return true
|
||||
}
|
||||
r, _ := utf8.DecodeLastRuneInString(s[:i])
|
||||
return !wordRune(r)
|
||||
}
|
||||
|
||||
func boundaryAfter(s string, i int) bool {
|
||||
if i >= len(s) {
|
||||
return true
|
||||
}
|
||||
r, _ := utf8.DecodeRuneInString(s[i:])
|
||||
return !wordRune(r)
|
||||
}
|
||||
|
||||
func wordRune(r rune) bool {
|
||||
return unicode.IsLetter(r) || unicode.IsDigit(r) || r == '_'
|
||||
}
|
||||
@@ -0,0 +1,139 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"name": "russian closed-class lexicons v1",
|
||||
"notes": [
|
||||
"Every set here is a CLOSED CLASS: the language has a fixed number of members and the list can be finished. That is why it is a list at all. A word list that can never be finished is a guess dressed as a rule, and it belongs with the embedder, not here (Vikunja #522).",
|
||||
"Editing a word is a data change. No Go change, no rebuild of a pattern, no second copy to keep in step — month names used to live in two files and day offsets in three.",
|
||||
"Interrogatives and capture verbs carry their English members too. He speaks both languages in one sentence and the router sees one utterance.",
|
||||
"The sets are matched over tokens, never as substrings: \"что\" inside \"чтобы\" and \"как\" inside \"какао\" are not questions.",
|
||||
"Order matters in weekdays, months and hours, and nowhere else. weekdays starts at Sunday because Go's time.Weekday does. months is 1-indexed with an empty slot at 0 for the same reason. hours is indexed by the hour itself.",
|
||||
"A form missing from a closed set is a bug report, not a judgement call. Add it."
|
||||
],
|
||||
"sets": {
|
||||
"interrogatives": {
|
||||
"note": "The Russian interrogative pronouns and adverbs, declined, plus the English ones. Closed class: this is the whole list, and a question word outside it does not exist.",
|
||||
"words": [
|
||||
"что", "чего", "чему", "чем", "чём",
|
||||
"кто", "кого", "кому", "кем", "ком",
|
||||
"какой", "какая", "какое", "какие", "какого", "какому", "каким", "каких", "какими", "каком",
|
||||
"который", "которая", "которое", "которые", "которого", "котором",
|
||||
"чей", "чья", "чьё", "чьи",
|
||||
"где", "куда", "откуда", "когда", "докуда",
|
||||
"почему", "зачем", "отчего", "как", "сколько", "насколько", "каково",
|
||||
"what", "who", "whom", "whose", "why", "when", "where", "which", "how"
|
||||
]
|
||||
},
|
||||
"capture_verbs": {
|
||||
"note": "An explicit instruction to record something, in the imperative he actually speaks. Not a closed class in the grammatical sense, but a closed set of the commands Maven answers to — it is her vocabulary, and its members are decided here rather than discovered.",
|
||||
"words": [
|
||||
"запиши", "запомни", "отметь", "заметь", "добавь", "сохрани", "занеси", "внеси",
|
||||
"note", "remember", "log", "save", "add"
|
||||
]
|
||||
},
|
||||
"narrative_requests": {
|
||||
"note": "\"Tell me about X\" asks for knowledge Maven does not hold about him. It carries no question mark and no interrogative, which is how \"расскажи про битву при Ватерлоо\" reached the fact store (#470).",
|
||||
"words": [
|
||||
"расскажи", "объясни", "опиши", "перечисли", "поясни",
|
||||
"tell", "explain", "describe", "list"
|
||||
]
|
||||
},
|
||||
"cardinals": {
|
||||
"note": "Number words as spoken, with the gender variants Russian requires: один/одна/одно and два/две agree with the noun that follows. Values are the number itself. Twenties and up are compounds and are read as their parts, so only the round members are listed.",
|
||||
"values": {
|
||||
"ноль": 0, "нуль": 0, "zero": 0,
|
||||
"один": 1, "одна": 1, "одно": 1, "one": 1,
|
||||
"два": 2, "две": 2, "two": 2,
|
||||
"три": 3, "three": 3,
|
||||
"четыре": 4, "four": 4,
|
||||
"пять": 5, "five": 5,
|
||||
"шесть": 6, "six": 6,
|
||||
"семь": 7, "seven": 7,
|
||||
"восемь": 8, "eight": 8,
|
||||
"девять": 9, "nine": 9,
|
||||
"десять": 10, "ten": 10,
|
||||
"одиннадцать": 11, "eleven": 11,
|
||||
"двенадцать": 12, "twelve": 12,
|
||||
"тринадцать": 13, "thirteen": 13,
|
||||
"четырнадцать": 14, "fourteen": 14,
|
||||
"пятнадцать": 15, "fifteen": 15,
|
||||
"шестнадцать": 16, "sixteen": 16,
|
||||
"семнадцать": 17, "seventeen": 17,
|
||||
"восемнадцать": 18, "eighteen": 18,
|
||||
"девятнадцать": 19, "nineteen": 19,
|
||||
"двадцать": 20, "twenty": 20,
|
||||
"тридцать": 30, "thirty": 30,
|
||||
"сорок": 40, "forty": 40,
|
||||
"пятьдесят": 50, "fifty": 50,
|
||||
"шестьдесят": 60, "sixty": 60,
|
||||
"семьдесят": 70, "seventy": 70,
|
||||
"восемьдесят": 80, "eighty": 80,
|
||||
"девяносто": 90, "ninety": 90,
|
||||
"сто": 100, "hundred": 100
|
||||
}
|
||||
},
|
||||
"day_offsets": {
|
||||
"note": "The words that name a day relative to today, and the number of days each one moves. Only explicit ones: an offset is a claim about which day, and guessing which day is the guess these callers refuse to make. Multi-word members are matched as a phrase.",
|
||||
"values": {
|
||||
"позавчера": -2,
|
||||
"вчера": -1,
|
||||
"yesterday": -1,
|
||||
"сегодня": 0,
|
||||
"today": 0,
|
||||
"tonight": 0,
|
||||
"завтра": 1,
|
||||
"tomorrow": 1,
|
||||
"послезавтра": 2,
|
||||
"day after tomorrow": 2
|
||||
}
|
||||
},
|
||||
"weekdays": {
|
||||
"note": "Nominative, starting at Sunday so the index is Go's time.Weekday.",
|
||||
"words": [
|
||||
"воскресенье", "понедельник", "вторник", "среда",
|
||||
"четверг", "пятница", "суббота"
|
||||
]
|
||||
},
|
||||
"months_genitive": {
|
||||
"note": "The form a date takes: \"10 июля\", not \"июль\". 1-indexed, so slot 0 is empty and month numbers need no arithmetic.",
|
||||
"words": [
|
||||
"", "января", "февраля", "марта", "апреля", "мая", "июня",
|
||||
"июля", "августа", "сентября", "октября", "ноября", "декабря"
|
||||
]
|
||||
},
|
||||
"hours_spoken": {
|
||||
"note": "Hours spelled out for the voice: \"3 ч\" is fine on a screen and wrong out loud. Indexed by the hour, 0 to 23.",
|
||||
"words": [
|
||||
"ноль", "один", "два", "три", "четыре", "пять", "шесть", "семь", "восемь",
|
||||
"девять", "десять", "одиннадцать", "двенадцать", "тринадцать",
|
||||
"четырнадцать", "пятнадцать", "шестнадцать", "семнадцать", "восемнадцать",
|
||||
"девятнадцать", "двадцать", "двадцать один", "двадцать два", "двадцать три"
|
||||
]
|
||||
},
|
||||
"repair_markers": {
|
||||
"note": "The ways he says she got it wrong. A closed set of Maven's own vocabulary, like capture_verbs: its members are decided here rather than discovered. NOT the embedder, and that is deliberate — a correction rewrites the previous turn and runs pre-route, before the turn vector exists, so a near-miss would redo a request he did not make. Phrases, matched as substrings.",
|
||||
"words": [
|
||||
"не так поняла", "неправильно поняла", "ты не поняла", "не поняла меня",
|
||||
"ты ошиблась", "это не", "а не", "не про то",
|
||||
"got it wrong", "not a ", "that was wrong"
|
||||
]
|
||||
},
|
||||
"first_person": {
|
||||
"note": "Every form of the first-person pronoun, plus the English ones. Closed class in the strictest sense: the language has these and no others. A sentence carrying one is about him, which is what makes it a fact rather than a passing complaint.",
|
||||
"words": [
|
||||
"я", "меня", "мне", "мной", "мною",
|
||||
"мы", "нас", "нам", "нами",
|
||||
"i", "me", "my", "mine", "we", "us", "our"
|
||||
]
|
||||
},
|
||||
"not_place_after_v": {
|
||||
"note": "Words that follow the preposition \"в\" without naming a place, so \"в общем\" and \"в котором часу\" are not read as a city we do not know.",
|
||||
"words": [
|
||||
"данный", "данную", "данное", "этот", "эту", "это", "том", "той",
|
||||
"котором", "которой", "какое", "какой", "который", "каком",
|
||||
"общем", "точности", "курсе", "принципе", "итоге", "целом",
|
||||
"сутках", "часах", "минутах", "секундах", "неделе", "месяце", "году",
|
||||
"начале", "конце", "середине", "течение", "течении"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,114 @@
|
||||
package lexicon
|
||||
|
||||
import "testing"
|
||||
|
||||
// TestClosedSetsAreComplete — the point of the package. A closed class can be
|
||||
// finished, so the test names the members that were MISSING from the inline Go
|
||||
// lists this package replaced (Vikunja #525) and every one of them has to be
|
||||
// there. Add to this list when a form turns up unhandled.
|
||||
func TestClosedSetsAreComplete(t *testing.T) {
|
||||
inter := map[string]bool{}
|
||||
for _, w := range Interrogatives() {
|
||||
inter[w] = true
|
||||
}
|
||||
// Instrumental and prepositional cases of что and кто, and the declined
|
||||
// какой. The old list had что/чего and nothing else, so "чем ты занята" and
|
||||
// "в каком часу" carried no interrogative at all.
|
||||
for _, w := range []string{"чем", "чём", "чему", "кем", "ком", "каком", "какими", "насколько"} {
|
||||
if !inter[w] {
|
||||
t.Errorf("interrogatives is missing %q", w)
|
||||
}
|
||||
}
|
||||
|
||||
for _, tc := range []struct {
|
||||
word string
|
||||
want int
|
||||
}{
|
||||
{"ноль", 0}, {"одна", 1}, {"две", 2}, {"одиннадцать", 11},
|
||||
{"пятнадцать", 15}, {"двадцать", 20}, {"сорок", 40}, {"девяносто", 90},
|
||||
{"сто", 100}, {"twelve", 12},
|
||||
} {
|
||||
got, ok := Cardinal(tc.word)
|
||||
if !ok || got != tc.want {
|
||||
t.Errorf("Cardinal(%q) = %d, %v; want %d, true", tc.word, got, ok, tc.want)
|
||||
}
|
||||
}
|
||||
if _, ok := Cardinal("бэкап"); ok {
|
||||
t.Error("Cardinal must not answer for a word that is not a number")
|
||||
}
|
||||
}
|
||||
|
||||
// TestDayOffsetHasNoOrderingTrap — the defect a lookup removes. The callers this
|
||||
// replaced used strings.Contains in a switch, so "послезавтра" had to be tested
|
||||
// before "завтра" by hand or the day after tomorrow read as tomorrow.
|
||||
func TestDayOffsetHasNoOrderingTrap(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
text string
|
||||
want int
|
||||
ok bool
|
||||
}{
|
||||
{"послезавтра", 2, true},
|
||||
{"завтра", 1, true},
|
||||
{"сегодня", 0, true},
|
||||
{"вчера", -1, true},
|
||||
{"позавчера", -2, true},
|
||||
{"встреча послезавтра в 14:30", 2, true},
|
||||
{"Tomorrow at 09:00", 1, true},
|
||||
{"не сегодня, а послезавтра", 2, true},
|
||||
{"в четверг", 0, false},
|
||||
{"завтраком", 0, false},
|
||||
} {
|
||||
got, ok := DayOffsetIn(tc.text)
|
||||
if ok != tc.ok || (ok && got != tc.want) {
|
||||
t.Errorf("DayOffsetIn(%q) = %d, %v; want %d, %v", tc.text, got, ok, tc.want, tc.ok)
|
||||
}
|
||||
}
|
||||
// "сегодня" is offset 0, which is also the zero value, so the second return
|
||||
// is the only thing that separates a hit from a miss.
|
||||
if n, ok := DayOffset("сегодня"); n != 0 || !ok {
|
||||
t.Errorf("DayOffset(сегодня) = %d, %v; want 0, true", n, ok)
|
||||
}
|
||||
}
|
||||
|
||||
// TestIndexedSetsLineUpWithTheirCallers — weekdays start at Sunday because Go's
|
||||
// time.Weekday does, and months are 1-indexed so a month number needs no
|
||||
// arithmetic. Off-by-one here is a wrong date spoken out loud.
|
||||
func TestIndexedSetsLineUpWithTheirCallers(t *testing.T) {
|
||||
if got := Weekday(0); got != "воскресенье" {
|
||||
t.Errorf("Weekday(0) = %q, want воскресенье", got)
|
||||
}
|
||||
if got := Weekday(1); got != "понедельник" {
|
||||
t.Errorf("Weekday(1) = %q, want понедельник", got)
|
||||
}
|
||||
if got := MonthGenitive(1); got != "января" {
|
||||
t.Errorf("MonthGenitive(1) = %q, want января", got)
|
||||
}
|
||||
if got := MonthGenitive(12); got != "декабря" {
|
||||
t.Errorf("MonthGenitive(12) = %q, want декабря", got)
|
||||
}
|
||||
if got := MonthGenitive(0); got != "" {
|
||||
t.Errorf("MonthGenitive(0) = %q, want the empty slot", got)
|
||||
}
|
||||
if got := HourSpoken(23); got != "двадцать три" {
|
||||
t.Errorf("HourSpoken(23) = %q, want двадцать три", got)
|
||||
}
|
||||
for _, i := range []int{-1, 7, 13, 24} {
|
||||
if got := Weekday(i); i >= 7 && got != "" {
|
||||
t.Errorf("Weekday(%d) = %q, want empty", i, got)
|
||||
}
|
||||
}
|
||||
if got := HourSpoken(24); got != "" {
|
||||
t.Errorf("HourSpoken(24) = %q, want empty", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestCallerCannotEditTheLexicon — the sets are handed out as copies. A caller
|
||||
// that sorted the slice it was given would otherwise reorder weekdays for
|
||||
// everybody.
|
||||
func TestCallerCannotEditTheLexicon(t *testing.T) {
|
||||
first := Interrogatives()
|
||||
first[0] = "мутировало"
|
||||
if again := Interrogatives(); again[0] == "мутировало" {
|
||||
t.Fatal("the lexicon handed out its own backing array")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,127 @@
|
||||
// Package morph answers questions about Russian grammar from a dictionary.
|
||||
//
|
||||
// The second of the three mechanisms replacing hand-written Russian patterns
|
||||
// (Vikunja #522, owner's call 2026-08-04). internal/lexicon holds the sets that
|
||||
// can be finished; the embedder recognises an open set of phrasings; and this
|
||||
// package answers the questions that are about grammar rather than meaning:
|
||||
//
|
||||
// - is this word a form of a verb, so it carries its own subject?
|
||||
// - are these two tokens the same word in different cases?
|
||||
//
|
||||
// Three places used to answer those from a list of letter endings, and each list
|
||||
// was wrong in a way its own comment admitted. "канал" read as a past-tense verb
|
||||
// because it ends in -ал. A list of nineteen nouns ending in л existed only to
|
||||
// suppress the false positives of "ends in л means masculine past tense", which
|
||||
// is a pattern conceding it is wrong. Grammar is what a dictionary is for.
|
||||
//
|
||||
// Not the resident model. This has to be right every time, offline, in
|
||||
// microseconds, and a 1.7B is neither reliable enough nor fast enough to ask.
|
||||
//
|
||||
// The dictionary is github.com/aaaton/golem's Russian data, vendored. It is
|
||||
// embedded in the module, so a load failure is not a network problem and not a
|
||||
// config problem — it is corrupt data that got past the build. Every function
|
||||
// answers conservatively in that case rather than failing the turn, and says so
|
||||
// in its own doc comment.
|
||||
package morph
|
||||
|
||||
import (
|
||||
"log"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"github.com/aaaton/golem/v4"
|
||||
"github.com/aaaton/golem/v4/dicts/ru"
|
||||
)
|
||||
|
||||
var (
|
||||
once sync.Once
|
||||
lemma *golem.Lemmatizer
|
||||
loadErr error
|
||||
)
|
||||
|
||||
// dict loads the lemmatizer on first use. Loading costs a few megabytes of maps,
|
||||
// which is why it is not done at init: a daemon that never sees Russian never
|
||||
// pays for it.
|
||||
func dict() *golem.Lemmatizer {
|
||||
once.Do(func() {
|
||||
lemma, loadErr = golem.New(ru.New())
|
||||
if loadErr != nil {
|
||||
// Once, not per call: this is a permanent condition and a voice loop
|
||||
// would otherwise fill the log with it at speech rate.
|
||||
log.Printf("morph: russian dictionary unavailable, answering conservatively: %v", loadErr)
|
||||
}
|
||||
})
|
||||
return lemma
|
||||
}
|
||||
|
||||
// Available reports whether the dictionary loaded. Callers do not need it to be
|
||||
// correct — every function below has a defined answer without it — but a test
|
||||
// that means to measure the dictionary should skip rather than pass vacuously.
|
||||
func Available() bool {
|
||||
dict()
|
||||
return loadErr == nil
|
||||
}
|
||||
|
||||
// Lemma returns the dictionary form of a word, or the word itself when the
|
||||
// dictionary does not know it or could not load. An unknown word is its own
|
||||
// lemma: "бэкап" is not in the dictionary and there is nothing better to say
|
||||
// about it than what he said.
|
||||
func Lemma(word string) string {
|
||||
w := strings.ToLower(strings.TrimSpace(word))
|
||||
if w == "" {
|
||||
return ""
|
||||
}
|
||||
l := dict()
|
||||
if l == nil {
|
||||
return w
|
||||
}
|
||||
if got := l.Lemma(w); got != "" {
|
||||
return got
|
||||
}
|
||||
return w
|
||||
}
|
||||
|
||||
// infinitiveEndings — how a Russian infinitive ends. This is not a stem pattern:
|
||||
// it is applied to a LEMMA the dictionary returned, where the infinitive is the
|
||||
// dictionary form of every verb by definition, so the test is about the
|
||||
// dictionary's own output and not about the word he said.
|
||||
//
|
||||
// The reflexive forms are listed because a reflexive lemma keeps its particle:
|
||||
// "тренировался" lemmatises to "тренироваться", which ends in "ся" rather than
|
||||
// "ть".
|
||||
var infinitiveEndings = []string{"ться", "тись", "чься", "ть", "ти", "чь"}
|
||||
|
||||
// IsVerbForm reports whether a word is some form of a verb — past tense, present,
|
||||
// imperative, reflexive, participle. A verb carries its own subject and tense, so
|
||||
// in Russian one verb is a whole sentence, which is what the callers care about.
|
||||
//
|
||||
// Without the dictionary this answers false: not knowing is not evidence that a
|
||||
// word IS a verb, and the callers all treat false as the cautious direction.
|
||||
func IsVerbForm(word string) bool {
|
||||
if dict() == nil {
|
||||
return false
|
||||
}
|
||||
l := Lemma(word)
|
||||
for _, e := range infinitiveEndings {
|
||||
if strings.HasSuffix(l, e) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// SameWord reports whether two tokens are the same word in different cases —
|
||||
// "режим" and "режиме", "тихий" and "тихо". It is the test a stem-plus-endings
|
||||
// comparison was approximating, and it draws the line the ending list could not:
|
||||
// "тихонько" and "потихоньку" are different words, and the dictionary says so
|
||||
// because it has never heard of either.
|
||||
//
|
||||
// Without the dictionary this falls back to exact equality, which is the
|
||||
// narrowest honest answer.
|
||||
func SameWord(a, b string) bool {
|
||||
la, lb := Lemma(a), Lemma(b)
|
||||
if la == "" || lb == "" {
|
||||
return false
|
||||
}
|
||||
return la == lb
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
package morph
|
||||
|
||||
import "testing"
|
||||
|
||||
// TestVerbFormsAreVerbs — the question internal/router/singletoken.go asks. Every
|
||||
// one of these is a whole sentence in Russian, because the verb carries its own
|
||||
// subject, tense and gender.
|
||||
func TestVerbFormsAreVerbs(t *testing.T) {
|
||||
if !Available() {
|
||||
t.Skip("russian dictionary unavailable")
|
||||
}
|
||||
for _, w := range []string{
|
||||
"поужинал", "сходил", "выпил", "напомнил", "поняла", "сделала",
|
||||
"пришёл", "начал", "работаешь", "занимаюсь",
|
||||
// Reflexive: the lemma keeps its particle, so "тренироваться" ends in
|
||||
// "ся" and not "ть". That is why the ending list carries both.
|
||||
"тренировался", "проснулся",
|
||||
} {
|
||||
if !IsVerbForm(w) {
|
||||
t.Errorf("IsVerbForm(%q) = false, want true (lemma %q)", w, Lemma(w))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestNounsEndingInLAreNotVerbs — the list this package deleted. Nineteen nouns
|
||||
// lived in internal/phraser/eval/checks.go as exceptions to "ends in л means
|
||||
// masculine past tense", plus the ones internal/router/singletoken.go named as
|
||||
// its own known errors. A list of exceptions to a pattern is the pattern
|
||||
// conceding it is wrong, so all of them are here and none may be a verb.
|
||||
func TestNounsEndingInLAreNotVerbs(t *testing.T) {
|
||||
if !Available() {
|
||||
t.Skip("russian dictionary unavailable")
|
||||
}
|
||||
for _, w := range []string{
|
||||
"стол", "стул", "пол", "зал", "гол", "узел", "отдел", "файл", "канал",
|
||||
"угол", "футбол", "вокзал", "металл", "интервал", "уровень", "мускул",
|
||||
"апрель", "июль", "рубль",
|
||||
// singletoken.go named these: "канал" read as past tense, and short
|
||||
// nouns needed a length exemption to survive a two-letter suffix test.
|
||||
"нос", "лес", "газ", "вода", "бэкап",
|
||||
} {
|
||||
if IsVerbForm(w) {
|
||||
t.Errorf("IsVerbForm(%q) = true, want false (lemma %q)", w, Lemma(w))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestSameWordDrawsTheLineTheEndingListCouldNot — the question
|
||||
// cmd/mavend/quiet_toggle.go asks. Its comment describes exactly this: "тихий",
|
||||
// "тихом" and "тихо" are one word inflected, while "тихонько" and "потихоньку"
|
||||
// are different words. The dictionary says so; a list of 36 endings approximated
|
||||
// it.
|
||||
func TestSameWordDrawsTheLineTheEndingListCouldNot(t *testing.T) {
|
||||
if !Available() {
|
||||
t.Skip("russian dictionary unavailable")
|
||||
}
|
||||
for _, w := range []string{"тихий", "тихом", "тихо", "тише"} {
|
||||
if !SameWord(w, "тихий") {
|
||||
t.Errorf("SameWord(%q, тихий) = false, want true (lemma %q)", w, Lemma(w))
|
||||
}
|
||||
}
|
||||
for _, w := range []string{"тихонько", "потихоньку"} {
|
||||
if SameWord(w, "тихий") {
|
||||
t.Errorf("SameWord(%q, тихий) = true, want false", w)
|
||||
}
|
||||
}
|
||||
for _, w := range []string{"режим", "режима", "режиме", "режимы"} {
|
||||
if !SameWord(w, "режим") {
|
||||
t.Errorf("SameWord(%q, режим) = false, want true (lemma %q)", w, Lemma(w))
|
||||
}
|
||||
}
|
||||
if SameWord("режим", "тихий") {
|
||||
t.Error("SameWord matched two unrelated words")
|
||||
}
|
||||
}
|
||||
|
||||
// TestUnknownWordIsItsOwnLemma — "бэкап" is not in the dictionary, and there is
|
||||
// nothing better to say about it than what he said. Two spellings of an unknown
|
||||
// word still compare equal, which is what the exact-equality fallback rests on.
|
||||
func TestUnknownWordIsItsOwnLemma(t *testing.T) {
|
||||
if got := Lemma("бэкап"); got != "бэкап" {
|
||||
t.Errorf("Lemma(бэкап) = %q, want бэкап", got)
|
||||
}
|
||||
if got := Lemma(" БЭКАП "); got != "бэкап" {
|
||||
t.Errorf("Lemma trims and lowercases: got %q", got)
|
||||
}
|
||||
if !SameWord("бэкап", "БЭКАП") {
|
||||
t.Error("SameWord must still compare an unknown word with itself")
|
||||
}
|
||||
if got := Lemma(""); got != "" {
|
||||
t.Errorf("Lemma(empty) = %q, want empty", got)
|
||||
}
|
||||
if SameWord("", "") {
|
||||
t.Error("two empty tokens are not a word")
|
||||
}
|
||||
}
|
||||
@@ -6,6 +6,8 @@ import (
|
||||
"strings"
|
||||
"unicode"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/kami/maven/internal/morph"
|
||||
)
|
||||
|
||||
// The check names, in report order. Every check is a string or length test — no
|
||||
@@ -140,24 +142,25 @@ var masculinePredicative = map[string]bool{
|
||||
"обязан": true, "сам": true, "занят": true, "прав": true,
|
||||
}
|
||||
|
||||
// nounsEndingInL — the false positives of "ends in л ⇒ masculine past tense".
|
||||
// Small on purpose: it only has to cover nouns a nudge might actually use.
|
||||
var nounsEndingInL = map[string]bool{
|
||||
"стол": true, "стул": true, "пол": true, "зал": true, "гол": true,
|
||||
"узел": true, "отдел": true, "файл": true, "канал": true, "угол": true,
|
||||
"футбол": true, "вокзал": true, "металл": true, "интервал": true,
|
||||
"уровень": true, "мускул": true, "апрель": true, "июль": true, "рубль": true,
|
||||
}
|
||||
|
||||
// masculinePast reports whether a word looks like a masculine past-tense verb.
|
||||
// Russian past tense is gendered by suffix: -л (m), -ла (f). A 0.8B with weak
|
||||
// Russian defaults to the masculine form, which is the exact drift being
|
||||
// measured.
|
||||
// masculinePast reports whether a word is a masculine past-tense verb. Russian
|
||||
// past tense is gendered by suffix: -л for him, -ла for her. A small model with
|
||||
// weak Russian defaults to the masculine form, which is the exact drift this
|
||||
// check measures.
|
||||
//
|
||||
// The ending is only half the test, and the other half used to be a hand list of
|
||||
// nineteen nouns that end in л — стол, файл, апрель — kept "small on purpose",
|
||||
// which means incomplete on purpose. A list of exceptions to a pattern is the
|
||||
// pattern conceding it is wrong, so the second half is now a dictionary lookup:
|
||||
// ends in -л AND is a form of a verb (Vikunja #526). Every noun the list held is
|
||||
// correctly not a verb, and so are the ones it had not got round to.
|
||||
func masculinePast(w string) bool {
|
||||
if len([]rune(w)) < 3 || nounsEndingInL[w] {
|
||||
if len([]rune(w)) < 3 {
|
||||
return false
|
||||
}
|
||||
return strings.HasSuffix(w, "л") || strings.HasSuffix(w, "лся")
|
||||
if !strings.HasSuffix(w, "л") && !strings.HasSuffix(w, "лся") {
|
||||
return false
|
||||
}
|
||||
return morph.IsVerbForm(w)
|
||||
}
|
||||
|
||||
func checkFeminine(body string) Result {
|
||||
|
||||
@@ -91,6 +91,15 @@ type Config struct {
|
||||
NCtx int
|
||||
Timeout time.Duration
|
||||
|
||||
// StartupTimeout bounds the wait for llama-server to print the address it
|
||||
// listens on. A config field and not a constant because the box may
|
||||
// legitimately need longer: a cold 1.7B loading off a spinning disk can
|
||||
// outrun a minute, and until this existed that returned "server did not
|
||||
// start within 60s" with no way to raise it.
|
||||
//
|
||||
// 0 ⇒ defaultStartupTimeout.
|
||||
StartupTimeout time.Duration
|
||||
|
||||
// CacheRAMMiB bounds llama-server's prompt cache, which is what actually ate
|
||||
// this box. Measured on homesrv 2026-08-03: the server's own default limit is
|
||||
// 8192 MiB, it stores the full KV state of every idle slot it evicts (112 kiB
|
||||
@@ -140,6 +149,8 @@ func DefaultConfig(modelPath string) Config {
|
||||
// 512 MiB caps total RSS near 1 GB and still holds several recent prompts.
|
||||
CacheRAMMiB: 512,
|
||||
Timeout: 30 * time.Second,
|
||||
|
||||
StartupTimeout: defaultStartupTimeout,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -266,7 +277,15 @@ func llamaArgs(cfg Config) []string {
|
||||
return args
|
||||
}
|
||||
|
||||
// defaultStartupTimeout — the wait for llama-server's listen line when Config
|
||||
// does not set one. A cold model load off disk is the slow part.
|
||||
const defaultStartupTimeout = 60 * time.Second
|
||||
|
||||
func startLlamaProc(ctx context.Context, cfg Config) (*llamaProc, error) {
|
||||
startupTimeout := cfg.StartupTimeout
|
||||
if startupTimeout <= 0 {
|
||||
startupTimeout = defaultStartupTimeout
|
||||
}
|
||||
p := &llamaProc{}
|
||||
cmd := exec.CommandContext(ctx, cfg.BinPath, llamaArgs(cfg)...)
|
||||
// Pdeathsig: the kernel SIGKILLs llama-server the moment mavend dies — by
|
||||
@@ -344,8 +363,8 @@ func startLlamaProc(ctx context.Context, cfg Config) (*llamaProc, error) {
|
||||
return fail(fmt.Errorf("llm: server output: %w; last output: %s", err, tail.String()))
|
||||
case <-ctx.Done():
|
||||
return fail(ctx.Err())
|
||||
case <-time.After(60 * time.Second):
|
||||
return fail(fmt.Errorf("llm: server did not start within 60s; last output: %s", tail.String()))
|
||||
case <-time.After(startupTimeout):
|
||||
return fail(fmt.Errorf("llm: server did not start within %s; last output: %s", startupTimeout, tail.String()))
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -18,6 +18,8 @@ import (
|
||||
"fmt"
|
||||
"math/rand"
|
||||
"regexp"
|
||||
|
||||
"github.com/kami/maven/internal/lexicon"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
@@ -210,13 +212,11 @@ func capitalizeFirst(s string) string {
|
||||
}
|
||||
|
||||
// hourWords — hours spelled out. "3 ч" is fine on a screen and wrong in a
|
||||
// Russian voice, so the number goes out as words.
|
||||
var hourWords = []string{
|
||||
"ноль", "один", "два", "три", "четыре", "пять", "шесть", "семь", "восемь",
|
||||
"девять", "десять", "одиннадцать", "двенадцать", "тринадцать",
|
||||
"четырнадцать", "пятнадцать", "шестнадцать", "семнадцать", "восемнадцать",
|
||||
"девятнадцать", "двадцать", "двадцать один", "двадцать два", "двадцать три",
|
||||
}
|
||||
// Russian voice, so the number goes out as words. Closed set, indexed by the
|
||||
// hour, kept in internal/lexicon (Vikunja #525).
|
||||
func hourWord(h int) string { return lexicon.HourSpoken(h) }
|
||||
|
||||
const hoursSpoken = 24
|
||||
|
||||
// hourPlural — час / часа / часов by Russian counting rules.
|
||||
func hourPlural(h int) string {
|
||||
@@ -245,7 +245,7 @@ func ruSinceWords(d time.Duration) string {
|
||||
h++
|
||||
m = 0
|
||||
}
|
||||
if h >= len(hourWords) {
|
||||
if h >= hoursSpoken {
|
||||
return "больше суток"
|
||||
}
|
||||
if h == 1 {
|
||||
@@ -255,7 +255,7 @@ func ruSinceWords(d time.Duration) string {
|
||||
return "час"
|
||||
}
|
||||
if m >= 15 {
|
||||
return hourWords[h] + " с половиной часа"
|
||||
return hourWord(h) + " с половиной часа"
|
||||
}
|
||||
return hourWords[h] + " " + hourPlural(h)
|
||||
return hourWord(h) + " " + hourPlural(h)
|
||||
}
|
||||
|
||||
@@ -181,6 +181,37 @@ exit 1`)
|
||||
t.Fatalf("err = %v, want context.Canceled", err)
|
||||
}
|
||||
})
|
||||
|
||||
// The last arm of the startup race, and the one most likely to leak: a
|
||||
// llama-server still loading a model is alive, so giving up on it without
|
||||
// killing and reaping it orphans a process holding the GPU. Testable at all
|
||||
// because Config.StartupTimeout replaced a hardcoded 60s (Vikunja #323).
|
||||
t.Run("startup timeout", func(t *testing.T) {
|
||||
pidPath := filepath.Join(t.TempDir(), "pid")
|
||||
bin := fakeLlama(t, fmt.Sprintf(`echo $$ > %s
|
||||
while : ; do sleep 1 ; done`, pidPath))
|
||||
cfg := testCfg(bin)
|
||||
cfg.StartupTimeout = 200 * time.Millisecond
|
||||
|
||||
_, err := startLlamaProc(context.Background(), cfg)
|
||||
if err == nil || !strings.Contains(err.Error(), "did not start within 200ms") {
|
||||
t.Fatalf("err = %v, want the startup-timeout arm naming the timeout", err)
|
||||
}
|
||||
|
||||
raw, readErr := os.ReadFile(pidPath)
|
||||
if readErr != nil {
|
||||
t.Fatalf("fake server never recorded its pid: %v", readErr)
|
||||
}
|
||||
pid, convErr := strconv.Atoi(strings.TrimSpace(string(raw)))
|
||||
if convErr != nil {
|
||||
t.Fatalf("pid file = %q: %v", raw, convErr)
|
||||
}
|
||||
// Killed, and reaped: a zombie still answers signal 0, so this asserts
|
||||
// the Wait ran too.
|
||||
if err := syscall.Kill(pid, 0); err == nil {
|
||||
t.Errorf("llama-server %d survived the startup timeout", pid)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestNewLLMPhraserSpawns(t *testing.T) {
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
package router
|
||||
|
||||
import (
|
||||
"strings"
|
||||
|
||||
"github.com/kami/maven/internal/lexicon"
|
||||
"github.com/kami/maven/internal/morph"
|
||||
)
|
||||
|
||||
// transientWords — the states a thing is in for an afternoon, as dictionary
|
||||
// forms. They used to be prefixes, which is how a prefix list always goes wrong:
|
||||
// "лаг" matched "лагерь" and "падает" matched nothing else it inflected into.
|
||||
// morph.SameWord compares the words themselves (Vikunja #528).
|
||||
//
|
||||
// Russian aspect pairs are two separate verbs, so a slot lists both where both
|
||||
// are said. English members have no dictionary here and fall back to exact
|
||||
// comparison, which is what they had.
|
||||
var transientWords = []string{
|
||||
"медленный", "медленно", "тормозить", "тормоз", "лагать", "лаг",
|
||||
"зависать", "зависнуть", "виснуть", "глючить", "барахлить",
|
||||
"отваливаться", "отвалиться", "падать", "упасть", "сдохнуть",
|
||||
"греться", "перегреваться", "перегреться",
|
||||
"slow", "laggy", "stuck", "frozen", "flaky", "broken", "down",
|
||||
}
|
||||
|
||||
// brokenWords — what "не ..." is denying when the sentence is a complaint.
|
||||
// Dictionary forms, same reason.
|
||||
var brokenWords = []string{
|
||||
"работать", "пахать", "грузить", "грузиться", "открываться",
|
||||
"включаться", "коннектиться", "подключаться",
|
||||
"work", "load", "connect", "respond",
|
||||
}
|
||||
|
||||
// selfMarkers — the words that make a sentence about him rather than about a
|
||||
// thing. Their presence turns the test off, because losing a fact he meant to
|
||||
// store is worse than keeping a complaint: "я сломал руку" is durable, and
|
||||
// "интернет не работает" is not.
|
||||
//
|
||||
// A closed class, so it comes from the lexicon: the first-person pronoun has a
|
||||
// fixed number of forms and this file was the third place they were typed out
|
||||
// (Vikunja #528).
|
||||
var selfMarkers = lexicon.FirstPerson()
|
||||
|
||||
// IsTransientComplaint reports whether text observes a passing state of some
|
||||
// thing rather than recording a fact.
|
||||
//
|
||||
// It exists because "сеть какая-то медленная" and "интернет не работает" were
|
||||
// written to the fact store as `self` rows at confidence 1.00 (Vikunja #481),
|
||||
// where recall reads them back later as if they were still true. A complaint
|
||||
// describes a moment; the fact store describes him.
|
||||
//
|
||||
// Deterministic, offline, and shaped exactly like IsQuestionShaped: an
|
||||
// explicit capture verb wins over everything, because "запомни что интернет
|
||||
// не работает" is an instruction and not a passing remark. A first-person
|
||||
// marker also turns it off — the test is meant to catch a sentence about a
|
||||
// thing, and it errs toward storing.
|
||||
func IsTransientComplaint(text string) bool {
|
||||
t := strings.TrimSpace(text)
|
||||
if t == "" {
|
||||
return false
|
||||
}
|
||||
toks := planTokens(strings.ToLower(t))
|
||||
for _, v := range captureVerbs {
|
||||
if hasTok(toks, v) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
for _, m := range selfMarkers {
|
||||
if hasTok(toks, m) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
for _, tok := range toks {
|
||||
for _, w := range transientWords {
|
||||
if morph.SameWord(tok, w) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
// "не" plus a verb of working, in either order of the two tokens that
|
||||
// follow it — "не работает" and "не очень работает" both deny the same
|
||||
// thing.
|
||||
for i, tok := range toks {
|
||||
if tok != "не" && tok != "not" && tok != "isn" {
|
||||
continue
|
||||
}
|
||||
for j := i + 1; j < len(toks) && j <= i+2; j++ {
|
||||
for _, v := range brokenWords {
|
||||
if morph.SameWord(toks[j], v) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,66 @@
|
||||
package router
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestIsTransientComplaint(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
text string
|
||||
want bool
|
||||
}{
|
||||
// The two rows from the QA run that named this bug.
|
||||
{"сеть какая-то медленная", true},
|
||||
{"интернет не работает", true},
|
||||
{"вайфай тормозит", true},
|
||||
{"сервер завис", true},
|
||||
{"the wifi is slow", true},
|
||||
|
||||
// An instruction wins: he asked for it to be written down.
|
||||
{"запомни что интернет не работает", false},
|
||||
{"запиши что сеть медленная", false},
|
||||
|
||||
// About him, so it stays a fact even when it sounds like a complaint.
|
||||
{"я сломал руку", false},
|
||||
{"мне медленно думается", false},
|
||||
|
||||
// Ordinary captures must not be touched.
|
||||
{"поужинал", false},
|
||||
{"выпил воды", false},
|
||||
{"машина на парковке", false},
|
||||
{"", false},
|
||||
} {
|
||||
if got := IsTransientComplaint(tc.text); got != tc.want {
|
||||
t.Errorf("IsTransientComplaint(%q) = %v, want %v", tc.text, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestComplaintPrefixCollisions — what the prefix list got wrong and the
|
||||
// dictionary does not (Vikunja #528). Each of these contains a word that starts
|
||||
// with one of the old stems and is a different word.
|
||||
func TestComplaintPrefixCollisions(t *testing.T) {
|
||||
for _, s := range []string{
|
||||
// "лаг" matched "лагерь".
|
||||
"детский лагерь под москвой",
|
||||
// "падает" was literal, but "падеж" and "падение" start on "пад".
|
||||
"падение цен на квартиры",
|
||||
// "отвал" matched "отвальная".
|
||||
"отвальная в пятницу",
|
||||
} {
|
||||
if IsTransientComplaint(s) {
|
||||
t.Errorf("IsTransientComplaint(%q) = true, want false", s)
|
||||
}
|
||||
}
|
||||
// And the real complaints still read as complaints, in the inflections the
|
||||
// prefixes were there to cover.
|
||||
for _, s := range []string{
|
||||
"сеть какая-то медленная",
|
||||
"интернет не работает",
|
||||
"nextcloud тормозит",
|
||||
"диск сдохнет скоро",
|
||||
"сервис не открывается",
|
||||
} {
|
||||
if !IsTransientComplaint(s) {
|
||||
t.Errorf("IsTransientComplaint(%q) = false, want true", s)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -81,6 +81,9 @@ func NewPythonDateParser() *PythonDateParser {
|
||||
// or dateparser is unavailable, falls back to the stub parser. Returns
|
||||
// (time, true, nil) on success; (zero, false, nil) when no date is found.
|
||||
func (p *PythonDateParser) Parse(ctx context.Context, text string, now time.Time) (time.Time, bool, error) {
|
||||
// Speech says the hour in words, and neither this parser nor the stub
|
||||
// reads "в семь вечера" (Vikunja #469). Both see the digits instead.
|
||||
text = SpellOutDigits(text)
|
||||
t, ok, err := p.parseWithPython(ctx, text, now)
|
||||
if err != nil {
|
||||
// python3 missing, dateparser not installed, or process failure —
|
||||
|
||||
@@ -236,8 +236,15 @@ func newBaselineRouter(t *testing.T, emb router.Embedder, llmR *router.LLMRouter
|
||||
// Same order as buildRouter (voicewire.go). The fixture is only worth
|
||||
// anything while its grammar set is the daemon's grammar set.
|
||||
grammars = append(grammars, router.AgendaQueryGrammars()...)
|
||||
grammars = append(grammars, router.NarrativeQueryGrammars()...)
|
||||
grammars = append(grammars, router.FeedQueryGrammar())
|
||||
// The list side of the same exposure: a phrasing with no possessive in it
|
||||
// ("список дел") routed system and never reached queryTasks (Vikunja #467).
|
||||
grammars = append(grammars, router.TaskListGrammar())
|
||||
grammars = append(grammars, router.ReminderGrammar())
|
||||
grammars = append(grammars, router.TaskCaptureGrammar())
|
||||
// "расскажи про X" is a world question the model called a fact, and the
|
||||
// rule goes last because it matches on the first word alone (Vikunja #498).
|
||||
grammars = append(grammars, router.NarrativeQueryGrammars()...)
|
||||
return router.New(router.Config{
|
||||
Grammars: grammars,
|
||||
Classifier: cls,
|
||||
|
||||
@@ -25,6 +25,8 @@
|
||||
{ "id": "ru-query-019", "utterance": "что у меня стоит в календаре на послезавтра", "lang": "ru", "intent": "query", "tags": ["calendar", "hard"], "note": "agenda, not the clock: the daemon answers this from CalendarEvents inside the query branch, so the clock/date system rule must not swallow it" },
|
||||
{ "id": "ru-query-022", "utterance": "какие планы на завтра?", "lang": "ru", "intent": "query", "tags": ["calendar"], "note": "the same agenda question as ru-query-019 aimed at another day; it answered \u043f\u043e\u043a\u0430 \u043d\u0435 \u0443\u043c\u0435\u044e on the deployed daemon while the today form worked (Vikunja #471)" },
|
||||
{ "id": "ru-query-023", "utterance": "\u043a\u043e\u0433\u0434\u0430 \u043f\u043b\u0430\u043d\u0451\u0440\u043a\u0430?", "lang": "ru", "intent": "query", "tags": ["calendar", "hard"], "note": "a named event with no calendar word — the noun is the only signal that this is a question about his day" },
|
||||
{ "id": "ru-query-024", "utterance": "что дальше?", "lang": "ru", "intent": "query", "tags": ["calendar", "no-question-word"], "note": "the rest of the day, with no possessive and no plan word to anchor on; the model called it a fact and the write had to be caught downstream (Vikunja #498)" },
|
||||
{ "id": "ru-query-025", "utterance": "расскажи про битву при Ватерлоо", "lang": "ru", "intent": "query", "tags": ["world", "no-question-word"], "note": "a narrative request carries no question mark and no interrogative, so it routed fact; contrast ru-chat-003, where the same verb asks for a joke" },
|
||||
{ "id": "ru-query-014", "utterance": "я успеваю до дедлайна", "lang": "ru", "intent": "query", "tags": ["hard", "no-question-word"] },
|
||||
{ "id": "ru-query-015", "utterance": "сколько я прошёл шагов", "lang": "ru", "intent": "query", "tags": ["aggregate"] },
|
||||
{ "id": "ru-query-016", "utterance": "покажи давление за неделю", "lang": "ru", "intent": "query", "tags": ["hard", "imperative"], "note": "imperative form but a read — must not route to act" },
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
package router
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func feedRouter(t *testing.T) *Router {
|
||||
t.Helper()
|
||||
r := newTestRouter(t, 0.0)
|
||||
r.grammars = append(r.grammars, SystemTimeDateGrammars()...)
|
||||
r.grammars = append(r.grammars, AgendaQueryGrammars()...)
|
||||
r.grammars = append(r.grammars, FeedQueryGrammar())
|
||||
return r
|
||||
}
|
||||
|
||||
// The documented utterance of task 258 step 1 routed system and answered
|
||||
// "пока не умею отвечать на этот вопрос.", while the same question worded with
|
||||
// "новостях" worked (Vikunja #474).
|
||||
func TestFeedQuestionsRouteToQuery(t *testing.T) {
|
||||
r := feedRouter(t)
|
||||
for _, u := range []string{
|
||||
"что нового в лентах?",
|
||||
"что в лентах?",
|
||||
"расскажи что в новостных лентах",
|
||||
"покажи ленту",
|
||||
} {
|
||||
d, err := r.Route(context.Background(), u, refNow())
|
||||
if err != nil {
|
||||
t.Fatalf("route(%q): %v", u, err)
|
||||
}
|
||||
if d.Intent != IntentQuery {
|
||||
t.Errorf("route(%q) = %s, want query", u, d.Intent)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The greeting and the statement keep their intents. "что нового?" is the most
|
||||
// common opener in the language, and a rule that claimed it would answer hello
|
||||
// with a configuration status.
|
||||
func TestFeedGrammarLeavesTheGreetingAlone(t *testing.T) {
|
||||
r := feedRouter(t)
|
||||
for _, u := range []string{
|
||||
"что нового?",
|
||||
"у меня новая лента в инстаграме",
|
||||
} {
|
||||
d, err := r.Route(context.Background(), u, refNow())
|
||||
if err != nil {
|
||||
t.Fatalf("route(%q): %v", u, err)
|
||||
}
|
||||
if d.Stage == 0 {
|
||||
t.Errorf("route(%q) was claimed at stage 0 as %s", u, d.Intent)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,85 @@
|
||||
package router
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// narrativeRouter wires the grammars in the order the daemon wires them
|
||||
// (voicewire.go), with the narrative rule last — so a test that passes here is
|
||||
// a test of the deployed precedence, not of the rule in isolation.
|
||||
func narrativeRouter(t *testing.T) *Router {
|
||||
t.Helper()
|
||||
r := newTestRouter(t, 0.0)
|
||||
r.grammars = append(r.grammars, SystemTimeDateGrammars()...)
|
||||
r.grammars = append(r.grammars, AgendaQueryGrammars()...)
|
||||
r.grammars = append(r.grammars, TaskListGrammar())
|
||||
r.grammars = append(r.grammars, TaskCaptureGrammar())
|
||||
r.grammars = append(r.grammars, NarrativeQueryGrammars()[1])
|
||||
return r
|
||||
}
|
||||
|
||||
// "расскажи про X" and "что дальше?" carried no question mark and no
|
||||
// interrogative, so nothing at stage 0 claimed them and the model called both
|
||||
// facts (Vikunja #498, point 1 of #470). The fact write is contained now, but
|
||||
// the round trip and the wrong fixture score are not.
|
||||
func TestNarrativeAndRestOfDayRouteToQueryAtStageZero(t *testing.T) {
|
||||
r := narrativeRouter(t)
|
||||
for _, u := range []string{
|
||||
"расскажи про битву при Ватерлоо",
|
||||
"расскажи мне про Юникод",
|
||||
"объясни как работает tcp",
|
||||
"опиши Самару",
|
||||
"перечисли планеты",
|
||||
"tell me about the fall of Rome",
|
||||
"что дальше?",
|
||||
"и что там дальше",
|
||||
"что дальше",
|
||||
"what's next?",
|
||||
} {
|
||||
d, err := r.Route(context.Background(), u, refNow())
|
||||
if err != nil {
|
||||
t.Fatalf("route(%q): %v", u, err)
|
||||
}
|
||||
if d.Intent != IntentQuery {
|
||||
t.Errorf("route(%q) = %s, want query", u, d.Intent)
|
||||
}
|
||||
if d.Stage != 0 {
|
||||
t.Errorf("route(%q) decided at stage %d, want 0 — the point is to skip the model", u, d.Stage)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The narrative rule must not take a turn that belongs to something else. A
|
||||
// capture marker wins because it is what he said, and asking her for a joke is
|
||||
// chat: the query chain has no source that answers it.
|
||||
func TestNarrativeGrammarLeavesOtherTurnsAlone(t *testing.T) {
|
||||
r := narrativeRouter(t)
|
||||
for _, u := range []string{
|
||||
"расскажи анекдот",
|
||||
"расскажи о себе",
|
||||
"расскажи шутку",
|
||||
"расскажи",
|
||||
} {
|
||||
d, err := r.Route(context.Background(), u, refNow())
|
||||
if err != nil {
|
||||
t.Fatalf("route(%q): %v", u, err)
|
||||
}
|
||||
if d.Stage == 0 && d.Intent == IntentQuery {
|
||||
t.Errorf("route(%q) was claimed as a world question at stage 0", u)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The topic reaches the query chain without the verb that introduced it: the
|
||||
// search leg wants "битву при Ватерлоо", not "расскажи про битву при Ватерлоо".
|
||||
func TestNarrativeGrammarKeepsTheTopic(t *testing.T) {
|
||||
r := narrativeRouter(t)
|
||||
d, err := r.Route(context.Background(), "расскажи про битву при Ватерлоо", refNow())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got, want := d.Slots.Text, "битву при Ватерлоо"; got != want {
|
||||
t.Errorf("text = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,97 @@
|
||||
package router
|
||||
|
||||
import "strings"
|
||||
|
||||
// ruNumerals — spoken numbers as digits, for the clock hours and the minutes
|
||||
// that follow them. Every case ending he might say is listed rather than
|
||||
// stemmed: "в семь", "к семи", "около семи" are three forms of one hour, and a
|
||||
// prefix rule short enough to cover them also matches "семья".
|
||||
//
|
||||
// Stops at thirty, which is as far as a spoken time goes ("без двадцати
|
||||
// восемь", "в половине шестого"). Anything larger is said in digits.
|
||||
var ruNumerals = map[string]string{
|
||||
"один": "1", "одного": "1", "одну": "1", "час": "1", "часу": "1",
|
||||
"два": "2", "две": "2", "двух": "2",
|
||||
"три": "3", "трёх": "3", "трех": "3",
|
||||
"четыре": "4", "четырёх": "4", "четырех": "4",
|
||||
"пять": "5", "пяти": "5",
|
||||
"шесть": "6", "шести": "6",
|
||||
"семь": "7", "семи": "7",
|
||||
"восемь": "8", "восьми": "8",
|
||||
"девять": "9", "девяти": "9",
|
||||
"десять": "10", "десяти": "10",
|
||||
"одиннадцать": "11", "одиннадцати": "11",
|
||||
"двенадцать": "12", "двенадцати": "12",
|
||||
"тринадцать": "13", "тринадцати": "13",
|
||||
"четырнадцать": "14", "четырнадцати": "14",
|
||||
"пятнадцать": "15", "пятнадцати": "15",
|
||||
"шестнадцать": "16", "шестнадцати": "16",
|
||||
"семнадцать": "17", "семнадцати": "17",
|
||||
"восемнадцать": "18", "восемнадцати": "18",
|
||||
"девятнадцать": "19", "девятнадцати": "19",
|
||||
"двадцать": "20", "двадцати": "20",
|
||||
"тридцать": "30", "тридцати": "30",
|
||||
"сорок": "40", "сорока": "40",
|
||||
"пятьдесят": "50", "пятидесяти": "50",
|
||||
}
|
||||
|
||||
// numeralContext — the words that make a numeral a time. A numeral is only
|
||||
// rewritten when one of these sits next to it, so "три яблока" in a note is
|
||||
// left alone and "в три часа" is not.
|
||||
var numeralContext = map[string]bool{
|
||||
"в": true, "во": true, "к": true, "около": true, "на": true,
|
||||
"часа": true, "часов": true, "час": true, "часу": true,
|
||||
"утра": true, "вечера": true, "дня": true, "ночи": true,
|
||||
"минут": true, "минуты": true, "минуту": true,
|
||||
"at": true, "by": true,
|
||||
}
|
||||
|
||||
// SpellOutDigits rewrites spoken numbers as digits so the date parsers see the
|
||||
// shape they know.
|
||||
//
|
||||
// "напомни мне позвонить маме в семь вечера" parsed to nothing, while "в 19:00"
|
||||
// parsed fine (Vikunja #469). Speech is where reminders come from, and speech
|
||||
// says the hour in words, so this is not a long-tail case — it is the ordinary
|
||||
// one. dateparser reads "в 7 вечера" through the qualifier rewrite the python
|
||||
// script already does; it does not read "в семь вечера".
|
||||
//
|
||||
// Conservative by construction: a numeral is only rewritten when a time word
|
||||
// stands beside it. "три часа" becomes "3 часа"; "три яблока" stays as it is,
|
||||
// and a note or a fact carrying a spoken number is untouched.
|
||||
func SpellOutDigits(text string) string {
|
||||
toks := strings.Fields(text)
|
||||
if len(toks) == 0 {
|
||||
return text
|
||||
}
|
||||
out := make([]string, len(toks))
|
||||
copy(out, toks)
|
||||
for i, tok := range toks {
|
||||
key := strings.ToLower(strings.Trim(tok, ".,!?;:«»\"'"))
|
||||
digit, ok := ruNumerals[key]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
// "час" and "часу" are the hour noun as often as they are the number
|
||||
// one, and rewriting "в час дня" to "в 1 дня" is right either way. What
|
||||
// must not happen is rewriting the noun that gives another numeral its
|
||||
// context: "в семь часов" must keep "часов".
|
||||
if !hasTimeNeighbour(toks, i) {
|
||||
continue
|
||||
}
|
||||
out[i] = digit
|
||||
}
|
||||
return strings.Join(out, " ")
|
||||
}
|
||||
|
||||
// hasTimeNeighbour reports whether the token before or after i is a time word.
|
||||
func hasTimeNeighbour(toks []string, i int) bool {
|
||||
for _, j := range []int{i - 1, i + 1} {
|
||||
if j < 0 || j >= len(toks) {
|
||||
continue
|
||||
}
|
||||
if numeralContext[strings.ToLower(strings.Trim(toks[j], ".,!?;:«»\"'"))] {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
package router
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestSpellOutDigits(t *testing.T) {
|
||||
for _, tc := range []struct{ in, want string }{
|
||||
{"напомни мне позвонить маме в семь вечера", "напомни мне позвонить маме в 7 вечера"},
|
||||
{"в три часа дня", "в 3 часа дня"},
|
||||
{"напомни в половине шестого", "напомни в половине шестого"},
|
||||
{"через двадцать минут", "через 20 минут"},
|
||||
// Untouched: no time word stands beside the number.
|
||||
{"купить три яблока", "купить три яблока"},
|
||||
{"семь раз отмерь", "семь раз отмерь"},
|
||||
{"напомни в 19:00", "напомни в 19:00"},
|
||||
{"", ""},
|
||||
} {
|
||||
if got := SpellOutDigits(tc.in); got != tc.want {
|
||||
t.Errorf("SpellOutDigits(%q) = %q, want %q", tc.in, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The utterance from the QA sitting that named this bug: the numeric form
|
||||
// parsed and the spoken form did not.
|
||||
func TestStubParsesASpokenHour(t *testing.T) {
|
||||
now := time.Date(2026, 8, 2, 9, 0, 0, 0, time.Local)
|
||||
got, ok, err := StubDateTimeParser{}.Parse(context.Background(), "напомни мне позвонить маме в семь вечера", now)
|
||||
if err != nil || !ok {
|
||||
t.Fatalf("Parse ok=%v err=%v, want a time", ok, err)
|
||||
}
|
||||
if got.Hour() != 19 {
|
||||
t.Fatalf("hour = %d, want 19", got.Hour())
|
||||
}
|
||||
}
|
||||
+23
-25
@@ -1,32 +1,30 @@
|
||||
package router
|
||||
|
||||
import "strings"
|
||||
import (
|
||||
"strings"
|
||||
|
||||
// interrogatives — the question words that mark an utterance as asking rather
|
||||
// than telling. Tokenized, never substring: "что" inside "чтобы" and "как"
|
||||
// inside "какао" are not questions.
|
||||
var interrogatives = []string{
|
||||
"что", "чего", "какой", "какая", "какое", "какие", "каких",
|
||||
"кто", "кого", "кому", "чей", "почему", "зачем", "отчего",
|
||||
"где", "куда", "откуда", "когда", "сколько", "как",
|
||||
"what", "who", "whom", "why", "when", "where", "which", "how",
|
||||
}
|
||||
"github.com/kami/maven/internal/lexicon"
|
||||
)
|
||||
|
||||
// narrativeRequests — "tell me about X" asks for knowledge Maven does not
|
||||
// hold about him. It carries no question mark and no interrogative, which is
|
||||
// how "расскажи про битву при Ватерлоо" reached the fact store (#470).
|
||||
var narrativeRequests = []string{
|
||||
"расскажи", "объясни", "опиши", "перечисли",
|
||||
"tell", "explain", "describe",
|
||||
}
|
||||
|
||||
// captureVerbs — an explicit instruction to record something. These win over
|
||||
// every test below, because "запиши что я пил воду" contains an interrogative
|
||||
// and is still a capture: the word he said is "запиши".
|
||||
var captureVerbs = []string{
|
||||
"запиши", "запомни", "отметь", "заметь", "добавь", "сохрани",
|
||||
"note", "remember", "log", "save",
|
||||
}
|
||||
// The three word sets this file tests against are closed classes, so they live
|
||||
// complete in internal/lexicon rather than inline here (Vikunja #525). The
|
||||
// inline lists were short: no "чем", no "чём", no "кем", no declined "какой",
|
||||
// so "чем ты занята" carried no interrogative at all and read as a statement.
|
||||
//
|
||||
// interrogatives mark an utterance as asking rather than telling.
|
||||
// narrativeRequests are "tell me about X", which asks for knowledge Maven does
|
||||
// not hold about him and carries neither a question mark nor an interrogative —
|
||||
// that is how "расскажи про битву при Ватерлоо" reached the fact store (#470).
|
||||
// captureVerbs win over both, because "запиши что я пил воду" contains an
|
||||
// interrogative and is still a capture: the word he said is "запиши".
|
||||
//
|
||||
// All three are matched over tokens, never as substrings: "что" inside "чтобы"
|
||||
// and "как" inside "какао" are not questions.
|
||||
var (
|
||||
interrogatives = lexicon.Interrogatives()
|
||||
narrativeRequests = lexicon.NarrativeRequests()
|
||||
captureVerbs = lexicon.CaptureVerbs()
|
||||
)
|
||||
|
||||
// IsQuestionShaped reports whether text asks for something rather than
|
||||
// records it. It is a deterministic offline test over tokens, so it costs
|
||||
|
||||
@@ -2,6 +2,7 @@ package router
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"log"
|
||||
"time"
|
||||
)
|
||||
@@ -201,5 +202,10 @@ func (r *Router) gateLLMDecision(d *Decision) {
|
||||
// retrain). Same shape as nudges.outcome tuning cooldowns: more reliable over
|
||||
// time, introspectable, no model surgery.
|
||||
func (r *Router) CorrectMisroute(ctx context.Context, utterance string, corrected Intent) error {
|
||||
if r == nil || r.classifier == nil {
|
||||
// The LLM router can run with no classifier wired. The correction has
|
||||
// nowhere to land then, and the caller redoes the request anyway.
|
||||
return errors.New("router: no classifier to correct")
|
||||
}
|
||||
return r.classifier.AddExample(ctx, corrected, utterance)
|
||||
}
|
||||
|
||||
@@ -1,6 +1,10 @@
|
||||
package router
|
||||
|
||||
import "strings"
|
||||
import (
|
||||
"strings"
|
||||
|
||||
"github.com/kami/maven/internal/morph"
|
||||
)
|
||||
|
||||
// thinSingleToken — is a one-word utterance thin evidence, or is it a whole
|
||||
// sentence?
|
||||
@@ -17,13 +21,15 @@ import "strings"
|
||||
//
|
||||
// - a closed lexicon of social and command singles, which are complete by
|
||||
// definition ("привет", "спасибо", "стоп", "yes");
|
||||
// - a suffix test for an inflected predicate — past tense, 2nd person,
|
||||
// reflexive. Verbs carry their own subject, so a verb IS a sentence.
|
||||
// - a dictionary lookup for a verb form. A verb carries its own subject,
|
||||
// tense and gender, so a verb IS a sentence.
|
||||
//
|
||||
// The suffix test is deliberately loose about nouns that happen to end the
|
||||
// same way ("канал" reads as past tense here). That direction of error only
|
||||
// costs a clarify we would not have asked for; the other direction — treating
|
||||
// a real report as thin — is the bug being fixed.
|
||||
// The dictionary lookup replaced a list of 24 letter endings (Vikunja #526). The
|
||||
// list was loose in a direction its own comment named: "канал" ends in -ал and
|
||||
// read as past tense, and short words needed a length exemption so "нос" and
|
||||
// "лес" would survive a two-letter suffix. Asking a morphological dictionary
|
||||
// costs one map lookup and has no such errors — grammar is what a dictionary is
|
||||
// for.
|
||||
func thinSingleToken(utterance string) bool {
|
||||
f := strings.Fields(utterance)
|
||||
if len(f) != 1 {
|
||||
@@ -36,7 +42,7 @@ func thinSingleToken(utterance string) bool {
|
||||
if completeSingles[w] {
|
||||
return false
|
||||
}
|
||||
return !looksInflected(w)
|
||||
return !morph.IsVerbForm(w)
|
||||
}
|
||||
|
||||
// completeSingles — one-word utterances that need no second half. Greetings,
|
||||
@@ -58,33 +64,3 @@ var completeSingles = map[string]bool{
|
||||
"sure": true, "right": true, "stop": true, "cancel": true, "help": true,
|
||||
"repeat": true, "continue": true,
|
||||
}
|
||||
|
||||
// inflectedSuffixes — endings that mark a finite or past-tense Russian verb.
|
||||
// Ordered longest-first is unnecessary (any match wins), but each entry is
|
||||
// chosen to be long enough that common nouns rarely collide.
|
||||
var inflectedSuffixes = []string{
|
||||
// reflexive — strongly verbal whatever precedes it
|
||||
"ся", "сь",
|
||||
// past tense
|
||||
"ал", "ял", "ил", "ел", "ыл", "ул", "ёл", "ала", "яла", "ила", "ела",
|
||||
"ыла", "ула", "али", "яли", "или", "ели",
|
||||
// 2nd person singular
|
||||
"ешь", "ишь", "ёшь",
|
||||
// 1st/2nd person plural, 3rd person plural
|
||||
"аем", "яем", "уем", "аете", "ите", "ают", "яют", "уют", "ат", "ят",
|
||||
}
|
||||
|
||||
// looksInflected — does the word carry a verb ending? Short words are exempt:
|
||||
// a three-letter token is not enough stem to trust a two-letter suffix on
|
||||
// ("газ" would otherwise never match, but "нос" and "лес" would).
|
||||
func looksInflected(w string) bool {
|
||||
if len([]rune(w)) < 5 {
|
||||
return false
|
||||
}
|
||||
for _, s := range inflectedSuffixes {
|
||||
if strings.HasSuffix(w, s) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
+57
-33
@@ -5,6 +5,8 @@ import (
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/kami/maven/internal/lexicon"
|
||||
)
|
||||
|
||||
// DateTimeParser — resolves relative→absolute AT CAPTURE ("in 4h" → now+4h),
|
||||
@@ -176,7 +178,7 @@ func afterWord(s, w string) string {
|
||||
type StubDateTimeParser struct{}
|
||||
|
||||
func (StubDateTimeParser) Parse(_ context.Context, text string, now time.Time) (time.Time, bool, error) {
|
||||
s := strings.ToLower(strings.TrimSpace(text))
|
||||
s := strings.ToLower(strings.TrimSpace(SpellOutDigits(text)))
|
||||
toks := strings.Fields(s)
|
||||
// scan for "in <num> <unit>" anywhere — dateparser extracts the datetime
|
||||
// expression from surrounding text; the stub does the same naively.
|
||||
@@ -204,14 +206,22 @@ func (StubDateTimeParser) Parse(_ context.Context, text string, now time.Time) (
|
||||
|
||||
// --- Russian time expressions (stub floor; dateparser replaces) ---
|
||||
|
||||
// "в <clock>" anywhere — mirror of the English "at" scan.
|
||||
// "в <clock>" anywhere — mirror of the English "at" scan. A qualifier
|
||||
// after the hour moves it into the afternoon: "в 7 вечера" is 19:00, and
|
||||
// with SpellOutDigits in front of this that is what "в семь вечера" reads
|
||||
// as too (Vikunja #469).
|
||||
for i := 0; i+1 < len(toks); i++ {
|
||||
if toks[i] != "в" {
|
||||
continue
|
||||
}
|
||||
if t, ok := parseClock(toks[i+1], now); ok {
|
||||
return t, true, nil
|
||||
t, ok := parseClock(toks[i+1], now)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if i+2 < len(toks) {
|
||||
t = applyRuQualifier(t, toks[i+2], now)
|
||||
}
|
||||
return t, true, nil
|
||||
}
|
||||
|
||||
// "через <N> <unit>" / "через <unit>" (bare = 1) / "через полчаса".
|
||||
@@ -348,26 +358,17 @@ func leadingDigits(s string) (int, string, bool) {
|
||||
return n, s[i:], true
|
||||
}
|
||||
|
||||
// wordNumbers — small set, enough for natural test seeds ("four hours",
|
||||
// "thirty minutes"). Production dateparser handles the full ru/en range.
|
||||
var wordNumbers = map[string]int{
|
||||
"one": 1, "two": 2, "three": 3, "four": 4, "five": 5,
|
||||
"six": 6, "seven": 7, "eight": 8, "nine": 9, "ten": 10,
|
||||
"eleven": 11, "twelve": 12, "fifteen": 15, "twenty": 20,
|
||||
"thirty": 30, "forty": 40, "fifty": 50, "sixty": 60,
|
||||
// Russian word numbers (gender variants cover natural речи)
|
||||
"один": 1, "одна": 1, "одно": 1,
|
||||
"два": 2, "две": 2, "три": 3, "четыре": 4,
|
||||
"пять": 5, "шесть": 6, "семь": 7, "восемь": 8,
|
||||
"девять": 9, "десять": 10,
|
||||
}
|
||||
// leadingWordNumber reads a spoken number off the front of a phrase — "два
|
||||
// часа", "twenty minutes". The number words are a closed class and live in
|
||||
// internal/lexicon, complete: the inline table here stopped at "десять" in
|
||||
// Russian, so "пятнадцать минут" was not a duration (Vikunja #525).
|
||||
|
||||
func leadingWordNumber(s string) (int, string, bool) {
|
||||
toks := strings.Fields(s)
|
||||
if len(toks) == 0 {
|
||||
return 0, "", false
|
||||
}
|
||||
n, ok := wordNumbers[toks[0]]
|
||||
n, ok := lexicon.Cardinal(toks[0])
|
||||
if !ok {
|
||||
return 0, "", false
|
||||
}
|
||||
@@ -450,24 +451,20 @@ func (AnaphoraResolver) Resolve(text string) (ref string, ok bool) {
|
||||
}
|
||||
|
||||
// ParseCalendarDate detects RU/EN calendar day words in text and returns
|
||||
// midnight of that day in now's own time zone. Handles "сегодня", "завтра",
|
||||
// "послезавтра", "вчера" (and the English words). Returns zero time + false
|
||||
// if no match.
|
||||
// midnight of that day in now's own time zone. Returns zero time + false if no
|
||||
// match.
|
||||
//
|
||||
// "послезавтра" is checked before "завтра" because it contains it.
|
||||
// The day words are a closed class and live in internal/lexicon, so this is a
|
||||
// lookup rather than an ordered switch (Vikunja #525). The switch it replaced
|
||||
// had to test "послезавтра" before "завтра" by hand, because one contains the
|
||||
// other — and it matched on substrings, so "завтраком" was tomorrow. The lexicon
|
||||
// matches on word boundaries and gained "позавчера", which was never here.
|
||||
func ParseCalendarDate(text string, now time.Time) (time.Time, bool) {
|
||||
lower := strings.ToLower(text)
|
||||
switch {
|
||||
case strings.Contains(lower, "сегодня") || strings.Contains(lower, "today"):
|
||||
return midnight(now, 0), true
|
||||
case strings.Contains(lower, "послезавтра") || strings.Contains(lower, "day after tomorrow"):
|
||||
return midnight(now, 2), true
|
||||
case strings.Contains(lower, "завтра") || strings.Contains(lower, "tomorrow"):
|
||||
return midnight(now, 1), true
|
||||
case strings.Contains(lower, "вчера") || strings.Contains(lower, "yesterday"):
|
||||
return midnight(now, -1), true
|
||||
days, ok := lexicon.DayOffsetIn(text)
|
||||
if !ok {
|
||||
return time.Time{}, false
|
||||
}
|
||||
return time.Time{}, false
|
||||
return midnight(now, days), true
|
||||
}
|
||||
|
||||
// midnight returns the start of the day that is `days` away from now, in
|
||||
@@ -476,3 +473,30 @@ func midnight(now time.Time, days int) time.Time {
|
||||
y, m, d := now.AddDate(0, 0, days).Date()
|
||||
return time.Date(y, m, d, 0, 0, 0, 0, now.Location())
|
||||
}
|
||||
|
||||
// applyRuQualifier moves an hour into the afternoon when he said "вечера" or
|
||||
// "дня" after it. Noon-crossing only: 7 becomes 19, and 19 stays 19. Morning
|
||||
// qualifiers need no arithmetic, they only confirm the hour as spoken.
|
||||
//
|
||||
// The date is recomputed rather than shifted, so an hour that parseClock
|
||||
// already pushed to tomorrow does not land two days out.
|
||||
func applyRuQualifier(t time.Time, qualifier string, now time.Time) time.Time {
|
||||
h := t.Hour()
|
||||
switch strings.Trim(strings.ToLower(qualifier), ".,!?;:") {
|
||||
case "вечера", "дня":
|
||||
if h < 12 {
|
||||
h += 12
|
||||
}
|
||||
case "утра", "ночи":
|
||||
if h == 12 {
|
||||
h = 0
|
||||
}
|
||||
default:
|
||||
return t
|
||||
}
|
||||
out := time.Date(now.Year(), now.Month(), now.Day(), h, t.Minute(), 0, 0, now.Location())
|
||||
if !out.After(now) {
|
||||
out = out.Add(24 * time.Hour)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
+142
-13
@@ -3,6 +3,9 @@ package router
|
||||
import (
|
||||
"regexp"
|
||||
"strings"
|
||||
"unicode"
|
||||
|
||||
"github.com/kami/maven/internal/lexicon"
|
||||
)
|
||||
|
||||
// Grammar — one stage-0 exact-match pattern. Wake-word + known command grammar
|
||||
@@ -196,6 +199,20 @@ func AgendaQueryGrammars() []Grammar {
|
||||
Pattern: regexp.MustCompile(`(?i)(^|\s)(план|дел)[а-я]*\s+(на|в|во|по)\s+` + dayWordPattern + `(\s|[?!.]|$)`),
|
||||
Build: agendaQueryBuild,
|
||||
},
|
||||
{
|
||||
// "что дальше?" — the rest of the day, with no possessive and no
|
||||
// plan word for the rules above to anchor on, so neither claimed
|
||||
// it and the model called it a fact (Vikunja #498). The predicate
|
||||
// for the same utterance already exists as IsRestOfDayQuery, one
|
||||
// layer down in the query chain; this is what gets the turn there.
|
||||
//
|
||||
// "и что там дальше" and "что потом дальше" are the same question,
|
||||
// and "what's next" splits into two tokens, hence the optional
|
||||
// middles rather than plain adjacency.
|
||||
Name: "rest-of-day-query",
|
||||
Pattern: regexp.MustCompile(`(?i)^\s*(и\s+)?(что|чего|what'?s?)\s+(там\s+|ещё\s+|еще\s+|потом\s+|у\s+меня\s+)?(дальше|next)(\s|[?!.]|$)`),
|
||||
Build: agendaQueryBuild,
|
||||
},
|
||||
{
|
||||
// A named event with no calendar word at all: "когда планёрка?",
|
||||
// "во сколько созвон". He is asking when something on his calendar
|
||||
@@ -208,6 +225,11 @@ func AgendaQueryGrammars() []Grammar {
|
||||
}
|
||||
}
|
||||
|
||||
// chatNarrativeTopics — the things "расскажи X" asks for that are not
|
||||
// questions about the world. She is being asked to entertain or to describe
|
||||
// herself, and the query chain has no source for either.
|
||||
var chatNarrativeTopics = regexp.MustCompile(`(?i)(анекдот|шутк|сказк|истори[юи]\s+на\s+ночь|о\s+себе|про\s+себя|о\s+нас|про\s+нас)`)
|
||||
|
||||
// NarrativeQueryGrammars — stage-0 grammars for the two question shapes that
|
||||
// carry no question mark and no interrogative, and so reached the resident
|
||||
// model with nothing deterministic in front of them (Vikunja #498).
|
||||
@@ -237,7 +259,7 @@ func NarrativeQueryGrammars() []Grammar {
|
||||
// Anchored at the start: "запиши что мне рассказали" is a capture,
|
||||
// and a narrative verb buried mid-utterance is not the shape.
|
||||
Name: "narrative-query",
|
||||
Pattern: regexp.MustCompile(`(?i)^\s*(расскажи|объясни|опиши|перечисли|tell|explain|describe)(\s+(.*))?$`),
|
||||
Pattern: narrativeQueryPattern,
|
||||
Build: narrativeQueryBuild,
|
||||
},
|
||||
}
|
||||
@@ -252,15 +274,27 @@ var entertainmentNouns = []string{
|
||||
"joke", "jokes", "story",
|
||||
}
|
||||
|
||||
// narrativeQueryBuild — the narrative shape is a query unless he also said one
|
||||
// of the capture verbs, or asked for entertainment. "расскажи и запиши" is him
|
||||
// asking for a note, and stage 0 must not take either off the cascade.
|
||||
// narrativeQueryBuild — the narrative shape is a query carrying its topic,
|
||||
// unless he also said one of the capture verbs, asked for entertainment, or
|
||||
// asked about her. "расскажи и запиши" is him asking for a note, and stage 0
|
||||
// must not take either off the cascade.
|
||||
//
|
||||
// The topic goes into Slots.Text rather than the whole utterance: the query
|
||||
// chain looks things up by it, and "расскажи мне про Ватерлоо" is a question
|
||||
// about Ватерлоо.
|
||||
func narrativeQueryBuild(m []string) (Decision, bool) {
|
||||
rest := ""
|
||||
if len(m) > 3 {
|
||||
rest = m[3]
|
||||
topic := strings.TrimSpace(m[2])
|
||||
// "расскажи" with nothing after it is a conversational opener, and there is
|
||||
// no topic to look up.
|
||||
if topic == "" {
|
||||
return Decision{}, false
|
||||
}
|
||||
for _, t := range planTokens(rest) {
|
||||
// Against the whole utterance, not the topic: "о себе" has its preposition
|
||||
// eaten by the pattern, leaving a bare "себе".
|
||||
if chatNarrativeTopics.MatchString(m[0]) {
|
||||
return Decision{}, false
|
||||
}
|
||||
for _, t := range planTokens(topic) {
|
||||
for _, v := range captureVerbs {
|
||||
if t == v {
|
||||
return Decision{}, false
|
||||
@@ -272,16 +306,111 @@ func narrativeQueryBuild(m []string) (Decision, bool) {
|
||||
}
|
||||
}
|
||||
}
|
||||
return agendaQueryBuild(m)
|
||||
return Decision{
|
||||
Stage: 0,
|
||||
Intent: IntentQuery,
|
||||
Confidence: 1.0,
|
||||
Slots: Slots{Text: topic},
|
||||
}, true
|
||||
}
|
||||
|
||||
// narrativeQueryPattern — "расскажи про X", built from the lexicon rather than
|
||||
// spelled out here (Vikunja #527). The verbs used to be a second copy of
|
||||
// lexicon.NarrativeRequests, and a second copy of a closed set is a set that
|
||||
// drifts: adding "поясни" in the data file left this rule not knowing it.
|
||||
//
|
||||
// Anchored at the start, which was the point of the old literal and still is:
|
||||
// "запиши что мне рассказали" is a capture, and a narrative verb buried
|
||||
// mid-utterance is not the shape. The dative and the preposition are eaten so
|
||||
// the topic slot comes out clean: "расскажи мне про Ватерлоо" leaves
|
||||
// "Ватерлоо".
|
||||
var narrativeQueryPattern = regexp.MustCompile(
|
||||
`(?is)^\s*(` + strings.Join(lexicon.NarrativeRequests(), "|") +
|
||||
`)(?:\s+(?:мне|нам|us|me))?(?:\s+(?:про|о|об|about))?(\s+.+)$`)
|
||||
|
||||
// FeedQueryGrammar — stage-0 rule for "что нового в лентах?", routed to
|
||||
// IntentQuery so it reaches queryFeeds.
|
||||
//
|
||||
// Same shape of defect as the agenda grammars: the model calls it system, and
|
||||
// replySystem has no feeds arm, so the documented utterance of task 258 step 1
|
||||
// answered "пока не умею отвечать на этот вопрос." while the same question
|
||||
// worded with "новостях" worked (Vikunja #474).
|
||||
//
|
||||
// An ask word at the front and a feed noun after it are both required, which
|
||||
// is the same pair ParseFeedQuery wants. "что нового?" on its own is a greeting
|
||||
// — the most common opener in the language — and vagueNouns in feeds.go exists
|
||||
// to keep it out of the feed reader; routing it to query here would put it
|
||||
// back. "у меня новая лента в инстаграме" carries the noun without the ask and
|
||||
// stays the statement it is.
|
||||
func FeedQueryGrammar() Grammar {
|
||||
return Grammar{
|
||||
Name: "feed-query",
|
||||
// (\s|[?!.]|$) rather than \b, which is ASCII-only and never fires next
|
||||
// to a Cyrillic letter.
|
||||
Pattern: regexp.MustCompile(`(?i)^\s*(что|какие|расскажи|покажи|почитай|прочитай)\s+.*(лент|новостн)[а-я]*(\s|[?!.]|$)`),
|
||||
Build: agendaQueryBuild,
|
||||
}
|
||||
}
|
||||
|
||||
// dayWordPattern — the day words an agenda question can name. Weekdays appear
|
||||
// in the accusative and prepositional forms the questions actually use ("в
|
||||
// среду", "на среде"), which is why the stems carry an inflection tail rather
|
||||
// than a fixed ending.
|
||||
const dayWordPattern = `(сегодня|завтра|послезавтра|выходн[а-я]+|недел[а-я]+|понедельник[а-я]*|вторник[а-я]*|сред[ауые][а-я]*|четверг[а-я]*|пятниц[ауые][а-я]*|суббот[ауые][а-я]*|воскресень[ея][а-я]*)`
|
||||
// среду", "на среде"), so each one contributes its stem plus an inflection
|
||||
// tail; the relative day words are exact.
|
||||
//
|
||||
// Built from the lexicon for the same reason as above. The literal that stood
|
||||
// here spelled all seven weekdays out a second time, in a third file after
|
||||
// cmd/mavend/voice.go and internal/ttsnorm.
|
||||
var dayWordPattern = buildDayWordPattern()
|
||||
|
||||
// agendaQueryBuild — shared Build for the agenda grammars. Confidence 1.0 on
|
||||
// buildDayWordPattern — one alternation over the relative day words, the
|
||||
// weekday stems, and the two period words that are in no closed set ("на
|
||||
// выходных", "на неделе" name a span, not a day).
|
||||
func buildDayWordPattern() string {
|
||||
alts := []string{`выходн[а-я]+`, `недел[а-я]+`}
|
||||
for _, w := range lexicon.DayOffsetWords() {
|
||||
if strings.Contains(w, " ") || !isCyrillic(w) {
|
||||
// Multi-word and English members belong to the offset lookup, not
|
||||
// to a Russian agenda pattern.
|
||||
continue
|
||||
}
|
||||
alts = append(alts, regexp.QuoteMeta(w))
|
||||
}
|
||||
for i := 0; i < 7; i++ {
|
||||
day := lexicon.Weekday(i)
|
||||
if day == "" {
|
||||
continue
|
||||
}
|
||||
alts = append(alts, weekdayStem(day)+`[а-я]*`)
|
||||
}
|
||||
return `(` + strings.Join(alts, "|") + `)`
|
||||
}
|
||||
|
||||
// weekdayStem trims the nominative ending off a weekday so the pattern matches
|
||||
// the case forms an agenda question uses: "среда" has to reach "в среду", and
|
||||
// "понедельник" already ends on its stem.
|
||||
func weekdayStem(day string) string {
|
||||
r := []rune(day)
|
||||
switch r[len(r)-1] {
|
||||
case 'а', 'я', 'е', 'о', 'ь':
|
||||
return string(r[:len(r)-1])
|
||||
}
|
||||
return day
|
||||
}
|
||||
|
||||
// isCyrillic reports whether every rune is Cyrillic. Used to keep the English
|
||||
// members of a bilingual lexicon set out of a Russian-only pattern.
|
||||
func isCyrillic(s string) bool {
|
||||
for _, r := range s {
|
||||
if !unicode.Is(unicode.Cyrillic, r) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return s != ""
|
||||
}
|
||||
|
||||
// agendaQueryBuild — shared Build for the agenda grammars and the feed one,
|
||||
// which all do the same single thing: keep the utterance out of IntentSystem
|
||||
// and let the query chain decide who answers it. Confidence 1.0 on
|
||||
// the intent only: the utterance travels intact and the query chain's own
|
||||
// matchers decide the rest.
|
||||
func agendaQueryBuild(m []string) (Decision, bool) {
|
||||
|
||||
@@ -248,3 +248,32 @@ func TaskCaptureGrammar() Grammar {
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// TaskListGrammar — stage 0 for "какие у меня задачи", "список дел", "что мне
|
||||
// нужно сделать" (Vikunja #467).
|
||||
//
|
||||
// The same exposure the capture marker had, pointed the other way. IsTaskListQuery
|
||||
// is a deterministic lookup that lives inside a query source, so it is only
|
||||
// consulted once the turn is already IntentQuery. A phrasing the model calls
|
||||
// system or note never reaches it, and "пока не умею" is what he hears — the
|
||||
// failure the agenda and feed rules were written for.
|
||||
//
|
||||
// Placed after the agenda rules, which already send "какие у меня задачи" to
|
||||
// query. What this adds is the phrasings with no possessive in them.
|
||||
func TaskListGrammar() Grammar {
|
||||
return Grammar{
|
||||
Name: "task-list-query",
|
||||
Pattern: regexp.MustCompile(`(?s)^\s*(.+)$`),
|
||||
Build: func(m []string) (Decision, bool) {
|
||||
if !IsTaskListQuery(m[1]) {
|
||||
return Decision{}, false
|
||||
}
|
||||
return Decision{
|
||||
Stage: 0,
|
||||
Intent: IntentQuery,
|
||||
Confidence: 1.0,
|
||||
Slots: Slots{Text: strings.TrimSpace(m[1])},
|
||||
}, true
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -119,3 +119,27 @@ func TestTaskCaptureGrammarClaimsTheMarker(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestTaskListGrammarClaimsTheAsk — a list question answered before the model,
|
||||
// including the phrasings with no possessive that used to route elsewhere.
|
||||
func TestTaskListGrammarClaimsTheAsk(t *testing.T) {
|
||||
g := TaskListGrammar()
|
||||
claimed := []string{"какие у меня задачи", "список дел", "что мне нужно сделать"}
|
||||
for _, u := range claimed {
|
||||
m := g.Pattern.FindStringSubmatch(u)
|
||||
if m == nil {
|
||||
t.Fatalf("%q did not match the grammar pattern", u)
|
||||
}
|
||||
d, ok := g.Build(m)
|
||||
if !ok || d.Intent != IntentQuery {
|
||||
t.Errorf("%q built %+v ok=%v; want a query", u, d, ok)
|
||||
}
|
||||
}
|
||||
passed := []string{"как дела", "напомни купить хлеб", "что docker делает"}
|
||||
for _, u := range passed {
|
||||
m := g.Pattern.FindStringSubmatch(u)
|
||||
if _, ok := g.Build(m); ok {
|
||||
t.Errorf("%q was claimed as a task list", u)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -267,3 +267,26 @@ func joinRU(rs []Ranked, limit int, withReasons bool) string {
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// Spoken — the tasks FormatRU actually named, in the order it named them
|
||||
// (Vikunja #448). "второй" has to mean the second thing she said, so the list
|
||||
// an ordinal resolves against is built here and not by a caller guessing how
|
||||
// the renderer split and truncated it.
|
||||
func Spoken(ranked []Ranked) []Ranked {
|
||||
var open, cands []Ranked
|
||||
for _, r := range ranked {
|
||||
if r.Status == StatusCandidate {
|
||||
cands = append(cands, r)
|
||||
} else {
|
||||
open = append(open, r)
|
||||
}
|
||||
}
|
||||
out := make([]Ranked, 0, 2*SpokenLimit)
|
||||
for _, group := range [][]Ranked{open, cands} {
|
||||
if len(group) > SpokenLimit {
|
||||
group = group[:SpokenLimit]
|
||||
}
|
||||
out = append(out, group...)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
@@ -0,0 +1,128 @@
|
||||
package tts
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Pronunciation dictionary (Vikunja #458).
|
||||
//
|
||||
// piper reads a Russian sentence with a Russian voice, and a Latin service id
|
||||
// inside that sentence comes out as letters or as noise: "Vikunja" is spelled
|
||||
// out, "SearXNG" is unreadable, and "homesrv" is read as if it were a word. The
|
||||
// fix is not a code change per name — it is a file of replacements applied to
|
||||
// the text before piper sees it.
|
||||
//
|
||||
// Spelling, not phonemes. piper has no lexicon input of its own here, so the
|
||||
// only lever is the text, and the entry for a name is how it should be spelled
|
||||
// in Russian for the voice to say it right. That also means a wrong entry is
|
||||
// visible: it is a word, and it is read out loud.
|
||||
//
|
||||
// The dictionary is data, so it ships as a file rather than a table in Go. A
|
||||
// name added to it needs no rebuild and no deploy of the daemon that owns the
|
||||
// text — only a restart of mavttsd, which is the process that reads it.
|
||||
|
||||
// Lexicon rewrites names into the spelling the voice reads correctly.
|
||||
//
|
||||
// The zero value is usable and rewrites nothing, so a daemon with no dictionary
|
||||
// configured behaves exactly as it did before this existed.
|
||||
type Lexicon struct {
|
||||
// Rules are held in one alternation rather than as a map, so a text is
|
||||
// scanned once however many entries there are, and the longest name wins
|
||||
// where two overlap ("Home Assistant" before "Home").
|
||||
re *regexp.Regexp
|
||||
// by lower-cased name, because the match is case-insensitive and the
|
||||
// replacement is not derived from what was matched.
|
||||
by map[string]string
|
||||
}
|
||||
|
||||
// LoadLexicon reads a dictionary file: a flat JSON object of name to spelling.
|
||||
//
|
||||
// {"Vikunja": "Викунья", "SearXNG": "сёрчиксэнджи"}
|
||||
//
|
||||
// An empty path returns an empty Lexicon and no error — the dictionary is off
|
||||
// unless configured, like every other optional capability. A path that is set
|
||||
// and unreadable IS an error: he asked for it, and silently saying names wrong
|
||||
// is the failure this exists to remove.
|
||||
func LoadLexicon(path string) (*Lexicon, error) {
|
||||
if strings.TrimSpace(path) == "" {
|
||||
return &Lexicon{}, nil
|
||||
}
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("tts: lexicon %s: %w", path, err)
|
||||
}
|
||||
var entries map[string]string
|
||||
if err := json.Unmarshal(raw, &entries); err != nil {
|
||||
return nil, fmt.Errorf("tts: lexicon %s: %w", path, err)
|
||||
}
|
||||
return NewLexicon(entries), nil
|
||||
}
|
||||
|
||||
// NewLexicon builds a lexicon from entries already in memory.
|
||||
func NewLexicon(entries map[string]string) *Lexicon {
|
||||
names := make([]string, 0, len(entries))
|
||||
by := make(map[string]string, len(entries))
|
||||
for name, say := range entries {
|
||||
name = strings.TrimSpace(name)
|
||||
if name == "" || strings.TrimSpace(say) == "" {
|
||||
continue
|
||||
}
|
||||
names = append(names, name)
|
||||
by[strings.ToLower(name)] = say
|
||||
}
|
||||
if len(names) == 0 {
|
||||
return &Lexicon{}
|
||||
}
|
||||
// Longest first: "Home Assistant" must match before "Home" does, and Go's
|
||||
// regexp alternation is leftmost-first, not longest-match.
|
||||
sort.Slice(names, func(i, j int) bool { return len(names[i]) > len(names[j]) })
|
||||
quoted := make([]string, len(names))
|
||||
for i, n := range names {
|
||||
quoted[i] = regexp.QuoteMeta(n)
|
||||
}
|
||||
// The boundaries are written out rather than left to \b, which is ASCII-only
|
||||
// and never fires next to a Cyrillic letter — so "в Vikunja," would not
|
||||
// match with \b on the left in a Russian sentence.
|
||||
pattern := `(?i)(^|[^\p{L}\p{N}_])(` + strings.Join(quoted, "|") + `)($|[^\p{L}\p{N}_])`
|
||||
return &Lexicon{re: regexp.MustCompile(pattern), by: by}
|
||||
}
|
||||
|
||||
// Apply rewrites every name in the text. Text with no name in it comes back
|
||||
// unchanged and untouched.
|
||||
func (l *Lexicon) Apply(text string) string {
|
||||
if l == nil || l.re == nil || text == "" {
|
||||
return text
|
||||
}
|
||||
// Twice, because two names separated by a single space share the character
|
||||
// between them and one pass consumes it: "Nexus Praxis" would leave the
|
||||
// second name alone otherwise.
|
||||
out := l.replaceOnce(text)
|
||||
return l.replaceOnce(out)
|
||||
}
|
||||
|
||||
func (l *Lexicon) replaceOnce(text string) string {
|
||||
return l.re.ReplaceAllStringFunc(text, func(m string) string {
|
||||
groups := l.re.FindStringSubmatch(m)
|
||||
if groups == nil {
|
||||
return m
|
||||
}
|
||||
say, ok := l.by[strings.ToLower(groups[2])]
|
||||
if !ok {
|
||||
return m
|
||||
}
|
||||
return groups[1] + say + groups[3]
|
||||
})
|
||||
}
|
||||
|
||||
// Size reports how many names are loaded, for the startup log line.
|
||||
func (l *Lexicon) Size() int {
|
||||
if l == nil {
|
||||
return 0
|
||||
}
|
||||
return len(l.by)
|
||||
}
|
||||
@@ -0,0 +1,85 @@
|
||||
package tts
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestLexiconRewritesNames(t *testing.T) {
|
||||
lex := NewLexicon(map[string]string{
|
||||
"Vikunja": "Викунья",
|
||||
"Home Assistant": "Хоум Ассистент",
|
||||
"Home": "Хоум",
|
||||
"GPU": "джи-пи-ю",
|
||||
})
|
||||
for _, tc := range []struct{ in, want string }{
|
||||
{"задача в Vikunja готова", "задача в Викунья готова"},
|
||||
// Case-insensitive: the router and the model both change the case of a
|
||||
// name on the way through.
|
||||
{"открой vikunja.", "открой Викунья."},
|
||||
// Longest first, or "Home Assistant" is read as "Хоум Assistant".
|
||||
{"Home Assistant не отвечает", "Хоум Ассистент не отвечает"},
|
||||
// Two names in a row share the space between them, which one pass
|
||||
// would consume.
|
||||
{"GPU GPU", "джи-пи-ю джи-пи-ю"},
|
||||
// Not a word boundary: a name inside a longer token is left alone.
|
||||
{"vikunjaless", "vikunjaless"},
|
||||
{"ничего не совпало", "ничего не совпало"},
|
||||
{"", ""},
|
||||
} {
|
||||
if got := lex.Apply(tc.in); got != tc.want {
|
||||
t.Errorf("Apply(%q) = %q, want %q", tc.in, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The zero value and an unconfigured path rewrite nothing, so a daemon with no
|
||||
// dictionary behaves as it did before this existed.
|
||||
func TestLexiconOffByDefault(t *testing.T) {
|
||||
var zero *Lexicon
|
||||
if got := zero.Apply("Vikunja"); got != "Vikunja" {
|
||||
t.Errorf("nil lexicon rewrote %q", got)
|
||||
}
|
||||
lex, err := LoadLexicon("")
|
||||
if err != nil {
|
||||
t.Fatalf("LoadLexicon(\"\"): %v", err)
|
||||
}
|
||||
if lex.Size() != 0 || lex.Apply("Vikunja") != "Vikunja" {
|
||||
t.Errorf("empty path produced a live lexicon of %d names", lex.Size())
|
||||
}
|
||||
}
|
||||
|
||||
// A path he set and that cannot be read is a startup failure. Saying names
|
||||
// wrong in silence is what the dictionary exists to stop.
|
||||
func TestLexiconLoadErrors(t *testing.T) {
|
||||
if _, err := LoadLexicon(filepath.Join(t.TempDir(), "nope.json")); err == nil {
|
||||
t.Error("a missing dictionary must be an error")
|
||||
}
|
||||
bad := filepath.Join(t.TempDir(), "bad.json")
|
||||
if err := os.WriteFile(bad, []byte("{not json"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := LoadLexicon(bad); err == nil {
|
||||
t.Error("an unparseable dictionary must be an error")
|
||||
}
|
||||
}
|
||||
|
||||
func TestLexiconRoundTripsAFile(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "lex.json")
|
||||
if err := os.WriteFile(path, []byte(`{"Praxis":"Праксис"," ":"skipped","Nexus":""}`), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
lex, err := LoadLexicon(path)
|
||||
if err != nil {
|
||||
t.Fatalf("LoadLexicon: %v", err)
|
||||
}
|
||||
// Blank names and blank spellings are dropped: an entry that says nothing
|
||||
// would delete the word it matched.
|
||||
if lex.Size() != 1 {
|
||||
t.Fatalf("Size = %d, want 1", lex.Size())
|
||||
}
|
||||
if got := lex.Apply("Praxis молчит"); got != "Праксис молчит" {
|
||||
t.Errorf("Apply = %q", got)
|
||||
}
|
||||
}
|
||||
@@ -7,10 +7,13 @@ import (
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/kami/maven/internal/lexicon"
|
||||
)
|
||||
|
||||
var months = [...]string{"", "января", "февраля", "марта", "апреля", "мая",
|
||||
"июня", "июля", "августа", "сентября", "октября", "ноября", "декабря"}
|
||||
// The month names are a closed class and live in internal/lexicon, 1-indexed,
|
||||
// which is also where the voice reply path reads them. There used to be a second
|
||||
// copy of the twelve names in cmd/mavend/ruwords.go (Vikunja #525).
|
||||
|
||||
var (
|
||||
reDateY = regexp.MustCompile(`\b(\d{1,2})\.(\d{1,2})\.(\d{4})\b`)
|
||||
@@ -47,7 +50,7 @@ func spokenDate(dd, mm, yyyy string) string {
|
||||
return dd + " " + mm + gap(yyyy)
|
||||
}
|
||||
day := strconv.Itoa(mustInt(dd))
|
||||
out := day + " " + months[mi]
|
||||
out := day + " " + lexicon.MonthGenitive(mi)
|
||||
if yyyy != "" {
|
||||
out += " " + yyyy
|
||||
}
|
||||
|
||||
+8
@@ -0,0 +1,8 @@
|
||||
data
|
||||
vendor
|
||||
.vscode
|
||||
# Testing and benchmarks
|
||||
*.out
|
||||
*.test
|
||||
pprof
|
||||
.DS_Store
|
||||
+21
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2019 Anton Södergren
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
+59
@@ -0,0 +1,59 @@
|
||||
SHELL:=/usr/bin/env bash
|
||||
default: all
|
||||
LANG=en
|
||||
all:
|
||||
# go get -u github.com/jteeuwen/go-bindata/...
|
||||
mkdir -p data
|
||||
$(MAKE) en sv fr es de it ru uk
|
||||
|
||||
package-all:
|
||||
$(MAKE) LANG=en package
|
||||
$(MAKE) LANG=sv package
|
||||
$(MAKE) LANG=fr package
|
||||
$(MAKE) LANG=es package
|
||||
$(MAKE) LANG=de package
|
||||
$(MAKE) LANG=it package
|
||||
$(MAKE) LANG=ru package
|
||||
$(MAKE) LANG=uk package
|
||||
|
||||
en:
|
||||
$(MAKE) LANG=en download package
|
||||
sv:
|
||||
$(MAKE) LANG=sv download package
|
||||
fr:
|
||||
$(MAKE) LANG=fr download package
|
||||
es:
|
||||
$(MAKE) LANG=es download package
|
||||
de:
|
||||
$(MAKE) LANG=de download package
|
||||
it:
|
||||
$(MAKE) LANG=it download package
|
||||
ru:
|
||||
$(MAKE) LANG=ru download package
|
||||
uk:
|
||||
$(MAKE) LANG=uk download package
|
||||
|
||||
download:
|
||||
curl https://raw.githubusercontent.com/michmech/lemmatization-lists/master/lemmatization-$(LANG).txt > data/$(LANG)
|
||||
|
||||
package:
|
||||
# Packaging $(LANG)
|
||||
go run cmd/simplify/simplify.go data/$(LANG) data/$(LANG).gz
|
||||
go run cmd/genpack/genpack.go -locale $(LANG) -path data/$(LANG).gz > v4/dicts/$(LANG)/pack.go
|
||||
# ----------------
|
||||
|
||||
benchcmp:
|
||||
# ensure no govenor weirdness
|
||||
# sudo cpufreq-set -g performance
|
||||
go test -test.benchmem=true -run=NONE -bench=. ./... > bench_current.test
|
||||
git stash save "stashing for benchcmp"
|
||||
@go test -test.benchmem=true -run=NONE -bench=. ./... > bench_head.test
|
||||
git stash pop
|
||||
benchcmp bench_head.test bench_current.test
|
||||
|
||||
profile:
|
||||
@mkdir -p pprof/
|
||||
go test -run=NONE -cpuprofile pprof/cpu.prof -memprofile pprof/mem.prof -bench .
|
||||
go tool pprof -pdf pprof/cpu.prof > pprof/cpu.pdf
|
||||
xdg-open pprof/cpu.pdf
|
||||
go tool pprof -weblist=.* pprof/cpu.prof
|
||||
+87
@@ -0,0 +1,87 @@
|
||||
# GoLem
|
||||
|
||||
This project is a dictionary based lemmatizer written in go.
|
||||
|
||||
Since v4 all dictionaries need to be gotten individually.
|
||||
|
||||
```
|
||||
go get github.com/aaaton/golem/v4
|
||||
```
|
||||
|
||||
|
||||
### What?
|
||||
|
||||
A [lemmatizer](https://en.wikipedia.org/wiki/Lemmatisation) is a tool that finds the base form of words.
|
||||
|
||||
| Lang | Input | Output |
|
||||
| ------- | ---------- | ------- |
|
||||
| English | aligning | align |
|
||||
| Swedish | sprungit | springa |
|
||||
| French | abattaient | abattre |
|
||||
|
||||
It's based on the dictionaries found on [michmech/lemmatization-lists](https://github.com/michmech/lemmatization-lists), which are available under the [Open Database License](https://opendatacommons.org/licenses/odbl/summary/). This project would not be feasible without them.
|
||||
|
||||
### Languages
|
||||
|
||||
At the moment golem supports English, Swedish, French, Spanish, Italian & German, but adding another language should be no more trouble than getting the dictionary for that language. Some of which are already available on lexiconista. Please let me know if there is something you would like to see in here, or fork the project and create a pull request.
|
||||
|
||||
English
|
||||
```
|
||||
go get github.com/aaaton/golem/v4/dicts/en
|
||||
```
|
||||
|
||||
Swedish
|
||||
```
|
||||
go get github.com/aaaton/golem/v4/dicts/sv
|
||||
```
|
||||
|
||||
French
|
||||
```
|
||||
go get github.com/aaaton/golem/v4/dicts/fr
|
||||
```
|
||||
|
||||
German
|
||||
```
|
||||
go get github.com/aaaton/golem/v4/dicts/de
|
||||
```
|
||||
|
||||
Spanish
|
||||
```
|
||||
go get github.com/aaaton/golem/v4/dicts/es
|
||||
```
|
||||
|
||||
Italian
|
||||
```
|
||||
go get github.com/aaaton/golem/v4/dicts/it
|
||||
```
|
||||
|
||||
### Basic usage
|
||||
|
||||
```golang
|
||||
package main
|
||||
|
||||
import (
|
||||
"github.com/aaaton/golem/v4"
|
||||
"github.com/aaaton/golem/v4/dicts/en"
|
||||
)
|
||||
|
||||
func main() {
|
||||
// the language packages are available under golem/dicts
|
||||
// "en" is for english
|
||||
lemmatizer, err := golem.New(en.New())
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
word := lemmatizer.Lemma("Abducting")
|
||||
if word != "abduct" {
|
||||
panic("The output is not what is expected!")
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Contributors
|
||||
|
||||
- axamon
|
||||
- charlesgiroux
|
||||
- glaslos
|
||||
- ptdewey
|
||||
+21
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2019 Anton Södergren
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
+43
File diff suppressed because one or more lines are too long
+101
@@ -0,0 +1,101 @@
|
||||
package golem
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// LanguagePack is what each language should implement
|
||||
type LanguagePack interface {
|
||||
GetResource() ([]byte, error)
|
||||
GetLocale() string
|
||||
}
|
||||
|
||||
// Lemmatizer is the key to lemmatizing a word in a language
|
||||
type Lemmatizer struct {
|
||||
m map[string]int
|
||||
v [][]string
|
||||
}
|
||||
|
||||
func newLemmatizerFromBytes(b []byte) (Lemmatizer, error) {
|
||||
lines := strings.Split(string(b), "\n")
|
||||
s := Lemmatizer{
|
||||
m: make(map[string]int),
|
||||
v: [][]string{},
|
||||
}
|
||||
// TODO: Would it be better to do with a reader
|
||||
// instead of loading the full thing into an array?
|
||||
|
||||
// br := bufio.NewReader(bytes.NewReader(b))
|
||||
// line, err := br.ReadString('\n')
|
||||
// for err == nil {
|
||||
// wordIndex := make(map[string])
|
||||
for _, line := range lines {
|
||||
if len(line) == 0 {
|
||||
continue
|
||||
}
|
||||
words := strings.Split(line, "\t")
|
||||
if len(words) < 2 {
|
||||
return s, fmt.Errorf("expected more than 1 form per word")
|
||||
}
|
||||
base := words[0]
|
||||
for _, word := range words {
|
||||
if index, ok := s.m[word]; ok {
|
||||
s.v[index] = append(s.v[index], word)
|
||||
} else {
|
||||
index := len(s.v)
|
||||
s.v = append(s.v, []string{base})
|
||||
s.m[word] = index
|
||||
}
|
||||
}
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
|
||||
// New produces a new Lemmatizer
|
||||
func New(pack LanguagePack) (*Lemmatizer, error) {
|
||||
resource, err := pack.GetResource()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf(`Could not open resource file for "%s"`, pack.GetLocale())
|
||||
}
|
||||
l, err := newLemmatizerFromBytes(resource)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf(`language %s is not valid: %s`, pack.GetLocale(), err)
|
||||
}
|
||||
return &l, nil
|
||||
}
|
||||
|
||||
// InDict checks if a certain word is in the dictionary
|
||||
func (l *Lemmatizer) InDict(word string) bool {
|
||||
_, ok := l.m[strings.ToLower(word)]
|
||||
return ok
|
||||
}
|
||||
|
||||
// Lemma gets one of the base forms of a word
|
||||
func (l *Lemmatizer) Lemma(word string) string {
|
||||
if out, ok := l.m[strings.ToLower(word)]; ok {
|
||||
return l.v[out][0]
|
||||
}
|
||||
return word
|
||||
}
|
||||
|
||||
// LemmaLower gets one of the base forms of a lower case word
|
||||
// expects `word` to be lowercased
|
||||
func (l *Lemmatizer) LemmaLower(word string) string {
|
||||
if out, ok := l.m[word]; ok {
|
||||
return l.v[out][0]
|
||||
}
|
||||
return word
|
||||
}
|
||||
|
||||
// Lemmas gets all the base forms of a word, if multiple exist
|
||||
func (l *Lemmatizer) Lemmas(word string) (out []string) {
|
||||
if index, ok := l.m[strings.ToLower(word)]; ok {
|
||||
out := l.v[index]
|
||||
// to get rid of the randomness, we sort the output
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
return []string{word}
|
||||
}
|
||||
Vendored
+6
-3
@@ -1,3 +1,9 @@
|
||||
# github.com/aaaton/golem/v4 v4.0.2
|
||||
## explicit; go 1.13
|
||||
github.com/aaaton/golem/v4
|
||||
# github.com/aaaton/golem/v4/dicts/ru v0.0.0-20250408131944-3488790fc110
|
||||
## explicit; go 1.13
|
||||
github.com/aaaton/golem/v4/dicts/ru
|
||||
# github.com/coder/websocket v1.8.12
|
||||
## explicit; go 1.19
|
||||
github.com/coder/websocket
|
||||
@@ -15,8 +21,6 @@ github.com/google/uuid
|
||||
# github.com/kami/hexis v0.0.0 => /home/kami/apps/hexis
|
||||
## explicit; go 1.25.5
|
||||
github.com/kami/hexis/pkg/client
|
||||
# github.com/kami/praxis v0.0.0 => /home/kami/apps/praxis
|
||||
## explicit; go 1.23
|
||||
# github.com/mattn/go-isatty v0.0.20
|
||||
## explicit; go 1.15
|
||||
github.com/mattn/go-isatty
|
||||
@@ -79,4 +83,3 @@ modernc.org/memory
|
||||
modernc.org/sqlite
|
||||
modernc.org/sqlite/lib
|
||||
modernc.org/sqlite/vtab
|
||||
# github.com/kami/nexus v0.0.0 => /home/kami/apps/nexus
|
||||
|
||||
Reference in New Issue
Block a user