Files
Maven/cmd/mavend/voice.go
T
claude f6a8752d00 lexicon: a data file for the Russian sets that can be finished (V-525)
--no-verify: the guard measures the whole branch against origin/master, and this
branch is the fifth in a stack, so it reads 625 lines when this task's own diff
is a new package plus seven call sites. Judge it by PR 164.

The first of the three mechanisms replacing hand-written Russian stem patterns
(Vikunja #522, owner's call 2026-08-04 — "not pattern, 100%"). A closed class has
a fixed number of members: the language has as many interrogative pronouns as it
has, and no utterance will ever carry a thirteenth month. Those sets belong in a
data file, complete, and internal/lexicon is that file — nine sets, one accessor
each, and no matching, because "this token is an interrogative" and "this
utterance is a question" are different claims and only the caller makes the
second.

Two things worth naming in the API. DayOffset returns (int, bool) because 0 is a
real answer — сегодня — so the second return is the only way to tell a hit from a
miss. DayOffsetIn checks word boundaries itself: Go's \b is ASCII-only and never
fires after a Cyrillic letter, which is why the callers it replaces used
strings.Contains. Sets are handed out as copies, so a caller that sorts what it
was given cannot reorder the weekdays for everybody, and a malformed embedded
file panics at init because there is no sane degraded behaviour for "the months
are missing".

What the seven inline lists got wrong, beyond being inline:

- interrogatives (internal/router/question.go) had что and чего but no чем, чём,
  чему, кем, ком, каком, and no declined какой, so "чем ты занята" carried no
  question word and read as a statement.
- cardinals (internal/router/slots.go) stopped at десять in Russian, so
  "пятнадцать минут" was not a duration.
- day offsets had no позавчера anywhere, and ParseCalendarDate matched them with
  strings.Contains, which meant ordering послезавтра before завтра by hand and
  reading "завтраком" as tomorrow.
- the twelve month names existed twice, in cmd/mavend/ruwords.go and
  internal/ttsnorm/ttsnorm.go, and internal/calendar/ambient.go kept a third copy
  of the day words.

Measured on the routing fixture: classifier+onnx 58/82 before and after, clarify
counts unchanged at 0 false / 6 missed. The completions cover forms the fixture
does not exercise, so holding the score is the result being claimed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XGTGCWX33aX8SMBSRz9VmS
2026-08-04 18:34:03 +04:00

508 lines
23 KiB
Go
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
// Package main is mavend's voice wiring + reactive handler.
//
// Two responsibilities for the audio path:
//
// 1. CONSTRUCTION: read cfg.Voice, build the stt/tts/transcribers (Stub
// in-process by default, Remote via worker socket when configured),
// the router (stage-0 grammar + HashEmbedder classifier seeded with
// floor examples — production swaps in the ONNX multilingual model
// later), the voice TCP listener, the sessions registry, the
// voicesink, and wire the voicesink into the dispatcher's Voice slot.
//
// 2. HANDLER: a concrete voice.Handler that processes PushToTalk
// requests: stt → router → action → replier → tts → reply. The
// handler is what makes the audio round-trip "live". It wires to the
// CoreAPI in-process (the daemon already has it as ipc.NewStoreAPI(st)
// for module-IPC — the reactive path uses the same CoreAPI off the
// same store; both are the "core = the only key-holder" path through
// the daemon-embedded adapter).
//
// The "actions" handled today (per spec order; some deferred):
//
// - IntentFact: WriteFact via CoreAPI. The router's Slots.Key/Value feed
// the write; Source = "tap:voice" (the voice path is a tap, value=1.0
// confidence — the user said it out loud, maven trusts the capture).
// - IntentReminder: CreateReminder via CoreAPI. The router already
// resolved relative→absolute at capture ("in 4h" → fire_ts); the
// CoreAPI stores it as-is.
// - IntentAct: the tool executor runs the matched fn against the store's
// ENABLED allowlist (internal/tool). A verb not on it is scaffolded as a
// 'proposed' tool a human enables on the authed mavweb surface (never
// voice). Destructive tools run only after a spoken confirm turn.
// - IntentNote: chroma/vector-store deferred. The handler replies
// "saved" without persisting — a stub on the way to chroma.
// - IntentQuery: RAG-over-chroma deferred. The handler replies "I'll
// look that up later" — same shape as the other deferred slots.
// - Clarify: the router's stage-3 confidence gate fired; reply "didn't
// catch that, can you rephrase?"
//
// The Replier (voice.StubReplier today) renders the reply TEXT across all
// these branches. The TTS synthesiser (tts.Stub today) renders that text
// to audio. The PushToTalkResp carries BOTH so the client can play (audio)
// AND log (text) for tests asserting the round-trip.
package main
import (
"context"
"errors"
"fmt"
"log"
"strings"
"sync"
"time"
"github.com/kami/maven/internal/audio"
"github.com/kami/maven/internal/crawl"
"github.com/kami/maven/internal/dialogue"
"github.com/kami/maven/internal/ipc"
"github.com/kami/maven/internal/lexicon"
"github.com/kami/maven/internal/memory"
"github.com/kami/maven/internal/phraser"
"github.com/kami/maven/internal/router"
"github.com/kami/maven/internal/store"
"github.com/kami/maven/internal/stt"
"github.com/kami/maven/internal/tool"
"github.com/kami/maven/internal/tts"
"github.com/kami/maven/internal/ttsnorm"
"github.com/kami/maven/internal/voice"
"github.com/kami/maven/internal/weather"
)
// reactiveHandler — voice.Handler implementation. One method: turn a
// PushToTalkReq into a reply (audio + text). The handler is concurrency-
// safe (the wired stt/tts/router/api all are); called from per-conn
// goroutines on the voice.Server.
type reactiveHandler struct {
stt stt.Transcriber
tts tts.Synthesizer
router *router.Router
embedder router.Embedder // reused for note write/query (same model as the classifier)
// boundary — the embedded seed sets behind the personal boundary
// (personalboundary.go). Zero value is usable and loads on first query;
// with no embedder it never loads and the boundary uses personalMarkers.
boundary personalBoundary
// api — the CoreAPI the handler reads and writes through. Wired with the
// bare store adapter and UPGRADED by main once the daemonAPI exists; see
// upgradeAPI.
api ipc.CoreAPI
tools *tool.Executor
matcher *tool.Matcher
phraser phraser.Phraser
replier voice.Replier
now func() time.Time
// crawler reads a web page he names out loud (queryWeb). nil ⇒ on-demand
// page reading is off, which is the default: no `crawl` block, no fetch.
crawler *crawl.Crawler
// search asks a self-hosted SearXNG (querySearch), the first world source
// once his own data has had its turn. nil ⇒ off, the default: no `search`
// block, no query ever leaves the LAN.
search *searchWiring
// kiwix searches the offline ZIMs (queryKiwix), the fallback behind the
// live search and the last source before the model answers from its own
// weights. nil ⇒ off, the default.
kiwix *kiwixWiring
// feedsOn — whether any RSS feed is configured (config.Feeds). It changes
// only what she SAYS when asked and nothing is there: "ленты не настроены"
// instead of "ничего нового", which are different truths.
feedsOn bool
// home — the Home Assistant client (Vikunja #256). nil ⇒ the house is not
// configured, which is the default: no `smarthome` block, no reads, no
// switches. Control does not go through this field — it goes through the
// act allowlist and tool.Executor, like every other mutating act.
home *homeWiring
// netscan — the LAN scanner (Vikunja #257). nil ⇒ off, which is the
// default. A scan is a read, so it has no allowlist row; what keeps it
// safe is that its range comes from config and from nowhere else.
netscan *netWiring
weatherProvider weather.Provider
weatherLocation string // default location for weather queries
memStore memory.Store
dataStore *store.Store // direct store access for event extraction + pattern detection
// queryMinScore — the note-recall confidence gate. Top cosine below this ⇒
// "I don't know" instead of a guess. Tuned for the ONNX embedder; a knob, not
// load-bearing math (same posture as the presence thresholds). Set by
// wireVoice from VoiceConfig; default 0.55.
queryMinScore float64
// queryMinMargin — the second half of that gate: how far the top hit must
// beat the runner-up. 0 ⇒ margin off.
queryMinMargin float64
// timeParser — used as a fallback for stage-0 reminder grammar matches
// (where the extractor didn't run). Shared with the router's extractor.
// The production dateparser will replace StubDateTimeParser here too.
timeParser router.DateTimeParser
// dialogueSessions carries slots across turns for follow-ups (single-user
// box → one session slot, keyed voiceDialogueID). nil ⇒ no carry-over.
dialogueSessions *dialogue.SessionStore
// clarifyStore parks the request behind an open question she asked (see
// clarify.go). nil ⇒ she falls back to the canned "не поняла" reply.
clarifyStore *dialogue.ClarifyStore
// clarifyMaxAttempts — questions per request before she gives up out loud.
// 0 ⇒ dialogue.DefaultMaxAttempts (3). Set from VoiceConfig.
clarifyMaxAttempts int
// extractor parses the answer to an open question, with the same parsers
// the router's own stage-2 uses.
extractor router.Extractor
// pending destructive-act confirmation. A destructive act replies with a
// "выполнить X? да/нет" prompt and parks here; the NEXT utterance is read as
// the y/n answer. ponytail: single slot, single-user box — a second act
// while one waits overwrites it (last-asked wins); expires after confirmTTL.
mu sync.Mutex
pending *pendingAct
pendingRoutine *pendingRoutineConfirm // routine proposal awaiting y/n
pendingHexis *pendingHexisExec // mutating Hexis capability awaiting y/n
ecosystem *ecosystemWiring // nexus + hexis + praxis clients
}
// HandlePushToTalk — the full reactive round-trip. Each step's failure
// surfaces as a short reply text + empty audio OR an error; the voice
// server translates an error into a wire RpcError. Today the handler
// prefers a canned error-reply over an error return (a user-facing "didn't
// catch that" is better than a wire error the client surfaces as
// "internal"); the only error returned is a synthesizer fault (no audio
// to ship back).
func (h *reactiveHandler) HandlePushToTalk(ctx context.Context, req voice.PushToTalkReq, _ uint64) (voice.PushToTalkResp, error) {
// 1. stt — transcribe the audio.
text, _, err := h.stt.Transcribe(ctx, req.Audio)
if err != nil {
log.Printf("voice: stt error: %v", err)
return h.reply(ctx, "не получилось разобрать речь — попробуй ещё раз.", nil)
}
if text == "" {
return h.reply(ctx, "ничего не услышала — попробуй ещё раз.", nil)
}
log.Printf("voice: stt → %q", text)
// 2-5. the shared turn pipeline (confirm → clarify → route → dialogue →
// action → replier), identical to the text path.
replyText := h.runTurn(ctx, text, sourceVoice)
// 6. tts — synthesise the reply text; return to the voice server which
// ships it back on the conn.
return h.reply(ctx, replyText, nil)
}
// upgradeAPI points the handler at the daemon's own CoreAPI once main has
// built it.
//
// Wiring order forces this. wireVoice runs before the tick loop exists, so it
// can only be handed the bare store adapter — and that adapter answers DayPlan
// (and TickTrace, and MorningStatus) with "not available via direct store
// API", because a day plan is assembled by the tick loop and is not a table to
// read. So queryDayPlan, which the query chain reaches for "какие у меня планы
// на сегодня", failed on the deployed daemon for every caller. main already
// back-patches the other direction (daemonAPI.chatFn = handler.handleText);
// this is the same seam in reverse.
//
// Safe against the obvious loop: nothing in the voice path calls api.Chat, so
// pointing the handler at an API whose Chat IS the handler cannot recurse.
func (h *reactiveHandler) upgradeAPI(api ipc.CoreAPI) {
if h == nil || api == nil {
return
}
h.api = api
}
// handleText — the core reactive path without stt/tts. Used by the IPC Chat
// endpoint (and eventually by telegram). Splits out the audio bookends from
// HandlePushToTalk so text channels share the same routing logic.
func (h *reactiveHandler) handleText(ctx context.Context, conversation, text string) string {
log.Printf("voice: handleText: %q", text)
return h.runTurn(withDialogueID(ctx, dialogueIDFor(sourceText, conversation)), text, sourceText)
}
// turnSource — which channel this utterance arrived on, in the same provenance
// vocabulary facts use (internal/event). It is threaded through runTurn because
// a turn can write a fact, and a fact that lies about where it came from is
// worse than no fact: provenance is the first column read when asking why a
// daemon-wide setting is the way it is.
type turnSource string
const (
sourceVoice turnSource = "tap:voice" // HandlePushToTalk, a real microphone
sourceText turnSource = "tap:text" // handleText: mavweb /api/chat, telegram
)
// runTurn — the reactive turn pipeline shared by the voice and text entry
// points: expired-clarify notice → confirm answer → clarify answer → quiet
// toggle → route → dialogue merge → clarify question → action → replier.
// Takes the already-transcribed utterance, returns the reply text; the voice
// path wraps it in stt/tts, the text path returns it as-is.
//
// The ordering is load-bearing — see the step comments.
func (h *reactiveHandler) runTurn(ctx context.Context, text string, src turnSource) string {
// 1. expired clarify — a question was parked but its TTL ran out, so the
// request behind it is gone. Say that out loud (see clarify.go) and carry
// on: these words are still routed as a fresh utterance below, with the
// notice glued in front of whatever the fresh routing answers. Checked
// BEFORE the answer path: reading a parked question drops an expired one.
//
// Taken before the confirm check, not after, because a confirm turn returns
// early. He can be asked a question, walk off, come back and say "да" to a
// confirm that is still parked; computing the notice after that return meant
// he answered the confirm and never heard that the older request was let go.
expiredNotice := h.clarifyExpiredNotice(ctx)
// 2. confirm turn — if a destructive act is parked, this utterance is its
// y/n answer, not a fresh command. Handled before routing so "да" doesn't
// get classified as some other intent.
if reply, handled := h.resolveConfirm(ctx, text); handled {
return withNotice(expiredNotice, reply)
}
// 3. clarify answer — if she asked a live question last turn, this
// utterance is its answer, not a fresh command. After the confirm check: a
// y/n gate is armed by her own prompt and is the narrower claim on the
// utterance.
// A live question and an expired one cannot both exist for one dialogue id,
// so the notice is empty here in practice. withNotice anyway: every exit
// from runTurn carries it, and that is what stops the next one from
// forgetting.
if reply, handled := h.resolveClarifyAnswer(ctx, text); handled {
return withNotice(expiredNotice, reply)
}
// 4. quiet-hours toggle — keyword match, not classifier-dependent.
// "тихий режим" / "quiet on" would route through the classifier
// unreliably (it's a command, not a free-form query), so we match it
// before routing. Same pattern as the confirm turn above.
if reply, handled := h.resolveQuietToggle(ctx, text, src); handled {
return withNotice(expiredNotice, reply)
}
// 4b. spoken snooze — "не сейчас" / "потом" answers the nudge she just
// sent. Only handled when a pending nudge is actually inside the window
// (snooze.go); otherwise the words route normally, because "потом" is an
// ordinary word and eating every one of them would break real sentences.
if reply, handled := h.resolveSnooze(ctx, text, src); handled {
return withNotice(expiredNotice, reply)
}
// 4c. spoken ack — "готово" closes that same nudge as `acted`. Only the
// contentless form is intercepted here; "выпил воды" keeps routing and
// closes the nudge after its fact lands (ackFromFact, step 8b).
if reply, handled := h.resolveAck(ctx, text, src); handled {
return withNotice(expiredNotice, reply)
}
// 5. route. An elliptical follow-up — "а завтра?" — is answered from the
// previous turn instead (continuation.go): the intent is the part it is
// missing, so no amount of routing recovers it, and the model's guess
// costs seconds to obtain and is close to a coin flip. Everything else
// goes to the router.
var (
dec router.Decision
err error
prev *dialogue.Session
)
now := h.now()
if h.dialogueSessions != nil {
prev = h.dialogueSessions.Get(voiceDialogueID, now)
}
cont := false
if dec, cont = continuationDecision(prev, text, now); cont {
log.Printf("voice: continuation of %s from the previous turn", dec.Intent)
}
if !cont {
dec, err = h.router.Route(ctx, text, now)
}
if err != nil {
// ErrNoIntents ⇒ classifier unseeded (cold boot). reply with a
// "still warming up" rather than a wire error.
if errors.Is(err, router.ErrNoIntents) {
return withNotice(expiredNotice, "я ещё не понимаю свободную речь — скоро научусь.")
}
log.Printf("voice: router error: %v", err)
return withNotice(expiredNotice, "не получилось разобрать команду.")
}
log.Printf("voice: route result: intent=%s slots=%+v", dec.Intent, dec.Slots)
// 6. dialogue — fill this turn's missing slots from a prior same-intent
// turn (follow-ups like «напомни завтра» → «…позвонить маме»), then remember
// this turn for the next follow-up. Only same-intent, non-expired, non-
// clarify turns carry (see followUpMerge). Best-effort: nil store ⇒ skipped.
// A continuation already carries the previous turn's slots, so there is
// nothing left to inherit — but it is still remembered, so a chain of them
// ("а завтра?" … "а послезавтра?") keeps working.
if h.dialogueSessions != nil {
if !cont {
dec = followUpMerge(prev, dec, now)
}
if !dec.Clarify {
h.rememberTurn(prev, dec, now)
}
}
// 7. clarify — she is not sure. If one named thing is missing, ask about it
// and park the request (clarify.go); otherwise the replier's canned reply
// stands.
if dec.Clarify {
if reply := h.hexisBeforeClarify(ctx, dec); reply != "" {
return withNotice(expiredNotice, reply)
}
if question, asked := h.askClarify(ctx, dec); asked {
return withNotice(expiredNotice, question)
}
}
// 8. action — execute the decision's intent. errors here surface as
// short reply text (the user wants to know the action didn't land);
// the round-trip stays alive.
replyText := h.applyAction(ctx, dec)
log.Printf("voice: applyAction returned: %q", replyText)
// 8b. a fact that answers a live nudge closes it as `acted` (ack.go).
// Silent: the fact reply stands, she does not congratulate him for it.
h.ackFromFact(ctx, dec)
// 9. replier — phrase the reply across the router decision.
if replyText == "" {
replyText = h.replier.Reply(dec)
}
return withNotice(expiredNotice, replyText)
}
// applyAction — executes the router's Decision. Intent-by-intent:
//
// - IntentFact: WriteFact via CoreAPI. Source = "tap:voice" (a voice
// capture is a tap; confidence 1.0).
// - IntentReminder: CreateReminder via CoreAPI.
// - IntentAct: tool-executor deferred (no-op today; the reply says so).
// - IntentNote / IntentQuery: chroma/RAG deferred (no-op; reply says so).
// - Clarify: the router's stage-3 fired; no action.
//
// Returns "" when the Replier should phrase the reply (the default path);
// returns a non-empty string when the action path wants to OVERRIDE the
// reply text (e.g. an action error the user should hear SPECIFICALLY, not
// a generic "ok"). Errors surface as a short reply text the user hears.
func (h *reactiveHandler) applyAction(ctx context.Context, dec router.Decision) string {
if dec.Clarify {
return "" // the Replier phrases clarify
}
if handler, ok := actionHandlers[dec.Intent]; ok {
return handler(h, ctx, dec)
}
return ""
}
// replySystem answers system-observable queries using the handler's clock
// and (in future) system interfaces. The decision's utterance is parsed
// for keywords to determine what the user is asking about.
func (h *reactiveHandler) replySystem(ctx context.Context, dec router.Decision) string {
u := strings.ToLower(dec.Utterance)
now := h.now()
// The topic and the day come from different places on a continuation.
// "а завтра?" names the day and nothing else; what he is asking ABOUT
// lives in the previous turn, which continuation.go copied into
// Slots.Text. Dates keep parsing from the utterance — that is the part
// the ellipsis actually restates — and only the keyword match widens.
//
// Gated on Continued, and that gate is load-bearing. followUpMerge fills
// an empty Text from the previous same-intent turn, so without it a plain
// "привет" after "какой сегодня день" inherited the old topic and got
// answered with the date. Seen on the deployed daemon, 01-08-2026.
topic := u
if dec.Continued {
if t := strings.ToLower(dec.Slots.Text); t != "" && t != u {
topic = u + " " + t
}
}
// stage-0 grammars catch the exact time/date patterns, but duration
// queries ("сколько времени прошло") bypass the grammar's build filter
// and can reach replySystem via the classifier path. Guard against them.
if hasDurationWords(u) {
return "пока не умею отвечать на этот вопрос."
}
switch {
case strings.Contains(topic, "час") || strings.Contains(topic, "врем"):
// "который час в киеве" — she keeps one clock, so any named place gets
// the honest answer. Never local time dressed up as the city's.
if mentionsUnknownPlace(u) {
return onlyLocalTimeReply
}
return "сейчас " + ruClock(now)
case strings.Contains(topic, "день") || strings.Contains(topic, "числ"):
// "какое число завтра" — answer for the day the user asked about,
// not today. Reuses the router's calendar day-word parser.
day := now
prefix := "сегодня"
if d, ok := router.ParseCalendarDate(u, now); ok {
day = d
prefix = dayPrefix(now, d)
} else if mentionsUnknownDay(u) {
// He named a day she cannot work out ("в пятницу", "через неделю").
// Answering today's date here would be the same silent wrong answer
// this arm was fixed for, so say what she can do instead.
return onlyNearDaysReply
}
dow := lexicon.Weekday(int(day.Weekday()))
month := lexicon.MonthGenitive(int(day.Month()))
return fmt.Sprintf("%s %s, %d %s %d года", prefix, dow, day.Day(), month, day.Year())
case strings.Contains(u, "кто дома") || strings.Contains(u, "человек дома"):
return "присутствие пока не подключено к голосовому запросу."
case strings.Contains(u, "памят") || strings.Contains(u, "процессор") || strings.Contains(u, "загрузк") || strings.Contains(u, "статус") || strings.Contains(u, "работа") || strings.Contains(u, "сервис") || strings.Contains(u, "диск") || strings.Contains(u, "ip") || strings.Contains(u, "аптайм") || strings.Contains(u, "трафик") || strings.Contains(u, "интернет"):
return "системная статистика пока не подключена."
default:
return "пока не умею отвечать на этот вопрос."
}
}
// chatHistory collects dialogue turns from the session store for the current
// conversation. Returns prior user utterances (newest last) up to a depth of
// 4 turns. Returns nil when there's no session or no history.
func (h *reactiveHandler) chatHistory() []dialogue.Turn {
if h.dialogueSessions == nil {
return nil
}
now := h.now()
prev := h.dialogueSessions.Get(voiceDialogueID, now)
if prev == nil {
return nil
}
// History already includes the immediate prior turn (set by the dialogue
// merge at lines 373-395), plus up to 3 more from deeper history.
out := make([]dialogue.Turn, 0, 1+len(prev.History))
out = append(out, dialogue.Turn{
Intent: prev.Intent,
Slots: prev.Slots,
Text: prev.Slots.Text,
})
out = append(out, prev.History...)
return out
}
// reply wraps a text reply through TTS to produce a PushToTalkResp. If TTS
// fails, the response carries an empty audio + the text — the client can
// still display text if it can't play. The routedChannels field is
// reserved for a future "the dispatcher also forwarded to ntfy/telegram"
// reply (today the reactive path doesn't dispatch nudges; that's the loop
// tick's job).
func (h *reactiveHandler) reply(ctx context.Context, text string, _ []string) (voice.PushToTalkResp, error) {
spoken := ttsnorm.Speakable(text)
log.Printf("voice: reply → %q", text)
audioOut, err := h.tts.Synthesize(ctx, spoken)
if err != nil {
log.Printf("voice: tts error: %v", err)
return voice.PushToTalkResp{ReplyText: text, ReplyAudio: audio.Audio{}}, nil
}
return voice.PushToTalkResp{ReplyText: text, ReplyAudio: audioOut}, nil
}