Compare commits
11 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| d773f1f72b | |||
| 80ac7579fb | |||
| bb6cb6d185 | |||
| a9067a5754 | |||
| 787cc56522 | |||
| 86dcd99de2 | |||
| 554181ccbd | |||
| 58635f1a69 | |||
| fd3d063e02 | |||
| ed491e23fc | |||
| 246db4e609 |
@@ -32,7 +32,11 @@ See `docs/rearchitecture.md` for the target architecture, `docs/design.md` for t
|
||||
GPU and the workstation has 16GB of VRAM. So the resident model, STT and TTS become preferred
|
||||
remotes with a floor on homesrv. The workstation is never assumed up. Fall back silently when
|
||||
it would only do the job better. Name the gap when the 1.7B cannot do it at all. The embedder
|
||||
stays on homesrv permanently, because it backs that floor. Read `docs/offload.md` before
|
||||
stays on homesrv permanently, because it backs that floor. It is multilingual-e5-small,
|
||||
quantized and asymmetric — `EmbedQuery` and `EmbedPassage` apply the `query:`/`passage:`
|
||||
prefixes it was trained with, and calling plain `Embed` on a note is a bug. It replaced
|
||||
MiniLM and bought ten points of recall@1 and 2.5× the speed; see
|
||||
`docs/evals/2026-08-04-recall-e5-small.md`. Read `docs/offload.md` before
|
||||
touching a daemon seam or adding a model caller. Vikunja #483 is the umbrella, #484 to #487
|
||||
are the work.
|
||||
|
||||
@@ -73,8 +77,8 @@ Pure-Go packages (`router`, `memory`, `mavweb`, …) run under a plain `go test
|
||||
| `mavweb` | HTTP UI + PWA (`/dash`, `/history`, `/trace`, `/notifications`, `/tools`); WebAuthn auth. Connects to mavend's socket. |
|
||||
| `mavsttd` | Speech-to-text (whisper.cpp, CGO). |
|
||||
| `mavttsd` | Text-to-speech (piper subprocess). |
|
||||
| `mavwaked` | Wake-word / VAD gate. |
|
||||
| `mavenclient` | Voice loop client (mic → stt → core → tts). |
|
||||
| `mavwaked` | Wake-word / VAD gate. **Not on homesrv** — see below. |
|
||||
| `mavenclient` | Voice loop client (mic → stt → core → tts). **Not on homesrv** — see below. |
|
||||
| `mavpoll` | Telegram long-poll reach. |
|
||||
| `mavcaldav` | CalDAV calendar sync. |
|
||||
| `mavmaild` | Mail reader (IMAP, read-only). Holds the IMAP password; core never sees it. |
|
||||
@@ -83,6 +87,15 @@ Daemons are wired socket-to-socket, not linked. `internal/ipc` is the client/ser
|
||||
protocol; the config in `deploy/mavend.json` (with `${VAR}` env expansion from gitignored
|
||||
`deploy/telegram.env`) sets socket paths, model paths, and the phraser/embedder blocks.
|
||||
|
||||
**Seven of the nine run on homesrv. `mavwaked` and `mavenclient` do not, and that is the
|
||||
decision, not an oversight** (Vikunja #463, `docs/plans/17-where-the-voice-loop-runs.md`).
|
||||
homesrv has a microphone — it is a laptop — but it is in the wrong room, so a wake-word
|
||||
daemon there listens to nobody. They belong on a client machine where he is standing.
|
||||
`ipc.Dial` already takes `tcp://host:port?token=...` through the netaddr seam, so nothing
|
||||
needs building to allow it, but no such machine exists yet. **The consequence: the wake
|
||||
word and the VAD gate are covered by unit tests and by nothing else, and no amount of
|
||||
sitting at the box changes that.** Push-to-talk through `/dash` is what QA actually covers.
|
||||
|
||||
## The ecosystem: Nexus, Praxis, Hexis
|
||||
|
||||
Maven is one of four services. It owns conversation and personal memory. It does not
|
||||
|
||||
@@ -16,7 +16,7 @@ PIPER_BIN := $(shell pwd)/deps/piper/piper
|
||||
PIPER_MODEL := $(shell pwd)/models/tts/ru_RU-irina-medium.onnx
|
||||
PIPER_ESPEAK := $(shell pwd)/deps/piper/espeak-ng-data
|
||||
|
||||
.PHONY: simulate stt-fixtures test-stt-golden all build build-stt build-tts build-daemon build-client build-waked build-web build-poll build-caldav clean test fmt-check vet run-stt run-tts run-web download-embedder deps-go eval-router eval-recall eval-phrasing eval-models build-gpud
|
||||
.PHONY: simulate stt-fixtures test-stt-golden all build build-stt build-tts build-daemon build-client build-waked build-web build-poll build-caldav clean test fmt-check vet run-stt run-tts run-web download-embedder deps-go eval-router eval-reach eval-recall eval-phrasing eval-models build-gpud
|
||||
|
||||
all: build
|
||||
|
||||
@@ -119,6 +119,14 @@ MAVEN_ONNX_LIB ?= $(shell pwd)/deps/onnxruntime-linux-x64-1.26.0/lib/libonnxrunt
|
||||
eval-router:
|
||||
MAVEN_ONNX_LIB="$(MAVEN_ONNX_LIB)" $(GO) test -v -count=1 ./internal/router/eval/
|
||||
|
||||
# eval-reach — score the held-out ecosystem reach fixture (internal/router/eval,
|
||||
# ru_ecosystem_v1.json). Answers "does a real Russian utterance actually arrive
|
||||
# at Praxis or Hexis", which routing accuracy alone does not say. Same
|
||||
# MAVEN_ONNX_LIB deal as eval-router; without it only the deterministic hash
|
||||
# ratchet runs. Vikunja #405.
|
||||
eval-reach:
|
||||
MAVEN_ONNX_LIB="$(MAVEN_ONNX_LIB)" $(GO) test -v -count=1 -run 'Reach|Praxis' ./internal/router/eval/
|
||||
|
||||
# eval-recall — score the held-out note-recall fixture (internal/memory/recalleval).
|
||||
# Answers "can she find the note again when it matters": recall@1, recall@3,
|
||||
# false recall and the query_min_score sweep. Same MAVEN_ONNX_LIB deal as
|
||||
@@ -197,7 +205,7 @@ deps-piper:
|
||||
# multilingual-e5-small: an asymmetric retrieval model. It is trained to match
|
||||
# a short question against a longer passage, which is what note recall is.
|
||||
# The quantized file is the one we download, deploy and measure — see
|
||||
# docs/evals/2026-07-31-recall.md.
|
||||
# docs/evals/2026-08-04-recall-e5-small.md for what the swap bought.
|
||||
EMBEDDER_DIR := $(shell pwd)/models/embedder/multilingual-e5-small
|
||||
EMBEDDER_MODEL_URL := https://huggingface.co/Xenova/multilingual-e5-small/resolve/main/onnx/model_quantized.onnx
|
||||
EMBEDDER_TOKENIZER_URL := https://huggingface.co/Xenova/multilingual-e5-small/resolve/main/tokenizer.json
|
||||
|
||||
@@ -73,11 +73,11 @@ func (h *reactiveHandler) actionFact(ctx context.Context, dec router.Decision) s
|
||||
// hears; storing the utterance meant recall answered with his own sentence
|
||||
// rather than the value. The utterance stays alongside as provenance —
|
||||
// readable on /trace, never the answer and never embedded.
|
||||
if h.memStore != nil {
|
||||
if h.recall.memStore != nil {
|
||||
text := store.FactRecallText(dec.Slots.Key, dec.Slots.Value)
|
||||
if vec, err := router.EmbedPassage(ctx, h.embedder, text); err != nil {
|
||||
if vec, err := router.EmbedPassage(ctx, h.recall.embedder, text); err != nil {
|
||||
log.Printf("voice: embed fact for memory: %v", err)
|
||||
} else if err := h.memStore.Insert(ctx, "fact:"+dec.Slots.Key+":"+strconv.FormatInt(now.Unix(), 10), vec, map[string]string{
|
||||
} else if err := h.recall.memStore.Insert(ctx, "fact:"+dec.Slots.Key+":"+strconv.FormatInt(now.Unix(), 10), vec, map[string]string{
|
||||
"source": "voice",
|
||||
"type": "fact",
|
||||
"text": text,
|
||||
|
||||
@@ -20,7 +20,7 @@ func (h *reactiveHandler) actionNote(ctx context.Context, dec router.Decision) s
|
||||
// embed the note text with the same model the classifier uses, persist
|
||||
// via CoreAPI (source=tap:voice). Semantic recall lives in `notes`, not
|
||||
// facts — no predicate reads it (spec's two-memory split).
|
||||
vec, err := router.EmbedPassage(ctx, h.embedder, dec.Utterance)
|
||||
vec, err := router.EmbedPassage(ctx, h.recall.embedder, dec.Utterance)
|
||||
if err != nil {
|
||||
log.Printf("voice: embed note: %v", err)
|
||||
return "не получилось сохранить заметку."
|
||||
@@ -33,8 +33,8 @@ func (h *reactiveHandler) actionNote(ctx context.Context, dec router.Decision) s
|
||||
}
|
||||
// Insert into long-term memory (best-effort, must not fail the note write).
|
||||
// text/ts in the meta make a Search hit self-describing (see bestRecall).
|
||||
if h.memStore != nil {
|
||||
if err := h.memStore.Insert(ctx, "note:"+strconv.FormatInt(noteID, 10), vec, map[string]string{
|
||||
if h.recall.memStore != nil {
|
||||
if err := h.recall.memStore.Insert(ctx, "note:"+strconv.FormatInt(noteID, 10), vec, map[string]string{
|
||||
"source": "voice",
|
||||
"type": "note",
|
||||
"text": dec.Utterance,
|
||||
|
||||
@@ -406,7 +406,7 @@ func (h *reactiveHandler) queryWeather(ctx context.Context, t *queryTurn) (strin
|
||||
// sources below both need, run once, in the position it always ran in. It
|
||||
// only claims the turn when the embedder fails.
|
||||
func (h *reactiveHandler) queryEmbed(ctx context.Context, t *queryTurn) (string, bool) {
|
||||
vec, err := router.EmbedQuery(ctx, h.embedder, t.dec.Utterance)
|
||||
vec, err := router.EmbedQuery(ctx, h.recall.embedder, t.dec.Utterance)
|
||||
if err != nil {
|
||||
log.Printf("voice: embed query: %v", err)
|
||||
return "не получилось найти ответ.", true
|
||||
@@ -426,15 +426,15 @@ func (h *reactiveHandler) queryEmbed(ctx context.Context, t *queryTurn) (string,
|
||||
// gate, was the bug — the set of questions Maven answers is unchanged, only
|
||||
// which memory gets to answer them.
|
||||
func (h *reactiveHandler) queryMemory(ctx context.Context, t *queryTurn) (string, bool) {
|
||||
if h.memStore == nil {
|
||||
if h.recall.memStore == nil {
|
||||
return "", false
|
||||
}
|
||||
hits, herr := h.memStore.Search(ctx, t.vec, 3)
|
||||
hits, herr := h.recall.memStore.Search(ctx, t.vec, 3)
|
||||
if herr != nil {
|
||||
log.Printf("voice: memory search: %v", herr)
|
||||
return "", false
|
||||
}
|
||||
hit, ok := bestRecall(hits, h.queryMinScore, h.queryMinMargin)
|
||||
hit, ok := bestRecall(hits, h.recall.minScore, h.recall.minMargin)
|
||||
if !ok {
|
||||
return "", false
|
||||
}
|
||||
@@ -479,7 +479,7 @@ func (h *reactiveHandler) queryNotes(ctx context.Context, t *queryTurn) (string,
|
||||
for i, n := range notes {
|
||||
noteScores[i] = n.Score
|
||||
}
|
||||
if !memory.ConfidentScores(noteScores, h.queryMinScore, h.queryMinMargin) {
|
||||
if !memory.ConfidentScores(noteScores, h.recall.minScore, h.recall.minMargin) {
|
||||
return "", false
|
||||
}
|
||||
// Same topic veto as queryMemory above: the best note must be about what
|
||||
@@ -766,8 +766,8 @@ func isPersonalQuery(utterance string) bool {
|
||||
// computed. Same shape as the cascade: the better test leads, the offline one
|
||||
// always answers.
|
||||
func (h *reactiveHandler) isPersonalTurn(ctx context.Context, t *queryTurn) bool {
|
||||
h.boundary.load(ctx, h.embedder)
|
||||
if personal, world, ok := h.boundary.score(t.vec); ok {
|
||||
h.recall.boundary.load(ctx, h.recall.embedder)
|
||||
if personal, world, ok := h.recall.boundary.score(t.vec); ok {
|
||||
if personal > world {
|
||||
log.Printf("voice: %q scores personal %.4f vs world %.4f", t.dec.Utterance, personal, world)
|
||||
return true
|
||||
|
||||
@@ -299,7 +299,7 @@ func TestClarifyExpiryIsAnnouncedAndWordsStillRoute(t *testing.T) {
|
||||
ctx := withDialogueID(context.Background(), dialogueIDFor(sourceText, ""))
|
||||
h, _, now := newClarifyHandler(t)
|
||||
emb := router.NewHashEmbedder(1024)
|
||||
h.embedder = emb
|
||||
h.recall.embedder = emb
|
||||
h.router = buildRouter(emb, h.matcher, 0.55, nil)
|
||||
|
||||
if _, asked := h.askClarify(ctx, clarifyDec(router.IntentReminder, router.Slots{Text: "напомни"}, "напомни")); !asked {
|
||||
|
||||
@@ -28,11 +28,10 @@ func TestApplyAction_FactCapture_QueuesEntityResolution(t *testing.T) {
|
||||
|
||||
h := &reactiveHandler{
|
||||
api: api,
|
||||
embedder: emb,
|
||||
recall: recallWiring{embedder: emb, memStore: memory.NewInMemoryStore()},
|
||||
router: rtr,
|
||||
replier: voice.NewStubReplier(),
|
||||
now: func() time.Time { return now },
|
||||
memStore: memory.NewInMemoryStore(),
|
||||
dataStore: st,
|
||||
}
|
||||
|
||||
|
||||
@@ -19,11 +19,10 @@ func newFactGateHandler(t *testing.T, now time.Time) (*reactiveHandler, ipc.Core
|
||||
emb := router.NewHashEmbedder(1024)
|
||||
h := &reactiveHandler{
|
||||
api: api,
|
||||
embedder: emb,
|
||||
recall: recallWiring{embedder: emb, memStore: memory.NewInMemoryStore()},
|
||||
router: buildRouter(emb, tool.NewMatcher(api), 0.55, nil),
|
||||
replier: voice.NewStubReplier(),
|
||||
now: func() time.Time { return now },
|
||||
memStore: memory.NewInMemoryStore(),
|
||||
dataStore: st,
|
||||
}
|
||||
return h, api
|
||||
@@ -44,7 +43,7 @@ func TestActionFact_QuestionIsNotWritten(t *testing.T) {
|
||||
if _, err := api.LatestFact(ctx, "go_version"); err == nil {
|
||||
t.Fatal("a question was stored as a fact about him")
|
||||
}
|
||||
hits, err := h.memStore.Search(ctx, mustEmbedPassage(t, h, "какая последняя версия языка Go?"), 3)
|
||||
hits, err := h.recall.memStore.Search(ctx, mustEmbedPassage(t, h, "какая последняя версия языка Go?"), 3)
|
||||
if err != nil {
|
||||
t.Fatalf("memory search: %v", err)
|
||||
}
|
||||
@@ -81,7 +80,7 @@ func TestActionFact_ExplicitCaptureStillWrites(t *testing.T) {
|
||||
// #493: what recall reads back is the fact, not the sentence he said.
|
||||
// queryMemory returns a fact's text verbatim, so the utterance sitting here
|
||||
// meant "запиши что я пил воду" was the answer to "когда я пил воду?".
|
||||
hits, err := h.memStore.Search(ctx, mustEmbedPassage(t, h, "вода"), 3)
|
||||
hits, err := h.recall.memStore.Search(ctx, mustEmbedPassage(t, h, "вода"), 3)
|
||||
if err != nil {
|
||||
t.Fatalf("memory search: %v", err)
|
||||
}
|
||||
@@ -117,7 +116,7 @@ func TestFactConfidence(t *testing.T) {
|
||||
|
||||
func mustEmbedPassage(t *testing.T, h *reactiveHandler, text string) []float32 {
|
||||
t.Helper()
|
||||
vec, err := router.EmbedQuery(context.Background(), h.embedder, text)
|
||||
vec, err := router.EmbedQuery(context.Background(), h.recall.embedder, text)
|
||||
if err != nil {
|
||||
t.Fatalf("embed %q: %v", text, err)
|
||||
}
|
||||
|
||||
@@ -29,12 +29,12 @@ func buildFeedHandler(t *testing.T, feedsOn bool, notes ...ipc.Note) *reactiveHa
|
||||
}
|
||||
}
|
||||
return &reactiveHandler{
|
||||
api: ipc.NewStoreAPI(st),
|
||||
replier: voice.NewStubReplier(),
|
||||
phraser: phraser.NewStub(),
|
||||
now: func() time.Time { return now },
|
||||
feedsOn: feedsOn,
|
||||
embedder: nil,
|
||||
api: ipc.NewStoreAPI(st),
|
||||
replier: voice.NewStubReplier(),
|
||||
phraser: phraser.NewStub(),
|
||||
now: func() time.Time { return now },
|
||||
feedsOn: feedsOn,
|
||||
recall: recallWiring{embedder: nil},
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -70,7 +70,7 @@ func TestONNXPersonalBoundary(t *testing.T) {
|
||||
{"how do i boil an egg", false},
|
||||
}
|
||||
|
||||
h := &reactiveHandler{embedder: emb}
|
||||
h := &reactiveHandler{recall: recallWiring{embedder: emb}}
|
||||
ctx := context.Background()
|
||||
wrong := 0
|
||||
for _, c := range cases {
|
||||
@@ -80,7 +80,7 @@ func TestONNXPersonalBoundary(t *testing.T) {
|
||||
}
|
||||
turn := &queryTurn{dec: router.Decision{Utterance: c.utterance}, vec: vec}
|
||||
got := h.isPersonalTurn(ctx, turn)
|
||||
p, w, ok := h.boundary.score(vec)
|
||||
p, w, ok := h.recall.boundary.score(vec)
|
||||
if !ok {
|
||||
t.Fatal("seeds did not load with a working embedder")
|
||||
}
|
||||
|
||||
@@ -85,15 +85,17 @@ func buildRecallHandler(t *testing.T, question string, mems []recallCase) (*reac
|
||||
|
||||
phr := &recordingPhraser{Stub: phraser.NewStub()}
|
||||
h := &reactiveHandler{
|
||||
api: ipc.NewStoreAPI(st),
|
||||
embedder: emb,
|
||||
api: ipc.NewStoreAPI(st),
|
||||
recall: recallWiring{
|
||||
embedder: emb,
|
||||
memStore: mem,
|
||||
minScore: 0.55,
|
||||
minMargin: 0.008,
|
||||
},
|
||||
replier: voice.NewStubReplier(),
|
||||
phraser: phr,
|
||||
now: func() time.Time { return now },
|
||||
memStore: mem,
|
||||
dataStore: st,
|
||||
queryMinScore: 0.55,
|
||||
queryMinMargin: 0.008,
|
||||
weatherProvider: nil,
|
||||
}
|
||||
return h, phr
|
||||
|
||||
@@ -26,11 +26,10 @@ func TestReactiveNotesReminders(t *testing.T) {
|
||||
|
||||
h := &reactiveHandler{
|
||||
api: api,
|
||||
embedder: emb,
|
||||
recall: recallWiring{embedder: emb, memStore: memory.NewInMemoryStore()},
|
||||
router: rtr,
|
||||
replier: voice.NewStubReplier(),
|
||||
now: func() time.Time { return now },
|
||||
memStore: memory.NewInMemoryStore(),
|
||||
dataStore: st,
|
||||
}
|
||||
|
||||
@@ -101,11 +100,10 @@ func TestSpokenTaskCaptureFilesATask(t *testing.T) {
|
||||
matcher := tool.NewMatcher(api)
|
||||
h := &reactiveHandler{
|
||||
api: api,
|
||||
embedder: emb,
|
||||
recall: recallWiring{embedder: emb, memStore: memory.NewInMemoryStore()},
|
||||
router: buildRouter(emb, matcher, 0.55, nil),
|
||||
replier: voice.NewStubReplier(),
|
||||
now: func() time.Time { return now },
|
||||
memStore: memory.NewInMemoryStore(),
|
||||
dataStore: st,
|
||||
}
|
||||
|
||||
|
||||
+35
-1
@@ -1,6 +1,9 @@
|
||||
package main
|
||||
|
||||
import "github.com/kami/maven/internal/memory"
|
||||
import (
|
||||
"github.com/kami/maven/internal/memory"
|
||||
"github.com/kami/maven/internal/router"
|
||||
)
|
||||
|
||||
// bestRecall is the read side of the long-term memory store: the top hit when
|
||||
// it clears the confidence gate. The index holds BOTH notes and facts, and
|
||||
@@ -23,3 +26,34 @@ func bestRecall(results []memory.Result, minScore, minMargin float64) (memory.Re
|
||||
}
|
||||
return results[0], true
|
||||
}
|
||||
|
||||
// recallWiring — the recall subsystem's dependencies, held as one group on
|
||||
// reactiveHandler (Vikunja #433). It is the worked example for the wiring
|
||||
// decision in docs/handler-wiring.md: cohesive groups of fields, not thirty
|
||||
// loose ones, so a handler names what it needs and the package can be split
|
||||
// later without exporting the whole struct.
|
||||
//
|
||||
// The zero value is usable and means "no recall": no embedder, no vector
|
||||
// store, and a gate that is never consulted because nothing is ever searched.
|
||||
type recallWiring struct {
|
||||
// embedder — reused for note write/query (same model as the classifier).
|
||||
embedder router.Embedder
|
||||
|
||||
// memStore — the vector index over notes and facts.
|
||||
memStore memory.Store
|
||||
|
||||
// boundary — the embedded seed sets behind the personal boundary
|
||||
// (personalboundary.go). Zero value is usable and loads on first query;
|
||||
// with no embedder it never loads and the boundary uses personalMarkers.
|
||||
boundary personalBoundary
|
||||
|
||||
// minScore — the note-recall confidence gate. Top cosine below this ⇒
|
||||
// "I don't know" instead of a guess. Tuned for the ONNX embedder; a knob,
|
||||
// not load-bearing math (same posture as the presence thresholds). Set by
|
||||
// wireVoice from VoiceConfig; default 0.55.
|
||||
minScore float64
|
||||
|
||||
// minMargin — the second half of that gate: how far the top hit must beat
|
||||
// the runner-up. 0 ⇒ margin off.
|
||||
minMargin float64
|
||||
}
|
||||
|
||||
@@ -428,20 +428,22 @@ func newSimWorld(t *testing.T, sc scenario) *simWorld {
|
||||
rtr := buildRouter(emb, matcher, config.DefaultRouterThreshold, router.NewLLMRouter(scripted))
|
||||
|
||||
w.handler = &reactiveHandler{
|
||||
stt: simTranscriber{},
|
||||
tts: simSynthesizer{},
|
||||
router: rtr,
|
||||
embedder: emb,
|
||||
stt: simTranscriber{},
|
||||
tts: simSynthesizer{},
|
||||
router: rtr,
|
||||
recall: recallWiring{
|
||||
embedder: emb,
|
||||
memStore: st.VectorMemory(),
|
||||
minScore: config.DefaultQueryMinScore,
|
||||
minMargin: config.DefaultQueryMinMargin,
|
||||
},
|
||||
api: api,
|
||||
matcher: matcher,
|
||||
tools: tool.NewExecutor(api, 5*time.Second),
|
||||
phraser: phraser.NewStub(),
|
||||
replier: newLLMReplier(scripted, nil),
|
||||
now: clock.Now,
|
||||
memStore: st.VectorMemory(),
|
||||
dataStore: st,
|
||||
queryMinScore: config.DefaultQueryMinScore,
|
||||
queryMinMargin: config.DefaultQueryMinMargin,
|
||||
timeParser: router.StubDateTimeParser{},
|
||||
dialogueSessions: dialogue.NewSessionStore(time.Hour),
|
||||
clarifyStore: dialogue.NewClarifyStore(time.Hour),
|
||||
|
||||
+10
-19
@@ -55,7 +55,6 @@ import (
|
||||
"github.com/kami/maven/internal/crawl"
|
||||
"github.com/kami/maven/internal/dialogue"
|
||||
"github.com/kami/maven/internal/ipc"
|
||||
"github.com/kami/maven/internal/memory"
|
||||
"github.com/kami/maven/internal/phraser"
|
||||
"github.com/kami/maven/internal/router"
|
||||
"github.com/kami/maven/internal/store"
|
||||
@@ -72,14 +71,16 @@ import (
|
||||
// safe (the wired stt/tts/router/api all are); called from per-conn
|
||||
// goroutines on the voice.Server.
|
||||
type reactiveHandler struct {
|
||||
stt stt.Transcriber
|
||||
tts tts.Synthesizer
|
||||
router *router.Router
|
||||
embedder router.Embedder // reused for note write/query (same model as the classifier)
|
||||
// boundary — the embedded seed sets behind the personal boundary
|
||||
// (personalboundary.go). Zero value is usable and loads on first query;
|
||||
// with no embedder it never loads and the boundary uses personalMarkers.
|
||||
boundary personalBoundary
|
||||
stt stt.Transcriber
|
||||
tts tts.Synthesizer
|
||||
router *router.Router
|
||||
|
||||
// recall — the note-and-fact recall subsystem: the embedder, the vector
|
||||
// store it writes into, the personal boundary, and the two numbers that
|
||||
// gate an answer. Grouped rather than spread across the handler because a
|
||||
// handler that recalls needs all five and a handler that does not needs
|
||||
// none of them (Vikunja #433, docs/handler-wiring.md).
|
||||
recall recallWiring
|
||||
// api — the CoreAPI the handler reads and writes through. Wired with the
|
||||
// bare store adapter and UPGRADED by main once the daemonAPI exists; see
|
||||
// upgradeAPI.
|
||||
@@ -123,18 +124,8 @@ type reactiveHandler struct {
|
||||
weatherProvider weather.Provider
|
||||
weatherLocation string // default location for weather queries
|
||||
|
||||
memStore memory.Store
|
||||
dataStore *store.Store // direct store access for event extraction + pattern detection
|
||||
|
||||
// queryMinScore — the note-recall confidence gate. Top cosine below this ⇒
|
||||
// "I don't know" instead of a guess. Tuned for the ONNX embedder; a knob, not
|
||||
// load-bearing math (same posture as the presence thresholds). Set by
|
||||
// wireVoice from VoiceConfig; default 0.55.
|
||||
queryMinScore float64
|
||||
// queryMinMargin — the second half of that gate: how far the top hit must
|
||||
// beat the runner-up. 0 ⇒ margin off.
|
||||
queryMinMargin float64
|
||||
|
||||
// timeParser — used as a fallback for stage-0 reminder grammar matches
|
||||
// (where the extractor didn't run). Shared with the router's extractor.
|
||||
// The production dateparser will replace StubDateTimeParser here too.
|
||||
|
||||
+21
-19
@@ -262,19 +262,18 @@ func wireVoice(cfg *config.Config, coreAPI ipc.CoreAPI, phr phraser.Phraser, mem
|
||||
|
||||
// ----- the handler (the reactive path; closes over stt / tts / router / coreAPI / memory) -----
|
||||
h := &reactiveHandler{
|
||||
stt: transcriber,
|
||||
tts: synthesizer,
|
||||
router: rtr,
|
||||
embedder: emb,
|
||||
api: coreAPI,
|
||||
tools: exec,
|
||||
matcher: matcher,
|
||||
replier: replier,
|
||||
phraser: phr,
|
||||
now: time.Now,
|
||||
feedsOn: cfg.Feeds != nil,
|
||||
home: w.home,
|
||||
netscan: w.netscan,
|
||||
stt: transcriber,
|
||||
tts: synthesizer,
|
||||
router: rtr,
|
||||
api: coreAPI,
|
||||
tools: exec,
|
||||
matcher: matcher,
|
||||
replier: replier,
|
||||
phraser: phr,
|
||||
now: time.Now,
|
||||
feedsOn: cfg.Feeds != nil,
|
||||
home: w.home,
|
||||
netscan: w.netscan,
|
||||
// nil unless `crawl.on_demand` is on: reading a page he names is a
|
||||
// capability, and capabilities are off unless configured.
|
||||
crawler: onDemandCrawler(cfg),
|
||||
@@ -283,18 +282,21 @@ func wireVoice(cfg *config.Config, coreAPI ipc.CoreAPI, phr phraser.Phraser, mem
|
||||
search: wireSearch(cfg),
|
||||
// nil unless a `kiwix` block names a server. Same swap-aware client the
|
||||
// router and replier use, so the rewriter follows a model swap.
|
||||
kiwix: wireKiwix(cfg, llmClient),
|
||||
weatherProvider: weatherProvider,
|
||||
weatherLocation: weatherLocation,
|
||||
memStore: memStore,
|
||||
kiwix: wireKiwix(cfg, llmClient),
|
||||
weatherProvider: weatherProvider,
|
||||
weatherLocation: weatherLocation,
|
||||
recall: recallWiring{
|
||||
embedder: emb,
|
||||
memStore: memStore,
|
||||
minScore: cfg.Voice.QueryMinScore,
|
||||
minMargin: cfg.Voice.QueryMinMargin,
|
||||
},
|
||||
dataStore: dataStore,
|
||||
dialogueSessions: dialogueSessions,
|
||||
clarifyStore: clarifyStore,
|
||||
// 0 here (unset config) ⇒ the dialogue default.
|
||||
clarifyMaxAttempts: cfg.Voice.ClarifyMaxAttempts,
|
||||
extractor: router.Extractor{Time: timeParser, Acts: matcher, Facts: router.DefaultFactParser{}},
|
||||
queryMinScore: cfg.Voice.QueryMinScore,
|
||||
queryMinMargin: cfg.Voice.QueryMinMargin,
|
||||
timeParser: timeParser,
|
||||
ecosystem: eco,
|
||||
}
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
# Ecosystem reach, measured — 2026-08-04
|
||||
|
||||
Vikunja #405. Measured at `86dcd99` on the 30-case held-out fixture
|
||||
`internal/router/eval/ru_ecosystem_v1.json`, scored by `make eval-reach`.
|
||||
|
||||
**Praxis reach is zero. Not low — zero, on all twelve cases, under both embedders.**
|
||||
|
||||
## What was measured
|
||||
|
||||
Reach is where an utterance *arrives*, not what it achieves. The derivation is in
|
||||
`internal/router/eval/reach.go` and mirrors `actionAct` in `cmd/mavend/actions_act.go`:
|
||||
|
||||
- **Praxis** needs `IntentAct` and a fn slot whose value is one of the aliases in
|
||||
`praxisCapabilities`.
|
||||
- **Hexis** needs `IntentAct` and non-empty `Slots.Text`. It also fires from
|
||||
`hexisBeforeClarify`, so a clarified act with text reaches Hexis before the clarify
|
||||
question is ever asked.
|
||||
- Everything else stays inside Maven.
|
||||
|
||||
The fixture holds 10 Hexis cases, 12 Praxis cases and 8 negatives. The negatives carry
|
||||
the expensive direction: an utterance that reaches a mutating path it had no business
|
||||
reaching is worse than one that never arrives, because he never gets asked about it.
|
||||
|
||||
## The numbers
|
||||
|
||||
| Path | Reached the right place | Missed | Overreach | Wrong service | p50 |
|
||||
|---|---|---|---|---|---|
|
||||
| classifier + hash embedder | 7/30 (23.3%) | 22 | 1 | 0 | 14.8µs |
|
||||
| classifier + e5-small (deployed) | 16/30 (53.3%) | 11 | 1 | 2 | 23.8ms |
|
||||
|
||||
Split by target, on the deployed embedder:
|
||||
|
||||
| Want | Passed |
|
||||
|---|---|
|
||||
| hexis | 9/10 |
|
||||
| none | 7/8 |
|
||||
| **praxis** | **0/12** |
|
||||
|
||||
## The finding
|
||||
|
||||
Hexis reach is fine. Nine of ten act-shaped home and homelab utterances land on the
|
||||
entity resolver, which is what the gate was built to do: it only needs the intent and
|
||||
some text, and both the act grammar and the embedder produce those.
|
||||
|
||||
Praxis reach is structurally impossible from free Russian, and the fixture makes that
|
||||
visible for the first time. `handlePraxisAct` dispatches on exact equality between
|
||||
`Slots.Fn` and a capability alias. The fn slot is filled by `DefaultActMatcher`, whose
|
||||
allowlist is the deployment's enabled tool names — `перезапусти`, `выключи`, and so on.
|
||||
No Praxis alias is in that list, so no utterance can ever put one in the slot. The
|
||||
Russian aliases in `praxisCapabilities` (`готово`, `принято`, `игнорировать`) read as if
|
||||
they match speech and they do not: they are compared against a fn slot, never against
|
||||
the utterance.
|
||||
|
||||
That means the whole lifecycle half of the Praxis contract — acknowledge, resolve,
|
||||
ignore, pin — has no voice path at all. Attention and changes have none either.
|
||||
|
||||
Three of the twelve got as far as the wrong place, which is the same defect seen from the
|
||||
other side: `"готово, закрывай"` and `"как дела у праксиса"` route to act with text, so
|
||||
they fall past the Praxis check into the Hexis one and go to entity resolution instead.
|
||||
|
||||
## The live services
|
||||
|
||||
All three are up and answer. With `no_proxy` set for the loopback (the host's `http_proxy`
|
||||
answers 503 for 127.0.0.1, which is the same trap `llmrouter_test.go` documents):
|
||||
|
||||
```
|
||||
127.0.0.1:8989/health -> {"status":"ok"} praxis
|
||||
127.0.0.1:9740/health -> {"status":"ok"} nexus
|
||||
127.0.0.1:9741/health -> {"status":"ok"} hexis
|
||||
127.0.0.1:8989/api/v1/tools/attention?limit=3 -> []
|
||||
127.0.0.1:8989/api/v1/tools/changes?limit=3 -> []
|
||||
```
|
||||
|
||||
So the boundary is not the problem, and an end-to-end run today would add nothing: both
|
||||
Praxis feeds are empty, so even a perfect reach score would produce "ничего не требует
|
||||
внимания". The gap is entirely on Maven's side of the wire.
|
||||
|
||||
## Not measured
|
||||
|
||||
**The resident model.** The LLM router is the deployed default, and these numbers are the
|
||||
classifier only. `mavend` spawns its llama-server on a container-local port
|
||||
(127.0.0.1:40063 inside `maven-mavend-1`), unreachable from the host, and the workstation
|
||||
at 192.168.1.105:8080 was down. The classifier is the failure floor and it is what always
|
||||
answers, so the floor is worth knowing on its own — but the LLM router could fill the fn
|
||||
slot with a literal `list_attention`, since the grammar lets it emit any string. Whether
|
||||
it does is the open question, and the fixture is ready for it.
|
||||
|
||||
## What this argues for
|
||||
|
||||
Not a new intent. The seven are frozen by prompt parity with the training workspace.
|
||||
|
||||
The cheap fix is stage 0: a grammar per Praxis capability that sets `Slots.Fn` to the
|
||||
canonical arm name, the same trick `AgendaQueryGrammars` used to take agenda questions off
|
||||
the model. It costs one regex per capability on every turn and it is deterministic, which
|
||||
for a lifecycle verb is the right trade — "отметь это как сделанное" should never be a
|
||||
similarity guess.
|
||||
|
||||
The second fix is smaller and separate: `entity_attention` aliases to grammar names only,
|
||||
so scoped attention ("что там с нексусом") needs a grammar before it can be reached at
|
||||
all.
|
||||
@@ -0,0 +1,88 @@
|
||||
# Note recall after the e5-small swap — 04-08-2026
|
||||
|
||||
Closes Vikunja #371, which asked for the swap and for this re-measurement. The embedder is no
|
||||
longer paraphrase-multilingual-MiniLM-L12-v2. It is **multilingual-e5-small**, quantized, with the
|
||||
`query:` / `passage:` prefixes it was trained with (`internal/router/onnxembedder.go`,
|
||||
`EmbedQuery` / `EmbedPassage`). `deploy/mavend.json` loads
|
||||
`models/embedder/multilingual-e5-small/model_quantized.onnx`, which is the same file `make
|
||||
download-embedder` fetches and the same file this run measured.
|
||||
|
||||
- Fixture + scorer: `internal/memory/recalleval/` — 32 cases now, not 30
|
||||
- Reproduce: `make eval-recall`
|
||||
- Commit: `b6abb19`
|
||||
- Gate as deployed: `query_min_score` 0.55, `query_min_margin` 0.008
|
||||
|
||||
The fixture grew since 31-07, so the case counts are not comparable row for row. The percentages
|
||||
are.
|
||||
|
||||
## Results
|
||||
|
||||
| | 31-07 MiniLM (onnx) | 04-08 e5-small (onnx) |
|
||||
|---|---|---|
|
||||
| **recall@1** | 60.0% (15/25) | **70.4% (19/27)** |
|
||||
| recall@3 | 80.0% (20/25) | **85.2% (23/27)** |
|
||||
| **answered after the gate** | 48.0% (12/25) | **63.0% (17/27)** |
|
||||
| **false recall** | 1/5 (20%) | **0/5** |
|
||||
| wrong note on top / tie on top | 10 / 0 | 8 / 0 |
|
||||
| ranked first, then silenced by the gate | 3 | 2 |
|
||||
| `hard` cases passed | 2/11 | 5/12 |
|
||||
| RU / EN passed | 13/24 / 3/6 | 18/26 / 4/6 |
|
||||
| latency p50 / p95 / max | 59ms / 148ms / 194ms | **23ms / 41ms / 62ms** |
|
||||
|
||||
The hash ratchet CI runs is unchanged in kind and still answers nothing after the gate: recall@1
|
||||
37.0%, recall@3 74.1%, 0/27 answered, 0/5 false. It is lexical and exists so CI has a deterministic
|
||||
floor. Never compare a hash number to an ONNX one.
|
||||
|
||||
## Findings
|
||||
|
||||
### 1. The swap paid on every axis at once, including latency
|
||||
|
||||
Ten points of recall@1, fifteen points of *answered*, the one false recall gone, and it is 2.5×
|
||||
faster because the quantized e5-small is 118MB against the 470MB fp32 file the old config loaded.
|
||||
Finding 3 of the 31-07 eval predicted the recall half and said nothing about speed; the speed came
|
||||
from fixing the second half of that finding, which was that the deployed path loaded a different
|
||||
file than the download target.
|
||||
|
||||
The concrete case that eval named is fixed. "из-за чего кончилось место" no longer returns the
|
||||
guitar-chords filler note. It now returns a homelab note, `n2` at 0.884, and the wanted note is
|
||||
still not in the top 3 — so the query moved from absurd to merely wrong. That is the shape of what
|
||||
is left.
|
||||
|
||||
### 2. The score distributions still overlap. The margin is what separates them
|
||||
|
||||
This is the part of #371's premise that did not come true. Right-note-first top-1 scores run
|
||||
0.791 / 0.857 / 0.890 (min / median / max). Must-stay-silent top-1 scores run 0.795 / 0.815 /
|
||||
0.835. The silent cases sit *inside* the answering range, so no value of `query_min_score` keeps
|
||||
every real recall and rejects every false one — the same verdict as 31-07, at a higher and tighter
|
||||
band of scores.
|
||||
|
||||
What separates them is the second-place gap. Margin top1-top2 for a right first hit: median 0.024.
|
||||
For a must-be-silent case: median 0.002, max 0.019. A false recall is a note that beats its
|
||||
neighbours by nothing, because nothing in the store is about the question. The sweep:
|
||||
|
||||
| margin | answered | false recall |
|
||||
|---|---|---|
|
||||
| 0.000 | 18/27 (67%) | 3/5 |
|
||||
| 0.005 | 17/27 (63%) | 1/5 |
|
||||
| **0.008 (deployed)** | **17/27 (63%)** | **0/5** |
|
||||
| 0.010 | 15/27 (56%) | 0/5 |
|
||||
| 0.015 | 12/27 (44%) | 0/5 |
|
||||
|
||||
0.008 is the knee: it is the smallest margin that silences all five, and the next step up costs two
|
||||
real answers for nothing. The score gate contributes almost nothing on its own — every value from
|
||||
0.00 to 0.70 answers the same 18 and admits the same 3 — so `query_min_score` is now close to inert
|
||||
and the margin is the live dial. Leave both where they are; #412 is where a further sweep belongs.
|
||||
|
||||
### 3. What is left is a retrieval problem, not a gate problem
|
||||
|
||||
Eight cases put the wrong note on top, and the failures cluster: `hard` 5/12, `preference` 5/9,
|
||||
`homelab` 8/13. Four of the eight have the right note in the top 3, so a reranker would collect
|
||||
them; the other four do not, so nothing downstream can. Two more rank first and are silenced by the
|
||||
margin — `en-hard-024` at 0.826 with margin 0.023, and `ru-home-026` at 0.846 with margin 0.001,
|
||||
which is a genuine near-tie against a second note that is also plausible.
|
||||
|
||||
Preference queries are the weakest class in a way that is not about the model. "когда запускать
|
||||
резервное копирование" and "как мне присылать оповещения" both return a fact, not the note that
|
||||
states the preference. Facts and notes are searched in one pass since #373, so a confidently-scored
|
||||
fact wins a question that a note answers better. That is a ranking policy question and it belongs
|
||||
in its own task, not in a threshold.
|
||||
@@ -0,0 +1,62 @@
|
||||
# How reactiveHandler is wired
|
||||
|
||||
*Last verified: 2026-08-04 @ b6abb19. Living doc: correct it in place, do not append.*
|
||||
|
||||
The decision Vikunja #433 asked for, and the rule that follows from it.
|
||||
|
||||
## The decision
|
||||
|
||||
**Group the fields into cohesive wiring structs. No container, no generator, no
|
||||
framework.** `reactiveHandler` (`cmd/mavend/voice.go`) stays the one type the voice
|
||||
server talks to. What changes is that a capability arrives as one named group, not as
|
||||
four more loose fields on a struct that already had thirty.
|
||||
|
||||
The pattern was already in the file before this was written down: `searchWiring`,
|
||||
`kiwixWiring`, `homeWiring`, `netWiring` and `ecosystemWiring` are all this shape, each
|
||||
`nil` when the capability is off. #433 makes it the rule rather than a habit, and adds
|
||||
the case the habit had missed — a group that is not a capability toggle.
|
||||
|
||||
## The worked example
|
||||
|
||||
`recallWiring` (`cmd/mavend/recall.go`) holds the five things the recall path needs:
|
||||
the embedder, the vector store, the personal boundary, and the score and margin that
|
||||
gate an answer. They used to sit in three separate places on the handler with the two
|
||||
gate numbers a hundred lines away from the store they gate.
|
||||
|
||||
Its zero value means "no recall", which is why it is a value and not a pointer. The
|
||||
`*Wiring` types that model an optional capability stay pointers, because `nil` is how
|
||||
"not configured" is spelled and a zero-valued search client would be a client pointed at
|
||||
nothing.
|
||||
|
||||
## Why not the alternatives
|
||||
|
||||
**Narrow consumer-side interfaces at each handler** is the more idiomatic Go answer and
|
||||
it is not rejected, only deferred. It is the right move at the point a handler is pulled
|
||||
into its own package, because that is when the import direction starts to matter. Doing
|
||||
it first would mean writing an interface per handler against a struct nobody can pass
|
||||
anywhere, which is churn bought against a package split that has not happened.
|
||||
|
||||
**A container or a wire-style generator** is rejected outright. This is one binary with
|
||||
one composition root (`wireVoice` in `cmd/mavend/voicewire.go`). Generated wiring would
|
||||
add a build step and a layer of indirection to solve a problem that is currently one
|
||||
composite literal long, and it would make the "is this capability configured" question
|
||||
harder to answer by reading, which is the question this file is mostly about.
|
||||
|
||||
## What this unlocks
|
||||
|
||||
The reason `cmd/mavend/` cannot split into `mavend/actions/` today is that every action
|
||||
handler is a method on a struct with thirty unexported fields: moving handlers to a
|
||||
subdirectory means exporting all of them or inventing an interface to pass through. That
|
||||
was the answer given on the tick.go and voice.go splits, and it is still true. Grouping
|
||||
is the step that makes it false later — a handler that takes `recallWiring` and nothing
|
||||
else can move without the other twenty-five fields following it.
|
||||
|
||||
## The rule
|
||||
|
||||
**A wiring change does not ride a feature PR.** The voice.go and tick.go splits were
|
||||
safe to merge because the moved code diffed identical, line for line. A regrouping that
|
||||
touches the confirm gate or the act allowlist is its own change, reviewed on its own, or
|
||||
it is not reviewable at all.
|
||||
|
||||
New capability, new group. A capability that adds four fields to `reactiveHandler`
|
||||
instead of one struct is the thing this decision exists to stop.
|
||||
@@ -103,9 +103,14 @@ query path.
|
||||
```
|
||||
|
||||
`Recognizes()` requires both `enabled` and a `model_path`, so a half-filled block reads as off
|
||||
rather than as a capability that fails every turn. With `enabled` and no model the daemon still
|
||||
attaches the three methods — profiles can be created, listed and deleted — and logs that
|
||||
recognition is blocked.
|
||||
rather than as a capability that fails every turn.
|
||||
|
||||
That gate was written, documented, and then never called. It is called now, and the behaviour it
|
||||
describes changed with it. `enabled` with no `model_path` used to attach all three methods and log
|
||||
that enrolment was on. Today `newSpeakerWiring` returns nil, so the methods are absent, and the log
|
||||
says why: there is nothing to embed with, so enrol, list and forget would all be no-ops. That is
|
||||
the one config shape where the operator most needs to be told otherwise, and it was the shape that
|
||||
lied.
|
||||
|
||||
## Still open
|
||||
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
# Plan: The work board surface
|
||||
|
||||
**The decision Vikunja #431 asked for. Written 04-08-2026.**
|
||||
|
||||
**Verdict: build it, in a smaller shape than the task imagined.** The board is worth
|
||||
moving out of the file. The intake form belongs on the `/tasks` page, not on the voice
|
||||
path. The argument is not built, now or later.
|
||||
|
||||
## Why it is worth building
|
||||
|
||||
The reason is the one the task gives and it holds: company rules forbid pointing Claude
|
||||
at work repos, and Maven is the one assistant on the box that work material may reach.
|
||||
No telemetry, no cloud model, no third-party account. That is not a preference here, it
|
||||
is the whole permission.
|
||||
|
||||
The build is also small, because most of it landed already:
|
||||
|
||||
| Piece | Where | State |
|
||||
|---|---|---|
|
||||
| task rows, dedupe, status lifecycle | `internal/store/migrations.go:149` and migration #15 | done |
|
||||
| capture from speech, urgency stripped | `router.ParseTaskCapture`, `router.TaskCaptureGrammar` | done |
|
||||
| recite the list on request | `router.IsTaskListQuery` | done |
|
||||
| a page to read and change the board | `/tasks` in `cmd/mavweb` | done |
|
||||
| counting a shape without judging it | `internal/memory/behavior.go` | done, as precedent |
|
||||
| a proposal he reads when he chooses | `/routines`, the proposed-routine queue | done, as precedent |
|
||||
|
||||
`tasks` already carries `status` (candidate, open, done, dropped), `due_ts`, `weight`,
|
||||
`source`, `evidence`, `ext_id` and `resolved_by`. Three things are missing. It has no
|
||||
definition of done and no blocked-on. There is no way to edit a task after capture:
|
||||
`SetTaskStatus` moves the status and nothing writes text, date or weight again. And
|
||||
there is no grouping he controls, because order is computed by `tasks.Rank` alone.
|
||||
|
||||
## Where the form lives, and why not voice
|
||||
|
||||
The task asks the form to refuse a capture with no definition of done. That refusal
|
||||
cannot live on the voice path, for two reasons.
|
||||
|
||||
**The parked-state mechanism is binary.** `resolveConfirm` in `cmd/mavend/confirm.go`
|
||||
answers yes or no against a slot with a 90-second life. Filling four fields over four
|
||||
turns is slot filling, which is a different mechanism and a new one. Nothing in the
|
||||
daemon does it today.
|
||||
|
||||
**The definition of done is the worst possible field to dictate.** It is the one string
|
||||
that has to be exact, because its whole purpose is to be unarguable later. Whisper
|
||||
transcribing a sentence of Russian work vocabulary is where exactness goes to die, and
|
||||
the capture path already had to strip a question mark that whisper invented.
|
||||
|
||||
So: voice captures a line and recites the list. The page is where a line becomes an
|
||||
item with a definition of done, a blocked-on and a date. A captured line lands as
|
||||
`candidate` and stays there until it is filled in, which is what `candidate` was for.
|
||||
|
||||
The refusal the task wants survives, moved: the page will not promote a candidate to
|
||||
`open` without a definition of done, the same way `ParseTaskCapture` will not file a
|
||||
marker with nothing after it. And the field must close on either outcome, so "it already
|
||||
works" counts as complete. A definition of done that only one result satisfies is a wish.
|
||||
|
||||
## Does the stage-0 trick stretch
|
||||
|
||||
The task asks this before any shape is committed to. It was checked. The answer is
|
||||
partly.
|
||||
|
||||
`TaskCaptureGrammar` matches every utterance and lets `ParseTaskCapture` decide inside
|
||||
`Build`, keeping the intent at `note` and leaving the frozen seven-intent contract alone.
|
||||
That trick stretches to **recite** and to **status change**: both are a marker plus a
|
||||
referent, both are a lookup, and a status change is a small closed verb set over a list
|
||||
he can see. It does not stretch to **intake**, because intake is not one utterance, and
|
||||
it does not need to, because intake moved to the page.
|
||||
|
||||
One cost to name. Each such grammar matches everything and runs its parser on every
|
||||
turn, ahead of the resident model. Two more of them is fine. A dozen would make stage 0
|
||||
a second router with no evaluation behind it, and at that point the frozen enum is the
|
||||
smaller problem.
|
||||
|
||||
## What is not built: the argument
|
||||
|
||||
Not now and not later behind a flag. The task is right about why, and
|
||||
`internal/memory/behavior.go` already argued it for habits: a 1.7B asked whether evidence
|
||||
proves anything will agree fluently and launder a guess into a decision. A wrong claim
|
||||
about his work, stated confidently, is the most expensive kind of wrong Maven can be.
|
||||
|
||||
The line is the same line behaviour memory drew. She may **count**:
|
||||
|
||||
- no state change in eleven days
|
||||
- blocked on a person, with no date
|
||||
- four of nine waiting on two people
|
||||
|
||||
Those are queries over rows. She may not assess whether a build proves anything, whether
|
||||
a blocker is real, or whether a task should be dropped.
|
||||
|
||||
## Persona
|
||||
|
||||
A progress tracker is a nag by default, and "not a nag" is hard. The line is already
|
||||
drawn twice in the codebase and it is drawn the same way here:
|
||||
|
||||
- A date he set becomes a reminder. He set it, so it is not her raising it.
|
||||
- A stall becomes a proposal he reads when he chooses, on a page, like `/routines`.
|
||||
- Ask what is on the board and she recites. She never opens with it.
|
||||
|
||||
The day plan is the place to watch. `tickLoop.dayPlan` reads calendar events, pending
|
||||
reminders and checklist facts, and it does not read tasks. Adding the board to the
|
||||
morning nudge is exactly the move that turns this into a nag, so the board goes on the
|
||||
page and into the answer when asked, and not into the unprompted morning message.
|
||||
|
||||
## The build, as tasks
|
||||
|
||||
1. Two columns on `tasks`: definition of done, and blocked-on. Blocked-on resolves
|
||||
through Nexus like any other person reference, because identity lives in Nexus.
|
||||
2. An edit path. Today a task is write-once except for its status, so the form has
|
||||
nothing to save into.
|
||||
3. `/tasks` grows the form: promote candidate to open only with a definition of done,
|
||||
set a date, set blocked-on. A date set here writes a reminder.
|
||||
4. A status-change grammar at stage 0, following `TaskCaptureGrammar`.
|
||||
5. Counted stall shapes on `/tasks`, phrased as counts. No assessment.
|
||||
|
||||
Note for whoever picks up 3: `/tasks` accepts its POST without the step-up gate, while
|
||||
`/routines` and `/tools` require a passkey. That was deliberate for capture. Adding an
|
||||
edit path is the moment to re-argue it, not to inherit it silently.
|
||||
|
||||
Each is separable and each is worth stopping after.
|
||||
@@ -0,0 +1,87 @@
|
||||
# Plan: What the ambient calendar path should be
|
||||
|
||||
**The decision Vikunja #432 asked for. Written 04-08-2026.**
|
||||
|
||||
**Verdict: keep the endpoint, change the contract.** The relay app sends structured
|
||||
fields, not a notification blob. The free-text parser stays as the degraded path, because
|
||||
there is a real case where the phone cannot produce structure. Delete the endpoint only
|
||||
if the answer to the one open question below is no.
|
||||
|
||||
## First, the task's premise is out of date
|
||||
|
||||
#432 states as confirmed that every ambient event lands on the day the notification was
|
||||
posted, because there is no date parsing at all. That was true when the task was filed
|
||||
and it is not true now.
|
||||
|
||||
`4e4c917` added `dayWords` and `dayOffset` (`internal/calendar/ambient.go:53`), so
|
||||
"завтра в 15:00" now dates to tomorrow. The same commit added `ambientPastGrace`, which
|
||||
refuses an event landing more than two hours before the notification, on the reasoning
|
||||
that the day was inferred and a stale inference is wrong rather than late. `45a5e37`
|
||||
(#482, this week) fixed a second dating bug the task did not know about: the wall clock
|
||||
was resolved against the notification's own zone, so every ambient meeting on a non-UTC
|
||||
box landed off by the deploy's UTC offset.
|
||||
|
||||
What is still missing is an explicit date. "5 августа в 15:00" and "12.08 15:00" carry no
|
||||
day word, so they date to today and the past-grace check drops them. That is a smaller
|
||||
defect than the one filed, and it fails safe rather than storing a wrong meeting.
|
||||
|
||||
The task's third question also has an answer, and the answer is yes. `internal/morning/plan.go:174`
|
||||
prefixes an uncertain item with "похоже, " and `cmd/mavend/actions_query.go:334` carries
|
||||
`Confidence < 1.0` into the calendar recital. Both readers hedge.
|
||||
|
||||
## The open question, and it is the owner's
|
||||
|
||||
**Can the phone read Android's calendar provider, or only the notification text?**
|
||||
|
||||
Everything follows from this and nothing in this repo can answer it.
|
||||
|
||||
A `NotificationListenerService` sees a title and a body. It cannot know a meeting's real
|
||||
start, end or organiser, because those are not in the notification. So if the relay is
|
||||
limited to the notification stream, free-text parsing on this side is not a choice, it is
|
||||
the only thing available, and #432's suggestion that the phone send structured JSON
|
||||
cannot be honoured.
|
||||
|
||||
If the app may instead read `CalendarContract`, it has the actual event rows, and the
|
||||
whole parser stops being necessary. That is the better shape by a wide margin: a real
|
||||
start and end, a real title, an explicit date, no clock-reading heuristic and no
|
||||
past-grace guard, because nothing is inferred.
|
||||
|
||||
Reading the phone's calendar provider does not break the constraint the design was built
|
||||
around. The refusal in `internal/calendar/ambient.go:10` is about holding a work
|
||||
credential **on the homelab**, which is what ties the box's blast radius to the employer.
|
||||
The phone already holds that session. Nothing new lands on homesrv either way.
|
||||
|
||||
**Assumption, and it needs his answer:** a managed work profile may block a third-party
|
||||
app from reading work calendar rows. If it does, the notification stream is all there is.
|
||||
|
||||
## The decision
|
||||
|
||||
**Keep the endpoint.** Deleting it costs the parser, the tests and the wg-facing token,
|
||||
and buys nothing while the question above is open. It is off unless `-ambient-token` is
|
||||
set, so an unbuilt relay carries no surface today.
|
||||
|
||||
**Make structured the primary shape.** `/api/ambient` should accept an event with an
|
||||
explicit start, end and title, and store it without parsing anything. Confidence stays
|
||||
below 1.0 and the source stays `ambient:notif`, because the provenance claim is unchanged:
|
||||
this is the phone telling Maven what it sees, not Maven reading a calendar.
|
||||
|
||||
**Keep the free-text shape as the degraded path.** It is what a notification-only relay
|
||||
can send, and it is already written and tested.
|
||||
|
||||
**Delete it instead if** the relay is not going to be built. That is the one answer that
|
||||
closes this without code, and it is his to give.
|
||||
|
||||
## What not to do
|
||||
|
||||
Do not add date parsing to the free-text path yet. That is the patch #432 explicitly
|
||||
refuses to accept as closure, and it is the wrong order: if the relay can send a date, no
|
||||
date parser is needed, and if it cannot, the parser is guessing at a date from text that
|
||||
was never meant to carry one.
|
||||
|
||||
## Follow-on, unrelated to the decision
|
||||
|
||||
`cmd/mavweb/ambient.go:33` records a known gap worth keeping visible: an ambient meeting
|
||||
writes `calendar_event_*` and never `calendar_busy`, so it is good enough to recite and
|
||||
not good enough to suppress a nudge. That is backwards. Suppressing a nudge is the
|
||||
lower-risk use of a low-confidence signal, and reciting one is the higher-risk use. It
|
||||
needs an expiry on the busy level, so it is its own task either way.
|
||||
@@ -0,0 +1,60 @@
|
||||
# Plan: Where mavwaked and mavenclient run
|
||||
|
||||
**The decision Vikunja #463 asked for. Written 04-08-2026.**
|
||||
|
||||
**Verdict: not in compose on homesrv. They run on a client machine in the room he is in.**
|
||||
The transport for that already exists and nothing needs building to allow it. What needs
|
||||
building is a way to check the wake path at all, which is a separate task.
|
||||
|
||||
## The state that prompted this
|
||||
|
||||
`docker-compose.yml` runs mavend, mavsttd, mavttsd, mavweb and mavpoll. `mavwaked` and
|
||||
`mavenclient` appear in no compose file and run as no host process. Both build under
|
||||
`make build`. So the wake word and the voice-activity gate are untested by construction:
|
||||
QA session 1 step 2 covers push-to-talk from `/dash` only, and #287 (voice session
|
||||
quality) can never be more than half-answered while this holds.
|
||||
|
||||
## The reason is not hardware
|
||||
|
||||
homesrv has a microphone. It is a Lenovo IdeaPad 5 Pro, and `/proc/asound/cards` lists
|
||||
the ACP digital mic array with capture devices at `/dev/snd/pcmC1D0c` and
|
||||
`/dev/snd/pcmC2D0c`. Adding `/dev/snd` to compose and joining the `audio` group would
|
||||
work.
|
||||
|
||||
It would also be pointless. A wake-word daemon is worth having in the room he is standing
|
||||
in. homesrv is a server, so its microphone hears the room the server is in, which is not
|
||||
where anybody talks to Maven. Wiring audio into a container to listen to an empty room is
|
||||
work spent on a capability nobody can use.
|
||||
|
||||
## The reason it belongs off-box
|
||||
|
||||
`mavenclient` is a client by name and by design: microphone, then STT, then core, then
|
||||
TTS. It is the one binary in the tree meant to run somewhere else. `mavwaked` is the gate
|
||||
in front of it, so it goes wherever the microphone goes.
|
||||
|
||||
Maven already has components that are not containers on homesrv. `mavupdate` is a
|
||||
host-side tool. The resident model, STT and TTS prefer the workstation and fall back to
|
||||
homesrv (`docs/offload.md`). Off-box is a shape this system already has.
|
||||
|
||||
**And the wire already supports it.** `ipc.Dial` takes a netaddr seam address:
|
||||
a bare path is the unix socket, and `tcp://host:port?token=...` reaches a core on another
|
||||
host, with the token checked in `internal/netaddr` before `internal/ipc` sees the
|
||||
connection. So a client machine reaching mavend over wg or the LAN needs no new protocol
|
||||
work. It needs mavend to listen on TCP, which `deploy/mavend.json` does not currently ask
|
||||
for.
|
||||
|
||||
## What this means for the QA plan
|
||||
|
||||
QA session 1 step 2 should say what it actually covers, which is push-to-talk through
|
||||
`/dash`. It should not read as though it covers the voice loop. The wake path is checked
|
||||
on the client machine or it is not checked, and today there is no client machine.
|
||||
|
||||
That is the honest state, and it is worse than the task suggests: this is not a
|
||||
configuration gap that a compose entry closes. Until a machine with a microphone runs
|
||||
`mavwaked` and `mavenclient` against a TCP-listening mavend, `internal/wake` and
|
||||
`cmd/mavwaked`'s VAD are covered by their unit tests and by nothing else.
|
||||
|
||||
## What was wrong in CLAUDE.md
|
||||
|
||||
The daemon table lists all nine binaries with no column for where they run, which is how
|
||||
this went unnoticed for as long as it did. It now says which two are not on the box.
|
||||
+35
-9
@@ -1,6 +1,6 @@
|
||||
# QA plan: checking Maven properly
|
||||
|
||||
*Last verified: 2026-08-02 @ 20aa2d5. Living doc: correct it in place, do not append.*
|
||||
*Last verified: 2026-08-04 @ 58635f1. Living doc: correct it in place, do not append.*
|
||||
|
||||
Written 2026-08-01, after the 35-PR stack landed and the box came back up.
|
||||
Refreshed 2026-08-02 against the live list, after PRs #85-#90.
|
||||
@@ -105,9 +105,9 @@ turns look misaligned when they are not.
|
||||
**Passes** (02-08-2026, five turns): `я рада`, `поняла`, `помогла`,
|
||||
`проверила`, `записала`, `грустна`, `ты` throughout, no pet names.
|
||||
2. Press push-to-talk on `/dash`. Say `привет`. Confirm a spoken reply comes
|
||||
back. This is the only check that covers mic to STT to core to TTS to
|
||||
speaker as one path. It is also the path the eleven-day outage most likely
|
||||
broke.
|
||||
back. This covers browser mic to STT to core to TTS as one path. It does
|
||||
**not** cover the wake word or the voice-activity gate, and no step here
|
||||
does — see below.
|
||||
3. Say `тихий режим`. Expect `тихий режим включён. буду реже напоминать.` **Passes.**
|
||||
4. Say `выключи тихий режим`. Expect `тихий режим выключен.` Negation must win. **Passes.**
|
||||
5. Say `в комнате тихо`. Quiet mode must NOT flip. Confirm on `/history` that no
|
||||
@@ -129,11 +129,18 @@ turns look misaligned when they are not.
|
||||
and stitch unrelated topics. Asked whether he should move flats, she opened
|
||||
with the weather. That is 287, and it is a phrasing problem, not a loop problem.
|
||||
|
||||
**The wake path cannot be checked as deployed.** `mavwaked` and `mavenclient`
|
||||
appear in no compose file and run as no host process. Step 2 covers only
|
||||
push-to-talk, from `/dash` through mavsttd and mavttsd. Wake word and VAD
|
||||
are untested by construction. Decide whether they belong in compose or on a
|
||||
client machine, and say which in the deploy docs. Tracked as **463**.
|
||||
**The wake path cannot be checked here, and that is now the decision rather
|
||||
than a gap.** `mavwaked` and `mavenclient` appear in no compose file and run as
|
||||
no host process. They are not going to. They belong on a client machine in the
|
||||
room he is standing in, because homesrv's microphone is real and in the wrong
|
||||
room — **463**, written up in `docs/plans/17-where-the-voice-loop-runs.md`.
|
||||
|
||||
So the wake word and the VAD gate are covered by their unit tests and by
|
||||
nothing else, and no session at this box changes that. Checking them needs a
|
||||
machine with a microphone running both binaries against a TCP-listening mavend.
|
||||
`ipc.Dial` already speaks `tcp://host:port?token=...`, so the work is a machine
|
||||
and a config line, not protocol work. Until then, **287** can only be
|
||||
half-answered, and step 2 above is push-to-talk, not the voice loop.
|
||||
|
||||
**319's single-token bug is fixed** (01-08-2026). Single-word Russian utterances no longer come
|
||||
back as `не совсем поняла — можешь переформулировать?`. `привет` and `поужинал`
|
||||
@@ -471,6 +478,25 @@ at any address (**478**). `allow_private` does work, measured both ways.
|
||||
`CaptureStart`. There is no `cmd/mavheard`, no mavweb route, and `mavenclient`
|
||||
never calls it (**480**). Two of its QA steps are also stale.
|
||||
|
||||
**Four stale QA steps were rewritten on 04-08-2026** under **480**, against the
|
||||
code rather than against what the plans said. All four failed the same way: the
|
||||
daemon was right and the step described an older daemon.
|
||||
|
||||
| Step | Said | Says now |
|
||||
|---|---|---|
|
||||
| 253/3 | boots with the methods unknown | refuses to boot, `config.go:1651` |
|
||||
| 253/10 | no `:transcript` note by default | true only with a summary present |
|
||||
| 255/5 | `speaker: enrolment on, recognition BLOCKED` | that line is gone, the capability stays off |
|
||||
| 252/3 | `vision: stored image <id-prefix>` | `vision: stored <id>`, `vision.go:199` |
|
||||
|
||||
Two of them are worth reading past the correction. 253/10 was false in exactly
|
||||
the scenario 253/16 creates, because `writeNotes` saves the transcript whenever
|
||||
the summary is empty so a dead llama-server does not lose the meeting. And 255/5
|
||||
changed because `Recognizes()` was written as the gate, documented as one, and
|
||||
never called — calling it turned `enabled` with no model from a half-working
|
||||
capability into a refusal. Enrolling into a store nothing can match against is
|
||||
not a working half.
|
||||
|
||||
**257, netscan.** Steps 2, 3 and 9 pass at unit level. Step 1 fails. Steps 4 to 8
|
||||
need the block enabled. Step 10 is Bluetooth and stays skipped.
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Maven — Re-architecture (Qwen3 resident model, revised 2026-07-18)
|
||||
|
||||
*Last verified: 2026-08-02 @ 7079a24. Living doc: correct it in place, do not append.*
|
||||
*Last verified: 2026-08-04 @ b6abb19. Living doc: correct it in place, do not append.*
|
||||
|
||||
> Supersedes the classifier-first routing model. Agreed in a design session
|
||||
> after diagnosing that homesrv deploys with a **stub phraser** (no LLM
|
||||
@@ -40,8 +40,10 @@ utterance
|
||||
are deferred until the main feature set is complete.
|
||||
- **Embedder demoted from router to tool** — it now backs `memory.search`
|
||||
(RAG) and gives the router a cheap "similar past notes/intents" hint. The
|
||||
router no longer depends on it clearing a threshold. Upgrade MiniLM → bge-m3
|
||||
for better RU retrieval later (model swap, not architecture).
|
||||
router no longer depends on it clearing a threshold. The MiniLM upgrade is
|
||||
done: it is multilingual-e5-small, asymmetric, with the `query:`/`passage:`
|
||||
prefixes (Vikunja #371, `docs/evals/2026-08-04-recall-e5-small.md`). A
|
||||
further swap is a model swap, not architecture (Vikunja #412).
|
||||
|
||||
### Router output
|
||||
- Constrained structured JSON action `{tool, args, escalate}` — NOT free-form
|
||||
|
||||
@@ -0,0 +1,295 @@
|
||||
package eval
|
||||
|
||||
import (
|
||||
"context"
|
||||
_ "embed"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/kami/maven/internal/router"
|
||||
)
|
||||
|
||||
//go:embed ru_ecosystem_v1.json
|
||||
var reachFixtureJSON []byte
|
||||
|
||||
// Service — where an utterance arrives. Reach stops at the service boundary on
|
||||
// purpose: whether Nexus knows the entity and whether Hexis holds a capability
|
||||
// for it are those services' answers, and a fixture that asserted them would be
|
||||
// measuring three systems and reporting one number.
|
||||
type Service string
|
||||
|
||||
const (
|
||||
// ServiceNone — the turn stays inside Maven.
|
||||
ServiceNone Service = "none"
|
||||
// ServicePraxis — handlePraxisAct dispatched to a capability.
|
||||
ServicePraxis Service = "praxis"
|
||||
// ServiceHexis — the text reached resolveEntityReference.
|
||||
ServiceHexis Service = "hexis"
|
||||
)
|
||||
|
||||
// ReachCase — one utterance and the service it must arrive at.
|
||||
//
|
||||
// WantCapability is informational and unscored: it says which Praxis arm the fn
|
||||
// slot should land on, so a failure reads as "reached Praxis, wrong arm" rather
|
||||
// than only "reached Praxis". Scoring it would assert an alias table this
|
||||
// package cannot import.
|
||||
type ReachCase struct {
|
||||
ID string `json:"id"`
|
||||
Utterance string `json:"utterance"`
|
||||
Lang string `json:"lang"`
|
||||
WantService Service `json:"want_service"`
|
||||
WantCapability string `json:"want_capability"`
|
||||
Tags []string `json:"tags"`
|
||||
Note string `json:"note"`
|
||||
}
|
||||
|
||||
// ReachFixture — the versioned envelope, same shape as Fixture.
|
||||
type ReachFixture struct {
|
||||
SchemaVersion int `json:"schema_version"`
|
||||
Name string `json:"name"`
|
||||
ReferenceNow string `json:"reference_now"`
|
||||
Notes []string `json:"notes"`
|
||||
Cases []ReachCase `json:"cases"`
|
||||
}
|
||||
|
||||
// LoadReach returns the embedded ecosystem fixture.
|
||||
func LoadReach() (ReachFixture, error) {
|
||||
var f ReachFixture
|
||||
if err := json.Unmarshal(reachFixtureJSON, &f); err != nil {
|
||||
return ReachFixture{}, fmt.Errorf("parse reach fixture: %w", err)
|
||||
}
|
||||
if f.SchemaVersion != SchemaVersion {
|
||||
return ReachFixture{}, fmt.Errorf("reach fixture schema_version %d, want %d", f.SchemaVersion, SchemaVersion)
|
||||
}
|
||||
if len(f.Cases) == 0 {
|
||||
return ReachFixture{}, fmt.Errorf("reach fixture has no cases")
|
||||
}
|
||||
return f, nil
|
||||
}
|
||||
|
||||
// Now — the fixture's reference clock, same contract as Fixture.Now.
|
||||
func (f ReachFixture) Now() (time.Time, error) {
|
||||
t, err := time.Parse(time.RFC3339, f.ReferenceNow)
|
||||
if err != nil {
|
||||
return time.Time{}, fmt.Errorf("parse reference_now %q: %w", f.ReferenceNow, err)
|
||||
}
|
||||
return t, nil
|
||||
}
|
||||
|
||||
// PraxisAliases — the fn slots handlePraxisAct dispatches on, copied from
|
||||
// praxisCapabilities in cmd/mavend/ecosystem_acts.go.
|
||||
//
|
||||
// It is a copy because the registry lives in package main and cannot be
|
||||
// imported. That is a drift risk and it is deliberate: the alternative is
|
||||
// lifting the whole capability registry out of the daemon, which is a
|
||||
// refactor this measurement should not be carrying. TestPraxisAliasesShape
|
||||
// asserts the arms are all present; a new capability adds a line here.
|
||||
var PraxisAliases = map[string]string{
|
||||
"list_attention": "list_attention",
|
||||
"attention": "list_attention",
|
||||
"внимание": "list_attention",
|
||||
"что требует внимания": "list_attention",
|
||||
"что нового": "list_attention",
|
||||
"acknowledge_item": "acknowledge_item",
|
||||
"принято": "acknowledge_item",
|
||||
"понял": "acknowledge_item",
|
||||
"поняла": "acknowledge_item",
|
||||
"resolve_item": "resolve_item",
|
||||
"сделано": "resolve_item",
|
||||
"готово": "resolve_item",
|
||||
"решено": "resolve_item",
|
||||
"ignore_item": "ignore_item",
|
||||
"игнорировать": "ignore_item",
|
||||
"неважно": "ignore_item",
|
||||
"pin_item": "pin_item",
|
||||
"закрепить": "pin_item",
|
||||
"list_changes": "list_changes",
|
||||
"changes": "list_changes",
|
||||
"изменения": "list_changes",
|
||||
"что изменилось": "list_changes",
|
||||
"entity_attention": "entity_attention",
|
||||
"entity_status": "entity_attention",
|
||||
}
|
||||
|
||||
// Reach derives which ecosystem service a decision arrives at, assuming all
|
||||
// three are configured. It mirrors actionAct in cmd/mavend/actions_act.go and
|
||||
// hexisBeforeClarify in cmd/mavend/ecosystem_acts.go, in their order:
|
||||
//
|
||||
// 1. A clarified act with text and no fn reaches Hexis before the clarify
|
||||
// question is ever asked. That path runs on the raw slots, so the matcher
|
||||
// does not get to fill fn first.
|
||||
// 2. Otherwise the act matcher may earn a fn from the text slot.
|
||||
// 3. A fn that is a Praxis capability alias dispatches to Praxis.
|
||||
// 4. Non-empty text reaches Hexis.
|
||||
// 5. Anything else stays inside Maven.
|
||||
//
|
||||
// It returns the service and, for Praxis, the capability the fn landed on.
|
||||
func Reach(d router.Decision, m router.ActMatcher) (Service, string) {
|
||||
if d.Intent != router.IntentAct {
|
||||
return ServiceNone, ""
|
||||
}
|
||||
if d.Clarify {
|
||||
if !d.Slots.HasFn && d.Slots.Text != "" {
|
||||
return ServiceHexis, ""
|
||||
}
|
||||
return ServiceNone, ""
|
||||
}
|
||||
fn, hasFn := d.Slots.Fn, d.Slots.HasFn
|
||||
if !hasFn && d.Slots.Text != "" && m != nil {
|
||||
if matched, _, ok := m.Match(d.Slots.Text); ok {
|
||||
fn, hasFn = matched, true
|
||||
}
|
||||
}
|
||||
if hasFn {
|
||||
if capability, ok := PraxisAliases[strings.ToLower(strings.TrimSpace(fn))]; ok {
|
||||
return ServicePraxis, capability
|
||||
}
|
||||
}
|
||||
if d.Slots.Text != "" {
|
||||
return ServiceHexis, ""
|
||||
}
|
||||
return ServiceNone, ""
|
||||
}
|
||||
|
||||
// ReachOutcome — one scored case.
|
||||
type ReachOutcome struct {
|
||||
Case ReachCase
|
||||
Decision router.Decision
|
||||
Got Service
|
||||
Capability string
|
||||
Err error
|
||||
Latency time.Duration
|
||||
Pass bool
|
||||
Reason string
|
||||
}
|
||||
|
||||
// ReachReport — the aggregate.
|
||||
//
|
||||
// The two miss directions are kept apart because they cost different things.
|
||||
// Missed is an utterance that should have reached a service and did not: he
|
||||
// asks again, or does it himself. Overreach is an utterance that reached a
|
||||
// service it had no business reaching, and on the Hexis side that is one
|
||||
// resolution away from executing a capability nobody asked for.
|
||||
type ReachReport struct {
|
||||
Name string
|
||||
Total int
|
||||
Passed int
|
||||
Missed int
|
||||
Overreach int
|
||||
// WrongService — reached a service, but the other one.
|
||||
WrongService int
|
||||
// WrongCapability — reached Praxis on the wrong arm. Reported, not failed.
|
||||
WrongCapability int
|
||||
Errors int
|
||||
Outcomes []ReachOutcome
|
||||
// ByService is keyed by the fixture's want_service.
|
||||
ByService map[string]TagStat
|
||||
ByTag map[string]TagStat
|
||||
P50, P95 time.Duration
|
||||
Max time.Duration
|
||||
}
|
||||
|
||||
// Accuracy — fraction of cases that arrived where the fixture says they must.
|
||||
func (r ReachReport) Accuracy() float64 {
|
||||
if r.Total == 0 {
|
||||
return 0
|
||||
}
|
||||
return float64(r.Passed) / float64(r.Total)
|
||||
}
|
||||
|
||||
// ScoreReach runs every case through the router and derives where it lands. It
|
||||
// never fails the run on a route error: an erroring case scores as a miss and
|
||||
// is counted, because "the model was down" and "the router was wrong" are
|
||||
// different numbers.
|
||||
func ScoreReach(ctx context.Context, name string, r Router, m router.ActMatcher, f ReachFixture) (ReachReport, error) {
|
||||
now, err := f.Now()
|
||||
if err != nil {
|
||||
return ReachReport{}, err
|
||||
}
|
||||
rep := ReachReport{
|
||||
Name: name,
|
||||
Total: len(f.Cases),
|
||||
ByService: map[string]TagStat{},
|
||||
ByTag: map[string]TagStat{},
|
||||
}
|
||||
lat := make([]time.Duration, 0, len(f.Cases))
|
||||
|
||||
for _, c := range f.Cases {
|
||||
start := time.Now()
|
||||
d, err := r.Route(ctx, c.Utterance, now)
|
||||
o := ReachOutcome{Case: c, Decision: d, Err: err, Latency: time.Since(start)}
|
||||
lat = append(lat, o.Latency)
|
||||
|
||||
switch {
|
||||
case err != nil:
|
||||
rep.Errors++
|
||||
o.Reason = fmt.Sprintf("route error: %v", err)
|
||||
default:
|
||||
o.Got, o.Capability = Reach(d, m)
|
||||
switch {
|
||||
case o.Got == c.WantService:
|
||||
o.Pass = true
|
||||
if c.WantCapability != "" && o.Capability != c.WantCapability {
|
||||
rep.WrongCapability++
|
||||
o.Reason = fmt.Sprintf("reached praxis on %q, want %q", o.Capability, c.WantCapability)
|
||||
}
|
||||
case c.WantService == ServiceNone:
|
||||
rep.Overreach++
|
||||
o.Reason = fmt.Sprintf("reached %s, want none (intent %q, clarify %v, text %q)",
|
||||
o.Got, d.Intent, d.Clarify, d.Slots.Text)
|
||||
case o.Got == ServiceNone:
|
||||
rep.Missed++
|
||||
o.Reason = fmt.Sprintf("stayed local, want %s (intent %q, clarify %v, fn %q)",
|
||||
c.WantService, d.Intent, d.Clarify, d.Slots.Fn)
|
||||
default:
|
||||
rep.WrongService++
|
||||
o.Reason = fmt.Sprintf("reached %s, want %s (fn %q)", o.Got, c.WantService, d.Slots.Fn)
|
||||
}
|
||||
}
|
||||
if o.Pass {
|
||||
rep.Passed++
|
||||
}
|
||||
bump(rep.ByService, string(c.WantService), o.Pass)
|
||||
for _, tag := range c.Tags {
|
||||
bump(rep.ByTag, tag, o.Pass)
|
||||
}
|
||||
rep.Outcomes = append(rep.Outcomes, o)
|
||||
}
|
||||
|
||||
sort.Slice(lat, func(i, j int) bool { return lat[i] < lat[j] })
|
||||
rep.P50, rep.P95 = percentile(lat, 0.50), percentile(lat, 0.95)
|
||||
if len(lat) > 0 {
|
||||
rep.Max = lat[len(lat)-1]
|
||||
}
|
||||
return rep, nil
|
||||
}
|
||||
|
||||
// String renders the report.
|
||||
func (r ReachReport) String() string {
|
||||
var b strings.Builder
|
||||
fmt.Fprintf(&b, "%s: %d/%d reached the right place (%.1f%%)\n", r.Name, r.Passed, r.Total, 100*r.Accuracy())
|
||||
fmt.Fprintf(&b, " missed: %d (should have reached, didn't) | overreach: %d (reached, shouldn't) | wrong service: %d | wrong praxis arm: %d | errors: %d\n",
|
||||
r.Missed, r.Overreach, r.WrongService, r.WrongCapability, r.Errors)
|
||||
fmt.Fprintf(&b, " latency: p50 %s p95 %s max %s\n", r.P50, r.P95, r.Max)
|
||||
fmt.Fprintf(&b, " by want: %s\n", renderStats(r.ByService))
|
||||
fmt.Fprintf(&b, " by tag: %s\n", renderStats(r.ByTag))
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// Failures — the per-case detail, sorted by ID so two runs diff cleanly. A
|
||||
// passing case with a wrong Praxis arm is listed too: it carries a reason.
|
||||
func (r ReachReport) Failures() string {
|
||||
var b strings.Builder
|
||||
out := append([]ReachOutcome(nil), r.Outcomes...)
|
||||
sort.Slice(out, func(i, j int) bool { return out[i].Case.ID < out[j].Case.ID })
|
||||
for _, o := range out {
|
||||
if o.Reason == "" {
|
||||
continue
|
||||
}
|
||||
fmt.Fprintf(&b, " %s %q: %s\n", o.Case.ID, o.Case.Utterance, o.Reason)
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
@@ -0,0 +1,228 @@
|
||||
package eval
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/kami/maven/internal/router"
|
||||
)
|
||||
|
||||
func TestLoadReachFixture(t *testing.T) {
|
||||
f, err := LoadReach()
|
||||
if err != nil {
|
||||
t.Fatalf("LoadReach: %v", err)
|
||||
}
|
||||
if _, err := f.Now(); err != nil {
|
||||
t.Fatalf("Now: %v", err)
|
||||
}
|
||||
valid := map[Service]bool{ServiceNone: true, ServicePraxis: true, ServiceHexis: true}
|
||||
seen := map[string]bool{}
|
||||
byService := map[Service]int{}
|
||||
for _, c := range f.Cases {
|
||||
if c.ID == "" || seen[c.ID] {
|
||||
t.Errorf("case %q: empty or duplicate id", c.ID)
|
||||
}
|
||||
seen[c.ID] = true
|
||||
if !valid[c.WantService] {
|
||||
t.Errorf("%s: want_service %q, want none|praxis|hexis", c.ID, c.WantService)
|
||||
}
|
||||
if c.Lang != "ru" && c.Lang != "en" {
|
||||
t.Errorf("%s: lang %q, want ru|en", c.ID, c.Lang)
|
||||
}
|
||||
// A capability only means something on the Praxis side, and it must
|
||||
// name an arm that exists — otherwise the case asserts a target the
|
||||
// dispatch can never hit.
|
||||
if c.WantCapability != "" {
|
||||
if c.WantService != ServicePraxis {
|
||||
t.Errorf("%s: want_capability on %q", c.ID, c.WantService)
|
||||
}
|
||||
if PraxisAliases[c.WantCapability] != c.WantCapability {
|
||||
t.Errorf("%s: capability %q is not a praxis arm", c.ID, c.WantCapability)
|
||||
}
|
||||
}
|
||||
byService[c.WantService]++
|
||||
}
|
||||
// Coverage floor. The negative cases matter most: without them a router
|
||||
// that sent everything to Hexis would score perfectly.
|
||||
for s := range valid {
|
||||
if byService[s] < 5 {
|
||||
t.Errorf("want_service %q has %d cases, want >= 5", s, byService[s])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestReachFixtureIsHeldOut — same rule as TestFixtureIsHeldOut. The classifier
|
||||
// routes by similarity to frozen seeds, so a case copied from models/seeds
|
||||
// would measure memorisation rather than reach.
|
||||
func TestReachFixtureIsHeldOut(t *testing.T) {
|
||||
f, err := LoadReach()
|
||||
if err != nil {
|
||||
t.Fatalf("LoadReach: %v", err)
|
||||
}
|
||||
seeds := loadSeeds(t)
|
||||
for _, c := range f.Cases {
|
||||
if src, ok := seeds[normalize(c.Utterance)]; ok {
|
||||
t.Errorf("%s: %q is verbatim in %s — not held out", c.ID, c.Utterance, src)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestReachDerivation pins the gate order that ScoreReach depends on. These are
|
||||
// the four branches of actionAct plus the clarify pre-empt, asserted directly
|
||||
// so a change to cmd/mavend/actions_act.go that this package no longer mirrors
|
||||
// fails here rather than silently moving the score.
|
||||
func TestReachDerivation(t *testing.T) {
|
||||
m := router.DefaultActMatcher{Fns: actFns}
|
||||
cases := []struct {
|
||||
name string
|
||||
dec router.Decision
|
||||
want Service
|
||||
capability string
|
||||
}{
|
||||
{
|
||||
name: "act with a praxis alias in the fn slot",
|
||||
dec: router.Decision{Intent: router.IntentAct, Slots: router.Slots{Fn: "list_attention", HasFn: true}},
|
||||
want: ServicePraxis, capability: "list_attention",
|
||||
},
|
||||
{
|
||||
name: "act with a russian praxis alias",
|
||||
dec: router.Decision{Intent: router.IntentAct, Slots: router.Slots{Fn: "готово", HasFn: true}},
|
||||
want: ServicePraxis, capability: "resolve_item",
|
||||
},
|
||||
{
|
||||
name: "act with an entity but no fn earns one from the matcher",
|
||||
dec: router.Decision{Intent: router.IntentAct, Slots: router.Slots{Text: "выключи свет в спальне"}},
|
||||
want: ServiceHexis,
|
||||
},
|
||||
{
|
||||
name: "act with a non-praxis fn and text goes to hexis",
|
||||
dec: router.Decision{Intent: router.IntentAct, Slots: router.Slots{Fn: "restart", HasFn: true, Text: "перезапусти гитею"}},
|
||||
want: ServiceHexis,
|
||||
},
|
||||
{
|
||||
name: "act with a fn and no text stays local",
|
||||
dec: router.Decision{Intent: router.IntentAct, Slots: router.Slots{Fn: "restart", HasFn: true}},
|
||||
want: ServiceNone,
|
||||
},
|
||||
{
|
||||
name: "a clarified act with text still reaches hexis",
|
||||
dec: router.Decision{Intent: router.IntentAct, Clarify: true, Slots: router.Slots{Text: "выключи это"}},
|
||||
want: ServiceHexis,
|
||||
},
|
||||
{
|
||||
name: "a clarified act that already has a fn does not",
|
||||
dec: router.Decision{Intent: router.IntentAct, Clarify: true, Slots: router.Slots{Fn: "restart", HasFn: true, Text: "перезапусти"}},
|
||||
want: ServiceNone,
|
||||
},
|
||||
{
|
||||
name: "no intent but act reaches nothing",
|
||||
dec: router.Decision{Intent: router.IntentQuery, Slots: router.Slots{Fn: "list_attention", HasFn: true, Text: "что требует внимания"}},
|
||||
want: ServiceNone,
|
||||
},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
got, capability := Reach(tc.dec, m)
|
||||
if got != tc.want {
|
||||
t.Errorf("Reach = %q, want %q", got, tc.want)
|
||||
}
|
||||
if capability != tc.capability {
|
||||
t.Errorf("capability = %q, want %q", capability, tc.capability)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestPraxisAliasesShape — the copy of praxisCapabilities in reach.go is a
|
||||
// drift risk (see its comment). This does not close it, but it does catch the
|
||||
// cheap half: an arm losing all its aliases, or an alias pointing at an arm
|
||||
// that no longer has a canonical name.
|
||||
func TestPraxisAliasesShape(t *testing.T) {
|
||||
arms := map[string]int{}
|
||||
for alias, capability := range PraxisAliases {
|
||||
if alias == "" || capability == "" {
|
||||
t.Errorf("empty alias or capability: %q → %q", alias, capability)
|
||||
}
|
||||
arms[capability]++
|
||||
}
|
||||
for _, want := range []string{
|
||||
"list_attention", "acknowledge_item", "resolve_item",
|
||||
"ignore_item", "pin_item", "list_changes", "entity_attention",
|
||||
} {
|
||||
if arms[want] == 0 {
|
||||
t.Errorf("praxis arm %q has no aliases", want)
|
||||
}
|
||||
// Every arm must be reachable by its own name, which is the alias the
|
||||
// LLM router's fn slot actually emits.
|
||||
if PraxisAliases[want] != want {
|
||||
t.Errorf("arm %q does not alias to itself", want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestReachBaselineHash — the deterministic ratchet. Same deal as
|
||||
// TestClassifierBaseline: the hash embedder never clears the confidence gate on
|
||||
// paraphrases, so almost everything lands on clarify, and the number this
|
||||
// asserts is the overreach count rather than the accuracy.
|
||||
//
|
||||
// Overreach is the direction worth a hard assertion. An utterance that should
|
||||
// stay inside Maven and instead resolves an entity through Nexus is one Hexis
|
||||
// capability away from executing something nobody asked for, and unlike a miss
|
||||
// he never gets asked about it.
|
||||
func TestReachBaselineHash(t *testing.T) {
|
||||
f, err := LoadReach()
|
||||
if err != nil {
|
||||
t.Fatalf("LoadReach: %v", err)
|
||||
}
|
||||
m := router.DefaultActMatcher{Fns: actFns}
|
||||
rep, err := ScoreReach(context.Background(), "reach: classifier+hash",
|
||||
newBaselineRouter(t, router.NewHashEmbedder(1024), nil), m, f)
|
||||
if err != nil {
|
||||
t.Fatalf("ScoreReach: %v", err)
|
||||
}
|
||||
t.Log("\n" + rep.String() + rep.Failures())
|
||||
|
||||
if rep.Overreach > 4 {
|
||||
t.Errorf("%d utterances reached a service they should not have, want <= 4:\n%s",
|
||||
rep.Overreach, rep.Failures())
|
||||
}
|
||||
}
|
||||
|
||||
// TestReachBaselineONNX — the deployed configuration: the cascade with the
|
||||
// multilingual e5 embedder. Opt-in via MAVEN_ONNX_LIB, same as TestONNXBaseline,
|
||||
// because deps/ is gitignored. `make eval-reach` points it at the vendored copy.
|
||||
//
|
||||
// Reports rather than asserts. This is the number Vikunja #405 asked for, and a
|
||||
// threshold invented alongside the first measurement is not a ratchet, it is a
|
||||
// guess written down twice.
|
||||
func TestReachBaselineONNX(t *testing.T) {
|
||||
lib := os.Getenv("MAVEN_ONNX_LIB")
|
||||
if lib == "" {
|
||||
t.Skip("MAVEN_ONNX_LIB unset — see AGENTS.md § Embedder model for intent routing")
|
||||
}
|
||||
model := filepath.Join("../../..", "models/embedder/multilingual-e5-small/model_quantized.onnx")
|
||||
tok := filepath.Join("../../..", "models/embedder/multilingual-e5-small/tokenizer.json")
|
||||
for _, p := range []string{lib, model, tok} {
|
||||
if _, err := os.Stat(p); err != nil {
|
||||
t.Skipf("missing %s: %v", p, err)
|
||||
}
|
||||
}
|
||||
emb, err := router.NewONNXEmbedder(model, tok, lib)
|
||||
if err != nil {
|
||||
t.Skipf("onnx embedder unavailable: %v", err)
|
||||
}
|
||||
defer emb.Close()
|
||||
|
||||
f, err := LoadReach()
|
||||
if err != nil {
|
||||
t.Fatalf("LoadReach: %v", err)
|
||||
}
|
||||
rep, err := ScoreReach(context.Background(), "reach: classifier+onnx",
|
||||
newBaselineRouter(t, emb, nil), router.DefaultActMatcher{Fns: actFns}, f)
|
||||
if err != nil {
|
||||
t.Fatalf("ScoreReach: %v", err)
|
||||
}
|
||||
t.Log("\n" + rep.String() + rep.Failures())
|
||||
}
|
||||
@@ -0,0 +1,284 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"name": "ru_ecosystem_v1",
|
||||
"reference_now": "2026-08-04T10:00:00+03:00",
|
||||
"notes": [
|
||||
"Held out from models/seeds/*.txt, same rule as ru_routing_v1: a case that is verbatim a seed measures memorisation.",
|
||||
"want_service is where the utterance must ARRIVE, not what it must achieve. Praxis means handlePraxisAct dispatched to a capability; hexis means the text reached resolveEntityReference. Whether the entity exists in Nexus and the capability exists in Hexis is the services' answer, not the router's.",
|
||||
"The reach model is derived in reach.go from cmd/mavend/actions_act.go. Praxis needs IntentAct AND a fn slot whose value is a praxis capability alias. Hexis needs IntentAct AND non-empty text, and it also fires from hexisBeforeClarify, so a clarified act still reaches it.",
|
||||
"want_capability is informational and is not scored. It records which praxis arm the fn should land on, so a report can say WHERE a praxis reach went wrong rather than only that it did.",
|
||||
"none means the utterance must NOT reach either service. Those cases are the expensive direction: a note or a query that lands on a mutating Hexis path is a wrong act, not a missed one."
|
||||
],
|
||||
"cases": [
|
||||
{
|
||||
"id": "eco-ru-001",
|
||||
"utterance": "выключи свет в спальне",
|
||||
"lang": "ru",
|
||||
"want_service": "hexis",
|
||||
"want_capability": "",
|
||||
"tags": ["home", "mutating"],
|
||||
"note": "The canonical act. Verb in the allowlist, entity text after it."
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-002",
|
||||
"utterance": "включи свет на кухне, пожалуйста",
|
||||
"lang": "ru",
|
||||
"want_service": "hexis",
|
||||
"want_capability": "",
|
||||
"tags": ["home", "mutating"],
|
||||
"note": ""
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-003",
|
||||
"utterance": "перезапусти вукунью",
|
||||
"lang": "ru",
|
||||
"want_service": "hexis",
|
||||
"want_capability": "",
|
||||
"tags": ["homelab", "mutating"],
|
||||
"note": "Service name in Russian, declined. Nexus owns the alias, the router only has to hand it over."
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-004",
|
||||
"utterance": "перезагрузи прокси",
|
||||
"lang": "ru",
|
||||
"want_service": "hexis",
|
||||
"want_capability": "",
|
||||
"tags": ["homelab", "mutating"],
|
||||
"note": ""
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-005",
|
||||
"utterance": "останови гитею на минуту",
|
||||
"lang": "ru",
|
||||
"want_service": "hexis",
|
||||
"want_capability": "",
|
||||
"tags": ["homelab", "mutating"],
|
||||
"note": "Trailing qualifier the entity resolver has to survive."
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-006",
|
||||
"utterance": "запусти бэкап на нексусе",
|
||||
"lang": "ru",
|
||||
"want_service": "hexis",
|
||||
"want_capability": "",
|
||||
"tags": ["homelab", "mutating"],
|
||||
"note": ""
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-007",
|
||||
"utterance": "закрой шторы в комнате",
|
||||
"lang": "ru",
|
||||
"want_service": "hexis",
|
||||
"want_capability": "",
|
||||
"tags": ["home", "mutating"],
|
||||
"note": ""
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-008",
|
||||
"utterance": "открой гараж",
|
||||
"lang": "ru",
|
||||
"want_service": "hexis",
|
||||
"want_capability": "",
|
||||
"tags": ["home", "mutating"],
|
||||
"note": "Two words. The single-token thinner spares it, the verb ending carries the subject."
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-009",
|
||||
"utterance": "поставь чайник",
|
||||
"lang": "ru",
|
||||
"want_service": "hexis",
|
||||
"want_capability": "",
|
||||
"tags": ["home", "mutating"],
|
||||
"note": ""
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-010",
|
||||
"utterance": "выключи телевизор в гостиной",
|
||||
"lang": "ru",
|
||||
"want_service": "hexis",
|
||||
"want_capability": "",
|
||||
"tags": ["home", "mutating"],
|
||||
"note": ""
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-011",
|
||||
"utterance": "что требует внимания",
|
||||
"lang": "ru",
|
||||
"want_service": "praxis",
|
||||
"want_capability": "list_attention",
|
||||
"tags": ["attention", "reading"],
|
||||
"note": "The task's own example. Query-shaped, and the reach model only dispatches to Praxis from IntentAct, so this is the case most likely to miss."
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-012",
|
||||
"utterance": "что сейчас требует внимания",
|
||||
"lang": "ru",
|
||||
"want_service": "praxis",
|
||||
"want_capability": "list_attention",
|
||||
"tags": ["attention", "reading"],
|
||||
"note": ""
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-013",
|
||||
"utterance": "что нового по проектам",
|
||||
"lang": "ru",
|
||||
"want_service": "praxis",
|
||||
"want_capability": "list_attention",
|
||||
"tags": ["attention", "reading"],
|
||||
"note": ""
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-014",
|
||||
"utterance": "что изменилось за сегодня",
|
||||
"lang": "ru",
|
||||
"want_service": "praxis",
|
||||
"want_capability": "list_changes",
|
||||
"tags": ["attention", "reading"],
|
||||
"note": ""
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-015",
|
||||
"utterance": "покажи изменения",
|
||||
"lang": "ru",
|
||||
"want_service": "praxis",
|
||||
"want_capability": "list_changes",
|
||||
"tags": ["attention", "reading"],
|
||||
"note": ""
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-016",
|
||||
"utterance": "отметь это как сделанное",
|
||||
"lang": "ru",
|
||||
"want_service": "praxis",
|
||||
"want_capability": "resolve_item",
|
||||
"tags": ["lifecycle", "mutating"],
|
||||
"note": "The task's second example. The item id is the value slot and is missing here, so the capability asks for it — reaching Praxis with no id is still reaching Praxis."
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-017",
|
||||
"utterance": "готово, закрывай",
|
||||
"lang": "ru",
|
||||
"want_service": "praxis",
|
||||
"want_capability": "resolve_item",
|
||||
"tags": ["lifecycle", "mutating"],
|
||||
"note": ""
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-018",
|
||||
"utterance": "принято, я это видел",
|
||||
"lang": "ru",
|
||||
"want_service": "praxis",
|
||||
"want_capability": "acknowledge_item",
|
||||
"tags": ["lifecycle", "mutating"],
|
||||
"note": "Acknowledged is not resolved. The alias must not collapse into the resolve arm."
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-019",
|
||||
"utterance": "игнорировать это пока",
|
||||
"lang": "ru",
|
||||
"want_service": "praxis",
|
||||
"want_capability": "ignore_item",
|
||||
"tags": ["lifecycle", "mutating"],
|
||||
"note": ""
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-020",
|
||||
"utterance": "закрепи этот пункт",
|
||||
"lang": "ru",
|
||||
"want_service": "praxis",
|
||||
"want_capability": "pin_item",
|
||||
"tags": ["lifecycle", "mutating"],
|
||||
"note": ""
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-021",
|
||||
"utterance": "что там с нексусом",
|
||||
"lang": "ru",
|
||||
"want_service": "praxis",
|
||||
"want_capability": "entity_attention",
|
||||
"tags": ["attention", "reading"],
|
||||
"note": "Scoped attention. The alias list for this arm is grammar names only, so free Russian reaching it depends on a grammar that does not exist yet."
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-022",
|
||||
"utterance": "как дела у праксиса",
|
||||
"lang": "ru",
|
||||
"want_service": "praxis",
|
||||
"want_capability": "entity_attention",
|
||||
"tags": ["attention", "reading"],
|
||||
"note": ""
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-023",
|
||||
"utterance": "напомни завтра в девять позвонить маме",
|
||||
"lang": "ru",
|
||||
"want_service": "none",
|
||||
"want_capability": "",
|
||||
"tags": ["negative"],
|
||||
"note": "A reminder is Maven's own store. Nothing outside her hears it."
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-024",
|
||||
"utterance": "запиши что мне понравился этот подход",
|
||||
"lang": "ru",
|
||||
"want_service": "none",
|
||||
"want_capability": "",
|
||||
"tags": ["negative"],
|
||||
"note": "A note carrying a verb-shaped word. It must not become an act."
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-025",
|
||||
"utterance": "какая завтра погода",
|
||||
"lang": "ru",
|
||||
"want_service": "none",
|
||||
"want_capability": "",
|
||||
"tags": ["negative"],
|
||||
"note": ""
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-026",
|
||||
"utterance": "мой рост сто восемьдесят два",
|
||||
"lang": "ru",
|
||||
"want_service": "none",
|
||||
"want_capability": "",
|
||||
"tags": ["negative"],
|
||||
"note": "A fact about him. The personal boundary means this never leaves the box."
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-027",
|
||||
"utterance": "как ты сегодня",
|
||||
"lang": "ru",
|
||||
"want_service": "none",
|
||||
"want_capability": "",
|
||||
"tags": ["negative"],
|
||||
"note": ""
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-028",
|
||||
"utterance": "выключи",
|
||||
"lang": "ru",
|
||||
"want_service": "none",
|
||||
"want_capability": "",
|
||||
"tags": ["negative", "ambiguous"],
|
||||
"note": "A verb with no target. Reaching Hexis with an empty entity text is the failure this case guards: it must clarify, not resolve."
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-029",
|
||||
"utterance": "сделай это",
|
||||
"lang": "ru",
|
||||
"want_service": "none",
|
||||
"want_capability": "",
|
||||
"tags": ["negative", "ambiguous"],
|
||||
"note": "The known dangerous case. It carries text, so under the current gates it does reach Hexis — the fixture says it should not, and that disagreement is the finding."
|
||||
},
|
||||
{
|
||||
"id": "eco-ru-030",
|
||||
"utterance": "во сколько у меня встреча",
|
||||
"lang": "ru",
|
||||
"want_service": "none",
|
||||
"want_capability": "",
|
||||
"tags": ["negative"],
|
||||
"note": "Stage 0 routes agenda questions to query. The calendar is Maven's, not Praxis's."
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user