Maven/cmd/mavend/topics.go

package main

import (
	"context"
	"log"
	"sync"

	"github.com/kami/maven/internal/router"
)

// Which subject is this question about — the weather, the house, the LAN, what
// needs looking at, his feeds, or none of them. Third of the three mechanisms replacing hand-written Russian
// patterns (Vikunja #522, owner's call 2026-08-04). internal/lexicon holds the
// sets that can be finished and internal/morph answers the grammar questions;
// this is for the sets that can never be finished, because "is this about the
// house" is a question about meaning and no word list closes it.
//
// The recognisers this replaces were each built the same way: a stem list,
// an ask test, a device-noun list, and a bail-out list for the neighbouring
// topic. Every one of their own comments admits the shape. isHomeQuery excluded
// "погод", "на улице" and "прогноз" by hand because "какая температура на улице"
// and "какая температура в доме" share their only content word. isNetworkQuery
// matched "сети" as a whole token because the substring lives inside "посетил",
// so "сколько машин я посетил" read as a request to scan the LAN. Those are not
// bugs in the lists, they are the lists being asked to do semantics.
//
// So the seeds decide, the same way the personal boundary does
// (personalboundary.go), against the same embedder and the same query vector the
// turn already carries. One difference in the gate, and it is deliberate. The
// boundary claims on the sign of the difference, because there a false claim
// costs one honest "не знаю". Here a false claim runs a network scan, or names a
// capability as off on a box where it is simply not the subject — so a topic has
// to win by a margin, and the losing side of a thin call falls through to the
// next query source, which is what the narrow regexes were achieving.
//
// A fourth subject joined on the same day: isAttentionQuery, "что требует
// внимания", which is Praxis's operational state and reached the web search
// before the source existed (Vikunja #475).
//
// A fifth joined on 05-08-2026: the feeds, "что нового в лентах". Its word lists
// were the last pair of hand-written Russian stem lists in the router (V-522),
// and they carried the same admission in their own comments — vagueNouns exists
// because "что нового?" is a greeting that matched a feed noun.
//
// The regexes stay as the offline floor, unchanged, for a handler with no
// embedder or a turn whose vector never got computed. They are allowed to remain
// narrow now precisely because they are no longer the only answer.

// topicLabel — the subjects worth telling apart, plus the one that means none of
// them. topicOther is a real class and not a threshold: a question needs
// somewhere to lose TO, and "интернет не работает" losing to a set that contains
// complaints is a better statement than it failing a number.
type topicLabel string

const (
	topicWeather topicLabel = "weather"
	topicHome    topicLabel = "home"
	topicNetwork topicLabel = "network"
	topicAttend  topicLabel = "attention"
	topicFeed    topicLabel = "feeds"
	topicList    topicLabel = "list"
	topicSelf    topicLabel = "self"
	topicOther   topicLabel = "other"
)

// topicMargin — how far a topic must clear the runner-up. Small, because the
// margins between neighbouring topics are small: measured on held-out
// utterances, a true weather question clears the home set by roughly 0.02 to
// 0.09 and the nearest wrong call sits under 0.01. It exists at all for the
// asymmetry named above — this gate spends a scan, so a coin-flip falls
// through rather than acts.
const topicMargin = 0.01

// topicSeedSets — frozen scoring data, like personalSeeds. Editing one moves a
// recogniser and has to be re-measured against TestONNXTopics, not eyeballed.
//
// Each set covers the phrasings its old regex covered, INCLUDING the ones it
// needed a bail-out list for: the weather set carries "какая температура на
// улице" and the home set "какая температура в доме", so the pair that forced
// isHomeQuery to exclude weather words by hand is now just two seeds sitting on
// their own sides.
var topicSeedSets = map[topicLabel][]string{
	topicWeather: {
		"какая сегодня погода",
		"какая температура на улице",
		"будет дождь сегодня",
		"на улице холодно",
		"прогноз погоды на завтра",
		"сколько градусов сейчас",
		"what is the weather like",
		"is it going to rain today",
	},
	topicHome: {
		"что включено в доме",
		"какая температура в доме",
		"свет в квартире горит",
		"сколько лампочек включено дома",
		"что у меня дома с датчиками",
		"умный дом что сейчас работает",
		"розетки в доме включены",
		"what is on in the house",
	},
	topicNetwork: {
		"какие устройства в сети",
		"кто в сети сейчас",
		"просканируй локальную сеть",
		"сколько машин в сетке",
		"покажи хосты в сети",
		"какие адреса заняты в локальной сети",
		"кто подключён к вайфаю",
		"what devices are on the network",
	},
	topicAttend: {
		"что требует внимания",
		"что не так сейчас",
		"на что мне посмотреть",
		"что важное я пропустил",
		"есть что-то срочное",
		"что там висит нерешённое",
		// With a thing named. Every other seed asks in the abstract, and the
		// attention question he actually asks names his services or his
		// projects.
		"что не так с сервисами",
		"что там с моими проектами",
		"what needs attention",
		"what needs looking at right now",
	},
	topicFeed: {
		"что нового в лентах",
		"какие новости",
		"что нового по технологиям",
		"почитай заголовки",
		// Two seeds carrying a day word beside the headlines. Without them
		// "какие сегодня заголовки" read as weather, because "какая сегодня
		// погода" is the nearest thing in the whole set with "сегодня" in it.
		"заголовки за сегодня",
		"какие главные новости за день",
		"покажи новости за сегодня",
		"что пишут в новостях",
		"что нового про политику",
		"расскажи что нового в ленте",
		"what is new in the feeds",
		"any news headlines today",
	},
	// Reading a standing list back, and only that. Adding to one and clearing
	// one stay on the phrase tables in internal/router/list.go — see its header
	// for why a span and a delete are not seed-shaped work.
	topicList: {
		"что в списке покупок",
		"что мне нужно купить",
		"прочитай список покупок",
		"покажи что в списке",
		"что осталось купить в магазине",
		"что мне нужно в аптеке",
		"какой у меня список покупок",
		"what is on my shopping list",
		"read me the grocery list",
	},
	// Questions about her (Vikunja #555). The set lives in self.go beside the
	// description it unlocks, so the two are edited together — a seed claiming
	// a question the description does not answer is the failure mode.
	//
	// "что ты умеешь" was a topicOther seed until this existed, put there so an
	// attention question had something to lose to. It is a self seed now, and
	// it cannot be both: a phrasing on two sides never clears the margin.
	topicSelf: selfSeeds,
	topicOther: {
		// A task question is not a list read-back. They collide on "что у меня",
		// and the list has its own table to lose to as well.
		"какие у меня задачи",
		"что у меня в делах",
		// The bare newness opener, which is a greeting and not a request for
		// headlines. It sits here on purpose: it is close enough to the feed
		// seeds that it will not clear topicMargin, and a thin call goes to
		// ParseFeedQuery, which declines a vague noun with no topic beside it.
		"что нового",
		"как дела",
		// Complaints, which are not requests to scan or to read the house.
		// isNetworkQuery's comment names this one: a scan she runs unasked is
		// the noisy behaviour the bounds exist to prevent.
		"интернет не работает",
		"вайфай тормозит",
		"свет погас",
		// Statements. "я дома" was the reason isHomeQuery needed an ask test.
		"я дома",
		"я уже дома",
		// The collision that made "сети" a whole-token match.
		"сколько машин я посетил",
		"сколько домов мы посмотрели",
		// Ordinary questions, his and the world's, so a topic has something
		// real to lose to rather than an arbitrary floor.
		"почему небо синее",
		"какая столица франции",
		"что я говорил про бэкапы",
		"что у меня сегодня по календарю",
		"напомни мне позвонить маме",
		"what did i say about backups",
		// World questions that name a day (Vikunja #553). Weather was the only
		// topic whose seeds carry a day word — four of its eight do — so every
		// "какой сегодня X" landed nearest it and cleared the margin: the
		// dollar rate by 0.0220 and a public holiday by 0.0398, against 0.0883
		// for a real weather question. The gate then asked "для какого города?"
		// about the dollar.
		//
		// The margin was not the knob. 0.0398 is not a coin flip, and raising
		// the bar far enough to catch it would take real weather questions with
		// it. What was missing is the negative class: a day word means the
		// question is about a day, and says nothing about whether it is about
		// the sky.
		"сколько стоит биткоин сегодня",
		"какой завтра праздник в стране",
		"во сколько сегодня восход солнца",
		"кто вчера победил в чемпионате",
		// The frame itself, twice. "какая сегодня погода" is a weather seed,
		// and the four above did not move "какой сегодня курс доллара" or
		// "что интересного произошло сегодня в мире" off weather, because what
		// pulls them is the frame and not the noun. A frame that both topics
		// use has to sit on both sides, or the side that owns it wins every
		// noun it has never seen.
		"какой сегодня курс валют",
		"что сегодня происходит в мире",
		// The same story one topic over, found while verifying V-554 on the
		// box: "кто изобрёл телефон" ran a LAN scan and answered "нашла 3
		// устройства". The network set opens with "кто в сети сейчас" and
		// names devices throughout, so a "кто ..." question about any device
		// noun landed there. A device has a history, and asking about it is
		// not asking what is plugged in.
		"кто изобрёл телефон",
		"когда появился первый компьютер",
		"как работает роутер",
	},
}

// topicIndex holds the embedded seeds. Zero value is usable and means "not
// loaded yet"; a handler built without an embedder never loads and every caller
// uses its own floor instead.
type topicIndex struct {
	once   sync.Once
	vecs   map[topicLabel][][]float32
	loaded bool
}

// load embeds every set once per process, on the QUERY side — a question
// compared with a question, for the reason personalBoundary.load gives.
func (x *topicIndex) load(ctx context.Context, emb router.Embedder) {
	x.once.Do(func() {
		if emb == nil {
			return
		}
		vecs := make(map[topicLabel][][]float32, len(topicSeedSets))
		for label, seeds := range topicSeedSets {
			out := make([][]float32, 0, len(seeds))
			for _, s := range seeds {
				v, err := router.EmbedQuery(ctx, emb, s)
				if err != nil {
					log.Printf("voice: topic seeds unavailable (%v); falling back to keyword matching", err)
					return
				}
				out = append(out, v)
			}
			vecs[label] = out
		}
		x.vecs, x.loaded = vecs, true
	})
}

// best returns the nearest label, how far it cleared the runner-up, and whether
// the seeds answered at all. ok is false when they are not loaded, which is the
// caller's signal to use its floor.
func (x *topicIndex) best(vec []float32) (label topicLabel, margin float64, ok bool) {
	if !x.loaded || len(vec) == 0 {
		return "", 0, false
	}
	first, second := -1.0, -1.0
	for l, seeds := range x.vecs {
		top := -1.0
		for _, s := range seeds {
			if c := cosine(vec, s); c > top {
				top = c
			}
		}
		switch {
		case top > first:
			label, first, second = l, top, first
		case top > second:
			second = top
		}
	}
	return label, first - second, true
}

// turnVector returns the turn's query vector, computing it on first ask and
// caching it on the turn.
//
// It exists because every topic source sits ABOVE the "embed" source in
// querySources, and that source was the only thing that ever set t.vec. So
// turnIsAbout was reading an empty vector on every deployed turn, best returned
// ok=false, and all six recognisers ran on their keyword floors — the seeds
// decided nothing outside the tests, which embed the utterance themselves and
// call best directly. Found on the box on 05-08-2026: "что мне нужно купить" was
// answered from an old note, and the seeds place it as the list by 0.0841.
//
// Computing here rather than moving the embed source up: the cost is paid by the
// turns that ask, the cache means queryEmbed below reuses this one, and the
// order of querySources stays what its comments argue for.
func (h *reactiveHandler) turnVector(ctx context.Context, t *queryTurn) []float32 {
	if len(t.vec) > 0 || h.recall.embedder == nil {
		return t.vec
	}
	vec, err := router.EmbedQuery(ctx, h.recall.embedder, t.dec.Utterance)
	if err != nil {
		// The floor answers. A topic source is not the place to fail a turn:
		// the recall sources below hit the same embedder and report it there.
		log.Printf("voice: topic vector for %q: %v", t.dec.Utterance, err)
		return nil
	}
	t.vec = vec
	return vec
}

// turnIsAbout — the recogniser every topic source calls. The seeds decide when
// the embedder is there, which is every deployed box; floor is the source's own
// keyword test, which answers when they are not.
//
// A topic that wins WITHOUT the margin is handed to the floor rather than
// claimed or dropped, and the near-miss is logged. That is the cascade shape
// again: the better test leads, the offline one always answers, and a thin call
// is exactly where the cheap high-precision test earns its place. Measured, the
// one held-out case that lands there is "вайфай опять отвалился", which reads as
// network by 0.0055; isNetworkQuery says no, so it stays the complaint it is.
func (h *reactiveHandler) turnIsAbout(ctx context.Context, t *queryTurn, want topicLabel, floor func(string) bool) bool {
	h.recall.topics.load(ctx, h.recall.embedder)
	label, margin, ok := h.recall.topics.best(h.turnVector(ctx, t))
	if !ok {
		return floor(t.dec.Utterance)
	}
	if label != want {
		return false
	}
	if margin < topicMargin {
		claimed := floor(t.dec.Utterance)
		log.Printf("voice: %q reads as %s by only %.4f; the keyword floor says %v", t.dec.Utterance, want, margin, claimed)
		return claimed
	}
	return true
}