query: let her search the live web before she reads the ZIMs

The offline encyclopedia was the only world source, and it reads what was true
when the ZIM was built. A self-hosted SearXNG now asks first and Kiwix is the
fallback for an empty result, an unreachable instance or no line out. Owner's
ruling, 2026-08-02.

internal/websearch is deliberately thin: no rewriter (SearXNG ranks through
real engines, so the Russian question goes out as he asked it), no page fetch,
no cache. It cannot read the store, so only the query string can leave the box.

The personal boundary is unchanged and still sits above this source, so a
question about him is never searched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BFaeSbLMEVG5ey8tejU3y2
This commit is contained in:
kami
2026-08-02 02:24:36 +04:00
parent a103708a08
commit 14e98334ad
11 changed files with 720 additions and 17 deletions
+80 -9
View File
@@ -112,14 +112,18 @@ var querySources = []querySource{
// has no answer in his data, and no outside source can supply one, so this
// stops the walk rather than let the encyclopedia and the model guess.
{name: "personal", answer: (*reactiveHandler).queryPersonal},
// The offline encyclopedia, after everything of his and before anything on
// the network. A question he can be answered from his own notes is answered
// from his own notes; only what is left over is looked up.
// The world, read live. Owner's ruling of 2026-08-02: a metasearch hit beats
// a frozen ZIM, so SearXNG asks before Kiwix does. Nothing of his is at
// stake by this point — the boundary above already stopped every question
// about him, and only the query string leaves the box.
{name: "search", answer: (*reactiveHandler).querySearch},
// The offline encyclopedia, now the fallback for when the line is down or
// the search comes back empty. It reads the way it always did; what changed
// is that it no longer gets first refusal on a world question.
{name: "kiwix", answer: (*reactiveHandler).queryKiwix},
// LAST before the model answers from memory, and that position is the whole
// design (Vikunja #259): local sources first. His memory, his notes and the
// offline ZIMs all get their turn before anything touches the network.
// The model does NOT: it
// design (Vikunja #259): everything of his, then the search, then the ZIMs,
// and only then a page he named. The model does NOT come first: it
// answers after this, because a URL he said out loud is an instruction and
// a 1.7B guessing at a page it cannot read is how contents get invented.
// This source only claims a turn where he named a URL, so it never competes
@@ -537,9 +541,76 @@ func (h *reactiveHandler) queryWeb(ctx context.Context, t *queryTurn) (string, b
// model's own answer than by more waiting.
const kiwixTimeout = 20 * time.Second
// queryKiwix — the offline encyclopedia. The last local source: everything of
// his has already had its turn and found nothing, so the question is a world
// question, and reading beats recalling for a 1.7B.
// searchTimeout — the whole metasearch source. websearch.Client already holds a
// per-request timeout from config; this is the outer bound on the turn, so a
// hung dial cannot outlive it either. Shorter than kiwixTimeout because there
// is no rewrite call in front of it: the question goes out verbatim.
const searchTimeout = 12 * time.Second
// querySearch — the live web, through a self-hosted SearXNG.
//
// Ahead of Kiwix by the owner's ruling of 2026-08-02: a search reads what is
// true today, a ZIM reads what was true when it was built, and the ZIM is the
// fallback for a box with no line out. Everything of his still answers first —
// the personal boundary is directly above this source, so a question ABOUT him
// never becomes a query.
//
// What leaves this process is the query string and nothing else. His notes, his
// facts, the persona block and the history do not travel with it: the websearch
// package cannot read the store. That is the CLAUDE.md rule made mechanical,
// not a promise about how the prompt is assembled.
//
// It claims the turn only when the search returns something. An empty result,
// an unreachable instance and a 403 from an instance without the JSON format
// all fall through to Kiwix, which is the point of the ordering.
func (h *reactiveHandler) querySearch(ctx context.Context, t *queryTurn) (string, bool) {
if h.search == nil {
// Off unless configured, same as the crawler and the ZIMs. Nothing is
// said about it: he never asked for a capability he did not enable.
return "", false
}
ctxS, cancel := context.WithTimeout(ctx, searchTimeout)
defer cancel()
// Verbatim. No rewriter: SearXNG ranks by meaning through real engines, and
// reducing "почему небо голубое" to English keywords would throw away the
// language he asked in along with the ranking that handles it.
resp, err := h.search.client.Search(ctxS, t.dec.Utterance, h.search.max)
if err != nil {
log.Printf("voice: search %q: %v", t.dec.Utterance, err)
return "", false
}
if resp.Empty() {
return "", false
}
// Logged on the way through, not only on failure. Without this there is no
// telling from the outside whether an answer came off the web, off a ZIM or
// out of the model's weights, and those are the cases worth telling apart.
log.Printf("voice: search: %q → %d answers, %d results", t.dec.Utterance, len(resp.Answers), len(resp.Results))
// Handed over the same way a note, a page or an article is: evidence for the
// question he asked, not something to recite. The trim is one budget over the
// joined block, so a long first snippet cannot crowd out the rest.
evidence := crawl.TrimRunes(strings.Join(resp.Snippets(), "\n"), h.search.runes)
var reply string
if h.phraser != nil {
var perr error
reply, perr = h.phraser.PhraseQuery(ctx, t.dec.Utterance, []string{evidence})
if perr != nil {
log.Printf("voice: search: phrase: %v", perr)
}
}
if reply == "" {
// No phraser, or it failed. Read back the best evidence rather than
// pretend the search did not happen.
return "вот что я нашла: " + crawl.TrimRunes(resp.Snippets()[0], 300), true
}
return reply, true
}
// queryKiwix — the offline encyclopedia, and the fallback behind querySearch:
// everything of his has already had its turn and the live search found nothing
// or could not be reached. Reading beats recalling for a 1.7B either way.
//
// What leaves this process is the search query and nothing else. His notes,
// his facts, the persona block and the history do not travel with it — the