kami/Maven
1
0
Fork 0
Code Issues 1 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

config: the two world sources move to world.go (V-410)

Browse Source 
Pure move. Search and Kiwix are one seam — the query chain tries the live
search, then the ZIMs — so they share a file, in the order they answer, with
the "only the query leaves the box" rule stated once at the top instead of
twice. KiwixConfig's own doc said it was searched "before anything touches
the network", which stopped being true on 2026-08-02 when search went in
front; corrected here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
claude
2026-08-06 01:19:58 +04:00
parent 9ef6f286a9
commit 9ea178be99
2 changed files with 151 additions and 140 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/config/config.go
+2 -140
View File
@@ -775,121 +775,6 @@ type CrawlConfig struct {
MaxRunes int `json:"max_runes,omitempty"`
}
// KiwixConfig — the offline encyclopedia. A kiwix-serve instance holding ZIM
// archives (Wikipedia, ifixit, devdocs) on the LAN, searched before anything
// touches the network. Dark until configured, same as every other reach.
//
// This is the "local sources first" rule in CLAUDE.md made concrete: a 1.7B
// does not know enough to answer a world question, but it can read. A local
// read costs nothing and leaves the box only as far as the LAN.
//
// Only the rewritten search query leaves this process. His notes, facts,
// persona block and history are never part of a request.
type KiwixConfig struct {
// URL — base address of kiwix-serve, e.g. "http://kiwix:8080". Empty ⇒ the
// whole block is normalised to nil and the source stays off.
URL string `json:"url,omitempty"`
// Book — the ZIM to search, by its catalog name, e.g.
// "wikipedia_en_all_maxi_2026-02". Take it from the /content/… href in
// /catalog/v2/entries; the display title is not the name.
//
// Required. kiwix-serve answers 400 to a search with an empty books.name,
// so a block without one is normalised to nil rather than left to fail one
// query at a time.
Book string `json:"book,omitempty"`
// BookRU — the ZIM to search when the question is in Russian, by the same
// catalog name. Empty ⇒ every question goes to Book.
//
// It exists because the rewriter is a workaround, not a feature (V-508). An
// English ZIM cannot match a Russian sentence, so the resident model turns
// the question into English keywords first, and that costs a model call and
// loses whatever the keywords drop. A Russian ZIM matches the question as he
// asked it. So a Cyrillic question searches this book verbatim and skips the
// rewrite, and the English book keeps answering English ones.
BookRU string `json:"book_ru,omitempty"`
// MaxResults — how many hits are asked for. 0 ⇒ DefaultKiwixResults.
// Only the top few reach the phraser regardless; the rest are context the
// snippet ranking throws away.
MaxResults int `json:"max_results,omitempty"`
// SnippetRunes — how much of the joined snippets is handed to the phraser.
// 0 ⇒ DefaultKiwixSnippetRunes. Sized against the 4096-token context, which
// also holds the persona block and the prompt.
SnippetRunes int `json:"snippet_runes,omitempty"`
// Rewrite — turn the Russian question into English keywords with the
// resident model before searching. The ZIMs are English and kiwix ranks by
// keyword, not meaning, so a Russian sentence matches nothing. Costs one
// short LLM call per query. Default true; set false only to measure the
// difference or when the books are Russian.
Rewrite *bool `json:"rewrite,omitempty"`
}
// RewriteEnabled — Rewrite with its default applied. Absent ⇒ on.
func (k *KiwixConfig) RewriteEnabled() bool {
return k.Rewrite == nil || *k.Rewrite
}
// Kiwix defaults, applied in Normalise.
const (
DefaultKiwixResults = 5
DefaultKiwixSnippetRunes = 1500
)
// SearchConfig — the self-hosted SearXNG instance she searches with.
//
// External search is allowed and off unless configured (CLAUDE.md). Configuring
// it is the whole opt-in: no `search` block, no query ever leaves the LAN.
//
// It sits AHEAD of Kiwix in the query chain, and that is the owner's ruling of
// 2026-08-02: a live search answers better than a frozen ZIM, and the ZIM is
// what she falls back to when the line is down. Everything of HIS still comes
// first — the personal boundary runs above both, so a question about him is
// never searched.
//
// Only the query string leaves the box. Notes, facts, the persona block and the
// history are never part of a request; internal/websearch cannot read the store.
type SearchConfig struct {
// URL — base address of the SearXNG instance, e.g. "http://searxng:9563".
// Empty ⇒ the whole block is normalised to nil and the source stays off.
//
// The instance needs `search.formats` to include `json` in its settings.yml.
// A stock install answers 403 to format=json, and then every search fails.
URL string `json:"url,omitempty"`
// MaxResults — how many hits are kept as evidence. 0 ⇒ DefaultSearchResults.
// Small on purpose: the snippets share a 4096-token context with the persona
// block and the prompt.
MaxResults int `json:"max_results,omitempty"`
// SnippetRunes — how much of the joined evidence reaches the phraser.
// 0 ⇒ DefaultSearchSnippetRunes.
SnippetRunes int `json:"snippet_runes,omitempty"`
// Language — SearXNG's `language` parameter, e.g. "ru", "en" or "auto".
// Empty ⇒ the instance default. He asks in Russian and in English, so
// pinning one language here is usually the wrong call.
Language string `json:"language,omitempty"`
// Engines — comma-separated engine names to restrict the search to, e.g.
// "duckduckgo,wikipedia". Empty ⇒ whatever the instance has enabled.
Engines string `json:"engines,omitempty"`
// Timeout — per-search budget. 0 ⇒ websearch.DefaultTimeout. SearXNG waits
// on the slowest upstream engine, so this is the knob that decides how long
// a voice turn can stall on a bad network.
Timeout Duration `json:"timeout,omitempty"`
}
// Search defaults, applied in Normalise.
const (
DefaultSearchResults = 4
DefaultSearchSnippetRunes = 1500
)
// CrawlWatchConfig — one page kept an eye on.
type CrawlWatchConfig struct {
Name string `json:"name"` // note source is "crawl:<name>"
@@ -1180,31 +1065,8 @@ func (c *Config) applyDefaults() {
c.Crawl = nil
}
// Same rule for the ZIM reader: no address or no book, nothing to search.
if c.Kiwix != nil && (strings.TrimSpace(c.Kiwix.URL) == "" || strings.TrimSpace(c.Kiwix.Book) == "") {
c.Kiwix = nil
}
if c.Kiwix != nil {
if c.Kiwix.MaxResults <= 0 {
c.Kiwix.MaxResults = DefaultKiwixResults
}
if c.Kiwix.SnippetRunes <= 0 {
c.Kiwix.SnippetRunes = DefaultKiwixSnippetRunes
}
}
// Same rule for the metasearch instance: no address, nothing to search.
if c.Search != nil && strings.TrimSpace(c.Search.URL) == "" {
c.Search = nil
}
if c.Search != nil {
if c.Search.MaxResults <= 0 {
c.Search.MaxResults = DefaultSearchResults
}
if c.Search.SnippetRunes <= 0 {
c.Search.SnippetRunes = DefaultSearchSnippetRunes
}
}
c.normaliseKiwix()
c.normaliseSearch()
c.normaliseWorkstation()
internal/config/world.go
+149
View File
@@ -0,0 +1,149 @@
package config
import "strings"
// The two world sources in the query chain, in the order they answer: a live
// SearXNG search first, the offline ZIMs behind it (owner's call, 2026-08-02).
// Everything of HIS still comes first — the personal boundary runs above both,
// so a question about him is never searched.
//
// Only the query string leaves the box in either case. Notes, facts, the
// persona block and the history are never part of a request; neither
// internal/websearch nor internal/kiwix can read the store.
// SearchConfig — the self-hosted SearXNG instance she searches with.
//
// External search is allowed and off unless configured (CLAUDE.md). Configuring
// it is the whole opt-in: no `search` block, no query ever leaves the LAN.
type SearchConfig struct {
// URL — base address of the SearXNG instance, e.g. "http://searxng:9563".
// Empty ⇒ the whole block is normalised to nil and the source stays off.
//
// The instance needs `search.formats` to include `json` in its settings.yml.
// A stock install answers 403 to format=json, and then every search fails.
URL string `json:"url,omitempty"`
// MaxResults — how many hits are kept as evidence. 0 ⇒ DefaultSearchResults.
// Small on purpose: the snippets share a 4096-token context with the persona
// block and the prompt.
MaxResults int `json:"max_results,omitempty"`
// SnippetRunes — how much of the joined evidence reaches the phraser.
// 0 ⇒ DefaultSearchSnippetRunes.
SnippetRunes int `json:"snippet_runes,omitempty"`
// Language — SearXNG's `language` parameter, e.g. "ru", "en" or "auto".
// Empty ⇒ the instance default. He asks in Russian and in English, so
// pinning one language here is usually the wrong call.
Language string `json:"language,omitempty"`
// Engines — comma-separated engine names to restrict the search to, e.g.
// "duckduckgo,wikipedia". Empty ⇒ whatever the instance has enabled.
Engines string `json:"engines,omitempty"`
// Timeout — per-search budget. 0 ⇒ websearch.DefaultTimeout. SearXNG waits
// on the slowest upstream engine, so this is the knob that decides how long
// a voice turn can stall on a bad network.
Timeout Duration `json:"timeout,omitempty"`
}
// Search defaults, applied in normaliseSearch.
const (
DefaultSearchResults = 4
DefaultSearchSnippetRunes = 1500
)
// normaliseSearch applies the block's defaults. No address, nothing to search.
func (c *Config) normaliseSearch() {
if c.Search != nil && strings.TrimSpace(c.Search.URL) == "" {
c.Search = nil
}
if c.Search == nil {
return
}
if c.Search.MaxResults <= 0 {
c.Search.MaxResults = DefaultSearchResults
}
if c.Search.SnippetRunes <= 0 {
c.Search.SnippetRunes = DefaultSearchSnippetRunes
}
}
// KiwixConfig — the offline encyclopedia. A kiwix-serve instance holding ZIM
// archives (Wikipedia, ifixit, devdocs) on the LAN, searched when the live
// search is empty, unreachable, or the line is down. Dark until configured,
// same as every other reach.
//
// This is the "local sources first" rule in CLAUDE.md made concrete: a 1.7B
// does not know enough to answer a world question, but it can read. A local
// read costs nothing and leaves the box only as far as the LAN.
type KiwixConfig struct {
// URL — base address of kiwix-serve, e.g. "http://kiwix:8080". Empty ⇒ the
// whole block is normalised to nil and the source stays off.
URL string `json:"url,omitempty"`
// Book — the ZIM to search, by its catalog name, e.g.
// "wikipedia_en_all_maxi_2026-02". Take it from the /content/… href in
// /catalog/v2/entries; the display title is not the name.
//
// Required. kiwix-serve answers 400 to a search with an empty books.name,
// so a block without one is normalised to nil rather than left to fail one
// query at a time.
Book string `json:"book,omitempty"`
// BookRU — the ZIM to search when the question is in Russian, by the same
// catalog name. Empty ⇒ every question goes to Book.
//
// It exists because the rewriter is a workaround, not a feature (V-508). An
// English ZIM cannot match a Russian sentence, so the resident model turns
// the question into English keywords first, and that costs a model call and
// loses whatever the keywords drop. A Russian ZIM matches the question as he
// asked it. So a Cyrillic question searches this book verbatim and skips the
// rewrite, and the English book keeps answering English ones.
BookRU string `json:"book_ru,omitempty"`
// MaxResults — how many hits are asked for. 0 ⇒ DefaultKiwixResults.
// Only the top few reach the phraser regardless; the rest are context the
// snippet ranking throws away.
MaxResults int `json:"max_results,omitempty"`
// SnippetRunes — how much of the joined snippets is handed to the phraser.
// 0 ⇒ DefaultKiwixSnippetRunes. Sized against the 4096-token context, which
// also holds the persona block and the prompt.
SnippetRunes int `json:"snippet_runes,omitempty"`
// Rewrite — turn the Russian question into English keywords with the
// resident model before searching. The ZIMs are English and kiwix ranks by
// keyword, not meaning, so a Russian sentence matches nothing. Costs one
// short LLM call per query. Default true; set false only to measure the
// difference or when the books are Russian.
Rewrite *bool `json:"rewrite,omitempty"`
}
// Kiwix defaults, applied in normaliseKiwix.
const (
DefaultKiwixResults = 5
DefaultKiwixSnippetRunes = 1500
)
// RewriteEnabled — Rewrite with its default applied. Absent ⇒ on.
func (k *KiwixConfig) RewriteEnabled() bool {
return k.Rewrite == nil || *k.Rewrite
}
// normaliseKiwix applies the block's defaults. No address or no book, nothing
// to search.
func (c *Config) normaliseKiwix() {
if c.Kiwix != nil && (strings.TrimSpace(c.Kiwix.URL) == "" || strings.TrimSpace(c.Kiwix.Book) == "") {
c.Kiwix = nil
}
if c.Kiwix == nil {
return
}
if c.Kiwix.MaxResults <= 0 {
c.Kiwix.MaxResults = DefaultKiwixResults
}
if c.Kiwix.SnippetRunes <= 0 {
c.Kiwix.SnippetRunes = DefaultKiwixSnippetRunes
}
}