the ZIM fallback fires fast, and reads Russian in Russian (V-508)

Verification, as the task asked. Drove что такое фотосинтез through
/api/chat with the search reachable, with the container stopped, and with
the host blackholed. Kiwix claims the turn in both failure cases, and a
stopped container costs nothing: DNS fails and the ZIM answers inside the
same second.

The blackhole is the case that hurts. The search waited its full 8-second
budget before the ZIM was asked and the turn took 15.4s against 3.5, which
he sits through with nothing being said. So the connect phase alone is now
capped at 1.5s. A reachable instance that is merely slow keeps the whole
budget, because it is fanning out to real engines.

The RU Wikipedia ZIM is on the box (owner moved it into the kiwix zims
dir), and kiwix-serve picked it up. A Cyrillic question now searches
book_ru verbatim and skips the RU->EN rewrite: that rewriter is the
workaround for an English book, and against a Russian one it is a
translation of his own words back at him. Catalog names come from the
filename, not the <name> field — books.name=wikipedia_ru_all returns
nothing.

Measurement in docs/evals/2026-08-05-kiwix-offline-fallback.md. The RU book
answering a driven turn needs a rebuild and is not verified yet.
This commit is contained in:
2026-08-05 15:52:54 +04:00
parent 2de5a339fb
commit 1f38e71d1a
9 changed files with 234 additions and 9 deletions
@@ -0,0 +1,65 @@
# Does the ZIM answer when the line is down? (V-508)
Measured 2026-08-05 on the deploy, through `POST /api/chat`. The question was
`что такое фотосинтез` in every run. Which source claimed is read off
`voice: query claimed by source` and off the badge V-539 added.
## It fires, and it is fast when the host is gone
`docker stop searxng`, then one question:
| | Claimed by | Turn |
|---|---|---|
| Search reachable | search | 3.5 s |
| Container stopped | kiwix | 3.5 s |
| Host blackholed | kiwix | 15.4 s |
With the container stopped, DNS failed and the ZIM answered inside the same
second:
```
15:40:51 voice: search "что такое фотосинтез": ... lookup searxng: no such host
15:40:51 voice: kiwix: "photosynthesis" → 5 hits, top "Photosynthesis"
15:40:53 voice: query claimed by source "kiwix"
```
The rewrite, the search and the reply all fit in the same turn budget as a live
search. The fallback works.
## The blackhole is the case that hurts
192.0.2.1 is reserved and routed nowhere. Pointing `search.url` at it is the
shape of a real outage: the router drops the packet instead of refusing it. The
search sat for its full 8-second budget before the ZIM was asked. The turn took
15.4 seconds against 3.5. He waits through all of it with nothing
being said.
Fixed by capping the connect phase alone at 1.5 s (`dialTimeout` in
`internal/websearch/searxng.go`). The instance is on the LAN, so a connection it
will ever accept is accepted in milliseconds. A reachable instance that is
merely slow still gets the whole 8 seconds. It is fanning out to real engines,
which is worth waiting for.
## The Russian ZIM is now on the box and is read directly
`wikipedia_ru_all_maxi_2026-02` (41 GB) was copied to the kiwix zims directory
and kiwix-serve picked it up. Note that the catalog name is derived from the
filename. `books.name=wikipedia_ru_all_maxi_2026-02` returns Фотосинтез,
С4-фотосинтез and Википедия. The `<name>` field in the catalog says
`wikipedia_ru_all`, which returns nothing.
A Cyrillic question now searches that book verbatim (`book_ru` in the `kiwix`
block). The rewriter was never a feature. An English ZIM cannot match a Russian
sentence, so the resident model translated the question into English keywords
first. That costs a model call. It also drops whatever the keywords do not carry.
Against a Russian book it is a translation of his own words back at him.
## Not measured here
- The Russian book answering a driven turn. The `book_ru` field is a binary
change, so it needs a rebuild the owner runs. The book itself was verified by
querying kiwix-serve directly.
- Recall against the Russian book compared with the rewrite path. Reading his
own language directly should win, and it was not scored.
- `ru.stackoverflow.com_mul_all_2026-02.zim` is still in the staging directory
and is wired to nothing.