kami/Maven
1
0
Fork 0
Code Issues 1 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Try the capitalized title too, and reach Париж (V-668)

Browse Source 
A ZIM title carries a leading capital and the utterance does not: /A/фотосинтез
is a 404 and /A/Фотосинтез is a 200. TitleCandidates tries the spoken form
first, so a title that begins lowercase on purpose keeps its chance.

That takes the measurement from four right to five, and the fifth is the one
that mattered. "столица Франции" returned "Список столиц Олимпийских игр"
and now returns Париж, through a title redirect the ZIM already held. The
2026-08-05 measurement named that case as the one no lexical signal could
reach. Retrieval by title reaches it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
This commit is contained in:
claude
2026-08-09 10:52:24 +04:00
parent 02f6e8ad4a
commit 96d97e8964
8 changed files with 119 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/evals/2026-08-09-kiwix-topic-retrieval.md
+20 -10
View File
@@ -60,6 +60,10 @@ A ZIM is also addressable by title, which nothing here used. `/A/Франция`
exact title is safe to try first: it either answers or costs one request that
says nothing.
The title has to carry its capital. `/A/фотосинтез` is a 404 and
`/A/Фотосинтез` is a 200. The spoken form is tried first anyway, so a title
that begins lowercase on purpose keeps its chance.
## What shipped, measured
`kiwix.Topic` drops the narrative request, the interrogative and a verb sitting
@@ -71,25 +75,31 @@ would take the topic off the rewriter's input.
| question | before | after |
|---|---|---|
| что такое TCP? | Перехват TCP-соединения | **TCP** (by title) |
| что такое фотосинтез | C4-фотосинтез | **Фотосинтез** |
| что такое фотосинтез | C4-фотосинтез | **Фотосинтез** (by title) |
| кто такой Линус Торвальдс? | Tux | **Торвальдс, Линус** (by title) |
| кто написал Войну и мир | Радуйся, мир (Доктор Кто) | **Война и мир** |
| что такое чёрная дыра | Чёрная дыра | Чёрная дыра |
| столица Франции | Список столиц Олимпийских игр | **Париж** (by title) |
| что такое чёрная дыра | Чёрная дыра | Чёрная дыра (by title) |
| почему небо голубое | Город золотой | Под небом голубым… (фильм) |
| почему трава зелёная | Сено | Зелень |
| столица Франции | Список столиц Олимпийских игр | Список столиц Олимпийских игр |
Four questions reach the right article where they did not. Two were already
right and stay right. Nothing regressed.
Five questions reach the right article where they did not. One was already
right and stays right. Nothing regressed.
"столица Франции" is the surprise. The 2026-08-05 measurement named it as the
case a quality gate must not break, because the answer is Париж and that word
is not in the question. The ZIM holds a title redirect, so asking for the
article titled "Столица Франции" returns Париж. Retrieval by title reaches an
answer that retrieval by keyword cannot.
## What is still wrong
Two of the eight are still not answered, and both are the same shape. The
question names no article. "почему небо голубое" is answered by Rayleigh
scattering and "столица Франции" by the lead of Франция. Neither title is in
the question. Keyword retrieval cannot bridge that and neither can a
threshold. The candidates are a semantic index over titles, or asking the
resident model for the article title rather than for keywords.
question names no article and no redirect covers it. "почему небо голубое" is
answered by Rayleigh scattering, and nothing in the question says so. Keyword
retrieval cannot bridge that and neither can a threshold. The candidates are a
semantic index over titles, or asking the resident model for the article title
rather than for keywords.
`Response.Empty()` is still the whole gate. A wrong article that the search
does return is still spoken. What this change buys is that the article is