claude
2ea39a3d41
Kiwix ranks by keyword overlap, which the package doc has said since it was written: "why is the sky blue" finds a TV episode. queryKiwix sent the whole Russian sentence, because the verbatim path added by V-508 skips the rewriter that would have reduced it. Measured against the Russian ZIM on 2026-08-09, over eight questions. Four reach the right article where they did not: TCP was "Перехват TCP-соединения" and is TCP, фотосинтез was "C4-фотосинтез" and is Фотосинтез, Линус Торвальдс was "Tux", and "кто написал Войну и мир" was "Радуйся, мир (Доктор Кто)". Two were already right and stay right. Two are still wrong and were wrong before. Nothing regressed. kiwix.Topic drops the narrative request, the interrogative and a verb behind one, and keeps everything else. A word it cannot classify is more likely the topic than noise. TitlePath tries the exact article first, since a ZIM is addressable by title and a wrong title is a 404. The gate this task set out to build does not exist. Query-to-passage cosine scored 0.79-0.91 on answerable questions and 0.75-0.84 on unanswerable ones, and the sets overlap. The wrong TCP article scored 0.8653, above five of six unanswerable rows. e5 measures topic, not whether the passage answers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
66 lines
2.0 KiB
Go
66 lines
2.0 KiB
Go
package kiwix
|
|
|
|
import (
|
|
"context"
|
|
"os"
|
|
"testing"
|
|
"time"
|
|
)
|
|
|
|
// TestLiveTopicBeatsTheSentence — the measurement V-668 turned on, kept as a
|
|
// test so the claim can be re-run rather than believed.
|
|
//
|
|
// It prints the article the old path returned and the article the new one
|
|
// returns, for the same question. It asserts nothing about which is better,
|
|
// because "is this the right article" is a human's call. It fails only if the
|
|
// two paths agree on every case, which would mean the change does nothing.
|
|
//
|
|
// MAVEN_KIWIX_URL=http://127.0.0.1:8034 make t PKG=./internal/kiwix/ RUN=TestLive V=1
|
|
func TestLiveTopicBeatsTheSentence(t *testing.T) {
|
|
base := os.Getenv("MAVEN_KIWIX_URL")
|
|
if base == "" {
|
|
t.Skip("MAVEN_KIWIX_URL unset — point it at the kiwix-server host port")
|
|
}
|
|
const book = "wikipedia_ru_all_maxi_2026-02"
|
|
c := New(base)
|
|
questions := []string{
|
|
"что такое TCP?",
|
|
"что такое фотосинтез",
|
|
"кто такой Линус Торвальдс?",
|
|
"кто написал Войну и мир",
|
|
"что такое чёрная дыра",
|
|
"почему небо голубое",
|
|
"почему трава зелёная",
|
|
"столица Франции",
|
|
}
|
|
moved := 0
|
|
for _, q := range questions {
|
|
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
|
|
before := firstTitle(ctx, c, q, book)
|
|
topic := Topic(q)
|
|
after := ""
|
|
if page, err := c.Article(ctx, TitlePath(book, topic), 400); err == nil && page.Text != "" {
|
|
after = page.Title + " (by title)"
|
|
} else {
|
|
after = firstTitle(ctx, c, topic, book)
|
|
}
|
|
cancel()
|
|
if before != after {
|
|
moved++
|
|
}
|
|
t.Logf("%-30s before=%-34q after=%q", q, before, after)
|
|
}
|
|
t.Logf("%d of %d questions reach a different article", moved, len(questions))
|
|
if moved == 0 {
|
|
t.Error("the topic path returns exactly what the sentence path returned")
|
|
}
|
|
}
|
|
|
|
func firstTitle(ctx context.Context, c *Client, pattern, book string) string {
|
|
hits, err := c.Search(ctx, pattern, book, 3)
|
|
if err != nil || len(hits) == 0 {
|
|
return "(nothing)"
|
|
}
|
|
return hits[0].Title
|
|
}
|