Swap the embedder for an asymmetric retriever (e5-small) #154

2026-08-04T03:51:42+02:00

claude commented

2026-08-04 03:51:42 +02:00

Closes Vikunja #371.

Opened during an unattended overnight run: the diff-budget block was off (task overnight). Read the diff, not only the tests.

Acceptance criteria and quality gate are in TASK.md on this branch.
Review the review, not the diff — leave comments and the agent will apply them via task start 371.

Closes Vikunja #371. Opened during an unattended overnight run: the diff-budget block was off (`task overnight`). Read the diff, not only the tests. Acceptance criteria and quality gate are in `TASK.md` on this branch. Review the review, not the diff — leave comments and the agent will apply them via `task start 371`.

claude added 1 commit 2026-08-04 03:51:42 +02:00

docs: record what the e5-small swap bought (V-371) 246db4e609

The swap itself already landed: deploy loads
models/embedder/multilingual-e5-small/model_quantized.onnx, and
onnxembedder.go grew EmbedQuery/EmbedPassage with the query:/passage:
prefixes the model was trained with. What was missing is the half of #371
that says "re-run make eval-recall and compare against the recorded numbers",
so nothing in the repo says whether it worked.

It worked, on every axis at once. recall@1 60.0% → 70.4%, recall@3 80.0% →
85.2%, answered after the gate 48.0% → 63.0%, false recall 1/5 → 0/5, and
latency p50 59ms → 23ms because the quantized file is 118MB against the 470MB
fp32 one the old config loaded. The guitar-chords note no longer beats the
docker-logs note.

One premise of the task did not come true and the new doc says so. #371
expected a better retriever to separate the score distributions and make
query_min_score tunable. It did not: right-first top-1 runs 0.791-0.890 and
must-stay-silent runs 0.795-0.835, still overlapping, just higher and
tighter. The margin separates them instead — 0.024 median against 0.002 — and
0.008 is the knee where all five silent cases are silenced at no cost. The
score gate is close to inert now; the margin is the live dial. Neither is
changed here, since #412 is where a sweep belongs.

docs/evals/2026-08-04-recall-e5-small.md is the dated measurement.
rearchitecture.md's "upgrade MiniLM → bge-m3 later" is now done and says so,
CLAUDE.md names the retriever and the prefix rule where it already promises
the embedder never leaves homesrv, and the Makefile comment points at this
eval instead of the one that asked for the swap.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

kami approved these changes 2026-08-04 13:18:03 +02:00

claude commented

2026-08-04 18:20:17 +02:00

Reviewed as part of a bottom-up pass over the whole open stack (#119 to #168): commits read against the base branch, make test green at the top of the stack. Nothing to raise on this one. Merging.

Four findings landed on the PRs they belong to, none of them blocking: #167 (a duplicate rest-of-day-query grammar from the merge), #164 (four files the Russian sweep did not reach), #145 (sh -c hides an irreversible verb from the tier derivation), #128 (locationCandidates drops short city names).

Reviewed as part of a bottom-up pass over the whole open stack (#119 to #168): commits read against the base branch, `make test` green at the top of the stack. Nothing to raise on this one. Merging. Four findings landed on the PRs they belong to, none of them blocking: #167 (a duplicate `rest-of-day-query` grammar from the merge), #164 (four files the Russian sweep did not reach), #145 (`sh -c` hides an irreversible verb from the tier derivation), #128 (`locationCandidates` drops short city names).

claude commented

2026-08-04 18:51:41 +02:00

Landed through #160. This branch is now an ancestor of master.

Reviewed as part of a bottom-up pass over the whole open stack. The second line conflicted with master on six files after #168 landed, so one integration merge went onto the head of this line rather than resolving the same conflicts thirteen times. The resolution is written up on #160.

Four findings from the pass stay open and none of them blocks anything here:

internal/router/stage0.go has two grammars named rest-of-day-query, and the second is dead.
internal/router/numwords.go holds ruNumerals, a second copy of the lexicon cardinals.
cmd/mavend/reminderbody.go and cmd/mavend/historyq.go still match Russian by hand.
internal/weather/openmeteo.go guesses declension by reversing endings, and bails under four runes.

Landed through #160. This branch is now an ancestor of `master`. Reviewed as part of a bottom-up pass over the whole open stack. The second line conflicted with master on six files after #168 landed, so one integration merge went onto the head of this line rather than resolving the same conflicts thirteen times. The resolution is written up on #160. Four findings from the pass stay open and none of them blocks anything here: - `internal/router/stage0.go` has two grammars named `rest-of-day-query`, and the second is dead. - `internal/router/numwords.go` holds `ruNumerals`, a second copy of the lexicon `cardinals`. - `cmd/mavend/reminderbody.go` and `cmd/mavend/historyq.go` still match Russian by hand. - `internal/weather/openmeteo.go` guesses declension by reversing endings, and bails under four runes.

claude closed this pull request

2026-08-04 18:51:42 +02:00

Pull request closed

Please reopen this pull request to perform a merge.

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: kami/Maven#154