# Note recall after the e5-small swap — 04-08-2026

Closes Vikunja #371, which asked for the swap and for this re-measurement. The embedder is no
longer paraphrase-multilingual-MiniLM-L12-v2. It is **multilingual-e5-small**, quantized, with the
`query:` / `passage:` prefixes it was trained with (`internal/router/onnxembedder.go`,
`EmbedQuery` / `EmbedPassage`). `deploy/mavend.json` loads
`models/embedder/multilingual-e5-small/model_quantized.onnx`, which is the same file `make
download-embedder` fetches and the same file this run measured.

- Fixture + scorer: `internal/memory/recalleval/` — 32 cases now, not 30
- Reproduce: `make eval-recall`
- Commit: `b6abb19`
- Gate as deployed: `query_min_score` 0.55, `query_min_margin` 0.008

The fixture grew since 31-07, so the case counts are not comparable row for row. The percentages
are.

## Results

| | 31-07 MiniLM (onnx) | 04-08 e5-small (onnx) |
|---|---|---|
| **recall@1** | 60.0% (15/25) | **70.4% (19/27)** |
| recall@3 | 80.0% (20/25) | **85.2% (23/27)** |
| **answered after the gate** | 48.0% (12/25) | **63.0% (17/27)** |
| **false recall** | 1/5 (20%) | **0/5** |
| wrong note on top / tie on top | 10 / 0 | 8 / 0 |
| ranked first, then silenced by the gate | 3 | 2 |
| `hard` cases passed | 2/11 | 5/12 |
| RU / EN passed | 13/24 / 3/6 | 18/26 / 4/6 |
| latency p50 / p95 / max | 59ms / 148ms / 194ms | **23ms / 41ms / 62ms** |

The hash ratchet CI runs is unchanged in kind and still answers nothing after the gate: recall@1
37.0%, recall@3 74.1%, 0/27 answered, 0/5 false. It is lexical and exists so CI has a deterministic
floor. Never compare a hash number to an ONNX one.

## Findings

### 1. The swap paid on every axis at once, including latency

Ten points of recall@1, fifteen points of *answered*, the one false recall gone, and it is 2.5×
faster because the quantized e5-small is 118MB against the 470MB fp32 file the old config loaded.
Finding 3 of the 31-07 eval predicted the recall half and said nothing about speed; the speed came
from fixing the second half of that finding, which was that the deployed path loaded a different
file than the download target.

The concrete case that eval named is fixed. "из-за чего кончилось место" no longer returns the
guitar-chords filler note. It now returns a homelab note, `n2` at 0.884, and the wanted note is
still not in the top 3 — so the query moved from absurd to merely wrong. That is the shape of what
is left.

### 2. The score distributions still overlap. The margin is what separates them

This is the part of #371's premise that did not come true. Right-note-first top-1 scores run
0.791 / 0.857 / 0.890 (min / median / max). Must-stay-silent top-1 scores run 0.795 / 0.815 /
0.835. The silent cases sit *inside* the answering range, so no value of `query_min_score` keeps
every real recall and rejects every false one — the same verdict as 31-07, at a higher and tighter
band of scores.

What separates them is the second-place gap. Margin top1-top2 for a right first hit: median 0.024.
For a must-be-silent case: median 0.002, max 0.019. A false recall is a note that beats its
neighbours by nothing, because nothing in the store is about the question. The sweep:

| margin | answered | false recall |
|---|---|---|
| 0.000 | 18/27 (67%) | 3/5 |
| 0.005 | 17/27 (63%) | 1/5 |
| **0.008 (deployed)** | **17/27 (63%)** | **0/5** |
| 0.010 | 15/27 (56%) | 0/5 |
| 0.015 | 12/27 (44%) | 0/5 |

0.008 is the knee: it is the smallest margin that silences all five, and the next step up costs two
real answers for nothing. The score gate contributes almost nothing on its own — every value from
0.00 to 0.70 answers the same 18 and admits the same 3 — so `query_min_score` is now close to inert
and the margin is the live dial. Leave both where they are; #412 is where a further sweep belongs.

### 3. What is left is a retrieval problem, not a gate problem

Eight cases put the wrong note on top, and the failures cluster: `hard` 5/12, `preference` 5/9,
`homelab` 8/13. Four of the eight have the right note in the top 3, so a reranker would collect
them; the other four do not, so nothing downstream can. Two more rank first and are silenced by the
margin — `en-hard-024` at 0.826 with margin 0.023, and `ru-home-026` at 0.846 with margin 0.001,
which is a genuine near-tie against a second note that is also plausible.

Preference queries are the weakest class in a way that is not about the model. "когда запускать
резервное копирование" and "как мне присылать оповещения" both return a fact, not the note that
states the preference. Facts and notes are searched in one pass since #373, so a confidently-scored
fact wins a question that a note answers better. That is a ranking policy question and it belongs
in its own task, not in a threshold.