Merge pull request 'llama-server holds 7.9GB RSS for a 1.1GB model, and its startup log goes nowhere' (#105) from task/496-recall-a-cross-language-question-loses-i into master

Reviewed-on: #105

docs/evals/2026-08-03-recall-topic-veto.md

@@ -0,0 +1,65 @@
+# Recall topic veto, what it costs and what it buys, 2026-08-03
+
+Vikunja #496. The task asked for a cross-language fix. Skip the topic veto in
+`memory.RecallAllowed` when the question and the hit are in different scripts.
+An English question would then stop losing a Russian note.
+
+No such case exists. No fixture case puts the question and its wanted note in
+different scripts. The case the task named is not one either.
+
+    en-hard-024
+      query  "what fixed the screen problem"
+      note   "the flicker went away once i swapped the display cable"
+
+Both are English. It is a paraphrase failure, not a language failure. A script
+test would not have changed a single case, and neither would a bilingual stem
+map.
+
+## What the veto is worth today
+
+Measured with the real embedder, multilingual-e5-small int8, gate 0.55, margin
+0.008. The first row is the veto as it ships. The second is `RecallAllowed`
+forced to true.
+
+| | cases passing | answered | false recall | silenced by gate |
+|---|---|---|---|---|
+| veto on | 22/32 | 17/27 | 0/5 | 2 |
+| veto off | 22/32 | 18/27 | 1/5 | 1 |
+
+The pass count does not move. The veto trades one true recall for one false one.
+It costs `en-hard-024` and it buys `ru-silent-029`:
+
+    ru-silent-029
+      query  "во сколько отходит поезд"
+      note   "погулял вдоль реки"        0.835, margin 0.019
+
+The second case counted as silenced by the gate is `ru-home-026` at margin
+0.001, which the margin gate stops. The veto has nothing to do with it.
+
+## Why no lexical rule separates the two
+
+`en-hard-024` and `ru-silent-029` are in the same lexical class. Both questions
+share zero content words with their hit, and neither carries a first-person
+marker. The scores sit on top of each other, 0.826 against 0.835, and so do the
+margins, 0.023 against 0.019. Only one thing separates them. A screen problem
+and a swapped display cable are the same event. A train and a river walk are
+not. The embedder scores that difference at nine thousandths.
+
+So the signal is semantic and the gate is lexical. Any rule cheap enough to sit
+in `RecallAllowed` and strong enough to recover `en-hard-024` also re-admits
+`ru-silent-029`, which puts false recall back to 1/5.
+
+One near-miss rule was tried on paper and rejected: let the veto pass when the
+hit itself is first person. It works on these two, because the English note says
+"i swapped" and the Russian note says only "погулял". It is backwards as a
+principle. A first-person note is exactly the personal note the veto keeps away
+from a world question. The rule would weaken the veto where it was designed to
+bite. It survives here only because Russian drops the pronoun.
+
+## Decision
+
+Accept the loss. `en-hard-024` stays silenced and false recall stays 0/5.
+
+The way out is a reranker, not a longer word list. Recall@3 is 85.2% against
+recall@1 at 70.4%, so the right note is usually in the returned set and ranked
+wrong. That is where the remaining points are, and it is not this task.

internal/memory/lexical.go

@@ -60,6 +60,14 @@ var firstPerson = map[string]bool{
 // kill one false one. A question about his own life keeps the embedder alone
 // as its judge. A question about the world has to name something the memory
 // actually mentions.
+//
+// The veto's price was re-measured on 2026-08-03 (#496,
+// docs/evals/2026-08-03-recall-topic-veto.md). It costs one true recall and
+// buys one false one, and the fixture pass count is the same either way. The
+// lost case is an English paraphrase, not the cross-language loss it was
+// reported as, and the fixture has no cross-language case at all. Do not add a
+// script test or a bilingual stem map for it — both are no-ops here. The
+// separating signal is semantic and belongs in a reranker, not in this file.
 func RecallAllowed(query, text string) bool {
 	if mentionsHim(query) {
 		return true

internal/memory/lexical_test.go

@@ -31,6 +31,22 @@ func TestRecallAllowed(t *testing.T) {
 	}
 }
 
+// The known cost of the veto and the thing that pays for it, both measured on
+// the held-out fixture with the real embedder (#496,
+// docs/evals/2026-08-03-recall-topic-veto.md). The two are one lexical class:
+// zero shared content words, no first-person marker, scores 0.826 against 0.835
+// and margins 0.023 against 0.019. Recovering the first re-admits the second,
+// which puts false recall back to 1/5. Anyone loosening the veto has to move
+// the first line without moving the second.
+func TestRecallVetoTradeIsPinned(t *testing.T) {
+	if RecallAllowed("what fixed the screen problem", "the flicker went away once i swapped the display cable") {
+		t.Error("en-hard-024 is expected to stay vetoed — if this passes now, re-measure false recall before celebrating")
+	}
+	if RecallAllowed("во сколько отходит поезд", "погулял вдоль реки") {
+		t.Error("ru-silent-029 must stay vetoed — this is the false recall the veto exists to stop")
+	}
+}
+
 // A question made only of filler has no topic word to match on, and the score
 // gate is then the only judge it can have.
 func TestRecallAllowedFallsBackWhenNothingToCompare(t *testing.T) {