Files
Maven/docs/evals/2026-08-03-personal-boundary.md
claude 86817d6d06 memory: score the personal boundary on seeds, not word lists (V-495)
"что я говорил про бэкапы?" is his data by definition, and nothing outside the
box has ever heard him say anything. The boundary matched possession words only,
so the question walked past it into SearXNG and came back answered out of a Habr
article about somebody else's backups.

A speech-verb marker class was written first and dropped. Russian gives every
verb a dozen surface forms and the "как я говорил, ..." preamble list has no end,
so each form the lexicon missed was one more question reaching the world, and a
missing verb looks exactly like no bug.

The boundary now embeds two frozen seed sets and scores the turn's own query
vector, already computed upstream, against both. Nearest side wins. The
possession markers stay as the offline floor for a handler with no embedder.

19/19 held-out utterances correct against multilingual-e5-small; see
docs/evals/2026-08-03-personal-boundary.md. The live probe on the deployed box is
not done.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 22:57:11 +04:00

2.0 KiB

Personal boundary, seed scoring vs possession markers, 2026-08-03

Vikunja #495. что я говорил про бэкапы? walked past the personal boundary into SearXNG and came back answered from a Habr article. The boundary matched possession words only, so a first-person speech verb was not a personal question.

What changed

The boundary now scores the turn's query vector against two frozen seed sets. It claims the turn when the personal side is nearer than the world side. Seeds and code are in cmd/mavend/personalboundary.go. The possession markers stay as the offline floor for a handler with no embedder.

A regex speech class was written first and dropped. Russian gives every verb a dozen surface forms, and the "как я говорил, ..." preamble list has no end. Each form the lexicon missed was one more question reaching the world.

Measurement

Embedder: multilingual-e5-small int8, the one homesrv runs. Both sides are embedded on the query side. Cases are held out, none of them a seed. make test runs the offline part. The scored part is opt-in through MAVEN_ONNX_LIB, like TestONNXRecall.

19/19 held-out utterances correct   (TestONNXPersonalBoundary)

true positive margins   +0.014 to +0.089
nearest true negative   -0.005  ("кто такой гагарин")

One case missed during the first pass and is not held out any more: as i said, what is the population of india, +0.008 to the personal side. It is a world seed now.

The gate is the sign of the difference and nothing tighter. The margins are too thin for a threshold. The asymmetry favours claiming: a false claim costs one honest "не знаю", a false pass sends his life to an upstream engine.

make eval-recall unchanged, 18/27 answered at gate 0.55. Recall does not touch this path.

Not verified

The live probe on the deployed box. The daemon was not rebuilt in this session. The reply to что я говорил про бэкапы? with no matching note is still untested against a real SearXNG.