kami
11831c6ace
The e5 embedder puts every cosine in one narrow band (0.79-0.89), so the absolute query_min_score gate cannot tell a real hit from a made-up question: any value under the band answers everything, any value above it answers nothing. False recall was 5/5. New gate asks whether one note is clearly the best instead: top1 - top2 > delta. New query_min_margin config knob, default 0.008, read off the sweep in the recall harness. The absolute floor stays as a second check. On the recall fixture with e5: answered 72% -> 68%, false recall 5/5 -> 1/5. Vikunja #359 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CGeSZxh1DCtRxmFVSYVGvJ
39 lines
1.4 KiB
Go
39 lines
1.4 KiB
Go
package memory
|
|
|
|
// Confidence gate for a recall. Two checks, both must pass before Maven says a
|
|
// note back:
|
|
//
|
|
// - minScore — an absolute cosine floor.
|
|
// - minMargin — the top hit must beat the runner-up by more than this.
|
|
//
|
|
// The margin is the one that carries the weight. The e5 embedder packs every
|
|
// score into a narrow high band (0.79-0.89 on the recall fixture), so an
|
|
// absolute floor cannot tell a real hit from a confident-looking miss: every
|
|
// value under the band admits everything, every value above it answers nothing.
|
|
// A margin asks a different question — "is this note clearly the best one, or
|
|
// is the whole shelf equally close?" — and a made-up question has no clear best.
|
|
//
|
|
// With one hit and no runner-up there is nothing to compare, so only the floor
|
|
// applies.
|
|
|
|
// ConfidentScores reports whether the top score clears both gates. scores must
|
|
// be sorted highest first. minMargin <= 0 turns the margin check off.
|
|
func ConfidentScores(scores []float64, minScore, minMargin float64) bool {
|
|
if len(scores) == 0 || scores[0] < minScore {
|
|
return false
|
|
}
|
|
if minMargin > 0 && len(scores) > 1 && scores[0]-scores[1] <= minMargin {
|
|
return false
|
|
}
|
|
return true
|
|
}
|
|
|
|
// Confident is ConfidentScores for search results.
|
|
func Confident(results []Result, minScore, minMargin float64) bool {
|
|
scores := make([]float64, len(results))
|
|
for i, r := range results {
|
|
scores[i] = r.Score
|
|
}
|
|
return ConfidentScores(scores, minScore, minMargin)
|
|
}
|