Make locative recall prove identity, not overlap (V-719)
The spare-key note scored 0.832 to 0.867 against a spare passport, a blue shirt, a blue document box and a car key. Score and margin cannot separate those: the right note runs 0.817 to 0.892 and the silent cases 0.787 to 0.874, so the ranges overlap and structure has to decide. RecallAllowed now takes two structural facts from the router. A locative question must corroborate every identity term against the candidate's subject, read up to its first dictionary-proven verb, so a location object in the note cannot answer for the thing being located. A turn that is not question-shaped needs a named shared topic even when it ends in '?', which is what "я отменил напоминание про молоко" lacked when it recalled an unrelated note at 0.825 with no runner-up to fail the margin. query_min_score moves 0.55 to 0.80 for tokenizer rev 2. The held-out fixture answers 14/27 real recalls and 0/14 false ones. LocativeAnswerVerifier is the resident-model second opinion, kept behind the deterministic gate and wired into nothing. The measurement that says why is docs/evals/2026-08-15-locative-answerability-verifier.md. --no-verify: master is the working branch this session by the owner's call. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -52,6 +52,14 @@ func TestDeployConfigLoads(t *testing.T) {
|
||||
if cfg.Voice.RouterThreshold <= 0 {
|
||||
t.Error("router threshold did not get its default")
|
||||
}
|
||||
// The recall thresholds are a measured pair. An explicit deployment value
|
||||
// silently overriding a retuned default would make the eval and the box run
|
||||
// different safety gates, so pin both directions here.
|
||||
if cfg.Voice.QueryMinScore != DefaultQueryMinScore || cfg.Voice.QueryMinMargin != DefaultQueryMinMargin {
|
||||
t.Errorf("deploy recall gate is %.3f/%.3f, defaults are %.3f/%.3f",
|
||||
cfg.Voice.QueryMinScore, cfg.Voice.QueryMinMargin,
|
||||
DefaultQueryMinScore, DefaultQueryMinMargin)
|
||||
}
|
||||
|
||||
// The second reach (V-649). The token is a ${VAR} that CI cannot resolve, so
|
||||
// the committed deployment makes the dark state explicit. Removing disabled
|
||||
|
||||
@@ -77,11 +77,15 @@ type ToolConfig struct {
|
||||
// Voice defaults, applied in normaliseVoice.
|
||||
const (
|
||||
DefaultRouterThreshold = 0.55
|
||||
DefaultQueryMinScore = 0.55
|
||||
// Read off the margin sweep in internal/memory/recalleval on the e5
|
||||
// embedder: 0.008 answers 68% of real questions (down from 72%) and cuts
|
||||
// false recall from 5/5 to 1/5. Every larger delta costs real recall
|
||||
// without removing that last one until 0.020, which drops recall to 44%.
|
||||
// The absolute half of the recall gate, recalibrated after tokenizer rev 2
|
||||
// changed the e5 score distribution. At 0.80 with the independent 0.008
|
||||
// margin and structural eligibility, the held-out fixture answers 14/27
|
||||
// real recalls and 0/14 false ones. The score distributions still overlap
|
||||
// (true minimum 0.817, silent maximum 0.874), so structure remains decisive.
|
||||
DefaultQueryMinScore = 0.80
|
||||
// The runner-up half is intentionally separate. Keep 0.008 as the ambiguity
|
||||
// floor measured before the structural gate; a single-hit store has no
|
||||
// runner-up, which is why score and structure are both required.
|
||||
DefaultQueryMinMargin = 0.008
|
||||
// DefaultClarifyMaxAttempts — see dialogue.DefaultMaxAttempts.
|
||||
DefaultClarifyMaxAttempts = 3
|
||||
@@ -258,8 +262,8 @@ type VoiceConfig struct {
|
||||
LLMRouter *bool `json:"llm_router,omitempty"`
|
||||
|
||||
// QueryMinScore — the note-recall confidence gate. Top cosine below this
|
||||
// ⇒ "I don't know" instead of a guess. Tuned for the ONNX embedder (0.55);
|
||||
// the HashEmbedder floor scores lexically and may never clear it. 0.55
|
||||
// ⇒ "I don't know" instead of a guess. Tuned for the corrected ONNX
|
||||
// embedder (0.80); the HashEmbedder floor scores lexically and may never clear it. 0.80
|
||||
// default if unset.
|
||||
QueryMinScore float64 `json:"query_min_score,omitempty"`
|
||||
|
||||
|
||||
@@ -68,3 +68,16 @@ func voiceConfigWith(t *testing.T, model, heads string) error {
|
||||
_, err := Load(writeConfig(t, body))
|
||||
return err
|
||||
}
|
||||
|
||||
func TestVoiceRecallGateDefaults(t *testing.T) {
|
||||
cfg, err := Load(writeConfig(t, `{"voice":{"enabled":true,"bind":"127.0.0.1:9100"}}`))
|
||||
if err != nil {
|
||||
t.Fatalf("Load: %v", err)
|
||||
}
|
||||
if got := cfg.Voice.QueryMinScore; got != DefaultQueryMinScore {
|
||||
t.Errorf("query score default = %.3f, want %.3f", got, DefaultQueryMinScore)
|
||||
}
|
||||
if got := cfg.Voice.QueryMinMargin; got != DefaultQueryMinMargin {
|
||||
t.Errorf("query margin default = %.3f, want %.3f", got, DefaultQueryMinMargin)
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user