Harden semantic boundaries and repair dialogue state

Replace nearest-neighbour personal routing with a frozen class-balanced linear head measured on historical, stratified, cross-validation, holdout, and fresh challenge gates (V-702). Close the four repair handoff holes, preserve nested clarification flows, and route Russian possession statements through structural grammar rather than lexical exceptions (V-573). Owner explicitly requested direct commits to master.
This commit is contained in:
2026-08-13 03:00:31 +04:00
parent 35c6ff5a71
commit 8015fdbb79
24 changed files with 2644 additions and 172 deletions
+54
View File
@@ -135,6 +135,13 @@ second getting it right. Every rule was added against a measurement.
and "сколько будет 17 на 23". Wired after the agenda rules and **before** the
feed and list rules. "что такое лента" is a definition question, and the feed
rule would take it on the noun alone.
- **Russian possession statements** (`PossessionStatementGrammar`, V-577,
2026-08-13). The closed grammatical frame `у меня …` anchors a plain
declaration as `IntentNote`; its remainder is open and contains no noun list.
Questions, captures, reminders and narrative requests keep their narrower
routes. Hyphenated indefinite pronouns such as `что-то`, `кто-нибудь` and
`кое-что` remain statements—the interrogative root inside that closed
morphology is not a question by itself.
`calendar-query` and `event-time-query` name the calendar as the destination.
The possessive agenda rules deliberately do not. "что у меня в списке покупок"
@@ -357,6 +364,32 @@ at all. It was measured on the box 2026-08-07
TCP?" and "сколько будет 17 на 23?" with "для какого города?". The feed answered
"какой у меня любимый язык?" with kernel headlines.
### How the personal boundary scores
The boundary was fitted from a frozen bilingual training corpus for two classes:
questions that require the owner's records, and questions an outside knowledge
source can answer. The model-backed gate fits one class-balanced logistic head
over those vectors, verifies it, and production loads its model-ID-pinned frozen
weights. The head reads the whole semantic vector; it does not let the one
training sentence with the nearest public noun decide.
The corpus covers remembered speech, possession, narrative, first-person
preambles, current advice/information and public proper nouns on both sides.
Each class contributes total training weight 0.5, so adding coverage to the
larger world class cannot shift the prior merely by class size. The corpus is
separate from both evaluation sets, and deterministic CI checks dimensions,
class balancing, leakage, and leave-one-shape-out generalisation.
The measured result is 29/29 historical regressions, 72/72 on a balanced
Russian/English stratified fixture, 99/104 four-fold training-corpus
cross-validation, and 24/24 on a challenge authored after the final
regularisation was frozen (V-702,
`docs/evals/2026-08-13-personal-boundary-linear-head.md`).
The decision threshold remains exactly personal > world. This boundary is
asymmetric: a false local claim costs an honest “не знаю”, while a false world
claim can send a question about the owner upstream.
### Who may drop the personal boundary
The personal boundary guesses, so naming `SourceWorld` drops it. That is what
@@ -480,3 +513,24 @@ All three reaches offer it as of 2026-08-06:
The turn source is still `tap:text` for both telegram and the web. So provenance
cannot tell a chat turn from a typed one.
### Repair and a parked question
An explicit spoken correction gets first refusal before `clarify-answer`
(V-573). It is narrower evidence than the mere presence of a parked question,
so it must not be scored as bad slot material or spend that question's retry.
The question is suspended and repeated after the repair reply. If redoing the
corrected turn exposes a missing slot of its own, that new question is pushed
on the dialogue stack; completing or cancelling it pops only the top and makes
the older flow audible again.
Completed clarifications and repairs deliberately enter different wrappers.
`finishClarified` acts on a decision whose parked gaps were already checked;
`finishRepaired` first validates the rebuilt decision against the current
required-slot schema. Both record the action as the next correctable turn.
The repair pointer is claimed atomically only after its window and target are
valid. A stale correction changes no state. A correction naming the intent
already used is acknowledged without redoing the action, and retains the
pointer for a genuine correction that follows. This avoids routing correction
prose as a fresh note and overwriting the referent it was meant to preserve.