mavend: a position resolves against the digest she last read (V-516)

The router names a position ("2", "last") or a demonstrative ("this"),
because only the daemon has the list. surfacedItems records the item ids
she read out, in the order she said them, and only for items she could
actually say: one Praxis returned without a title has no position in what
he heard.

resolveSurfacedPosition maps the reference to an id before dispatch, and
its second return says whether the turn is still Praxis's. A position that
names nothing keeps the turn and clears the slot, so the capability asks
which пункт -- he said "второй пункт" and deserves to hear there is no
second one. A demonstrative that resolves to nothing gives the turn BACK,
because "я это сделал" was probably never about a пункт. "это" also needs
the list to hold exactly one item: pointing at one of five is a guess, and
a wrong guess here transitions the wrong item.

No TTL, unlike the pending confirmation. A stale position resolves to an
item Praxis will report as already acknowledged, which is a harmless
answer, where a stale confirmation would execute something.

Measured, make eval-reach, classifier + ONNX: 16/30 -> 27/30 overall,
praxis 0/12 -> 11/12, lifecycle 0/5 -> 5/5, attention 0/7 -> 6/7, hexis
and none unchanged, p50 20.6ms -> 16.5ms. make eval-router: 60/84, 0 false
clarifies, and no failure in that list comes from a stage-0 decision.
Details and the two judgement calls in docs/evals/2026-08-05-praxis-reach.md.
This commit is contained in:
2026-08-05 13:11:04 +04:00
parent 549d4c8380
commit bb51c28a19
7 changed files with 330 additions and 0 deletions
+63
View File
@@ -0,0 +1,63 @@
# Praxis reach at stage 0, 2026-08-05
Vikunja #516. Measured with `make eval-reach` on the held-out ecosystem fixture
(`internal/router/eval/ru_ecosystem_v1.json`, 30 cases), classifier + ONNX embedder,
no llama-server in the run. The LLM arm was not measured, so judge a cascade
number again before quoting one.
## Result
| | before | after |
|---|---|---|
| overall | 16/30 (53.3%) | 27/30 (90.0%) |
| by want: praxis | 0/12 | 11/12 |
| by want: hexis | 9/10 | 9/10 |
| by want: none | 7/8 | 7/8 |
| by tag: lifecycle | 0/5 | 5/5 |
| by tag: attention | 0/7 | 6/7 |
| by tag: reading | 0/7 | 6/7 |
| wrong praxis arm | 0 | 0 |
| p50 latency | 20.6ms | 16.5ms |
## Why it was zero
Not a tuning gap. `handlePraxisAct` dispatches on exact equality between
`Slots.Fn` and a capability alias, and the fn slot is filled by `DefaultActMatcher`
from the deployment's enabled tool names. No Praxis alias is on that list, so no
utterance could put one in the slot. The Russian aliases in `praxisCapabilities`
read as if they matched speech. They are compared against a fn slot and never
against an utterance.
`PraxisGrammars()` (`internal/router/praxis.go`) fills the slot at stage 0, wired in
`buildRouter` before the capture marker because "отметь" is a capture verb.
## The three misses that remain
- `eco-ru-006` "запусти бэкап на нексусе", a Hexis case, routed note. Pre-existing.
- `eco-ru-028` "выключи", reached Hexis, should have asked. Pre-existing.
- `eco-ru-021` "что там с нексусом" wants scoped attention. Deliberately not
claimed. "что там с X" also opens "что там с погодой". Routing a weather
question to Nexus is worse than one missed fixture case.
## Two judgement calls worth re-arguing
**A lifecycle word alone does not transition an item.** "готово" is what he says
about the thing he just finished. So the rules split lifecycle words by mood. An
imperative he says to her ("закрывай") claims the turn bare, and the capability
asks which пункт. A stative ("готово", "принято") needs an item named beside it.
The bare-imperative arm also requires that nothing else in the sentence is being
acted on. "закрой шторы в комнате" is an imperative too. Without that guard it took
a house command to Praxis, measured at hexis 8/10 mid-change.
**A demonstrative resolves only against a one-item digest.** "отметь это как
сделанное" points at what she just read. `resolveSurfacedPosition` maps it to an id
only when exactly one item was spoken. With two or more it gives the turn back to
the cascade rather than transitioning one of them at random. With no digest at all
it gives the turn back too, because "я это сделал" was never about a пункт.
## Routing fixture
`make eval-router`, same run: classifier + ONNX 60/84 (71.4% full and intent-only),
0 false clarifies, 6 missed clarifies (the known `amb-*` set). No failure in that
list comes from a stage-0 decision. Every one carries a classifier confidence score.