bbefda66e2
The third run of the same 140 turns, with the harness fix in. Sixty-eight turns name a source. Two findings the wording could not carry. The unfixed homelab turns are claimed by weather and by feeds, which the destination fixture predicted. And agenda questions are claimed by the personal boundary and by Praxis, not by the calendar: 3 of 6, the same 3 of 6 the destination fixture and every routing-head seed score. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
133 lines
5.8 KiB
Markdown
133 lines
5.8 KiB
Markdown
# The same fortnight, after V-655 merged
|
||
|
||
Date: 2026-08-08, a few hours after `2026-08-08-two-weeks.md`.
|
||
Build: `f8fa0d1` on master, the five compose services rebuilt and recreated.
|
||
Same driver, same 140 turns, same reach. This is the diff that baseline was for.
|
||
|
||
Master now carries V-655, V-659, V-660 and V-661. The change under test is V-655. A query source that decides by seed similarity
|
||
is marked `guesses: true`. It is dropped when the cascade names a different
|
||
destination.
|
||
|
||
## Two things confound the comparison and one of them matters
|
||
|
||
**The workstation was up for the re-run.** `llama-server` on 192.168.1.105
|
||
answered a health probe with 200. So routing completed through `llm.Pair`
|
||
against gemma-4-12b, which is the arm that names a destination. Its state
|
||
during the baseline was not recorded. So a difference here may be the merge, or
|
||
may be the better router, and this run cannot separate them.
|
||
|
||
**The store carried over**, as the baseline said it would. Facts written by the
|
||
first run were present from turn 1 of the second.
|
||
|
||
## Numbers
|
||
|
||
| | baseline `beb093a` | after `f8fa0d1` |
|
||
|---|---|---|
|
||
| turns | 140 | 140 |
|
||
| p50 | 1.5s | 1.2s |
|
||
| p95 | 7.1s | 3.0s |
|
||
| max | 33.7s | 4.2s |
|
||
| transport errors | 0 | 0 |
|
||
| turns carrying a failure string | 41 | 38 |
|
||
|
||
| string in the reply | before | after |
|
||
|---|---|---|
|
||
| `на какое время поставить напоминание` | 13 | 13 |
|
||
| `не нашла у тебя такой записи` | 8 | 9 |
|
||
| `Такую команду я не знаю` | 8 | 8 |
|
||
| `для какого города` | 6 | 4 |
|
||
| `В какой день` | 6 | 6 |
|
||
| `пока не умею` | 5 | 1 |
|
||
| `Когда?` | 3 | 3 |
|
||
|
||
Read the latency as unattributed. The workstation confound covers all of it.
|
||
|
||
## Defect 2 is the one this was for: four of six fixed
|
||
|
||
| utterance | before | after |
|
||
|---|---|---|
|
||
| `что такое TCP?` | `для какого города?` | a real definition |
|
||
| `сколько будет 17 на 23?` | `для какого города?` | search, which has no answer |
|
||
| `какой у меня любимый язык?` | kernel headlines | `не нашла у тебя такой записи` |
|
||
| `что я сохранил про Сочи?` | `Хорошо, сохраню.` | answered as a question |
|
||
| `какая скорость у меня сейчас?` | `для какого города?` | `для какого города?` |
|
||
| `хватает ли места под новые бэкапы?` | kernel headlines | kernel headlines |
|
||
|
||
`что такое TCP?` is the clean win. `WorldQueryGrammars` names `world` at stage
|
||
0, weather is dropped, and search answers.
|
||
|
||
`сколько будет 17 на 23?` moved source and not outcome. Weather no longer claims it. Search
|
||
cannot do arithmetic, so the reply says the sources have no product of 17 and
|
||
23. That is an honest gap where it used to be a wrong
|
||
question. Arithmetic has no destination in the enum.
|
||
|
||
`что я сохранил про Сочи?` was defect 3 and it is gone. The utterance is no
|
||
longer read as a capture.
|
||
|
||
**The two that did not move are both homelab questions.** They are exactly the
|
||
cluster the destination fixture flagged. `SourceRecall`, `SourceNetwork` and
|
||
`SourceAttention` overlap on every question about the box. Five of the seven
|
||
floor cases in that fixture are homelab operations for the same reason. So this
|
||
is the enum, not the walk.
|
||
|
||
## Defect 1 did not move at all
|
||
|
||
Twenty-six turns still carry a parked clarify tail, the same count as the
|
||
baseline. `спасибо тебе` answers `Сейчас 21:12. Это утра или вечера?` and
|
||
`спокойной ночи` answers `хорошо, напомню послезавтра в 10:00.`
|
||
|
||
V-655 was never going to touch this. A parked clarify is dialogue state and not
|
||
a query source. It remains the single worst thing about talking to her. The week test, the
|
||
fortnight test and this re-run all report it unchanged.
|
||
|
||
## A gap in the harness, fixed and re-run the same day
|
||
|
||
`ipc.ChatReply.Source` came back empty on all 140 turns, in both runs. The
|
||
driver read the redirect parameter `src` and `cmd/mavweb/chat.go` writes `s`.
|
||
So every finding above is read off the reply text instead of off the badge.
|
||
|
||
Fixed in V-662 and the 140 turns were driven a third time. Sixty-eight of them
|
||
name a source. The rest are not query turns and never reach `queryWalk`.
|
||
|
||
| source | turns |
|
||
|---|---|
|
||
| search | 27 |
|
||
| memory | 13 |
|
||
| personal | 9 |
|
||
| weather | 5 |
|
||
| calendar | 3 |
|
||
| attention | 3 |
|
||
| list | 2 |
|
||
| feeds | 2 |
|
||
| tasks, money, self, habits | 1 each |
|
||
|
||
## What the badge shows that the wording did not
|
||
|
||
The two unfixed homelab turns are now direct evidence.
|
||
|
||
```text
|
||
какая скорость у меня сейчас? -> weather
|
||
хватает ли места под новые бэкапы? -> feeds
|
||
```
|
||
|
||
Both are guessing sources claiming a turn about the box, exactly as the
|
||
destination fixture predicted.
|
||
|
||
The badge also names a defect the wording hid. **Agenda questions are being
|
||
claimed by the personal boundary and by Praxis, not by the calendar.**
|
||
|
||
```text
|
||
во сколько у меня встреча? -> personal не нашла у тебя такой записи
|
||
когда у меня встреча? -> attention у Praxis нет источников
|
||
что у меня в понедельник? -> personal не нашла у тебя такой записи
|
||
```
|
||
|
||
Calendar claimed 3 turns of the 6 that asked about the calendar. That is the
|
||
same 3/6 the destination fixture scores and the same 3/6 every seed of the
|
||
routing head scores. Three measurements agree. The cause is the one V-660 named. The possessive
|
||
agenda rules claim these at stage 0 and name no destination, so the walk
|
||
reaches `personal` and `attention` first.
|
||
|
||
This is the third independent confirmation that the possessive agenda rules
|
||
should name the calendar. That call is still the owner's.
|