claude
68b2aa9137
V-655 fixed four of the six turns a guessing query source claimed. The clean win is 'что такое TCP?', which stage 0 names world and search now answers instead of weather asking for a city. 'что я сохранил про Сочи?' is no longer read as a capture. The two that did not move are both homelab questions, which is the enum and not the walk. SourceRecall, SourceNetwork and SourceAttention overlap on every question about the box, and the destination fixture already flagged that cluster. The parked clarify is unchanged at 26 turns. It is dialogue state and no query source could have touched it. Latency is reported and not attributed. The workstation was up for the re-run and its state during the baseline was never recorded. Also corrects the baseline's '49 of 140 turns'. That was the sum of occurrences. It is 41 turns.
92 lines
4.3 KiB
Markdown
92 lines
4.3 KiB
Markdown
# The same fortnight, after V-655 merged
|
||
|
||
Date: 2026-08-08, a few hours after `2026-08-08-two-weeks.md`.
|
||
Build: `f8fa0d1` on master, the five compose services rebuilt and recreated.
|
||
Same driver, same 140 turns, same reach. This is the diff that baseline was for.
|
||
|
||
Master now carries V-655, V-659, V-660 and V-661. The change under test is V-655. A query source that decides by seed similarity
|
||
is marked `guesses: true`. It is dropped when the cascade names a different
|
||
destination.
|
||
|
||
## Two things confound the comparison and one of them matters
|
||
|
||
**The workstation was up for the re-run.** `llama-server` on 192.168.1.105
|
||
answered a health probe with 200. So routing completed through `llm.Pair`
|
||
against gemma-4-12b, which is the arm that names a destination. Its state
|
||
during the baseline was not recorded. So a difference here may be the merge, or
|
||
may be the better router, and this run cannot separate them.
|
||
|
||
**The store carried over**, as the baseline said it would. Facts written by the
|
||
first run were present from turn 1 of the second.
|
||
|
||
## Numbers
|
||
|
||
| | baseline `beb093a` | after `f8fa0d1` |
|
||
|---|---|---|
|
||
| turns | 140 | 140 |
|
||
| p50 | 1.5s | 1.2s |
|
||
| p95 | 7.1s | 3.0s |
|
||
| max | 33.7s | 4.2s |
|
||
| transport errors | 0 | 0 |
|
||
| turns carrying a failure string | 41 | 38 |
|
||
|
||
| string in the reply | before | after |
|
||
|---|---|---|
|
||
| `на какое время поставить напоминание` | 13 | 13 |
|
||
| `не нашла у тебя такой записи` | 8 | 9 |
|
||
| `Такую команду я не знаю` | 8 | 8 |
|
||
| `для какого города` | 6 | 4 |
|
||
| `В какой день` | 6 | 6 |
|
||
| `пока не умею` | 5 | 1 |
|
||
| `Когда?` | 3 | 3 |
|
||
|
||
Read the latency as unattributed. The workstation confound covers all of it.
|
||
|
||
## Defect 2 is the one this was for: four of six fixed
|
||
|
||
| utterance | before | after |
|
||
|---|---|---|
|
||
| `что такое TCP?` | `для какого города?` | a real definition |
|
||
| `сколько будет 17 на 23?` | `для какого города?` | search, which has no answer |
|
||
| `какой у меня любимый язык?` | kernel headlines | `не нашла у тебя такой записи` |
|
||
| `что я сохранил про Сочи?` | `Хорошо, сохраню.` | answered as a question |
|
||
| `какая скорость у меня сейчас?` | `для какого города?` | `для какого города?` |
|
||
| `хватает ли места под новые бэкапы?` | kernel headlines | kernel headlines |
|
||
|
||
`что такое TCP?` is the clean win. `WorldQueryGrammars` names `world` at stage
|
||
0, weather is dropped, and search answers.
|
||
|
||
`сколько будет 17 на 23?` moved source and not outcome. Weather no longer claims it. Search
|
||
cannot do arithmetic, so the reply says the sources have no product of 17 and
|
||
23. That is an honest gap where it used to be a wrong
|
||
question. Arithmetic has no destination in the enum.
|
||
|
||
`что я сохранил про Сочи?` was defect 3 and it is gone. The utterance is no
|
||
longer read as a capture.
|
||
|
||
**The two that did not move are both homelab questions.** They are exactly the
|
||
cluster the destination fixture flagged. `SourceRecall`, `SourceNetwork` and
|
||
`SourceAttention` overlap on every question about the box. Five of the seven
|
||
floor cases in that fixture are homelab operations for the same reason. So this
|
||
is the enum, not the walk.
|
||
|
||
## Defect 1 did not move at all
|
||
|
||
Twenty-six turns still carry a parked clarify tail, the same count as the
|
||
baseline. `спасибо тебе` answers `Сейчас 21:12. Это утра или вечера?` and
|
||
`спокойной ночи` answers `хорошо, напомню послезавтра в 10:00.`
|
||
|
||
V-655 was never going to touch this. A parked clarify is dialogue state and not
|
||
a query source. It remains the single worst thing about talking to her. The week test, the
|
||
fortnight test and this re-run all report it unchanged.
|
||
|
||
## A gap in the harness
|
||
|
||
`ipc.ChatReply.Source` came back empty on all 140 turns, in both runs. The
|
||
driver reads it from the redirect query string and there is nothing there. So
|
||
the badge that says which query source claimed a turn is invisible to the
|
||
harness, and every finding above is read off the reply text instead.
|
||
|
||
That is worth fixing before the next re-run. Reading the source directly would
|
||
have shown the two homelab misses without inferring them from the wording.
|