Measure the registry against the run it describes

ARCHITECTURE.md step 0. eval/chapter-truth.json holds the 38 occurrences the
user walked crop by crop on the 19:44 run of 2026-08-12, and audit_registry.py
now prints purity per cluster and fragmentation per person against it. All six
baseline numbers reproduce.

Rows key on page-space geometry, purity is a share, and fragmentation is a
count of ids, so nothing in the file names a panel_id or a character_id. The
fifth cycle re-crops and calls /characters/reset, and the file survives both.
That was the ordering trap in the handoff.

NEXT.md said 2 of woman B's 9 crops were really woman A and never said which.
They are panel_order 31 and 33, identified from p030 and p032.

Four fixes to the audit itself, all pre-existing:

- 20 characters counted where 14 are live and 6 are merge losers kept on purpose
- the assignment spread keyed on name, so the two Seonhos summed into one line
- the default worked example was panel_index 7, a panel vision skips. NEXT.md's
  "panel 7" is panel_order 7, one lower
- nothing about skipped panels. 41 of 116 are skip=True, four checked and all
  four correct, and they hold 28 of the chapter's 122 dialogue lines

That last count is the measured case for an offscreen speaker_ref kind: 23% of
dialogue sits on panels with no character to attribute to.

Checks: audit_registry.py --selftest covers the IoU match, the greedy tie-break
and the purity maths with no database. ruff check . exits 0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-13 23:32:05 +04:00
parent a67f1c2501
commit bc19f9e9fd
8 changed files with 1001 additions and 14 deletions
+25 -1
View File
@@ -37,6 +37,21 @@ weakness is that its biggest character has no name.
Coverage fell because gemma now clears 12 crops instead of naming them from a shortlist that did not
contain the right person. Coverage is still the `has_face` gate plus those refusals, and nothing else.
Four numbers the audit was misreporting, found while building the truth file on 2026-08-13:
- **41 of 116 panels are `skip=True`**, so all 119 detections come from 75 panels. Four were checked by
eye. All four are right: a balloon-only panel, the chapter title card, a traffic light, an SFX word.
The skip rate is not a defect. It does mean the coverage number describes two thirds of the chapter.
- **Those skipped panels hold 28 of the chapter's 122 dialogue lines**, and a skipped panel has no
character at all. So 23% of dialogue can never resolve to a visible speaker, whatever the linker
does. This is the evidence for `offscreen` as a fourth `speaker_ref` kind, and it is now measured.
- **"panel 7" in this file means `panel_order` 7, which is `panel_index` 6**, the wide office shot with
6 detections. `panel_index` 7 is the balloon-only panel vision skips. `audit_registry.py` defaulted
to 7 and printed an empty worked example.
- **The registry holds 20 rows, of which 14 are live** and 6 are merge losers kept on purpose. The
audit was printing 20 as the cast size. It also keyed the spread on `name`, so the two characters
both named `Seonho` were summed into one line of 11.
## Next
1. **Re-crop the chapter and run the fifth cycle.** Four changes are written and tested since the fourth
@@ -75,6 +90,14 @@ contain the right person. Coverage is still the `has_face` gate plus those refus
carries a DoD. Its build order starts at step 0, `eval/chapter-truth.json` plus purity and
fragmentation printed by `audit_registry.py`. Nothing below it is measurable without that file.
**Step 0's character half is done, 2026-08-13.** `eval/chapter-truth.json` holds 38 labelled
occurrences keyed on page-space geometry, and all six baseline numbers print `= baseline`
(`decisions/measurement-spine.md`). The ordering trap is closed. The file survives both the re-crop
and the reset, so the fifth cycle can be measured against it. Run the audit before and after.
The dialogue half is not written and needs the chapter watched panel by panel, 30 lines with their
true speaker. It is the only part of step 0 still open.
Steps 1 and 2 are independent and cheap: `plane` and `species` per detection, and the
face-plus-person embedding already queued in item 1 above. The weak cannot-link is step 3 and depends
on step 2, not on `plane` alone. Co-presence as a hard constraint is load-bearing while cosine cannot
@@ -173,7 +196,8 @@ contain the right person. Coverage is still the `has_face` gate plus those refus
(`caveats/speaker-attribution.md#tail-is-not-geometry`) is now behind item 5. With no name to attach,
geometry buys nothing.
11. Resolve a speaker answer across the whole dialogue window, not just the answering panel. The last 3
unresolved refs describe a neighbouring panel in the same 8-panel call.
unresolved refs describe a neighbouring panel in the same 8-panel call. The window is also the only
way to reach the 28 lines that sit on skipped panels, measured above.
12. Start Phase 2 from `ROADMAP.md`. Set SQLite `busy_timeout` before any concurrency work
(`caveats/audit-open.md#sqlite-locking`).