Measure the registry against the run it describes
ARCHITECTURE.md step 0. eval/chapter-truth.json holds the 38 occurrences the user walked crop by crop on the 19:44 run of 2026-08-12, and audit_registry.py now prints purity per cluster and fragmentation per person against it. All six baseline numbers reproduce. Rows key on page-space geometry, purity is a share, and fragmentation is a count of ids, so nothing in the file names a panel_id or a character_id. The fifth cycle re-crops and calls /characters/reset, and the file survives both. That was the ordering trap in the handoff. NEXT.md said 2 of woman B's 9 crops were really woman A and never said which. They are panel_order 31 and 33, identified from p030 and p032. Four fixes to the audit itself, all pre-existing: - 20 characters counted where 14 are live and 6 are merge losers kept on purpose - the assignment spread keyed on name, so the two Seonhos summed into one line - the default worked example was panel_index 7, a panel vision skips. NEXT.md's "panel 7" is panel_order 7, one lower - nothing about skipped panels. 41 of 116 are skip=True, four checked and all four correct, and they hold 28 of the chapter's 122 dialogue lines That last count is the measured case for an offscreen speaker_ref kind: 23% of dialogue sits on panels with no character to attribute to. Checks: audit_registry.py --selftest covers the IoU match, the greedy tie-break and the purity maths with no database. ruff check . exits 0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -37,6 +37,21 @@ weakness is that its biggest character has no name.
|
||||
Coverage fell because gemma now clears 12 crops instead of naming them from a shortlist that did not
|
||||
contain the right person. Coverage is still the `has_face` gate plus those refusals, and nothing else.
|
||||
|
||||
Four numbers the audit was misreporting, found while building the truth file on 2026-08-13:
|
||||
|
||||
- **41 of 116 panels are `skip=True`**, so all 119 detections come from 75 panels. Four were checked by
|
||||
eye. All four are right: a balloon-only panel, the chapter title card, a traffic light, an SFX word.
|
||||
The skip rate is not a defect. It does mean the coverage number describes two thirds of the chapter.
|
||||
- **Those skipped panels hold 28 of the chapter's 122 dialogue lines**, and a skipped panel has no
|
||||
character at all. So 23% of dialogue can never resolve to a visible speaker, whatever the linker
|
||||
does. This is the evidence for `offscreen` as a fourth `speaker_ref` kind, and it is now measured.
|
||||
- **"panel 7" in this file means `panel_order` 7, which is `panel_index` 6**, the wide office shot with
|
||||
6 detections. `panel_index` 7 is the balloon-only panel vision skips. `audit_registry.py` defaulted
|
||||
to 7 and printed an empty worked example.
|
||||
- **The registry holds 20 rows, of which 14 are live** and 6 are merge losers kept on purpose. The
|
||||
audit was printing 20 as the cast size. It also keyed the spread on `name`, so the two characters
|
||||
both named `Seonho` were summed into one line of 11.
|
||||
|
||||
## Next
|
||||
|
||||
1. **Re-crop the chapter and run the fifth cycle.** Four changes are written and tested since the fourth
|
||||
@@ -75,6 +90,14 @@ contain the right person. Coverage is still the `has_face` gate plus those refus
|
||||
carries a DoD. Its build order starts at step 0, `eval/chapter-truth.json` plus purity and
|
||||
fragmentation printed by `audit_registry.py`. Nothing below it is measurable without that file.
|
||||
|
||||
**Step 0's character half is done, 2026-08-13.** `eval/chapter-truth.json` holds 38 labelled
|
||||
occurrences keyed on page-space geometry, and all six baseline numbers print `= baseline`
|
||||
(`decisions/measurement-spine.md`). The ordering trap is closed. The file survives both the re-crop
|
||||
and the reset, so the fifth cycle can be measured against it. Run the audit before and after.
|
||||
|
||||
The dialogue half is not written and needs the chapter watched panel by panel, 30 lines with their
|
||||
true speaker. It is the only part of step 0 still open.
|
||||
|
||||
Steps 1 and 2 are independent and cheap: `plane` and `species` per detection, and the
|
||||
face-plus-person embedding already queued in item 1 above. The weak cannot-link is step 3 and depends
|
||||
on step 2, not on `plane` alone. Co-presence as a hard constraint is load-bearing while cosine cannot
|
||||
@@ -173,7 +196,8 @@ contain the right person. Coverage is still the `has_face` gate plus those refus
|
||||
(`caveats/speaker-attribution.md#tail-is-not-geometry`) is now behind item 5. With no name to attach,
|
||||
geometry buys nothing.
|
||||
11. Resolve a speaker answer across the whole dialogue window, not just the answering panel. The last 3
|
||||
unresolved refs describe a neighbouring panel in the same 8-panel call.
|
||||
unresolved refs describe a neighbouring panel in the same 8-panel call. The window is also the only
|
||||
way to reach the 28 lines that sit on skipped panels, measured above.
|
||||
12. Start Phase 2 from `ROADMAP.md`. Set SQLite `busy_timeout` before any concurrency work
|
||||
(`caveats/audit-open.md#sqlite-locking`).
|
||||
|
||||
|
||||
Reference in New Issue
Block a user