Measure the registry against the run it describes
ARCHITECTURE.md step 0. eval/chapter-truth.json holds the 38 occurrences the user walked crop by crop on the 19:44 run of 2026-08-12, and audit_registry.py now prints purity per cluster and fragmentation per person against it. All six baseline numbers reproduce. Rows key on page-space geometry, purity is a share, and fragmentation is a count of ids, so nothing in the file names a panel_id or a character_id. The fifth cycle re-crops and calls /characters/reset, and the file survives both. That was the ordering trap in the handoff. NEXT.md said 2 of woman B's 9 crops were really woman A and never said which. They are panel_order 31 and 33, identified from p030 and p032. Four fixes to the audit itself, all pre-existing: - 20 characters counted where 14 are live and 6 are merge losers kept on purpose - the assignment spread keyed on name, so the two Seonhos summed into one line - the default worked example was panel_index 7, a panel vision skips. NEXT.md's "panel 7" is panel_order 7, one lower - nothing about skipped panels. 41 of 116 are skip=True, four checked and all four correct, and they hold 28 of the chapter's 122 dialogue lines That last count is the measured case for an offscreen speaker_ref kind: 23% of dialogue sits on panels with no character to attribute to. Checks: audit_registry.py --selftest covers the IoU match, the greedy tie-break and the purity maths with no database. ruff check . exits 0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -54,3 +54,6 @@ still live belongs in `caveats/`.
|
||||
| [A name is a word set, not a string](identity-naming.md#alias-grouping) | closed, GPU pending |
|
||||
| [A confident caption names a character on its own](identity-naming.md#caption-is-strong) | closed, GPU pending |
|
||||
| [A name belongs to one character](identity-naming.md#one-name-one-character) | closed, GPU pending |
|
||||
| [The truth file keys on page-space geometry](measurement-spine.md#truth-is-page-geometry) | closed |
|
||||
| [Purity is a share, so it never names an id](measurement-spine.md#purity-is-id-free) | closed |
|
||||
| [The truth is 38 assignments, and holds no dialogue yet](measurement-spine.md#truth-scope-is-38) | open |
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
# The measurement spine
|
||||
|
||||
`eval/chapter-truth.json` plus the purity and fragmentation section of `audit_registry.py`. This is
|
||||
`ARCHITECTURE.md` step 0, and nothing below it in that build order is measurable without it.
|
||||
|
||||
## The truth file keys on page-space geometry {#truth-is-page-geometry}
|
||||
|
||||
**Closed, 2026-08-13.** Reproduces the baseline on the 19:44 run of 2026-08-12.
|
||||
|
||||
A truth row cannot name a `panel_id` or a `character_id`. A re-crop remints panel ids, and
|
||||
`/characters/reset` remints character ids. The fifth cycle does both, so a truth file keyed on either
|
||||
would die the moment it was first used.
|
||||
|
||||
Each row carries `[x1, y1, x2, y2]` in page space, which is the panel's `bbox` offset plus the
|
||||
detection box. The page strip is the fetched artifact and does not change when panels are re-sliced, so
|
||||
the key survives. `audit_registry.py:_score` matches a row to an assignment on the same page at
|
||||
`IOU_MIN = 0.5`, greedily, best overlap first, each row and each assignment used once.
|
||||
|
||||
The 19:44 `panel_order`, `local_id` and `character_id` are kept per row under `from_19_44`, for
|
||||
debugging only. Nothing matches on them.
|
||||
|
||||
## Purity is a share, so it never names an id {#purity-is-id-free}
|
||||
|
||||
**Closed, 2026-08-13.**
|
||||
|
||||
Purity of a cluster is the largest share of one true person inside it. The lead's 16 assignments hold
|
||||
14 of him, so 0.88, and the truth file never has to say which id he was. Fragmentation of a person is
|
||||
the number of clusters holding their occurrences. Woman A has 15 occurrences over 2 ids, so 2.
|
||||
|
||||
So both numbers survive a reset. The definitions in `ARCHITECTURE.md` were not explicit about that.
|
||||
A cluster is reported under its dominant person. Every other person in it prints as a `wrong:` line,
|
||||
which is where the photograph and the chibi show up on the lead.
|
||||
|
||||
The `baseline` block in the truth file holds the 19:44 numbers, and each printed line says
|
||||
`= baseline` or the delta. That makes step 0's DoD a check rather than an eyeball.
|
||||
|
||||
## The truth is 38 assignments, and holds no dialogue yet {#truth-scope-is-38}
|
||||
|
||||
**Open, 2026-08-13.** The character half is done, the dialogue half is not written.
|
||||
|
||||
The eyeball pass covered three characters, 38 of the run's 60 assignments. The other 10 characters were
|
||||
never checked and carry no truth row. The audit prints them as out of scope rather than as correct.
|
||||
Anything claiming chapter-wide accuracy off this file is overreaching.
|
||||
|
||||
The prose in `NEXT.md` left two rows unnamed. They were identified on 2026-08-13 by re-cropping `p030`
|
||||
and `p032`: `panel_order` 31 and 33 on woman B's id are both woman A. That closes the recorded
|
||||
arithmetic of 7 correct and 2 wrong, and it is why woman A's fragmentation is 2.
|
||||
|
||||
One judgement is recorded in the file rather than hidden. The chibi of woman B at `panel_order` 28
|
||||
counts as her, because it is a stylised rendering of the subject. The chibi of the lead at
|
||||
`panel_order` 20 does not, because it is a drawing inside the scene. `not_people` names the second kind.
|
||||
|
||||
`ARCHITECTURE.md` also asks for 30 dialogue lines with their true speaker, typed
|
||||
`visible | offscreen | narrator | unknown`. `dialogue` is an empty array. No such pass exists, because
|
||||
`NEXT.md` records no per-line speaker truth. Writing it means watching the chapter panel by panel. The
|
||||
audit prints nothing about speakers until it is filled.
|
||||
Reference in New Issue
Block a user