kami/manga-recap-pipeline
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
master
manga-recap-pipeline/eval/chapter-truth.json
T
kami bc19f9e9fd Measure the registry against the run it describes 
ARCHITECTURE.md step 0. eval/chapter-truth.json holds the 38 occurrences the
user walked crop by crop on the 19:44 run of 2026-08-12, and audit_registry.py
now prints purity per cluster and fragmentation per person against it. All six
baseline numbers reproduce.

Rows key on page-space geometry, purity is a share, and fragmentation is a
count of ids, so nothing in the file names a panel_id or a character_id. The
fifth cycle re-crops and calls /characters/reset, and the file survives both.
That was the ordering trap in the handoff.

NEXT.md said 2 of woman B's 9 crops were really woman A and never said which.
They are panel_order 31 and 33, identified from p030 and p032.

Four fixes to the audit itself, all pre-existing:

- 20 characters counted where 14 are live and 6 are merge losers kept on purpose
- the assignment spread keyed on name, so the two Seonhos summed into one line
- the default worked example was panel_index 7, a panel vision skips. NEXT.md's
  "panel 7" is panel_order 7, one lower
- nothing about skipped panels. 41 of 116 are skip=True, four checked and all
  four correct, and they hold 28 of the chapter's 122 dialogue lines

That last count is the measured case for an offscreen speaker_ref kind: 23% of
dialogue sits on panels with no character to attribute to.

Checks: audit_registry.py --selftest covers the IoU match, the greedy tie-break
and the purity maths with no database. ruff check . exits 0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-13 23:32:05 +04:00

670 lines
14 KiB
JSON
Raw Permalink Blame History

{
"chapter_id": "7c944dd4-e972-42c7-ba60-9f6939548e80",
"manga_id": "ef105a86-4b7e-4ac4-b45c-b7d83b8f5b5e",
"source": "the eyeball pass over the 19:44 UTC run of 2026-08-12, recorded as prose in NEXT.md. The two woman_a crops sitting on woman_b's id were unnamed there and were identified on 2026-08-13 by re-cropping p030 and p032 from s3://panels.",
"scope": "three characters walked crop by crop, 38 of the run's 60 assignments. The other 10 characters were never checked by eye and carry no truth row.",
"keyed_on": "page-space pixel box, [x1,y1,x2,y2] = the panel's bbox offset plus the detection box, matched by IoU. Not panel_id and not character_id: a re-crop remints panel ids and /characters/reset remints character ids, and the fifth cycle does both.",
"people": {
"lead": {
"label": "the lead, named LIM SEONHO in the 19:44 registry",
"gender": "m"
},
"woman_a": {
"label": "woman A, black bob, cream top with black trim, pearl necklace. A main character the 19:44 registry never named. Expected to become Choi Haeseon from the p040 caption.",
"gender": "f"
},
"woman_b": {
"label": "woman B, the green-dress woman, brown side ponytail with a scrunchie. Wrongly named Seonho in the 19:44 registry.",
"gender": "f"
}
},
"not_people": {
"photo_of_lead": "a photograph of the lead inside a panel",
"chibi_of_lead": "a chibi drawing of the lead on a monitor"
},
"baseline": {
"run": "2026-08-12 19:44-19:52 UTC, on a registry reset beforehand",
"detections": 119,
"assignments": 60,
"clusters": {
"lead": {
"assignments": 16,
"correct": 14,
"purity": 0.88
},
"woman_a": {
"assignments": 13,
"correct": 13,
"purity": 1.0
},
"woman_b": {
"assignments": 9,
"correct": 7,
"purity": 0.78
}
},
"fragmentation": {
"lead": 1,
"woman_a": 2,
"woman_b": 1
}
},
"occurrences": [
{
"page": 0,
"box": [
0,
53814,
810,
54790
],
"person": "woman_a",
"from_19_44": {
"panel_order": 38,
"local_id": "person_1",
"character_id": "character_2b1b12a1",
"confidence": 1.0
}
},
{
"page": 0,
"box": [
0,
56215,
862,
57486
],
"person": "woman_a",
"from_19_44": {
"panel_order": 40,
"local_id": "person_1",
"character_id": "character_2b1b12a1",
"confidence": 1.0
}
},
{
"page": 0,
"box": [
0,
60385,
900,
61496
],
"person": "woman_a",
"from_19_44": {
"panel_order": 43,
"local_id": "person_1",
"character_id": "character_2b1b12a1",
"confidence": 1.0
}
},
{
"page": 0,
"box": [
496,
66848,
620,
67051
],
"person": "woman_a",
"from_19_44": {
"panel_order": 48,
"local_id": "person_2",
"character_id": "character_2b1b12a1",
"confidence": 1.0
}
},
{
"page": 0,
"box": [
58,
68579,
760,
69375
],
"person": "woman_a",
"from_19_44": {
"panel_order": 49,
"local_id": "person_1",
"character_id": "character_2b1b12a1",
"confidence": 1.0
}
},
{
"page": 0,
"box": [
285,
78844,
409,
79211
],
"person": "woman_a",
"from_19_44": {
"panel_order": 57,
"local_id": "person_1",
"character_id": "character_2b1b12a1",
"confidence": 0.9
}
},
{
"page": 0,
"box": [
0,
82703,
757,
83432
],
"person": "woman_a",
"from_19_44": {
"panel_order": 59,
"local_id": "person_1",
"character_id": "character_2b1b12a1",
"confidence": 0.9
}
},
{
"page": 0,
"box": [
310,
86176,
688,
87211
],
"person": "woman_a",
"from_19_44": {
"panel_order": 61,
"local_id": "person_1",
"character_id": "character_2b1b12a1",
"confidence": 0.9
}
},
{
"page": 0,
"box": [
628,
90916,
898,
91743
],
"person": "woman_a",
"from_19_44": {
"panel_order": 65,
"local_id": "person_2",
"character_id": "character_2b1b12a1",
"confidence": 0.9
}
},
{
"page": 0,
"box": [
0,
94674,
844,
95892
],
"person": "woman_a",
"from_19_44": {
"panel_order": 67,
"local_id": "person_1",
"character_id": "character_2b1b12a1",
"confidence": 1.0
}
},
{
"page": 0,
"box": [
0,
120322,
898,
121699
],
"person": "woman_a",
"from_19_44": {
"panel_order": 86,
"local_id": "person_1",
"character_id": "character_2b1b12a1",
"confidence": 1.0
}
},
{
"page": 0,
"box": [
0,
151177,
539,
152023
],
"person": "woman_a",
"from_19_44": {
"panel_order": 108,
"local_id": "person_1",
"character_id": "character_2b1b12a1",
"confidence": 1.0
}
},
{
"page": 0,
"box": [
0,
157221,
862,
158723
],
"person": "woman_a",
"from_19_44": {
"panel_order": 111,
"local_id": "person_1",
"character_id": "character_2b1b12a1",
"confidence": 0.897
}
},
{
"page": 0,
"box": [
0,
8758,
900,
10118
],
"person": "lead",
"from_19_44": {
"panel_order": 10,
"local_id": "person_1",
"character_id": "character_b1dd5659",
"confidence": 0.0
}
},
{
"page": 0,
"box": [
412,
11199,
900,
11986
],
"person": "lead",
"from_19_44": {
"panel_order": 11,
"local_id": "person_2",
"character_id": "character_b1dd5659",
"confidence": 0.9
}
},
{
"page": 0,
"box": [
523,
16210,
716,
16532
],
"person": "lead",
"from_19_44": {
"panel_order": 13,
"local_id": "person_2",
"character_id": "character_b1dd5659",
"confidence": 0.826
}
},
{
"page": 0,
"box": [
454,
20833,
715,
21006
],
"person": "photo_of_lead",
"from_19_44": {
"panel_order": 17,
"local_id": "person_1",
"character_id": "character_b1dd5659",
"confidence": 0.9
},
"note": "a photograph of him inside the panel, not him"
},
{
"page": 0,
"box": [
202,
22420,
900,
23566
],
"person": "lead",
"from_19_44": {
"panel_order": 18,
"local_id": "person_1",
"character_id": "character_b1dd5659",
"confidence": 1.0
}
},
{
"page": 0,
"box": [
0,
24339,
899,
25041
],
"person": "lead",
"from_19_44": {
"panel_order": 19,
"local_id": "person_1",
"character_id": "character_b1dd5659",
"confidence": 1.0
}
},
{
"page": 0,
"box": [
30,
26777,
202,
27204
],
"person": "chibi_of_lead",
"from_19_44": {
"panel_order": 20,
"local_id": "person_3",
"character_id": "character_b1dd5659",
"confidence": 1.0
},
"note": "a chibi of him on a monitor, art inside the panel"
},
{
"page": 0,
"box": [
106,
51044,
741,
51945
],
"person": "lead",
"from_19_44": {
"panel_order": 36,
"local_id": "person_1",
"character_id": "character_b1dd5659",
"confidence": 0.9
}
},
{
"page": 0,
"box": [
26,
64499,
730,
65435
],
"person": "lead",
"from_19_44": {
"panel_order": 47,
"local_id": "person_1",
"character_id": "character_b1dd5659",
"confidence": 1.0
}
},
{
"page": 0,
"box": [
382,
66803,
464,
67054
],
"person": "lead",
"from_19_44": {
"panel_order": 48,
"local_id": "person_1",
"character_id": "character_b1dd5659",
"confidence": 0.888
}
},
{
"page": 0,
"box": [
97,
71438,
619,
72061
],
"person": "lead",
"from_19_44": {
"panel_order": 51,
"local_id": "person_1",
"character_id": "character_b1dd5659",
"confidence": 1.0
}
},
{
"page": 0,
"box": [
463,
91111,
675,
91494
],
"person": "lead",
"from_19_44": {
"panel_order": 65,
"local_id": "person_1",
"character_id": "character_b1dd5659",
"confidence": 1.0
}
},
{
"page": 0,
"box": [
0,
97794,
900,
98950
],
"person": "lead",
"from_19_44": {
"panel_order": 69,
"local_id": "person_1",
"character_id": "character_b1dd5659",
"confidence": 1.0
}
},
{
"page": 0,
"box": [
251,
110663,
788,
112008
],
"person": "lead",
"from_19_44": {
"panel_order": 80,
"local_id": "person_1",
"character_id": "character_b1dd5659",
"confidence": 1.0
}
},
{
"page": 0,
"box": [
316,
114988,
653,
115534
],
"person": "lead",
"from_19_44": {
"panel_order": 82,
"local_id": "person_1",
"character_id": "character_b1dd5659",
"confidence": 1.0
}
},
{
"page": 0,
"box": [
0,
123335,
899,
124244
],
"person": "lead",
"from_19_44": {
"panel_order": 88,
"local_id": "person_1",
"character_id": "character_b1dd5659",
"confidence": 1.0
}
},
{
"page": 0,
"box": [
648,
4620,
767,
4830
],
"person": "woman_b",
"from_19_44": {
"panel_order": 7,
"local_id": "person_6",
"character_id": "character_f0d4e901",
"confidence": 1.0
}
},
{
"page": 0,
"box": [
94,
11408,
540,
11986
],
"person": "woman_b",
"from_19_44": {
"panel_order": 11,
"local_id": "person_1",
"character_id": "character_f0d4e901",
"confidence": 0.0
}
},
{
"page": 0,
"box": [
350,
15975,
437,
16256
],
"person": "woman_b",
"from_19_44": {
"panel_order": 13,
"local_id": "person_1",
"character_id": "character_f0d4e901",
"confidence": 1.0
}
},
{
"page": 0,
"box": [
0,
35632,
421,
36197
],
"person": "woman_b",
"from_19_44": {
"panel_order": 26,
"local_id": "person_1",
"character_id": "character_f0d4e901",
"confidence": 0.933
}
},
{
"page": 0,
"box": [
79,
38398,
390,
38652
],
"person": "woman_b",
"from_19_44": {
"panel_order": 28,
"local_id": "person_1",
"character_id": "character_f0d4e901",
"confidence": 1.0
},
"note": "a chibi of her as an emote, counted as her. Distinct from chibi_of_lead, which is a drawing inside the scene rather than a stylised rendering of the subject."
},
{
"page": 0,
"box": [
0,
41191,
435,
42178
],
"person": "woman_b",
"from_19_44": {
"panel_order": 29,
"local_id": "person_1",
"character_id": "character_f0d4e901",
"confidence": 1.0
}
},
{
"page": 0,
"box": [
322,
43342,
886,
44191
],
"person": "woman_b",
"from_19_44": {
"panel_order": 30,
"local_id": "person_1",
"character_id": "character_f0d4e901",
"confidence": 0.943
}
},
{
"page": 0,
"box": [
471,
45263,
525,
45385
],
"person": "woman_a",
"from_19_44": {
"panel_order": 31,
"local_id": "person_3",
"character_id": "character_f0d4e901",
"confidence": 1.0
},
"note": "woman_a in her cream top, verified 2026-08-13 on a p030 zoom"
},
{
"page": 0,
"box": [
81,
47159,
899,
48564
],
"person": "woman_a",
"from_19_44": {
"panel_order": 33,
"local_id": "person_1",
"character_id": "character_f0d4e901",
"confidence": 0.853
},
"note": "woman_a, unmistakable at full panel width"
}
],
"dialogue": [],
"dialogue_todo": "ARCHITECTURE.md's spine also asks for 30 dialogue lines with their true speaker, typed visible | offscreen | narrator | unknown. No such eyeball pass has been done: NEXT.md records no per-line speaker truth, and the dialogue stage is cleared, so there is nothing to transcribe. It has to be watched panel by panel and written here."
}
Reference in New Issue View Git Blame Copy Permalink