kami/manga-recap-pipeline
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

File the over-merge findings: two tracklet fixes, and cosine is not identity

Browse Source 
The user checked the lead's crops. LIM SEONHO's 36 assignments cover at least
six different men, a chibi and a cat, so 36 of 68 is a bug. 22 came from
native resolves and 14 from reconcile merges.

Two fixes are written in the orchestrator repo, tested, not deployed and not
run on a GPU. `link_tracklets` caps a tracklet's panel span, because `window`
bounded each pair while transitivity was unbounded and the lead's 22 native
assignments came from tracklets spanning 22 and 30 panels. And one shared
appearance tokenizer drops generic words, because whole chains hung on the
word `short` and one pair on the word `hair`.

The obvious third fix is ruled out by measurement. Over all 22 crop
embeddings, the cat scores up to 0.82 against men, two different men score
0.93, and the highest pair is 0.96. No threshold separates them, so
crop-to-crop cosine is not a link signal.

Item 1 of the agreed plan, sending the live cast instead of a cosine top-k
gallery, is not started.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
kami
2026-08-12 23:35:15 +04:00
parent 54bd126cd7
commit de7e34bc0f
7 changed files with 237 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files
caveats/CLAUDE.md
+1
View File
@@ -43,3 +43,4 @@ a complaint, so give it one or drop it.
| [Vision does not separate a background extra from cast](speaker-attribution.md#extras-as-cast) | 2026-08-12 panel 7 |
| [Cast reference profiles are enrolled from wrong crops](speaker-attribution.md#poisoned-reference-set) | 2026-08-12 panel 7 |
| [Identity coverage has fallen on every run since the gate landed](audit-open.md#coverage-trend) | before the next downstream run |
| [Embedding cosine on a person crop cannot tell two people apart](audit-open.md#cosine-not-identity) | before building on cosine |
caveats/audit-open.md
+25
View File
@@ -212,3 +212,28 @@ The plumbing works, the labels are missing. Precision and recall on the current
pass over `7c944dd4` first, via `POST /review/identity/label` per detection. Label against the bbox, not
the `local_id`. Vision is non-deterministic and reassigns `local_id` top-to-bottom every run. A label
taken before a vision rerun then points at whoever now occupies that slot.
## Embedding cosine on a person crop cannot tell two people apart {#cosine-not-identity}
Measured 2026-08-12 on the 18:07 run's 22 crops for `character_2367d70c`, the embeddings pulled from
`manga/<manga_id>/characters/_crops/*.npy` and L2-normalised. 1152 dimensions.
| pair | cosine |
| --- | --- |
| the cat at `p098` against any man | up to 0.82 |
| the red-robe chibi at `p088` against any man | up to 0.75 |
| two visibly different men (`order 60~80`, `88~110`) | 0.93, 0.92 |
| the highest pair in the matrix (`47~51`, one man, one shirt) | 0.96 |
Different people reach 0.93 and the same person reaches 0.96. No threshold separates them. So a
crop-to-crop cosine link is not available, and the `candidates` overlap condition in `link_tracklets` is
close to noise for the same reason: the top-k is chosen by this metric, so every crop shortlists the same
few rows.
Inference, not yet tested: identity embeds the whole person box, which holds background, clothing and pose.
Those change between scenes while every crop shares one art style, so cosine measures "manga crop of a
person in an office". `face_detect` already finds the face and pairs it to the person box for `has_face`,
so embedding the face region instead is a small change.
Revisit trigger: before building anything else on cosine. The test is to crop the faces of these same 22
detections, embed them, and recompute this matrix. If a threshold appears, embed faces.