Files

T

kami a9d64fe80a Record panel 7 against the art, and the A/V gap it hid

Somebody watched chapter.mp4 for the first time. Two failures came out of it
that no stage counter could see.

chapter.mp4 is video 436.39s over audio 363.67s, so narration finishes 72.7s
before the picture. The 49 clips are clean: all 25fps, video and audio agree
to 0.03s, summing to 363.6s. A per-round probe puts the loss in the final
round of _assemble_batched, which turns 359s of video into 100s while the
audio survives. Round 0 is correct. Round 1 differs by holding a 7th input,
the leftover clip that skips encoding, so the tree mixes concat output, xfade
output and a raw clip. Not fixed.

worker_render.py gains an FPS constant, fps normalization in the xfade branch
to match concat, _stream_dur, and a self-check that compares video against
audio instead of asserting the file is non-empty. That old check is how a 20%
sync failure shipped. The fps inconsistency is real but not proven to be the
shipped cause. Pinning -r on the output was tried and reverted: it drops
frames to force CFR, which the concat branch comment already warned about.

Panel 7 checked against the art has zero correct identity bindings out of two,
and Seonho, the one character who matters, is unbound. bbox values are
consumed as absolute pixels; on a 900x1650 panel that puts all six boxes in
the top third, two inside a speech balloon. Identity therefore embeds crops of
balloon edges and window frames, which is how confidence 0.9 lands on the
wrong person. The colleague has no name in the story and was labelled Choi
Haeseon; that row holds 25 of 26 assignments, so it is the label the pipeline
stamps on any unnamed woman.

Four caveats added. Two earlier claims are withdrawn in place: rescaling bbox
by 1000 does not make the boxes correct, and the constraint is not 16 nameless
rows needing names. Cast profiles already exist, since all 53 rows populate
ref_image_uris and embedding_uri, but they are enrolled from the wrong crops.

worker_render.py self-check passes. No pipeline ran.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

2026-08-12 01:07:17 +04:00

9.1 KiB

Raw Blame History

speaker-attribution

Limits found by cross-checking the 2026-08-11 chapter run against the panel images.

Nothing attributes a speaker in a multi-character panel

The false tail label is gone (decisions/speaker-attribution.md#no-fake-tail). What replaced it is a refusal, not an answer: with two or more characters present, every speech line now returns unknown. On a chapter like this one that costs 31 of 81 speech lines their speaker. The narration then falls back to a generic-handle. That is the honest floor, and it is not the fix.

The measurement that forced it, on job 778297bc, chapter 7c944dd4. Three two-character panels were checked against the art. All three are wrong, each with the two speakers swapped:

panel key	line	truth	pipeline
`p010.png`	"…definitely an Egen guy, Seonho!"	the woman	Seonho, the person addressed
`p010.png`	"Y-you think so?"	Seonho	Choi Haeseon
`p012.png`	"Want me to send you the link?"	the woman	the man
`p059.png`	"If team leader Choi says it, it must be true."	the man	Choi Haeseon

The last row needs no image: the line refers to Choi in the third person and is attributed to Choi.

The grounded path exists and almost never fires. Only 2 of 81 speech lines got som_face, because attribution marks need face_detect boxes that survive _pair_faces_to_present, and these webtoon close-ups rarely produce them. Inference, not measured: the face detector was not instrumented.

worker_vision.py:356 already carries the ponytail: note that multi-character attribution needs per-balloon geometry. bubble_detect.py:9 records that the det/seg heads carry balloon fill and tail tips and are unused.

Revisit trigger: the share of narrated lines with a named speaker is the Phase 1 headline metric (ROADMAP.md). The 30% read on this run counted attributions the sample says are wrong. The next run will read lower and will be the first honest number. Raising it means binding a balloon to a speaker by tail geometry, using the unused det/seg heads.

One character id covers two different women

character_afa7623b is stored as "black bob, white sweater" and is assigned both to that person (p059.png) and to the brown-bob green-top coworker (p010.png, p012.png). It took 25 of the 26 identity assignments in the chapter, against 113 detected people. Coverage is 23%.

Revisit trigger: any work on the identity Tier-2 decider. A single id absorbing a whole chapter is the signature to watch for.

The character registry carries five weeks of wrong names

The registry holds 53 characters for manga ef105a86, 41 of them unnamed, with "Kei" three times and "Kanade" twice. Kei, Kanade, Zen, Rico, K3, and Watanabe occur zero times in this chapter's text. Only Haeseon and Seonho do. /stage/clear leaves the per-manga registry intact by design, so every rerun inherits the whole pile.

Duplicate rows also make a correct name unresolvable. This manga holds Choi Haeseon, Seonho with aliases ["Lim Seonho", "Seonho"], and a separate Lim Seonho. An answer of "Lim Seonho" matches two rows, so normalize_speaker returns candidates and raises ambiguous-speaker instead of binding. The pipeline read the name correctly and still cannot name the speaker.

Revisit trigger: before any run that is meant to produce a clean baseline. Either scope the registry to a chapter or add a reviewed reset. Merging the duplicate rows needs the reversible-merge design first (caveats/audit-open.md#destructive-reconcile).

One invented word still halts the chapter

The multi-word name failure is fixed (decisions/speaker-attribution.md#multiword-cast-names). The blast radius it exposed is not. run_stage_script retries a rejected beat once, then raises, so a single unsupported token ends the run at that beat. On job 778297bc one of the 29 lost beats cited ['Blur'], an onomatopoeia the model invented. The verifier was right, and the whole chapter still stopped.

Revisit trigger: the next unsupported-proper-noun halt that is a true positive. The likely answer is to flag the beat for review and continue, which is #136 gate work, not a verifier change.

Every `bbox` is read in the wrong coordinate space

Vision's bbox values are stored and consumed as absolute pixels. On panel 7c944dd4-e972-42c7-ba60-9f6939548e80_p007 (crop 900x1650) all six boxes then land in the top third of the panel, two of them inside the "YEAH!" speech balloon. Divided by 1000 against the panel's own dimensions, four of the six fit their subjects tightly.

Two places in the code assert pixels, and the art contradicts both:

worker_vision.py:271, prompt text: pixel bounding box [x1,y1,x2,y2] (top-left, bottom-right corners)
worker_identity.py:91, comment: vision emits [x1, y1, x2, y2] pixel corners (gemma4's native bbox convention)

Who pays: identity embeds _crop_bbox(img, ch["bbox"]) at worker_identity.py:200, so it matches faces against crops of balloons and window frames. On panel 7 that produced Choi Haeseon at confidence 0.9 from a crop of a balloon edge and Lim Seonho at 0.9 from an empty window frame. Blank crops embed alike, which is a plausible mechanism for one row absorbing 25 of 26 assignments. The face pairing at worker_vision.py:57 reads the same numbers and was not checked.

Rescaling is necessary and not sufficient. After scaling, person_1 still sits on a window frame with nobody in it, and person_6 clips its subject and runs onto the frame. worker_vision.py:38 already calls the box "coarse, imprecise".

Revisit trigger: before any further identity or balloon-geometry work. Nothing downstream of bbox can be judged while the crops are wrong.

Identity cannot say "a person with no name"

The colleague on panel 7 has no name in the story. She was assigned Choi Haeseon at confidence 0.9. Across the chapter that row holds 25 of 26 assignments, so in practice it is the label the pipeline stamps on any unnamed woman. Narration then calls her Choi Haeseon and inherits that row's gender, which is the direct cause of the user's 0:20 and 1:51 notes and of the gender flips at 1:45 and the closing line.

This is the same shape as invariant 6 in CLAUDE.md, which forbids minting a character from an unparseable model answer. The missing rule: never attach a name to a detection that carries no name evidence. A recurring unnamed person needs a stable anonymous identity, so narration says "the colleague" every time.

match() at worker_identity.py:69 does abstain, returning None below threshold, so the 0.9 came from cosine clearing the threshold on a wrong crop. Whether the Tier-2 gemma resolver can answer "none of these" was not verified.

Revisit trigger: immediately after the bbox space is settled.

Vision does not separate a background extra from cast

Panel 7 is a wide establishing shot. Vision emitted 6 characters. Two matter: Seonho in the foreground and the unnamed colleague. Three are background office extras, and one (person_1) is a window frame with nobody in it. All six reach identity as equal candidates.

Who pays: the roadmap's framing figure, "26 of 113 detected people carry an identity", counted mostly extras, so it measured nothing useful and should not be quoted again.

Revisit trigger: with #no-anonymous-identity, since both change what identity is allowed to return.

Cast reference profiles are enrolled from wrong crops

characters carries ref_image_uris and embedding_uri, and all 53 rows have both populated. So the cast-profile mechanism exists. It is enrolled through #bbox-wrong-space, so the stored references are crops of balloon edges, window frames and background extras rather than of faces.

The visual comparison people reach for as the fix is already implemented, so do not build it again. /vision/resolve at worker_vision.py:963 sends the query crop plus up to 3 labelled reference images per candidate. build_resolve_prompt already tells the model to judge face shape first, to treat hair and outfit as secondary, that two people sharing a hair colour are not the same, and to answer 0 for NONE when unsure. choice: 0 becomes a new character and an out-of-range index becomes unresolved. The ref_image_uris column is republished as reference_image_uris at worker_identity.py:152 and :161, so the references reach the model.

That is why this caveat is about the pixels and not the prompt. The resolver compares a crop of a balloon edge against references enrolled from window frames and background extras, then sometimes answers "same". Nothing gates enrollment on the crop holding a face.

Who pays: every later match, because the reference set defines what a character looks like. Fixing #bbox-wrong-space without re-enrolling leaves the poisoned references in place.

Revisit trigger: as soon as #bbox-wrong-space lands, re-enroll from corrected crops and treat the existing ref_image_uris and embedding_uri values as invalid.

9.1 KiB Raw Blame History