Adds decision entries for the unpaired set-of-mark label, the interjection verifier false positive, and the vision-blob clearing bug, plus the per-run speaker audit script used to measure the chapter. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
5.6 KiB
Speaker attribution and cast names
Settled 2026-08-11 from the quality cross-check of job 778297bc
(JOURNAL.md, caveats/speaker-attribution.md). No GPU work ran and no pipeline run was executed
after the change. Both claims rest on source and on the CPU-only self-checks named below.
Files: worker_vision.py on workpc, correctness.py and test_script_verify.py in the homesrv
orchestrator (/mnt/server/home/kami/docker-apps/manga-infra/orchestrator/).
A model guess is never labelled tail
Closed. speaker_method names how a speaker was established, and nothing may claim geometry it did
not read. _annotate_speaker_methods stamped tail, the highest-trust label, on any line whose
speaker matched a local_id present in the panel, keeping gemma's confidence of 1.0. No balloon was
ever consulted.
Evidence: three of three sampled two-character panels had both speakers swapped
(caveats/speaker-attribution.md#tail-is-not-geometry). 31 of 81 speech lines carried tail with two
or more characters present.
The label is gone. With two or more characters present the guess is dropped: speaker becomes
unknown, confidence 0.0, method unknown. With one character present the claim equals the solo
backstop, so it is kept as model_solo at confidence 0.7. Grounded som_face and solo_prior rows are
untouched, because the function still skips any row that already carries a method.
Forbids: minting a provenance label for evidence that was not read, and shipping a multi-character
attribution as truth before balloon geometry exists.
Check: python worker_vision.py, the crowd/lone cases.
Cost: the named-speaker share will fall. The 30% headline was measured on attributions the sample says are wrong, so the lower number is the first honest one.
The model's speaker answer is resolved against what the prompt showed
Closed. build_dialogue_prompt renders a present character as
- person_1: brown ponytail, green dress (f). gemma answers with any part of that line, so every part
of it must map back to the local_id. It did not, and normalize_speaker classified each unmatched
answer as a free-form name that no registry entry could match.
Evidence, measured on 36 panels of the cancelled first rerun. 15 lines carried a description and 9 a bare
local_id with no identity assignment. 2 carried a stale P1 mark label, 2 a name with the gender
marker attached. That is 28 of 51 speech lines. Only 3 resolved to a character_id.
_apply_speaker_labels now takes present and resolves the id, the name, the description, and each of
those plus the gender marker. A key shared by two present characters is dropped, because it identifies
neither. An id-shaped answer naming nobody present becomes unknown. A trailing gender marker is
stripped, so an off-panel Seonho (m) still matches the registry name Seonho.
normalize_speaker refuses an id-shaped value independently, because the worker is a separate process
and the contract is load-bearing (invariant 7).
Forbids: showing the model a label the worker cannot resolve back, and treating an unmatched speaker
string as a name.
Check: python worker_vision.py, the shown/twins cases. pytest test_correctness.py,
test_an_id_shaped_speaker_is_never_a_name.
An unpaired mark grounds nothing
Closed. _set_of_mark labels a detected face unknown when gated pairing
(decisions/audit-phase1.md#gated-face-pairing) matched it to no present character. An answer pointing
at such a mark identifies nobody, so it must not carry som_face, the highest-trust provenance.
Evidence: all 7 som_face lines in the first 36 panels of the 2026-08-11 rerun had speaker_ref kind
unknown. The label sat on lines with no speaker. Same defect class as the fake tail.
Forbids: deriving a provenance label from the label map without checking what the label resolved to.
Check: python worker_vision.py, the unpaired case.
An interjection is not a name and not a misquote
Closed. verify_script must stay quiet on valid narration, because run_stage_script retries once
and then raises (decisions/audit-phase1.md#verifier-false-positives).
Evidence: the narrator wrote "...Hm?" for the source line "Uh... hum...?". Both rules fired at once.
Hm was absent from the source words, and a 6-character quote needs 5 matching characters to ground,
so it got 4. That halted the script stage at 112/116 on job 778297bc.
Interjections join _STOPWORDS. A quote of three letters or fewer is no longer grounded-checked. That
holds the same line as the dialogue prompt's 1-3 character noise rule. The stage then passed 116/116.
Forbids: scoring a quote too short for the ratio to carry meaning.
Check: pytest test_script_verify.py, test_an_interjection_is_not_a_name_or_a_misquote.
Cast names enter the verifier tokenized
Closed. verify_script compares single capitalized tokens, so every allowed name must be present as
tokens. allowed was built from cast_names verbatim, which put "choi haeseon" in the set as one
string while the checker looked up Choi and Haeseon separately.
Evidence: the script stage failed at 87/116 on job 778297bc. 28 of the 29 lost beats cite
unsupported-proper-noun: ['Choi', 'Haeseon']. A one-word name such as Seonho always passed, which
is why this survived the Phase 1 verifier work
(decisions/audit-phase1.md#verifier-false-positives).
Forbids: adding any future allow-list to verify_script as whole strings.
Check: pytest test_script_verify.py, test_multiword_cast_name_is_supported.