Files
kami 63f7918a3e Record all six defects and the 9% baseline
Adds decision entries for the unpaired set-of-mark label, the interjection
verifier false positive, and the vision-blob clearing bug, plus the per-run
speaker audit script used to measure the chapter.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-11 23:59:32 +04:00

5.6 KiB

Speaker attribution and cast names

Settled 2026-08-11 from the quality cross-check of job 778297bc (JOURNAL.md, caveats/speaker-attribution.md). No GPU work ran and no pipeline run was executed after the change. Both claims rest on source and on the CPU-only self-checks named below.

Files: worker_vision.py on workpc, correctness.py and test_script_verify.py in the homesrv orchestrator (/mnt/server/home/kami/docker-apps/manga-infra/orchestrator/).

A model guess is never labelled tail

Closed. speaker_method names how a speaker was established, and nothing may claim geometry it did not read. _annotate_speaker_methods stamped tail, the highest-trust label, on any line whose speaker matched a local_id present in the panel, keeping gemma's confidence of 1.0. No balloon was ever consulted.

Evidence: three of three sampled two-character panels had both speakers swapped (caveats/speaker-attribution.md#tail-is-not-geometry). 31 of 81 speech lines carried tail with two or more characters present.

The label is gone. With two or more characters present the guess is dropped: speaker becomes unknown, confidence 0.0, method unknown. With one character present the claim equals the solo backstop, so it is kept as model_solo at confidence 0.7. Grounded som_face and solo_prior rows are untouched, because the function still skips any row that already carries a method.

Forbids: minting a provenance label for evidence that was not read, and shipping a multi-character attribution as truth before balloon geometry exists. Check: python worker_vision.py, the crowd/lone cases.

Cost: the named-speaker share will fall. The 30% headline was measured on attributions the sample says are wrong, so the lower number is the first honest one.

The model's speaker answer is resolved against what the prompt showed

Closed. build_dialogue_prompt renders a present character as - person_1: brown ponytail, green dress (f). gemma answers with any part of that line, so every part of it must map back to the local_id. It did not, and normalize_speaker classified each unmatched answer as a free-form name that no registry entry could match.

Evidence, measured on 36 panels of the cancelled first rerun. 15 lines carried a description and 9 a bare local_id with no identity assignment. 2 carried a stale P1 mark label, 2 a name with the gender marker attached. That is 28 of 51 speech lines. Only 3 resolved to a character_id.

_apply_speaker_labels now takes present and resolves the id, the name, the description, and each of those plus the gender marker. A key shared by two present characters is dropped, because it identifies neither. An id-shaped answer naming nobody present becomes unknown. A trailing gender marker is stripped, so an off-panel Seonho (m) still matches the registry name Seonho.

normalize_speaker refuses an id-shaped value independently, because the worker is a separate process and the contract is load-bearing (invariant 7).

Forbids: showing the model a label the worker cannot resolve back, and treating an unmatched speaker string as a name. Check: python worker_vision.py, the shown/twins cases. pytest test_correctness.py, test_an_id_shaped_speaker_is_never_a_name.

An unpaired mark grounds nothing

Closed. _set_of_mark labels a detected face unknown when gated pairing (decisions/audit-phase1.md#gated-face-pairing) matched it to no present character. An answer pointing at such a mark identifies nobody, so it must not carry som_face, the highest-trust provenance.

Evidence: all 7 som_face lines in the first 36 panels of the 2026-08-11 rerun had speaker_ref kind unknown. The label sat on lines with no speaker. Same defect class as the fake tail.

Forbids: deriving a provenance label from the label map without checking what the label resolved to. Check: python worker_vision.py, the unpaired case.

An interjection is not a name and not a misquote

Closed. verify_script must stay quiet on valid narration, because run_stage_script retries once and then raises (decisions/audit-phase1.md#verifier-false-positives).

Evidence: the narrator wrote "...Hm?" for the source line "Uh... hum...?". Both rules fired at once. Hm was absent from the source words, and a 6-character quote needs 5 matching characters to ground, so it got 4. That halted the script stage at 112/116 on job 778297bc.

Interjections join _STOPWORDS. A quote of three letters or fewer is no longer grounded-checked. That holds the same line as the dialogue prompt's 1-3 character noise rule. The stage then passed 116/116.

Forbids: scoring a quote too short for the ratio to carry meaning. Check: pytest test_script_verify.py, test_an_interjection_is_not_a_name_or_a_misquote.

Cast names enter the verifier tokenized

Closed. verify_script compares single capitalized tokens, so every allowed name must be present as tokens. allowed was built from cast_names verbatim, which put "choi haeseon" in the set as one string while the checker looked up Choi and Haeseon separately.

Evidence: the script stage failed at 87/116 on job 778297bc. 28 of the 29 lost beats cite unsupported-proper-noun: ['Choi', 'Haeseon']. A one-word name such as Seonho always passed, which is why this survived the Phase 1 verifier work (decisions/audit-phase1.md#verifier-false-positives).

Forbids: adding any future allow-list to verify_script as whole strings. Check: pytest test_script_verify.py, test_multiword_cast_name_is_supported.