Files
manga-recap-pipeline/decisions/speaker-attribution.md
T
kami 63f7918a3e Record all six defects and the 9% baseline
Adds decision entries for the unpaired set-of-mark label, the interjection
verifier false positive, and the vision-blob clearing bug, plus the per-run
speaker audit script used to measure the chapter.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-11 23:59:32 +04:00

97 lines
5.6 KiB
Markdown

# Speaker attribution and cast names
Settled 2026-08-11 from the quality cross-check of job `778297bc`
(`JOURNAL.md`, `caveats/speaker-attribution.md`). No GPU work ran and no pipeline run was executed
after the change. Both claims rest on source and on the CPU-only self-checks named below.
Files: `worker_vision.py` on workpc, `correctness.py` and `test_script_verify.py` in the homesrv
orchestrator (`/mnt/server/home/kami/docker-apps/manga-infra/orchestrator/`).
## A model guess is never labelled `tail` {#no-fake-tail}
**Closed.** `speaker_method` names how a speaker was established, and nothing may claim geometry it did
not read. `_annotate_speaker_methods` stamped `tail`, the highest-trust label, on any line whose
speaker matched a `local_id` present in the panel, keeping gemma's confidence of 1.0. No balloon was
ever consulted.
Evidence: three of three sampled two-character panels had both speakers swapped
(`caveats/speaker-attribution.md#tail-is-not-geometry`). 31 of 81 speech lines carried `tail` with two
or more characters present.
The label is gone. With two or more characters present the guess is dropped: `speaker` becomes
`unknown`, confidence 0.0, method `unknown`. With one character present the claim equals the solo
backstop, so it is kept as `model_solo` at confidence 0.7. Grounded `som_face` and `solo_prior` rows are
untouched, because the function still skips any row that already carries a method.
Forbids: minting a provenance label for evidence that was not read, and shipping a multi-character
attribution as truth before balloon geometry exists.
Check: `python worker_vision.py`, the `crowd`/`lone` cases.
Cost: the named-speaker share will fall. The 30% headline was measured on attributions the sample says
are wrong, so the lower number is the first honest one.
## The model's speaker answer is resolved against what the prompt showed {#prompt-label-answers}
**Closed.** `build_dialogue_prompt` renders a present character as
`- person_1: brown ponytail, green dress (f)`. gemma answers with any part of that line, so every part
of it must map back to the `local_id`. It did not, and `normalize_speaker` classified each unmatched
answer as a free-form name that no registry entry could match.
Evidence, measured on 36 panels of the cancelled first rerun. 15 lines carried a description and 9 a bare
`local_id` with no identity assignment. 2 carried a stale `P1` mark label, 2 a name with the gender
marker attached. That is 28 of 51 speech lines. Only 3 resolved to a `character_id`.
`_apply_speaker_labels` now takes `present` and resolves the id, the name, the description, and each of
those plus the gender marker. A key shared by two present characters is dropped, because it identifies
neither. An id-shaped answer naming nobody present becomes `unknown`. A trailing gender marker is
stripped, so an off-panel `Seonho (m)` still matches the registry name `Seonho`.
`normalize_speaker` refuses an id-shaped value independently, because the worker is a separate process
and the contract is load-bearing (invariant 7).
Forbids: showing the model a label the worker cannot resolve back, and treating an unmatched speaker
string as a name.
Check: `python worker_vision.py`, the `shown`/`twins` cases. `pytest test_correctness.py`,
`test_an_id_shaped_speaker_is_never_a_name`.
## An unpaired mark grounds nothing {#unpaired-mark}
**Closed.** `_set_of_mark` labels a detected face `unknown` when gated pairing
(`decisions/audit-phase1.md#gated-face-pairing`) matched it to no present character. An answer pointing
at such a mark identifies nobody, so it must not carry `som_face`, the highest-trust provenance.
Evidence: all 7 `som_face` lines in the first 36 panels of the 2026-08-11 rerun had `speaker_ref` kind
`unknown`. The label sat on lines with no speaker. Same defect class as the fake `tail`.
Forbids: deriving a provenance label from the label map without checking what the label resolved to.
Check: `python worker_vision.py`, the `unpaired` case.
## An interjection is not a name and not a misquote {#interjection-false-positive}
**Closed.** `verify_script` must stay quiet on valid narration, because `run_stage_script` retries once
and then raises (`decisions/audit-phase1.md#verifier-false-positives`).
Evidence: the narrator wrote `"...Hm?"` for the source line `"Uh... hum...?"`. Both rules fired at once.
`Hm` was absent from the source words, and a 6-character quote needs 5 matching characters to ground,
so it got 4. That halted the `script` stage at 112/116 on job `778297bc`.
Interjections join `_STOPWORDS`. A quote of three letters or fewer is no longer grounded-checked. That
holds the same line as the dialogue prompt's 1-3 character noise rule. The stage then passed 116/116.
Forbids: scoring a quote too short for the ratio to carry meaning.
Check: `pytest test_script_verify.py`, `test_an_interjection_is_not_a_name_or_a_misquote`.
## Cast names enter the verifier tokenized {#multiword-cast-names}
**Closed.** `verify_script` compares single capitalized tokens, so every allowed name must be present as
tokens. `allowed` was built from `cast_names` verbatim, which put `"choi haeseon"` in the set as one
string while the checker looked up `Choi` and `Haeseon` separately.
Evidence: the `script` stage failed at 87/116 on job `778297bc`. 28 of the 29 lost beats cite
`unsupported-proper-noun: ['Choi', 'Haeseon']`. A one-word name such as `Seonho` always passed, which
is why this survived the Phase 1 verifier work
(`decisions/audit-phase1.md#verifier-false-positives`).
Forbids: adding any future allow-list to `verify_script` as whole strings.
Check: `pytest test_script_verify.py`, `test_multiword_cast_name_is_supported`.