diff --git a/caveats/speaker-attribution.md b/caveats/speaker-attribution.md index 3bb106f..3a6ff10 100644 --- a/caveats/speaker-attribution.md +++ b/caveats/speaker-attribution.md @@ -50,8 +50,14 @@ The registry holds 53 characters for manga `ef105a86`, 41 of them unnamed, with Haeseon and Seonho do. `/stage/clear` leaves the per-manga registry intact by design, so every rerun inherits the whole pile. +Duplicate rows also make a correct name unresolvable. This manga holds `Choi Haeseon`, `Seonho` with +aliases `["Lim Seonho", "Seonho"]`, and a separate `Lim Seonho`. An answer of "Lim Seonho" matches two +rows, so `normalize_speaker` returns candidates and raises `ambiguous-speaker` instead of binding. The +pipeline read the name correctly and still cannot name the speaker. + **Revisit trigger:** before any run that is meant to produce a clean baseline. Either scope the -registry to a chapter or add a reviewed reset. +registry to a chapter or add a reviewed reset. Merging the duplicate rows needs the reversible-merge +design first (`caveats/audit-open.md#destructive-reconcile`). ## One invented word still halts the chapter {#one-word-halts-chapter} diff --git a/decisions/CLAUDE.md b/decisions/CLAUDE.md index f54fa0a..e8d5022 100644 --- a/decisions/CLAUDE.md +++ b/decisions/CLAUDE.md @@ -31,4 +31,5 @@ still live belongs in `caveats/`. | [The orchestrator creates missing buckets at startup](storage-layout.md#ensure-buckets) | closed | | [RustFS is staged, not adopted](storage-layout.md#rustfs-staged) | open | | [A model guess is never labelled `tail`](speaker-attribution.md#no-fake-tail) | closed | +| [The model's speaker answer is resolved against what the prompt showed](speaker-attribution.md#prompt-label-answers) | closed | | [Cast names enter the verifier tokenized](speaker-attribution.md#multiword-cast-names) | closed | diff --git a/decisions/speaker-attribution.md b/decisions/speaker-attribution.md index 21c3124..4c867ac 100644 --- a/decisions/speaker-attribution.md +++ b/decisions/speaker-attribution.md @@ -30,6 +30,30 @@ Check: `python worker_vision.py`, the `crowd`/`lone` cases. Cost: the named-speaker share will fall. The 30% headline was measured on attributions the sample says are wrong, so the lower number is the first honest one. +## The model's speaker answer is resolved against what the prompt showed {#prompt-label-answers} + +**Closed.** `build_dialogue_prompt` renders a present character as +`- person_1: brown ponytail, green dress (f)`. gemma answers with any part of that line, so every part +of it must map back to the `local_id`. It did not, and `normalize_speaker` classified each unmatched +answer as a free-form name that no registry entry could match. + +Evidence, measured on 36 panels of the cancelled first rerun. 15 lines carried a description and 9 a bare +`local_id` with no identity assignment. 2 carried a stale `P1` mark label, 2 a name with the gender +marker attached. That is 28 of 51 speech lines. Only 3 resolved to a `character_id`. + +`_apply_speaker_labels` now takes `present` and resolves the id, the name, the description, and each of +those plus the gender marker. A key shared by two present characters is dropped, because it identifies +neither. An id-shaped answer naming nobody present becomes `unknown`. A trailing gender marker is +stripped, so an off-panel `Seonho (m)` still matches the registry name `Seonho`. + +`normalize_speaker` refuses an id-shaped value independently, because the worker is a separate process +and the contract is load-bearing (invariant 7). + +Forbids: showing the model a label the worker cannot resolve back, and treating an unmatched speaker +string as a name. +Check: `python worker_vision.py`, the `shown`/`twins` cases. `pytest test_correctness.py`, +`test_an_id_shaped_speaker_is_never_a_name`. + ## Cast names enter the verifier tokenized {#multiword-cast-names} **Closed.** `verify_script` compares single capitalized tokens, so every allowed name must be present as diff --git a/worker_vision.py b/worker_vision.py index 297c131..b1b1133 100644 --- a/worker_vision.py +++ b/worker_vision.py @@ -165,8 +165,14 @@ def _apply_speaker_labels(dialogue: list, label_map: dict, present: list | None if not s: continue if s in label_map: - d["speaker"] = label_map[s] - d["speaker_method"] = "som_face" + lid = label_map[s] + # _set_of_mark labels a detected face `unknown` when gated pairing matched it to no present + # character. An answer pointing at such a mark grounds nothing, so it must not carry + # `som_face`, the highest-trust label. 7 of 7 som_face lines on the first 36 panels of the + # 2026-08-11 rerun were this case. + d["speaker"] = lid + if lid != "unknown": + d["speaker_method"] = "som_face" continue bare = _GENDER_SUFFIX.sub("", s).strip() lid = keys.get(s.casefold()) or keys.get(bare.casefold()) @@ -1044,6 +1050,10 @@ if __name__ == "__main__": lbl = _apply_speaker_labels([{"speaker": "P1"}, {"speaker": "Aria"}, {"speaker": "unknown"}], {"P1": "person_3"}) assert lbl[0]["speaker"] == "person_3" and lbl[1]["speaker"] == "Aria" and lbl[2]["speaker"] == "unknown" + assert lbl[0]["speaker_method"] == "som_face" + # a mark whose face paired to nobody present grounds nothing, so it gets no som_face label + unpaired = _apply_speaker_labels([{"speaker": "P2"}], {"P2": "unknown"})[0] + assert unpaired["speaker"] == "unknown" and "speaker_method" not in unpaired, unpaired # gemma answers with what the prompt SHOWED, not the id: description, name+gender, bare id, junk id. # Each shape cost real lines on job 778297bc by falling through as a free-form name. shown = [{"local_id": "person_1", "desc": "brown ponytail, green dress", "gender": "f"},