Stop labelling a model guess as a read tail

_annotate_speaker_methods stamped `tail`, the highest-trust provenance, on any
line whose speaker matched a present local_id, at gemma's confidence of 1.0. No
balloon was read. Three of three sampled two-character panels had the speakers
swapped, so a multi-character guess is now dropped to unknown, and a solo-panel
guess is kept as model_solo at 0.7.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-11 23:23:56 +04:00
parent 36c7cc946f
commit e8941d8ceb
7 changed files with 133 additions and 39 deletions
+2
View File
@@ -30,3 +30,5 @@ still live belongs in `caveats/`.
| [One bucket per artifact class, not everything under `manga`](storage-layout.md#bucket-per-artifact) | closed |
| [The orchestrator creates missing buckets at startup](storage-layout.md#ensure-buckets) | closed |
| [RustFS is staged, not adopted](storage-layout.md#rustfs-staged) | open |
| [A model guess is never labelled `tail`](speaker-attribution.md#no-fake-tail) | closed |
| [Cast names enter the verifier tokenized](speaker-attribution.md#multiword-cast-names) | closed |
+45
View File
@@ -0,0 +1,45 @@
# Speaker attribution and cast names
Settled 2026-08-11 from the quality cross-check of job `778297bc`
(`JOURNAL.md`, `caveats/speaker-attribution.md`). No GPU work ran and no pipeline run was executed
after the change. Both claims rest on source and on the CPU-only self-checks named below.
Files: `worker_vision.py` on workpc, `correctness.py` and `test_script_verify.py` in the homesrv
orchestrator (`/mnt/server/home/kami/docker-apps/manga-infra/orchestrator/`).
## A model guess is never labelled `tail` {#no-fake-tail}
**Closed.** `speaker_method` names how a speaker was established, and nothing may claim geometry it did
not read. `_annotate_speaker_methods` stamped `tail`, the highest-trust label, on any line whose
speaker matched a `local_id` present in the panel, keeping gemma's confidence of 1.0. No balloon was
ever consulted.
Evidence: three of three sampled two-character panels had both speakers swapped
(`caveats/speaker-attribution.md#tail-is-not-geometry`). 31 of 81 speech lines carried `tail` with two
or more characters present.
The label is gone. With two or more characters present the guess is dropped: `speaker` becomes
`unknown`, confidence 0.0, method `unknown`. With one character present the claim equals the solo
backstop, so it is kept as `model_solo` at confidence 0.7. Grounded `som_face` and `solo_prior` rows are
untouched, because the function still skips any row that already carries a method.
Forbids: minting a provenance label for evidence that was not read, and shipping a multi-character
attribution as truth before balloon geometry exists.
Check: `python worker_vision.py`, the `crowd`/`lone` cases.
Cost: the named-speaker share will fall. The 30% headline was measured on attributions the sample says
are wrong, so the lower number is the first honest one.
## Cast names enter the verifier tokenized {#multiword-cast-names}
**Closed.** `verify_script` compares single capitalized tokens, so every allowed name must be present as
tokens. `allowed` was built from `cast_names` verbatim, which put `"choi haeseon"` in the set as one
string while the checker looked up `Choi` and `Haeseon` separately.
Evidence: the `script` stage failed at 87/116 on job `778297bc`. 28 of the 29 lost beats cite
`unsupported-proper-noun: ['Choi', 'Haeseon']`. A one-word name such as `Seonho` always passed, which
is why this survived the Phase 1 verifier work
(`decisions/audit-phase1.md#verifier-false-positives`).
Forbids: adding any future allow-list to `verify_script` as whole strings.
Check: `pytest test_script_verify.py`, `test_multiword_cast_name_is_supported`.