manga-recap-pipeline/decisions/identity-bbox.md

# identity-bbox

The coordinate space of a vision character box, and what reads it.

## A vision `bbox` arrives on gemma's 0-1000 grid, and `/vision` converts it to pixels {#bbox-is-normalized}

**Closed, 2026-08-12.**

`build_detect_prompt` asks for a "pixel bounding box". The model answers on its own normalized grid
regardless. Measured over all 113 detections of job `778297bc`, read straight from `/review/identity`:

| test | result |
| --- | --- |
| boxes with `x2` past the 900px panel width | **47 of 113** |
| boxes with `y2` past 1000, on panels 1257 to 2307px tall | **0 of 113** |
| boxes clamped at exactly 1000 | 21 in x, 5 in y |
| coordinate range over every box | `[0, 1000]` |

Pixels cannot behave that way. A person standing in the lower half of a 2307px panel needs `y2` near
2000, and it never once exceeds 1000.

Consumed as pixels the boxes collapse into the top-left corner of the panel. Four consumers were reading
them:

* `worker_identity._crop_bbox` at `worker_identity.py:200`, which embeds the crop. This is why a crop of
  a speech balloon's edge matched `Choi Haeseon` at 0.9.
* `_pair_faces_to_present` in `worker_vision.py`, which compares real detector face boxes, in pixels,
  against these. The gate could almost never pass, which is the mechanism behind the 7 `unknown` results
  out of 7 `som_face` lines already recorded at `worker_vision.py:169`.
* the set-of-mark boxes drawn for attribution.
* the review UI, which crops client-side off the panel PNG.

`/vision` now calls `_bbox_to_pixels(characters, w, h)` before returning, so all four see pixels and no
consumer needs to know the grid existed. Verified by drawing the converted boxes on panel 7. Five of six
land on their subject, including `person_5`, who is Seonho in the foreground and had no identity.
`person_1` still frames a window mullion with nobody in it, which is `#extras-as-cast`, not this.

The prompt text still says "pixel bounding box". Rewording it changes what the model emits and needs a
GPU run to re-verify, so the boundary converts instead. The `ponytail:` note on `_bbox_to_pixels` records
that. It also records the trap: a model that really answered in pixels would be scaled down here.

**Consequence: every assignment in the registry came from a wrong crop.** The existing embeddings and
`ref_image_uris` are enrolled on balloons and window frames. Re-running identity
is what makes the registry mean anything. The anonymous-identity and extra-versus-cast work cannot be
judged until that rerun happens.

## A stage result proves nothing until the worker is newer than the edit {#stale-worker-invalidates}

**Closed, 2026-08-12.** The first rerun after the bbox fix reproduced the defect exactly: 46 of 110
boxes past the 900px panel width, coordinates clamping at 1000, `y2` never once past 1000 on panels up
to 2307px tall. The same fingerprint as `#bbox-is-normalized` measured before the fix.

The fix was not wrong. It was not loaded.

```
vision worker process started   12:00:09
worker_vision.py modified       12:11:35
8113bdf, which contains _bbox_to_pixels, committed  12:16:22
```

Python binds a module once, at process start. `./start_workers.sh` had launched the worker eleven
minutes before the file changed, so `/vision` served pre-fix code for the whole run and returned raw
grid boxes. Nothing in the result said so. The stage reported `completed 116/116`, the orchestrator
recorded no error, and identity and reconcile ran to completion on top of it. Cost: one full
vision + identity + reconcile cycle, plus a registry reset to undo the 8 characters it minted.

The rerun against a restarted worker gives the opposite reading over the same 110 detections: 0 boxes
past the width, 0 past the height, one coordinate on 1000 which is now a real pixel value, and a
deepest box reaching 100% down its panel with `max y2 = 2307`. Boxes track the panel, so they are
pixels.

`check_stale.sh` compares every running worker's process start against its module's mtime and exits
non-zero if any is stale. This failure mode was already known as advice — the render worker "must be
restarted by hand to pick up an edit" — and advice did not stop it happening. Run the check before any
stage run that is meant to prove a code change.

Forbids: citing a stage result as evidence about a code change without establishing that the worker
serving it postdates the change.

## A detection with no detected face never enrolls or binds {#face-gates-enrollment}

**Closed, 2026-08-12. Written and self-checked, not yet proven on a GPU run.**

Fixing the coordinate space made the extras problem worse, not better. With the boxes finally landing
on their subjects, panel 7's four background extras became four good crops of four irrelevant people,
and one of them bound to `Seonho` at confidence 1.00. Before the fix the same detection was a crop of
scenery and matched nothing much. Correct geometry turned a harmless failure into a poisoned reference
set for the lead.

The measured panel 7 outcome, converted boxes, against the art:

| box | who | assigned |
| --- | --- | --- |
| `[457, 657, 642, 937]` | Seonho, foreground | `Seonho` |
| `[669, 591, 763, 822]` | the colleague, unnamed in the story | `character_f7a4fd`, anonymous |
| `[428, 386, 496, 526]` | background extra | none |
| `[34, 414, 122, 564]` | background extra | none |
| `[498, 386, 568, 533]` | background extra | `character_d72710` at 0.94 |
| `[31, 554, 94, 728]` | background extra | `Seonho` at 1.00 |

`/vision` now stamps `has_face` on every character by running `face_detect.detect_faces` on the panel
and reusing `_pair_faces_to_present` for containment, so the gate uses the same margin and the same
global shortest-first assignment as the speaker path. `worker_identity.py` skips a character with
`has_face is False` before it crops, embeds, matches or mints.

Two properties are deliberate. It **fails open**: a missing or raising detector marks every character
`True`, because dropping a whole panel's cast is worse than the over-detection the gate exists to trim.
And it gates on `is False`, not falsiness, so a vision blob written before this change (no key) behaves
as it did rather than silently dropping every character.

Cost: a cast member drawn from behind, or in a style the detector misses, now takes no identity on that
panel. That is the abstain this pipeline already prefers to a wrong bind
(`caveats/speaker-attribution.md#no-anonymous-identity`).

Forbids: enrolling a reference crop, or binding a character, from a region no face detector confirms.

## A resolver NONE mints an anonymous character, it does not clear the crop {#none-mints-an-anonymous-character}

**Closed, 2026-08-12. Written and self-checked, not yet proven on a GPU run.**

`caveats/speaker-attribution.md#no-anonymous-identity` asked whether the Tier-2 gemma resolver can answer
"none of these". It can, and it always could. `/vision/resolve` at `worker_vision.py:1071` maps `choice: 0`
to `state="new"`, an out-of-range index to `state="unresolved"`, and a parse failure to `unresolved` as
well. The abstain path was never the defect.

The defect was one branch on the other side of the contract. The orchestrator read only
`v.get("character_id")` and treated every falsy value the same way: `unassign_identity` on every crop of
the tracklet. So a deliberate "this is a real person the roster does not hold" and a hallucinated index
both produced nothing, and the unnamed colleague was `unknown` on every panel she appeared on. The stale
`ponytail:` comment above that block named the reason nobody fixed it, and the reason was real: minting a
character needs an `embedding_uri`, and the orchestrator cannot compute one. siglip is resident in the
identity worker, gemma is resident in the vision worker, and `session_manager` forbids both at once.

What removes the blocker is carrying the embedding, not a third GPU pass. `/identity/resolve` already
computes an embedding per crop and already uploads the crop to
`s3://manga/{manga}/characters/_crops/{panel}_{local}.png`. It now writes the embedding to the same key
with a `.npy` suffix and returns `emb_uri` in each shortlist entry. The mint is then a local
`create_character(manga_id, None, appearance, [crop_uri], emb_uri, gender)`, and the existing per-tracklet
assign loop binds every member to it.

`tracklets.resolve_outcome` holds the three-way decision as a pure function, so the branch that runs is
the branch the self-check covers: `known` on a named answer, `mint` on `state="new"` with an `emb_uri`,
`clear` on `unresolved`, on a NONE with no embedding, and on an older worker that sends no `state` at all.

Two limits are deliberate. A tracklet's candidate gallery is built before the loop mints anything, so one
person split across two unlinked tracklets still gets two anonymous ids; `run_stage_reconcile` merges
unnamed twins on appearance overlap and is what folds them. And an anonymous character's text sheet
(`worker_vision._sheet`) carries no name, so gemma re-recognising it on a later panel leans on the
reference images rather than the description.

Contract: `shortlists[].emb_uri` is new in the `/identity/resolve` response. Invariant 7 — both repos
changed in the same session.

Forbids: treating an absent `character_id` as one outcome. A resolver that answered and a resolver that
failed are different facts.

## The extras gate runs at enrollment and at narration, not at the speaker prompt {#extras-gate-consumers}

**Closed, 2026-08-12. Written and self-checked, not yet proven on a GPU run.**

`#face-gates-enrollment` stops a faceless detection taking an identity. It does not stop the detection
being narrated, because three places read the raw vision character list and only one of them consults an
assignment. Panel 7 is the worked example: six detections, four of them extras or scenery.

`worker_scene.build_scene` already drops an unassigned detection from `characters` and `present`
(`worker_scene.py:63`), so an extra never reached the cast list. Its **action** did. `actions` was built
from every detection, and that list is what the script prompt renders and what the verifier uses as
evidence. So "standing at the window" arrived as a fact about the panel with no character attached, and
the verifier confirmed it because the action really was in the blob. Both now skip `has_face is False`.

`service._beat` picks the cinematographer's "who" from the first three detections, falling back to a
detection's action when it has no name. An extra could take a slot and steer the camera. Also gated.

`service._present_characters` is deliberately **not** gated. It builds the dialogue stage's candidate
speaker list and the set-of-mark boxes. Two reasons. The failure is already contained: an extra chosen as
the speaker has no identity assignment, so `normalize_dialogue` resolves it to unknown rather than to a
wrong name. And the gate's own cost lands hardest here, because a character drawn from behind has no face
box, so gating would delete a real speaker from the only list that can attribute their line.

All three gates test `is False`, never falsiness. A vision blob written before the gate existed carries no
`has_face` key, and a panel whose detector failed is marked `True` by the fail-open path. Both keep their
previous behaviour.

Forbids: adding a fourth consumer of `vision["characters"]` without deciding which side of this line it is
on. The blob keeps every detection on purpose, so the audit can still see what was gated.