kami/manga-recap-pipeline
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Record two GPU cycles: the gate works, coverage is the open question

Browse Source 
No worker code changed. This is the evidence from the 2026-08-12 16:39 and
17:38 runs, and where each finding now lives.

The fourth session's four identity changes all work on real panels. Panel 7's
two wrong bindings are gone. The lead going unassigned there is correct and
was measured, not assumed: face_detect finds one face on the whole panel at
conf 0.599, nothing else above 0.056 even at a 0.04 threshold, and the crop
shows him drawn from behind.

Two decisions, both closed: a roster name is a guess so it never reaches
detection, and merged_into is exactly one hop deep. Two caveats, both open:
detection can order a bbox backwards (1 in 117), and identity coverage has
fallen on every run since the gate landed (70 -> 61 -> 50).

Coverage is the thing to settle next, and not by reading the number.
identity_labels already holds 145 rows of ground truth.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
kami
2026-08-12 21:51:36 +04:00
parent c51871348f
commit 18b49c43bd
7 changed files with 366 additions and 120 deletions
Download Patch File Download Diff File Expand all files Collapse all files
HANDOFF.md
+117 -81
View File
@@ -1,124 +1,160 @@
# HANDOFF, 2026-08-12 (fourth session of the day)
# HANDOFF, 2026-08-12 (fifth session)
Live state is in `NEXT.md`. This file is only what this session did. The previous handoff is in
`JOURNAL.md`.
## Asked
"How much will `has_face` help with character and identity problems?", then in order: fix 2b, fix
extra-versus-cast, 2d.
"go ahead" on the GPU cycle the fourth session left staged. Then "what do we do now". Then go ahead on
the fixes its evidence asked for, and run it again.
## Result
Identity item 2 in `NEXT.md` is now done as far as it can go without a run. Four changes are written,
self-checked and deployed. **None has run on a GPU.** No pipeline stage was executed this session.
Two full GPU cycles ran. All four of the fourth session's identity changes are now proven on real panels.
Four more fixes were written on top, all orchestrator-side, all deployed. Coverage fell twice and that is
the open question.
| item | was | now |
| --- | --- | --- |
| 2a bbox space | done last session | unchanged |
| 2b abstain and stay abstained | resolver blamed, unverified | fixed in the orchestrator, `f6804e7` + `ffda3df` |
| 2c extra versus cast | open | two consumers gated, `ca46617` + `8b27aec` |
| 2d merge and split | open | examples gone; merge made non-destructive, `00096cc` |
| cycle | detections | assignments | coverage | registry | named |
| --- | --- | --- | --- | --- | --- |
| baseline (13:11, pre-change) | 110 | 77 | 70% | 8 | 1 |
| run A (16:39-16:45) | 110 | 67 | 61% | 16 | 2 |
| run B (17:38-17:44) | 117 | 59 | 50% | 20 | 2 |
## What each change was, and where the cause turned out to be
Vision is non-deterministic, so detection counts move between runs. Each cycle is ~6 minutes:
vision ~3m50s, identity ~1m25s, reconcile ~50s.
**2b was never the resolver.** `/vision/resolve` at `worker_vision.py:1071` already returns
`state="new"` for `choice: 0` and `state="unresolved"` for an out-of-range index. `service.py` read only
`v.get("character_id")` and ran `unassign_identity` on every crop of the tracklet for either one. A
deliberate NONE produced nothing, so an unnamed recurring person was `unknown` on every panel.
## First, a correction the session started with
The old `ponytail:` comment above that block named the real blocker and was right. Minting needs an
`embedding_uri` the orchestrator cannot compute. siglip is resident in the identity worker and gemma in
the vision worker. Fixed by carrying the embedding, not by adding a GPU pass.
`/identity/resolve` writes each crop's embedding to the crop's key with a `.npy` suffix and returns
`emb_uri`. `tracklets.resolve_outcome` holds the three-way decision as a pure function.
`decisions/identity-bbox.md#none-mints-an-anonymous-character`.
The vision/identity/reconcile timestamps in `/job/status` are UTC. The git log is local, UTC+4. The run
that looked like a completed rerun was the pre-change baseline: it finished 13:17 local, and
`_mark_has_face` was not committed until 19:07. The fourth session's handoff was right that nothing had
run.
**2c: `build_scene` already dropped extras from the cast list.** `worker_scene.py:63` skips an unassigned
detection, so extras never reached `characters` or `present`. Their **actions** did. `actions` was built
from every detection. That list is the script prompt's content and the verifier's evidence. So a background
extra's "standing at the window" arrived as a fact with no character attached, and the verifier confirmed
it. Gated `worker_scene`'s `actions` and `service._beat` on `has_face is False`. Left
`service._present_characters` ungated on purpose, reasoned out in
`decisions/identity-bbox.md#extras-gate-consumers`.
## Run A: the four staged changes, verified
**2d's worked examples no longer exist.** The registry reset deleted `Lim Seonho` and
`character_afa7623b`. The registry is 8 rows, one named. Built the safety net for the coming rerun
instead, since `reconcile` runs inside it: `merge_characters` sets `merged_into = keeper` rather than
deleting, and stamps every repointed assignment `method = merged_from:<loser_id>`. Roster readers filter
`merged_into IS NULL`, lookup by id does not.
`caveats/audit-open.md#destructive-reconcile` is half-closed.
| change | verdict |
| --- | --- |
| `has_face` stamp | on all 110 detections, 67 true / 43 gated (39%) |
| extras gate | panel 7's two wrong bindings gone |
| NONE mints | 16 characters where 8 existed |
| non-destructive merge | 7 rows carry `merged_into`, 9 assignments stamped `merged_from:` |
## Measured, read-only, before writing anything
Panel 7, the worked example:
```
registry: 8 characters, 1 named -> ['Seonho']
detections: 110 assignments: 77 = 70% coverage
spread: Seonho 36, character_565c88 24, character_759e23 9, character_f7a4fd 3,
character_25f682 3, character_d72710 1, character_823aba 1
bbox space: 77/110 exceed 1000, 1 on 1000 -> PIXELS
person_1 Seonho 1.00 -> -- none -- has_face=False
person_2 character_f7a4fd 0.00 -> character_519d2b 0.00 has_face=True
person_4 character_d72710 0.94 -> -- none -- WRONG binding removed
person_5 Seonho 1.00 -> -- none -- WRONG binding removed
```
That killed the assumed cause of 2b. Anonymous ids already recur, so the identity worker's own
pending-promote path gives stable anonymous identities. Only the gemma NONE branch was discarding people.
`person_1` going unassigned is correct, and this was measured, not assumed. Ran `face_detect.detect_faces`
on `p006.png` directly: one face on the whole panel, conf 0.599, inside `person_2`. Swept the threshold to
0.04 and nothing else appears above 0.056. The crop shows the lead drawn from behind at his desk, back of
the head and headphones. Detector right, gate right, lead unidentifiable in that panel.
## Deliberately not built
`person_1`'s box now frames the lead. The old "empty window mullion" note was written against pre-fix
boxes and is retired.
- **The unmerge path and the split.** No wrong merge has been seen since the crops were fixed. Undoing
one today is a hand-written SQL walk of the two records above. Revisit trigger is in the caveat.
- **`service._present_characters` gating.** An extra picked as speaker already resolves to unknown. A
character drawn from behind has no face box. Gating would delete a real speaker from the only list that
can attribute their line.
- **Vision still emits extras into the blob.** Deliberate, so the audit can see what was gated.
## Run A's new defect: a roster hint named the wrong man
Verified against the art, three real people:
- `character_92129ac7` "Lim Seonho", 22 assignments. The p010 caption reads `LIM SEONHO (29)`, yellow
plaid shirt, headphones, matching the roster's description. Correct.
- `character_556aef60`, 25 assignments, unnamed. The woman with short black hair and pearls, the roster's
second character, whom the roster itself calls "Unknown". Correct.
- `character_dbadfff7` "Seonho", 15 assignments. A different man, glasses, dark clothing, wearing the
lead's roster name.
p020 assigned `Seonho` to `person_1` and `Lim Seonho` to `person_3`, two people in one panel.
## What was changed, and where the cause was
**Roster hints no longer seed detection.** Deleted the two lines at `service.py:882` that appended
`_roster_char_hints` to `known_characters`. `build_detect_prompt` drops any hint without a name. Passing a
nameless hint would have contributed nothing, so removing the seeding was the only real option.
Names now reach detection from registry rows only, which carry embeddings and were named from an in-panel
caption or address. The roster still feeds `roster_cast` in `run_stage_dialogue`, where names match
against speech rather than faces.
**`merge_characters` keeps `merged_into` one hop deep.** Two halves, one per direction, and the second was
missed on the first attempt:
- resolve the keeper to its chain root before merging (a bounded 64-step walk, the cap only so a cycle
cannot hang reconcile)
- repoint the loser's own dependents to the keeper when the loser is retired
The keeper walk alone does not work. Run B still produced `477c1894 -> a92d9df4 -> 4fb94c15`. At merge
time that pair was fine. The chain formed later, when a row that was already somebody's keeper was itself
retired.
**`reset_registry` deletes `name_claims`** for the manga's panels and reports the count. Confirmed live:
run B's reset reported `name_claims: 5`, the orphans that had pointed at `character_afa7623b` since two
resets earlier.
**`audit_registry.py` is in the repo.** It had only ever been `docker cp`'d, so every rebuild dropped it.
`Dockerfile` has `COPY . .`, so it is baked now and that trap is gone.
## Run B: what the fixes did
The registry split is fixed. "Lim Seonho" came back as ONE row holding 25 assignments. That answers the
open risk from before the run: dropping the roster hint did not split the lead across the panels before
his caption. "Seonho" fell from 15 assignments to 1.
Two things run B surfaced:
- **A second naming mechanism, untouched by the fix.** Even with no roster hint, p011 and p026 emit
`name: "Seonho"` on `person_2` from in-panel text. Detection reads a name off the panel and attaches it
to the wrong body. That is what keeps the glasses man named, now at 1 assignment instead of 15.
- **One degenerate bbox.** `p007 person_1` is `[226, 417, 130, 551]`, x1 > x2. One in 117. It crops to
nothing, so that detection can never enroll or match. `_bbox_to_pixels` clamps each coordinate but never
orders the corners.
## Not done
- **The degenerate-box guard.** It is a worker change (`worker_vision.py:_bbox_to_pixels`), so it needs a
vision worker restart and a third GPU cycle to prove. Not started, nothing half-edited.
- **The coverage trend.** 70% -> 61% -> 50%. Part is the `has_face` gate, which is stable at ~39% of
detections across both runs. Framed against face-bearing detections only, run B assigned 59 of 72, or
82%. Nothing establishes whether the rest is correct abstention or lost cast. Settle that number before
trusting the registry.
- **Nothing downstream re-ran.** The job is parked at `dialogue waiting` with every stage below it
cleared. No clip or chapter has been rebuilt against the new cast.
## Checks
```bash
.venv/bin/python worker_identity.py # ok
.venv/bin/python worker_scene.py # ok, 3 new cases on the actions gate
./check_stale.sh # exit 0, all 9 workers current
/usr/bin/ssh kami@192.168.1.104 "cd /home/kami/docker-apps/manga-infra/orchestrator && python3 -m pytest -q --ignore=test_api.py" # 116 passed
/usr/bin/ssh kami@192.168.1.104 "cd /home/kami/docker-apps/manga-infra/orchestrator && python3 -m pytest -q --ignore=test_api.py" # 118 passed, was 116
./check_stale.sh # exit 0, all 9 workers current
```
Deployed and verified inside the running container: `resolve_outcome` returns `mint`, `_beat` drops a
faceless detection, `merged_into` exists on the live database with 0 rows merged. Vision, identity and
scene workers restarted.
Both new merge tests were confirmed to fail with their fix disabled, then the fix was restored and the
suite re-run. The deployed container was verified by parsing its source, not grepping it. The first
attempt gave a false negative, matching `_roster_char_hints` inside the comment that explains its removal.
## Next command
Four changes ride one GPU cycle. Coverage is 70% and is the number to beat. A gate that abstains too hard
shows up there before it shows up on panel 7. Watch the identity log line for `minted N anonymous`.
The third cycle, after adding the corner-ordering guard to `_bbox_to_pixels` and restarting the vision
worker. Watch coverage against 50% and the degenerate box count against 1.
```bash
cd /home/kami/Programs/n8n-worker && ./check_stale.sh # must exit 0
J=778297bc-e7ce-439d-91b5-8a027060d17f; M=ef105a86-4b7e-4ac4-b45c-b7d83b8f5b5e
/usr/bin/ssh kami@192.168.1.104 "curl -s -X POST http://127.0.0.1:9090/characters/reset -H 'Content-Type: application/json' -d '{\"manga_id\":\"$M\",\"confirm\":true}'"
/usr/bin/ssh kami@192.168.1.104 "curl -s -X POST http://127.0.0.1:9090/stage/clear -H 'Content-Type: application/json' -d '{\"job_id\":\"$J\",\"stage\":\"vision\"}'"
# restart the identity worker here, see the trap below
for S in vision identity reconcile; do /usr/bin/ssh kami@192.168.1.104 "curl -s --max-time 5400 -X POST http://127.0.0.1:9090/stage/run -H 'Content-Type: application/json' -d '{\"job_id\":\"$J\",\"stage\":\"$S\"}'"; done
/usr/bin/ssh kami@192.168.1.104 "docker exec manga-orchestrator python3 /app/audit_registry.py 7c944dd4-e972-42c7-ba60-9f6939548e80 6"
```
Panel 7 (`panel_index 6`) is the worked example. Before this session's changes:
```
person_1 [457, 657, 642, 937] Seonho 0.00 foreground, correct
person_2 [669, 591, 763, 822] character_f7a4fd 0.00 the unnamed colleague, correct
person_3 [428, 386, 496, 526] -- none -- extra
person_4 [498, 386, 568, 533] character_d72710 0.94 extra, WRONG
person_5 [31, 554, 94, 728] Seonho 1.00 extra bound to the lead, WRONG
person_6 [34, 414, 122, 564] -- none -- extra
```
`person_4` and `person_5` are what `has_face` must remove without taking `person_1` or `person_2` with
them. Restart the identity worker after every reset: it caches the known list in-process and only
invalidates on enrollment.
## Traps confirmed or found
- **`tmux respawn-window -k` does not re-run the window command.** It leaves a bare shell. Both vision and
identity sat dead for two minutes before an empty `/health` caught it. Now in `AGENTS.md`.
- `docker compose up --build orchestrator` recreates the container and drops any `docker cp`'d file, so
`audit_registry.py` needs re-copying after every rebuild.
- Two test assertions asserted the old destructive merge (`test_db.py:235`, `test_merge_refs.py:37`). They
were rewritten, not deleted: the invariant changed on purpose.
- The orchestrator image bakes its source. Editing the repo on homesrv does nothing until the rebuild.
- **`kill $(pgrep -f "worker_identity:app")` kills the shell running it.** The pattern matches the calling
command line. It killed this session's own script mid-way, so the `send-keys` relaunch never fired and
the identity worker sat dead. Restart it with the two `tmux send-keys` lines only, then poll `/health`.
- **Verify deployed code by parsing it, not by grepping it.** A comment explaining a removal contains the
name of the thing removed.
- **A test that cannot fail proves nothing.** Both merge tests were run with their fix disabled first.
- Panel-7 line numbers move between runs, because vision is non-deterministic and `local_id` is assigned
top-to-bottom per run. In run B the lead is `person_5`, not `person_1`. Compare by bbox, not by id.
JOURNAL.md
+127
View File
@@ -459,3 +459,130 @@ block, verified on the live database (`merged_into` present, 0 rows merged). Com
Deliberately not built: the unmerge path and the split. No wrong merge has been observed since the crops
were fixed, so the consumer of these records waits for one. The forward case is partly covered by 2b, since
a resolver NONE now mints instead of folding a stranger into the nearest match.
## 2026-08-12, fifth session — the GPU cycle all four changes were waiting for
Asked: "go ahead" on the rerun.
First correction: the vision/identity/reconcile timestamps in `/job/status` are UTC and the git log is
local (UTC+4), so the run that looked like a rerun was the pre-change baseline. `_mark_has_face` landed at
19:07 local, the run finished 13:17 local. The handoff was right that nothing had run.
Reset the registry for `ef105a86` (8 characters, 77 assignments, 101 sources, 142 S3 objects), cleared
`vision` and everything downstream (116 vision results), restarted the identity worker for its in-process
known-character cache, then ran vision, identity and reconcile. 6 minutes wall: vision 3m48s, identity
1m25s, reconcile 48s. All three `completed`, 116/116/16.
### What the four changes did
| change | verdict |
| --- | --- |
| `has_face` stamp | works, present on all 110 detections, 67 true / 43 gated (39%) |
| extras gate | works, panel 7's two wrong bindings are gone |
| NONE mints | works, 16 characters minted where 8 existed |
| non-destructive merge | works, 7 rows carry `merged_into`, 9 assignments stamped `merged_from:` |
Coverage went 70% -> 61% (77 -> 67 assignments over the same 110 detections). That is the gate's price and
it is mostly correct, see below.
### Panel 7, the worked example, before and after
```
person_1 Seonho 1.00 -> -- none -- has_face=False
person_2 character_f7a4fd 0.00 -> character_519d2b 0.00 has_face=True
person_4 character_d72710 0.94 -> -- none -- WRONG binding removed
person_5 Seonho 1.00 -> -- none -- WRONG binding removed
```
The two wrong bindings the gate existed to kill are dead. It also took `person_1`, and that is correct:
ran `face_detect.detect_faces` on p006.png directly and it finds exactly one face on the whole panel, conf
0.599, inside `person_2`. Swept the threshold to 0.04 and nothing else appears above 0.056. Looked at the
crop: `person_1` is the lead drawn from behind at his desk, back of the head and headphones, no face in
frame. The detector is right and the gate is right. The cost is that a back-turned character cannot be
identified from that panel at all.
Also worth recording: `person_1`'s new box frames the lead correctly. The old note that it framed an empty
window mullion was written against the pre-fix boxes.
### The new defect: a roster hint named the wrong man
The registry holds three real people and one wrong name.
- `character_92129ac7` "Lim Seonho", 22 assignments. Verified against the art: the p010 introduction panel
captioned `LIM SEONHO (29)`, yellow plaid shirt and headphones, which is the roster's description for
"Seonho". This is the lead and the name is right.
- `character_556aef60`, 25 assignments, unnamed. Verified: the woman with short black hair and pearls, the
roster's second character, whom the roster itself calls "Unknown".
- `character_dbadfff7` "Seonho", 15 assignments. Verified: a different man, glasses, dark clothing, in
profile. He carries the lead's roster name.
p020 assigns `Seonho` to `person_1` and `Lim Seonho` to `person_3`, so the pipeline holds them as two
people in one panel. inference: the roster hint injected at `service.py:882` puts a name in front of
detection, and detection attached it to the wrong face before the caption panel could mint the real one.
This is not the alias-merge case and merging the two rows would be wrong. It needs name binding to require
evidence, the way `name_claims` already does for captions and address.
### Two smaller things the run exposed
- **Merge chains.** `character_e1ab7776 -> character_521c301f -> character_556aef60`. `merged_into` points
at a row that is itself merged, so a single-hop resolve lands on a retired character. Roster readers are
fine because they filter `merged_into IS NULL`. Anything that follows one hop is not.
- **Orphan `name_claims`.** All 5 rows point at `character_afa7623b`, which the reset deleted. The reset
clears characters and assignments but not claims.
Not run: dialogue and everything downstream. The job sits at `dialogue waiting`.
### Same session — the three fixes the run's evidence asked for
All three are orchestrator-side, so no worker changed and `check_stale.sh` is not in play.
- **Roster hints no longer seed detection.** Deleted the two lines at `service.py:882` that appended
`_roster_char_hints` to `known_characters`. `build_detect_prompt` drops any hint without a name, so a
nameless hint would have contributed nothing anyway. Names now reach detection only from registry rows,
which are embedding-backed and were themselves named from an in-panel caption or address. The roster
still feeds `roster_cast` in `run_stage_dialogue`, where names are matched against speech.
- **`merge_characters` resolves the keeper to its chain root** before merging, so `merged_into` stays one
hop deep. Bounded 64-step walk, the cap only exists so a cycle cannot hang reconcile.
- **`reset_registry` deletes `name_claims`** for the manga's panels and reports the count.
Checks: 117 passed on homesrv, up from 116. The new chain test was confirmed to fail with the walk
disabled, then the walk was restored and the suite re-run. Deployed by rebuilding the image; verified
inside the running container by parsing the deployed source, not by grepping it, because the first check
matched the word `_roster_char_hints` inside the comment that explains its removal.
Also copied `audit_registry.py` into the repo before the rebuild. It had only ever been `docker cp`'d, so
every rebuild dropped it. `Dockerfile` has `COPY . .`, so it is baked now and the trap is gone.
Open risk on the next run: without the roster hint the lead is unnamed until the p010 caption, so panels 1
to 9 may mint him as an anonymous character that reconcile then has to merge. Watch whether "Lim Seonho"
comes back as one row or two.
### Same session — run B, 17:38-17:44
Reset (16 characters, 67 assignments, 67 sources, **5 name_claims**, 156 S3 objects), cleared `vision`,
restarted the identity worker, ran the three stages. The `name_claims: 5` line is the orphan fix confirmed
on live data.
117 detections, 59 assignments, 50% coverage, 20 characters, 72 with `has_face` (45 gated).
The registry split is fixed. "Lim Seonho" is one row with 25 assignments, "Seonho" fell from 15 to 1. So
dropping the roster hint did not split the lead across the panels before his caption, which was the risk
recorded before the run.
Two findings, both now filed:
- detection still names from in-panel text and can attach the name to the wrong body (p011, p026), which
is `decisions/identity-bbox.md#roster-does-not-name`'s "not covered" paragraph
- one degenerate bbox, `caveats/audit-open.md#degenerate-bbox`
And one fix of my own that was incomplete: the merge chain came back as
`477c1894 -> a92d9df4 -> 4fb94c15` with the keeper walk deployed. The walk only sees the pair being merged
now. Retiring a row that is already somebody's keeper needed the second half, a repoint of the loser's
dependents. Both halves and both tests are in `decisions/identity-bbox.md#merge-chains-flatten`.
Checks: 118 passed, was 116. Each new merge test was run with its fix disabled and confirmed to fail
first. Deployed by rebuild and verified by parsing the container's source.
Coverage is now 70 -> 61 -> 50 across three runs and is the open question, recorded as
`caveats/audit-open.md#coverage-trend`. `identity_labels` already holds 145 rows of ground truth, so the
next move is to score with `eval_identity.py` rather than to keep reading the coverage number.
NEXT.md
+54 -39
View File
@@ -1,51 +1,66 @@
# NEXT
Updated 2026-08-12. What this session did is in `HANDOFF.md`.
Updated 2026-08-12 (fifth session). What the fourth session did is in `HANDOFF.md`, the run is in
`JOURNAL.md`.
## State
The chapter runs end to end. The A/V sync defect is fixed and `chapter.mp4` is rebuilt: video 364.120s
against audio 364.122s at `25/1`. The identity defects are still in the output.
against audio 364.122s at `25/1`.
Job `778297bc-e7ce-439d-91b5-8a027060d17f`, chapter `7c944dd4-e972-42c7-ba60-9f6939548e80`, 116 panels,
`status=completed`, finished 2026-08-11T20:08:16Z. `s3://video/` holds 49 clips and a 50MiB
`chapter.mp4`. The user watched it and read out 19 defects. They are grouped by cause in `HANDOFF.md`.
Job `778297bc-e7ce-439d-91b5-8a027060d17f`, chapter `7c944dd4-e972-42c7-ba60-9f6939548e80`, 116 panels.
Two GPU cycles ran on 2026-08-12, the last 17:38-17:44 UTC on a reset registry. The job is `running` and
parked at `dialogue waiting`. Everything from `dialogue` down is cleared and stale.
One number sets the agenda:
Seven changes are now proven on real panels: the fourth session's four, plus this session's roster-hint
removal, merge-chain flattening and `name_claims` reset. The registry after the last run:
- Panel 7 checked against the art has **zero correct identity bindings** out of two, and the one
character who matters is unbound. `HANDOFF.md#panel-7-walked-against-the-art` has the table. The
coordinate cause is fixed. The registry built on it is not.
- `character_713ea2ce` "LIM SEONHO", 25 assignments, the lead, one row, named from the p010 caption.
- `character_4fb94c15`, 18 assignments, the woman, correct and unnamed.
- `character_023ba5a3` "Seonho", 1 assignment, still the wrong man, down from 15.
Coverage is 50%, from 61% and 70% before it. That trend is the open question and is filed as
`caveats/audit-open.md#coverage-trend`.
## Next
1. **Re-run vision and identity.** The `bbox` space is settled and converted at `/vision`
(`decisions/identity-bbox.md#bbox-is-normalized`). Every stored box, embedding and `ref_image_uris` in
the registry came from the wrong space. The fix changes nothing until those stages run again.
This is GPU work and needs the user's go-ahead. Clear `vision` and everything downstream of it, or
accept that the boxes in the database stay normalized while new ones are pixels.
Watch two things on the rerun. Whether `som_face` still returns `unknown` on every face, since gated
pairing was comparing pixel face boxes against 0-1000 character boxes. And whether `Choi Haeseon` still
absorbs every unnamed woman, which is item (b) below and independent of the crops.
1. **Settle coverage before trusting the registry.** 70% -> 61% -> 50% over three runs. `has_face` gates a
steady 39%, and against face-bearing detections alone the last run assigned 59 of 72 (82%). Nothing
separates correct abstention from lost cast. `identity_labels` already holds 145 rows of human ground
truth, and `eval_identity.py` already scores against it. Measure precision and recall instead of
reading the coverage number (`caveats/audit-open.md#coverage-trend`).
2. **Order the corners in `_bbox_to_pixels`.** `p007 person_1` came back `[226, 417, 130, 551]`, x1 > x2,
1 in 117. It crops to nothing, so that detection is silently lost
(`caveats/audit-open.md#degenerate-bbox`). Two `min`/`max` pairs. It is a worker change, so it needs a
vision restart and a GPU cycle to prove.
3. **Detection still names from in-panel text and can hit the wrong body.** p011 and p026 emit
`name: "Seonho"` on `person_2` with the roster hint gone. That is the residue of the naming defect and
the reason the glasses man is named at all
(`decisions/identity-bbox.md#roster-does-not-name`, "not covered").
4. **Decide what a back-turned character costs.** `has_face` gates 39% of detections. Panel 7's lead is at
his desk from behind. `face_detect` finds one face on the whole panel at conf 0.599, and nothing else
above 0.056 even at a 0.04 threshold. The gate is right and the detector is right. The lead is still
unidentifiable there. Options are a body or head detector alongside the face one, or letting the
tracklet carry identity across a back-turned panel. Do not lower `FACE_CONF`, the sweep shows nothing
to find.
Smaller follow-on: nine `_audio_dur` calls in `worker_render.py` measure finished clips with
`format=duration`. So the durations reported to the orchestrator are blind to per-clip drift.
They position no filter, so invariant 9 does not cover them. Worth converting to `_stream_dur`.
2. **Fix identity, in this order.** Panel 7 is the worked example and
`HANDOFF.md#panel-7-walked-against-the-art` carries the evidence. Do not start at the registry.
5. **Fix identity, in this order.** All of 2a-2d below are done and now proven on a GPU. Kept for the
evidence trail.
a. ~~Settle the `bbox` coordinate space.~~ **Done 2026-08-12**, proven over all 113 detections and
checked by eye on panel 7, where five of six converted boxes land on their subject
(`decisions/identity-bbox.md#bbox-is-normalized`). `person_1` still frames an empty window mullion,
which is (c).
b. ~~Let identity abstain and stay abstained.~~ **Done 2026-08-12, not yet run on a GPU**
(`decisions/identity-bbox.md#bbox-is-normalized`). The rerun settles the last doubt: `person_1` now
frames the lead at his desk, not the window mullion the pre-fix box caught.
b. ~~Let identity abstain and stay abstained.~~ **Done 2026-08-12, run and verified on a GPU**
(`decisions/identity-bbox.md#none-mints-an-anonymous-character`). The resolver could always answer
"none of these". The orchestrator was discarding the answer: it read only `character_id`, so a
deliberate NONE and a hallucinated index both unassigned every crop of the tracklet. A NONE now
mints an anonymous character from the crop, using the embedding `/identity/resolve` ships beside
it as `emb_uri`. Deployed: image rebuilt, `resolve_outcome` verified inside the container.
c. ~~Separate extra from cast.~~ **Done 2026-08-12, not yet run on a GPU**
c. ~~Separate extra from cast.~~ **Done 2026-08-12, run and verified on a GPU**
(`decisions/identity-bbox.md#face-gates-enrollment`,
`decisions/identity-bbox.md#extras-gate-consumers`). `has_face` stops a faceless detection
enrolling, and two more consumers now skip it: `worker_scene`'s `actions`, which is the script
@@ -53,11 +68,11 @@ One number sets the agenda:
`_present_characters` stays ungated on purpose, reasoned out in the decision.
The remaining gap is that vision still emits extras into the blob, which is deliberate so the
audit can see what was gated.
d. **The worked examples are gone.** The registry reset deleted `Lim Seonho` and
`character_afa7623b`. The current registry is 8 rows, one named (`Seonho`), so there is nothing to
merge or split until the rerun mints a new set.
d. ~~Merge and split.~~ **Safety net done and now exercised.** The rerun's `reconcile` merged 7 rows
and stamped 9 assignments `merged_from:`, so the non-destructive path is proven on real data. It also
produced the chain in item 3, which is the first thing to fix in it.
What was done instead is the safety net for that rerun, since `reconcile` runs inside it. A merge no
A merge no
longer deletes the losing row: it sets `merged_into`, and stamps every repointed assignment with
`method = merged_from:<loser_id>`. A wrong merge now costs a hand-written SQL walk, not a full
rebaseline (`caveats/audit-open.md#destructive-reconcile`).
@@ -69,9 +84,9 @@ One number sets the agenda:
**Cast profiles already exist. Do not rebuild them.** The user asked whether the main cast could get a
profile built from reference frames and reused. `characters` already carries `ref_image_uris` and
`embedding_uri`, and all 53 rows have both populated. The mechanism is not missing, it is enrolled
from the wrong crops, so today it stores references to balloon edges and window frames. Step (a) is
what makes it work. Three things are genuinely absent and are the smaller follow-on:
`embedding_uri`, and every row has both populated (16 rows after the rerun, 53 before it). The
mechanism was never missing. It was enrolled from the wrong crops, so it stored balloon edges and
window frames. Step (a) fixed that, and the audit now reports 0 characters missing a ref crop. Three things are genuinely absent and are the smaller follow-on:
- no quality gate on enrollment, so nothing checks that a reference crop holds a face at all
- nothing re-enrolls a reference set once it is written, so the wrong crops persist
@@ -91,21 +106,21 @@ One number sets the agenda:
The chibi at 1:35 will survive all of this. He genuinely is brown hair plus a yellow shirt, so a
profile match is correct on appearance and wrong on reality. That needs item 4 below, plus requiring
a real face before a crop can enroll.
3. **Stop the narration inventing facts.** 0:43, 2:03, 2:05 and 2:15 assert things no panel shows. The
6. **Stop the narration inventing facts.** 0:43, 2:03, 2:05 and 2:15 assert things no panel shows. The
correctness verifier passed 116/116 because it checks quotes and names, never invented claims.
4. **Teach vision that art inside a panel is not the scene.** A chibi on a monitor became "a man holding
7. **Teach vision that art inside a panel is not the scene.** A chibi on a monitor became "a man holding
a drink" at 1:35. A colleague pointing into the distance became "pointing towards the screen" at
1:59.
5. **`layers` writes nothing** and reports `completed 116/116`, so no clip has parallax and a still
8. **`layers` writes nothing** and reports `completed 116/116`, so no clip has parallax and a still
holds for 28s from 2:24 (`caveats/audit-open.md#layers-writes-nothing`).
6. **Clear the stale job error.** The completed job still carries `error: "partial: 112/116 completed"`
9. **Clear the stale job error.** The completed job still carries `error: "partial: 112/116 completed"`
(`caveats/audit-open.md#stale-job-error`).
7. Balloon-to-speaker geometry via the unused `det`/`seg` heads
(`caveats/speaker-attribution.md#tail-is-not-geometry`) is now behind item 2. With no name to attach,
10. Balloon-to-speaker geometry via the unused `det`/`seg` heads
(`caveats/speaker-attribution.md#tail-is-not-geometry`) is now behind item 5. With no name to attach,
geometry buys nothing.
8. Resolve a speaker answer across the whole dialogue window, not just the answering panel. The last 3
11. Resolve a speaker answer across the whole dialogue window, not just the answering panel. The last 3
unresolved refs describe a neighbouring panel in the same 8-panel call.
9. Start Phase 2 from `ROADMAP.md`. Set SQLite `busy_timeout` before any concurrency work
12. Start Phase 2 from `ROADMAP.md`. Set SQLite `busy_timeout` before any concurrency work
(`caveats/audit-open.md#sqlite-locking`).
## Lesson worth keeping
caveats/CLAUDE.md
+2
View File
@@ -42,3 +42,5 @@ a complaint, so give it one or drop it.
| [Identity cannot say "a person with no name"](speaker-attribution.md#no-anonymous-identity) | 2026-08-12 panel 7 |
| [Vision does not separate a background extra from cast](speaker-attribution.md#extras-as-cast) | 2026-08-12 panel 7 |
| [Cast reference profiles are enrolled from wrong crops](speaker-attribution.md#poisoned-reference-set) | 2026-08-12 panel 7 |
| [Detection can order a bbox backwards](audit-open.md#degenerate-bbox) | next vision run |
| [Identity coverage has fallen on every run since the gate landed](audit-open.md#coverage-trend) | before the next downstream run |
caveats/audit-open.md
+26
View File
@@ -181,3 +181,29 @@ Every clip in that run therefore has no parallax. The stage is a sibling of
Costs: a silent quality regression that no status field reveals.
Revisit when: parallax matters for a deliverable, or before quoting this run as a full-pipeline pass.
## Detection can order a bbox backwards {#degenerate-bbox}
`p007 person_1` came back as `[226, 417, 130, 551]` on the 2026-08-12 17:38 run: x1 greater than x2. One
detection in 117. `_bbox_to_pixels` clamps every coordinate into the panel but never orders the corners,
so the box survives as a zero-or-negative-width region. It crops to nothing, so that detection can never
enroll, embed or match, and it is silently lost rather than reported.
The guard is two `min`/`max` pairs in `_bbox_to_pixels`. It was not written this session because it is a
worker change and needs a vision worker restart plus a GPU cycle to prove.
Revisit trigger: the next vision run. Count degenerate boxes against 1 in 117.
## Identity coverage has fallen on every run since the gate landed {#coverage-trend}
70% -> 61% -> 50% across the 13:11 baseline, the 16:39 run and the 17:38 run. The `has_face` gate explains
part of it and is stable, gating 39% of detections on both post-change runs. Against face-bearing
detections only, the 17:38 run assigned 59 of 72, or 82%.
Nothing yet separates correct abstention from lost cast, and both fixes that could have caused the second
drop landed together. A back-turned lead is a correct abstention. A real character the resolver refused is
not, and the two are indistinguishable in the coverage number alone.
Revisit trigger: before trusting the registry for a downstream run. `identity_labels` already exists for
exactly this and holds 145 rows of human ground truth, so `eval_identity.py` can score precision against
recall instead of counting assignments.
decisions/CLAUDE.md
+2
View File
@@ -45,3 +45,5 @@ still live belongs in `caveats/`.
| [A detection with no detected face never enrolls or binds](identity-bbox.md#face-gates-enrollment) | closed |
| [A resolver NONE mints an anonymous character, it does not clear the crop](identity-bbox.md#none-mints-an-anonymous-character) | closed |
| [The extras gate runs at enrollment and at narration, not at the speaker prompt](identity-bbox.md#extras-gate-consumers) | closed |
| [A roster name is a guess, so it never reaches detection](identity-bbox.md#roster-does-not-name) | closed |
| [`merged_into` is exactly one hop deep](identity-bbox.md#merge-chains-flatten) | closed |
decisions/identity-bbox.md
+38
View File
@@ -183,3 +183,41 @@ previous behaviour.
Forbids: adding a fourth consumer of `vision["characters"]` without deciding which side of this line it is
on. The blob keeps every detection on purpose, so the audit can still see what was gated.
## A roster name is a guess, so it never reaches detection {#roster-does-not-name}
`service.py` seeded `known_characters` with `_roster_char_hints` before every detect call, so the chapter
roster's names were in front of gemma before any panel had identified anyone. On the 2026-08-12 16:39 run
that put "Seonho (short brown hair, yellow plaid shirt)" on a different man wearing glasses, who then held
15 assignments under the lead's name. The real lead was minted separately from the p010 caption as
"Lim Seonho", and p020 held both as two people in one panel.
The seeding is removed. A name now reaches detection only from a registry row, and a registry row is named
from an in-panel caption or address through `name_claims`. The distinction is evidence: a roster name is a
claim about art nobody has looked at yet, and a registry row carries an embedding plus the panel that
named it.
The roster still feeds `roster_cast` in `run_stage_dialogue`. There the names are matched against speech,
not against faces, so a wrong guess costs an unresolved speaker rather than a wrong face binding.
Confirmed on the 17:38 run: "Lim Seonho" came back as one row with 25 assignments, so dropping the hint
did not split the lead across the panels before his caption. "Seonho" fell to 1 assignment.
Not covered: detection still reads a name off the panel and can attach it to the wrong body. p011 and p026
emit `name: "Seonho"` on `person_2` with no roster hint present. That is a separate mechanism and it is
what keeps the glasses man named at all.
## `merged_into` is exactly one hop deep {#merge-chains-flatten}
A merge chain means `merged_into` points at a row that is itself merged, so anything resolving one hop
lands on a retired character. Roster readers filter `merged_into IS NULL` and were never affected.
Two rules, one per direction, and both are needed:
- `merge_characters` resolves the keeper to its chain root before merging. Bounded 64-step walk, the cap
only exists so a cycle cannot hang reconcile.
- retiring a loser repoints every row whose `merged_into` was that loser.
The keeper walk alone is not enough, which the 17:38 run proved by producing
`477c1894 -> a92d9df4 -> 4fb94c15` with the walk deployed. At merge time that pair was fine. The chain
formed later, when a row that was already somebody's keeper was itself retired.