kami/manga-recap-pipeline
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Record two GPU cycles: the gate works, coverage is the open question

Browse Source 
No worker code changed. This is the evidence from the 2026-08-12 16:39 and
17:38 runs, and where each finding now lives.

The fourth session's four identity changes all work on real panels. Panel 7's
two wrong bindings are gone. The lead going unassigned there is correct and
was measured, not assumed: face_detect finds one face on the whole panel at
conf 0.599, nothing else above 0.056 even at a 0.04 threshold, and the crop
shows him drawn from behind.

Two decisions, both closed: a roster name is a guess so it never reaches
detection, and merged_into is exactly one hop deep. Two caveats, both open:
detection can order a bbox backwards (1 in 117), and identity coverage has
fallen on every run since the gate landed (70 -> 61 -> 50).

Coverage is the thing to settle next, and not by reading the number.
identity_labels already holds 145 rows of ground truth.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
kami
2026-08-12 21:51:36 +04:00
parent c51871348f
commit 18b49c43bd
7 changed files with 366 additions and 120 deletions
Download Patch File Download Diff File Expand all files Collapse all files
HANDOFF.md
+117 -81
View File
@@ -1,124 +1,160 @@
# HANDOFF, 2026-08-12 (fourth session of the day)
# HANDOFF, 2026-08-12 (fifth session)
Live state is in `NEXT.md`. This file is only what this session did. The previous handoff is in
`JOURNAL.md`.
## Asked
"How much will `has_face` help with character and identity problems?", then in order: fix 2b, fix
extra-versus-cast, 2d.
"go ahead" on the GPU cycle the fourth session left staged. Then "what do we do now". Then go ahead on
the fixes its evidence asked for, and run it again.
## Result
Identity item 2 in `NEXT.md` is now done as far as it can go without a run. Four changes are written,
self-checked and deployed. **None has run on a GPU.** No pipeline stage was executed this session.
Two full GPU cycles ran. All four of the fourth session's identity changes are now proven on real panels.
Four more fixes were written on top, all orchestrator-side, all deployed. Coverage fell twice and that is
the open question.
| item | was | now |
| --- | --- | --- |
| 2a bbox space | done last session | unchanged |
| 2b abstain and stay abstained | resolver blamed, unverified | fixed in the orchestrator, `f6804e7` + `ffda3df` |
| 2c extra versus cast | open | two consumers gated, `ca46617` + `8b27aec` |
| 2d merge and split | open | examples gone; merge made non-destructive, `00096cc` |
| cycle | detections | assignments | coverage | registry | named |
| --- | --- | --- | --- | --- | --- |
| baseline (13:11, pre-change) | 110 | 77 | 70% | 8 | 1 |
| run A (16:39-16:45) | 110 | 67 | 61% | 16 | 2 |
| run B (17:38-17:44) | 117 | 59 | 50% | 20 | 2 |
## What each change was, and where the cause turned out to be
Vision is non-deterministic, so detection counts move between runs. Each cycle is ~6 minutes:
vision ~3m50s, identity ~1m25s, reconcile ~50s.
**2b was never the resolver.** `/vision/resolve` at `worker_vision.py:1071` already returns
`state="new"` for `choice: 0` and `state="unresolved"` for an out-of-range index. `service.py` read only
`v.get("character_id")` and ran `unassign_identity` on every crop of the tracklet for either one. A
deliberate NONE produced nothing, so an unnamed recurring person was `unknown` on every panel.
## First, a correction the session started with
The old `ponytail:` comment above that block named the real blocker and was right. Minting needs an
`embedding_uri` the orchestrator cannot compute. siglip is resident in the identity worker and gemma in
the vision worker. Fixed by carrying the embedding, not by adding a GPU pass.
`/identity/resolve` writes each crop's embedding to the crop's key with a `.npy` suffix and returns
`emb_uri`. `tracklets.resolve_outcome` holds the three-way decision as a pure function.
`decisions/identity-bbox.md#none-mints-an-anonymous-character`.
The vision/identity/reconcile timestamps in `/job/status` are UTC. The git log is local, UTC+4. The run
that looked like a completed rerun was the pre-change baseline: it finished 13:17 local, and
`_mark_has_face` was not committed until 19:07. The fourth session's handoff was right that nothing had
run.
**2c: `build_scene` already dropped extras from the cast list.** `worker_scene.py:63` skips an unassigned
detection, so extras never reached `characters` or `present`. Their **actions** did. `actions` was built
from every detection. That list is the script prompt's content and the verifier's evidence. So a background
extra's "standing at the window" arrived as a fact with no character attached, and the verifier confirmed
it. Gated `worker_scene`'s `actions` and `service._beat` on `has_face is False`. Left
`service._present_characters` ungated on purpose, reasoned out in
`decisions/identity-bbox.md#extras-gate-consumers`.
## Run A: the four staged changes, verified
**2d's worked examples no longer exist.** The registry reset deleted `Lim Seonho` and
`character_afa7623b`. The registry is 8 rows, one named. Built the safety net for the coming rerun
instead, since `reconcile` runs inside it: `merge_characters` sets `merged_into = keeper` rather than
deleting, and stamps every repointed assignment `method = merged_from:<loser_id>`. Roster readers filter
`merged_into IS NULL`, lookup by id does not.
`caveats/audit-open.md#destructive-reconcile` is half-closed.
| change | verdict |
| --- | --- |
| `has_face` stamp | on all 110 detections, 67 true / 43 gated (39%) |
| extras gate | panel 7's two wrong bindings gone |
| NONE mints | 16 characters where 8 existed |
| non-destructive merge | 7 rows carry `merged_into`, 9 assignments stamped `merged_from:` |
## Measured, read-only, before writing anything
Panel 7, the worked example:
```
registry: 8 characters, 1 named -> ['Seonho']
detections: 110 assignments: 77 = 70% coverage
spread: Seonho 36, character_565c88 24, character_759e23 9, character_f7a4fd 3,
character_25f682 3, character_d72710 1, character_823aba 1
bbox space: 77/110 exceed 1000, 1 on 1000 -> PIXELS
person_1 Seonho 1.00 -> -- none -- has_face=False
person_2 character_f7a4fd 0.00 -> character_519d2b 0.00 has_face=True
person_4 character_d72710 0.94 -> -- none -- WRONG binding removed
person_5 Seonho 1.00 -> -- none -- WRONG binding removed
```
That killed the assumed cause of 2b. Anonymous ids already recur, so the identity worker's own
pending-promote path gives stable anonymous identities. Only the gemma NONE branch was discarding people.
`person_1` going unassigned is correct, and this was measured, not assumed. Ran `face_detect.detect_faces`
on `p006.png` directly: one face on the whole panel, conf 0.599, inside `person_2`. Swept the threshold to
0.04 and nothing else appears above 0.056. The crop shows the lead drawn from behind at his desk, back of
the head and headphones. Detector right, gate right, lead unidentifiable in that panel.
## Deliberately not built
`person_1`'s box now frames the lead. The old "empty window mullion" note was written against pre-fix
boxes and is retired.
- **The unmerge path and the split.** No wrong merge has been seen since the crops were fixed. Undoing
one today is a hand-written SQL walk of the two records above. Revisit trigger is in the caveat.
- **`service._present_characters` gating.** An extra picked as speaker already resolves to unknown. A
character drawn from behind has no face box. Gating would delete a real speaker from the only list that
can attribute their line.
- **Vision still emits extras into the blob.** Deliberate, so the audit can see what was gated.
## Run A's new defect: a roster hint named the wrong man
Verified against the art, three real people:
- `character_92129ac7` "Lim Seonho", 22 assignments. The p010 caption reads `LIM SEONHO (29)`, yellow
plaid shirt, headphones, matching the roster's description. Correct.
- `character_556aef60`, 25 assignments, unnamed. The woman with short black hair and pearls, the roster's
second character, whom the roster itself calls "Unknown". Correct.
- `character_dbadfff7` "Seonho", 15 assignments. A different man, glasses, dark clothing, wearing the
lead's roster name.
p020 assigned `Seonho` to `person_1` and `Lim Seonho` to `person_3`, two people in one panel.
## What was changed, and where the cause was
**Roster hints no longer seed detection.** Deleted the two lines at `service.py:882` that appended
`_roster_char_hints` to `known_characters`. `build_detect_prompt` drops any hint without a name. Passing a
nameless hint would have contributed nothing, so removing the seeding was the only real option.
Names now reach detection from registry rows only, which carry embeddings and were named from an in-panel
caption or address. The roster still feeds `roster_cast` in `run_stage_dialogue`, where names match
against speech rather than faces.
**`merge_characters` keeps `merged_into` one hop deep.** Two halves, one per direction, and the second was
missed on the first attempt:
- resolve the keeper to its chain root before merging (a bounded 64-step walk, the cap only so a cycle
cannot hang reconcile)
- repoint the loser's own dependents to the keeper when the loser is retired
The keeper walk alone does not work. Run B still produced `477c1894 -> a92d9df4 -> 4fb94c15`. At merge
time that pair was fine. The chain formed later, when a row that was already somebody's keeper was itself
retired.
**`reset_registry` deletes `name_claims`** for the manga's panels and reports the count. Confirmed live:
run B's reset reported `name_claims: 5`, the orphans that had pointed at `character_afa7623b` since two
resets earlier.
**`audit_registry.py` is in the repo.** It had only ever been `docker cp`'d, so every rebuild dropped it.
`Dockerfile` has `COPY . .`, so it is baked now and that trap is gone.
## Run B: what the fixes did
The registry split is fixed. "Lim Seonho" came back as ONE row holding 25 assignments. That answers the
open risk from before the run: dropping the roster hint did not split the lead across the panels before
his caption. "Seonho" fell from 15 assignments to 1.
Two things run B surfaced:
- **A second naming mechanism, untouched by the fix.** Even with no roster hint, p011 and p026 emit
`name: "Seonho"` on `person_2` from in-panel text. Detection reads a name off the panel and attaches it
to the wrong body. That is what keeps the glasses man named, now at 1 assignment instead of 15.
- **One degenerate bbox.** `p007 person_1` is `[226, 417, 130, 551]`, x1 > x2. One in 117. It crops to
nothing, so that detection can never enroll or match. `_bbox_to_pixels` clamps each coordinate but never
orders the corners.
## Not done
- **The degenerate-box guard.** It is a worker change (`worker_vision.py:_bbox_to_pixels`), so it needs a
vision worker restart and a third GPU cycle to prove. Not started, nothing half-edited.
- **The coverage trend.** 70% -> 61% -> 50%. Part is the `has_face` gate, which is stable at ~39% of
detections across both runs. Framed against face-bearing detections only, run B assigned 59 of 72, or
82%. Nothing establishes whether the rest is correct abstention or lost cast. Settle that number before
trusting the registry.
- **Nothing downstream re-ran.** The job is parked at `dialogue waiting` with every stage below it
cleared. No clip or chapter has been rebuilt against the new cast.
## Checks
```bash
.venv/bin/python worker_identity.py # ok
.venv/bin/python worker_scene.py # ok, 3 new cases on the actions gate
./check_stale.sh # exit 0, all 9 workers current
/usr/bin/ssh kami@192.168.1.104 "cd /home/kami/docker-apps/manga-infra/orchestrator && python3 -m pytest -q --ignore=test_api.py" # 116 passed
/usr/bin/ssh kami@192.168.1.104 "cd /home/kami/docker-apps/manga-infra/orchestrator && python3 -m pytest -q --ignore=test_api.py" # 118 passed, was 116
./check_stale.sh # exit 0, all 9 workers current
```
Deployed and verified inside the running container: `resolve_outcome` returns `mint`, `_beat` drops a
faceless detection, `merged_into` exists on the live database with 0 rows merged. Vision, identity and
scene workers restarted.
Both new merge tests were confirmed to fail with their fix disabled, then the fix was restored and the
suite re-run. The deployed container was verified by parsing its source, not grepping it. The first
attempt gave a false negative, matching `_roster_char_hints` inside the comment that explains its removal.
## Next command
Four changes ride one GPU cycle. Coverage is 70% and is the number to beat. A gate that abstains too hard
shows up there before it shows up on panel 7. Watch the identity log line for `minted N anonymous`.
The third cycle, after adding the corner-ordering guard to `_bbox_to_pixels` and restarting the vision
worker. Watch coverage against 50% and the degenerate box count against 1.
```bash
cd /home/kami/Programs/n8n-worker && ./check_stale.sh # must exit 0
J=778297bc-e7ce-439d-91b5-8a027060d17f; M=ef105a86-4b7e-4ac4-b45c-b7d83b8f5b5e
/usr/bin/ssh kami@192.168.1.104 "curl -s -X POST http://127.0.0.1:9090/characters/reset -H 'Content-Type: application/json' -d '{\"manga_id\":\"$M\",\"confirm\":true}'"
/usr/bin/ssh kami@192.168.1.104 "curl -s -X POST http://127.0.0.1:9090/stage/clear -H 'Content-Type: application/json' -d '{\"job_id\":\"$J\",\"stage\":\"vision\"}'"
# restart the identity worker here, see the trap below
for S in vision identity reconcile; do /usr/bin/ssh kami@192.168.1.104 "curl -s --max-time 5400 -X POST http://127.0.0.1:9090/stage/run -H 'Content-Type: application/json' -d '{\"job_id\":\"$J\",\"stage\":\"$S\"}'"; done
/usr/bin/ssh kami@192.168.1.104 "docker exec manga-orchestrator python3 /app/audit_registry.py 7c944dd4-e972-42c7-ba60-9f6939548e80 6"
```
Panel 7 (`panel_index 6`) is the worked example. Before this session's changes:
```
person_1 [457, 657, 642, 937] Seonho 0.00 foreground, correct
person_2 [669, 591, 763, 822] character_f7a4fd 0.00 the unnamed colleague, correct
person_3 [428, 386, 496, 526] -- none -- extra
person_4 [498, 386, 568, 533] character_d72710 0.94 extra, WRONG
person_5 [31, 554, 94, 728] Seonho 1.00 extra bound to the lead, WRONG
person_6 [34, 414, 122, 564] -- none -- extra
```
`person_4` and `person_5` are what `has_face` must remove without taking `person_1` or `person_2` with
them. Restart the identity worker after every reset: it caches the known list in-process and only
invalidates on enrollment.
## Traps confirmed or found
- **`tmux respawn-window -k` does not re-run the window command.** It leaves a bare shell. Both vision and
identity sat dead for two minutes before an empty `/health` caught it. Now in `AGENTS.md`.
- `docker compose up --build orchestrator` recreates the container and drops any `docker cp`'d file, so
`audit_registry.py` needs re-copying after every rebuild.
- Two test assertions asserted the old destructive merge (`test_db.py:235`, `test_merge_refs.py:37`). They
were rewritten, not deleted: the invariant changed on purpose.
- The orchestrator image bakes its source. Editing the repo on homesrv does nothing until the rebuild.
- **`kill $(pgrep -f "worker_identity:app")` kills the shell running it.** The pattern matches the calling
command line. It killed this session's own script mid-way, so the `send-keys` relaunch never fired and
the identity worker sat dead. Restart it with the two `tmux send-keys` lines only, then poll `/health`.
- **Verify deployed code by parsing it, not by grepping it.** A comment explaining a removal contains the
name of the thing removed.
- **A test that cannot fail proves nothing.** Both merge tests were run with their fix disabled first.
- Panel-7 line numbers move between runs, because vision is non-deterministic and `local_id` is assigned
top-to-bottom per run. In run B the lead is `person_5`, not `person_1`. Compare by bbox, not by id.