kami/manga-recap-pipeline
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Hand off the seventh session: cast gallery run, naming fixes, architecture

Browse Source 
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
kami
2026-08-13 00:41:11 +04:00
parent 97cb4831f9
commit dae9dbc93a
1 changed files with 77 additions and 80 deletions
Download Patch File Download Diff File Expand all files Collapse all files
HANDOFF.md
+77 -80
View File
@@ -1,124 +1,121 @@
# HANDOFF, 2026-08-12 (sixth session) # HANDOFF, 2026-08-12/13 (seventh session)
Live state is in `NEXT.md`. This file is only what this session did. The previous handoff is in Live state is in `NEXT.md`. This file is only what this session did. The previous handoff is in
`JOURNAL.md`. `JOURNAL.md`.
## Asked ## Asked
The two context files, then "go" on the third GPU cycle, then "commit", then "what's next". Then the user "go" on item 1 of the previous plan, then a fourth GPU cycle. Then "commit and let's figure out how to
checked the lead's crops in the review UI and said the assignments were wrong. Then "test on one of the deal with the 2b1b". Then a long architecture message, "mind turning the architecture in docs?", plus a
panels first". Then "mind presenting the panels with boxes". Then "so, we didn't fix it properly?". Then question about panel merging before cropping. Then "well, I'd like the merge to be wired actually. and
"go with the 1". then 3 of your fixes."
## Result ## Result
One worker fix proven on a GPU. One measurement that killed the plan's item 1. One GPU cycle. Then the user Item 1 written, run on a GPU, committed. Naming diagnosed from a real dialogue run and three fixes written
found the registry is over-merged, which invalidates the cycle's headline numbers, and two more fixes were against it. The crop merge wired. `ARCHITECTURE.md` written. Nothing has run on a GPU since 20:25 UTC.
written for it. The third fix the user chose, item 1 of the three options, is NOT started.
| metric | 17:38 run | 18:07 run |
| --- | --- | --- |
| detections | 117 | 119 |
| assignments | 59 | 68 |
| coverage | 50% | 57% |
| degenerate boxes | 1 | 0 |
| assigned among face-bearing | 59/72 = 82% | 68/71 = 96% |
| assigned among gated | -- | 0 |
| chains deeper than one hop | 1 | 0 |
| `merged_from` stamps | 9 | 22 |
Cycle timings, 18:07-18:13 UTC: vision 116/116 in 3m59s, identity 116/116 in 1m24s, reconcile 18/18 in 50s.
## Committed and proven on a GPU ## Committed and proven on a GPU
`_bbox_to_pixels` sorts each coordinate pair after clamping `a386e9d` here, `2daaa84` in the orchestrator. **The resolver gallery is the live cast, not cosine's top-k**
(`decisions/identity-bbox.md#bbox-corners-ordered`). 0 degenerate boxes over 119 detections, against 1 in (`decisions/identity-bbox.md#cast-is-the-gallery`). `tracklets.cast_gallery` builds it from
117. Commit `54bd126` on `restore-runtime`. Its caveat is deleted. `get_known_characters`, gender-compatible, named first, `GALLERY_CAP = 10`, re-read per tracklet. The
`if s.get("candidates")` guard is gone, so a crop with an empty cosine top-k now reaches the resolver.
`worker_vision.REF_BUDGET = 12` spreads reference images, `max(1, min(3, 12 // n))` apiece.
## Measured, no code Fourth cycle, 19:44-19:52 UTC: vision 116/116 3m55s, identity 116/116 2m44s, reconcile 20/20 44s. Ran with
the span cap and `GENERIC` tokenizer, which had never touched a GPU either.
**The 145 ground-truth labels are for a different manga.** Every row in `identity_labels` keys to chapter | metric | 18:07 run | 19:44 run |
`8ca8249b`, cast "Rico" and "Ikekin", 81 panels. Chapter `7c944dd4` has zero, so | --- | --- | --- |
`/review/identity?job_id=778297bc...` returns `labeled: 0, accuracy: null`. Scoring `8ca8249b` gives 7/138 | detections | 119 | 119 |
on a run with 44 assignments over 246 panels, predating every fix. | assignments | 68 | 60 |
| coverage | 57% | 50% |
| tracklets / crops | 12 / 64 | 33 / 72 |
| lead's assignments | 36 | 16 |
| characters after reconcile | 18 | 14 |
| minted / cleared | -- | 10 / 12 |
**Coverage is the `has_face` gate and nothing else.** All 68 assignments landed on face-bearing detections Coverage fell because gemma clears 12 crops instead of naming them wrongly. Checked by eye and confirmed by
and none on a gated one. Recall among face-bearing detections is 96%, up from 82%. the user. The lead's 16 are 14 him plus 2 art-in-art. `character_2b1b12a1` holds 13, all correct.
`character_f0d4e901` holds 9, of which 2 are `2b1b12a1`.
**Embedding cosine cannot separate people.** All 22 crop embeddings for the lead, pulled from ## Measured, then fixed
`manga/<manga_id>/characters/_crops/*.npy`, 1152 dims, L2-normalised. The cat scores up to 0.82 against
men, two different men score 0.93, the highest pair in the matrix is 0.96
(`caveats/audit-open.md#cosine-not-identity`). This rules out a crop-to-crop cosine link, which was the
fix proposed one message before the measurement.
## Written, tested, NOT deployed and NOT run on a GPU Dialogue ran 116/116 in 5m57s at 20:19 UTC to test the naming path. Six claims, three defects:
Both in `/mnt/server/home/kami/docker-apps/manga-infra/orchestrator/`, uncommitted there: ```
p040 character_2b1b12a1 "Choi Haeseon" caption 1.00 -> NOT promoted
p010 character_b1dd5659 "Lim Seonho" caption 1.00 -> conflict flag
p047 character_b1dd5659 "Seonho" address 0.90 -> conflict flag
p011 character_f0d4e901 "Seonho" address 1.00 -> PROMOTED (wrong body)
p026 character_f0d4e901 "Seonho" address 1.00 -> PROMOTED (wrong body)
p110 character_028d4a49 "Haeseon" address 1.00 -> already named
```
- `tracklets.py`: `link_tracklets` rejects a merge whose group would span more than `window` panels ## Written, tested, NOT run on a GPU
(`decisions/identity-bbox.md#tracklet-span-cap`). `window` bounded each pair, transitivity was unbounded,
and the lead's 22 native assignments came from 3 tracklets spanning 0, 22 and 30 panels.
- `tracklets.py` + `service.py`: one `appearance_tokens` with a `GENERIC` stopword set, and
`service._appearance_tokens` delegates to it, so reconcile's pre-filter is fixed too
(`decisions/identity-bbox.md#generic-tokens`). Whole chains hung on the word `short`, one pair on the
word `hair`.
On the same 22 crops, candidate overlap forced to pass: 3 tracklets at worst span 30 becomes 9 at worst Orchestrator, committed `2927927`, `db.add_name_claim` + `decisions/identity-naming.md`:
span 8.
- `alias_groups`: one name's word set inside another's is the same name, longer wins
- `STRONG_EVIDENCE` gains `caption` at confidence >= 0.9
- a name held by another live character refuses to promote, files `name-already-taken`
Here, committed `97cb483`:
- `merge_faceless_captions` wired into both crop endpoints. It existed and was never called.
- `ARCHITECTURE.md`, the target shape from the user's design, each section carrying what exists today.
## Not started ## Not started
**Item 1, which the user chose: stop letting cosine pick the gallery.** `run_stage_identity` builds - **The head/body split is not fixed.** The wired merge only folds a fragment with text and no face. A body
`union_cands` from the members' cosine top-k. So a metric that cannot separate people decides who gemma is fragment has no dialogue, so `_merge_plan` leaves it solo. No evidence gathered on how often it happens.
even allowed to consider. There are 9 live characters. Send the live cast instead, gender-gated, capped and - The three cheap items from `ARCHITECTURE.md`: a `plane` field per detection, same-panel co-presence as a
logged when truncated. Two traps found while reading it: weak cannot-link, `offscreen` as a fourth `speaker_ref` kind.
- `caveats/audit-open.md#gallery-cap-drops-the-unnamed`: the cap fired at `p097` (16 -> 10) and `p109`
- `/vision/resolve` sends up to 3 reference images per candidate (`worker_vision.py:1057`), so 9 candidates (11 -> 10) and dropped exactly the freshly minted anonymous rows.
is 27 images plus the query. It needs a cap.
- only a crop with a non-empty cosine shortlist enters `shortlists` at all, via the
`if s.get("candidates")` guard in `service.py`. An empty top-k drops the crop from resolution entirely.
Also open: nothing downstream re-ran, the job is still parked at `dialogue waiting`, and vision boxes cats
as people and dresses them (`p081`, `p108`).
## Checks ## Checks
```bash ```bash
.venv/bin/python worker_vision.py # self-check ok, including the swapped-corner assert .venv/bin/python worker_vision.py # ok, including the reference-budget asserts
./check_stale.sh # exit 0 before the cycle and after it .venv/bin/python worker_crop.py # ok
/usr/bin/ssh kami@192.168.1.104 "cd /home/kami/docker-apps/manga-infra/orchestrator && python3 tracklets.py && python3 -m pytest -q --ignore=test_api.py" # self-check ok, 118 passed ./check_stale.sh # exit 0 before the cycle
/usr/bin/ssh kami@192.168.1.104 "cd /home/kami/docker-apps/manga-infra/orchestrator && python3 tracklets.py && python3 test_name_binding.py && python3 -m pytest -q --ignore=test_api.py" # 121 passed
``` ```
Every new assert was confirmed to fail with its fix disabled: the span cap returns `[[0, 1, 2]]`, and the Each new assert was confirmed to fail with its fix disabled.
generic-word pair links with `GENERIC` emptied.
## Next command ## Next command
Deploy the two orchestrator fixes and run the fourth cycle. Re-crop and run the fifth cycle. **Clear the panels prefix first** or the merge will not take effect:
`crop_webtoon` skips an upload when the key exists.
```bash ```bash
cd /home/kami/Programs/n8n-worker && ./check_stale.sh # must exit 0 cd /home/kami/Programs/n8n-worker && ./check_stale.sh # restart crop, it is stale after the wiring
/usr/bin/ssh kami@192.168.1.104 "cd /home/kami/docker-apps/manga-infra/orchestrator && docker compose up -d --build orchestrator" /usr/bin/ssh kami@192.168.1.104 "cd /home/kami/docker-apps/manga-infra/orchestrator && docker compose up -d --build orchestrator"
J=778297bc-e7ce-439d-91b5-8a027060d17f; M=ef105a86-4b7e-4ac4-b45c-b7d83b8f5b5e J=778297bc-e7ce-439d-91b5-8a027060d17f; M=ef105a86-4b7e-4ac4-b45c-b7d83b8f5b5e; C=7c944dd4-e972-42c7-ba60-9f6939548e80
/usr/bin/ssh kami@192.168.1.104 "mc rm --recursive --force homesrv/panels/$M/$C/panels/"
/usr/bin/ssh kami@192.168.1.104 "curl -s -X POST http://127.0.0.1:9090/characters/reset -H 'Content-Type: application/json' -d '{\"manga_id\":\"$M\",\"confirm\":true}'" /usr/bin/ssh kami@192.168.1.104 "curl -s -X POST http://127.0.0.1:9090/characters/reset -H 'Content-Type: application/json' -d '{\"manga_id\":\"$M\",\"confirm\":true}'"
# the reset returns restart_identity_worker: true -- honour it, see the traps # honour restart_identity_worker: true
/usr/bin/ssh kami@192.168.1.104 "curl -s -X POST http://127.0.0.1:9090/stage/clear -H 'Content-Type: application/json' -d '{\"job_id\":\"$J\",\"stage\":\"vision\"}'" /usr/bin/ssh kami@192.168.1.104 "curl -s -X POST http://127.0.0.1:9090/stage/clear -H 'Content-Type: application/json' -d '{\"job_id\":\"$J\",\"stage\":\"crop\"}'"
for S in vision identity reconcile; do /usr/bin/ssh kami@192.168.1.104 "curl -s --max-time 5400 -X POST http://127.0.0.1:9090/stage/run -H 'Content-Type: application/json' -d '{\"job_id\":\"$J\",\"stage\":\"$S\"}'"; done for S in crop vision identity reconcile dialogue; do /usr/bin/ssh kami@192.168.1.104 "curl -s --max-time 7200 -X POST http://127.0.0.1:9090/stage/run -H 'Content-Type: application/json' -d '{\"job_id\":\"$J\",\"stage\":\"$S\"}'"; done
/usr/bin/ssh kami@192.168.1.104 "docker logs manga-orchestrator --since 1h 2>&1 | grep tracklet" # expect ~30 tracklets, was 12
/usr/bin/ssh kami@192.168.1.104 "docker exec manga-orchestrator python3 /app/audit_registry.py 7c944dd4-e972-42c7-ba60-9f6939548e80 6"
``` ```
Then re-check the lead's crops by eye. A stage counter will not show this defect. Expect: fewer than 116 panels, `2b1b12a1` named `Choi Haeseon`, no `conflicting-name-claims` on the lead,
and a `name-already-taken` flag where the female `Seonho` used to be.
## Traps confirmed or found ## Traps confirmed or found
- **`crop_webtoon` skips an upload when the key exists.** Right for a resume, silently wrong after any
slicing change. Clear `s3://panels/<manga>/<chapter>/panels/` before re-cropping.
- Restart a worker with `tmux send-keys -t manga-workers:<window> C-c`, then re-send the launch line from - Restart a worker with `tmux send-keys -t manga-workers:<window> C-c`, then re-send the launch line from
`start_workers.sh` with its `MIOPEN_ENV` prefix. A `pgrep` kill matches the calling shell. `start_workers.sh` with its `MIOPEN_ENV` prefix. A `pgrep` kill matches the calling shell.
- `POST /characters/reset` returns `restart_identity_worker: true`. Honour it, the worker caches the - `POST /characters/reset` returns `restart_identity_worker: true`. Honour it.
registry. - `docker compose up --build orchestrator` drops any `docker cp`'d script, `audit_registry.py` included.
- `identity_assignments` has no `method` column. The `merged_from:` stamps live in - A heredoc piped into `docker exec` over `/usr/bin/ssh` produces no output. `scp` the file, `docker cp`
`identity_assignment_sources`. `chapters` has no `title` column. The orchestrator image has no numpy. it, then run it.
- A heredoc piped into `docker exec` over `/usr/bin/ssh` silently produces no output. Write the script to a
file, `scp` it, `docker cp` it, then run it.
- Mixing `echo` with a `tar cf -` stream over ssh corrupts the archive. Separate the calls.
- `panel_order` and the panel filename differ by one: `panel_order` 10 is `p009.png`. - `panel_order` and the panel filename differ by one: `panel_order` 10 is `p009.png`.
- The contact-sheet script that found the identity errors is `sheet.py` in the session scratchpad, not
committed. It reads `assigns.json` dumped from `identity_assignments` and pulls crops from
`manga/<manga_id>/characters/_crops/`.