kami/manga-recap-pipeline
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

File the over-merge findings: two tracklet fixes, and cosine is not identity

Browse Source 
The user checked the lead's crops. LIM SEONHO's 36 assignments cover at least
six different men, a chibi and a cat, so 36 of 68 is a bug. 22 came from
native resolves and 14 from reconcile merges.

Two fixes are written in the orchestrator repo, tested, not deployed and not
run on a GPU. `link_tracklets` caps a tracklet's panel span, because `window`
bounded each pair while transitivity was unbounded and the lead's 22 native
assignments came from tracklets spanning 22 and 30 panels. And one shared
appearance tokenizer drops generic words, because whole chains hung on the
word `short` and one pair on the word `hair`.

The obvious third fix is ruled out by measurement. Over all 22 crop
embeddings, the cat scores up to 0.82 against men, two different men score
0.93, and the highest pair is 0.96. No threshold separates them, so
crop-to-crop cosine is not a link signal.

Item 1 of the agreed plan, sending the live cast instead of a cosine top-k
gallery, is not started.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
kami
2026-08-12 23:35:15 +04:00
parent 54bd126cd7
commit de7e34bc0f
7 changed files with 237 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files
HANDOFF.md
+76 -40
View File
@@ -5,13 +5,16 @@ Live state is in `NEXT.md`. This file is only what this session did. The previou
## Asked
The two files, then "go" on the third GPU cycle.
The two context files, then "go" on the third GPU cycle, then "commit", then "what's next". Then the user
checked the lead's crops in the review UI and said the assignments were wrong. Then "test on one of the
panels first". Then "mind presenting the panels with boxes". Then "so, we didn't fix it properly?". Then
"go with the 1".
## Result
One worker fix, one measurement that killed the plan's first item, one GPU cycle. The cycle proved the new
fix and the fifth session's merge fix. Coverage rose for the first time, 50% to 57%. It is now explained
entirely by the `has_face` gate.
One worker fix proven on a GPU. One measurement that killed the plan's item 1. One GPU cycle. Then the user
found the registry is over-merged, which invalidates the cycle's headline numbers, and two more fixes were
written for it. The third fix the user chose, item 1 of the three options, is NOT started.
| metric | 17:38 run | 18:07 run |
| --- | --- | --- |
@@ -24,65 +27,98 @@ entirely by the `has_face` gate.
| chains deeper than one hop | 1 | 0 |
| `merged_from` stamps | 9 | 22 |
## The corner-ordering fix
Cycle timings, 18:07-18:13 UTC: vision 116/116 in 3m59s, identity 116/116 in 1m24s, reconcile 18/18 in 50s.
`_bbox_to_pixels` sorts each coordinate pair after clamping. Three lines
(`decisions/identity-bbox.md#bbox-corners-ordered`). The self-check feeds it run B's real
`[226, 417, 130, 551]` and asserts `[117, 688, 203, 909]`. Unsorted the call returns `[203, 688, 117, 909]`,
so the assert fails without the fix.
## Committed and proven on a GPU
The run returned 0 degenerate boxes over 119 detections. The caveat is deleted and the decision is indexed.
`_bbox_to_pixels` sorts each coordinate pair after clamping
(`decisions/identity-bbox.md#bbox-corners-ordered`). 0 degenerate boxes over 119 detections, against 1 in
117. Commit `54bd126` on `restore-runtime`. Its caveat is deleted.
## The measurement that killed item 1
## Measured, no code
The plan said to score precision with `eval_identity.py` against the 145 rows of `identity_labels`. Every
one of those rows belongs to chapter `8ca8249b`, a different manga with cast "Rico" and "Ikekin", spread
over 81 panels. Chapter `7c944dd4` has zero labels, so
`/review/identity?job_id=778297bc...` returns `labeled: 0, correct: 0, accuracy: null`.
**The 145 ground-truth labels are for a different manga.** Every row in `identity_labels` keys to chapter
`8ca8249b`, cast "Rico" and "Ikekin", 81 panels. Chapter `7c944dd4` has zero, so
`/review/identity?job_id=778297bc...` returns `labeled: 0, accuracy: null`. Scoring `8ca8249b` gives 7/138
on a run with 44 assignments over 246 panels, predating every fix.
Scoring `8ca8249b` anyway gives 7/138, with 113 rows labelled as a real person and left unassigned. That
chapter's identity run is stale: 44 assignments over 246 panels, predating every fix. The number measures
old code on the wrong chapter.
**Coverage is the `has_face` gate and nothing else.** All 68 assignments landed on face-bearing detections
and none on a gated one. Recall among face-bearing detections is 96%, up from 82%.
So the eval path is proven end to end and the ground truth is absent. A precision number needs a hand pass
over `7c944dd4` through `POST /review/identity/label`, keyed by bbox rather than `local_id`.
**Embedding cosine cannot separate people.** All 22 crop embeddings for the lead, pulled from
`manga/<manga_id>/characters/_crops/*.npy`, 1152 dims, L2-normalised. The cat scores up to 0.82 against
men, two different men score 0.93, the highest pair in the matrix is 0.96
(`caveats/audit-open.md#cosine-not-identity`). This rules out a crop-to-crop cosine link, which was the
fix proposed one message before the measurement.
## What the cycle settled
## Written, tested, NOT deployed and NOT run on a GPU
**Coverage is the gate and nothing else.** All 68 assignments landed on face-bearing detections and none on
a gated one. Recall among face-bearing detections is 96%, up from 82%. The resolver is not losing cast, so
the suspicion in `caveats/audit-open.md#coverage-trend` is closed.
Both in `/mnt/server/home/kami/docker-apps/manga-infra/orchestrator/`, uncommitted there:
**The merge-chain fix holds under load.** 9 merges this run against 7 last run, and 22 `merged_from`
stamps. No chain is deeper than one hop.
- `tracklets.py`: `link_tracklets` rejects a merge whose group would span more than `window` panels
(`decisions/identity-bbox.md#tracklet-span-cap`). `window` bounded each pair, transitivity was unbounded,
and the lead's 22 native assignments came from 3 tracklets spanning 0, 22 and 30 panels.
- `tracklets.py` + `service.py`: one `appearance_tokens` with a `GENERIC` stopword set, and
`service._appearance_tokens` delegates to it, so reconcile's pre-filter is fixed too
(`decisions/identity-bbox.md#generic-tokens`). Whole chains hung on the word `short`, one pair on the
word `hair`.
## What the cycle opened
On the same 22 crops, candidate overlap forced to pass: 3 tracklets at worst span 30 becomes 9 at worst
span 8.
**The lead may be absorbing.** 36 of 68 assignments, 53%, against 25 of 59 before. `audit_registry.py`
flags it. A protagonist in half the panels looks identical to an over-merge without labels.
## Not started
**Panel 7 got worse.** 5 of its 6 detections carry `has_face = False`, and `person_6`, the one that does,
went unassigned. The previous run bound its colleague. That is the cost of requiring a face, `NEXT.md`
item 4.
**Item 1, which the user chose: stop letting cosine pick the gallery.** `run_stage_identity` builds
`union_cands` from the members' cosine top-k. So a metric that cannot separate people decides who gemma is
even allowed to consider. There are 9 live characters. Send the live cast instead, gender-gated, capped and
logged when truncated. Two traps found while reading it:
## Not done
- `/vision/resolve` sends up to 3 reference images per candidate (`worker_vision.py:1057`), so 9 candidates
is 27 images plus the query. It needs a cap.
- only a crop with a non-empty cosine shortlist enters `shortlists` at all, via the
`if s.get("candidates")` guard in `service.py`. An empty top-k drops the crop from resolution entirely.
- Nothing downstream re-ran. The job is still parked at `dialogue waiting` with every stage below cleared.
- Nothing is committed. All edits are in the working tree.
Also open: nothing downstream re-ran, the job is still parked at `dialogue waiting`, and vision boxes cats
as people and dresses them (`p081`, `p108`).
## Checks
```bash
.venv/bin/python worker_vision.py # self-check ok, including the swapped-corner assert
./check_stale.sh # exit 0 before the run and after it
./check_stale.sh # exit 0 before the cycle and after it
/usr/bin/ssh kami@192.168.1.104 "cd /home/kami/docker-apps/manga-infra/orchestrator && python3 tracklets.py && python3 -m pytest -q --ignore=test_api.py" # self-check ok, 118 passed
```
## Traps confirmed
Every new assert was confirmed to fail with its fix disabled: the span cap returns `[[0, 1, 2]]`, and the
generic-word pair links with `GENERIC` emptied.
## Next command
Deploy the two orchestrator fixes and run the fourth cycle.
```bash
cd /home/kami/Programs/n8n-worker && ./check_stale.sh # must exit 0
/usr/bin/ssh kami@192.168.1.104 "cd /home/kami/docker-apps/manga-infra/orchestrator && docker compose up -d --build orchestrator"
J=778297bc-e7ce-439d-91b5-8a027060d17f; M=ef105a86-4b7e-4ac4-b45c-b7d83b8f5b5e
/usr/bin/ssh kami@192.168.1.104 "curl -s -X POST http://127.0.0.1:9090/characters/reset -H 'Content-Type: application/json' -d '{\"manga_id\":\"$M\",\"confirm\":true}'"
# the reset returns restart_identity_worker: true -- honour it, see the traps
/usr/bin/ssh kami@192.168.1.104 "curl -s -X POST http://127.0.0.1:9090/stage/clear -H 'Content-Type: application/json' -d '{\"job_id\":\"$J\",\"stage\":\"vision\"}'"
for S in vision identity reconcile; do /usr/bin/ssh kami@192.168.1.104 "curl -s --max-time 5400 -X POST http://127.0.0.1:9090/stage/run -H 'Content-Type: application/json' -d '{\"job_id\":\"$J\",\"stage\":\"$S\"}'"; done
/usr/bin/ssh kami@192.168.1.104 "docker logs manga-orchestrator --since 1h 2>&1 | grep tracklet" # expect ~30 tracklets, was 12
/usr/bin/ssh kami@192.168.1.104 "docker exec manga-orchestrator python3 /app/audit_registry.py 7c944dd4-e972-42c7-ba60-9f6939548e80 6"
```
Then re-check the lead's crops by eye. A stage counter will not show this defect.
## Traps confirmed or found
- Restart a worker with `tmux send-keys -t manga-workers:<window> C-c`, then re-send the launch line from
`start_workers.sh` with the `MIOPEN_ENV` prefix. `pgrep`-based kills match the calling shell.
`start_workers.sh` with its `MIOPEN_ENV` prefix. A `pgrep` kill matches the calling shell.
- `POST /characters/reset` returns `restart_identity_worker: true`. Honour it, the worker caches the
registry.
- `identity_assignments` has no `method` column. The `merged_from:` stamps live in
`identity_assignment_sources`.
- `chapters` has no `title` column.
`identity_assignment_sources`. `chapters` has no `title` column. The orchestrator image has no numpy.
- A heredoc piped into `docker exec` over `/usr/bin/ssh` silently produces no output. Write the script to a
file, `scp` it, `docker cp` it, then run it.
- Mixing `echo` with a `tar cf -` stream over ssh corrupts the archive. Separate the calls.
- `panel_order` and the panel filename differ by one: `panel_order` 10 is `p009.png`.