diff --git a/HANDOFF.md b/HANDOFF.md index b6db079..b8b7fad 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -1,124 +1,121 @@ -# HANDOFF, 2026-08-12 (sixth session) +# HANDOFF, 2026-08-12/13 (seventh session) Live state is in `NEXT.md`. This file is only what this session did. The previous handoff is in `JOURNAL.md`. ## Asked -The two context files, then "go" on the third GPU cycle, then "commit", then "what's next". Then the user -checked the lead's crops in the review UI and said the assignments were wrong. Then "test on one of the -panels first". Then "mind presenting the panels with boxes". Then "so, we didn't fix it properly?". Then -"go with the 1". +"go" on item 1 of the previous plan, then a fourth GPU cycle. Then "commit and let's figure out how to +deal with the 2b1b". Then a long architecture message, "mind turning the architecture in docs?", plus a +question about panel merging before cropping. Then "well, I'd like the merge to be wired actually. and +then 3 of your fixes." ## Result -One worker fix proven on a GPU. One measurement that killed the plan's item 1. One GPU cycle. Then the user -found the registry is over-merged, which invalidates the cycle's headline numbers, and two more fixes were -written for it. The third fix the user chose, item 1 of the three options, is NOT started. - -| metric | 17:38 run | 18:07 run | -| --- | --- | --- | -| detections | 117 | 119 | -| assignments | 59 | 68 | -| coverage | 50% | 57% | -| degenerate boxes | 1 | 0 | -| assigned among face-bearing | 59/72 = 82% | 68/71 = 96% | -| assigned among gated | -- | 0 | -| chains deeper than one hop | 1 | 0 | -| `merged_from` stamps | 9 | 22 | - -Cycle timings, 18:07-18:13 UTC: vision 116/116 in 3m59s, identity 116/116 in 1m24s, reconcile 18/18 in 50s. +Item 1 written, run on a GPU, committed. Naming diagnosed from a real dialogue run and three fixes written +against it. The crop merge wired. `ARCHITECTURE.md` written. Nothing has run on a GPU since 20:25 UTC. ## Committed and proven on a GPU -`_bbox_to_pixels` sorts each coordinate pair after clamping -(`decisions/identity-bbox.md#bbox-corners-ordered`). 0 degenerate boxes over 119 detections, against 1 in -117. Commit `54bd126` on `restore-runtime`. Its caveat is deleted. +`a386e9d` here, `2daaa84` in the orchestrator. **The resolver gallery is the live cast, not cosine's top-k** +(`decisions/identity-bbox.md#cast-is-the-gallery`). `tracklets.cast_gallery` builds it from +`get_known_characters`, gender-compatible, named first, `GALLERY_CAP = 10`, re-read per tracklet. The +`if s.get("candidates")` guard is gone, so a crop with an empty cosine top-k now reaches the resolver. +`worker_vision.REF_BUDGET = 12` spreads reference images, `max(1, min(3, 12 // n))` apiece. -## Measured, no code +Fourth cycle, 19:44-19:52 UTC: vision 116/116 3m55s, identity 116/116 2m44s, reconcile 20/20 44s. Ran with +the span cap and `GENERIC` tokenizer, which had never touched a GPU either. -**The 145 ground-truth labels are for a different manga.** Every row in `identity_labels` keys to chapter -`8ca8249b`, cast "Rico" and "Ikekin", 81 panels. Chapter `7c944dd4` has zero, so -`/review/identity?job_id=778297bc...` returns `labeled: 0, accuracy: null`. Scoring `8ca8249b` gives 7/138 -on a run with 44 assignments over 246 panels, predating every fix. +| metric | 18:07 run | 19:44 run | +| --- | --- | --- | +| detections | 119 | 119 | +| assignments | 68 | 60 | +| coverage | 57% | 50% | +| tracklets / crops | 12 / 64 | 33 / 72 | +| lead's assignments | 36 | 16 | +| characters after reconcile | 18 | 14 | +| minted / cleared | -- | 10 / 12 | -**Coverage is the `has_face` gate and nothing else.** All 68 assignments landed on face-bearing detections -and none on a gated one. Recall among face-bearing detections is 96%, up from 82%. +Coverage fell because gemma clears 12 crops instead of naming them wrongly. Checked by eye and confirmed by +the user. The lead's 16 are 14 him plus 2 art-in-art. `character_2b1b12a1` holds 13, all correct. +`character_f0d4e901` holds 9, of which 2 are `2b1b12a1`. -**Embedding cosine cannot separate people.** All 22 crop embeddings for the lead, pulled from -`manga//characters/_crops/*.npy`, 1152 dims, L2-normalised. The cat scores up to 0.82 against -men, two different men score 0.93, the highest pair in the matrix is 0.96 -(`caveats/audit-open.md#cosine-not-identity`). This rules out a crop-to-crop cosine link, which was the -fix proposed one message before the measurement. +## Measured, then fixed -## Written, tested, NOT deployed and NOT run on a GPU +Dialogue ran 116/116 in 5m57s at 20:19 UTC to test the naming path. Six claims, three defects: -Both in `/mnt/server/home/kami/docker-apps/manga-infra/orchestrator/`, uncommitted there: +``` +p040 character_2b1b12a1 "Choi Haeseon" caption 1.00 -> NOT promoted +p010 character_b1dd5659 "Lim Seonho" caption 1.00 -> conflict flag +p047 character_b1dd5659 "Seonho" address 0.90 -> conflict flag +p011 character_f0d4e901 "Seonho" address 1.00 -> PROMOTED (wrong body) +p026 character_f0d4e901 "Seonho" address 1.00 -> PROMOTED (wrong body) +p110 character_028d4a49 "Haeseon" address 1.00 -> already named +``` -- `tracklets.py`: `link_tracklets` rejects a merge whose group would span more than `window` panels - (`decisions/identity-bbox.md#tracklet-span-cap`). `window` bounded each pair, transitivity was unbounded, - and the lead's 22 native assignments came from 3 tracklets spanning 0, 22 and 30 panels. -- `tracklets.py` + `service.py`: one `appearance_tokens` with a `GENERIC` stopword set, and - `service._appearance_tokens` delegates to it, so reconcile's pre-filter is fixed too - (`decisions/identity-bbox.md#generic-tokens`). Whole chains hung on the word `short`, one pair on the - word `hair`. +## Written, tested, NOT run on a GPU -On the same 22 crops, candidate overlap forced to pass: 3 tracklets at worst span 30 becomes 9 at worst -span 8. +Orchestrator, committed `2927927`, `db.add_name_claim` + `decisions/identity-naming.md`: + +- `alias_groups`: one name's word set inside another's is the same name, longer wins +- `STRONG_EVIDENCE` gains `caption` at confidence >= 0.9 +- a name held by another live character refuses to promote, files `name-already-taken` + +Here, committed `97cb483`: + +- `merge_faceless_captions` wired into both crop endpoints. It existed and was never called. +- `ARCHITECTURE.md`, the target shape from the user's design, each section carrying what exists today. ## Not started -**Item 1, which the user chose: stop letting cosine pick the gallery.** `run_stage_identity` builds -`union_cands` from the members' cosine top-k. So a metric that cannot separate people decides who gemma is -even allowed to consider. There are 9 live characters. Send the live cast instead, gender-gated, capped and -logged when truncated. Two traps found while reading it: - -- `/vision/resolve` sends up to 3 reference images per candidate (`worker_vision.py:1057`), so 9 candidates - is 27 images plus the query. It needs a cap. -- only a crop with a non-empty cosine shortlist enters `shortlists` at all, via the - `if s.get("candidates")` guard in `service.py`. An empty top-k drops the crop from resolution entirely. - -Also open: nothing downstream re-ran, the job is still parked at `dialogue waiting`, and vision boxes cats -as people and dresses them (`p081`, `p108`). +- **The head/body split is not fixed.** The wired merge only folds a fragment with text and no face. A body + fragment has no dialogue, so `_merge_plan` leaves it solo. No evidence gathered on how often it happens. +- The three cheap items from `ARCHITECTURE.md`: a `plane` field per detection, same-panel co-presence as a + weak cannot-link, `offscreen` as a fourth `speaker_ref` kind. +- `caveats/audit-open.md#gallery-cap-drops-the-unnamed`: the cap fired at `p097` (16 -> 10) and `p109` + (11 -> 10) and dropped exactly the freshly minted anonymous rows. ## Checks ```bash -.venv/bin/python worker_vision.py # self-check ok, including the swapped-corner assert -./check_stale.sh # exit 0 before the cycle and after it -/usr/bin/ssh kami@192.168.1.104 "cd /home/kami/docker-apps/manga-infra/orchestrator && python3 tracklets.py && python3 -m pytest -q --ignore=test_api.py" # self-check ok, 118 passed +.venv/bin/python worker_vision.py # ok, including the reference-budget asserts +.venv/bin/python worker_crop.py # ok +./check_stale.sh # exit 0 before the cycle +/usr/bin/ssh kami@192.168.1.104 "cd /home/kami/docker-apps/manga-infra/orchestrator && python3 tracklets.py && python3 test_name_binding.py && python3 -m pytest -q --ignore=test_api.py" # 121 passed ``` -Every new assert was confirmed to fail with its fix disabled: the span cap returns `[[0, 1, 2]]`, and the -generic-word pair links with `GENERIC` emptied. +Each new assert was confirmed to fail with its fix disabled. ## Next command -Deploy the two orchestrator fixes and run the fourth cycle. +Re-crop and run the fifth cycle. **Clear the panels prefix first** or the merge will not take effect: +`crop_webtoon` skips an upload when the key exists. ```bash -cd /home/kami/Programs/n8n-worker && ./check_stale.sh # must exit 0 +cd /home/kami/Programs/n8n-worker && ./check_stale.sh # restart crop, it is stale after the wiring /usr/bin/ssh kami@192.168.1.104 "cd /home/kami/docker-apps/manga-infra/orchestrator && docker compose up -d --build orchestrator" -J=778297bc-e7ce-439d-91b5-8a027060d17f; M=ef105a86-4b7e-4ac4-b45c-b7d83b8f5b5e +J=778297bc-e7ce-439d-91b5-8a027060d17f; M=ef105a86-4b7e-4ac4-b45c-b7d83b8f5b5e; C=7c944dd4-e972-42c7-ba60-9f6939548e80 +/usr/bin/ssh kami@192.168.1.104 "mc rm --recursive --force homesrv/panels/$M/$C/panels/" /usr/bin/ssh kami@192.168.1.104 "curl -s -X POST http://127.0.0.1:9090/characters/reset -H 'Content-Type: application/json' -d '{\"manga_id\":\"$M\",\"confirm\":true}'" -# the reset returns restart_identity_worker: true -- honour it, see the traps -/usr/bin/ssh kami@192.168.1.104 "curl -s -X POST http://127.0.0.1:9090/stage/clear -H 'Content-Type: application/json' -d '{\"job_id\":\"$J\",\"stage\":\"vision\"}'" -for S in vision identity reconcile; do /usr/bin/ssh kami@192.168.1.104 "curl -s --max-time 5400 -X POST http://127.0.0.1:9090/stage/run -H 'Content-Type: application/json' -d '{\"job_id\":\"$J\",\"stage\":\"$S\"}'"; done -/usr/bin/ssh kami@192.168.1.104 "docker logs manga-orchestrator --since 1h 2>&1 | grep tracklet" # expect ~30 tracklets, was 12 -/usr/bin/ssh kami@192.168.1.104 "docker exec manga-orchestrator python3 /app/audit_registry.py 7c944dd4-e972-42c7-ba60-9f6939548e80 6" +# honour restart_identity_worker: true +/usr/bin/ssh kami@192.168.1.104 "curl -s -X POST http://127.0.0.1:9090/stage/clear -H 'Content-Type: application/json' -d '{\"job_id\":\"$J\",\"stage\":\"crop\"}'" +for S in crop vision identity reconcile dialogue; do /usr/bin/ssh kami@192.168.1.104 "curl -s --max-time 7200 -X POST http://127.0.0.1:9090/stage/run -H 'Content-Type: application/json' -d '{\"job_id\":\"$J\",\"stage\":\"$S\"}'"; done ``` -Then re-check the lead's crops by eye. A stage counter will not show this defect. +Expect: fewer than 116 panels, `2b1b12a1` named `Choi Haeseon`, no `conflicting-name-claims` on the lead, +and a `name-already-taken` flag where the female `Seonho` used to be. ## Traps confirmed or found +- **`crop_webtoon` skips an upload when the key exists.** Right for a resume, silently wrong after any + slicing change. Clear `s3://panels///panels/` before re-cropping. - Restart a worker with `tmux send-keys -t manga-workers: C-c`, then re-send the launch line from `start_workers.sh` with its `MIOPEN_ENV` prefix. A `pgrep` kill matches the calling shell. -- `POST /characters/reset` returns `restart_identity_worker: true`. Honour it, the worker caches the - registry. -- `identity_assignments` has no `method` column. The `merged_from:` stamps live in - `identity_assignment_sources`. `chapters` has no `title` column. The orchestrator image has no numpy. -- A heredoc piped into `docker exec` over `/usr/bin/ssh` silently produces no output. Write the script to a - file, `scp` it, `docker cp` it, then run it. -- Mixing `echo` with a `tar cf -` stream over ssh corrupts the archive. Separate the calls. +- `POST /characters/reset` returns `restart_identity_worker: true`. Honour it. +- `docker compose up --build orchestrator` drops any `docker cp`'d script, `audit_registry.py` included. +- A heredoc piped into `docker exec` over `/usr/bin/ssh` produces no output. `scp` the file, `docker cp` + it, then run it. - `panel_order` and the panel filename differ by one: `panel_order` 10 is `p009.png`. +- The contact-sheet script that found the identity errors is `sheet.py` in the session scratchpad, not + committed. It reads `assigns.json` dumped from `identity_assignments` and pulls crops from + `manga//characters/_crops/`.