From c19a726455bceb9088704690898c37d474d7168d Mon Sep 17 00:00:00 2001 From: kami Date: Thu, 13 Aug 2026 23:41:11 +0400 Subject: [PATCH] Stop dropping strip pixels, so panel bboxes tile the page Answering whether the caption merge breaks the page-space key in eval/chapter-truth.json. It does not. _merge_plan only groups adjacent fragments and slice_webtoon cuts at gutter midpoints, so a vstacked panel's [x, y0, w, sum(heights)] is the exact page region. The defect is one line up. slice_webtoon skipped any segment shorter than min_seg instead of absorbing it, so this chapter's 116 panels have a 62px hole between panel_order 70 and 71 and lose 18px at the strip's end. A merge spanning that hole reports a bbox 62px short of the page span it covers, and every consumer mapping a detection back to the page is then wrong by that much, silently. The 62px held the credits line, found at p_050.png row 338. A short band is now held and absorbed by the next segment, or by the previous one when it is last. min_seg still suppresses a tiny panel and no strip pixel is discarded. The self-check asserts the bboxes tile the strip end to end. Checks: worker_crop.py self-check passes and fails on the old code, where the first band starts at 49 instead of 0. ruff check . exits 0. Co-Authored-By: Claude Opus 5 --- HANDOFF.md | 127 +++++++++++++++------------------ JOURNAL.md | 14 ++++ NEXT.md | 8 +++ decisions/CLAUDE.md | 1 + decisions/measurement-spine.md | 18 +++++ worker_crop.py | 28 +++++++- 6 files changed, 124 insertions(+), 72 deletions(-) diff --git a/HANDOFF.md b/HANDOFF.md index 56c2ab9..1d3dd37 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -1,106 +1,93 @@ -# HANDOFF, 2026-08-13 (eighth session) +# HANDOFF, 2026-08-13 (ninth session) Live state is in `NEXT.md`. This file is only what this session did. The previous handoff is in `JOURNAL.md`. ## Asked -Push the repo to Gitea and open a PR to `master`. Then, from the PR review: fix the s3 URI constants, -run ruff and fix what it finds, push, run a ponytail audit, fix, push, merge. Then audit. +Get up to speed from the eighth handoff, pick the next work, then commit to `master`. Then: can the +merged-panel bug be fixed. ## Result -No GPU work. Nothing ran on a pipeline stage. PR #1 is open at -`https://gitea.kvmx.ru/kami/manga-recap-pipeline/pulls/1`, `master` <- `restore-runtime`. +No GPU work. No stage ran. `master` is at `bc19f9e` and pushed, plus the crop fix below still to commit. -`master` held only the reconstruction commit `ff6a512`. All 27 commits of real work sat unpushed on -`restore-runtime`. +`ARCHITECTURE.md` step 0, the measurement spine, is built for the character half and its DoD is met. ## Landed -| commit | what | +| what | where | | --- | --- | -| `12d4ed8` | the `ARCHITECTURE.md` rewrite, which was finished but uncommitted | -| `be63b22` | deleted a 0-byte file named `II, d[i+4:i+12]))\n"; done`, added by `8113bdf` | -| `bec9411` | s3 URI templates into `transport.py`, plus `ruff.toml` and the lint fixes | +| `eval/chapter-truth.json`, 38 labelled occurrences | new file | +| purity and fragmentation, plus a `--selftest` | `audit_registry.py` | +| a short slice is absorbed, not dropped | `worker_crop.py:slice_webtoon` | +| the design, 4 sections | `decisions/measurement-spine.md` | -### S3 URIs +### Step 0, the measurement spine -Eight templates now live in `transport.py`: `PANEL_URI`, `PAGE_PANEL_URI`, `AUDIO_URI`, -`AUDIO_FLAT_URI`, `LAYER_URI`, `CLIP_URI`, `CHAPTER_URI`, `CHAR_PNG_URI`, `CHAR_NPY_URI`. Five workers -formatted their own before. +``` +truth: 38 labelled occurrences, 38 matched an assignment at IoU >= 0.5, 0 unmatched + LIM SEONHO [b1dd5659] lead 14/16 = 0.88 = baseline (wrong: photo_of_lead 1, chibi_of_lead 1) + character_2b1b12a1 woman_a 13/13 = 1.00 = baseline + Seonho [f0d4e901] woman_b 7/9 = 0.78 = baseline (wrong: woman_a 2) + fragmentation lead 1, woman_a 2, woman_b 1 all = baseline +``` -`transport.ids_from_uri` replaces three separate copies of the same parse in `worker_tts`, -`worker_layers` and `worker_render`. It raises on a uri too short to carry the ids instead of returning -a wrong pair. `worker_render._mc_from_uri` is gone, its 6 call sites repointed. +Rows key on page-space geometry. Purity is the largest share of one true person in a cluster. +Fragmentation is the count of ids holding one person. So neither needs a `character_id`. The file +survives the re-crop and the `/characters/reset` that the fifth cycle runs. That was the handoff's +open ordering trap. -### Lint +`NEXT.md` said 2 of woman B's 9 crops were really woman A and never said which. They are `panel_order` +31 and 33, identified from `p030` and `p032`. -`ruff check .` exits 0. Ruff's defaults found 115. Fixed: implicit `Optional` in 8 signatures, an -unparenthesized implicit concatenation inside the ASS filter list, 5 `subprocess.run` calls now saying -`check=False` out loud, 1 unused import, 1 duplicate exception handler, 1 non-executable shebang, -4 `dict()` calls and 2 `startswith` chains. +### The crop fix, answering "can we fix the merged panels bug" -12 rules are off in `ruff.toml`, each with its reason. The ASYNC ones matter: ffmpeg and ffprobe run -synchronously inside `async def` endpoints, so a busy worker cannot answer `/health`. That is already -`caveats/audit-open.md#blocking-event-loop`, tracked as [#199], and it needs a per-handler refactor -rather than a lint fix. +There is no merged-panel bug. `_merge_plan` only groups adjacent fragments and `slice_webtoon` cuts at +gutter midpoints, so a vstacked panel's `[x, y0, w, sum(heights)]` is the exact page region. -`CLAUDE.md` said "There is no lint or build step". It now names `ruff check .` and the rule that every -output URI is a template in `transport.py`. +The real defect was one line up. A segment under `min_seg=64` was skipped outright, so the 116 panels +have a 62px hole between `panel_order` 70 and 71 and lose 18px at the strip's end. A merge spanning that +hole would report a bbox 62px short of the page span. Every page-space mapping across it would then be +wrong by that much, silently. The 62px held the credits line at `p_050.png` row 338. `slice_webtoon` now +absorbs a short band into its neighbour (`decisions/measurement-spine.md#slices-tile-the-strip`). -## Audit findings +### Audit fixes, all pre-existing -Applied: +- 20 characters counted where 14 are live and 6 are merge losers kept on purpose +- the assignment spread keyed on `name`, so the two characters named `Seonho` summed into one line of 11 +- the default worked example was `panel_index` 7, a panel vision skips. `NEXT.md`'s "panel 7" is + `panel_order` 7, one lower, the office shot with 6 detections +- nothing about skipped panels. 41 of 116 are `skip=True`, four checked and all four correct -- `delete:` `attic/worker_ocr.py` and `attic/worker_parse.py`, 224 lines, imported by nothing and named - in no doc. `attic/char-recognition.md` and `attic/plan-workpc.md` kept, they are design history. -- `delete:` `worker_vision._panel_size`, 8 lines, one reference and it is the definition. +## Open -Found and NOT applied, in order of size: - -- `shrink:` `call_gemma4`, `_extract_json` and `_strip_thought` each exist twice, in `worker_vision.py` - and `worker_script.py`, and **have already diverged**. `worker_vision.call_gemma4` is 21 lines and - takes a content list, `worker_script`'s is 8 and takes a prompt string plus a system prompt. The two - `_extract_json` bodies carry the same comment about `raw_decode` but different error text. This is - the live risk: `caveats/audit-open.md#repair-fabricates` says the JSON repair pass can fabricate - dialogue, and a fix would land in one copy. A shared `gemma.py` costs one new file and removes about - 25 duplicated lines. Not done because it touches the two largest workers and nothing has run on a GPU - since. -- `stdlib:` `worker_render._wrap2`, 16 lines of greedy word-wrap with a 2-line cap, is close to - `textwrap.wrap(text, width, max_lines=2, placeholder="…")`. Not identical: `_wrap2` does - `rstrip(".,")` before the ellipsis, and textwrap reserves width for the placeholder, so the break - points differ. It renders burned-in subtitles, so the diff is user-visible. Verify against the render - self-check before swapping. -- `shrink:` `_letterbox` and `_load` are duplicated between `bubble_detect.py` and `face_detect.py`, - two ONNX detectors with the same preprocessing. About 16 lines. -- `delete:` `worker_layers.py:14` points `LAYERED_WORKFLOW` at `legacy/qwen_layered_workflow.json`. - `legacy/` was **never tracked in git** and is absent from disk, so the module's self-check has never - been able to pass here. This is the other half of `caveats/audit-open.md#layers-writes-nothing`. +- **Step 0's dialogue half.** 30 lines with their true speaker, typed + `visible | offscreen | narrator | unknown`. `dialogue` is an empty array in the truth file. It needs + the chapter watched panel by panel, because no per-line speaker truth exists anywhere to transcribe. +- **28 of 122 dialogue lines sit on skipped panels**, which hold no character at all. 23% of dialogue + cannot resolve to a visible speaker under any linker. This is the measured case for `offscreen` as a + fourth `speaker_ref` kind, and it is not built. +- The eighth handoff's five unapplied audit findings are all still unapplied, `gemma.py` first. ## Checks ```bash -ruff check . # All checks passed -.venv/bin/python worker_render.py # ok, ffmpeg ran, about 4 minutes +.venv/bin/python audit_registry.py --selftest # IoU match, greedy tie-break, purity maths +.venv/bin/python worker_crop.py # the slice must tile the strip with no hole +ruff check . # All checks passed ``` -Self-checks pass: `transport`, `collage`, `bubble_detect`, `test_vision_parse`, `worker_crop`, -`worker_scene`, `worker_script`, `worker_identity`, `worker_tts`, `session_manager`, `worker_vision`, -`worker_render`, `face_detect`. - -`worker_layers` fails, and did before this branch, on the missing `legacy/` file above. +The live audit ran against `/data/manga.db` in `manga-orchestrator` and reproduced every baseline number. +Both files must be inside the container and a rebuild drops them, recipe in `AGENTS.md`. ## Next command -The fifth GPU cycle is still the next pipeline work. It is blocked only on the GPU being free. The -exact sequence is in `JOURNAL.md` under the seventh session, and `NEXT.md` item 1 holds the -expectations. **Clear the panels prefix first** or the wired caption merge -will not take effect. +The fifth GPU cycle, unchanged as the next pipeline work, and now measurable on both sides. The GPU is +idle. The workers are **down**, so it starts with `./start_workers.sh` then `./check_stale.sh`. -Before that, or instead of it while the GPU is busy, `ARCHITECTURE.md` step 0 is the measurement spine: -`eval/chapter-truth.json` plus purity and fragmentation printed by `audit_registry.py`. +`slice_webtoon` changed, so the re-crop is now mandatory rather than merely planned, and +**clear `s3://panels///panels/` first** or `crop_webtoon` keeps the old images. -One ordering trap, unresolved: the truth file cannot be keyed on `panel_id` or `character_id`. The -fifth cycle re-crops and calls `/characters/reset`, which destroys both. Key it on page-space geometry, -or write it after the fifth cycle rather than before. +Run the audit after the cycle and compare against `= baseline`. Expect the panel count to drop and one +panel to grow by 62px. A row printed as `unmatched` means the detection moved more than half its area. diff --git a/JOURNAL.md b/JOURNAL.md index 7114e24..d1974b1 100644 --- a/JOURNAL.md +++ b/JOURNAL.md @@ -801,3 +801,17 @@ visible speaker to attribute to under any linker. That is the case for `offscree Step 0's dialogue half is not written. It needs 30 lines watched panel by panel, and `NEXT.md` holds no per-line speaker truth to transcribe. + +Then the question of whether the geometry key survives the caption merge. It does, and checking it found +a different defect one line up. `_merge_plan` only groups adjacent fragments, and `slice_webtoon` cuts at +gutter midpoints, so a vstacked panel's `[x, y0, w, sum(heights)]` is the exact page region. + +But the 116 panels do not tile the strip. A 62px hole sits between `panel_order` 70 and 71, and 18px are +missing at the end, because `slice_webtoon` skipped any segment under `min_seg=64` instead of absorbing +it. The 62px held the chapter's credits line, `글 | H2J · 그림 | 이루리다`. Finding it meant reading the +PNG headers of all 79 tiles to locate the band at `p_050.png` row 338. Worthless content, and a hole in the +page-space y of everything after it. `slice_webtoon` now absorbs a short band into its neighbour +(`decisions/measurement-spine.md#slices-tile-the-strip`), and its self-check asserts the bboxes tile the +strip end to end. + +Nothing ran on a GPU this session. diff --git a/NEXT.md b/NEXT.md index b87d458..070bb99 100644 --- a/NEXT.md +++ b/NEXT.md @@ -98,6 +98,14 @@ Four numbers the audit was misreporting, found while building the truth file on The dialogue half is not written and needs the chapter watched panel by panel, 30 lines with their true speaker. It is the only part of step 0 still open. + **The key held up under a real check, after one crop fix.** A merged panel's bbox is exact. + `_merge_plan` only groups adjacent fragments, and `slice_webtoon` cuts at gutter midpoints. One thing + was not exact. A segment under `min_seg=64` was dropped outright, so the 116 panels had a 62px hole + between `panel_order` 70 and 71, and lost 18px at the strip's end. A merge spanning that hole would + have reported a bbox 62px short. `slice_webtoon` now absorbs a short band instead of dropping it + (`decisions/measurement-spine.md#slices-tile-the-strip`). The dropped 62px held the credits line, so + nothing of value was lost this time. + Steps 1 and 2 are independent and cheap: `plane` and `species` per detection, and the face-plus-person embedding already queued in item 1 above. The weak cannot-link is step 3 and depends on step 2, not on `plane` alone. Co-presence as a hard constraint is load-bearing while cosine cannot diff --git a/decisions/CLAUDE.md b/decisions/CLAUDE.md index 048da74..259fafc 100644 --- a/decisions/CLAUDE.md +++ b/decisions/CLAUDE.md @@ -56,4 +56,5 @@ still live belongs in `caveats/`. | [A name belongs to one character](identity-naming.md#one-name-one-character) | closed, GPU pending | | [The truth file keys on page-space geometry](measurement-spine.md#truth-is-page-geometry) | closed | | [Purity is a share, so it never names an id](measurement-spine.md#purity-is-id-free) | closed | +| [A slice under `min_seg` is absorbed, not dropped](measurement-spine.md#slices-tile-the-strip) | closed | | [The truth is 38 assignments, and holds no dialogue yet](measurement-spine.md#truth-scope-is-38) | open | diff --git a/decisions/measurement-spine.md b/decisions/measurement-spine.md index 46cdaf4..3c320c3 100644 --- a/decisions/measurement-spine.md +++ b/decisions/measurement-spine.md @@ -19,6 +19,24 @@ the key survives. `audit_registry.py:_score` matches a row to an assignment on t The 19:44 `panel_order`, `local_id` and `character_id` are kept per row under `from_19_44`, for debugging only. Nothing matches on them. +## A slice under `min_seg` is absorbed, not dropped {#slices-tile-the-strip} + +**Closed, 2026-08-13.** `worker_crop.slice_webtoon`, checked in its `__main__`. + +The page-space key above needs the panel bboxes to tile the strip. They did not. `slice_webtoon` skipped +any segment shorter than `min_seg`, so its pixels reached nothing and the y chain gained a hole. On the +2026-08-12 chapter it happened twice. 62px went missing between `panel_order` 70 and 71, holding the +credits line `글 | H2J · 그림 | 이루리다`, and 18px at the bottom of the strip. + +62px is small and that content was worthless, which is why it went unnoticed for the whole project. The +rule is not harmless. The same threshold drops a 62px caption. `merge_faceless_captions` vstacks +adjacent fragments, so a merge spanning a hole reports a bbox 62px shorter than the page span it covers. +Every consumer mapping a detection back to the page is then wrong by that much, silently. + +A short band is now held and absorbed by the next segment. The last band is absorbed by the previous one. +`min_seg` still does its job, which is to suppress a tiny panel, and no strip pixel is discarded. +The self-check asserts the bboxes tile the strip end to end and that no panel is under `min_seg`. + ## Purity is a share, so it never names an id {#purity-is-id-free} **Closed, 2026-08-13.** diff --git a/worker_crop.py b/worker_crop.py index 1d8eca6..b09cecd 100644 --- a/worker_crop.py +++ b/worker_crop.py @@ -45,7 +45,10 @@ def slice_webtoon(img, bg_thresh=235, min_gap=20, min_seg=64, max_seg=2500, blan Defaults tuned on real webtoon strips: 235/20 finds true gutters over full-color art; max_seg=2500 (~one phone screen) caps full-bleed art that has no internal gutter. 186: a row is 'blank' when >=blank_frac of its pixels are bright — NOT every pixel (min>=thresh), - because one dark speck/border pixel/stray letter in a true gutter used to defeat the whole cut.""" + because one dark speck/border pixel/stray letter in a true gutter used to defeat the whole cut. + min_seg suppresses a tiny panel, it does not discard the band: a short band is absorbed by its + neighbour, so the returned bboxes tile the strip with no gap + (`decisions/measurement-spine.md#slices-tile-the-strip`).""" gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) if img.ndim == 3 else img row_blank = (gray >= bg_thresh).mean(axis=1) >= blank_frac h, w = gray.shape[:2] @@ -62,15 +65,22 @@ def slice_webtoon(img, bg_thresh=235, min_gap=20, min_seg=64, max_seg=2500, blan i += 1 cuts.append(h) out = [] + held = None # start of a band too short to stand alone, waiting to be absorbed for a, b in zip(cuts, cuts[1:]): + a = a if held is None else held if b - a < min_seg: - continue + held = a # a short band is absorbed by the NEXT segment, never dropped: dropping + continue # it loses strip content and puts a hole in page-space y + held = None n = -(-(b - a) // max_seg) step = (b - a) // n for s in range(n): y0 = a + s * step y1 = b if s == n - 1 else a + (s + 1) * step out.append((img[y0:y1], [0, y0, w, y1 - y0])) + if held is not None and out: # a short band at the bottom has no next segment + _, (x, y0, bw, _) = out[-1] + out[-1] = (img[y0:h], [x, y0, bw, h - y0]) return out or [(img, [0, 0, w, h])] @@ -317,6 +327,20 @@ if __name__ == "__main__": speckled[75, 40] = 0 # stray dark pixel inside the top band — used to defeat min()>=thresh assert len(slice_webtoon(speckled)) == 2, "gutter detection must tolerate a speck" + # a band under min_seg is absorbed, never dropped: the bboxes must tile the strip with no gap. + # the real chapter lost 62px of credits this way, and with it the page-space y of everything after. + short = np.full((400, 100, 3), 255, np.uint8) + short[0:40] = 0 # a 40px band, under min_seg=64 + short[60:110] = 0 # gutter, then two real segments + short[160:380] = 0 + segs = slice_webtoon(short, min_seg=64) + covered = [b for _, b in segs] + assert covered[0][1] == 0, covered # first band absorbed forward, strip starts at 0 + for prev, nxt in zip(covered, covered[1:]): + assert prev[1] + prev[3] == nxt[1], covered # contiguous, no hole + assert covered[-1][1] + covered[-1][3] == 400, covered # the tail band is absorbed too + assert all(b[3] >= 64 for b in covered), covered # and min_seg still suppresses tiny panels + # faceless-caption merge plan (pure grouping). F=face T=text per fragment, top-to-bottom. assert _merge_plan([True, False], [False, True]) == [[0, 1]] # caption below a face -> absorbed up assert _merge_plan([False, True], [True, False]) == [[0, 1]] # caption above a face -> absorbed down