c19a726455
Answering whether the caption merge breaks the page-space key in eval/chapter-truth.json. It does not. _merge_plan only groups adjacent fragments and slice_webtoon cuts at gutter midpoints, so a vstacked panel's [x, y0, w, sum(heights)] is the exact page region. The defect is one line up. slice_webtoon skipped any segment shorter than min_seg instead of absorbing it, so this chapter's 116 panels have a 62px hole between panel_order 70 and 71 and lose 18px at the strip's end. A merge spanning that hole reports a bbox 62px short of the page span it covers, and every consumer mapping a detection back to the page is then wrong by that much, silently. The 62px held the credits line, found at p_050.png row 338. A short band is now held and absorbed by the next segment, or by the previous one when it is last. min_seg still suppresses a tiny panel and no strip pixel is discarded. The self-check asserts the bboxes tile the strip end to end. Checks: worker_crop.py self-check passes and fails on the old code, where the first band starts at 49 instead of 0. ruff check . exits 0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
818 lines
50 KiB
Markdown
818 lines
50 KiB
Markdown
# JOURNAL
|
|
|
|
Append-only, newest last. One block per session or run. Not a changelog: this records what happened on
|
|
the day a number was produced, so a later postmortem can find it.
|
|
|
|
## 2026-08-11 Audit second pass [no task]
|
|
|
|
Command: none. Source reading only.
|
|
Outcome: finished. `AUDIT.md` grew from 565 to 771 lines with a `## Second-pass findings` section:
|
|
4 new P0, 6 new P1, 13 P2, 5 additions to the Phase 1 list.
|
|
Produced: commit `6d9df5b`, `AUDIT.md:566`.
|
|
|
|
## 2026-08-11 Audit Phase 1 implemented [#203]
|
|
|
|
Command: `python worker_scene.py worker_script.py worker_vision.py session_manager.py`,
|
|
`pytest -q --ignore=test_api.py` in the orchestrator.
|
|
Outcome: finished. All self-checks pass, 108 orchestrator tests pass. No GPU work, no pipeline run.
|
|
`test_api.py` was skipped because fastapi is not installed in the workpc venv.
|
|
Produced: `decisions/audit-phase1.md`, `caveats/audit-open.md`, `ROADMAP.md`, and this scaffold.
|
|
|
|
One existing test asserted the bug: `test_name_binding.test_conflict_flags_and_stays_unnamed` relied on
|
|
orphan flags leaking into every chapter, because it never created panel rows. It now creates them.
|
|
|
|
## 2026-08-11 Orchestrator half committed and deployed [no task]
|
|
|
|
Command: `pytest -q --ignore=test_api.py`, then `docker compose up -d --build orchestrator` on homesrv.
|
|
Outcome: finished. 108 tests pass. Commits `1c60710` (orchestrator half) and `94bd4d8` (minio pin) in
|
|
`/mnt/server/home/kami/docker-apps`. Orchestrator and minio both answer health on homesrv.
|
|
|
|
The rebuild recreated `minio` as a side effect and it crash-looped with `exec format error`: the
|
|
compose pin was the arm64 manifest digest of `minio/minio:latest` and homesrv is amd64. Repinned to the
|
|
amd64 digest. Nothing about Phase 1 caused this, but any compose action that recreates minio would have
|
|
hit it, so it was latent, not new.
|
|
|
|
Still unrun against a real chapter.
|
|
|
|
## 2026-08-11 S3 viewer and storage swap, tasks #116/#117 [#116 #117]
|
|
|
|
Command: docker compose on homesrv, `dig`, `openssl s_client`. No pipeline, no GPU.
|
|
Outcome: partial. Viewer works, storage swap staged and unfinished.
|
|
|
|
#117 needed no new software. `stowage` at `~/docker-apps/stowage` was already configured against the
|
|
manga MinIO and had been dead since 2026-07-19 with `exec /sbin/tini: exec format error`: its digest
|
|
pin was the arm64 manifest. Repinned to amd64 `sha256:91be7f13`, chowned `data/` to uid 65532 for the
|
|
new image, and it serves. MinIO had the identical bug, repinned to `sha256:a1a8bd4a`. A sweep of all
|
|
470 local images on homesrv found exactly those two arm64; nothing else in the homelab is affected.
|
|
|
|
#116 is staged, not done. `rustfs` runs alongside MinIO on `127.0.0.1:9010/9011`, pinned
|
|
`sha256:19b105cc`, data at `/mnt/hdd2/rustfs`. Buckets are empty: the `mc` mirror of
|
|
`audio layers manga panels raw video` (350M, all in `manga`) has NOT run. `/mnt/hdd2/minio/data` is
|
|
untouched and is the rollback. RustFS is `1.0.0-beta.12`, labeled `build-type=prerelease`. Cutover
|
|
would give rustfs 9000/9001 and repoint `MINIO_ENDPOINT=minio:9000` in the orchestrator plus
|
|
`stowage/config.yaml`; `transport.py:95` needs no change if rustfs takes `192.168.1.104:9000`.
|
|
|
|
Side quest, unrelated to the pipeline: the shared 41-domain cert stopped renewing. Root cause was DNS,
|
|
not nginx. Every `*.kvmx.ru` name pointed at a hard A record for `109.229.102.117` while the line had
|
|
moved to `109.229.127.149`; the Mercusys DDNS at `kvmx-home.mercusysddns.com` was correct the whole
|
|
time but nothing in the zone referenced it. Fixed with `CNAME * -> kvmx-home.mercusysddns.com` at
|
|
reg.ru. Certificate now issues.
|
|
|
|
Two measurement traps worth remembering. The ISP transparently intercepts ports 80 and 443 by
|
|
Host/SNI, so `curl` from workpc to ANY address returns kvmx.ru content and proves nothing about
|
|
external reachability; bare TCP connects also succeed against arbitrary addresses and then hang. Three
|
|
wrong root causes came out of trusting those probes before checking them.
|
|
|
|
Also patched `~/scripts/migrate-kvmx-https.sh:54` on homesrv. `need_stream_module` used
|
|
`sudo -n nginx -V` and `sudo -n nginx -T`; the NOPASSWD rule covers only `nginx -t`, so it reported
|
|
"stream module is not loaded" whenever it meant "could not ask for a password". Both checks now run
|
|
without sudo. `bash -n` passes and both conditions evaluate true.
|
|
|
|
## 2026-08-11 Per-artifact buckets, rustfs buckets, baseline chapter run [#116]
|
|
|
|
Command: `mc mb` on rustfs, `docker compose up -d --build orchestrator`, `pytest -q --ignore=test_api.py`,
|
|
`./start_workers.sh`, then `/job/create` + `/stage/clear` + `/job/resume` for chapter
|
|
`7c944dd4-e972-42c7-ba60-9f6939548e80` of "Teto X Egen" as job `778297bc-e7ce-439d-91b5-8a027060d17f`.
|
|
Outcome: partial. Storage split landed and is proven by the run. The run itself was still in `direct`
|
|
when the session ended.
|
|
Produced: `decisions/storage-layout.md`, `caveats/speaker-attribution.md`, 109 orchestrator tests pass.
|
|
|
|
Artifacts now split one bucket per class instead of everything under `manga`
|
|
(`decisions/storage-layout.md#bucket-per-artifact`). Both MinIO and rustfs hold all six buckets. The
|
|
run put 79 pages in `raw` and 116 panel crops in `panels`, so the split works end to end.
|
|
|
|
Two containers on homesrv had been dead for two weeks and blocked the work. `manga-fetch` was exited,
|
|
so `/job/create` failed with `httpx.ConnectError`; `manga-web` was exited, so `manga.kvmx.ru` had
|
|
nothing behind it on port 8083. Both started with `docker compose up -d`. Neither is related to the
|
|
storage change. Neither was caught by any check, because nothing watches these containers.
|
|
|
|
Stage timings, 116 panels: crop 85s, vision ~4min, identity ~1min, reconcile ~7min for 35 pairs,
|
|
dialogue ~8min. Faster than the 2026-07-17 run at 75 panels. The webtoon crop that 500'd in July
|
|
succeeded this time.
|
|
|
|
Quality cross-check against the panel images, the point of the run. Dialogue text extraction is
|
|
accurate. Character detection is accurate. Speaker attribution is not: three of three sampled
|
|
two-character panels attribute both speakers to the wrong person, always swapped
|
|
(`caveats/speaker-attribution.md#tail-is-not-geometry`). 24 of 81 speech lines resolve to a named
|
|
character, which is the Phase 1 headline metric at 30%, and the sample says that 30% is not
|
|
trustworthy. 26 of 113 detected people got an identity, and 25 of those 26 went to one character that
|
|
turns out to cover two different women.
|
|
|
|
The run then reached `scene` 116/116 and failed in `script` at 87/116, not on OOM: 28 beats were
|
|
rejected by the script verifier as `unsupported-proper-noun: ['Choi', 'Haeseon']`
|
|
(`caveats/speaker-attribution.md#multiword-name-verifier`). No two-word cast name can pass that check.
|
|
|
|
## 2026-08-11 Speaker provenance and the multi-word cast name
|
|
|
|
Command: `.venv/bin/python worker_vision.py`, `pytest -q --ignore=test_api.py` in the orchestrator.
|
|
Outcome: both pass, 110 orchestrator tests. Nothing deployed, no GPU work, no pipeline run.
|
|
Produced: `decisions/speaker-attribution.md`, two caveats rewritten.
|
|
|
|
`_annotate_speaker_methods` stopped stamping `tail` on a model guess. With two or more characters
|
|
present the guess is dropped to `unknown` at confidence 0.0. With one present it is kept as
|
|
`model_solo` at 0.7, the same claim the solo backstop already makes
|
|
(`decisions/speaker-attribution.md#no-fake-tail`). Grounded `som_face` and `solo_prior` rows are
|
|
untouched. Nothing outside `worker_vision.py` reads the literal `tail`, checked across both repos.
|
|
|
|
`verify_script` now tokenizes each cast name into `allowed`, so `Choi Haeseon` passes as two tokens
|
|
(`decisions/speaker-attribution.md#multiword-cast-names`). That is the 28 beats job `778297bc` lost.
|
|
|
|
The remaining `['Blur']` beat is a true positive that still halts the whole chapter, now recorded as
|
|
`caveats/speaker-attribution.md#one-word-halts-chapter`.
|
|
|
|
Neither fix is live. The orchestrator container is not rebuilt and the workers are not restarted.
|
|
|
|
## 2026-08-11 Rerun from dialogue: the honest speaker number is 9%
|
|
|
|
Command: `/job/cancel`, `/stage/clear dialogue`, `./start_workers.sh`, `/job/resume` on job
|
|
`778297bc-e7ce-439d-91b5-8a027060d17f`, twice. `docker compose up -d --build orchestrator` three times.
|
|
Outcome: `dialogue` 116/116. 112 orchestrator tests pass, `worker_vision.py` self-check passes.
|
|
|
|
The named-speaker share is 9%, 9 of 95 speech lines, down from a reported 30% that counted fake tails.
|
|
Multi-character panels contribute 0 of 40 lines by design. Single-character panels give 9 of 55. All 9
|
|
binds are `Choi Haeseon`, the row that covers two different women.
|
|
|
|
Five defects, four of them found by measuring the run rather than by reading code.
|
|
|
|
1. The fake `tail` label, fixed before the run (`decisions/speaker-attribution.md#no-fake-tail`).
|
|
2. The multi-word cast name in the script verifier
|
|
(`decisions/speaker-attribution.md#multiword-cast-names`).
|
|
3. `/stage/clear dialogue` deleted nothing and reported success. dialogue and direct write onto the
|
|
per-panel vision blob and had no `_STAGE_TABLES` entry, so `run_stage_dialogue` saw
|
|
`"dialogue" in vision` and would have skipped all 116 panels. The proof is the second clear:
|
|
116 dialogue blobs and 75 direct blobs stripped that the first had left. This is
|
|
`caveats/audit-open.md#dishonest-clearing` firing exactly where it was filed.
|
|
4. gemma answers the speaker field with whatever the prompt showed, most often the character
|
|
description, and every such answer became a free-form name that no registry entry matched. 28 of 51
|
|
sampled lines (`decisions/speaker-attribution.md#prompt-label-answers`). After the fix, 3 of 95.
|
|
5. All 7 `som_face` lines pointed at a mark whose face paired to no present character, so the
|
|
highest-trust provenance sat on a line with no speaker. Same defect class as the fake tail.
|
|
|
|
Identity is now the binding constraint, not attribution. 26 of 113 detected people carry an identity,
|
|
23%, and 25 of the 26 are the one over-merged row. Even perfect balloon binding caps this chapter near
|
|
23% named. The person who does hold an identity is stored as `Lim Seonho` while a separate row is named
|
|
`Seonho` with alias `Lim Seonho`, so either name matches two rows, raises `ambiguous-speaker` and binds
|
|
nothing.
|
|
|
|
Two more defects surfaced after `dialogue` finished. All 7 `som_face` lines pointed at a mark whose face
|
|
paired to no present character (`decisions/speaker-attribution.md#unpaired-mark`). Then `script` halted
|
|
at 112/116 because the narrator wrote `"...Hm?"` for the source line `"Uh... hum...?"`, and both verifier
|
|
rules fired on that two-letter interjection
|
|
(`decisions/speaker-attribution.md#interjection-false-positive`). After the fix, `script` passed 116/116,
|
|
the first time this chapter has cleared the verifier. `tts` then ran for the first time.
|
|
|
|
The run then completed end to end for the first time: `tts` 116/116, `layers` 116/116, `render` 116/116,
|
|
`assemble` 1/1, finished 2026-08-11T20:08:16Z. `s3://video/` holds 49 clips and a 50MiB `chapter.mp4`,
|
|
`s3://audio/` 49 objects at 32MiB. The per-artifact bucket split is now proven for every class except
|
|
layers (`decisions/storage-layout.md#bucket-per-artifact`).
|
|
|
|
Two honesty defects surfaced at the finish, both recorded rather than fixed. `layers` reported
|
|
`completed 116/116` with an empty bucket, and the completed job still carries
|
|
`error: "partial: 112/116 completed"` from the failure three resumes earlier.
|
|
|
|
## 2026-08-12 — the video got watched
|
|
|
|
No pipeline ran. The user watched `chapter.mp4` for the first time and read out 19 timestamped defects.
|
|
That found more than the previous four sessions of measuring, because the recorded metrics were all
|
|
measuring whether code ran rather than whether the result was right.
|
|
|
|
Two measurements came out of it. First, `chapter.mp4` is video 436.39s over audio 363.67s, so the
|
|
narration finishes 72.7s before the picture and the gap accumulates. The 49 clips are clean: every one is
|
|
25fps exactly, video and audio agree to 0.03s, and they sum to 363.6s. Assembly adds 72.7s of video and
|
|
no audio. Second, this manga holds 19 character rows of which 3 carry a name, and `Choi Haeseon` holds 25
|
|
of the chapter's 26 identity assignments. That is why the video calls the colleague Choi, never names the
|
|
MC, and flips gender.
|
|
|
|
The A/V bug was narrowed with a per-round probe over the 49 real clips. Round 0 of `_assemble_batched` is
|
|
correct, losing only the xfade overlap per group. Round 1 turns 359s of video into 100s while the audio
|
|
survives at 358.79s. The round-1 filtergraph is arithmetically correct, and re-running the same chain by
|
|
hand over only the 6 encoded intermediates gives a correct 348.24s with no warnings. Round 1 differs by
|
|
holding a 7th input: the leftover 49th clip, which `_assemble_batched` passes through un-encoded. That
|
|
passthrough is a third path beside `concat` and `xfade` and is the prime suspect.
|
|
|
|
`worker_render.py` gained an `FPS = 25` constant, fps normalization in the xfade branch to match the
|
|
concat branch, a `_stream_dur` helper, and a self-check that compares video against audio rather than
|
|
asserting the file is non-empty. The old check only asserted `getsize(out) > 0`, which is how a 20% sync
|
|
failure shipped. Pinning `-r FPS` on the output encodes was tried and reverted: it collapsed the chapter
|
|
to exactly 100.00s by dropping frames to force CFR, which the comment at the concat branch already
|
|
warned about. None of it is committed and none of it fixes the chapter yet.
|
|
|
|
The fps inconsistency between the two branches is real but not proven to be the shipped cause. The scene
|
|
graphs hold 356 `cut` against 6 `fade_black`, so the real run's final round most likely stayed on the
|
|
concat branch where no mixing happens.
|
|
|
|
### Panel 7, checked against the art
|
|
|
|
The same day, the user pulled up panel 7 and checked every detection by eye. It overturned the framing
|
|
this file carried an hour earlier, and it overturned two theories I proposed before being corrected.
|
|
|
|
Panel `7c944dd4-e972-42c7-ba60-9f6939548e80_p007`, a wide establishing shot of an office through a
|
|
window, crop 900x1650. Vision emitted 6 characters. Zero of the two identity bindings are correct and the
|
|
one character who matters is unbound. `person_5`, described as "yellow sweater", is Seonho in the
|
|
foreground and got no identity. `person_6` is the colleague, who has no name in the story, and was
|
|
assigned `Choi Haeseon` at 0.9. `person_2` is a background extra and was assigned `Lim Seonho` at 0.9.
|
|
`person_1` is a window frame with nobody in it. `person_3` and `person_4` are background extras.
|
|
|
|
Three defects stack, recorded as `caveats/speaker-attribution.md#bbox-wrong-space`,
|
|
`#no-anonymous-identity` and `#extras-as-cast`. The `bbox` values are consumed as absolute pixels, and on
|
|
this panel that puts all six boxes in the top third with two inside a speech balloon. Divided by 1000
|
|
four of the six fit tightly. Identity therefore embedded crops of balloon edges and window frames, which
|
|
is how a 0.9 confidence lands on the wrong person. Blank crops embed alike, a plausible mechanism for one
|
|
row absorbing 25 of 26 assignments.
|
|
|
|
Two claims I made and had to withdraw. First, that rescaling by 1000 makes the boxes correct: after
|
|
scaling, `person_1` still sits on an empty window frame and `person_6` clips its subject, and the
|
|
descriptions are unreliable anyway, since `person_6` reads "white shirt" for a green dress. Second, that
|
|
the constraint is 16 nameless rows needing names. The opposite is true. The pipeline mints names onto
|
|
people who have none, and at least one nameless row is a real recurring person who should stay nameless.
|
|
|
|
The "26 of 113 detected people carry an identity" figure that framed the roadmap counted mostly
|
|
background extras. It should not be quoted again.
|
|
|
|
## 2026-08-12, chapter assembly, root cause and fix
|
|
|
|
Reproduced the A/V collapse offline with 49 synthetic clips at `ASSEMBLE_BATCH=8` and six `fade_black`
|
|
boundaries. It came out worse than the shipped run: **two round-0 groups of 8 fresh clips collapsed on
|
|
their own**, so the single-item passthrough theory from yesterday is dead
|
|
(`decisions/chapter-assembly.md#passthrough-innocent`).
|
|
|
|
Bisected one collapsing group by truncating the chain stage by stage:
|
|
|
|
```
|
|
k=7 out= 52.52 correct
|
|
k=8 out= 52.52 the last xfade contributed nothing
|
|
[v6][n7]xfade=duration=0.050:offset=52.500 <- [v6] is 52.52s long, 0.02s of margin
|
|
```
|
|
|
|
`_xfade_chain` took its durations from `_audio_dur`, which is `format=duration`, which is
|
|
`max(video, audio)`. Each clip's audio outlasts its video by about a frame, so the offset accumulator
|
|
crept ahead of the picture. Once the creep passed the transition width, xfade emitted the transition and
|
|
threw away the second input and every clip after it, at `rc 0` with nothing on stderr.
|
|
|
|
Fix: offsets come from `min(_stream_dur(v), _stream_dur(a))`, every input is floored to a whole frame
|
|
count and `trim`/`atrim`ed on both streams, and `_check_assembled` now verifies each encode against the
|
|
predicted timeline instead of trusting the exit code
|
|
(`decisions/chapter-assembly.md#offsets-from-min-stream`, `#check-assembled`).
|
|
|
|
Verified on the 49 real clips of chapter `7c944dd4`, re-downloaded from MinIO:
|
|
|
|
```
|
|
before r1 n=7 XFADE in v=359.29 a=359.60 -> out v= 99.96 a=358.79
|
|
after r1 n=7 XFADE in v=359.61 a=359.62 -> out v=358.76 a=358.76
|
|
chapter v=358.76 a=358.76 gap=+0.00 (shipped: v=436.39 a=363.67 gap=+72.72)
|
|
```
|
|
|
|
`worker_render.py` `__main__` passes. Two checks were added there, because the existing 4-clip A/V assert
|
|
passed all the way through the broken build. One asserts the frame-exact `trim` on both streams, one
|
|
assembles three clips whose audio outlasts their video by 0.4s. Mutation-tested by putting `_audio_dur`
|
|
back: the new check fires with `video=1.80 audio=3.56 expected=3.56`.
|
|
|
|
Not done: `s3://video/.../chapter.mp4` is still the broken 436s file. Rebuilding it means clearing the
|
|
`assemble` stage and resuming, which is CPU-only and was not run.
|
|
|
|
## 2026-08-12, the chapter rebuilt, and the bbox space settled
|
|
|
|
**The rebuild came out byte-identical to the broken file.** Clearing `assemble` and resuming produced
|
|
video 436.392031s over audio 363.674667s and `nb_frames` 9902 again, which proved the xfade fix committed
|
|
earlier today never runs for this chapter. With all-`cut` transitions `assemble` takes the `else` branch,
|
|
a `concat` demuxer with `-c copy`.
|
|
|
|
Reproduced that path offline in seconds and got the shipped numbers exactly. The cause is mixed frame
|
|
rates: 14 of the 49 clips are `r_frame_rate=30/1` at `time_base=1/15360`, the other 35 are `25/1` at
|
|
`1/12800`. `-c copy` writes the output in the first input's timebase, so those 14 play `15360/12800 = 1.2`
|
|
too long with their audio untouched. `collage_cmd` hardcoded `-r 30`, which yesterday's `FPS` sweep
|
|
missed. `decisions/chapter-assembly.md#mixed-rate-stream-copy`.
|
|
|
|
Fixed `collage_cmd` to emit `-r FPS`, and made `assemble` probe `r_frame_rate` across the clips and route
|
|
mixed rates through the re-encoding tree. Rebuilt:
|
|
|
|
```
|
|
before v=436.392 a=363.675 nb_frames=9902 avg_frame_rate=22.69
|
|
after v=364.120 a=364.122 nb_frames=9101 r=25/1
|
|
```
|
|
|
|
The 14 clips in the bucket are still 30fps. Assembly normalizes them, so the chapter is correct without
|
|
re-rendering, but the fast stream-copy path stays disabled for this chapter until `render` re-runs.
|
|
|
|
**The `bbox` space is 0-1000, not pixels.** Pulled all 113 detections from `/review/identity` and
|
|
measured: 47 boxes have `x2` past the 900px panel width, none has `y2` past 1000 on panels 1257 to 2307px
|
|
tall, 21 clamp at exactly 1000 in x, and the whole range is `[0, 1000]`. `/vision` now converts to pixels
|
|
before returning, so identity crops, gated face pairing, the set-of-mark boxes and the review UI all read
|
|
pixels (`decisions/identity-bbox.md#bbox-is-normalized`).
|
|
|
|
Checked by eye the way the user did. Drew the converted boxes on panel 7: five of six land on their
|
|
subject, including `person_5`, who is Seonho in the foreground with headphones and carried no identity.
|
|
`person_1` still frames an empty window mullion, which is the extra-versus-cast caveat, not this one.
|
|
|
|
Not done: `vision` and `identity` have not re-run, so every box, embedding and `ref_image_uris` in the
|
|
registry is still from the wrong space. That rerun is GPU work and was not started.
|
|
|
|
## 2026-08-12, third session — the rerun, and the run that did not count
|
|
|
|
Asked: move on from the chapter rebuild. Scope chosen mid-session: reset the registry, rerun vision,
|
|
identity and reconcile only, and inspect before spending the ~80 minutes downstream.
|
|
|
|
**The registry had to be wiped first, and nothing could do it.** `clear_stage_data` maps `identity` to
|
|
`identity_assignments` only and spares `characters` on purpose (`db.py:790`), because the registry is
|
|
cross-run. So a rerun would have matched fresh crops against 19 stale grid-space embeddings and kept the
|
|
duplicate `Lim Seonho` / `Seonho` rows that raise `ambiguous-speaker`. Added `db.reset_registry` and
|
|
`POST /characters/reset` (`confirm=true` required), which also clears `identity_assignment_sources` —
|
|
untouched by any stage clear, and a leftover `manual` row there makes `assign_identity` refuse the next
|
|
model assignment (`db.py:663`). Covered by `test_db.py:TestResetRegistry`; 115 orchestrator tests pass.
|
|
The orchestrator image bakes its source, so it needed `docker compose up -d --build orchestrator`.
|
|
|
|
**The first full cycle was void: the worker was serving pre-fix code.** vision + identity + reconcile
|
|
completed 116/116/20 and reproduced the defect exactly — 46 of 110 boxes past the 900px panel width,
|
|
coordinates clamping at 1000. The vision worker started 12:00:09, `worker_vision.py` changed 12:11:35,
|
|
and the commit carrying `_bbox_to_pixels` landed 12:16:22. Python had already bound the old module.
|
|
Nothing in the stage output said so (`decisions/identity-bbox.md#stale-worker-invalidates`). Added
|
|
`check_stale.sh`, which compares every worker's process start against its module mtime and exits
|
|
non-zero; mutation-tested by touching `worker_tts.py`.
|
|
|
|
**The rerun against restarted workers.** 8 minutes for all three stages.
|
|
|
|
```
|
|
before stale run after
|
|
x2 past panel width 47/113 46/110 0/110
|
|
coords exactly on 1000 21 27 1
|
|
max y2 1000 1000 2307
|
|
characters 19 8 8
|
|
coverage 26/113 = 23% 77/110 = 70% 77/110 = 70%
|
|
top character's share 96% Haeseon 47% 47%
|
|
```
|
|
|
|
`Choi Haeseon`, which had absorbed 25 of 26 assignments, no longer exists in the registry. On panel 7
|
|
Seonho is bound for the first time, and the unnamed colleague took an anonymous id instead of being
|
|
called `Choi Haeseon` at 0.9.
|
|
|
|
**Fixing the boxes made the extras problem worse.** With crops finally landing on their subjects, a
|
|
background extra bound to `Seonho` at confidence 1.00, putting an extra into the lead's reference set.
|
|
`/vision` now stamps `has_face` per character via `face_detect` + `_pair_faces_to_present`, and identity
|
|
skips `has_face is False` before it crops or embeds
|
|
(`decisions/identity-bbox.md#face-gates-enrollment`). Fails open on a missing or raising detector.
|
|
Self-checked in both workers. **Not yet proven on a GPU run** — that is the next command.
|
|
|
|
Deleted along the way and not recoverable: the rebuilt `chapter.mp4` and all 49 clips, by the cascade
|
|
from `/stage/clear vision`. The user chose not to keep a copy.
|
|
|
|
## 2026-08-12, fourth session — identity 2b, the resolver NONE branch
|
|
|
|
Asked: "how much will `has_face` help with character and identity problems?", then "fix 2b first".
|
|
|
|
Answered the first honestly: `has_face` reaches 2 of 6 detections on panel 7 and nothing else. It does not
|
|
touch naming or merging, and it cannot touch the chibi at 1:35, because an anime face detector detects a
|
|
chibi face. Also flagged its real cost: the model is face-only by design, so back-turned cast lose
|
|
enrollment along with the extras, and coverage is where that shows up first.
|
|
|
|
Measured before writing anything, read-only, no GPU:
|
|
|
|
```
|
|
registry: 8 characters, 1 named -> ['Seonho']
|
|
detections: 110 assignments: 77 = 70% coverage
|
|
spread: Seonho 36, character_565c88 24, character_759e23 9, character_f7a4fd 3,
|
|
character_25f682 3, character_d72710 1, character_823aba 1
|
|
```
|
|
|
|
That killed the assumed cause. Anonymous ids already recur, so the identity worker's own
|
|
pending-promote path gives stable anonymous identities. The defect was elsewhere.
|
|
|
|
`/vision/resolve` can answer "none of these" and always could: `worker_vision.py:1071` returns
|
|
`state="new"` for `choice: 0` and `state="unresolved"` for an out-of-range index. `service.py` read only
|
|
`character_id` and unassigned every crop of the tracklet for either. The stale `ponytail:` comment above
|
|
that block named the real blocker and was right: minting needs an `embedding_uri` the orchestrator cannot
|
|
compute, since siglip and gemma cannot both be resident.
|
|
|
|
Fixed by carrying the embedding, not by adding a GPU pass. `/identity/resolve` writes each crop's
|
|
embedding to the crop's key with a `.npy` suffix and returns `emb_uri`. `tracklets.resolve_outcome` holds
|
|
the three-way decision as a pure function. `service.py` mints via the existing `create_character` and
|
|
falls into the existing assign loop. `decisions/identity-bbox.md#none-mints-an-anonymous-character`.
|
|
|
|
Checks: `worker_identity self-check ok`, `tracklets self-check ok`, 115 passed on homesrv.
|
|
Deployed: image rebuilt, `resolve_outcome` verified inside the running container, `audit_registry.py`
|
|
re-copied after the recreate. Vision and identity restarted, `./check_stale.sh` exits 0.
|
|
|
|
New trap: `tmux respawn-window -k` does not re-run the window command. It leaves a bare shell and the
|
|
worker down. Both workers were dead for two minutes before `/health` caught it.
|
|
|
|
Not run: the GPU cycle. `has_face` and the NONE mint are both unproven on real panels and now land in the
|
|
same run.
|
|
|
|
## 2026-08-12, fourth session, continued — identity 2c, extras versus cast
|
|
|
|
Asked: "fix extra-vs-cast?"
|
|
|
|
Traced the three places that read `vision["characters"]` raw before changing anything. That corrected an
|
|
inference made earlier in the session. `build_scene` already drops an unassigned detection from
|
|
`characters` and `present` (`worker_scene.py:63`), so extras never reached the cast list at all.
|
|
|
|
The leak was their ACTIONS. `actions` was built from every detection, and that list is what the script
|
|
prompt renders and what the correctness verifier uses as evidence. So a background extra's "standing at the
|
|
window" arrived as a fact about the panel with no character attached, and the verifier confirmed it,
|
|
because the action really was in the blob. That is a second mechanism behind the invented-narration
|
|
complaints in item 3, independent of the model inventing anything.
|
|
|
|
Gated two consumers on `has_face is False`, matching the enrollment gate's semantics exactly:
|
|
`worker_scene`'s `actions`/`action`, and `service._beat`, which picks the director's "who" from the first
|
|
three detections and falls back to an action when a detection has no name.
|
|
|
|
Left `service._present_characters` ungated on purpose. It builds the dialogue stage's candidate speakers
|
|
and the set-of-mark boxes. An extra picked as speaker already resolves to unknown, not to a wrong name, so
|
|
the failure is contained. And the gate's cost lands hardest there, since a character drawn from behind has
|
|
no face box and gating would delete a real speaker from the only list that can attribute their line.
|
|
`decisions/identity-bbox.md#extras-gate-consumers`.
|
|
|
|
Checks: `worker_scene self-check ok` with three cases (cast action survives, faceless dropped, missing key
|
|
survives), 115 passed on homesrv. Deployed: image rebuilt, `_beat` verified inside the container, scene
|
|
worker restarted, `./check_stale.sh` clean. Committed `ca46617` and `8b27aec`.
|
|
|
|
Still not run on a GPU. Three changes now ride the same cycle: `has_face`, the NONE mint, and this.
|
|
|
|
## 2026-08-12, fourth session, continued — identity 2d, merge and split
|
|
|
|
Asked: "2d? merge and split?"
|
|
|
|
Checked the registry before planning anything, and 2d as written is stale. The registry reset earlier today
|
|
deleted both worked examples. There is no `Lim Seonho` to merge into, `character_afa7623b` does not exist,
|
|
and the current registry is 8 rows with one named character (`Seonho`). Nothing to merge or split until the
|
|
rerun mints a new set.
|
|
|
|
So the useful work was the safety net for that rerun, since `reconcile` runs inside it. The caveat's cost
|
|
line was the reason: one bad merge was unrecoverable without rebaselining the whole manga, and the cycle
|
|
about to run includes a merge pass over embeddings nobody has seen yet.
|
|
|
|
`merge_characters` no longer deletes the loser. It sets `merged_into = keeper`, so the row keeps its
|
|
embedding, description and gender, and it stamps every repointed assignment with
|
|
`method = merged_from:<loser_id>` in `identity_assignment_sources`. That reuses a free-form column on a
|
|
table that already existed rather than adding a merge log. `source` is left alone, so a reviewer's `manual`
|
|
assignment keeps its veto in `assign_identity` after being repointed.
|
|
|
|
Roster readers filter `merged_into IS NULL`, including the name-dedup in `create_character`, which would
|
|
otherwise fold new crops back into a character reconcile had retired. Lookup by id does not filter, because
|
|
an assignment or a narration reference may still point at a merged id.
|
|
|
|
Two existing assertions asserted the old destructive behaviour (`test_db.py:235`, `test_merge_refs.py:37`)
|
|
and were rewritten, not deleted: the invariant changed on purpose.
|
|
|
|
Checks: 116 passed on homesrv, up from 115. Additive `ALTER TABLE` through the existing `init_db` migration
|
|
block, verified on the live database (`merged_into` present, 0 rows merged). Committed `00096cc`.
|
|
|
|
Deliberately not built: the unmerge path and the split. No wrong merge has been observed since the crops
|
|
were fixed, so the consumer of these records waits for one. The forward case is partly covered by 2b, since
|
|
a resolver NONE now mints instead of folding a stranger into the nearest match.
|
|
|
|
## 2026-08-12, fifth session — the GPU cycle all four changes were waiting for
|
|
|
|
Asked: "go ahead" on the rerun.
|
|
|
|
First correction: the vision/identity/reconcile timestamps in `/job/status` are UTC and the git log is
|
|
local (UTC+4), so the run that looked like a rerun was the pre-change baseline. `_mark_has_face` landed at
|
|
19:07 local, the run finished 13:17 local. The handoff was right that nothing had run.
|
|
|
|
Reset the registry for `ef105a86` (8 characters, 77 assignments, 101 sources, 142 S3 objects), cleared
|
|
`vision` and everything downstream (116 vision results), restarted the identity worker for its in-process
|
|
known-character cache, then ran vision, identity and reconcile. 6 minutes wall: vision 3m48s, identity
|
|
1m25s, reconcile 48s. All three `completed`, 116/116/16.
|
|
|
|
### What the four changes did
|
|
|
|
| change | verdict |
|
|
| --- | --- |
|
|
| `has_face` stamp | works, present on all 110 detections, 67 true / 43 gated (39%) |
|
|
| extras gate | works, panel 7's two wrong bindings are gone |
|
|
| NONE mints | works, 16 characters minted where 8 existed |
|
|
| non-destructive merge | works, 7 rows carry `merged_into`, 9 assignments stamped `merged_from:` |
|
|
|
|
Coverage went 70% -> 61% (77 -> 67 assignments over the same 110 detections). That is the gate's price and
|
|
it is mostly correct, see below.
|
|
|
|
### Panel 7, the worked example, before and after
|
|
|
|
```
|
|
person_1 Seonho 1.00 -> -- none -- has_face=False
|
|
person_2 character_f7a4fd 0.00 -> character_519d2b 0.00 has_face=True
|
|
person_4 character_d72710 0.94 -> -- none -- WRONG binding removed
|
|
person_5 Seonho 1.00 -> -- none -- WRONG binding removed
|
|
```
|
|
|
|
The two wrong bindings the gate existed to kill are dead. It also took `person_1`, and that is correct:
|
|
ran `face_detect.detect_faces` on p006.png directly and it finds exactly one face on the whole panel, conf
|
|
0.599, inside `person_2`. Swept the threshold to 0.04 and nothing else appears above 0.056. Looked at the
|
|
crop: `person_1` is the lead drawn from behind at his desk, back of the head and headphones, no face in
|
|
frame. The detector is right and the gate is right. The cost is that a back-turned character cannot be
|
|
identified from that panel at all.
|
|
|
|
Also worth recording: `person_1`'s new box frames the lead correctly. The old note that it framed an empty
|
|
window mullion was written against the pre-fix boxes.
|
|
|
|
### The new defect: a roster hint named the wrong man
|
|
|
|
The registry holds three real people and one wrong name.
|
|
|
|
- `character_92129ac7` "Lim Seonho", 22 assignments. Verified against the art: the p010 introduction panel
|
|
captioned `LIM SEONHO (29)`, yellow plaid shirt and headphones, which is the roster's description for
|
|
"Seonho". This is the lead and the name is right.
|
|
- `character_556aef60`, 25 assignments, unnamed. Verified: the woman with short black hair and pearls, the
|
|
roster's second character, whom the roster itself calls "Unknown".
|
|
- `character_dbadfff7` "Seonho", 15 assignments. Verified: a different man, glasses, dark clothing, in
|
|
profile. He carries the lead's roster name.
|
|
|
|
p020 assigns `Seonho` to `person_1` and `Lim Seonho` to `person_3`, so the pipeline holds them as two
|
|
people in one panel. inference: the roster hint injected at `service.py:882` puts a name in front of
|
|
detection, and detection attached it to the wrong face before the caption panel could mint the real one.
|
|
This is not the alias-merge case and merging the two rows would be wrong. It needs name binding to require
|
|
evidence, the way `name_claims` already does for captions and address.
|
|
|
|
### Two smaller things the run exposed
|
|
|
|
- **Merge chains.** `character_e1ab7776 -> character_521c301f -> character_556aef60`. `merged_into` points
|
|
at a row that is itself merged, so a single-hop resolve lands on a retired character. Roster readers are
|
|
fine because they filter `merged_into IS NULL`. Anything that follows one hop is not.
|
|
- **Orphan `name_claims`.** All 5 rows point at `character_afa7623b`, which the reset deleted. The reset
|
|
clears characters and assignments but not claims.
|
|
|
|
Not run: dialogue and everything downstream. The job sits at `dialogue waiting`.
|
|
|
|
### Same session — the three fixes the run's evidence asked for
|
|
|
|
All three are orchestrator-side, so no worker changed and `check_stale.sh` is not in play.
|
|
|
|
- **Roster hints no longer seed detection.** Deleted the two lines at `service.py:882` that appended
|
|
`_roster_char_hints` to `known_characters`. `build_detect_prompt` drops any hint without a name, so a
|
|
nameless hint would have contributed nothing anyway. Names now reach detection only from registry rows,
|
|
which are embedding-backed and were themselves named from an in-panel caption or address. The roster
|
|
still feeds `roster_cast` in `run_stage_dialogue`, where names are matched against speech.
|
|
- **`merge_characters` resolves the keeper to its chain root** before merging, so `merged_into` stays one
|
|
hop deep. Bounded 64-step walk, the cap only exists so a cycle cannot hang reconcile.
|
|
- **`reset_registry` deletes `name_claims`** for the manga's panels and reports the count.
|
|
|
|
Checks: 117 passed on homesrv, up from 116. The new chain test was confirmed to fail with the walk
|
|
disabled, then the walk was restored and the suite re-run. Deployed by rebuilding the image; verified
|
|
inside the running container by parsing the deployed source, not by grepping it, because the first check
|
|
matched the word `_roster_char_hints` inside the comment that explains its removal.
|
|
|
|
Also copied `audit_registry.py` into the repo before the rebuild. It had only ever been `docker cp`'d, so
|
|
every rebuild dropped it. `Dockerfile` has `COPY . .`, so it is baked now and the trap is gone.
|
|
|
|
Open risk on the next run: without the roster hint the lead is unnamed until the p010 caption, so panels 1
|
|
to 9 may mint him as an anonymous character that reconcile then has to merge. Watch whether "Lim Seonho"
|
|
comes back as one row or two.
|
|
|
|
### Same session — run B, 17:38-17:44
|
|
|
|
Reset (16 characters, 67 assignments, 67 sources, **5 name_claims**, 156 S3 objects), cleared `vision`,
|
|
restarted the identity worker, ran the three stages. The `name_claims: 5` line is the orphan fix confirmed
|
|
on live data.
|
|
|
|
117 detections, 59 assignments, 50% coverage, 20 characters, 72 with `has_face` (45 gated).
|
|
|
|
The registry split is fixed. "Lim Seonho" is one row with 25 assignments, "Seonho" fell from 15 to 1. So
|
|
dropping the roster hint did not split the lead across the panels before his caption, which was the risk
|
|
recorded before the run.
|
|
|
|
Two findings, both now filed:
|
|
|
|
- detection still names from in-panel text and can attach the name to the wrong body (p011, p026), which
|
|
is `decisions/identity-bbox.md#roster-does-not-name`'s "not covered" paragraph
|
|
- one degenerate bbox, `caveats/audit-open.md#degenerate-bbox`
|
|
|
|
And one fix of my own that was incomplete: the merge chain came back as
|
|
`477c1894 -> a92d9df4 -> 4fb94c15` with the keeper walk deployed. The walk only sees the pair being merged
|
|
now. Retiring a row that is already somebody's keeper needed the second half, a repoint of the loser's
|
|
dependents. Both halves and both tests are in `decisions/identity-bbox.md#merge-chains-flatten`.
|
|
|
|
Checks: 118 passed, was 116. Each new merge test was run with its fix disabled and confirmed to fail
|
|
first. Deployed by rebuild and verified by parsing the container's source.
|
|
|
|
Coverage is now 70 -> 61 -> 50 across three runs and is the open question, recorded as
|
|
`caveats/audit-open.md#coverage-trend`. `identity_labels` already holds 145 rows of ground truth, so the
|
|
next move is to score with `eval_identity.py` rather than to keep reading the coverage number.
|
|
|
|
## 2026-08-12, sixth session — the ground truth is for the wrong manga
|
|
|
|
No GPU work. Two things, both cheap.
|
|
|
|
**`_bbox_to_pixels` orders its corners.** Each coordinate pair is sorted after clamping. The self-check
|
|
feeds it run B's real `[226, 417, 130, 551]` and asserts `[117, 688, 203, 909]`. Unsorted the call returns
|
|
`[203, 688, 117, 909]`, so the assert fails without the fix. `worker_vision.py` self-check passes.
|
|
The vision worker is now STALE and needs a restart before the next cycle.
|
|
|
|
**Item 1 of the plan cannot run as written.** All 145 rows of `identity_labels` belong to chapter
|
|
`8ca8249b`, a different manga with cast "Rico" and "Ikekin", spread over 81 panels. Chapter `7c944dd4`
|
|
has none, so `/review/identity?job_id=778297bc...` returns `labeled: 0, correct: 0, accuracy: null`.
|
|
|
|
Scoring `8ca8249b` anyway gives 7/138, with 113 rows labelled as a real person and left unassigned. That
|
|
chapter's identity run is stale: 44 assignments over 246 panels, predating every fix. The number measures
|
|
old code on the wrong chapter.
|
|
|
|
So the eval path is proven end to end and the ground truth is absent. A precision number for the current
|
|
registry needs a hand-labelling pass over `7c944dd4` through `POST /review/identity/label`, keyed by bbox
|
|
rather than by `local_id`, because vision reassigns `local_id` every run.
|
|
|
|
## 2026-08-12 18:07-18:13 UTC, third GPU cycle
|
|
|
|
Restarted vision (window 3) and identity (`/characters/reset` returned `restart_identity_worker: true`),
|
|
both by `send-keys C-c` then re-sending the launch line, never by `pgrep`. `check_stale.sh` exit 0 before
|
|
the run. Reset dropped 20 characters, 59 assignments, 69 sources, 168 S3 objects.
|
|
|
|
vision 116/116 in 3m59s, identity 116/116 in 1m24s, reconcile 18/18 in 50s.
|
|
|
|
| metric | 17:38 run | 18:07 run |
|
|
| --- | --- | --- |
|
|
| detections | 117 | 119 |
|
|
| assignments | 59 | 68 |
|
|
| coverage | 50% | 57% |
|
|
| degenerate boxes | 1 | **0** |
|
|
| `has_face` true / gated | 72 / 45 | 71 / 48 |
|
|
| assigned among face-bearing | 59/72 = 82% | **68/71 = 96%** |
|
|
| assigned among gated | -- | 0 |
|
|
| registry rows / merged | 20 / 7 | 18 / 9 |
|
|
| `merged_from` stamps | 9 | 22 |
|
|
| chains deeper than one hop | 1 | **0** |
|
|
|
|
The corner-ordering fix works: 0 degenerate boxes. The merge-chain fix holds under a heavier merge load,
|
|
9 merges and no chain.
|
|
|
|
**Coverage is now the gate and nothing else.** Every assignment landed on a face-bearing detection and none
|
|
on a gated one. Recall among face-bearing detections is 96%. So the 43% with no assignment is the 40% the
|
|
gate drops plus 3 detections, and the resolver is not losing cast. That closes the part of
|
|
`#coverage-trend` that suspected the resolver.
|
|
|
|
**The open question moved to precision.** The lead holds 36 of 68 assignments, 53%, against 25 of 59
|
|
before, and `audit_registry.py` flags it as absorbing. The merge count rose from 7 to 9 over the same
|
|
interval. Whether 36 is a protagonist in half the panels or an over-merge cannot be told apart without
|
|
labels, which is item 1.
|
|
|
|
Panel 7 got worse, not better: 5 of its 6 detections carry `has_face = False`, and `person_6`, the one that
|
|
does, went unassigned. The previous run bound its colleague. That is the cost of requiring a face, item 4.
|
|
|
|
## 2026-08-12, sixth session, later — the lead was over-merged, and cosine cannot fix it
|
|
|
|
The user checked the lead's crops in the review UI. `LIM SEONHO`'s 36 assignments cover at least six
|
|
different men, a chibi, and a cat. So 36 of 68 is a bug, not a protagonist.
|
|
|
|
Split by mechanism: 22 native resolves, 14 from reconcile merges. Both are broken, for different reasons.
|
|
|
|
**The tracklet linker chained across the chapter.** `window=8` bounds each pair, transitivity was
|
|
unbounded. The 22 native assignments came from 3 tracklets spanning 0, 22 and 30 panels, visible as two
|
|
confidence blocks of eleven rows at 0.95 and ten at 1.0. Fixed by capping the merged group's span
|
|
(`decisions/identity-bbox.md#tracklet-span-cap`).
|
|
|
|
**Both tokenizers linked on words that are not identity.** Whole chains hung on `short`; one pair linked on
|
|
the word `hair` itself. There is now one tokenizer with a `GENERIC` stopword set, shared with reconcile's
|
|
pre-filter (`decisions/identity-bbox.md#generic-tokens`).
|
|
|
|
Measured on the same 22 real crops, candidate overlap forced to pass: 3 tracklets at worst span 30 becomes
|
|
9 at worst span 8.
|
|
|
|
**Then the obvious next fix was ruled out by measurement.** Pulled all 22 crop embeddings from MinIO and
|
|
computed the pairwise cosine. The cat scores up to 0.82 against men, two different men score 0.93, and the
|
|
highest pair in the matrix is 0.96. No threshold exists, so crop-to-crop cosine is not a link signal, and
|
|
the `candidates` overlap condition is close to noise. Filed as
|
|
`caveats/audit-open.md#cosine-not-identity`. Inference: identity embeds the whole person box, so cosine
|
|
measures scene and style, not face.
|
|
|
|
Also seen: vision boxes cats as people and dresses them. `p081` and `p108` are cats, described
|
|
`short brown / yellow plaid shirt` and `short brown / white t-shirt`.
|
|
|
|
Checks: `tracklets.py` self-check ok, orchestrator 118 passed. Both new asserts confirmed to fail with
|
|
their fix disabled. Nothing deployed, nothing re-run on a GPU.
|
|
|
|
Artefact: `lead_tracklets.png`, the 22 panels with boxes, insets and tracklet grouping. Session scratchpad
|
|
only, not committed.
|
|
|
|
## 2026-08-12, 19:44-19:52 UTC — fourth GPU cycle: the cast is the gallery
|
|
|
|
Three fixes ran together for the first time: the tracklet span cap and the shared `GENERIC` tokenizer,
|
|
both written last session and never deployed, plus the new one
|
|
(`decisions/identity-bbox.md#cast-is-the-gallery`). Job `778297bc`, chapter `7c944dd4`, registry reset,
|
|
vision cleared, `vision -> identity -> reconcile`. Vision 116/116 in 3m55s, identity 116/116 in 2m44s,
|
|
reconcile 20/20 in 44s.
|
|
|
|
| metric | 18:07 run | 19:44 run |
|
|
| --- | --- | --- |
|
|
| detections | 119 | 119 |
|
|
| assignments | 68 | 60 |
|
|
| coverage | 57% | 50% |
|
|
| tracklets over crops | 12 / 64 | 33 / 72 |
|
|
| lead's assignments | 36 | 16 |
|
|
| top character's share | -- | 16/60 = 27% |
|
|
| characters after reconcile | 18 | 14 |
|
|
| minted / cleared | -- | 10 / 12 |
|
|
|
|
**Coverage went down and that is the fix working.** gemma cleared 12 crops it used to be forced to name
|
|
from a cosine top-k that did not contain the right person. 72 crops entered resolution against 64, because
|
|
a crop with an empty cosine shortlist is no longer dropped.
|
|
|
|
**Checked by eye, which is the only check that sees this.** Contact sheets of every assigned crop, per
|
|
character, confirmed by the user. The lead holds 16 crops. 14 are him and 2 are art inside a panel, the
|
|
photograph at `order 17` and the chibi at `order 20`. `character_2b1b12a1` holds 13, all of them her, and
|
|
she is a main character the registry never named. `character_f0d4e901` holds 9, of which 7 are her and 2
|
|
are `2b1b12a1` (`order 31`, `order 33`). Against the 18:07 run, where the lead's 36 covered six different
|
|
men, a chibi and a cat.
|
|
|
|
Every wrong crop on the lead is one defect: vision treats art inside a panel as the scene. It is not a
|
|
linker failure and no identity change will fix it.
|
|
|
|
**The cap fired twice and dropped the wrong rows.** `GALLERY_CAP = 10` truncated a 16-row gender-compatible
|
|
cast at `p097` and an 11-row one at `p109`. `cast_gallery` orders named first, so what it drops is exactly
|
|
the recently minted anonymous rows, which is the population a later tracklet most needs to match against.
|
|
Filed as `caveats/audit-open.md#gallery-cap-drops-the-unnamed`.
|
|
|
|
Checks: `worker_vision.py` self-check ok, `tracklets.py` self-check ok, orchestrator 118 passed,
|
|
`./check_stale.sh` exit 0 before the cycle. Vision and identity workers both restarted, the second because
|
|
`/characters/reset` asked for it.
|
|
|
|
Artefacts: `sheet_*.png`, one contact sheet per character. Session scratchpad only, not committed.
|
|
|
|
## 2026-08-13 — the dialogue stage names nobody, and why
|
|
|
|
Ran `dialogue` 116/116 in 5m57s on the fourth cycle's registry, to see whether the fixed identity lets the
|
|
existing `name_claims` path name the female lead. It does not, and the six claims it produced name three
|
|
separate defects.
|
|
|
|
```
|
|
p040 character_2b1b12a1 "Choi Haeseon" caption 1.00 -> NOT promoted
|
|
p010 character_b1dd5659 "Lim Seonho" caption 1.00 -> conflict flag
|
|
p047 character_b1dd5659 "Seonho" address 0.90 -> conflict flag
|
|
p011 character_f0d4e901 "Seonho" address 1.00 -> PROMOTED
|
|
p026 character_f0d4e901 "Seonho" address 1.00 -> PROMOTED
|
|
p110 character_028d4a49 "Haeseon" address 1.00 -> already named
|
|
```
|
|
|
|
All three are fixed in `db.add_name_claim` and recorded in `decisions/identity-naming.md`: alias grouping,
|
|
a confident caption as strong evidence, and one name per character. `test_name_binding.py` replays these
|
|
six claims, 121 tests pass, and each new assert was confirmed to fail with its fix disabled.
|
|
|
|
Also wired `merge_faceless_captions` into both crop endpoints. It had been written and never called; both
|
|
endpoints called the non-destructive `context_fragment_links` instead, and no decision recorded that
|
|
choice. It does not cover the head-in-one-shot, body-in-the-next split that prompted the question, because
|
|
a body fragment has no text and `_merge_plan` only folds a fragment that has text and no face.
|
|
|
|
Found while wiring it: `crop_webtoon` skips an upload when the key exists, which is right for a resume and
|
|
silently wrong after a slicing change. Documented at the line and in `NEXT.md`.
|
|
|
|
Wrote `ARCHITECTURE.md` from the user's design: region graph, occurrence/identity/name, speaker as a scored
|
|
graph edge with a typed union, narrative plane for art-in-art, and a persistent story state machine. Every
|
|
section carries what exists against it today. Nothing in it is built.
|
|
|
|
Nothing ran on a GPU after the dialogue stage.
|
|
|
|
## 2026-08-13 — the measurement spine, built and checked against the run it describes
|
|
|
|
`ARCHITECTURE.md` step 0. No GPU, no stage run. The 19:44 run of 2026-08-12 was still in
|
|
`/data/manga.db`, which is what made this possible before the fifth cycle destroys it.
|
|
|
|
`eval/chapter-truth.json` holds 38 labelled occurrences over the three characters the user walked crop by
|
|
crop. `audit_registry.py` now matches them to the run's assignments and prints purity per cluster and
|
|
fragmentation per person:
|
|
|
|
```
|
|
truth: 38 labelled occurrences, 38 matched an assignment at IoU >= 0.5, 0 unmatched
|
|
LIM SEONHO [b1dd5659] lead 14/16 = 0.88 = baseline (wrong: photo_of_lead 1, chibi_of_lead 1)
|
|
character_2b1b12a1 woman_a 13/13 = 1.00 = baseline
|
|
Seonho [f0d4e901] woman_b 7/9 = 0.78 = baseline (wrong: woman_a 2)
|
|
fragmentation lead 1, woman_a 2, woman_b 1 all = baseline
|
|
```
|
|
|
|
`NEXT.md` recorded that 2 of woman B's 9 crops were really woman A, and never said which. They are
|
|
`panel_order` 31 and 33, identified by pulling `p030` and `p032` from `s3://panels` and looking. Woman A
|
|
wears a cream top with black trim and a pearl necklace, and both crops are unmistakably her at zoom.
|
|
That closes the 7-and-2 arithmetic and is why her fragmentation is 2.
|
|
|
|
The design is in `decisions/measurement-spine.md`. Rows key on page-space geometry rather than
|
|
`panel_id`, purity is a share and fragmentation a count of ids rather than anything naming a
|
|
`character_id`. So the file survives both the re-crop and `/characters/reset` that the fifth cycle runs.
|
|
`audit_registry.py --selftest` covers the IoU match, the greedy tie-break and the purity maths without a
|
|
database.
|
|
|
|
Four things the audit itself was getting wrong, all pre-existing:
|
|
|
|
- it counted 20 characters where 14 are live and 6 are merge losers kept on purpose
|
|
- it keyed the assignment spread on `name`, so the two characters both named `Seonho` summed to one line
|
|
- its default worked example was `panel_index` 7, a balloon-only panel vision skips. `NEXT.md`'s
|
|
"panel 7" is `panel_order` 7, one lower, the office shot with 6 detections
|
|
- it said nothing about skipped panels. 41 of 116 are `skip=True`, so 119 detections come from 75
|
|
panels. Four skips were checked by eye and all four are correct
|
|
|
|
The skip count carries one new fact worth more than the audit fix. Those 41 panels hold 28 of the
|
|
chapter's 122 dialogue lines, and a skipped panel has no character at all. 23% of the dialogue has no
|
|
visible speaker to attribute to under any linker. That is the case for `offscreen` as a fourth
|
|
`speaker_ref` kind, now measured rather than assumed.
|
|
|
|
Step 0's dialogue half is not written. It needs 30 lines watched panel by panel, and `NEXT.md` holds no
|
|
per-line speaker truth to transcribe.
|
|
|
|
Then the question of whether the geometry key survives the caption merge. It does, and checking it found
|
|
a different defect one line up. `_merge_plan` only groups adjacent fragments, and `slice_webtoon` cuts at
|
|
gutter midpoints, so a vstacked panel's `[x, y0, w, sum(heights)]` is the exact page region.
|
|
|
|
But the 116 panels do not tile the strip. A 62px hole sits between `panel_order` 70 and 71, and 18px are
|
|
missing at the end, because `slice_webtoon` skipped any segment under `min_seg=64` instead of absorbing
|
|
it. The 62px held the chapter's credits line, `글 | H2J · 그림 | 이루리다`. Finding it meant reading the
|
|
PNG headers of all 79 tiles to locate the band at `p_050.png` row 338. Worthless content, and a hole in the
|
|
page-space y of everything after it. `slice_webtoon` now absorbs a short band into its neighbour
|
|
(`decisions/measurement-spine.md#slices-tile-the-strip`), and its self-check asserts the bboxes tile the
|
|
strip end to end.
|
|
|
|
Nothing ran on a GPU this session.
|