Files
kami c19a726455 Stop dropping strip pixels, so panel bboxes tile the page
Answering whether the caption merge breaks the page-space key in
eval/chapter-truth.json. It does not. _merge_plan only groups adjacent
fragments and slice_webtoon cuts at gutter midpoints, so a vstacked panel's
[x, y0, w, sum(heights)] is the exact page region.

The defect is one line up. slice_webtoon skipped any segment shorter than
min_seg instead of absorbing it, so this chapter's 116 panels have a 62px hole
between panel_order 70 and 71 and lose 18px at the strip's end. A merge
spanning that hole reports a bbox 62px short of the page span it covers, and
every consumer mapping a detection back to the page is then wrong by that much,
silently. The 62px held the credits line, found at p_050.png row 338.

A short band is now held and absorbed by the next segment, or by the previous
one when it is last. min_seg still suppresses a tiny panel and no strip pixel
is discarded. The self-check asserts the bboxes tile the strip end to end.

Checks: worker_crop.py self-check passes and fails on the old code, where the
first band starts at 49 instead of 0. ruff check . exits 0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-13 23:41:11 +04:00

818 lines
50 KiB
Markdown

# JOURNAL
Append-only, newest last. One block per session or run. Not a changelog: this records what happened on
the day a number was produced, so a later postmortem can find it.
## 2026-08-11 Audit second pass [no task]
Command: none. Source reading only.
Outcome: finished. `AUDIT.md` grew from 565 to 771 lines with a `## Second-pass findings` section:
4 new P0, 6 new P1, 13 P2, 5 additions to the Phase 1 list.
Produced: commit `6d9df5b`, `AUDIT.md:566`.
## 2026-08-11 Audit Phase 1 implemented [#203]
Command: `python worker_scene.py worker_script.py worker_vision.py session_manager.py`,
`pytest -q --ignore=test_api.py` in the orchestrator.
Outcome: finished. All self-checks pass, 108 orchestrator tests pass. No GPU work, no pipeline run.
`test_api.py` was skipped because fastapi is not installed in the workpc venv.
Produced: `decisions/audit-phase1.md`, `caveats/audit-open.md`, `ROADMAP.md`, and this scaffold.
One existing test asserted the bug: `test_name_binding.test_conflict_flags_and_stays_unnamed` relied on
orphan flags leaking into every chapter, because it never created panel rows. It now creates them.
## 2026-08-11 Orchestrator half committed and deployed [no task]
Command: `pytest -q --ignore=test_api.py`, then `docker compose up -d --build orchestrator` on homesrv.
Outcome: finished. 108 tests pass. Commits `1c60710` (orchestrator half) and `94bd4d8` (minio pin) in
`/mnt/server/home/kami/docker-apps`. Orchestrator and minio both answer health on homesrv.
The rebuild recreated `minio` as a side effect and it crash-looped with `exec format error`: the
compose pin was the arm64 manifest digest of `minio/minio:latest` and homesrv is amd64. Repinned to the
amd64 digest. Nothing about Phase 1 caused this, but any compose action that recreates minio would have
hit it, so it was latent, not new.
Still unrun against a real chapter.
## 2026-08-11 S3 viewer and storage swap, tasks #116/#117 [#116 #117]
Command: docker compose on homesrv, `dig`, `openssl s_client`. No pipeline, no GPU.
Outcome: partial. Viewer works, storage swap staged and unfinished.
#117 needed no new software. `stowage` at `~/docker-apps/stowage` was already configured against the
manga MinIO and had been dead since 2026-07-19 with `exec /sbin/tini: exec format error`: its digest
pin was the arm64 manifest. Repinned to amd64 `sha256:91be7f13`, chowned `data/` to uid 65532 for the
new image, and it serves. MinIO had the identical bug, repinned to `sha256:a1a8bd4a`. A sweep of all
470 local images on homesrv found exactly those two arm64; nothing else in the homelab is affected.
#116 is staged, not done. `rustfs` runs alongside MinIO on `127.0.0.1:9010/9011`, pinned
`sha256:19b105cc`, data at `/mnt/hdd2/rustfs`. Buckets are empty: the `mc` mirror of
`audio layers manga panels raw video` (350M, all in `manga`) has NOT run. `/mnt/hdd2/minio/data` is
untouched and is the rollback. RustFS is `1.0.0-beta.12`, labeled `build-type=prerelease`. Cutover
would give rustfs 9000/9001 and repoint `MINIO_ENDPOINT=minio:9000` in the orchestrator plus
`stowage/config.yaml`; `transport.py:95` needs no change if rustfs takes `192.168.1.104:9000`.
Side quest, unrelated to the pipeline: the shared 41-domain cert stopped renewing. Root cause was DNS,
not nginx. Every `*.kvmx.ru` name pointed at a hard A record for `109.229.102.117` while the line had
moved to `109.229.127.149`; the Mercusys DDNS at `kvmx-home.mercusysddns.com` was correct the whole
time but nothing in the zone referenced it. Fixed with `CNAME * -> kvmx-home.mercusysddns.com` at
reg.ru. Certificate now issues.
Two measurement traps worth remembering. The ISP transparently intercepts ports 80 and 443 by
Host/SNI, so `curl` from workpc to ANY address returns kvmx.ru content and proves nothing about
external reachability; bare TCP connects also succeed against arbitrary addresses and then hang. Three
wrong root causes came out of trusting those probes before checking them.
Also patched `~/scripts/migrate-kvmx-https.sh:54` on homesrv. `need_stream_module` used
`sudo -n nginx -V` and `sudo -n nginx -T`; the NOPASSWD rule covers only `nginx -t`, so it reported
"stream module is not loaded" whenever it meant "could not ask for a password". Both checks now run
without sudo. `bash -n` passes and both conditions evaluate true.
## 2026-08-11 Per-artifact buckets, rustfs buckets, baseline chapter run [#116]
Command: `mc mb` on rustfs, `docker compose up -d --build orchestrator`, `pytest -q --ignore=test_api.py`,
`./start_workers.sh`, then `/job/create` + `/stage/clear` + `/job/resume` for chapter
`7c944dd4-e972-42c7-ba60-9f6939548e80` of "Teto X Egen" as job `778297bc-e7ce-439d-91b5-8a027060d17f`.
Outcome: partial. Storage split landed and is proven by the run. The run itself was still in `direct`
when the session ended.
Produced: `decisions/storage-layout.md`, `caveats/speaker-attribution.md`, 109 orchestrator tests pass.
Artifacts now split one bucket per class instead of everything under `manga`
(`decisions/storage-layout.md#bucket-per-artifact`). Both MinIO and rustfs hold all six buckets. The
run put 79 pages in `raw` and 116 panel crops in `panels`, so the split works end to end.
Two containers on homesrv had been dead for two weeks and blocked the work. `manga-fetch` was exited,
so `/job/create` failed with `httpx.ConnectError`; `manga-web` was exited, so `manga.kvmx.ru` had
nothing behind it on port 8083. Both started with `docker compose up -d`. Neither is related to the
storage change. Neither was caught by any check, because nothing watches these containers.
Stage timings, 116 panels: crop 85s, vision ~4min, identity ~1min, reconcile ~7min for 35 pairs,
dialogue ~8min. Faster than the 2026-07-17 run at 75 panels. The webtoon crop that 500'd in July
succeeded this time.
Quality cross-check against the panel images, the point of the run. Dialogue text extraction is
accurate. Character detection is accurate. Speaker attribution is not: three of three sampled
two-character panels attribute both speakers to the wrong person, always swapped
(`caveats/speaker-attribution.md#tail-is-not-geometry`). 24 of 81 speech lines resolve to a named
character, which is the Phase 1 headline metric at 30%, and the sample says that 30% is not
trustworthy. 26 of 113 detected people got an identity, and 25 of those 26 went to one character that
turns out to cover two different women.
The run then reached `scene` 116/116 and failed in `script` at 87/116, not on OOM: 28 beats were
rejected by the script verifier as `unsupported-proper-noun: ['Choi', 'Haeseon']`
(`caveats/speaker-attribution.md#multiword-name-verifier`). No two-word cast name can pass that check.
## 2026-08-11 Speaker provenance and the multi-word cast name
Command: `.venv/bin/python worker_vision.py`, `pytest -q --ignore=test_api.py` in the orchestrator.
Outcome: both pass, 110 orchestrator tests. Nothing deployed, no GPU work, no pipeline run.
Produced: `decisions/speaker-attribution.md`, two caveats rewritten.
`_annotate_speaker_methods` stopped stamping `tail` on a model guess. With two or more characters
present the guess is dropped to `unknown` at confidence 0.0. With one present it is kept as
`model_solo` at 0.7, the same claim the solo backstop already makes
(`decisions/speaker-attribution.md#no-fake-tail`). Grounded `som_face` and `solo_prior` rows are
untouched. Nothing outside `worker_vision.py` reads the literal `tail`, checked across both repos.
`verify_script` now tokenizes each cast name into `allowed`, so `Choi Haeseon` passes as two tokens
(`decisions/speaker-attribution.md#multiword-cast-names`). That is the 28 beats job `778297bc` lost.
The remaining `['Blur']` beat is a true positive that still halts the whole chapter, now recorded as
`caveats/speaker-attribution.md#one-word-halts-chapter`.
Neither fix is live. The orchestrator container is not rebuilt and the workers are not restarted.
## 2026-08-11 Rerun from dialogue: the honest speaker number is 9%
Command: `/job/cancel`, `/stage/clear dialogue`, `./start_workers.sh`, `/job/resume` on job
`778297bc-e7ce-439d-91b5-8a027060d17f`, twice. `docker compose up -d --build orchestrator` three times.
Outcome: `dialogue` 116/116. 112 orchestrator tests pass, `worker_vision.py` self-check passes.
The named-speaker share is 9%, 9 of 95 speech lines, down from a reported 30% that counted fake tails.
Multi-character panels contribute 0 of 40 lines by design. Single-character panels give 9 of 55. All 9
binds are `Choi Haeseon`, the row that covers two different women.
Five defects, four of them found by measuring the run rather than by reading code.
1. The fake `tail` label, fixed before the run (`decisions/speaker-attribution.md#no-fake-tail`).
2. The multi-word cast name in the script verifier
(`decisions/speaker-attribution.md#multiword-cast-names`).
3. `/stage/clear dialogue` deleted nothing and reported success. dialogue and direct write onto the
per-panel vision blob and had no `_STAGE_TABLES` entry, so `run_stage_dialogue` saw
`"dialogue" in vision` and would have skipped all 116 panels. The proof is the second clear:
116 dialogue blobs and 75 direct blobs stripped that the first had left. This is
`caveats/audit-open.md#dishonest-clearing` firing exactly where it was filed.
4. gemma answers the speaker field with whatever the prompt showed, most often the character
description, and every such answer became a free-form name that no registry entry matched. 28 of 51
sampled lines (`decisions/speaker-attribution.md#prompt-label-answers`). After the fix, 3 of 95.
5. All 7 `som_face` lines pointed at a mark whose face paired to no present character, so the
highest-trust provenance sat on a line with no speaker. Same defect class as the fake tail.
Identity is now the binding constraint, not attribution. 26 of 113 detected people carry an identity,
23%, and 25 of the 26 are the one over-merged row. Even perfect balloon binding caps this chapter near
23% named. The person who does hold an identity is stored as `Lim Seonho` while a separate row is named
`Seonho` with alias `Lim Seonho`, so either name matches two rows, raises `ambiguous-speaker` and binds
nothing.
Two more defects surfaced after `dialogue` finished. All 7 `som_face` lines pointed at a mark whose face
paired to no present character (`decisions/speaker-attribution.md#unpaired-mark`). Then `script` halted
at 112/116 because the narrator wrote `"...Hm?"` for the source line `"Uh... hum...?"`, and both verifier
rules fired on that two-letter interjection
(`decisions/speaker-attribution.md#interjection-false-positive`). After the fix, `script` passed 116/116,
the first time this chapter has cleared the verifier. `tts` then ran for the first time.
The run then completed end to end for the first time: `tts` 116/116, `layers` 116/116, `render` 116/116,
`assemble` 1/1, finished 2026-08-11T20:08:16Z. `s3://video/` holds 49 clips and a 50MiB `chapter.mp4`,
`s3://audio/` 49 objects at 32MiB. The per-artifact bucket split is now proven for every class except
layers (`decisions/storage-layout.md#bucket-per-artifact`).
Two honesty defects surfaced at the finish, both recorded rather than fixed. `layers` reported
`completed 116/116` with an empty bucket, and the completed job still carries
`error: "partial: 112/116 completed"` from the failure three resumes earlier.
## 2026-08-12 — the video got watched
No pipeline ran. The user watched `chapter.mp4` for the first time and read out 19 timestamped defects.
That found more than the previous four sessions of measuring, because the recorded metrics were all
measuring whether code ran rather than whether the result was right.
Two measurements came out of it. First, `chapter.mp4` is video 436.39s over audio 363.67s, so the
narration finishes 72.7s before the picture and the gap accumulates. The 49 clips are clean: every one is
25fps exactly, video and audio agree to 0.03s, and they sum to 363.6s. Assembly adds 72.7s of video and
no audio. Second, this manga holds 19 character rows of which 3 carry a name, and `Choi Haeseon` holds 25
of the chapter's 26 identity assignments. That is why the video calls the colleague Choi, never names the
MC, and flips gender.
The A/V bug was narrowed with a per-round probe over the 49 real clips. Round 0 of `_assemble_batched` is
correct, losing only the xfade overlap per group. Round 1 turns 359s of video into 100s while the audio
survives at 358.79s. The round-1 filtergraph is arithmetically correct, and re-running the same chain by
hand over only the 6 encoded intermediates gives a correct 348.24s with no warnings. Round 1 differs by
holding a 7th input: the leftover 49th clip, which `_assemble_batched` passes through un-encoded. That
passthrough is a third path beside `concat` and `xfade` and is the prime suspect.
`worker_render.py` gained an `FPS = 25` constant, fps normalization in the xfade branch to match the
concat branch, a `_stream_dur` helper, and a self-check that compares video against audio rather than
asserting the file is non-empty. The old check only asserted `getsize(out) > 0`, which is how a 20% sync
failure shipped. Pinning `-r FPS` on the output encodes was tried and reverted: it collapsed the chapter
to exactly 100.00s by dropping frames to force CFR, which the comment at the concat branch already
warned about. None of it is committed and none of it fixes the chapter yet.
The fps inconsistency between the two branches is real but not proven to be the shipped cause. The scene
graphs hold 356 `cut` against 6 `fade_black`, so the real run's final round most likely stayed on the
concat branch where no mixing happens.
### Panel 7, checked against the art
The same day, the user pulled up panel 7 and checked every detection by eye. It overturned the framing
this file carried an hour earlier, and it overturned two theories I proposed before being corrected.
Panel `7c944dd4-e972-42c7-ba60-9f6939548e80_p007`, a wide establishing shot of an office through a
window, crop 900x1650. Vision emitted 6 characters. Zero of the two identity bindings are correct and the
one character who matters is unbound. `person_5`, described as "yellow sweater", is Seonho in the
foreground and got no identity. `person_6` is the colleague, who has no name in the story, and was
assigned `Choi Haeseon` at 0.9. `person_2` is a background extra and was assigned `Lim Seonho` at 0.9.
`person_1` is a window frame with nobody in it. `person_3` and `person_4` are background extras.
Three defects stack, recorded as `caveats/speaker-attribution.md#bbox-wrong-space`,
`#no-anonymous-identity` and `#extras-as-cast`. The `bbox` values are consumed as absolute pixels, and on
this panel that puts all six boxes in the top third with two inside a speech balloon. Divided by 1000
four of the six fit tightly. Identity therefore embedded crops of balloon edges and window frames, which
is how a 0.9 confidence lands on the wrong person. Blank crops embed alike, a plausible mechanism for one
row absorbing 25 of 26 assignments.
Two claims I made and had to withdraw. First, that rescaling by 1000 makes the boxes correct: after
scaling, `person_1` still sits on an empty window frame and `person_6` clips its subject, and the
descriptions are unreliable anyway, since `person_6` reads "white shirt" for a green dress. Second, that
the constraint is 16 nameless rows needing names. The opposite is true. The pipeline mints names onto
people who have none, and at least one nameless row is a real recurring person who should stay nameless.
The "26 of 113 detected people carry an identity" figure that framed the roadmap counted mostly
background extras. It should not be quoted again.
## 2026-08-12, chapter assembly, root cause and fix
Reproduced the A/V collapse offline with 49 synthetic clips at `ASSEMBLE_BATCH=8` and six `fade_black`
boundaries. It came out worse than the shipped run: **two round-0 groups of 8 fresh clips collapsed on
their own**, so the single-item passthrough theory from yesterday is dead
(`decisions/chapter-assembly.md#passthrough-innocent`).
Bisected one collapsing group by truncating the chain stage by stage:
```
k=7 out= 52.52 correct
k=8 out= 52.52 the last xfade contributed nothing
[v6][n7]xfade=duration=0.050:offset=52.500 <- [v6] is 52.52s long, 0.02s of margin
```
`_xfade_chain` took its durations from `_audio_dur`, which is `format=duration`, which is
`max(video, audio)`. Each clip's audio outlasts its video by about a frame, so the offset accumulator
crept ahead of the picture. Once the creep passed the transition width, xfade emitted the transition and
threw away the second input and every clip after it, at `rc 0` with nothing on stderr.
Fix: offsets come from `min(_stream_dur(v), _stream_dur(a))`, every input is floored to a whole frame
count and `trim`/`atrim`ed on both streams, and `_check_assembled` now verifies each encode against the
predicted timeline instead of trusting the exit code
(`decisions/chapter-assembly.md#offsets-from-min-stream`, `#check-assembled`).
Verified on the 49 real clips of chapter `7c944dd4`, re-downloaded from MinIO:
```
before r1 n=7 XFADE in v=359.29 a=359.60 -> out v= 99.96 a=358.79
after r1 n=7 XFADE in v=359.61 a=359.62 -> out v=358.76 a=358.76
chapter v=358.76 a=358.76 gap=+0.00 (shipped: v=436.39 a=363.67 gap=+72.72)
```
`worker_render.py` `__main__` passes. Two checks were added there, because the existing 4-clip A/V assert
passed all the way through the broken build. One asserts the frame-exact `trim` on both streams, one
assembles three clips whose audio outlasts their video by 0.4s. Mutation-tested by putting `_audio_dur`
back: the new check fires with `video=1.80 audio=3.56 expected=3.56`.
Not done: `s3://video/.../chapter.mp4` is still the broken 436s file. Rebuilding it means clearing the
`assemble` stage and resuming, which is CPU-only and was not run.
## 2026-08-12, the chapter rebuilt, and the bbox space settled
**The rebuild came out byte-identical to the broken file.** Clearing `assemble` and resuming produced
video 436.392031s over audio 363.674667s and `nb_frames` 9902 again, which proved the xfade fix committed
earlier today never runs for this chapter. With all-`cut` transitions `assemble` takes the `else` branch,
a `concat` demuxer with `-c copy`.
Reproduced that path offline in seconds and got the shipped numbers exactly. The cause is mixed frame
rates: 14 of the 49 clips are `r_frame_rate=30/1` at `time_base=1/15360`, the other 35 are `25/1` at
`1/12800`. `-c copy` writes the output in the first input's timebase, so those 14 play `15360/12800 = 1.2`
too long with their audio untouched. `collage_cmd` hardcoded `-r 30`, which yesterday's `FPS` sweep
missed. `decisions/chapter-assembly.md#mixed-rate-stream-copy`.
Fixed `collage_cmd` to emit `-r FPS`, and made `assemble` probe `r_frame_rate` across the clips and route
mixed rates through the re-encoding tree. Rebuilt:
```
before v=436.392 a=363.675 nb_frames=9902 avg_frame_rate=22.69
after v=364.120 a=364.122 nb_frames=9101 r=25/1
```
The 14 clips in the bucket are still 30fps. Assembly normalizes them, so the chapter is correct without
re-rendering, but the fast stream-copy path stays disabled for this chapter until `render` re-runs.
**The `bbox` space is 0-1000, not pixels.** Pulled all 113 detections from `/review/identity` and
measured: 47 boxes have `x2` past the 900px panel width, none has `y2` past 1000 on panels 1257 to 2307px
tall, 21 clamp at exactly 1000 in x, and the whole range is `[0, 1000]`. `/vision` now converts to pixels
before returning, so identity crops, gated face pairing, the set-of-mark boxes and the review UI all read
pixels (`decisions/identity-bbox.md#bbox-is-normalized`).
Checked by eye the way the user did. Drew the converted boxes on panel 7: five of six land on their
subject, including `person_5`, who is Seonho in the foreground with headphones and carried no identity.
`person_1` still frames an empty window mullion, which is the extra-versus-cast caveat, not this one.
Not done: `vision` and `identity` have not re-run, so every box, embedding and `ref_image_uris` in the
registry is still from the wrong space. That rerun is GPU work and was not started.
## 2026-08-12, third session — the rerun, and the run that did not count
Asked: move on from the chapter rebuild. Scope chosen mid-session: reset the registry, rerun vision,
identity and reconcile only, and inspect before spending the ~80 minutes downstream.
**The registry had to be wiped first, and nothing could do it.** `clear_stage_data` maps `identity` to
`identity_assignments` only and spares `characters` on purpose (`db.py:790`), because the registry is
cross-run. So a rerun would have matched fresh crops against 19 stale grid-space embeddings and kept the
duplicate `Lim Seonho` / `Seonho` rows that raise `ambiguous-speaker`. Added `db.reset_registry` and
`POST /characters/reset` (`confirm=true` required), which also clears `identity_assignment_sources`
untouched by any stage clear, and a leftover `manual` row there makes `assign_identity` refuse the next
model assignment (`db.py:663`). Covered by `test_db.py:TestResetRegistry`; 115 orchestrator tests pass.
The orchestrator image bakes its source, so it needed `docker compose up -d --build orchestrator`.
**The first full cycle was void: the worker was serving pre-fix code.** vision + identity + reconcile
completed 116/116/20 and reproduced the defect exactly — 46 of 110 boxes past the 900px panel width,
coordinates clamping at 1000. The vision worker started 12:00:09, `worker_vision.py` changed 12:11:35,
and the commit carrying `_bbox_to_pixels` landed 12:16:22. Python had already bound the old module.
Nothing in the stage output said so (`decisions/identity-bbox.md#stale-worker-invalidates`). Added
`check_stale.sh`, which compares every worker's process start against its module mtime and exits
non-zero; mutation-tested by touching `worker_tts.py`.
**The rerun against restarted workers.** 8 minutes for all three stages.
```
before stale run after
x2 past panel width 47/113 46/110 0/110
coords exactly on 1000 21 27 1
max y2 1000 1000 2307
characters 19 8 8
coverage 26/113 = 23% 77/110 = 70% 77/110 = 70%
top character's share 96% Haeseon 47% 47%
```
`Choi Haeseon`, which had absorbed 25 of 26 assignments, no longer exists in the registry. On panel 7
Seonho is bound for the first time, and the unnamed colleague took an anonymous id instead of being
called `Choi Haeseon` at 0.9.
**Fixing the boxes made the extras problem worse.** With crops finally landing on their subjects, a
background extra bound to `Seonho` at confidence 1.00, putting an extra into the lead's reference set.
`/vision` now stamps `has_face` per character via `face_detect` + `_pair_faces_to_present`, and identity
skips `has_face is False` before it crops or embeds
(`decisions/identity-bbox.md#face-gates-enrollment`). Fails open on a missing or raising detector.
Self-checked in both workers. **Not yet proven on a GPU run** — that is the next command.
Deleted along the way and not recoverable: the rebuilt `chapter.mp4` and all 49 clips, by the cascade
from `/stage/clear vision`. The user chose not to keep a copy.
## 2026-08-12, fourth session — identity 2b, the resolver NONE branch
Asked: "how much will `has_face` help with character and identity problems?", then "fix 2b first".
Answered the first honestly: `has_face` reaches 2 of 6 detections on panel 7 and nothing else. It does not
touch naming or merging, and it cannot touch the chibi at 1:35, because an anime face detector detects a
chibi face. Also flagged its real cost: the model is face-only by design, so back-turned cast lose
enrollment along with the extras, and coverage is where that shows up first.
Measured before writing anything, read-only, no GPU:
```
registry: 8 characters, 1 named -> ['Seonho']
detections: 110 assignments: 77 = 70% coverage
spread: Seonho 36, character_565c88 24, character_759e23 9, character_f7a4fd 3,
character_25f682 3, character_d72710 1, character_823aba 1
```
That killed the assumed cause. Anonymous ids already recur, so the identity worker's own
pending-promote path gives stable anonymous identities. The defect was elsewhere.
`/vision/resolve` can answer "none of these" and always could: `worker_vision.py:1071` returns
`state="new"` for `choice: 0` and `state="unresolved"` for an out-of-range index. `service.py` read only
`character_id` and unassigned every crop of the tracklet for either. The stale `ponytail:` comment above
that block named the real blocker and was right: minting needs an `embedding_uri` the orchestrator cannot
compute, since siglip and gemma cannot both be resident.
Fixed by carrying the embedding, not by adding a GPU pass. `/identity/resolve` writes each crop's
embedding to the crop's key with a `.npy` suffix and returns `emb_uri`. `tracklets.resolve_outcome` holds
the three-way decision as a pure function. `service.py` mints via the existing `create_character` and
falls into the existing assign loop. `decisions/identity-bbox.md#none-mints-an-anonymous-character`.
Checks: `worker_identity self-check ok`, `tracklets self-check ok`, 115 passed on homesrv.
Deployed: image rebuilt, `resolve_outcome` verified inside the running container, `audit_registry.py`
re-copied after the recreate. Vision and identity restarted, `./check_stale.sh` exits 0.
New trap: `tmux respawn-window -k` does not re-run the window command. It leaves a bare shell and the
worker down. Both workers were dead for two minutes before `/health` caught it.
Not run: the GPU cycle. `has_face` and the NONE mint are both unproven on real panels and now land in the
same run.
## 2026-08-12, fourth session, continued — identity 2c, extras versus cast
Asked: "fix extra-vs-cast?"
Traced the three places that read `vision["characters"]` raw before changing anything. That corrected an
inference made earlier in the session. `build_scene` already drops an unassigned detection from
`characters` and `present` (`worker_scene.py:63`), so extras never reached the cast list at all.
The leak was their ACTIONS. `actions` was built from every detection, and that list is what the script
prompt renders and what the correctness verifier uses as evidence. So a background extra's "standing at the
window" arrived as a fact about the panel with no character attached, and the verifier confirmed it,
because the action really was in the blob. That is a second mechanism behind the invented-narration
complaints in item 3, independent of the model inventing anything.
Gated two consumers on `has_face is False`, matching the enrollment gate's semantics exactly:
`worker_scene`'s `actions`/`action`, and `service._beat`, which picks the director's "who" from the first
three detections and falls back to an action when a detection has no name.
Left `service._present_characters` ungated on purpose. It builds the dialogue stage's candidate speakers
and the set-of-mark boxes. An extra picked as speaker already resolves to unknown, not to a wrong name, so
the failure is contained. And the gate's cost lands hardest there, since a character drawn from behind has
no face box and gating would delete a real speaker from the only list that can attribute their line.
`decisions/identity-bbox.md#extras-gate-consumers`.
Checks: `worker_scene self-check ok` with three cases (cast action survives, faceless dropped, missing key
survives), 115 passed on homesrv. Deployed: image rebuilt, `_beat` verified inside the container, scene
worker restarted, `./check_stale.sh` clean. Committed `ca46617` and `8b27aec`.
Still not run on a GPU. Three changes now ride the same cycle: `has_face`, the NONE mint, and this.
## 2026-08-12, fourth session, continued — identity 2d, merge and split
Asked: "2d? merge and split?"
Checked the registry before planning anything, and 2d as written is stale. The registry reset earlier today
deleted both worked examples. There is no `Lim Seonho` to merge into, `character_afa7623b` does not exist,
and the current registry is 8 rows with one named character (`Seonho`). Nothing to merge or split until the
rerun mints a new set.
So the useful work was the safety net for that rerun, since `reconcile` runs inside it. The caveat's cost
line was the reason: one bad merge was unrecoverable without rebaselining the whole manga, and the cycle
about to run includes a merge pass over embeddings nobody has seen yet.
`merge_characters` no longer deletes the loser. It sets `merged_into = keeper`, so the row keeps its
embedding, description and gender, and it stamps every repointed assignment with
`method = merged_from:<loser_id>` in `identity_assignment_sources`. That reuses a free-form column on a
table that already existed rather than adding a merge log. `source` is left alone, so a reviewer's `manual`
assignment keeps its veto in `assign_identity` after being repointed.
Roster readers filter `merged_into IS NULL`, including the name-dedup in `create_character`, which would
otherwise fold new crops back into a character reconcile had retired. Lookup by id does not filter, because
an assignment or a narration reference may still point at a merged id.
Two existing assertions asserted the old destructive behaviour (`test_db.py:235`, `test_merge_refs.py:37`)
and were rewritten, not deleted: the invariant changed on purpose.
Checks: 116 passed on homesrv, up from 115. Additive `ALTER TABLE` through the existing `init_db` migration
block, verified on the live database (`merged_into` present, 0 rows merged). Committed `00096cc`.
Deliberately not built: the unmerge path and the split. No wrong merge has been observed since the crops
were fixed, so the consumer of these records waits for one. The forward case is partly covered by 2b, since
a resolver NONE now mints instead of folding a stranger into the nearest match.
## 2026-08-12, fifth session — the GPU cycle all four changes were waiting for
Asked: "go ahead" on the rerun.
First correction: the vision/identity/reconcile timestamps in `/job/status` are UTC and the git log is
local (UTC+4), so the run that looked like a rerun was the pre-change baseline. `_mark_has_face` landed at
19:07 local, the run finished 13:17 local. The handoff was right that nothing had run.
Reset the registry for `ef105a86` (8 characters, 77 assignments, 101 sources, 142 S3 objects), cleared
`vision` and everything downstream (116 vision results), restarted the identity worker for its in-process
known-character cache, then ran vision, identity and reconcile. 6 minutes wall: vision 3m48s, identity
1m25s, reconcile 48s. All three `completed`, 116/116/16.
### What the four changes did
| change | verdict |
| --- | --- |
| `has_face` stamp | works, present on all 110 detections, 67 true / 43 gated (39%) |
| extras gate | works, panel 7's two wrong bindings are gone |
| NONE mints | works, 16 characters minted where 8 existed |
| non-destructive merge | works, 7 rows carry `merged_into`, 9 assignments stamped `merged_from:` |
Coverage went 70% -> 61% (77 -> 67 assignments over the same 110 detections). That is the gate's price and
it is mostly correct, see below.
### Panel 7, the worked example, before and after
```
person_1 Seonho 1.00 -> -- none -- has_face=False
person_2 character_f7a4fd 0.00 -> character_519d2b 0.00 has_face=True
person_4 character_d72710 0.94 -> -- none -- WRONG binding removed
person_5 Seonho 1.00 -> -- none -- WRONG binding removed
```
The two wrong bindings the gate existed to kill are dead. It also took `person_1`, and that is correct:
ran `face_detect.detect_faces` on p006.png directly and it finds exactly one face on the whole panel, conf
0.599, inside `person_2`. Swept the threshold to 0.04 and nothing else appears above 0.056. Looked at the
crop: `person_1` is the lead drawn from behind at his desk, back of the head and headphones, no face in
frame. The detector is right and the gate is right. The cost is that a back-turned character cannot be
identified from that panel at all.
Also worth recording: `person_1`'s new box frames the lead correctly. The old note that it framed an empty
window mullion was written against the pre-fix boxes.
### The new defect: a roster hint named the wrong man
The registry holds three real people and one wrong name.
- `character_92129ac7` "Lim Seonho", 22 assignments. Verified against the art: the p010 introduction panel
captioned `LIM SEONHO (29)`, yellow plaid shirt and headphones, which is the roster's description for
"Seonho". This is the lead and the name is right.
- `character_556aef60`, 25 assignments, unnamed. Verified: the woman with short black hair and pearls, the
roster's second character, whom the roster itself calls "Unknown".
- `character_dbadfff7` "Seonho", 15 assignments. Verified: a different man, glasses, dark clothing, in
profile. He carries the lead's roster name.
p020 assigns `Seonho` to `person_1` and `Lim Seonho` to `person_3`, so the pipeline holds them as two
people in one panel. inference: the roster hint injected at `service.py:882` puts a name in front of
detection, and detection attached it to the wrong face before the caption panel could mint the real one.
This is not the alias-merge case and merging the two rows would be wrong. It needs name binding to require
evidence, the way `name_claims` already does for captions and address.
### Two smaller things the run exposed
- **Merge chains.** `character_e1ab7776 -> character_521c301f -> character_556aef60`. `merged_into` points
at a row that is itself merged, so a single-hop resolve lands on a retired character. Roster readers are
fine because they filter `merged_into IS NULL`. Anything that follows one hop is not.
- **Orphan `name_claims`.** All 5 rows point at `character_afa7623b`, which the reset deleted. The reset
clears characters and assignments but not claims.
Not run: dialogue and everything downstream. The job sits at `dialogue waiting`.
### Same session — the three fixes the run's evidence asked for
All three are orchestrator-side, so no worker changed and `check_stale.sh` is not in play.
- **Roster hints no longer seed detection.** Deleted the two lines at `service.py:882` that appended
`_roster_char_hints` to `known_characters`. `build_detect_prompt` drops any hint without a name, so a
nameless hint would have contributed nothing anyway. Names now reach detection only from registry rows,
which are embedding-backed and were themselves named from an in-panel caption or address. The roster
still feeds `roster_cast` in `run_stage_dialogue`, where names are matched against speech.
- **`merge_characters` resolves the keeper to its chain root** before merging, so `merged_into` stays one
hop deep. Bounded 64-step walk, the cap only exists so a cycle cannot hang reconcile.
- **`reset_registry` deletes `name_claims`** for the manga's panels and reports the count.
Checks: 117 passed on homesrv, up from 116. The new chain test was confirmed to fail with the walk
disabled, then the walk was restored and the suite re-run. Deployed by rebuilding the image; verified
inside the running container by parsing the deployed source, not by grepping it, because the first check
matched the word `_roster_char_hints` inside the comment that explains its removal.
Also copied `audit_registry.py` into the repo before the rebuild. It had only ever been `docker cp`'d, so
every rebuild dropped it. `Dockerfile` has `COPY . .`, so it is baked now and the trap is gone.
Open risk on the next run: without the roster hint the lead is unnamed until the p010 caption, so panels 1
to 9 may mint him as an anonymous character that reconcile then has to merge. Watch whether "Lim Seonho"
comes back as one row or two.
### Same session — run B, 17:38-17:44
Reset (16 characters, 67 assignments, 67 sources, **5 name_claims**, 156 S3 objects), cleared `vision`,
restarted the identity worker, ran the three stages. The `name_claims: 5` line is the orphan fix confirmed
on live data.
117 detections, 59 assignments, 50% coverage, 20 characters, 72 with `has_face` (45 gated).
The registry split is fixed. "Lim Seonho" is one row with 25 assignments, "Seonho" fell from 15 to 1. So
dropping the roster hint did not split the lead across the panels before his caption, which was the risk
recorded before the run.
Two findings, both now filed:
- detection still names from in-panel text and can attach the name to the wrong body (p011, p026), which
is `decisions/identity-bbox.md#roster-does-not-name`'s "not covered" paragraph
- one degenerate bbox, `caveats/audit-open.md#degenerate-bbox`
And one fix of my own that was incomplete: the merge chain came back as
`477c1894 -> a92d9df4 -> 4fb94c15` with the keeper walk deployed. The walk only sees the pair being merged
now. Retiring a row that is already somebody's keeper needed the second half, a repoint of the loser's
dependents. Both halves and both tests are in `decisions/identity-bbox.md#merge-chains-flatten`.
Checks: 118 passed, was 116. Each new merge test was run with its fix disabled and confirmed to fail
first. Deployed by rebuild and verified by parsing the container's source.
Coverage is now 70 -> 61 -> 50 across three runs and is the open question, recorded as
`caveats/audit-open.md#coverage-trend`. `identity_labels` already holds 145 rows of ground truth, so the
next move is to score with `eval_identity.py` rather than to keep reading the coverage number.
## 2026-08-12, sixth session — the ground truth is for the wrong manga
No GPU work. Two things, both cheap.
**`_bbox_to_pixels` orders its corners.** Each coordinate pair is sorted after clamping. The self-check
feeds it run B's real `[226, 417, 130, 551]` and asserts `[117, 688, 203, 909]`. Unsorted the call returns
`[203, 688, 117, 909]`, so the assert fails without the fix. `worker_vision.py` self-check passes.
The vision worker is now STALE and needs a restart before the next cycle.
**Item 1 of the plan cannot run as written.** All 145 rows of `identity_labels` belong to chapter
`8ca8249b`, a different manga with cast "Rico" and "Ikekin", spread over 81 panels. Chapter `7c944dd4`
has none, so `/review/identity?job_id=778297bc...` returns `labeled: 0, correct: 0, accuracy: null`.
Scoring `8ca8249b` anyway gives 7/138, with 113 rows labelled as a real person and left unassigned. That
chapter's identity run is stale: 44 assignments over 246 panels, predating every fix. The number measures
old code on the wrong chapter.
So the eval path is proven end to end and the ground truth is absent. A precision number for the current
registry needs a hand-labelling pass over `7c944dd4` through `POST /review/identity/label`, keyed by bbox
rather than by `local_id`, because vision reassigns `local_id` every run.
## 2026-08-12 18:07-18:13 UTC, third GPU cycle
Restarted vision (window 3) and identity (`/characters/reset` returned `restart_identity_worker: true`),
both by `send-keys C-c` then re-sending the launch line, never by `pgrep`. `check_stale.sh` exit 0 before
the run. Reset dropped 20 characters, 59 assignments, 69 sources, 168 S3 objects.
vision 116/116 in 3m59s, identity 116/116 in 1m24s, reconcile 18/18 in 50s.
| metric | 17:38 run | 18:07 run |
| --- | --- | --- |
| detections | 117 | 119 |
| assignments | 59 | 68 |
| coverage | 50% | 57% |
| degenerate boxes | 1 | **0** |
| `has_face` true / gated | 72 / 45 | 71 / 48 |
| assigned among face-bearing | 59/72 = 82% | **68/71 = 96%** |
| assigned among gated | -- | 0 |
| registry rows / merged | 20 / 7 | 18 / 9 |
| `merged_from` stamps | 9 | 22 |
| chains deeper than one hop | 1 | **0** |
The corner-ordering fix works: 0 degenerate boxes. The merge-chain fix holds under a heavier merge load,
9 merges and no chain.
**Coverage is now the gate and nothing else.** Every assignment landed on a face-bearing detection and none
on a gated one. Recall among face-bearing detections is 96%. So the 43% with no assignment is the 40% the
gate drops plus 3 detections, and the resolver is not losing cast. That closes the part of
`#coverage-trend` that suspected the resolver.
**The open question moved to precision.** The lead holds 36 of 68 assignments, 53%, against 25 of 59
before, and `audit_registry.py` flags it as absorbing. The merge count rose from 7 to 9 over the same
interval. Whether 36 is a protagonist in half the panels or an over-merge cannot be told apart without
labels, which is item 1.
Panel 7 got worse, not better: 5 of its 6 detections carry `has_face = False`, and `person_6`, the one that
does, went unassigned. The previous run bound its colleague. That is the cost of requiring a face, item 4.
## 2026-08-12, sixth session, later — the lead was over-merged, and cosine cannot fix it
The user checked the lead's crops in the review UI. `LIM SEONHO`'s 36 assignments cover at least six
different men, a chibi, and a cat. So 36 of 68 is a bug, not a protagonist.
Split by mechanism: 22 native resolves, 14 from reconcile merges. Both are broken, for different reasons.
**The tracklet linker chained across the chapter.** `window=8` bounds each pair, transitivity was
unbounded. The 22 native assignments came from 3 tracklets spanning 0, 22 and 30 panels, visible as two
confidence blocks of eleven rows at 0.95 and ten at 1.0. Fixed by capping the merged group's span
(`decisions/identity-bbox.md#tracklet-span-cap`).
**Both tokenizers linked on words that are not identity.** Whole chains hung on `short`; one pair linked on
the word `hair` itself. There is now one tokenizer with a `GENERIC` stopword set, shared with reconcile's
pre-filter (`decisions/identity-bbox.md#generic-tokens`).
Measured on the same 22 real crops, candidate overlap forced to pass: 3 tracklets at worst span 30 becomes
9 at worst span 8.
**Then the obvious next fix was ruled out by measurement.** Pulled all 22 crop embeddings from MinIO and
computed the pairwise cosine. The cat scores up to 0.82 against men, two different men score 0.93, and the
highest pair in the matrix is 0.96. No threshold exists, so crop-to-crop cosine is not a link signal, and
the `candidates` overlap condition is close to noise. Filed as
`caveats/audit-open.md#cosine-not-identity`. Inference: identity embeds the whole person box, so cosine
measures scene and style, not face.
Also seen: vision boxes cats as people and dresses them. `p081` and `p108` are cats, described
`short brown / yellow plaid shirt` and `short brown / white t-shirt`.
Checks: `tracklets.py` self-check ok, orchestrator 118 passed. Both new asserts confirmed to fail with
their fix disabled. Nothing deployed, nothing re-run on a GPU.
Artefact: `lead_tracklets.png`, the 22 panels with boxes, insets and tracklet grouping. Session scratchpad
only, not committed.
## 2026-08-12, 19:44-19:52 UTC — fourth GPU cycle: the cast is the gallery
Three fixes ran together for the first time: the tracklet span cap and the shared `GENERIC` tokenizer,
both written last session and never deployed, plus the new one
(`decisions/identity-bbox.md#cast-is-the-gallery`). Job `778297bc`, chapter `7c944dd4`, registry reset,
vision cleared, `vision -> identity -> reconcile`. Vision 116/116 in 3m55s, identity 116/116 in 2m44s,
reconcile 20/20 in 44s.
| metric | 18:07 run | 19:44 run |
| --- | --- | --- |
| detections | 119 | 119 |
| assignments | 68 | 60 |
| coverage | 57% | 50% |
| tracklets over crops | 12 / 64 | 33 / 72 |
| lead's assignments | 36 | 16 |
| top character's share | -- | 16/60 = 27% |
| characters after reconcile | 18 | 14 |
| minted / cleared | -- | 10 / 12 |
**Coverage went down and that is the fix working.** gemma cleared 12 crops it used to be forced to name
from a cosine top-k that did not contain the right person. 72 crops entered resolution against 64, because
a crop with an empty cosine shortlist is no longer dropped.
**Checked by eye, which is the only check that sees this.** Contact sheets of every assigned crop, per
character, confirmed by the user. The lead holds 16 crops. 14 are him and 2 are art inside a panel, the
photograph at `order 17` and the chibi at `order 20`. `character_2b1b12a1` holds 13, all of them her, and
she is a main character the registry never named. `character_f0d4e901` holds 9, of which 7 are her and 2
are `2b1b12a1` (`order 31`, `order 33`). Against the 18:07 run, where the lead's 36 covered six different
men, a chibi and a cat.
Every wrong crop on the lead is one defect: vision treats art inside a panel as the scene. It is not a
linker failure and no identity change will fix it.
**The cap fired twice and dropped the wrong rows.** `GALLERY_CAP = 10` truncated a 16-row gender-compatible
cast at `p097` and an 11-row one at `p109`. `cast_gallery` orders named first, so what it drops is exactly
the recently minted anonymous rows, which is the population a later tracklet most needs to match against.
Filed as `caveats/audit-open.md#gallery-cap-drops-the-unnamed`.
Checks: `worker_vision.py` self-check ok, `tracklets.py` self-check ok, orchestrator 118 passed,
`./check_stale.sh` exit 0 before the cycle. Vision and identity workers both restarted, the second because
`/characters/reset` asked for it.
Artefacts: `sheet_*.png`, one contact sheet per character. Session scratchpad only, not committed.
## 2026-08-13 — the dialogue stage names nobody, and why
Ran `dialogue` 116/116 in 5m57s on the fourth cycle's registry, to see whether the fixed identity lets the
existing `name_claims` path name the female lead. It does not, and the six claims it produced name three
separate defects.
```
p040 character_2b1b12a1 "Choi Haeseon" caption 1.00 -> NOT promoted
p010 character_b1dd5659 "Lim Seonho" caption 1.00 -> conflict flag
p047 character_b1dd5659 "Seonho" address 0.90 -> conflict flag
p011 character_f0d4e901 "Seonho" address 1.00 -> PROMOTED
p026 character_f0d4e901 "Seonho" address 1.00 -> PROMOTED
p110 character_028d4a49 "Haeseon" address 1.00 -> already named
```
All three are fixed in `db.add_name_claim` and recorded in `decisions/identity-naming.md`: alias grouping,
a confident caption as strong evidence, and one name per character. `test_name_binding.py` replays these
six claims, 121 tests pass, and each new assert was confirmed to fail with its fix disabled.
Also wired `merge_faceless_captions` into both crop endpoints. It had been written and never called; both
endpoints called the non-destructive `context_fragment_links` instead, and no decision recorded that
choice. It does not cover the head-in-one-shot, body-in-the-next split that prompted the question, because
a body fragment has no text and `_merge_plan` only folds a fragment that has text and no face.
Found while wiring it: `crop_webtoon` skips an upload when the key exists, which is right for a resume and
silently wrong after a slicing change. Documented at the line and in `NEXT.md`.
Wrote `ARCHITECTURE.md` from the user's design: region graph, occurrence/identity/name, speaker as a scored
graph edge with a typed union, narrative plane for art-in-art, and a persistent story state machine. Every
section carries what exists against it today. Nothing in it is built.
Nothing ran on a GPU after the dialogue stage.
## 2026-08-13 — the measurement spine, built and checked against the run it describes
`ARCHITECTURE.md` step 0. No GPU, no stage run. The 19:44 run of 2026-08-12 was still in
`/data/manga.db`, which is what made this possible before the fifth cycle destroys it.
`eval/chapter-truth.json` holds 38 labelled occurrences over the three characters the user walked crop by
crop. `audit_registry.py` now matches them to the run's assignments and prints purity per cluster and
fragmentation per person:
```
truth: 38 labelled occurrences, 38 matched an assignment at IoU >= 0.5, 0 unmatched
LIM SEONHO [b1dd5659] lead 14/16 = 0.88 = baseline (wrong: photo_of_lead 1, chibi_of_lead 1)
character_2b1b12a1 woman_a 13/13 = 1.00 = baseline
Seonho [f0d4e901] woman_b 7/9 = 0.78 = baseline (wrong: woman_a 2)
fragmentation lead 1, woman_a 2, woman_b 1 all = baseline
```
`NEXT.md` recorded that 2 of woman B's 9 crops were really woman A, and never said which. They are
`panel_order` 31 and 33, identified by pulling `p030` and `p032` from `s3://panels` and looking. Woman A
wears a cream top with black trim and a pearl necklace, and both crops are unmistakably her at zoom.
That closes the 7-and-2 arithmetic and is why her fragmentation is 2.
The design is in `decisions/measurement-spine.md`. Rows key on page-space geometry rather than
`panel_id`, purity is a share and fragmentation a count of ids rather than anything naming a
`character_id`. So the file survives both the re-crop and `/characters/reset` that the fifth cycle runs.
`audit_registry.py --selftest` covers the IoU match, the greedy tie-break and the purity maths without a
database.
Four things the audit itself was getting wrong, all pre-existing:
- it counted 20 characters where 14 are live and 6 are merge losers kept on purpose
- it keyed the assignment spread on `name`, so the two characters both named `Seonho` summed to one line
- its default worked example was `panel_index` 7, a balloon-only panel vision skips. `NEXT.md`'s
"panel 7" is `panel_order` 7, one lower, the office shot with 6 detections
- it said nothing about skipped panels. 41 of 116 are `skip=True`, so 119 detections come from 75
panels. Four skips were checked by eye and all four are correct
The skip count carries one new fact worth more than the audit fix. Those 41 panels hold 28 of the
chapter's 122 dialogue lines, and a skipped panel has no character at all. 23% of the dialogue has no
visible speaker to attribute to under any linker. That is the case for `offscreen` as a fourth
`speaker_ref` kind, now measured rather than assumed.
Step 0's dialogue half is not written. It needs 30 lines watched panel by panel, and `NEXT.md` holds no
per-line speaker truth to transcribe.
Then the question of whether the geometry key survives the caption merge. It does, and checking it found
a different defect one line up. `_merge_plan` only groups adjacent fragments, and `slice_webtoon` cuts at
gutter midpoints, so a vstacked panel's `[x, y0, w, sum(heights)]` is the exact page region.
But the 116 panels do not tile the strip. A 62px hole sits between `panel_order` 70 and 71, and 18px are
missing at the end, because `slice_webtoon` skipped any segment under `min_seg=64` instead of absorbing
it. The 62px held the chapter's credits line, `글 | H2J · 그림 | 이루리다`. Finding it meant reading the
PNG headers of all 79 tiles to locate the band at `p_050.png` row 338. Worthless content, and a hole in the
page-space y of everything after it. `slice_webtoon` now absorbs a short band into its neighbour
(`decisions/measurement-spine.md#slices-tile-the-strip`), and its self-check asserts the bboxes tile the
strip end to end.
Nothing ran on a GPU this session.