# HANDOFF, 2026-08-12 (second session of the day)

Live state is in `NEXT.md`. This file is only what this session did. The previous handoff's content is
now in `JOURNAL.md` and in `decisions/`.

## Asked

Get up to speed from the previous handoff. Then: rebuild the chapter, and what about characters.

## Result

Two things landed. The A/V gap is fixed and the rebuilt `chapter.mp4` is in the bucket. The `bbox`
coordinate space is settled with measurements instead of comments, and converted at the boundary.

One correction to carry forward. **The first fix of the day named the wrong cause.** Commit `1457556`
claimed the xfade offset drift was the shipped 72.7s gap. The rebuild came out byte-identical to the
broken file, which disproved it. Both are real defects. Only the second one shipped.

## The chapter, rebuilt

`s3://video/ef105a86-4b7e-4ac4-b45c-b7d83b8f5b5e/7c944dd4-e972-42c7-ba60-9f6939548e80/chapter.mp4`

```
before  v=436.392  a=363.675  nb_frames=9902  avg_frame_rate=22.69   gap +72.72
after   v=364.120  a=364.122  nb_frames=9101  r_frame_rate=25/1      gap  -0.002
```

Cause: `assemble` sends an all-`cut` chapter down a `concat` demuxer with `-c copy`. That path writes
the output in the **first** input's `time_base` and reinterprets every later packet in it. 14 of the 49 clips
are `30/1` at `1/15360`, the other 35 are `25/1` at `1/12800`, so those 14 play `15360/12800 = 1.2` too
long with their audio untouched. `collage_cmd` hardcoded `-r 30`.
`decisions/chapter-assembly.md#mixed-rate-stream-copy`.

Fixes: `collage_cmd` emits `-r FPS`. `assemble` probes `_fps_of` across the clips and routes mixed rates
through `_assemble_batched`, whose branches both normalize with `fps={FPS}`.

A second, latent defect on the transition path was fixed and committed separately. `_xfade_chain` took
offsets from `format=duration`, which is `max(video, audio)`. The accumulator crept past the end of its
input, and ffmpeg discarded whole clips at `rc 0` with nothing on stderr.
`decisions/chapter-assembly.md#offsets-from-min-stream`.

The single-item passthrough theory from the previous handoff is dead, recorded void at
`decisions/chapter-assembly.md#passthrough-innocent`. The one-path rewrite it recommended is not needed.

**Still open here.** The 14 clips in the bucket are still 30fps. Assembly normalizes them, so the chapter
is correct, but the fast stream-copy path stays off for this chapter until `render` re-runs. Nobody has
watched the rebuilt video yet. The 2:52 slide transition and the 28s static hold from 2:24 were both
supposed to be re-judged after the sync fix.

## Characters: the `bbox` space, settled

All 113 detections, straight from `/review/identity`:

| test | result |
| --- | --- |
| boxes with `x2` past the 900px panel width | **47 of 113** |
| boxes with `y2` past 1000, on panels 1257 to 2307px tall | **0 of 113** |
| boxes clamped at exactly 1000 | 21 in x, 5 in y |
| coordinate range over every box | `[0, 1000]` |

Gemma's native 0-1000 grid. Not pixels. `worker_vision.py` prompt text and the old
`worker_identity.py:91` comment both claimed pixels and both were wrong.

`/vision` now calls `_bbox_to_pixels(characters, w, h)` before returning. Four consumers are fixed at
once: `_crop_bbox` in identity, `_pair_faces_to_present`, the set-of-mark boxes, and the review UI's
client-side crop. The pairing one was comparing real pixel face boxes against 0-1000 character boxes,
which is the likely mechanism behind 7 `unknown` out of 7 `som_face` lines.
`decisions/identity-bbox.md#bbox-is-normalized`.

Checked by eye on panel 7, not just asserted. Five of six converted boxes land on their subject. That
includes `person_5`, who is Seonho in the foreground with headphones and carried no identity. `person_1`
still frames an empty window mullion, which is `caveats/speaker-attribution.md#extras-as-cast`.

Converted boxes for panel 7, for whoever redraws the overlay:

```
person_1 [226, 414, 286, 553]   window mullion, nobody
person_2 [ 34, 558, 106, 749]   background extra, was assigned Lim Seonho
person_3 [428, 384, 494, 533]   background extra
person_4 [498, 389, 561, 549]   background extra
person_5 [460, 657, 631, 939]   Seonho, foreground. was assigned nothing
person_6 [646, 591, 767, 794]   the colleague, no name in the story. was assigned Choi Haeseon at 0.9
```

**The registry is unchanged and still wrong.** Every stored box, embedding and `ref_image_uris` was
enrolled from the wrong space. `vision` and `identity` have to re-run before any of it means anything,
and that is GPU work nobody authorized. `caveats/speaker-attribution.md#bbox-wrong-space` is marked
resolved with the rerun pending.

## Checks

Every self-check runs from the repo root and passes:

```bash
.venv/bin/python worker_render.py     # about 4 minutes, real ffmpeg
.venv/bin/python worker_vision.py
.venv/bin/python worker_identity.py
```

Three checks were added, because the existing ones passed all the way through both shipped defects:

- `_fps_of(collage clip) == "25/1"`, on a real collage encode. This is the one that would have caught the
  mixed-rate bug at the source.
- three clips whose audio outlasts their video by 0.4s, assembled through the xfade branch. Mutation
  tested by restoring `_audio_dur`: fires with `video=1.80 audio=3.56 expected=3.56`.
- `_bbox_to_pixels` against panel 7's real `person_5` box, asserting the result covers the lower half of a
  1650px panel, which a raw grid value cannot.

`_check_assembled` now runs after every encode on both paths, because ffmpeg returns 0 while dropping
whole inputs.

## Next command

Watch the rebuilt chapter before anything else. That is what found every real defect so far.

```bash
/usr/bin/ssh kami@192.168.1.104 'mc cat homesrv/video/ef105a86-4b7e-4ac4-b45c-b7d83b8f5b5e/7c944dd4-e972-42c7-ba60-9f6939548e80/chapter.mp4' > chapter.mp4
```

Then, with a go-ahead, the vision and identity rerun in `NEXT.md` item 1.

## Traps confirmed again this session

- `tmux` session `manga-workers` was gone and both systemd units were inactive. `./start_workers.sh`
  starts 9 windows. The render worker is window 9 and must be restarted by hand to pick up an edit.
- Plain `ssh` is the kitty ssh kitten and refuses non-interactive stdin. Use `/usr/bin/ssh`.
- `cd $dir && .venv/bin/python` fails, because the venv path is relative to the repo root. Use the
  absolute interpreter path when the working directory is elsewhere.
- The whole assembly investigation ran offline on 49 downloaded clips with no GPU and no orchestrator.
  Re-download with the command in `NEXT.md`.