kami
8113bdfc8b
The rebuild after 1457556 came out byte-identical to the broken file,
which proved the xfade fix never runs for this chapter. An all-cut chapter
goes down the concat demuxer with -c copy, which writes the output in the
FIRST input's time_base and reinterprets every later packet in it. 14 of
49 clips are 30/1 at 1/15360 against 35 at 25/1 at 1/12800, so those 14
play 15360/12800 = 1.2 too long with their audio untouched. collage_cmd
hardcoded -r 30 and yesterday's FPS sweep missed it.
collage_cmd now emits -r FPS, and assemble probes r_frame_rate across the
clips and routes mixed rates through the re-encoding tree. Rebuilt
chapter.mp4 is 364.120s video against 364.122s audio at 25/1, from
436.392 over 363.675.
Also settle the bbox coordinate space, measured over all 113 detections:
47 boxes have x2 past the 900px panel width, none has y2 past 1000 on
panels up to 2307px tall, and the range is exactly [0, 1000]. It is
gemma's normalized grid, not pixels, whatever the prompt asks for.
/vision converts before returning, which fixes identity's crop, the gated
face pairing that was comparing pixel face boxes against grid boxes, the
set-of-mark boxes and the review UI at once. Checked by eye on panel 7:
five of six boxes now land on their subject, including the foreground
character who had no identity.
The registry still holds boxes and embeddings enrolled from the wrong
space. vision and identity have to re-run, which is GPU work and was not
started.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
131 lines
6.4 KiB
Markdown
131 lines
6.4 KiB
Markdown
# HANDOFF, 2026-08-12 (second session of the day)
|
|
|
|
Live state is in `NEXT.md`. This file is only what this session did. The previous handoff's content is
|
|
now in `JOURNAL.md` and in `decisions/`.
|
|
|
|
## Asked
|
|
|
|
Get up to speed from the previous handoff. Then: rebuild the chapter, and what about characters.
|
|
|
|
## Result
|
|
|
|
Two things landed. The A/V gap is fixed and the rebuilt `chapter.mp4` is in the bucket. The `bbox`
|
|
coordinate space is settled with measurements instead of comments, and converted at the boundary.
|
|
|
|
One correction to carry forward. **The first fix of the day named the wrong cause.** Commit `1457556`
|
|
claimed the xfade offset drift was the shipped 72.7s gap. The rebuild came out byte-identical to the
|
|
broken file, which disproved it. Both are real defects. Only the second one shipped.
|
|
|
|
## The chapter, rebuilt
|
|
|
|
`s3://video/ef105a86-4b7e-4ac4-b45c-b7d83b8f5b5e/7c944dd4-e972-42c7-ba60-9f6939548e80/chapter.mp4`
|
|
|
|
```
|
|
before v=436.392 a=363.675 nb_frames=9902 avg_frame_rate=22.69 gap +72.72
|
|
after v=364.120 a=364.122 nb_frames=9101 r_frame_rate=25/1 gap -0.002
|
|
```
|
|
|
|
Cause: `assemble` sends an all-`cut` chapter down a `concat` demuxer with `-c copy`. That path writes
|
|
the output in the **first** input's `time_base` and reinterprets every later packet in it. 14 of the 49 clips
|
|
are `30/1` at `1/15360`, the other 35 are `25/1` at `1/12800`, so those 14 play `15360/12800 = 1.2` too
|
|
long with their audio untouched. `collage_cmd` hardcoded `-r 30`.
|
|
`decisions/chapter-assembly.md#mixed-rate-stream-copy`.
|
|
|
|
Fixes: `collage_cmd` emits `-r FPS`. `assemble` probes `_fps_of` across the clips and routes mixed rates
|
|
through `_assemble_batched`, whose branches both normalize with `fps={FPS}`.
|
|
|
|
A second, latent defect on the transition path was fixed and committed separately. `_xfade_chain` took
|
|
offsets from `format=duration`, which is `max(video, audio)`. The accumulator crept past the end of its
|
|
input, and ffmpeg discarded whole clips at `rc 0` with nothing on stderr.
|
|
`decisions/chapter-assembly.md#offsets-from-min-stream`.
|
|
|
|
The single-item passthrough theory from the previous handoff is dead, recorded void at
|
|
`decisions/chapter-assembly.md#passthrough-innocent`. The one-path rewrite it recommended is not needed.
|
|
|
|
**Still open here.** The 14 clips in the bucket are still 30fps. Assembly normalizes them, so the chapter
|
|
is correct, but the fast stream-copy path stays off for this chapter until `render` re-runs. Nobody has
|
|
watched the rebuilt video yet. The 2:52 slide transition and the 28s static hold from 2:24 were both
|
|
supposed to be re-judged after the sync fix.
|
|
|
|
## Characters: the `bbox` space, settled
|
|
|
|
All 113 detections, straight from `/review/identity`:
|
|
|
|
| test | result |
|
|
| --- | --- |
|
|
| boxes with `x2` past the 900px panel width | **47 of 113** |
|
|
| boxes with `y2` past 1000, on panels 1257 to 2307px tall | **0 of 113** |
|
|
| boxes clamped at exactly 1000 | 21 in x, 5 in y |
|
|
| coordinate range over every box | `[0, 1000]` |
|
|
|
|
Gemma's native 0-1000 grid. Not pixels. `worker_vision.py` prompt text and the old
|
|
`worker_identity.py:91` comment both claimed pixels and both were wrong.
|
|
|
|
`/vision` now calls `_bbox_to_pixels(characters, w, h)` before returning. Four consumers are fixed at
|
|
once: `_crop_bbox` in identity, `_pair_faces_to_present`, the set-of-mark boxes, and the review UI's
|
|
client-side crop. The pairing one was comparing real pixel face boxes against 0-1000 character boxes,
|
|
which is the likely mechanism behind 7 `unknown` out of 7 `som_face` lines.
|
|
`decisions/identity-bbox.md#bbox-is-normalized`.
|
|
|
|
Checked by eye on panel 7, not just asserted. Five of six converted boxes land on their subject. That
|
|
includes `person_5`, who is Seonho in the foreground with headphones and carried no identity. `person_1`
|
|
still frames an empty window mullion, which is `caveats/speaker-attribution.md#extras-as-cast`.
|
|
|
|
Converted boxes for panel 7, for whoever redraws the overlay:
|
|
|
|
```
|
|
person_1 [226, 414, 286, 553] window mullion, nobody
|
|
person_2 [ 34, 558, 106, 749] background extra, was assigned Lim Seonho
|
|
person_3 [428, 384, 494, 533] background extra
|
|
person_4 [498, 389, 561, 549] background extra
|
|
person_5 [460, 657, 631, 939] Seonho, foreground. was assigned nothing
|
|
person_6 [646, 591, 767, 794] the colleague, no name in the story. was assigned Choi Haeseon at 0.9
|
|
```
|
|
|
|
**The registry is unchanged and still wrong.** Every stored box, embedding and `ref_image_uris` was
|
|
enrolled from the wrong space. `vision` and `identity` have to re-run before any of it means anything,
|
|
and that is GPU work nobody authorized. `caveats/speaker-attribution.md#bbox-wrong-space` is marked
|
|
resolved with the rerun pending.
|
|
|
|
## Checks
|
|
|
|
Every self-check runs from the repo root and passes:
|
|
|
|
```bash
|
|
.venv/bin/python worker_render.py # about 4 minutes, real ffmpeg
|
|
.venv/bin/python worker_vision.py
|
|
.venv/bin/python worker_identity.py
|
|
```
|
|
|
|
Three checks were added, because the existing ones passed all the way through both shipped defects:
|
|
|
|
- `_fps_of(collage clip) == "25/1"`, on a real collage encode. This is the one that would have caught the
|
|
mixed-rate bug at the source.
|
|
- three clips whose audio outlasts their video by 0.4s, assembled through the xfade branch. Mutation
|
|
tested by restoring `_audio_dur`: fires with `video=1.80 audio=3.56 expected=3.56`.
|
|
- `_bbox_to_pixels` against panel 7's real `person_5` box, asserting the result covers the lower half of a
|
|
1650px panel, which a raw grid value cannot.
|
|
|
|
`_check_assembled` now runs after every encode on both paths, because ffmpeg returns 0 while dropping
|
|
whole inputs.
|
|
|
|
## Next command
|
|
|
|
Watch the rebuilt chapter before anything else. That is what found every real defect so far.
|
|
|
|
```bash
|
|
/usr/bin/ssh kami@192.168.1.104 'mc cat homesrv/video/ef105a86-4b7e-4ac4-b45c-b7d83b8f5b5e/7c944dd4-e972-42c7-ba60-9f6939548e80/chapter.mp4' > chapter.mp4
|
|
```
|
|
|
|
Then, with a go-ahead, the vision and identity rerun in `NEXT.md` item 1.
|
|
|
|
## Traps confirmed again this session
|
|
|
|
- `tmux` session `manga-workers` was gone and both systemd units were inactive. `./start_workers.sh`
|
|
starts 9 windows. The render worker is window 9 and must be restarted by hand to pick up an edit.
|
|
- Plain `ssh` is the kitty ssh kitten and refuses non-interactive stdin. Use `/usr/bin/ssh`.
|
|
- `cd $dir && .venv/bin/python` fails, because the venv path is relative to the repo root. Use the
|
|
absolute interpreter path when the working directory is elsewhere.
|
|
- The whole assembly investigation ran offline on 49 downloaded clips with no GPU and no orchestrator.
|
|
Re-download with the command in `NEXT.md`.
|