Fix the real A/V gap: a stream copy across mixed frame rates
The rebuild after 1457556 came out byte-identical to the broken file,
which proved the xfade fix never runs for this chapter. An all-cut chapter
goes down the concat demuxer with -c copy, which writes the output in the
FIRST input's time_base and reinterprets every later packet in it. 14 of
49 clips are 30/1 at 1/15360 against 35 at 25/1 at 1/12800, so those 14
play 15360/12800 = 1.2 too long with their audio untouched. collage_cmd
hardcoded -r 30 and yesterday's FPS sweep missed it.
collage_cmd now emits -r FPS, and assemble probes r_frame_rate across the
clips and routes mixed rates through the re-encoding tree. Rebuilt
chapter.mp4 is 364.120s video against 364.122s audio at 25/1, from
436.392 over 363.675.
Also settle the bbox coordinate space, measured over all 113 detections:
47 boxes have x2 past the 900px panel width, none has y2 past 1000 on
panels up to 2307px tall, and the range is exactly [0, 1000]. It is
gemma's normalized grid, not pixels, whatever the prompt asks for.
/vision converts before returning, which fixes identity's crop, the gated
face pairing that was comparing pixel face boxes against grid boxes, the
set-of-mark boxes and the review UI at once. Checked by eye on panel 7:
five of six boxes now land on their subject, including the foreground
character who had no identity.
The registry still holds boxes and embeddings enrolled from the wrong
space. vision and identity have to re-run, which is GPU work and was not
started.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -2,10 +2,40 @@
|
||||
|
||||
Settled questions about `_assemble_batched` / `_assemble_once` / `_xfade_chain` in `worker_render.py`.
|
||||
|
||||
## xfade offsets are computed from `min(video, audio)`, never `format=duration` {#offsets-from-min-stream}
|
||||
## The shipped 72.7s gap was a stream copy across mixed frame rates {#mixed-rate-stream-copy}
|
||||
|
||||
**Closed, 2026-08-12.**
|
||||
|
||||
`assemble` routes an all-`cut` chapter to a `concat` demuxer with `-c copy`. That path writes the output
|
||||
with the **first** input's `time_base` and reinterprets every later packet in it.
|
||||
|
||||
14 of this chapter's 49 clips came off `collage_cmd`, which hardcoded `-r 30`. They carry
|
||||
`r_frame_rate=30/1` and `time_base=1/15360`. The other 35 are `25/1` at `1/12800`. Copied into the first
|
||||
clip's timebase, those 14 play `15360/12800 = 1.2` times too long while their audio is untouched. That is
|
||||
the 1.2001 ratio, and the whole of video 436.39s over audio 363.67s.
|
||||
|
||||
Reproduced offline by running the same `-c copy` concat over the 49 real clips. Duration 436.392031 and
|
||||
`nb_frames` 9902, identical to the shipped file. Seconds to run, no GPU.
|
||||
|
||||
Two changes hold it closed:
|
||||
|
||||
* `collage_cmd` emits `-r FPS` like every other clip path, and the `__main__` self-check asserts
|
||||
`_fps_of(clip) == "25/1"` on a real collage encode.
|
||||
* `assemble` probes `r_frame_rate` across the clips and sends mixed rates through `_assemble_batched`,
|
||||
whose branches both normalize with `fps={FPS}`. Only a single shared rate keeps the stream copy.
|
||||
|
||||
Verified end to end. The rebuilt `chapter.mp4` is video 364.120s against audio 364.122s at `25/1`.
|
||||
|
||||
**An earlier version of this file, and commit `1457556`, blamed `#offsets-from-min-stream` below for the
|
||||
shipped gap. That was wrong.** The rebuild came out byte-identical to the broken file, which proved the
|
||||
xfade tree never ran for this chapter. The entry below is a real defect and stays closed on its own
|
||||
evidence. It was not this one.
|
||||
|
||||
## xfade offsets are computed from `min(video, audio)`, never `format=duration` {#offsets-from-min-stream}
|
||||
|
||||
**Closed, 2026-08-12.** A real latent defect on the transition path. Not the cause of the shipped gap,
|
||||
see `#mixed-rate-stream-copy` above.
|
||||
|
||||
`_xfade_chain` accumulates `cum += dur[i] - td` and hands each boundary `offset=cum-td`. That offset is
|
||||
an assertion about where input `i-1` still has frames. It fed on `_audio_dur`, which probes
|
||||
`format=duration`, which is `max(video, audio)`. A rendered clip's audio outlasts its video by about a
|
||||
|
||||
Reference in New Issue
Block a user