Fix the 72.7s A/V gap: xfade offsets ran off the end of their input
_xfade_chain positioned every transition using _audio_dur, which probes format=duration, which is max(video, audio). A clip's audio outlasts its video by about a frame, so the offset accumulator crept ahead of the real picture timeline. Once the creep exceeded the transition width, xfade emitted the transition and silently discarded the second input and every clip downstream, exiting 0 with nothing on stderr. That is the whole of the shipped chapter's 436.39s of video over 363.67s of audio. Offsets now come from min(video, audio). Every input is floored to a whole frame count and trimmed on both streams, so the accumulator tracks the real timeline instead of estimating it. _check_assembled verifies each encode against the predicted length and against its own audio, because both assembly branches drop stream time without failing. Verified over the 49 real clips of chapter 7c944dd4: the round that turned 359s of video into 100s now loses 0.85s, and the chapter comes out 358.76s video against 358.76s audio. The single-item passthrough was not the cause. Two round-0 groups of 8 fresh clips collapse without one, recorded void in decisions/. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -36,3 +36,6 @@ still live belongs in `caveats/`.
|
||||
| [An interjection is not a name and not a misquote](speaker-attribution.md#interjection-false-positive) | closed |
|
||||
| [Cast names enter the verifier tokenized](speaker-attribution.md#multiword-cast-names) | closed |
|
||||
| [Clearing a stage strips the vision blob it wrote](storage-layout.md#clear-vision-blob) | closed |
|
||||
| [xfade offsets come from `min(video, audio)`, never `format=duration`](chapter-assembly.md#offsets-from-min-stream) | closed |
|
||||
| [Assembly verifies its own output instead of trusting ffmpeg's exit code](chapter-assembly.md#check-assembled) | closed |
|
||||
| [The single-item passthrough is not the assembly bug](chapter-assembly.md#passthrough-innocent) | void |
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
# chapter-assembly
|
||||
|
||||
Settled questions about `_assemble_batched` / `_assemble_once` / `_xfade_chain` in `worker_render.py`.
|
||||
|
||||
## xfade offsets are computed from `min(video, audio)`, never `format=duration` {#offsets-from-min-stream}
|
||||
|
||||
**Closed, 2026-08-12.**
|
||||
|
||||
`_xfade_chain` accumulates `cum += dur[i] - td` and hands each boundary `offset=cum-td`. That offset is
|
||||
an assertion about where input `i-1` still has frames. It fed on `_audio_dur`, which probes
|
||||
`format=duration`, which is `max(video, audio)`. A rendered clip's audio outlasts its video by about a
|
||||
frame. So every boundary pushed the accumulator further ahead of the picture.
|
||||
|
||||
Once the accumulated overshoot exceeds the transition width, the xfade window starts after the last frame
|
||||
of input `i-1`. ffmpeg emits the transition and then **silently discards input `i` and every clip
|
||||
downstream of it**. `rc 0`, no warning on stderr, output file present and playable. Measured on a group of
|
||||
8 real clips, chain truncated at each stage:
|
||||
|
||||
```
|
||||
k=7 out= 52.52 correct
|
||||
k=8 out= 52.52 the last xfade contributed nothing
|
||||
[v6][n7]xfade=duration=0.050:offset=52.500 <- [v6] is only 52.52s long, so 0.02s of margin
|
||||
```
|
||||
|
||||
This is the whole cause of the shipped chapter being video 436.39s over audio 363.67s. It reproduces with
|
||||
synthetic clips in about four minutes and needs no GPU.
|
||||
|
||||
Two changes hold it closed:
|
||||
|
||||
* `_assemble_once` probes `min(_stream_dur(p, "v"), _stream_dur(p, "a"))`. The minimum, because either
|
||||
stream running long breaks a different half of the graph.
|
||||
* `_xfade_chain` floors every input to a whole frame count. It applies `trim` and `atrim` to both streams,
|
||||
so the accumulator tracks the real timeline instead of estimating it. Transition widths are quantized to
|
||||
frames for the same reason.
|
||||
|
||||
Verified over the 49 real clips of chapter `7c944dd4`. The round that previously turned 359s of video into
|
||||
100s now loses 0.85s. The chapter comes out video 358.76s against audio 358.76s, agreeing to the frame.
|
||||
|
||||
Forbidden from here: `_audio_dur` in anything that positions a filter. It is fine for "how long is this
|
||||
clip roughly", nothing else.
|
||||
|
||||
## Assembly verifies its own output instead of trusting ffmpeg's exit code {#check-assembled}
|
||||
|
||||
**Closed, 2026-08-12.**
|
||||
|
||||
Both assembly branches drop stream time without failing. xfade discards inputs as above. The concat
|
||||
demuxer with `-vsync cfr` drops video frames to force a constant rate. Neither is an error to ffmpeg.
|
||||
|
||||
So `_assemble_once` calls `_check_assembled(out, expect)` after every encode. It compares the output's
|
||||
video stream against the predicted timeline and against its own audio stream. It raises when either is off
|
||||
by more than `ASSEMBLE_TOL_S`, which is 0.5s. That tolerance covers frame boundaries and aac padding. A
|
||||
dropped input is off by whole seconds.
|
||||
|
||||
Without this the failure stays invisible until somebody watches the video. That is how a 50MiB chapter
|
||||
with 72.7s of silent picture reached the bucket while every stage counter read success.
|
||||
|
||||
## The single-item passthrough is not the bug {#passthrough-innocent}
|
||||
|
||||
**Void, 2026-08-12.** Cited in `HANDOFF.md` for 2026-08-11 as the prime suspect and must not be cited
|
||||
again.
|
||||
|
||||
With 49 clips and `ASSEMBLE_BATCH=8`, round 0 makes six groups of 8 plus a leftover group of 1, which
|
||||
`_assemble_batched` carries forward un-encoded. The theory was that mixing that raw clip with six encoded
|
||||
intermediates broke round 1. The instrumented run disproves it: **two round-0 groups of 8 fresh clips
|
||||
collapse on their own**, before any passthrough exists.
|
||||
|
||||
```
|
||||
n=8 XFADE in v= 63.52 a= 63.60 -> out v= 52.52 a= 62.77 lost_v= +11.00
|
||||
n=8 XFADE in v= 58.12 a= 58.20 -> out v= 12.04 a= 56.83 lost_v= +46.08
|
||||
```
|
||||
|
||||
Round 1's 259s loss was the cascade. Its inputs already held 309s of video against 364s of audio, and
|
||||
`durs` read the audio.
|
||||
|
||||
Consequence for the plan: collapsing `concat`, `xfade` and the passthrough into one path was the
|
||||
recommended fix in `NEXT.md` and is **not needed**. Three paths are fine once each one positions filters
|
||||
on a real timeline. The tree keeps the bounded memory it was built for.
|
||||
Reference in New Issue
Block a user