manga-recap-pipeline/NEXT.md

# NEXT

Updated 2026-08-12. What this session did is in `HANDOFF.md`.

## State

The chapter runs end to end. The A/V sync defect is fixed and `chapter.mp4` is rebuilt: video 364.120s
against audio 364.122s at `25/1`. The identity defects are still in the output.

Job `778297bc-e7ce-439d-91b5-8a027060d17f`, chapter `7c944dd4-e972-42c7-ba60-9f6939548e80`, 116 panels,
`status=completed`, finished 2026-08-11T20:08:16Z. `s3://video/` holds 49 clips and a 50MiB
`chapter.mp4`. The user watched it and read out 19 defects. They are grouped by cause in `HANDOFF.md`.

One number sets the agenda:

- Panel 7 checked against the art has **zero correct identity bindings** out of two, and the one
  character who matters is unbound. `HANDOFF.md#panel-7-walked-against-the-art` has the table. The
  coordinate cause is fixed. The registry built on it is not.

## Next

1. **Re-run vision and identity.** The `bbox` space is settled and converted at `/vision`
   (`decisions/identity-bbox.md#bbox-is-normalized`). Every stored box, embedding and `ref_image_uris` in
   the registry came from the wrong space. The fix changes nothing until those stages run again.
   This is GPU work and needs the user's go-ahead. Clear `vision` and everything downstream of it, or
   accept that the boxes in the database stay normalized while new ones are pixels.

   Watch two things on the rerun. Whether `som_face` still returns `unknown` on every face, since gated
   pairing was comparing pixel face boxes against 0-1000 character boxes. And whether `Choi Haeseon` still
   absorbs every unnamed woman, which is item (b) below and independent of the crops.

   Smaller follow-on: nine `_audio_dur` calls in `worker_render.py` measure finished clips with
   `format=duration`. So the durations reported to the orchestrator are blind to per-clip drift.
   They position no filter, so invariant 9 does not cover them. Worth converting to `_stream_dur`.
2. **Fix identity, in this order.** Panel 7 is the worked example and
   `HANDOFF.md#panel-7-walked-against-the-art` carries the evidence. Do not start at the registry.

   a. ~~Settle the `bbox` coordinate space.~~ **Done 2026-08-12**, proven over all 113 detections and
      checked by eye on panel 7, where five of six converted boxes land on their subject
      (`decisions/identity-bbox.md#bbox-is-normalized`). `person_1` still frames an empty window mullion,
      which is (c).
   b. ~~Let identity abstain and stay abstained.~~ **Done 2026-08-12, not yet run on a GPU**
      (`decisions/identity-bbox.md#none-mints-an-anonymous-character`). The resolver could always answer
      "none of these". The orchestrator was discarding the answer: it read only `character_id`, so a
      deliberate NONE and a hallucinated index both unassigned every crop of the tracklet. A NONE now
      mints an anonymous character from the crop, using the embedding `/identity/resolve` ships beside
      it as `emb_uri`. Deployed: image rebuilt, `resolve_outcome` verified inside the container.
   c. ~~Separate extra from cast.~~ **Done 2026-08-12, not yet run on a GPU**
      (`decisions/identity-bbox.md#face-gates-enrollment`,
      `decisions/identity-bbox.md#extras-gate-consumers`). `has_face` stops a faceless detection
      enrolling, and two more consumers now skip it: `worker_scene`'s `actions`, which is the script
      prompt's content and the verifier's evidence, and `service._beat`, the director's "who".
      `_present_characters` stays ungated on purpose, reasoned out in the decision.
      The remaining gap is that vision still emits extras into the blob, which is deliberate so the
      audit can see what was gated.
   d. **The worked examples are gone.** The registry reset deleted `Lim Seonho` and
      `character_afa7623b`. The current registry is 8 rows, one named (`Seonho`), so there is nothing to
      merge or split until the rerun mints a new set.

      What was done instead is the safety net for that rerun, since `reconcile` runs inside it. A merge no
      longer deletes the losing row: it sets `merged_into`, and stamps every repointed assignment with
      `method = merged_from:<loser_id>`. A wrong merge now costs a hand-written SQL walk, not a full
      rebaseline (`caveats/audit-open.md#destructive-reconcile`).

      Deliberately not built: the unmerge path and the split. No wrong merge has been observed since the
      crops were fixed, so the consumer of those records waits for one. The forward case is partly covered
      by 2b, because a resolver NONE now mints rather than folding a stranger into the nearest match.
      Splitting a character that is ALREADY over-merged still needs a re-embed pass over its detections.

   **Cast profiles already exist. Do not rebuild them.** The user asked whether the main cast could get a
   profile built from reference frames and reused. `characters` already carries `ref_image_uris` and
   `embedding_uri`, and all 53 rows have both populated. The mechanism is not missing, it is enrolled
   from the wrong crops, so today it stores references to balloon edges and window frames. Step (a) is
   what makes it work. Three things are genuinely absent and are the smaller follow-on:

   - no quality gate on enrollment, so nothing checks that a reference crop holds a face at all
   - nothing re-enrolls a reference set once it is written, so the wrong crops persist

   **The visual "is this them?" check is already built. Do not write it again.** `/vision/resolve` at
   `worker_vision.py:963` sends the query crop plus up to 3 labelled reference images per candidate.
   `build_resolve_prompt` tells the model to judge face shape first, to treat hair and outfit as
   secondary, that two people sharing a hair colour are not the same, and to answer `0` for NONE when
   unsure. `choice: 0` becomes a new character, an out-of-range index becomes `unresolved`, and
   `ref_image_uris` is republished as `reference_image_uris` at `worker_identity.py:152` and `:161`. The
   mechanism, the prompt and the abstain path are all correct. They are fed crops of the wrong region,
   which is step (a).
   - no human gate to name, merge or split the clusters. The user wants this as a minor adjustment on
     top, not as the mechanism. The `gates` table and the review gates from [#136] are the place to hang
     it

   The chibi at 1:35 will survive all of this. He genuinely is brown hair plus a yellow shirt, so a
   profile match is correct on appearance and wrong on reality. That needs item 4 below, plus requiring
   a real face before a crop can enroll.
3. **Stop the narration inventing facts.** 0:43, 2:03, 2:05 and 2:15 assert things no panel shows. The
   correctness verifier passed 116/116 because it checks quotes and names, never invented claims.
4. **Teach vision that art inside a panel is not the scene.** A chibi on a monitor became "a man holding
   a drink" at 1:35. A colleague pointing into the distance became "pointing towards the screen" at
   1:59.
5. **`layers` writes nothing** and reports `completed 116/116`, so no clip has parallax and a still
   holds for 28s from 2:24 (`caveats/audit-open.md#layers-writes-nothing`).
6. **Clear the stale job error.** The completed job still carries `error: "partial: 112/116 completed"`
   (`caveats/audit-open.md#stale-job-error`).
7. Balloon-to-speaker geometry via the unused `det`/`seg` heads
   (`caveats/speaker-attribution.md#tail-is-not-geometry`) is now behind item 2. With no name to attach,
   geometry buys nothing.
8. Resolve a speaker answer across the whole dialogue window, not just the answering panel. The last 3
   unresolved refs describe a neighbouring panel in the same 8-panel call.
9. Start Phase 2 from `ROADMAP.md`. Set SQLite `busy_timeout` before any concurrency work
   (`caveats/audit-open.md#sqlite-locking`).

## Lesson worth keeping

Every metric recorded before this session said the pipeline was fine or nearly fine. `script` 116/116,
"9 named speech lines", `layers` 116/116, `assemble` 1/1. Watching two and a half minutes of output
found a 20% sync failure, a cast that is 84% anonymous, invented narration, and a stage that writes
nothing while reporting success. Stage counters measure whether code ran. They say nothing about whether
the result is correct. Watch the output before trusting a number.

## Running the pieces

```bash
./start_workers.sh              # session_manager + 9 workers, each a uvicorn in a tmux window
tmux attach -t manga-workers    # per-worker logs
.venv/bin/python worker_render.py    # self-check, runs real ffmpeg, about 4 minutes
```

Read the state, or clear a stage and resume:

```bash
/usr/bin/ssh kami@192.168.1.104 "curl -s 'http://127.0.0.1:9090/job/status?job_id=778297bc-e7ce-439d-91b5-8a027060d17f'"
/usr/bin/ssh kami@192.168.1.104 "curl -s -X POST http://127.0.0.1:9090/stage/clear -H 'Content-Type: application/json' -d '{\"job_id\":\"778297bc-e7ce-439d-91b5-8a027060d17f\",\"stage\":\"<stage>\"}'"
/usr/bin/ssh kami@192.168.1.104 "curl -s -X POST 'http://127.0.0.1:9090/job/resume?job_id=778297bc-e7ce-439d-91b5-8a027060d17f'"
```

Traps: `tmux respawn-window -k` does NOT re-run the window's command, it drops to a bare shell and the
worker stays down. Restart one worker by re-sending the `send-keys` line from `start_workers.sh`, and
confirm with `/health` plus `./check_stale.sh`. `docker compose up --build orchestrator` recreates the
container, which drops any `docker cp`'d script such as `audit_registry.py`.
Plain `ssh` is the kitty ssh kitten and refuses non-interactive stdin, so use `/usr/bin/ssh`.
`mc` aliases on homesrv are `homesrv` and `mio`. `local` returns Access Denied and `rfs` is the empty
rustfs. `cp` is aliased to `cp -i` and hangs on overwrite, so use `/usr/bin/cp -f`.

Re-fixing assembly needs the real clips, which the session scratchpad no longer holds:

```bash
/usr/bin/ssh kami@192.168.1.104 'P=homesrv/video/ef105a86-4b7e-4ac4-b45c-b7d83b8f5b5e/7c944dd4-e972-42c7-ba60-9f6939548e80; mc cp -q -r $P/clips/ /tmp/rclips/; cd /tmp/rclips && tar cf - .' | tar xf - -C clips/
```

## Storage and viewer, tasks #116/#117

[#117] is done. `stowage` serves the manga buckets. It was never a MinIO problem: the container had been
dead since 2026-07-19 on an arm64 digest pin.

[#116] is closer but not cut over. Artifacts split one bucket per class
(`decisions/storage-layout.md#bucket-per-artifact`), and both MinIO and `rustfs` hold all six buckets.
`rustfs` on `127.0.0.1:9010/9011` is still empty and nothing is repointed, so MinIO serves every read
and write. Remaining: `mc mirror` the live buckets, verify counts and sizes, then decide on cutover
(`decisions/storage-layout.md#rustfs-staged`).

Two containers on homesrv had been dead for two weeks and now run. `manga-fetch` is the one
`/job/create` needs. `manga-web` is what `manga.kvmx.ru` proxies to on 8083. Nothing watches them, and
nothing watches the workers.

## Open questions

Four Phase 1 items have no Vikunja task, because writing to the tracker was not asked for: the speaker
contract fix, the verifier rules, the tracklet constraints, and the flag resolution path. Only [#203]
existed and is closed by `decisions/audit-phase1.md#unlocked-model-load`.

Three audit items are deliberately not done and are recorded as caveats rather than silently dropped:
honest stage clearing, ComfyUI under the session mutex, and reversible identity merges. Each needs a
design decision, not a patch.

Carried over from the reconstruction: `.venv` needs the ROCm torch wheel reinstalled, and `dots.tts/`,
`legacy/`, `RESUME_SPEC.md`, `pipeline-design-notes.md`, `spec-v2.md` are unrecoverable.