manga-recap-pipeline/caveats/audit-open.md

Open limits from the 2026-08-11 audit

Everything here was read in source during the audit and deliberately left unfixed in Phase 1. The fixed findings live in decisions/audit-phase1.md. Line numbers are from the audit and may drift.

Reconcile deletes the losing character irreversibly

Reconciliation used to delete the losing character row. Clearing the reconcile stage did not undo it, and nothing recorded which detections had been the loser's.

Half-closed 2026-08-12. The loss is no longer unrecoverable. merge_characters marks the loser merged_into = keeper instead of deleting it, so its embedding, description and gender survive, and every repointed assignment is stamped method = merged_from:<loser_id> in identity_assignment_sources. Those two records are enough to walk a merge backwards. Roster readers filter merged_into IS NULL. Lookup by id does not, so an assignment still pointing at a merged id resolves.

What is still missing is the mechanism that consumes them: there is no unmerge, and no split. Undoing a merge today means a manual SQL walk of the two records above.

Costs: a wrong merge needs hand-written SQL to undo, not a rebaseline. Revisit when: a reviewer reports a wrong merge, or the review gates from [#136] get a place to hang name/merge/split actions. No wrong merge has been seen since the crops were fixed, so building the unmerge before either trigger would be speculative.

Clearing a stage does not undo what it wrote

The dialogue and direct half of this is fixed and proven (decisions/storage-layout.md#clear-vision-blob). It cost a wasted rerun on 2026-08-11 first: the clear returned {"ok": true}, deleted nothing, and the stage skipped all 116 panels.

What remains: clearing identity preserves the per-manga registry by design, so a rerun inherits every character it ever minted (caveats/speaker-attribution.md#registry-pollution). Nothing verifies that a clear emptied what it claimed.

Costs: a rerun silently reuses stale output, which reads as a reproducible result. Revisit when: any stage is scheduled concurrently or resumed automatically. A stage must be idempotent before either is safe.

ComfyUI uses the GPU outside the session mutex

The layers stage calls ComfyUI directly and takes no lease. Another job can load gemma, siglip2, or dots while ComfyUI holds the same GPU.

Costs: out-of-memory failures that look random and land on an unrelated stage. Revisit when: layers is enabled on a real run, or a second concurrent job is allowed. Workaround: MAX_CONCURRENT_JOBS=1 keeps one pipeline at a time, which is the current default.

The JSON repair pass can fabricate dialogue

call_gemma4_json hands the model its own truncated text and asks for the JSON it should have been (worker_vision.py). The repair call carries no image. On a response truncated by max_tokens, the model completes dialogue it can no longer see. What it adds is indistinguishable downstream from transcribed text.

Costs: invented lines enter the script with normal provenance. Revisit when: schema-constrained generation lands, which removes most of this path. Workaround: resend the image on repair, or retry a truncated response instead of repairing it.

/review/preview pins a solo clip into the final video

review_preview renders one panel and calls save_clip(panel_id, ...). _render_one_beat returns early when a clip already exists for the leader. Previewing a beat leader therefore drops the rest of the beat's panels. review_retts gets this right by calling delete_clip first.

Costs: a reviewer silently corrupts the output by looking at it. Revisit when: the review UI is used on a real chapter. Workaround: never preview a beat leader, or delete the clip row afterwards.

Stored embeddings carry no model or pooling version

embed_crop uses pooler_output when present and falls back to mean-pooled patch tokens otherwise. The two paths produce different vector spaces, and the fixed 0.85 threshold is valid for one of them. Nothing beside a stored .npy records which model, revision, or pooling produced it.

Costs: a transformers upgrade mixes incompatible vectors into one gallery with no error. Revisit when: transformers or the siglip2 revision is upgraded. Before, not after. Workaround: none. Write the model id and pooling mode beside the vector and refuse cross-version comparison.

Worker endpoints block the event loop

Endpoints declared async def run blocking MinIO, OpenCV, torch, and ffmpeg calls directly on the event loop. A busy worker cannot answer /health or /unload.

Costs: /health/workers reports a working worker as unreachable, and the session manager's 30-second /unload can time out exactly when VRAM needs freeing. Revisit when: a stage stalls on /unload, or before any bounded parallelism lands. Workaround: def instead of async def moves each handler to the threadpool. Tracked as [#199] for the render worker.

SQLite has no busy timeout

get_conn opens a connection per call with no busy_timeout. WAL tolerates one writer. PIPELINE=1 already writes clips from concurrent tasks while TTS writes audio.

Costs: planned CPU parallelism will surface as database is locked before it surfaces as throughput. Revisit when: Phase 2 concurrency work starts. Set the timeout first. Workaround: keep PIPELINE off.

layers runs after tts

STAGES orders layers after tts, while run_stage_tts warns that eager rendering under PIPELINE=1 needs layers to run first.

Costs: with the flag on, solo beats always render without parallax. Revisit when: a real run enables PIPELINE=1. Workaround: keep PIPELINE off, or reorder STAGES.

completed means something different in each stage

A dropped vision panel fails the stage and halts the pipeline. A failed direction window counts its panels as done. Layers always finishes completed.

Costs: the acceptance metrics in ROADMAP.md cannot be read across stages. Revisit when: a baseline measurement is taken. The numbers are meaningless until then. Workaround: none.

Identity worker caches characters the orchestrator has deleted

worker_identity._known_cache is invalidated only by _persist_char. The reconcile stage deletes losing characters directly in the orchestrator database. A long-lived worker keeps shortlisting and assigning ids that no longer exist.

Costs: assignments point at rows that are gone. Revisit when: reconcile runs on a chapter without a worker restart between stages. Tracked as [#201], which proposes caching per manga_id. Workaround: restart the identity worker after reconcile.

A character seen once gets no assignment at all

worker_identity._pending holds full crop images in worker memory for a whole chapter, keyed by session. That is durable per-chapter state inside a worker documented as stateless, and it is lost on restart. A character seen exactly once receives no assignment, not even a chapter-local handle.

Costs: one-off characters vanish from the scene graph. Revisit when: chapter-local tracklet persistence lands (ROADMAP.md, Phase 3). Workaround: none.

MinIO credentials are hardcoded in committed source

Defaults live in transport.py and service.py.

Costs: the credentials are in git history for anyone who gets the repo. Revisit when: the repo leaves this machine, or MinIO is reachable outside the LAN. Workaround: the environment variables already override them. Set them and remove the defaults.

Assemble marks a job completed with no clips

run_stage_assemble does not check that clip_uris is non-empty before assembling, then marks the job completed.

Costs: a failed chapter reports success. Revisit when: any run reports completed without a video. One if not clip_uris guard fixes it. Workaround: none.

Reviewer timestamps drift against the crossfaded video

/review/panels sums per-panel audio durations. Assemble crossfades clips using the per-beat transitions, so every non-cut transition shortens the real video.

Costs: reviewer timestamps drift further out of sync the further into the chapter they scrub. Revisit when: the review UI is used for timing work. Workaround: subtract the transition overlaps by hand.

A completed job keeps the error from an earlier failure

/job/resume does not clear jobs.error. Job 778297bc finished every stage and still reports status: "completed" beside error: "partial: 112/116 completed", a message from three resumes earlier.

Costs: any reader of the error field sees a failure on a successful job. The review UI and any future alerting both read it. Revisit when: anything branches on jobs.error, or a run is judged by its status alone.

layers reports success on an empty bucket

Measured on job 778297bc: layers reported completed 116/116 while s3://layers/ held 0 objects. Every clip in that run therefore has no parallax. The stage is a sibling of #inconsistent-stage-policy, but this is the measured instance: worker_layers.py's own self-check already fails on the missing legacy/qwen_layered_workflow.json, and the stage still reports done for every panel.

Costs: a silent quality regression that no status field reveals. Revisit when: parallax matters for a deliverable, or before quoting this run as a full-pipeline pass.

Identity coverage has fallen on every run since the gate landed

70% -> 61% -> 50% -> 57% across the four 2026-08-12 runs. The gate is stable at 39 to 40% of detections.

The 18:07 run resolves most of this. Coverage is now the gate and nothing else. All 68 assignments went to face-bearing detections and none to a gated one. Recall among face-bearing detections is 68 of 71, or 96%, up from 82%. The resolver is not losing cast. The 43% that carries no assignment is the 40% the gate drops plus 3 detections.

So the coverage number no longer measures the resolver. What it measures is the cost of requiring a face, which is item 4 in NEXT.md. Panel 7 is the case: 5 of its 6 detections carry has_face = False, and the one that does is unassigned.

The open question moved. The lead now holds 36 of 68 assignments, or 53%, against 25 of 59 before, and audit_registry.py flags that as absorbing. Whether 36 is the protagonist appearing in half the panels or an over-merge cannot be told apart without labels.

Revisit trigger: before trusting the registry for a downstream run.

The 145 ground-truth labels do not cover this chapter. Measured 2026-08-12. All 145 rows in identity_labels key to chapter 8ca8249b, a different manga with cast "Rico" and "Ikekin", over 81 distinct panels. Chapter 7c944dd4 has zero labels, so /review/identity?job_id=778297bc... returns labeled: 0, accuracy: null. Scoring 8ca instead gives 7/138, because its identity run is stale: 44 assignments over 246 panels, predating all seven fixes. That number measures the old code on the wrong chapter and settles nothing.

The plumbing works, the labels are missing. Precision and recall on the current registry need a hand pass over 7c944dd4 first, via POST /review/identity/label per detection. Label against the bbox, not the local_id. Vision is non-deterministic and reassigns local_id top-to-bottom every run. A label taken before a vision rerun then points at whoever now occupies that slot.

Embedding cosine on a person crop cannot tell two people apart

Measured 2026-08-12 on the 18:07 run's 22 crops for character_2367d70c, the embeddings pulled from manga/<manga_id>/characters/_crops/*.npy and L2-normalised. 1152 dimensions.

pair	cosine
the cat at p098 against any man	up to 0.82
the red-robe chibi at p088 against any man	up to 0.75
two visibly different men (order 60~80, 88~110)	0.93, 0.92
the highest pair in the matrix (47~51, one man, one shirt)	0.96

Different people reach 0.93 and the same person reaches 0.96. No threshold separates them. So a crop-to-crop cosine link is not available, and the candidates overlap condition in link_tracklets is close to noise for the same reason: the top-k is chosen by this metric, so every crop shortlists the same few rows.

Inference, not yet tested: identity embeds the whole person box, which holds background, clothing and pose. Those change between scenes while every crop shares one art style, so cosine measures "manga crop of a person in an office". face_detect already finds the face and pairs it to the person box for has_face, so embedding the face region instead is a small change.

Revisit trigger: before building anything else on cosine. The test is to crop the faces of these same 22 detections, embed them, and recompute this matrix. If a threshold appears, embed faces.

The gallery cap drops exactly the rows a tracklet needs

Open. Observed on the 19:44 run of 2026-08-12.

cast_gallery orders the live cast named-first, then registry order, and run_stage_identity truncates the tail at GALLERY_CAP = 10. Registry order is creation order, so the tail is the anonymous characters this very run minted — and a later tracklet of a recurring unnamed person is precisely what needs to match one of those. The chapter has two named rows, so on this run the cap dropped 6 unnamed rows at p097 and 1 at p109, and a tracklet that should have joined one of them can only mint a duplicate instead.

It is capped because every candidate costs at least one reference image in the resolve prompt beside the query crop. The cap is a VRAM and context budget, not a modelling choice.

Revisit when the cast on a chapter routinely exceeds 10 gender-compatible rows, which it already did here. The fix is to order the gallery by how many assignments each character already holds in this chapter, so the tail is the rows nobody has matched rather than the rows nobody has named yet. That needs one count query per tracklet.