manga-recap-pipeline/caveats/audit-open.md

# Open limits from the 2026-08-11 audit

Everything here was read in source during the audit and deliberately left unfixed in Phase 1. The
fixed findings live in `decisions/audit-phase1.md`. Line numbers are from the audit and may drift.

## Reconcile deletes the losing character irreversibly {#destructive-reconcile}

Reconciliation used to delete the losing character row. Clearing the reconcile stage did not undo it, and
nothing recorded which detections had been the loser's.

**Half-closed 2026-08-12.** The loss is no longer unrecoverable. `merge_characters` marks the loser
`merged_into = keeper` instead of deleting it, so its embedding, description and gender survive, and every
repointed assignment is stamped `method = merged_from:<loser_id>` in `identity_assignment_sources`. Those
two records are enough to walk a merge backwards. Roster readers filter `merged_into IS NULL`. Lookup by
id does not, so an assignment still pointing at a merged id resolves.

What is still missing is the mechanism that consumes them: there is no unmerge, and no split. Undoing a
merge today means a manual SQL walk of the two records above.

Costs: a wrong merge needs hand-written SQL to undo, not a rebaseline.
Revisit when: a reviewer reports a wrong merge, or the review gates from [#136] get a place to hang
name/merge/split actions. No wrong merge has been seen since the crops were fixed, so building the unmerge
before either trigger would be speculative.

## Clearing a stage does not undo what it wrote {#dishonest-clearing}

The dialogue and direct half of this is fixed and proven
(`decisions/storage-layout.md#clear-vision-blob`). It cost a wasted rerun on 2026-08-11 first: the clear
returned `{"ok": true}`, deleted nothing, and the stage skipped all 116 panels.

What remains: clearing identity preserves the per-manga registry by design, so a rerun inherits every
character it ever minted (`caveats/speaker-attribution.md#registry-pollution`). Nothing verifies that a
clear emptied what it claimed.

Costs: a rerun silently reuses stale output, which reads as a reproducible result.
Revisit when: any stage is scheduled concurrently or resumed automatically. A stage must be idempotent
before either is safe.

## ComfyUI uses the GPU outside the session mutex {#comfyui-unscheduled}

The layers stage calls ComfyUI directly and takes no lease. Another job can load gemma, siglip2, or
dots while ComfyUI holds the same GPU.

Costs: out-of-memory failures that look random and land on an unrelated stage.
Revisit when: layers is enabled on a real run, or a second concurrent job is allowed.
Workaround: `MAX_CONCURRENT_JOBS=1` keeps one pipeline at a time, which is the current default.

## The JSON repair pass can fabricate dialogue {#repair-fabricates}

`call_gemma4_json` hands the model its own truncated text and asks for the JSON it should have been
(`worker_vision.py`). The repair call carries no image. On a response truncated by `max_tokens`, the
model completes dialogue it can no longer see. What it adds is indistinguishable downstream from
transcribed text.

Costs: invented lines enter the script with normal provenance.
Revisit when: schema-constrained generation lands, which removes most of this path.
Workaround: resend the image on repair, or retry a truncated response instead of repairing it.

## `/review/preview` pins a solo clip into the final video {#preview-overwrites-clip}

`review_preview` renders one panel and calls `save_clip(panel_id, ...)`. `_render_one_beat` returns
early when a clip already exists for the leader. Previewing a beat leader therefore drops the rest of
the beat's panels. `review_retts` gets this right by calling `delete_clip` first.

Costs: a reviewer silently corrupts the output by looking at it.
Revisit when: the review UI is used on a real chapter.
Workaround: never preview a beat leader, or delete the clip row afterwards.

## Stored embeddings carry no model or pooling version {#untagged-embeddings}

`embed_crop` uses `pooler_output` when present and falls back to mean-pooled patch tokens otherwise.
The two paths produce different vector spaces, and the fixed 0.85 threshold is valid for one of them.
Nothing beside a stored `.npy` records which model, revision, or pooling produced it.

Costs: a transformers upgrade mixes incompatible vectors into one gallery with no error.
Revisit when: transformers or the siglip2 revision is upgraded. Before, not after.
Workaround: none. Write the model id and pooling mode beside the vector and refuse cross-version
comparison.

## Worker endpoints block the event loop {#blocking-event-loop}

Endpoints declared `async def` run blocking MinIO, OpenCV, torch, and ffmpeg calls directly on the
event loop. A busy worker cannot answer `/health` or `/unload`.

Costs: `/health/workers` reports a working worker as unreachable, and the session manager's 30-second
`/unload` can time out exactly when VRAM needs freeing.
Revisit when: a stage stalls on `/unload`, or before any bounded parallelism lands.
Workaround: `def` instead of `async def` moves each handler to the threadpool. Tracked as [#199] for
the render worker.

## SQLite has no busy timeout {#sqlite-locking}

`get_conn` opens a connection per call with no `busy_timeout`. WAL tolerates one writer. `PIPELINE=1`
already writes clips from concurrent tasks while TTS writes audio.

Costs: planned CPU parallelism will surface as `database is locked` before it surfaces as throughput.
Revisit when: Phase 2 concurrency work starts. Set the timeout first.
Workaround: keep `PIPELINE` off.

## `layers` runs after `tts` {#layers-after-tts}

`STAGES` orders `layers` after `tts`, while `run_stage_tts` warns that eager rendering under
`PIPELINE=1` needs layers to run first.

Costs: with the flag on, solo beats always render without parallax.
Revisit when: a real run enables `PIPELINE=1`.
Workaround: keep `PIPELINE` off, or reorder `STAGES`.

## `completed` means something different in each stage {#inconsistent-stage-policy}

A dropped vision panel fails the stage and halts the pipeline. A failed direction window counts its
panels as done. Layers always finishes completed.

Costs: the acceptance metrics in `ROADMAP.md` cannot be read across stages.
Revisit when: a baseline measurement is taken. The numbers are meaningless until then.
Workaround: none.

## Identity worker caches characters the orchestrator has deleted {#stale-known-cache}

`worker_identity._known_cache` is invalidated only by `_persist_char`. The reconcile stage deletes
losing characters directly in the orchestrator database. A long-lived worker keeps shortlisting and
assigning ids that no longer exist.

Costs: assignments point at rows that are gone.
Revisit when: reconcile runs on a chapter without a worker restart between stages. Tracked as [#201],
which proposes caching per `manga_id`.
Workaround: restart the identity worker after reconcile.

## A character seen once gets no assignment at all {#pending-in-worker-memory}

`worker_identity._pending` holds full crop images in worker memory for a whole chapter, keyed by
session. That is durable per-chapter state inside a worker documented as stateless, and it is lost on
restart. A character seen exactly once receives no assignment, not even a chapter-local handle.

Costs: one-off characters vanish from the scene graph.
Revisit when: chapter-local tracklet persistence lands (`ROADMAP.md`, Phase 3).
Workaround: none.

## MinIO credentials are hardcoded in committed source {#hardcoded-credentials}

Defaults live in `transport.py` and `service.py`.

Costs: the credentials are in git history for anyone who gets the repo.
Revisit when: the repo leaves this machine, or MinIO is reachable outside the LAN.
Workaround: the environment variables already override them. Set them and remove the defaults.

## Assemble marks a job completed with no clips {#empty-assemble}

`run_stage_assemble` does not check that `clip_uris` is non-empty before assembling, then marks the job
completed.

Costs: a failed chapter reports success.
Revisit when: any run reports completed without a video. One `if not clip_uris` guard fixes it.
Workaround: none.

## Reviewer timestamps drift against the crossfaded video {#timeline-drift}

`/review/panels` sums per-panel audio durations. Assemble crossfades clips using the per-beat
transitions, so every non-`cut` transition shortens the real video.

Costs: reviewer timestamps drift further out of sync the further into the chapter they scrub.
Revisit when: the review UI is used for timing work.
Workaround: subtract the transition overlaps by hand.

## A completed job keeps the error from an earlier failure {#stale-job-error}

`/job/resume` does not clear `jobs.error`. Job `778297bc` finished every stage and still reports
`status: "completed"` beside `error: "partial: 112/116 completed"`, a message from three resumes earlier.

Costs: any reader of the error field sees a failure on a successful job. The review UI and any future
alerting both read it.
Revisit when: anything branches on `jobs.error`, or a run is judged by its status alone.

## `layers` reports success on an empty bucket {#layers-writes-nothing}

Measured on job `778297bc`: `layers` reported `completed 116/116` while `s3://layers/` held 0 objects.
Every clip in that run therefore has no parallax. The stage is a sibling of
[#inconsistent-stage-policy](#inconsistent-stage-policy), but this is the measured instance:
`worker_layers.py`'s own self-check already fails on the missing
`legacy/qwen_layered_workflow.json`, and the stage still reports done for every panel.

Costs: a silent quality regression that no status field reveals.
Revisit when: parallax matters for a deliverable, or before quoting this run as a full-pipeline pass.

## Identity coverage has fallen on every run since the gate landed {#coverage-trend}

70% -> 61% -> 50% -> 57% across the four 2026-08-12 runs. The gate is stable at 39 to 40% of detections.

**The 18:07 run resolves most of this.** Coverage is now the gate and nothing else. All 68 assignments went
to face-bearing detections and none to a gated one. Recall among face-bearing detections is 68 of 71, or
96%, up from 82%. The resolver is not losing cast. The 43% that carries no assignment is the 40% the gate
drops plus 3 detections.

So the coverage number no longer measures the resolver. What it measures is the cost of requiring a face,
which is item 4 in `NEXT.md`. Panel 7 is the case: 5 of its 6 detections carry `has_face = False`, and the
one that does is unassigned.

The open question moved. The lead now holds 36 of 68 assignments, or 53%, against 25 of 59 before, and
`audit_registry.py` flags that as absorbing. Whether 36 is the protagonist appearing in half the panels or
an over-merge cannot be told apart without labels.

Revisit trigger: before trusting the registry for a downstream run.

**The 145 ground-truth labels do not cover this chapter.** Measured 2026-08-12. All 145 rows in
`identity_labels` key to chapter `8ca8249b`, a different manga with cast "Rico" and "Ikekin", over 81
distinct panels. Chapter `7c944dd4` has zero labels, so `/review/identity?job_id=778297bc...` returns
`labeled: 0, accuracy: null`. Scoring 8ca instead gives 7/138, because its identity run is stale: 44
assignments over 246 panels, predating all seven fixes. That number measures the old code on the wrong
chapter and settles nothing.

The plumbing works, the labels are missing. Precision and recall on the current registry need a hand
pass over `7c944dd4` first, via `POST /review/identity/label` per detection. Label against the bbox, not
the `local_id`. Vision is non-deterministic and reassigns `local_id` top-to-bottom every run. A label
taken before a vision rerun then points at whoever now occupies that slot.

## Embedding cosine on a person crop cannot tell two people apart {#cosine-not-identity}

Measured 2026-08-12 on the 18:07 run's 22 crops for `character_2367d70c`, the embeddings pulled from
`manga/<manga_id>/characters/_crops/*.npy` and L2-normalised. 1152 dimensions.

| pair | cosine |
| --- | --- |
| the cat at `p098` against any man | up to 0.82 |
| the red-robe chibi at `p088` against any man | up to 0.75 |
| two visibly different men (`order 60~80`, `88~110`) | 0.93, 0.92 |
| the highest pair in the matrix (`47~51`, one man, one shirt) | 0.96 |

Different people reach 0.93 and the same person reaches 0.96. No threshold separates them. So a
crop-to-crop cosine link is not available, and the `candidates` overlap condition in `link_tracklets` is
close to noise for the same reason: the top-k is chosen by this metric, so every crop shortlists the same
few rows.

Inference, not yet tested: identity embeds the whole person box, which holds background, clothing and pose.
Those change between scenes while every crop shares one art style, so cosine measures "manga crop of a
person in an office". `face_detect` already finds the face and pairs it to the person box for `has_face`,
so embedding the face region instead is a small change.

Revisit trigger: before building anything else on cosine. The test is to crop the faces of these same 22
detections, embed them, and recompute this matrix. If a threshold appears, embed faces.

## The gallery cap drops exactly the rows a tracklet needs {#gallery-cap-drops-the-unnamed}

**Open. Observed on the 19:44 run of 2026-08-12.**

`cast_gallery` orders the live cast named-first, then registry order, and `run_stage_identity` truncates
the tail at `GALLERY_CAP = 10`. Registry order is creation order, so the tail is the anonymous characters
this very run minted — and a later tracklet of a recurring unnamed person is precisely what needs to match
one of those. The chapter has two named rows, so on this run the cap dropped 6 unnamed rows at `p097` and
1 at `p109`, and a tracklet that should have joined one of them can only mint a duplicate instead.

It is capped because every candidate costs at least one reference image in the resolve prompt beside the
query crop. The cap is a VRAM and context budget, not a modelling choice.

**Revisit when** the cast on a chapter routinely exceeds 10 gender-compatible rows, which it already did
here. The fix is to order the gallery by how many assignments each character already holds in this chapter,
so the tail is the rows nobody has matched rather than the rows nobody has named yet. That needs one count
query per tracklet.

## The gemma helpers exist twice and have diverged {#gemma-helpers-duplicated}

**Open. Found by the repo audit of 2026-08-13, not yet by a failure.**

`call_gemma4`, `_extract_json` and `_strip_thought` are defined in both `worker_vision.py` and
`worker_script.py`. They are no longer the same code. `worker_vision.call_gemma4` is 21 lines and takes a
content list; `worker_script`'s is 8 and takes a prompt string plus a system prompt. Both `_extract_json`
bodies carry the same comment about `raw_decode` stopping at the first object, and raise different errors.

Who pays: whoever fixes the JSON path. `#repair-fabricates` above says the repair pass can fabricate
dialogue. A fix written against one copy leaves the other worker on the old behaviour, and no self-check
compares them.

**Revisit when** either JSON path is touched for any reason. The fix is a `gemma.py` holding the three
helpers, imported by both workers, which removes about 25 duplicated lines and costs one file.