manga-recap-pipeline/AUDIT.md

Manga recap pipeline audit

Date: 2026-08-11

Scope: workpc workers plus the homesrv orchestrator. This audit excludes FFmpeg changes and does not recommend Magi or its embeddings. No GPU workloads were run during the audit.

Status

Phase 1 landed on 2026-08-11. This file is the reasoning, not the state.

where the finding went	file
implemented, with evidence and a runnable check	decisions/audit-phase1.md
deliberately not done, with a revisit trigger	caveats/audit-open.md
still ahead, in order	ROADMAP.md
what to do next	NEXT.md

Every P0 below is closed. The sections that name one now point at its decision instead of repeating it.

Outcome

The strongest performance gains are available without replacing the current models:

1. Keep Gemma resident across compatible stages, reducing roughly seven chapter-level Gemma loads to two.
2. Batch SigLIP crop embeddings instead of running one forward pass per detected character.
3. Add bounded parallelism to CPU/network work such as page uploads, framed-page cropping, prefetching, and scene construction.
4. Stop repeatedly downloading and base64-encoding the same panels and character references.
5. Generate schema-constrained JSON and batch text-only script work by scene.

Correctness fixes should land before throughput work because the current tracklet and naming paths can contaminate the permanent character registry.

Highest-priority findings

P0: tracklet gender gate uses the wrong enum — CLOSED, decisions/audit-phase1.md#tracklet-hard-constraints

orchestrator/tracklets.py checks male|female, while vision emits m|f|unknown. The tracklet hard gender gate therefore never activates on normal pipeline data.

P0: tracklets lack co-presence constraints — CLOSED, decisions/audit-phase1.md#tracklet-hard-constraints

The linker has no same-panel exclusion and uses transitive union-find. Two similar-looking people in the same panel can be joined, and weak links such as AB and BC can merge A and C even when the endpoints are incompatible.

P0: face-to-character pairing is unconditional — CLOSED, decisions/audit-phase1.md#gated-face-pairing

worker_vision._pair_faces_to_present() assigns every detected face to the nearest vision character as long as an unused character exists. Despite the docstring, there is no distance or overlap threshold. A distant or unrelated face can therefore receive another character's identity and speaker label.

P0: roster hints can become permanent identity evidence

The roster is described as hints-only, but those names are fed into detection. Any name emitted on a crop causes worker_identity to persist it immediately. A coarse appearance-to-roster guess can therefore contaminate a permanent character gallery.

P0: script repair feedback is discarded — CLOSED, decisions/audit-phase1.md#verifier-false-positives

The orchestrator sends beat and verifier_feedback on a failed-script retry, but worker_script.ScriptInput does not define either field and the prompt builder does not consume them. The second call is another stochastic attempt rather than a targeted correction.

P0: action evidence is missing from the verifier — CLOSED, decisions/audit-phase1.md#verifier-false-positives

The beat builder reads plural actions, while the scene worker emits singular action. Script validation therefore receives little or no action evidence and cannot reliably detect invented or omitted actions.

P0: GPU leases are unsafe for concurrent jobs — CLOSED, decisions/audit-phase1.md#no-lease-stealing

When /session/open returns 409, the homesrv proxy assumes the active lease is stale and closes it. A second legitimate job can terminate the first job's model. heartbeat_session() exists but is not used by the pipeline.

P1: repeated model cold starts

A normal chapter can load Gemma separately for:

1. roster;
2. vision;
3. identity adjudication;
4. reconcile;
5. dialogue;
6. direction;
7. script.

The source itself notes that each load takes tens of seconds to minutes. Removing approximately five of these loads is likely the largest safe wall-time improvement.

P1: repeated transfers and encoding

• /direct/window downloads every image once for grouping and again for shot design.
• Identity resolver calls repeatedly download the same candidate reference images.
• Each local image is base64-expanded into the JSON sent to the local llama-server.
• Dialogue overlap causes repeated download and CPU set-of-mark work for overlap panels.

P1: per-item GPU and database work

• SigLIP runs one forward pass per character crop rather than a bounded batch.
• Only one embedding URI is loaded per known character even when its reference gallery has several views.
• Scene construction makes one homesrv-to-workpc HTTP call per panel for a cheap JSON join.
• Many stage loops repeatedly open SQLite connections and commit one row at a time.

Performance and scheduling plan

1. Keep Gemma resident across model phases

Use this chapter schedule:

CPU:    fetch -> crop
Gemma:  roster -> vision
SigLIP: batched identity embeddings and shortlists
Gemma:  identity adjudication -> reconcile -> dialogue -> direct -> CPU scene join -> script
Dots:   TTS
Comfy:  optional layers

This needs pipeline-owned model leases instead of every stage opening and closing its own lease. Scene construction should run locally on homesrv or as a batch while the second Gemma lease remains open.

ComfyUI must be represented as a GPU resource too. It currently bypasses the session manager, so another job could load Gemma, SigLIP, or Dots while ComfyUI is using the same GPU.

2. Batch SigLIP inference

For each request or chapter batch:

1. download a panel once;
2. clamp and crop all detected characters;
3. preprocess a bounded image batch;
4. run one model forward pass;
5. compare the resulting matrix with the known gallery in one vectorized operation.

Batch size should be tuned against VRAM rather than hard-coded optimistically.

3. Add safe bounded CPU parallelism

• Fetch/upload pages with concurrency 4-8.
• Crop framed pages with concurrency 2-4, buffer results, then insert them in page order.
• Keep whole-strip webtoon restitching sequential.
• Pre-download the next vision/dialogue window while Gemma processes the current window.
• Precompute face/text detections with a small CPU worker pool while avoiding ONNX thread oversubscription.
• Batch scene construction and SQLite reads/writes.

GPU calls should remain serialized initially. Multiple llama-server slots divide the configured context among slots and need an explicit VRAM/context benchmark before enabling concurrent multimodal requests.

4. Eliminate repeated image movement

Add an ephemeral /dev/shm LRU keyed by S3 URI plus object version/etag. It is a cache, not durable worker state, and may be dropped at any time.

Reuse downloaded image paths between the two direction passes and cache candidate reference images during the identity phase. Configure llama-server with --media-path /dev/shm and send file:// image paths instead of base64 data URIs. The installed llama.cpp version supports local media paths and schema-constrained responses.

Reference: llama-server documentation.

5. Use JSON-schema-constrained generation

Provide a JSON schema for detect, dialogue, roster, direction, same-person, and identity-resolution calls. This should eliminate most malformed-output repair calls and make missing fields explicit. Keep semantic validation after parsing because a structurally valid answer can still be wrong.

6. Reduce model call count

• Dialogue context should contain the current scene's recent 12-20 turns, not up to 400 lines from the whole chapter.
• Reuse direction images first, then A/B test combining grouping and shot selection into one structured call.
• Replace the per-window LLM direction summary with deterministic structured continuity where possible.
• Generate 6-12 consecutive script beats per text-only call, grouped by scene, with stable beat IDs and targeted per-beat repair.

Character-recognition plan

The critical path keeps the current detector, SigLIP shortlisting, tracklets, and Gemma resolver.

1. Correct the tracklet spine

• Canonicalize gender as m|f|unknown and species as a validated enum everywhere.
• Add hard same-panel/co-presence cannot-links.
• Treat manual identity assignments as immutable constraints.
• Prevent transitive bridges from joining tracks whose hard constraints disagree.
• Give every detection a chapter-local provisional track ID, even if it never becomes a global character.
• Move provisional state out of the workpc process and into a resumable homesrv chapter artifact.

2. Improve visual evidence

• Validate and clamp all VLM bounding boxes before cropping.
• Match detected faces to person/body boxes with gated IoU and normalized distance assignment rather than unconditional nearest-neighbour matching.
• Store both face and body crops.
• Score crop quality using face size, detector confidence, occlusion, and sharpness.
• Send Gemma the best 2-4 query views for a tracklet. The current resolver sends only one representative query crop; its candidate union is not a real multi-view query gallery.
• Store multiple embedding vectors per global character and use gallery maximum or centroid similarity only for shortlisting.

Face/body evidence plus local temporal constraints are supported by dedicated comic and manga re-ID work: Identity-Aware Semi-Supervised Learning for Comic Character Re-Identification and Unsupervised Manga Character Re-identification via Face-body and Spatial-temporal Associated Clustering.

3. Separate naming from identity

• Keep roster names as candidate vocabulary, never identity evidence.
• Detection may emit a typed name claim only when the panel visibly binds the name to that character.
• Require self-introduction, name-tag, or corroborated address evidence before promotion.
• Preserve aliases and honorific forms instead of stripping and discarding them.
• Never merge galleries on name equality alone.
• Resolve the tracklet first, then attach corroborated names to its global identity.

4. Add global constraints

Identity resolution should enforce:

• must-link membership within a validated tracklet;
• cannot-link for co-present characters;
• gender and species hard gates;
• manual locks;
• at most one appearance of a global character per panel;
• open-set outcomes known|new|unresolved without deleting the chapter-local track ID.

Gemma should receive a compact labelled identity board containing several query views and selected candidate gallery views, rather than up to fifteen separately downloaded image parts for every representative crop.

5. Improve identity evaluation

Extend the existing labels and evaluator to report:

• false merges versus false splits;
• tracklet purity and fragmentation;
• global identity accuracy;
• name-binding accuracy;
• unknown/NONE rate;
• speaker accuracy conditioned on identity correctness.

A replacement embedding backbone should only be considered if the structural fixes plateau, using the same labeled chapters for a controlled A/B comparison.

Story-coherence plan

1. Build an evidence ledger before prose

Create a deterministic artifact for every beat:

beat
  -> ordered panels
  -> ordered actions and chronology
  -> dialogue/region IDs and exact text
  -> typed speaker references, confidence, and method
  -> characters present
  -> location and time
  -> named entities
  -> uncertainty flags

This becomes the only factual input to script generation and validation.

2. Make context scene-scoped and evidence-bearing

• Reset speaker-turn context on strong location/time/scene changes.
• Keep a short structured story state rather than repeatedly summarizing generated prose.
• Give a script chunk only facts that occur before or inside that chunk.
• Do not pass a whole-chapter premise as if it were already-known story state; it can leak future events.
• Keep stable chapter-local handles for unresolved recurring characters so they do not disappear from the scene graph while awaiting global promotion.

3. Generate narration in ordered scene chunks

Generate several consecutive beats in one text-only call and return:

{"beats":[{"beat_id":"b01","text":"...","evidence_ids":["p001_r1"]}]}

This provides more continuity and fewer calls than one independent request per beat while keeping output small enough for targeted validation and repair.

4. Respect uncertainty

Low-confidence or weak-method speaker assignments should become someone, an unattributed quote, or a review flag instead of a confident named assertion. Identity and speaker uncertainty must survive into the script prompt.

5. Strengthen verification

Validate:

• exact/fuzzy quote grounding;
• permitted character and entity names;
• action coverage and unsupported actions;
• event chronology;
• future-fact leakage;
• repeated facts and repeated sentence openings;
• evidence IDs returned by the model.

Feed the exact failures into the repair prompt. A final text-only consistency audit may report affected beat IDs, but should not rewrite the whole chapter automatically.

Stage clearing and resumability

Before parallel or identity work, stage ownership of shared data needs to be explicit:

• dialogue and direction currently mutate the shared vision JSON;
• clearing dialogue does not remove its keys, so a rerun can treat old dialogue as completed;
• reconcile merges are destructive and are not reversed by clearing the reconcile stage;
• clearing identity preserves the per-manga registry, so it is not a clean identity rerun;
• name claims attached to a merged-away character are not repointed with the identity assignments.

Use stage-specific artifacts or explicit key deletion, and preserve provenance through merges. A stage must be genuinely idempotent before it is scheduled concurrently or resumed automatically.

Further design ideas

1. Make identity merges reversible

Reconciliation currently deletes the losing character. Replace destructive merging with a versioned cluster or redirect model:

character_cluster
  canonical_id
  member_ids[]
  merge_evidence[]
  cannot_link[]
  version

Assignments resolve through the canonical cluster while the original identities remain recoverable. A reviewer can remove one bad merge edge without clearing and rebuilding the entire identity stage.

Preserve negative evidence as a first-class artifact. Co-presence is particularly valuable: if A and B appear together as distinct detections, they are a permanent cannot-link unless a reviewer overrides it.

2. Separate detection, identity, and naming confidence

These are different questions and should never share one confidence value:

• detection confidence: is this actually a story character?
• identity confidence: which recurring visual identity is it?
• naming confidence: what is that identity's canonical name?

A person may be confidently detected and tracked as chapter_person_3 while remaining unnamed. That is enough for coherent narration such as "the colleague" without contaminating the global registry.

3. Use an uncertainty-driven identity cascade

Spend computation according to ambiguity and downstream impact:

hard constraints resolve the case
  -> accept cheaply

strong tracklet plus clear gallery margin
  -> accept without Gemma

ambiguous candidates
  -> multi-view Gemma resolution

high-impact unresolved identity
  -> human review

one-off background person
  -> chapter-local handle only

High-impact detections are recurring, speaking, named, or referenced by several future beats. A one-panel extra should not consume the same identity budget as a protagonist.

4. Add model-aware scheduling across jobs

Strict chapter-by-chapter execution causes extra model swaps when several jobs are queued. A resource-aware scheduler can group ready work by resident model:

ready Gemma detect phases
-> ready SigLIP phases
-> ready Gemma post-identity phases
-> ready Dots phases

This improves total throughput at the cost of some per-job latency. Add fairness and maximum-wait limits so one long title cannot starve other jobs. GPU conflicts must wait in a queue; they must never terminate the active owner's lease.

5. Cache artifacts by dependency fingerprint

Replace broad "clear stage X and everything downstream" behavior with content-addressed artifacts keyed by:

hash(input URIs
     + relevant upstream JSON
     + model version
     + prompt/schema version
     + stage settings)

Consequences:

• changing narration style does not rerun vision;
• correcting one identity invalidates only affected scenes and beats;
• changing direction does not invalidate dialogue;
• a resume reuses every artifact whose dependencies still match;
• prompt experiments can run side-by-side instead of overwriting the baseline.

Store dependency and version metadata beside every stage artifact so invalidation is explainable in the review UI.

6. Detect repeated artwork before expensive stages

Use CPU-side perceptual hashing after masking detected text regions:

• same artwork and same text: reuse visual and dialogue artifacts;
• same artwork but different text: reuse detection/identity, rerun dialogue;
• near-duplicate establishing shot: mark visual_repeat_of so narration does not describe it again;
• exact duplicate with no new information: curate it out early.

Text masking is required because two visually identical panels may contain different dialogue. Combine the art hash with text-region geometry and a text-content hash when available.

7. Make the manga registry temporal

Names, aliases, relationships, and facts need an evidence position:

known_from: chapter 12, panel 37
source: self_intro
confidence: 0.97

The script can then use only knowledge available at that point. This prevents later chapters or out-of-order processing from leaking names and revelations into earlier recaps. A configurable spoiler_safe versus omniscient_recap policy can decide whether later knowledge may backfill earlier narration.

The sampled chapter premise needs the same guard because a synopsis built from the whole chapter can reveal its payoff in the opening beat.

8. Represent dialogue as a relation graph

A spoken name often identifies the addressee or a third party rather than the speaker. Store:

speaker_ref
addressed_character_ref
mentioned_character_refs[]
spoken_text
name_surface_form
honorific
region_id
speaker_evidence

For example, "Seonho-oppa!" is evidence about the addressee and their relationship, not the speaker's name. Honorifics should be normalized but preserved as evidence instead of being stripped and discarded.

9. Treat speaker attribution as constrained assignment

For every speech region, build candidates from:

• bubble-tail or geometric evidence;
• visible faces and bodies;
• previous and following speaker turns;
• chapter-local identities;
• the off-panel cast;
• same-panel cannot-links.

Assign speakers jointly across a conversation window rather than deciding every line independently. Use Gemma only to adjudicate ambiguous regions. Require region_id in the model output so numbered set-of-mark regions are structurally connected to returned dialogue instead of remaining visual suggestions.

10. Separate stable identity traits from scene appearance

Store durable and temporary appearance separately:

identity_appearance:
  face, hair, species, stable marks

scene_appearance:
  outfit, accessories, injuries, disguise, chibi/age form

Clothing can be strong evidence inside one local scene but should be down-weighted across scene or time changes. This also lets narration mention an outfit change without creating a new global character.

11. Add metamorphic pipeline tests

Test invariants rather than only fixed outputs:

• changing dialogue window size must not move text to another panel;
• inserting a scenery panel must not change neighboring identities;
• resuming halfway through identity must match a clean run;
• renumbering panel-local IDs must not change global identity;
• distinct co-present characters must never merge;
• changing narration style must not change facts or quotes;
• processing chapters in a different order must not silently rewrite prior identities.

These tests exercise cross-stage contracts and resumability failures that per-file self-checks cannot catch.

12. Roll out identity changes in shadow mode

Write new identity results to a versioned shadow artifact without affecting scripts or the canonical registry. The review UI can compare:

current:   Choi Haeseon
candidate: unresolved
reason:    co-presence conflict

Promote a new resolver only after it improves the labeled-chapter metrics. Shadow mode prevents tuning work from polluting the production gallery.

The highest-value additions beyond the core plan are reversible identity clusters with cannot-links, dependency-fingerprinted artifacts, and uncertainty-driven review for recurring/speaking/named characters.

Implementation sequence for approval

Phase 1: correctness and scheduling safety — DONE 2026-08-11

Implemented and verified by CPU-only self-checks. Nothing ran on the GPU. What landed, with evidence and the check that covers it, is in decisions/audit-phase1.md.

Two items from this list were NOT implemented and are recorded with a revisit trigger instead: honest stage clearing (caveats/audit-open.md#dishonest-clearing) and ComfyUI under the GPU scheduler (caveats/audit-open.md#comfyui-unscheduled). Both need a design decision, not a patch.

Phase 2: safe throughput improvements

Now tracked in ROADMAP.md, with the done-when condition for each phase.

Verification: unit checks first, followed by one labeled chapter after GPU availability is approved.

Phases 3 and 4

Now tracked in ROADMAP.md.

Acceptance criteria

Now tracked in ROADMAP.md, together with the approval boundary: measure Phase 2 before deciding whether the larger identity and narration changes are worth their size.

Second-pass findings (2026-08-11)

Added after reading the workers and the orchestrator against each other. Every item was read in source and is cited by file and line. Nothing was executed and no GPU work was run.

The first pass audited each side on its own terms. Most of what follows lives in the seam between them: one repo changed a field's meaning and the other still reads the old one. The existing per-file __main__ self-checks cannot catch any of it, because each one asserts its own side of the contract.

P0: the scene stage discards every speaker — CLOSED, decisions/audit-phase1.md#speaker-ref-is-canonical

normalize_dialogue rewrites row["speaker"] to a character_id or None (orchestrator/correctness.py:47). run_stage_dialogue saves that shape into the vision blob (orchestrator/service.py:1159-1160). worker_scene.build_scene still reads that field as a panel-local id and maps it through id_by_local (worker_scene.py:78). A character_id is never a key in that map, so every lookup returns None.

Every scene graph therefore reaches the script worker with speaker: None, and _render_line writes Someone says "..." for all of it (worker_script.py:48). Set-of-mark face attribution, the solo-speaker backstop, and window turn-taking all compute the right answer and then lose it one stage later.

The same field breaks the dialogue resume path: service.py:1180 pushes the stored value into recent as a speaker NAME, so a resumed run feeds raw strings like character_ab12cd34 to the transcription prompt as if they were people.

worker_scene's self-check still passes because it feeds the pre-change contract (worker_scene.py:138). Fix: prefer speaker_ref when its kind is character_id, keep the local-id path as the fallback. This is the strongest argument for the cross-stage metamorphic tests proposed above.

P0: the script verifier rejects correct narration, and its retry is a no-op — CLOSED, decisions/audit-phase1.md#verifier-false-positives

verify_script fails a beat on two rules that fire on valid output (orchestrator/correctness.py:100-116).

1. unsupported-proper-noun flags any capitalized token outside the cast, the entities, the beat's source words, and a 20-word stop list. Ordinary sentence-initial words are not in that list: Suddenly, Behind, Inside, Both, Everyone, After, Two. So is Someone, which the previous finding guarantees the narrator will emit constantly.
2. misquote compares each quoted span against a WHOLE source line with SequenceMatcher at 0.82. The narration prompt explicitly asks for a short quote of the actual words (worker_script.py:130-132). A 15-character quote taken from a 40-character line scores about 0.55 and fails. The prompt and the verifier ask for opposite things.

The failure is not soft. run_stage_script retries once, then raises, so the beat produces no narration (service.py:1384-1386). done never reaches total, _finalize_stage marks the stage failed, and _run_pipeline stops the whole chapter. One false positive halts a run.

The retry is also inert for the reason already recorded above: ScriptInput defines neither beat nor verifier_feedback (worker_script.py:13-23), and pydantic v2 ignores unknown fields by default, so both are dropped without an error.

Fix order: correct the two rules first, then wire the feedback fields. Fixing the plumbing alone makes the model retry against a broken oracle.

P0: correctness flags block TTS permanently in the default configuration — CLOSED, decisions/audit-phase1.md#flag-resolution

run_stage_tts refuses to start while any unresolved flag of kind script-verifier, ambiguous-speaker, conflicting-name-claims, or partial-dialogue exists (service.py:1419-1423). The only code that ever resolves a flag is resolve_correctness_flags, called from /review/approve for the script gate (service.py:1955-1956).

GATES defaults to off (service.py:312). The autonomous pipeline therefore has no path that clears a flag. The first flag of those kinds wedges the chapter until a human calls an endpoint that the autonomous mode never mentions. These flags are not rare: partial-dialogue is raised whenever a window response omits one panel, and dialogue_envelope treats a legitimately silent panel inside a partial window as unresolved (correctness.py:62), so one missing panel flags every wordless panel beside it.

There is a second, worse variant. Flag rows survive their panels. get_correctness_flags LEFT JOINs panels and accepts rows where p.panel_id IS NULL (db.py:573), while resolve_correctness_flags can only resolve flags whose panel still exists (db.py:579). Clearing the crop stage deletes panel rows (db.py:798-799). Any flag raised against a deleted panel becomes visible to every chapter and can never be resolved, blocking TTS for all future jobs.

Fix: resolve by flag identity rather than by surviving panel, and give the autonomous path an explicit policy (auto-resolve below a rank, or fail the stage loudly) instead of an unreachable gate.

P0: awaiting_review is immediately overwritten by failed — CLOSED, decisions/audit-phase1.md#flag-resolution

run_stage_tts sets awaiting_review and then raises (service.py:1422-1423). The raise unwinds into _run_pipeline's catch-all, which sets the job to failed (service.py:391-393). The review state the stage just recorded is gone before anyone can read it, and the operator sees a generic failure rather than a queue of flags. The gate path at service.py:337-341 returns instead of raising and does not have this problem.

P1: set-of-mark face pairing is enabled by default, contrary to its own documentation — CLOSED, decisions/audit-phase1.md#gated-face-pairing

worker_vision.py:22 states the feature is off by default and should be enabled per title once tuned. worker_vision.py:31 reads SOM_ATTRIBUTION with a default of "1". It is on.

That matters because of the unconditional pairing already recorded as a P0. The mispairing does not stay local. _set_of_mark writes the wrong name into the legend the model reads (worker_vision.py:99-104), the model attributes a line to that face label, and _apply_speaker_labels converts the label back into a local_id and stamps speaker_method = "som_face" (worker_vision.py:117). A geometric guess is laundered into the highest-trust provenance value the system has. Gate the pairing before trusting that label, or default the flag to off as documented.

The pairing is also greedy in face order rather than a joint assignment (worker_vision.py:42-52), so the first face processed can claim a character that fits a later face far better.

P1: the JSON repair pass can fabricate content — OPEN, caveats/audit-open.md#repair-fabricates

When a response fails to parse, call_gemma4_json sends the model its own truncated text and asks for the JSON it should have been (worker_vision.py:174-178). The repair call carries no image. On a response truncated by max_tokens, the model is being asked to complete dialogue it can no longer see. Anything it adds is invented and is indistinguishable downstream from transcribed text.

Schema-constrained generation, already recommended above, removes most of this path. Until then the repair pass should re-send the image, or a truncated response should be retried rather than repaired.

P1: /review/preview silently replaces a beat clip — OPEN, caveats/audit-open.md#preview-overwrites-clip

review_preview renders one panel alone and calls save_clip(panel_id, ...) (service.py:1902-1904). Its docstring calls this harmless because assemble would regenerate it. It does not. _render_one_beat returns early when a clip already exists for the leader (service.py:1591-1592). Previewing a beat leader therefore pins the solo preview into the final video and drops the rest of the beat's panels. review_retts gets this right by calling delete_clip first (service.py:2008). Preview should write to a scratch key or delete the clip row afterwards.

P1: the session manager can orphan a llama-server and hold the GPU — CLOSED, decisions/audit-phase1.md#unlocked-model-load

open_session claims the slot, releases the lock, then spawns and health-waits for up to 300 seconds (session_manager.py:108-119). A /session/close arriving during that window finds proc = None, tears down nothing, and clears _active. The process that finishes starting afterwards is unreferenced and keeps its VRAM until someone kills it by hand. The next open spawns a second server on the same port, and _health_wait cannot tell the two apart because it only probes the port (session_manager.py:51-60).

Related asymmetry: _supervise_once calls _start_subprocess while holding _lock (session_manager.py:169-181), so a respawn blocks /session/active, /session/close, and /session/open for the full health wait. The open path was deliberately written to avoid exactly this.

P1: nothing limits concurrent jobs — CLOSED, decisions/audit-phase1.md#no-lease-stealing

_background_jobs accepts any number of pipelines (service.py:302-303, service.py:427-428). Each opens its own model sessions. Combined with the 409 lease stealing already recorded, two jobs terminate each other's models rather than queueing. The queue has to live at the orchestrator as job admission control. A fix inside session_proxy alone still lets two pipelines interleave stages against one GPU.

P1: two cheap throughput wins the plan does not name

Order the shared prefix first so llama-server reuses its KV cache. /direct/window sends the same panel images twice, once for grouping and once for shot design (worker_vision.py:702-733). The audit above frames this as duplicate downloads. The larger cost is the second vision-encoder prefill on the GPU. llama-server reuses the longest common prompt prefix across requests, and the two calls currently differ in their first token because the instruction text precedes the images. Putting the images first and the differing instruction last makes the second pass nearly free on prefill. The same reordering helps any stage that issues several calls over one image set.

Give stored embeddings a version tag. embed_crop uses pooler_output when present and silently falls back to mean-pooled patch tokens otherwise (worker_identity.py:48-52). Those two paths produce different vector spaces, and the fixed 0.85 threshold is only valid for one of them. Nothing recorded beside a stored .npy says which model, revision, or pooling produced it, so a transformers upgrade mixes incompatible vectors into one gallery with no error. Write the model id and pooling mode next to the vector and refuse to compare across versions.

P2: smaller confirmed defects — partly closed, see decisions/audit-phase1.md#related and caveats/audit-open.md

• An out-of-range choice from the resolver is mapped to NONE and then reported as state: "new" (worker_vision.py:898-899), so a hallucinated index mints a brand new character. It should be unresolved, like a parse failure.
• _extract_json matches greedily from the first { to the last } (worker_vision.py:130). Two objects or any trailing braced prose produce an unparseable span and burn a repair call. json.JSONDecoder().raw_decode from the first brace is exact.
• worker_identity._known_cache is invalidated only by _persist_char (worker_identity.py:128-132). The reconcile stage deletes losing characters directly in the orchestrator database (db.py:506), so a long-lived worker keeps shortlisting and assigning ids that no longer exist.
• _pending holds full crop images in worker memory for a whole chapter and is keyed by session (worker_identity.py:25). This is durable per-chapter state inside a worker documented as stateless, it is lost on restart, and a character seen exactly once receives no assignment at all, not even a chapter-local handle.
• get_conn opens a connection per call with no busy_timeout (db.py:190-197). WAL tolerates one writer. PIPELINE=1 already writes clips from concurrent tasks while TTS writes audio, so the planned CPU parallelism will surface as database is locked before it surfaces as throughput.
• Worker endpoints declared async def run blocking MinIO, OpenCV, torch, and ffmpeg calls directly on the event loop (worker_identity.py:177, worker_tts.py:170, worker_vision.py:230). A busy worker cannot answer /health or /unload. That makes /health/workers report a working worker as unreachable (service.py:206-216) and puts the session manager's 30-second /unload at risk exactly when VRAM needs freeing (session_manager.py:93). def instead of async def moves each to the threadpool.
• layers runs after tts in STAGES (db.py:330-333), while run_stage_tts warns that eager rendering under PIPELINE=1 needs layers to run first (service.py:1427-1429). With the flag on, solo beats always render without parallax.
• Stage failure policy is inconsistent. A dropped vision panel fails the stage and halts the pipeline. A failed direction window counts its panels as done (service.py:1247), and layers always finishes completed (service.py:1503). completed does not mean the same thing across stages, which makes the acceptance metrics below hard to read.
• run_stage_assemble does not check that clip_uris is non-empty before assembling and then marks the job completed (service.py:1661-1677).
• The /review/panels timeline sums per-panel audio durations (service.py:1750), but assemble crossfades clips using the per-beat transitions (service.py:1671-1674). Every non-cut transition shortens the real video, so reviewer timestamps drift further out of sync the further into the chapter they scrub.
• MinIO credentials are hardcoded as defaults in committed source (transport.py:95-99, service.py:74-76).

What this changes in the plan — DONE

All five additions landed in Phase 1. See decisions/audit-phase1.md.

The acceptance metric they suggested, the share of narrated lines whose speaker is a named character rather than Someone, is now in ROADMAP.md. It has not been measured yet.