Audit Phase 1: correctness and scheduling safety
Implements every P0 from AUDIT.md plus four P1s, across both halves of the pipeline. Verified by CPU-only self-checks and the orchestrator test suite. No GPU work ran and no pipeline ran. workpc: - worker_scene: read speaker_ref, not the rewritten speaker field. Every line narrated as "Someone" before this. Emit `actions` for the verifier. - worker_script: declare beat + verifier_feedback (pydantic dropped both, so the retry was blind) and render them as a repair prompt. - worker_vision: gate face->identity pairing on containment, assign globally shortest-first, map an out-of-range resolver index to `unresolved` instead of minting a character, parse JSON with raw_decode. - session_manager: tear down a server whose lease vanished mid-load, and spawn the supervisor respawn unlocked. orchestrator (edited in place, NOT committed there): - tracklets: canonicalize gender, add co-presence cannot-links, block transitive bridges across a hard constraint. - correctness: stop failing valid narration on sentence-initial capitals and short quotes; read action evidence from the singular key. - db: stop orphan flags leaking into every chapter; resolve by flag id. - service: TTS returns instead of raising under GATES, auto-resolves under autonomous mode; job admission control; registry names on dialogue resume. - session_proxy: queue on 409 instead of stealing the lease; run heartbeats. Docs restructured per the repo-structure layout: CLAUDE.md is a pointer table, NEXT.md replaces HANDOFF.md, plus ROADMAP.md, JOURNAL.md, decisions/ and caveats/. AUDIT.md now points at those instead of restating them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XD7cAy81MZrc7gCr6aZGWr
This commit is contained in:
@@ -5,6 +5,19 @@ Date: 2026-08-11
|
|||||||
Scope: workpc workers plus the homesrv orchestrator. This audit excludes FFmpeg changes and does not
|
Scope: workpc workers plus the homesrv orchestrator. This audit excludes FFmpeg changes and does not
|
||||||
recommend Magi or its embeddings. No GPU workloads were run during the audit.
|
recommend Magi or its embeddings. No GPU workloads were run during the audit.
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
Phase 1 landed on 2026-08-11. This file is the reasoning, not the state.
|
||||||
|
|
||||||
|
| where the finding went | file |
|
||||||
|
| --- | --- |
|
||||||
|
| implemented, with evidence and a runnable check | `decisions/audit-phase1.md` |
|
||||||
|
| deliberately not done, with a revisit trigger | `caveats/audit-open.md` |
|
||||||
|
| still ahead, in order | `ROADMAP.md` |
|
||||||
|
| what to do next | `NEXT.md` |
|
||||||
|
|
||||||
|
Every P0 below is closed. The sections that name one now point at its decision instead of repeating it.
|
||||||
|
|
||||||
## Outcome
|
## Outcome
|
||||||
|
|
||||||
The strongest performance gains are available without replacing the current models:
|
The strongest performance gains are available without replacing the current models:
|
||||||
@@ -21,18 +34,18 @@ contaminate the permanent character registry.
|
|||||||
|
|
||||||
## Highest-priority findings
|
## Highest-priority findings
|
||||||
|
|
||||||
### P0: tracklet gender gate uses the wrong enum
|
### P0: tracklet gender gate uses the wrong enum — CLOSED, decisions/audit-phase1.md#tracklet-hard-constraints
|
||||||
|
|
||||||
`orchestrator/tracklets.py` checks `male|female`, while vision emits `m|f|unknown`. The tracklet hard
|
`orchestrator/tracklets.py` checks `male|female`, while vision emits `m|f|unknown`. The tracklet hard
|
||||||
gender gate therefore never activates on normal pipeline data.
|
gender gate therefore never activates on normal pipeline data.
|
||||||
|
|
||||||
### P0: tracklets lack co-presence constraints
|
### P0: tracklets lack co-presence constraints — CLOSED, decisions/audit-phase1.md#tracklet-hard-constraints
|
||||||
|
|
||||||
The linker has no same-panel exclusion and uses transitive union-find. Two similar-looking people in the
|
The linker has no same-panel exclusion and uses transitive union-find. Two similar-looking people in the
|
||||||
same panel can be joined, and weak links such as A~B and B~C can merge A and C even when the endpoints are
|
same panel can be joined, and weak links such as A~B and B~C can merge A and C even when the endpoints are
|
||||||
incompatible.
|
incompatible.
|
||||||
|
|
||||||
### P0: face-to-character pairing is unconditional
|
### P0: face-to-character pairing is unconditional — CLOSED, decisions/audit-phase1.md#gated-face-pairing
|
||||||
|
|
||||||
`worker_vision._pair_faces_to_present()` assigns every detected face to the nearest vision character as
|
`worker_vision._pair_faces_to_present()` assigns every detected face to the nearest vision character as
|
||||||
long as an unused character exists. Despite the docstring, there is no distance or overlap threshold.
|
long as an unused character exists. Despite the docstring, there is no distance or overlap threshold.
|
||||||
@@ -44,18 +57,18 @@ The roster is described as hints-only, but those names are fed into detection. A
|
|||||||
causes `worker_identity` to persist it immediately. A coarse appearance-to-roster guess can therefore
|
causes `worker_identity` to persist it immediately. A coarse appearance-to-roster guess can therefore
|
||||||
contaminate a permanent character gallery.
|
contaminate a permanent character gallery.
|
||||||
|
|
||||||
### P0: script repair feedback is discarded
|
### P0: script repair feedback is discarded — CLOSED, decisions/audit-phase1.md#verifier-false-positives
|
||||||
|
|
||||||
The orchestrator sends `beat` and `verifier_feedback` on a failed-script retry, but `worker_script.ScriptInput`
|
The orchestrator sends `beat` and `verifier_feedback` on a failed-script retry, but `worker_script.ScriptInput`
|
||||||
does not define either field and the prompt builder does not consume them. The second call is another
|
does not define either field and the prompt builder does not consume them. The second call is another
|
||||||
stochastic attempt rather than a targeted correction.
|
stochastic attempt rather than a targeted correction.
|
||||||
|
|
||||||
### P0: action evidence is missing from the verifier
|
### P0: action evidence is missing from the verifier — CLOSED, decisions/audit-phase1.md#verifier-false-positives
|
||||||
|
|
||||||
The beat builder reads plural `actions`, while the scene worker emits singular `action`. Script validation
|
The beat builder reads plural `actions`, while the scene worker emits singular `action`. Script validation
|
||||||
therefore receives little or no action evidence and cannot reliably detect invented or omitted actions.
|
therefore receives little or no action evidence and cannot reliably detect invented or omitted actions.
|
||||||
|
|
||||||
### P0: GPU leases are unsafe for concurrent jobs
|
### P0: GPU leases are unsafe for concurrent jobs — CLOSED, decisions/audit-phase1.md#no-lease-stealing
|
||||||
|
|
||||||
When `/session/open` returns 409, the homesrv proxy assumes the active lease is stale and closes it. A second
|
When `/session/open` returns 409, the homesrv proxy assumes the active lease is stale and closes it. A second
|
||||||
legitimate job can terminate the first job's model. `heartbeat_session()` exists but is not used by the
|
legitimate job can terminate the first job's model. `heartbeat_session()` exists but is not used by the
|
||||||
@@ -508,60 +521,29 @@ dependency-fingerprinted artifacts, and uncertainty-driven review for recurring/
|
|||||||
|
|
||||||
## Implementation sequence for approval
|
## Implementation sequence for approval
|
||||||
|
|
||||||
### Phase 1: correctness and scheduling safety
|
### Phase 1: correctness and scheduling safety — DONE 2026-08-11
|
||||||
|
|
||||||
- Fix enum mismatches and add tracklet cannot-links.
|
Implemented and verified by CPU-only self-checks. Nothing ran on the GPU. What landed, with evidence
|
||||||
- Gate face/body pairing.
|
and the check that covers it, is in `decisions/audit-phase1.md`.
|
||||||
- Wire action evidence and verifier feedback.
|
|
||||||
- Make stage clearing/resume behavior honest.
|
|
||||||
- Replace 409 lease stealing with a queue/wait policy.
|
|
||||||
- Run actual lease heartbeats.
|
|
||||||
- Register ComfyUI under the same GPU resource scheduler.
|
|
||||||
|
|
||||||
Verification: CPU-only unit/self-checks. No full pipeline or GPU run until explicitly scheduled.
|
Two items from this list were NOT implemented and are recorded with a revisit trigger instead:
|
||||||
|
honest stage clearing (`caveats/audit-open.md#dishonest-clearing`) and ComfyUI under the GPU scheduler
|
||||||
|
(`caveats/audit-open.md#comfyui-unscheduled`). Both need a design decision, not a patch.
|
||||||
|
|
||||||
### Phase 2: safe throughput improvements
|
### Phase 2: safe throughput improvements
|
||||||
|
|
||||||
- Introduce the two-phase Gemma lease schedule.
|
Now tracked in `ROADMAP.md`, with the done-when condition for each phase.
|
||||||
- Add JSON-schema outputs.
|
|
||||||
- Add local media paths and the ephemeral file/reference cache.
|
|
||||||
- Reuse direction downloads.
|
|
||||||
- Batch SigLIP embeddings.
|
|
||||||
- Batch scene/DB operations.
|
|
||||||
- Add bounded fetch and framed-page crop concurrency.
|
|
||||||
|
|
||||||
Verification: unit checks first, followed by one labeled chapter after GPU availability is approved.
|
Verification: unit checks first, followed by one labeled chapter after GPU availability is approved.
|
||||||
|
|
||||||
### Phase 3: multi-view constrained identity
|
### Phases 3 and 4
|
||||||
|
|
||||||
- Persist chapter-local tracklets.
|
Now tracked in `ROADMAP.md`.
|
||||||
- Add face/body galleries and crop-quality selection.
|
|
||||||
- Resolve with multi-view query evidence and global constraints.
|
|
||||||
- Separate name claims from visual identity.
|
|
||||||
- Expand identity evaluation.
|
|
||||||
|
|
||||||
### Phase 4: evidence-ledger narration
|
|
||||||
|
|
||||||
- Build ordered beat evidence artifacts.
|
|
||||||
- Add scene-scoped story and turn-taking state.
|
|
||||||
- Generate script chunks with evidence IDs.
|
|
||||||
- Add confidence-aware wording and targeted verification/repair.
|
|
||||||
|
|
||||||
## Acceptance criteria
|
## Acceptance criteria
|
||||||
|
|
||||||
Capture a baseline and compare the same labeled chapter after every phase:
|
Now tracked in `ROADMAP.md`, together with the approval boundary: measure Phase 2 before deciding
|
||||||
|
whether the larger identity and narration changes are worth their size.
|
||||||
- stage wall time and model-load time;
|
|
||||||
- number of Gemma calls and repair calls;
|
|
||||||
- MinIO bytes downloaded/uploaded;
|
|
||||||
- prompt-evaluation and generation timings;
|
|
||||||
- tracklet false merges/splits and identity accuracy;
|
|
||||||
- name and speaker accuracy;
|
|
||||||
- unresolved correctness flags;
|
|
||||||
- script repetition, unsupported facts, and verifier retry rate.
|
|
||||||
|
|
||||||
Recommended approval boundary: implement Phases 1 and 2 first, measure them, then decide whether to proceed
|
|
||||||
with the larger identity and narration changes.
|
|
||||||
|
|
||||||
## Second-pass findings (2026-08-11)
|
## Second-pass findings (2026-08-11)
|
||||||
|
|
||||||
@@ -572,7 +554,7 @@ The first pass audited each side on its own terms. Most of what follows lives in
|
|||||||
one repo changed a field's meaning and the other still reads the old one. The existing per-file
|
one repo changed a field's meaning and the other still reads the old one. The existing per-file
|
||||||
`__main__` self-checks cannot catch any of it, because each one asserts its own side of the contract.
|
`__main__` self-checks cannot catch any of it, because each one asserts its own side of the contract.
|
||||||
|
|
||||||
### P0: the scene stage discards every speaker
|
### P0: the scene stage discards every speaker — CLOSED, decisions/audit-phase1.md#speaker-ref-is-canonical
|
||||||
|
|
||||||
`normalize_dialogue` rewrites `row["speaker"]` to a `character_id` or `None`
|
`normalize_dialogue` rewrites `row["speaker"]` to a `character_id` or `None`
|
||||||
(`orchestrator/correctness.py:47`). `run_stage_dialogue` saves that shape into the vision blob
|
(`orchestrator/correctness.py:47`). `run_stage_dialogue` saves that shape into the vision blob
|
||||||
@@ -592,7 +574,7 @@ they were people.
|
|||||||
(`worker_scene.py:138`). Fix: prefer `speaker_ref` when its kind is `character_id`, keep the local-id path as
|
(`worker_scene.py:138`). Fix: prefer `speaker_ref` when its kind is `character_id`, keep the local-id path as
|
||||||
the fallback. This is the strongest argument for the cross-stage metamorphic tests proposed above.
|
the fallback. This is the strongest argument for the cross-stage metamorphic tests proposed above.
|
||||||
|
|
||||||
### P0: the script verifier rejects correct narration, and its retry is a no-op
|
### P0: the script verifier rejects correct narration, and its retry is a no-op — CLOSED, decisions/audit-phase1.md#verifier-false-positives
|
||||||
|
|
||||||
`verify_script` fails a beat on two rules that fire on valid output (`orchestrator/correctness.py:100-116`).
|
`verify_script` fails a beat on two rules that fire on valid output (`orchestrator/correctness.py:100-116`).
|
||||||
|
|
||||||
@@ -616,7 +598,7 @@ are dropped without an error.
|
|||||||
Fix order: correct the two rules first, then wire the feedback fields. Fixing the plumbing alone makes the
|
Fix order: correct the two rules first, then wire the feedback fields. Fixing the plumbing alone makes the
|
||||||
model retry against a broken oracle.
|
model retry against a broken oracle.
|
||||||
|
|
||||||
### P0: correctness flags block TTS permanently in the default configuration
|
### P0: correctness flags block TTS permanently in the default configuration — CLOSED, decisions/audit-phase1.md#flag-resolution
|
||||||
|
|
||||||
`run_stage_tts` refuses to start while any unresolved flag of kind `script-verifier`, `ambiguous-speaker`,
|
`run_stage_tts` refuses to start while any unresolved flag of kind `script-verifier`, `ambiguous-speaker`,
|
||||||
`conflicting-name-claims`, or `partial-dialogue` exists (`service.py:1419-1423`). The only code that ever
|
`conflicting-name-claims`, or `partial-dialogue` exists (`service.py:1419-1423`). The only code that ever
|
||||||
@@ -638,7 +620,7 @@ resolved, blocking TTS for all future jobs.
|
|||||||
Fix: resolve by flag identity rather than by surviving panel, and give the autonomous path an explicit
|
Fix: resolve by flag identity rather than by surviving panel, and give the autonomous path an explicit
|
||||||
policy (auto-resolve below a rank, or fail the stage loudly) instead of an unreachable gate.
|
policy (auto-resolve below a rank, or fail the stage loudly) instead of an unreachable gate.
|
||||||
|
|
||||||
### P0: `awaiting_review` is immediately overwritten by `failed`
|
### P0: `awaiting_review` is immediately overwritten by `failed` — CLOSED, decisions/audit-phase1.md#flag-resolution
|
||||||
|
|
||||||
`run_stage_tts` sets `awaiting_review` and then raises (`service.py:1422-1423`). The raise unwinds into
|
`run_stage_tts` sets `awaiting_review` and then raises (`service.py:1422-1423`). The raise unwinds into
|
||||||
`_run_pipeline`'s catch-all, which sets the job to `failed` (`service.py:391-393`). The review state the
|
`_run_pipeline`'s catch-all, which sets the job to `failed` (`service.py:391-393`). The review state the
|
||||||
@@ -646,7 +628,7 @@ stage just recorded is gone before anyone can read it, and the operator sees a g
|
|||||||
queue of flags. The gate path at `service.py:337-341` returns instead of raising and does not have this
|
queue of flags. The gate path at `service.py:337-341` returns instead of raising and does not have this
|
||||||
problem.
|
problem.
|
||||||
|
|
||||||
### P1: set-of-mark face pairing is enabled by default, contrary to its own documentation
|
### P1: set-of-mark face pairing is enabled by default, contrary to its own documentation — CLOSED, decisions/audit-phase1.md#gated-face-pairing
|
||||||
|
|
||||||
`worker_vision.py:22` states the feature is off by default and should be enabled per title once tuned.
|
`worker_vision.py:22` states the feature is off by default and should be enabled per title once tuned.
|
||||||
`worker_vision.py:31` reads `SOM_ATTRIBUTION` with a default of `"1"`. It is on.
|
`worker_vision.py:31` reads `SOM_ATTRIBUTION` with a default of `"1"`. It is on.
|
||||||
@@ -661,7 +643,7 @@ default the flag to off as documented.
|
|||||||
The pairing is also greedy in face order rather than a joint assignment (`worker_vision.py:42-52`), so the
|
The pairing is also greedy in face order rather than a joint assignment (`worker_vision.py:42-52`), so the
|
||||||
first face processed can claim a character that fits a later face far better.
|
first face processed can claim a character that fits a later face far better.
|
||||||
|
|
||||||
### P1: the JSON repair pass can fabricate content
|
### P1: the JSON repair pass can fabricate content — OPEN, caveats/audit-open.md#repair-fabricates
|
||||||
|
|
||||||
When a response fails to parse, `call_gemma4_json` sends the model its own truncated text and asks for the
|
When a response fails to parse, `call_gemma4_json` sends the model its own truncated text and asks for the
|
||||||
JSON it should have been (`worker_vision.py:174-178`). The repair call carries no image. On a response
|
JSON it should have been (`worker_vision.py:174-178`). The repair call carries no image. On a response
|
||||||
@@ -671,7 +653,7 @@ adds is invented and is indistinguishable downstream from transcribed text.
|
|||||||
Schema-constrained generation, already recommended above, removes most of this path. Until then the repair
|
Schema-constrained generation, already recommended above, removes most of this path. Until then the repair
|
||||||
pass should re-send the image, or a truncated response should be retried rather than repaired.
|
pass should re-send the image, or a truncated response should be retried rather than repaired.
|
||||||
|
|
||||||
### P1: `/review/preview` silently replaces a beat clip
|
### P1: `/review/preview` silently replaces a beat clip — OPEN, caveats/audit-open.md#preview-overwrites-clip
|
||||||
|
|
||||||
`review_preview` renders one panel alone and calls `save_clip(panel_id, ...)` (`service.py:1902-1904`). Its
|
`review_preview` renders one panel alone and calls `save_clip(panel_id, ...)` (`service.py:1902-1904`). Its
|
||||||
docstring calls this harmless because assemble would regenerate it. It does not. `_render_one_beat` returns
|
docstring calls this harmless because assemble would regenerate it. It does not. `_render_one_beat` returns
|
||||||
@@ -680,7 +662,7 @@ pins the solo preview into the final video and drops the rest of the beat's pane
|
|||||||
right by calling `delete_clip` first (`service.py:2008`). Preview should write to a scratch key or delete the
|
right by calling `delete_clip` first (`service.py:2008`). Preview should write to a scratch key or delete the
|
||||||
clip row afterwards.
|
clip row afterwards.
|
||||||
|
|
||||||
### P1: the session manager can orphan a llama-server and hold the GPU
|
### P1: the session manager can orphan a llama-server and hold the GPU — CLOSED, decisions/audit-phase1.md#unlocked-model-load
|
||||||
|
|
||||||
`open_session` claims the slot, releases the lock, then spawns and health-waits for up to 300 seconds
|
`open_session` claims the slot, releases the lock, then spawns and health-waits for up to 300 seconds
|
||||||
(`session_manager.py:108-119`). A `/session/close` arriving during that window finds `proc = None`, tears
|
(`session_manager.py:108-119`). A `/session/close` arriving during that window finds `proc = None`, tears
|
||||||
@@ -693,7 +675,7 @@ Related asymmetry: `_supervise_once` calls `_start_subprocess` while holding `_l
|
|||||||
(`session_manager.py:169-181`), so a respawn blocks `/session/active`, `/session/close`, and `/session/open`
|
(`session_manager.py:169-181`), so a respawn blocks `/session/active`, `/session/close`, and `/session/open`
|
||||||
for the full health wait. The open path was deliberately written to avoid exactly this.
|
for the full health wait. The open path was deliberately written to avoid exactly this.
|
||||||
|
|
||||||
### P1: nothing limits concurrent jobs
|
### P1: nothing limits concurrent jobs — CLOSED, decisions/audit-phase1.md#no-lease-stealing
|
||||||
|
|
||||||
`_background_jobs` accepts any number of pipelines (`service.py:302-303`, `service.py:427-428`). Each opens
|
`_background_jobs` accepts any number of pipelines (`service.py:302-303`, `service.py:427-428`). Each opens
|
||||||
its own model sessions. Combined with the 409 lease stealing already recorded, two jobs terminate each
|
its own model sessions. Combined with the 409 lease stealing already recorded, two jobs terminate each
|
||||||
@@ -717,7 +699,7 @@ vector spaces, and the fixed 0.85 threshold is only valid for one of them. Nothi
|
|||||||
vectors into one gallery with no error. Write the model id and pooling mode next to the vector and refuse to
|
vectors into one gallery with no error. Write the model id and pooling mode next to the vector and refuse to
|
||||||
compare across versions.
|
compare across versions.
|
||||||
|
|
||||||
### P2: smaller confirmed defects
|
### P2: smaller confirmed defects — partly closed, see `decisions/audit-phase1.md#related` and `caveats/audit-open.md`
|
||||||
|
|
||||||
- An out-of-range `choice` from the resolver is mapped to NONE and then reported as `state: "new"`
|
- An out-of-range `choice` from the resolver is mapped to NONE and then reported as `state: "new"`
|
||||||
(`worker_vision.py:898-899`), so a hallucinated index mints a brand new character. It should be
|
(`worker_vision.py:898-899`), so a hallucinated index mints a brand new character. It should be
|
||||||
@@ -755,16 +737,9 @@ compare across versions.
|
|||||||
- MinIO credentials are hardcoded as defaults in committed source (`transport.py:95-99`,
|
- MinIO credentials are hardcoded as defaults in committed source (`transport.py:95-99`,
|
||||||
`service.py:74-76`).
|
`service.py:74-76`).
|
||||||
|
|
||||||
### What this changes in the plan
|
### What this changes in the plan — DONE
|
||||||
|
|
||||||
Add to Phase 1, before any throughput work:
|
All five additions landed in Phase 1. See `decisions/audit-phase1.md`.
|
||||||
|
|
||||||
1. Restore the speaker field across the dialogue, scene, and script boundary.
|
The acceptance metric they suggested, the share of narrated lines whose speaker is a named character
|
||||||
2. Correct the two verifier rules, then wire `beat` and `verifier_feedback`.
|
rather than `Someone`, is now in `ROADMAP.md`. It has not been measured yet.
|
||||||
3. Give correctness flags a resolution path that does not require a disabled gate, and stop the TTS block
|
|
||||||
from erasing `awaiting_review`.
|
|
||||||
4. Decide the set-of-mark default deliberately, and gate face pairing before the label is trusted.
|
|
||||||
5. Add job admission control at the orchestrator, not only lease queueing in the proxy.
|
|
||||||
|
|
||||||
Add to the acceptance criteria: the share of narrated lines whose speaker is a named character rather than
|
|
||||||
`Someone`. That single number would have caught the first finding on the day it landed.
|
|
||||||
|
|||||||
@@ -1,67 +1,72 @@
|
|||||||
# CLAUDE.md
|
# CLAUDE.md
|
||||||
|
|
||||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
Goal, invariants, and working rules. Read this first.
|
||||||
|
|
||||||
## What this is
|
| file | holds |
|
||||||
|
| --- | --- |
|
||||||
|
| `NEXT.md` | the current state and the live plan |
|
||||||
|
| `ROADMAP.md` | the ordered outcomes past the current one |
|
||||||
|
| `JOURNAL.md` | what was run and when, append-only |
|
||||||
|
| `decisions/` | every settled question, indexed in `decisions/CLAUDE.md` |
|
||||||
|
| `caveats/` | every known limit and its revisit trigger, indexed in `caveats/CLAUDE.md` |
|
||||||
|
| `AGENTS.md` | commands, with the traps beside them |
|
||||||
|
| `AUDIT.md` | the 2026-08-11 pipeline audit, the source of the roadmap |
|
||||||
|
| `spec-v3.md` | current quality and look work, marked DONE/TODO per item |
|
||||||
|
|
||||||
The **workpc compute half** of a manga→narrated-video pipeline. This repo holds stateless
|
Do not restate a finding here. Point at the decision.
|
||||||
GPU/CPU workers only. State, job scheduling, and stage orchestration live in a **separate
|
|
||||||
homesrv orchestrator repo** (`/mnt/server/home/kami/apps/Maven/` region) — not here. The two
|
|
||||||
communicate over a fixed HTTP contract; never add sqlite or durable state to a worker.
|
|
||||||
|
|
||||||
Machine split (memorize): workers run on **workpc** (RX 7900 GRE, ROCm). MinIO + orchestrator
|
## The goal
|
||||||
run on **homesrv** (`192.168.1.104`, CPU-only). Data dir `/mnt/server/home/kami/` is an SSHFS
|
|
||||||
mount of homesrv.
|
|
||||||
|
|
||||||
## Run / test
|
This is the **workpc compute half** of a manga to narrated-video pipeline. It holds stateless GPU and
|
||||||
|
CPU workers only. State, job scheduling, and stage orchestration live in a separate homesrv
|
||||||
|
orchestrator repo (`/mnt/server/home/kami/docker-apps/manga-infra/orchestrator/`). The two talk over a
|
||||||
|
fixed HTTP contract. The hard part is that one GPU serves every model, so the schedule, not the model,
|
||||||
|
sets the wall time.
|
||||||
|
|
||||||
|
Machine split: workers run on **workpc** (RX 7900 GRE, ROCm). MinIO and the orchestrator run on
|
||||||
|
**homesrv** (`192.168.1.104`, CPU-only). `/mnt/server/home/kami/` is an SSHFS mount of homesrv.
|
||||||
|
|
||||||
|
## Invariants
|
||||||
|
|
||||||
|
1. **No durable state in a worker.** No sqlite, no cross-request memory. A worker pulls inputs from
|
||||||
|
MinIO by URI, does one stage, pushes outputs back, returns URIs. A `/dev/shm` cache is allowed
|
||||||
|
because it may be dropped at any time.
|
||||||
|
2. **One warm model at a time.** Every GPU stage takes a lease from `session_manager.py` on 8095. A 409
|
||||||
|
is a queue signal, never a stale lease
|
||||||
|
(`decisions/audit-phase1.md#no-lease-stealing`).
|
||||||
|
3. **Never hold `_lock` across a model load or a health wait** in `session_manager.py`
|
||||||
|
(`decisions/audit-phase1.md#unlocked-model-load`).
|
||||||
|
4. **A dialogue row's speaker is `speaker_ref`.** The flat `speaker` field is a compatibility value and
|
||||||
|
holds a `character_id`, not a panel-local id
|
||||||
|
(`decisions/audit-phase1.md#speaker-ref-is-canonical`).
|
||||||
|
5. **A stage never raises after setting `awaiting_review`.** The pipeline's catch-all overwrites it
|
||||||
|
with `failed` (`decisions/audit-phase1.md#flag-resolution`).
|
||||||
|
6. **Never mint a character from an unparseable or out-of-range model answer.** That is `unresolved`
|
||||||
|
(`decisions/audit-phase1.md#hallucinated-index`).
|
||||||
|
7. **The HTTP contract with the orchestrator is load-bearing.** Changing a worker's request or response
|
||||||
|
shape means reconciling the orchestrator in the same session. Neither repo's self-checks can catch a
|
||||||
|
contract break, because each asserts its own side.
|
||||||
|
8. **`ponytail:` comments mark deliberate simplifications** and name the upgrade path. Respect them.
|
||||||
|
|
||||||
|
## Working rules
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
./start_workers.sh # dev: session_manager + 9 workers, each a uvicorn in a tmux window
|
./start_workers.sh # dev: session_manager + 9 workers, each a uvicorn in a tmux window
|
||||||
tmux attach -t manga-workers # watch logs; per-worker window
|
tmux attach -t manga-workers # per-worker logs
|
||||||
sudo systemd/install.sh # production: one systemd unit per process (User=kami)
|
sudo systemd/install.sh # production: one systemd unit per process (User=kami)
|
||||||
|
|
||||||
python session_manager.py # each module has a __main__ self-check (assert-based, no framework)
|
.venv/bin/python worker_scene.py # every module has an assert-based __main__ self-check
|
||||||
python transport.py # run these to verify a file after editing it
|
.venv/bin/python test_vision_parse.py
|
||||||
python test_vision_parse.py # the one standalone pytest-free test
|
|
||||||
|
cd /mnt/server/home/kami/docker-apps/manga-infra/orchestrator && pytest -q --ignore=test_api.py
|
||||||
```
|
```
|
||||||
|
|
||||||
There is no lint/build step. `.venv` is the ROCm torch env; workers import `transport` by module name.
|
There is no lint or build step. `.venv` is the ROCm torch env. Workers import `transport` by module
|
||||||
|
name. Ports: crop 8000, vision 8002, identity 8003, scene 8004, script 8005, tts 8006, layers 8007,
|
||||||
|
render 8008, session_manager 8095.
|
||||||
|
|
||||||
## Architecture
|
- Any non-trivial logic gets ONE runnable check in `__main__`, assert-based, no framework. Run the file
|
||||||
|
to verify it.
|
||||||
**Workers are stateless HTTP stages.** Each `worker_*.py` is a FastAPI app on a fixed port. It
|
- Editing a worker's request or response shape means editing the orchestrator too, in the same session.
|
||||||
pulls inputs from MinIO by URI to local disk (`/dev/shm`), does one stage, pushes outputs back,
|
- Update `NEXT.md` alongside any change that moves the plan, and append to `JOURNAL.md` after a run.
|
||||||
returns URIs. No cross-request memory. Ports: crop 8000, vision 8002, identity 8003, scene 8004,
|
- Do not run GPU work or a full pipeline without asking.
|
||||||
script 8005, tts 8006, layers 8007, render 8008, **session_manager 8095**.
|
|
||||||
|
|
||||||
**`transport.py`** — shared MinIO client (`get`/`put`/`put_bytes`/`exists`) + `install_logging(app, name)`
|
|
||||||
(one log line per request with panel id). URIs are `s3://bucket/key`. Import it in every worker.
|
|
||||||
|
|
||||||
**`session_manager.py`** — the **GPU mutex**. Only one warm model at a time on the single local GPU.
|
|
||||||
Two model kinds:
|
|
||||||
- *subprocess* (`gemma4`): it spawns/health-waits/terminates `llama-server`, and a supervisor
|
|
||||||
respawns it in-place if it crashes mid-session (keeps the same session_id + port).
|
|
||||||
- *in-process* (`siglip2`, `dots`): returns `port=None`; the **worker** loads the transformers model
|
|
||||||
itself and must expose `/unload` so `/session/close` can free the ~5GB VRAM (the mutex alone
|
|
||||||
can't reclaim it). Leases have a TTL + heartbeat; a reaper force-closes stale ones.
|
|
||||||
|
|
||||||
A GPU worker's flow: `/session/open {model}` → (409 if busy) → do work against the returned port or
|
|
||||||
its own resident model → `/session/close`. CPU workers (crop) take no session.
|
|
||||||
|
|
||||||
## Conventions
|
|
||||||
|
|
||||||
- **`ponytail:` comments** mark deliberate simplifications and name the upgrade path — respect them,
|
|
||||||
don't "fix" them without reason.
|
|
||||||
- Any non-trivial logic gets ONE runnable check in `__main__` (assert-based `demo`/self-check), not a
|
|
||||||
test suite. Follow that pattern; run the file to verify.
|
|
||||||
- The HTTP contract with the orchestrator is load-bearing and shared across repos — changing a
|
|
||||||
worker's request/response shape means reconciling the orchestrator too (see commit history:
|
|
||||||
"reconcile worker contracts").
|
|
||||||
|
|
||||||
## Specs & docs
|
|
||||||
|
|
||||||
- `spec-v3.md` — current quality/look work (narration voice, panel curation, render overhaul), marked
|
|
||||||
DONE/TODO per item. `legacy/` holds the old single-repo workers.
|
|
||||||
- `AGENTS.md` — use `scripts/analyze_video_frames.sh` (not manual seeking) to diagnose render output;
|
|
||||||
its frames are `/tmp` artifacts, never commit them.
|
|
||||||
- `collage.py` (behind `COLLAGE` flag) and the render worker carry the animated-layout planners.
|
|
||||||
|
|||||||
-54
@@ -1,54 +0,0 @@
|
|||||||
# HANDOFF: audit second pass (2026-08-11)
|
|
||||||
|
|
||||||
## What was asked
|
|
||||||
Review `AUDIT.md` (untracked, first pass, 565 lines) and add anything missing.
|
|
||||||
|
|
||||||
## What was done
|
|
||||||
Read both halves of the pipeline and checked the first pass against source. No code changed.
|
|
||||||
No GPU work, no pipeline run, no tests executed.
|
|
||||||
|
|
||||||
Files read in full:
|
|
||||||
- workpc: `worker_vision.py` (1008), `worker_identity.py` (322), `worker_script.py` (300),
|
|
||||||
`worker_scene.py` (170), `worker_tts.py` (241), `session_manager.py` (238), `transport.py` (216),
|
|
||||||
`worker_crop.py` (first 120 of 359).
|
|
||||||
- homesrv `/mnt/server/home/kami/docker-apps/manga-infra/orchestrator/`: `service.py` (2048),
|
|
||||||
`db.py` (815), `session_proxy.py` (302), `correctness.py` (116), `tracklets.py` (121),
|
|
||||||
`minio_layout.py` (105), `grouping.py` (75).
|
|
||||||
|
|
||||||
Result: `AUDIT.md` grew from 565 to 745 lines. New section `## Second-pass findings (2026-08-11)`
|
|
||||||
at line 566. 4 new P0, 6 new P1, 13 P2, plus 5 additions to the Phase 1 list and 1 to the
|
|
||||||
acceptance criteria.
|
|
||||||
|
|
||||||
Four first-pass claims were confirmed in source and are not restated in the new section:
|
|
||||||
|
|
||||||
- tracklet gender enum: `tracklets.py:43` uses `male|female`, vision emits `m|f|unknown`.
|
|
||||||
- missing action evidence: `worker_scene.py:85` emits `action`, `correctness.py:91` reads `actions`.
|
|
||||||
- dropped verifier feedback: `worker_script.py:13-23` defines neither field.
|
|
||||||
- 409 lease stealing: `session_proxy.py:40-49`.
|
|
||||||
|
|
||||||
## Top 4 new findings
|
|
||||||
1. `worker_scene.py:78` reads `speaker` as a local id, but `correctness.py:47` already rewrote it
|
|
||||||
to a character_id. Every lookup returns None, so all narration says `Someone`.
|
|
||||||
2. `correctness.py:106-115` fails valid narration on sentence-initial capitals and on short
|
|
||||||
quotes, and `service.py:1384-1386` turns that into a halted chapter.
|
|
||||||
3. `service.py:1419-1423` blocks TTS on correctness flags that only `/review/approve` can clear,
|
|
||||||
while `GATES` defaults to off (`service.py:312`).
|
|
||||||
4. `session_manager.py:108-119` can orphan a llama-server that keeps its VRAM.
|
|
||||||
|
|
||||||
## Still open
|
|
||||||
- Nothing from this session is half-finished. `AUDIT.md` is complete as written.
|
|
||||||
- No finding has been fixed. All are report-only.
|
|
||||||
- `AUDIT.md` is still untracked as of this session's start. Decide whether it belongs in git.
|
|
||||||
- Prose linter reports ~97 style hits in `AUDIT.md`. Almost all are in the first-pass sections
|
|
||||||
(lines 1-565), which were left as the author wrote them.
|
|
||||||
- Carried over from the earlier reconstruction handoff of the same day:
|
|
||||||
- `.venv/` is gone. Rebuild it, then install the ROCm torch wheel the way workpc had it.
|
|
||||||
- `dots.tts/` and `legacy/` are not recoverable from transcripts.
|
|
||||||
- `RESUME_SPEC.md`, `pipeline-design-notes.md`, and `spec-v2.md` are unrecoverable.
|
|
||||||
|
|
||||||
## Next command
|
|
||||||
```
|
|
||||||
cd /home/kami/Programs/n8n-worker
|
|
||||||
sed -n '566,745p' AUDIT.md # read the new section
|
|
||||||
```
|
|
||||||
Then pick the Phase 1 order at `AUDIT.md:511` as amended at the end of the new section.
|
|
||||||
+22
@@ -0,0 +1,22 @@
|
|||||||
|
# JOURNAL
|
||||||
|
|
||||||
|
Append-only, newest last. One block per session or run. Not a changelog: this records what happened on
|
||||||
|
the day a number was produced, so a later postmortem can find it.
|
||||||
|
|
||||||
|
## 2026-08-11 Audit second pass [no task]
|
||||||
|
|
||||||
|
Command: none. Source reading only.
|
||||||
|
Outcome: finished. `AUDIT.md` grew from 565 to 771 lines with a `## Second-pass findings` section:
|
||||||
|
4 new P0, 6 new P1, 13 P2, 5 additions to the Phase 1 list.
|
||||||
|
Produced: commit `6d9df5b`, `AUDIT.md:566`.
|
||||||
|
|
||||||
|
## 2026-08-11 Audit Phase 1 implemented [#203]
|
||||||
|
|
||||||
|
Command: `python worker_scene.py worker_script.py worker_vision.py session_manager.py`,
|
||||||
|
`pytest -q --ignore=test_api.py` in the orchestrator.
|
||||||
|
Outcome: finished. All self-checks pass, 108 orchestrator tests pass. No GPU work, no pipeline run.
|
||||||
|
`test_api.py` was skipped because fastapi is not installed in the workpc venv.
|
||||||
|
Produced: `decisions/audit-phase1.md`, `caveats/audit-open.md`, `ROADMAP.md`, and this scaffold.
|
||||||
|
|
||||||
|
One existing test asserted the bug: `test_name_binding.test_conflict_flags_and_stays_unnamed` relied on
|
||||||
|
orphan flags leaking into every chapter, because it never created panel rows. It now creates them.
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
# NEXT
|
||||||
|
|
||||||
|
Updated 2026-08-11. Replaces the old `HANDOFF.md`.
|
||||||
|
|
||||||
|
## State
|
||||||
|
|
||||||
|
Audit Phase 1 is implemented and green. Nothing is half-finished.
|
||||||
|
|
||||||
|
Changed on workpc: `worker_scene.py`, `worker_script.py`, `worker_vision.py`, `session_manager.py`.
|
||||||
|
Changed on homesrv (`/mnt/server/home/kami/docker-apps/manga-infra/orchestrator/`): `tracklets.py`,
|
||||||
|
`correctness.py`, `db.py`, `service.py`, `session_proxy.py`, `test_script_verify.py`,
|
||||||
|
`test_name_binding.py`.
|
||||||
|
|
||||||
|
What landed and why: `decisions/audit-phase1.md`. What was left open: `caveats/audit-open.md`.
|
||||||
|
|
||||||
|
Verification: CPU-only self-checks and the orchestrator test suite. 108 orchestrator tests pass
|
||||||
|
(`test_api.py` is excluded on workpc because fastapi is not installed in this venv). No GPU work ran
|
||||||
|
and no pipeline ran, so none of this is confirmed against a real chapter.
|
||||||
|
|
||||||
|
The orchestrator changes are edited in place on the SSHFS mount and are NOT committed. Its git root is
|
||||||
|
`/mnt/server/home/kami/docker-apps`. Its container also needs a rebuild or restart to pick them up.
|
||||||
|
|
||||||
|
## Next
|
||||||
|
|
||||||
|
1. Commit the orchestrator half in `/mnt/server/home/kami/docker-apps` and restart the container.
|
||||||
|
2. Run one labeled chapter end to end and record the baseline numbers from `ROADMAP.md`, especially the
|
||||||
|
share of narrated lines with a named speaker. That number is the check on the largest Phase 1 fix.
|
||||||
|
3. Start Phase 2 from `ROADMAP.md`. Set SQLite `busy_timeout` before any concurrency work
|
||||||
|
(`caveats/audit-open.md#sqlite-locking`).
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
Four Phase 1 items have no Vikunja task and were not created, because writing to the tracker was not
|
||||||
|
asked for: the speaker contract fix, the verifier rules, the tracklet constraints, and the flag
|
||||||
|
resolution path. Only [#203] existed and is now closed by `decisions/audit-phase1.md#unlocked-model-load`.
|
||||||
|
|
||||||
|
Three audit items are deliberately not done and are recorded as caveats rather than silently dropped:
|
||||||
|
honest stage clearing, ComfyUI under the session mutex, and reversible identity merges. Each needs a
|
||||||
|
design decision, not a patch.
|
||||||
|
|
||||||
|
Carried over from the reconstruction: `.venv` needs the ROCm torch wheel reinstalled, and `dots.tts/`,
|
||||||
|
`legacy/`, `RESUME_SPEC.md`, `pipeline-design-notes.md`, `spec-v2.md` are unrecoverable.
|
||||||
+66
@@ -0,0 +1,66 @@
|
|||||||
|
# ROADMAP
|
||||||
|
|
||||||
|
Ordered outcomes past the current one. Phase 1 landed on 2026-08-11 and moved to
|
||||||
|
`decisions/audit-phase1.md`. The reasoning behind every phase is in `AUDIT.md`.
|
||||||
|
|
||||||
|
Approval boundary from the audit: ship Phase 2 and measure it. Then decide whether Phase 3 and 4 are
|
||||||
|
worth the size of the change.
|
||||||
|
|
||||||
|
## Phase 2: safe throughput
|
||||||
|
|
||||||
|
Unblocks: a chapter that finishes in a fraction of the current wall time without changing any model.
|
||||||
|
Done when: the same labeled chapter runs with two gemma loads instead of seven. The acceptance numbers
|
||||||
|
below are recorded in a report.
|
||||||
|
|
||||||
|
- Two-phase gemma lease schedule, so the model stays resident across compatible stages.
|
||||||
|
- JSON-schema-constrained generation for detect, dialogue, roster, direction, same-person, and resolve.
|
||||||
|
- `--media-path /dev/shm` and `file://` image paths instead of base64 data URIs.
|
||||||
|
- An ephemeral `/dev/shm` cache keyed by S3 URI plus etag, and reused direction downloads.
|
||||||
|
- Batched siglip embeddings, one forward pass per bounded image batch.
|
||||||
|
- Batched scene construction and SQLite writes. Set `busy_timeout` first
|
||||||
|
(`caveats/audit-open.md#sqlite-locking`).
|
||||||
|
- Bounded fetch and framed-page crop concurrency.
|
||||||
|
- Images before the differing instruction in every multi-call prompt, so llama-server reuses its KV
|
||||||
|
cache prefix across the two direction passes.
|
||||||
|
|
||||||
|
## Phase 3: multi-view constrained identity
|
||||||
|
|
||||||
|
Unblocks: an identity that survives a reviewer disagreeing with it.
|
||||||
|
Done when: false merges and false splits are reported per labeled chapter and the resolver beats the
|
||||||
|
Phase 1 baseline on both.
|
||||||
|
|
||||||
|
- Chapter-local tracklets persisted as a resumable homesrv artifact, not worker memory.
|
||||||
|
- Face and body galleries with crop-quality selection, and 2-4 query views per tracklet.
|
||||||
|
- Global constraints at resolution: must-link inside a tracklet, cannot-link for co-presence, gender
|
||||||
|
and species gates, manual locks, one appearance per character per panel.
|
||||||
|
- Name claims separated from visual identity. Roster names stay candidate vocabulary.
|
||||||
|
- Reversible merges: a cluster with `canonical_id`, `member_ids`, and `cannot_link`, replacing the
|
||||||
|
destructive delete (`caveats/audit-open.md#destructive-reconcile`).
|
||||||
|
- Identity evaluation extended to purity, fragmentation, name binding, and speaker accuracy.
|
||||||
|
|
||||||
|
## Phase 4: evidence-ledger narration
|
||||||
|
|
||||||
|
Unblocks: narration that can be checked against the panel rather than trusted.
|
||||||
|
Done when: every beat carries an evidence artifact and the verifier reports quote grounding, action
|
||||||
|
coverage, and future-fact leakage against it.
|
||||||
|
|
||||||
|
- Ordered beat evidence: panels, actions, chronology, dialogue ids, typed speaker refs, uncertainty.
|
||||||
|
- Scene-scoped story and turn-taking state, reset on location and time changes.
|
||||||
|
- Script chunks of 6-12 consecutive beats per text-only call, returning evidence ids.
|
||||||
|
- Confidence-aware wording: a weak speaker becomes `someone`, not a confident name.
|
||||||
|
- Targeted per-beat repair driven by the exact failures.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
Capture a baseline and compare the same labeled chapter after every phase.
|
||||||
|
|
||||||
|
- Stage wall time and model-load time.
|
||||||
|
- Gemma call count and repair call count.
|
||||||
|
- MinIO bytes moved, prompt-evaluation and generation timings.
|
||||||
|
- Tracklet false merges and splits, identity accuracy.
|
||||||
|
- Name and speaker accuracy, unresolved correctness flags.
|
||||||
|
- Script repetition, unsupported facts, verifier retry rate.
|
||||||
|
- The share of narrated lines whose speaker is a named character rather than `Someone`. That one
|
||||||
|
number would have caught `decisions/audit-phase1.md#speaker-ref-is-canonical` the day it landed.
|
||||||
|
|
||||||
|
Read `caveats/audit-open.md#inconsistent-stage-policy` before trusting any per-stage count.
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
# caveats/
|
||||||
|
|
||||||
|
One entry per known breakage, deferral, or trap. An entry names what fails, who pays for it, and the
|
||||||
|
concrete trigger that makes fixing it worth the time.
|
||||||
|
|
||||||
|
A caveat that gets resolved moves to `decisions/` or gets deleted. A caveat with no revisit trigger is
|
||||||
|
a complaint, so give it one or drop it.
|
||||||
|
|
||||||
|
## Rules for this directory
|
||||||
|
|
||||||
|
* One file per source, one `##` section per limit.
|
||||||
|
* State the limit as what fails, not as a topic. "Reconcile deletes the losing character" beats
|
||||||
|
"Reconcile".
|
||||||
|
* Delete an entry the moment its trigger fires and the work lands.
|
||||||
|
|
||||||
|
## Index
|
||||||
|
|
||||||
|
| caveat | source |
|
||||||
|
| --- | --- |
|
||||||
|
| [Reconcile deletes the losing character irreversibly](audit-open.md#destructive-reconcile) | AUDIT.md |
|
||||||
|
| [Clearing a stage does not undo what it wrote](audit-open.md#dishonest-clearing) | AUDIT.md |
|
||||||
|
| [ComfyUI uses the GPU outside the session mutex](audit-open.md#comfyui-unscheduled) | AUDIT.md |
|
||||||
|
| [The JSON repair pass can fabricate dialogue](audit-open.md#repair-fabricates) | AUDIT.md |
|
||||||
|
| [`/review/preview` pins a solo clip into the final video](audit-open.md#preview-overwrites-clip) | AUDIT.md |
|
||||||
|
| [Stored embeddings carry no model or pooling version](audit-open.md#untagged-embeddings) | AUDIT.md |
|
||||||
|
| [Worker endpoints block the event loop](audit-open.md#blocking-event-loop) | AUDIT.md |
|
||||||
|
| [SQLite has no busy timeout, so parallelism will surface as lock errors](audit-open.md#sqlite-locking) | AUDIT.md |
|
||||||
|
| [`layers` runs after `tts`, so pipelined solo beats lose parallax](audit-open.md#layers-after-tts) | AUDIT.md |
|
||||||
|
| [`completed` means something different in each stage](audit-open.md#inconsistent-stage-policy) | AUDIT.md |
|
||||||
|
| [Identity worker caches characters the orchestrator has deleted](audit-open.md#stale-known-cache) | AUDIT.md |
|
||||||
|
| [A character seen once gets no assignment at all](audit-open.md#pending-in-worker-memory) | AUDIT.md |
|
||||||
|
| [MinIO credentials are hardcoded in committed source](audit-open.md#hardcoded-credentials) | AUDIT.md |
|
||||||
|
| [Assemble marks a job completed with no clips](audit-open.md#empty-assemble) | AUDIT.md |
|
||||||
|
| [Reviewer timestamps drift against the crossfaded video](audit-open.md#timeline-drift) | AUDIT.md |
|
||||||
@@ -0,0 +1,149 @@
|
|||||||
|
# Open limits from the 2026-08-11 audit
|
||||||
|
|
||||||
|
Everything here was read in source during the audit and deliberately left unfixed in Phase 1. The
|
||||||
|
fixed findings live in `decisions/audit-phase1.md`. Line numbers are from the audit and may drift.
|
||||||
|
|
||||||
|
## Reconcile deletes the losing character irreversibly {#destructive-reconcile}
|
||||||
|
|
||||||
|
Reconciliation deletes the losing character row (`db.py:506`). Clearing the reconcile stage does not
|
||||||
|
undo it, and name claims attached to the merged-away character are not repointed.
|
||||||
|
|
||||||
|
Costs: one bad merge is unrecoverable without rebuilding the identity stage for the whole manga.
|
||||||
|
Revisit when: identity work resumes, or a reviewer reports a wrong merge on a real chapter.
|
||||||
|
Workaround: none. Clear identity and rerun, which loses the good merges too.
|
||||||
|
|
||||||
|
## Clearing a stage does not undo what it wrote {#dishonest-clearing}
|
||||||
|
|
||||||
|
Dialogue and direction mutate the shared vision JSON. Clearing dialogue leaves its keys in place, so a
|
||||||
|
rerun treats old dialogue as completed. Clearing identity preserves the per-manga registry.
|
||||||
|
|
||||||
|
Costs: a rerun silently reuses stale output, which reads as a reproducible result.
|
||||||
|
Revisit when: any stage is scheduled concurrently or resumed automatically. A stage must be idempotent
|
||||||
|
before either is safe.
|
||||||
|
Workaround: delete the keys by hand, or clear from `crop` down.
|
||||||
|
|
||||||
|
## ComfyUI uses the GPU outside the session mutex {#comfyui-unscheduled}
|
||||||
|
|
||||||
|
The layers stage calls ComfyUI directly and takes no lease. Another job can load gemma, siglip2, or
|
||||||
|
dots while ComfyUI holds the same GPU.
|
||||||
|
|
||||||
|
Costs: out-of-memory failures that look random and land on an unrelated stage.
|
||||||
|
Revisit when: layers is enabled on a real run, or a second concurrent job is allowed.
|
||||||
|
Workaround: `MAX_CONCURRENT_JOBS=1` keeps one pipeline at a time, which is the current default.
|
||||||
|
|
||||||
|
## The JSON repair pass can fabricate dialogue {#repair-fabricates}
|
||||||
|
|
||||||
|
`call_gemma4_json` hands the model its own truncated text and asks for the JSON it should have been
|
||||||
|
(`worker_vision.py`). The repair call carries no image. On a response truncated by `max_tokens`, the
|
||||||
|
model completes dialogue it can no longer see. What it adds is indistinguishable downstream from
|
||||||
|
transcribed text.
|
||||||
|
|
||||||
|
Costs: invented lines enter the script with normal provenance.
|
||||||
|
Revisit when: schema-constrained generation lands, which removes most of this path.
|
||||||
|
Workaround: resend the image on repair, or retry a truncated response instead of repairing it.
|
||||||
|
|
||||||
|
## `/review/preview` pins a solo clip into the final video {#preview-overwrites-clip}
|
||||||
|
|
||||||
|
`review_preview` renders one panel and calls `save_clip(panel_id, ...)`. `_render_one_beat` returns
|
||||||
|
early when a clip already exists for the leader. Previewing a beat leader therefore drops the rest of
|
||||||
|
the beat's panels. `review_retts` gets this right by calling `delete_clip` first.
|
||||||
|
|
||||||
|
Costs: a reviewer silently corrupts the output by looking at it.
|
||||||
|
Revisit when: the review UI is used on a real chapter.
|
||||||
|
Workaround: never preview a beat leader, or delete the clip row afterwards.
|
||||||
|
|
||||||
|
## Stored embeddings carry no model or pooling version {#untagged-embeddings}
|
||||||
|
|
||||||
|
`embed_crop` uses `pooler_output` when present and falls back to mean-pooled patch tokens otherwise.
|
||||||
|
The two paths produce different vector spaces, and the fixed 0.85 threshold is valid for one of them.
|
||||||
|
Nothing beside a stored `.npy` records which model, revision, or pooling produced it.
|
||||||
|
|
||||||
|
Costs: a transformers upgrade mixes incompatible vectors into one gallery with no error.
|
||||||
|
Revisit when: transformers or the siglip2 revision is upgraded. Before, not after.
|
||||||
|
Workaround: none. Write the model id and pooling mode beside the vector and refuse cross-version
|
||||||
|
comparison.
|
||||||
|
|
||||||
|
## Worker endpoints block the event loop {#blocking-event-loop}
|
||||||
|
|
||||||
|
Endpoints declared `async def` run blocking MinIO, OpenCV, torch, and ffmpeg calls directly on the
|
||||||
|
event loop. A busy worker cannot answer `/health` or `/unload`.
|
||||||
|
|
||||||
|
Costs: `/health/workers` reports a working worker as unreachable, and the session manager's 30-second
|
||||||
|
`/unload` can time out exactly when VRAM needs freeing.
|
||||||
|
Revisit when: a stage stalls on `/unload`, or before any bounded parallelism lands.
|
||||||
|
Workaround: `def` instead of `async def` moves each handler to the threadpool. Tracked as [#199] for
|
||||||
|
the render worker.
|
||||||
|
|
||||||
|
## SQLite has no busy timeout {#sqlite-locking}
|
||||||
|
|
||||||
|
`get_conn` opens a connection per call with no `busy_timeout`. WAL tolerates one writer. `PIPELINE=1`
|
||||||
|
already writes clips from concurrent tasks while TTS writes audio.
|
||||||
|
|
||||||
|
Costs: planned CPU parallelism will surface as `database is locked` before it surfaces as throughput.
|
||||||
|
Revisit when: Phase 2 concurrency work starts. Set the timeout first.
|
||||||
|
Workaround: keep `PIPELINE` off.
|
||||||
|
|
||||||
|
## `layers` runs after `tts` {#layers-after-tts}
|
||||||
|
|
||||||
|
`STAGES` orders `layers` after `tts`, while `run_stage_tts` warns that eager rendering under
|
||||||
|
`PIPELINE=1` needs layers to run first.
|
||||||
|
|
||||||
|
Costs: with the flag on, solo beats always render without parallax.
|
||||||
|
Revisit when: a real run enables `PIPELINE=1`.
|
||||||
|
Workaround: keep `PIPELINE` off, or reorder `STAGES`.
|
||||||
|
|
||||||
|
## `completed` means something different in each stage {#inconsistent-stage-policy}
|
||||||
|
|
||||||
|
A dropped vision panel fails the stage and halts the pipeline. A failed direction window counts its
|
||||||
|
panels as done. Layers always finishes completed.
|
||||||
|
|
||||||
|
Costs: the acceptance metrics in `ROADMAP.md` cannot be read across stages.
|
||||||
|
Revisit when: a baseline measurement is taken. The numbers are meaningless until then.
|
||||||
|
Workaround: none.
|
||||||
|
|
||||||
|
## Identity worker caches characters the orchestrator has deleted {#stale-known-cache}
|
||||||
|
|
||||||
|
`worker_identity._known_cache` is invalidated only by `_persist_char`. The reconcile stage deletes
|
||||||
|
losing characters directly in the orchestrator database. A long-lived worker keeps shortlisting and
|
||||||
|
assigning ids that no longer exist.
|
||||||
|
|
||||||
|
Costs: assignments point at rows that are gone.
|
||||||
|
Revisit when: reconcile runs on a chapter without a worker restart between stages. Tracked as [#201],
|
||||||
|
which proposes caching per `manga_id`.
|
||||||
|
Workaround: restart the identity worker after reconcile.
|
||||||
|
|
||||||
|
## A character seen once gets no assignment at all {#pending-in-worker-memory}
|
||||||
|
|
||||||
|
`worker_identity._pending` holds full crop images in worker memory for a whole chapter, keyed by
|
||||||
|
session. That is durable per-chapter state inside a worker documented as stateless, and it is lost on
|
||||||
|
restart. A character seen exactly once receives no assignment, not even a chapter-local handle.
|
||||||
|
|
||||||
|
Costs: one-off characters vanish from the scene graph.
|
||||||
|
Revisit when: chapter-local tracklet persistence lands (`ROADMAP.md`, Phase 3).
|
||||||
|
Workaround: none.
|
||||||
|
|
||||||
|
## MinIO credentials are hardcoded in committed source {#hardcoded-credentials}
|
||||||
|
|
||||||
|
Defaults live in `transport.py` and `service.py`.
|
||||||
|
|
||||||
|
Costs: the credentials are in git history for anyone who gets the repo.
|
||||||
|
Revisit when: the repo leaves this machine, or MinIO is reachable outside the LAN.
|
||||||
|
Workaround: the environment variables already override them. Set them and remove the defaults.
|
||||||
|
|
||||||
|
## Assemble marks a job completed with no clips {#empty-assemble}
|
||||||
|
|
||||||
|
`run_stage_assemble` does not check that `clip_uris` is non-empty before assembling, then marks the job
|
||||||
|
completed.
|
||||||
|
|
||||||
|
Costs: a failed chapter reports success.
|
||||||
|
Revisit when: any run reports completed without a video. One `if not clip_uris` guard fixes it.
|
||||||
|
Workaround: none.
|
||||||
|
|
||||||
|
## Reviewer timestamps drift against the crossfaded video {#timeline-drift}
|
||||||
|
|
||||||
|
`/review/panels` sums per-panel audio durations. Assemble crossfades clips using the per-beat
|
||||||
|
transitions, so every non-`cut` transition shortens the real video.
|
||||||
|
|
||||||
|
Costs: reviewer timestamps drift further out of sync the further into the chapter they scrub.
|
||||||
|
Revisit when: the review UI is used for timing work.
|
||||||
|
Workaround: subtract the transition overlaps by hand.
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
# decisions/
|
||||||
|
|
||||||
|
One entry per settled question about this pipeline. An entry names the claim, the evidence, the date,
|
||||||
|
and what it forbids or permits next.
|
||||||
|
|
||||||
|
No entry is a plan. `NEXT.md` holds the plan, `ROADMAP.md` holds the horizon, and a limitation that is
|
||||||
|
still live belongs in `caveats/`.
|
||||||
|
|
||||||
|
## Rules for this directory
|
||||||
|
|
||||||
|
* One file per topic, one `##` section per question, so the index can link an anchor.
|
||||||
|
* An entry cites a file and line, a commit, or a test. No citation means it is an opinion, and an
|
||||||
|
opinion gets deleted rather than defended.
|
||||||
|
* **Closed** means acting on it is safe. **Open** means investigated and undecided. **Void** means the
|
||||||
|
evidence turned out to be invalid and the claim must not be cited again.
|
||||||
|
* Rewrite an entry when something contradicts it, and say what changed.
|
||||||
|
|
||||||
|
## Index
|
||||||
|
|
||||||
|
| decision | state |
|
||||||
|
| --- | --- |
|
||||||
|
| [A dialogue row's speaker is `speaker_ref`, not `speaker`](audit-phase1.md#speaker-ref-is-canonical) | closed |
|
||||||
|
| [Script verification must not fail on sentence-initial capitals or short quotes](audit-phase1.md#verifier-false-positives) | closed |
|
||||||
|
| [Tracklet links are transitive over similarity, never across a hard constraint](audit-phase1.md#tracklet-hard-constraints) | closed |
|
||||||
|
| [A detected face takes an identity only when it falls inside that character's box](audit-phase1.md#gated-face-pairing) | closed |
|
||||||
|
| [A 409 from the GPU mutex is a queue signal, not a stale lease](audit-phase1.md#no-lease-stealing) | closed |
|
||||||
|
| [Correctness flags resolve by flag id, and never block autonomous TTS](audit-phase1.md#flag-resolution) | closed |
|
||||||
|
| [An out-of-range resolver index is `unresolved`, never a new character](audit-phase1.md#hallucinated-index) | closed |
|
||||||
|
| [The session manager holds no lock across a model load](audit-phase1.md#unlocked-model-load) | closed |
|
||||||
@@ -0,0 +1,144 @@
|
|||||||
|
# Audit Phase 1: correctness and scheduling safety
|
||||||
|
|
||||||
|
Settled 2026-08-11 from `AUDIT.md`. Every entry was verified by a runnable check in the same commit.
|
||||||
|
No GPU work ran and no pipeline run was executed. Each claim rests on source and on the CPU-only
|
||||||
|
self-checks named below.
|
||||||
|
|
||||||
|
Files: `worker_scene.py`, `worker_script.py`, `worker_vision.py`, `session_manager.py` on workpc, and
|
||||||
|
`tracklets.py`, `correctness.py`, `db.py`, `service.py`, `session_proxy.py` in the homesrv orchestrator
|
||||||
|
(`/mnt/server/home/kami/docker-apps/manga-infra/orchestrator/`).
|
||||||
|
|
||||||
|
## Speaker ref is canonical {#speaker-ref-is-canonical}
|
||||||
|
|
||||||
|
**Closed.** A dialogue row's speaker is `speaker_ref` when its kind is `character_id`. The flat
|
||||||
|
`speaker` field is a compatibility value. The orchestrator already rewrote it from a panel-local id to
|
||||||
|
a `character_id`.
|
||||||
|
|
||||||
|
Evidence: `correctness.py:normalize_dialogue` rewrites `row["speaker"]`. `worker_scene.build_scene`
|
||||||
|
mapped that value through `id_by_local`, whose keys are panel-local ids. Every lookup returned `None`,
|
||||||
|
so `worker_script._render_line` narrated every line as `Someone`.
|
||||||
|
|
||||||
|
Forbids: reading `speaker` as a panel-local id anywhere downstream of the dialogue stage.
|
||||||
|
Check: `python worker_scene.py`, the `p007` case.
|
||||||
|
|
||||||
|
The same field leaked into the dialogue resume path, where a stored `character_id` was pushed into
|
||||||
|
`recent` as a speaker name. It now renders the registry name (`service.py`, `run_stage_dialogue`).
|
||||||
|
|
||||||
|
## Verifier false positives {#verifier-false-positives}
|
||||||
|
|
||||||
|
**Closed.** `verify_script` must stay quiet on valid narration. A failure is not soft. The script stage
|
||||||
|
retries once and then raises, so one false positive halts the whole chapter.
|
||||||
|
|
||||||
|
Two rules fired on correct output.
|
||||||
|
|
||||||
|
1. `unsupported-proper-noun` flagged any capitalized token outside a 20-word stop list. That caught
|
||||||
|
ordinary sentence openers (`Suddenly`, `Behind`, `Inside`, `Both`, `Everyone`, `After`, `Two`) and
|
||||||
|
`Someone`. A capital opening a sentence is now grammar unless the word falls outside a real
|
||||||
|
narration vocabulary. A capital in the middle of a sentence still counts as name evidence.
|
||||||
|
2. `misquote` compared each quoted span against a WHOLE source line at ratio 0.82. The narration prompt
|
||||||
|
asks for a short quote of the actual words. A 15-character quote from a 40-character line scored
|
||||||
|
about 0.55. Grounding now matches the longest contiguous span instead.
|
||||||
|
|
||||||
|
Forbids: adding a verifier rule without a test that a correct beat passes it.
|
||||||
|
Check: `pytest test_script_verify.py` in the orchestrator.
|
||||||
|
|
||||||
|
Verifier feedback now reaches the retry. `worker_script.ScriptInput` declares `beat` and
|
||||||
|
`verifier_feedback`. Pydantic v2 dropped both silently before, so the retry was another blind sample.
|
||||||
|
The repair prompt names each failure and lists the exact quotable lines from the beat.
|
||||||
|
|
||||||
|
## Tracklet hard constraints {#tracklet-hard-constraints}
|
||||||
|
|
||||||
|
**Closed.** Tracklet linking is transitive over similarity evidence and never across a hard constraint.
|
||||||
|
|
||||||
|
The hard constraints are gender and co-presence. Gender is canonicalized first. Vision emits
|
||||||
|
`m|f|unknown` while the registry says `male|female`, and the old gate tested only the second spelling,
|
||||||
|
so it never fired on real data. Co-presence means two detections in one panel are two people by
|
||||||
|
construction. A merge is rejected when any cross pair between the two groups violates either rule.
|
||||||
|
A weak chain can no longer bridge two people seen together.
|
||||||
|
|
||||||
|
Forbids: comparing a raw gender string against a literal enum anywhere in the identity path.
|
||||||
|
Check: `python tracklets.py` and `pytest test_tracklets.py`.
|
||||||
|
|
||||||
|
## Gated face pairing {#gated-face-pairing}
|
||||||
|
|
||||||
|
**Closed.** A detected face takes a character's identity only when its centre falls inside that
|
||||||
|
character's gemma bbox. The box is grown by 25% first. An unpaired face stays `unknown`.
|
||||||
|
|
||||||
|
This matters because the label becomes `speaker_method="som_face"`, the highest-trust provenance the
|
||||||
|
pipeline records. Unconditional nearest-neighbour pairing laundered a geometric guess into evidence.
|
||||||
|
Pairs are now taken globally shortest-first, so the first face processed cannot claim a character that
|
||||||
|
fits a later face better.
|
||||||
|
|
||||||
|
Set-of-mark attribution stays ON by default now that the pairing is gated. The docstring said "off by
|
||||||
|
default" while the flag read `SOM_ATTRIBUTION` with default `"1"`. The code and the comment now agree.
|
||||||
|
|
||||||
|
Forbids: trusting a `som_face` speaker without the containment gate.
|
||||||
|
Check: `python worker_vision.py`.
|
||||||
|
|
||||||
|
## No lease stealing {#no-lease-stealing}
|
||||||
|
|
||||||
|
**Closed.** A 409 from `/session/open` means another job legitimately holds the GPU. The proxy queues
|
||||||
|
on it. Only the session manager's TTL reaper clears a dead lease, because only it can tell a dead
|
||||||
|
lease from a busy one.
|
||||||
|
|
||||||
|
`session_proxy.open_session` previously closed the active lease and retried, so two jobs terminated
|
||||||
|
each other's models. Job admission control now bounds concurrent pipelines through
|
||||||
|
`MAX_CONCURRENT_JOBS`, default 1. A lease queue alone still lets two pipelines interleave stages
|
||||||
|
against one GPU.
|
||||||
|
|
||||||
|
`heartbeat_session` is finally called. `open_session` starts a beat task and `close_session` cancels
|
||||||
|
it, so a stage running past its TTL is no longer reaped mid-run. Beating from open and close keeps all
|
||||||
|
nine stage call sites unchanged.
|
||||||
|
|
||||||
|
Forbids: closing a session the caller did not open.
|
||||||
|
|
||||||
|
## Flag resolution {#flag-resolution}
|
||||||
|
|
||||||
|
**Closed.** Correctness flags resolve by flag id, and unresolved flags never wedge an autonomous run.
|
||||||
|
|
||||||
|
Two defects combined into a permanent block. `get_correctness_flags` accepted rows whose panel no
|
||||||
|
longer existed. A flag raised against a panel that a crop rerun deleted became visible to every
|
||||||
|
chapter and could never be resolved. `resolve_correctness_flags` could only touch flags whose
|
||||||
|
panel still existed. The view no longer admits orphans, and resolution takes explicit flag ids.
|
||||||
|
|
||||||
|
The TTS block also had no reachable exit. It refused to start on four flag kinds that only
|
||||||
|
`/review/approve` clears, and `GATES` defaults to off. With gates on, TTS now returns rather than
|
||||||
|
raises, so `awaiting_review` survives instead of being overwritten by `failed` in the pipeline's
|
||||||
|
catch-all. With gates off, the flags are logged and cleared, because no reviewer exists to clear them.
|
||||||
|
|
||||||
|
Forbids: raising out of a stage that has just set `awaiting_review`.
|
||||||
|
Check: `pytest test_name_binding.py` in the orchestrator.
|
||||||
|
|
||||||
|
## Hallucinated index {#hallucinated-index}
|
||||||
|
|
||||||
|
**Closed.** An out-of-range `choice` from `/vision/resolve` is `unresolved`, like a parse failure. Only
|
||||||
|
an explicit `0` means NONE and mints a new character. Mapping a bad index to NONE created a brand new
|
||||||
|
entry in the permanent registry from a hallucination.
|
||||||
|
|
||||||
|
Check: `python worker_vision.py`.
|
||||||
|
|
||||||
|
## Unlocked model load {#unlocked-model-load} [#203]
|
||||||
|
|
||||||
|
**Closed.** The session manager never holds `_lock` across a model load or a health wait.
|
||||||
|
|
||||||
|
`open_session` was already written that way, but it could orphan the server it spawned. A
|
||||||
|
`/session/close` arriving during the load found `proc = None`, tore down nothing, and cleared
|
||||||
|
`_active`. The finished server then held its VRAM unreferenced, and the next open spawned a second one
|
||||||
|
on the same port. Open now tears down its own process and reports 409 when its lease vanished mid-load.
|
||||||
|
|
||||||
|
`_supervise_once` had the opposite asymmetry. It called `_start_subprocess` while holding `_lock`,
|
||||||
|
which blocked `/session/active`, `/session/close`, and `/session/open` for the full health wait. It now
|
||||||
|
claims the respawn by clearing `proc`, spawns unlocked, and tears the new process down if the lease
|
||||||
|
disappeared meanwhile.
|
||||||
|
|
||||||
|
Check: `python session_manager.py`.
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
`_extract_json` in both `worker_vision.py` and `worker_script.py` now uses
|
||||||
|
`json.JSONDecoder().raw_decode` from the first brace. The greedy `\{.*\}` ran to the LAST brace in the
|
||||||
|
reply. A second object or trailing braced prose burned a repair call on a response that parsed fine.
|
||||||
|
|
||||||
|
`worker_scene.build_scene` emits `actions` as a list beside the joined `action` string, because
|
||||||
|
`correctness.build_beat_artifact` reads the plural key. The verifier was receiving no action evidence
|
||||||
|
at all. The beat builder also falls back to splitting the singular string.
|
||||||
+47
-11
@@ -116,7 +116,12 @@ def open_session(req: OpenReq):
|
|||||||
with _lock:
|
with _lock:
|
||||||
if _active is not None and _active["session_id"] == session_id:
|
if _active is not None and _active["session_id"] == session_id:
|
||||||
_active["proc"] = proc
|
_active["proc"] = proc
|
||||||
return {"session_id": session_id, "port": cfg.get("port")}
|
return {"session_id": session_id, "port": cfg.get("port")}
|
||||||
|
# The lease was closed or reaped while we were spawning (close saw proc=None and tore down
|
||||||
|
# nothing). The server we just started is unreferenced and would hold its VRAM until someone
|
||||||
|
# killed it by hand, and the next open would spawn a SECOND one on the same port.
|
||||||
|
_teardown({"proc": proc})
|
||||||
|
raise HTTPException(409, "session closed while the model was loading")
|
||||||
|
|
||||||
|
|
||||||
@app.post("/session/close")
|
@app.post("/session/close")
|
||||||
@@ -162,8 +167,8 @@ def _supervise_once():
|
|||||||
talks to the port directly, so if we don't bring it back every remaining panel of the stage fails
|
talks to the port directly, so if we don't bring it back every remaining panel of the stage fails
|
||||||
with 'connection refused'. Restarting under the SAME session keeps the lease + port valid so the
|
with 'connection refused'. Restarting under the SAME session keeps the lease + port valid so the
|
||||||
orchestrator never has to re-open. Returns True if a respawn happened."""
|
orchestrator never has to re-open. Returns True if a respawn happened."""
|
||||||
|
global _active
|
||||||
with _lock:
|
with _lock:
|
||||||
global _active
|
|
||||||
sess = _active
|
sess = _active
|
||||||
if not sess:
|
if not sess:
|
||||||
return False
|
return False
|
||||||
@@ -174,16 +179,29 @@ def _supervise_once():
|
|||||||
proc.wait(timeout=1) # reap the zombie
|
proc.wait(timeout=1) # reap the zombie
|
||||||
except subprocess.TimeoutExpired:
|
except subprocess.TimeoutExpired:
|
||||||
pass
|
pass
|
||||||
print(f"[supervise] gemma subprocess died (rc={proc.returncode}) session {sess['session_id']}; "
|
# claim the respawn before releasing the lock: proc=None makes the next _supervise_once pass
|
||||||
f"respawning", flush=True)
|
# return early, so the health wait can't be entered twice for one death.
|
||||||
try:
|
sess["proc"] = None
|
||||||
sess["proc"] = _start_subprocess(MODELS[sess["model"]])
|
session_id, model = sess["session_id"], sess["model"]
|
||||||
sess["last_beat"] = time.time() # don't count the downtime against the TTL reaper
|
# ponytail: spawn + health-wait UNLOCKED, like open_session. Holding _lock here blocked
|
||||||
|
# /session/active, /close and /open for the full health wait (up to 300s).
|
||||||
|
print(f"[supervise] gemma subprocess died (rc={proc.returncode}) session {session_id}; "
|
||||||
|
f"respawning", flush=True)
|
||||||
|
try:
|
||||||
|
new_proc = _start_subprocess(MODELS[model])
|
||||||
|
except Exception as e:
|
||||||
|
print(f"[supervise] respawn failed: {e}; clearing session so the mutex frees", flush=True)
|
||||||
|
with _lock:
|
||||||
|
if _active is not None and _active["session_id"] == session_id:
|
||||||
|
_active = None
|
||||||
|
return False
|
||||||
|
with _lock:
|
||||||
|
if _active is not None and _active["session_id"] == session_id:
|
||||||
|
_active["proc"] = new_proc
|
||||||
|
_active["last_beat"] = time.time() # don't count the downtime against the TTL reaper
|
||||||
return True
|
return True
|
||||||
except Exception as e:
|
_teardown({"proc": new_proc}) # lease went away mid-respawn; don't orphan the server
|
||||||
print(f"[supervise] respawn failed: {e}; clearing session so the mutex frees", flush=True)
|
return False
|
||||||
_active = None
|
|
||||||
return False
|
|
||||||
|
|
||||||
|
|
||||||
def _reaper():
|
def _reaper():
|
||||||
@@ -229,6 +247,24 @@ if __name__ == "__main__":
|
|||||||
close_session(SessionReq(session_id=s4["session_id"]))
|
close_session(SessionReq(session_id=s4["session_id"]))
|
||||||
assert active() is None
|
assert active() is None
|
||||||
|
|
||||||
|
# a close landing DURING the spawn must not orphan the server: open reports 409 and tears it down.
|
||||||
|
MODELS["gemma4"]["binary"] = "/bin/sleep"
|
||||||
|
MODELS["gemma4"]["port"] = None
|
||||||
|
real_start = _start_subprocess
|
||||||
|
|
||||||
|
def racing_start(cfg):
|
||||||
|
p = real_start(cfg)
|
||||||
|
close_session(SessionReq(session_id=_active["session_id"])) # close mid-load
|
||||||
|
return p
|
||||||
|
_start_subprocess = racing_start
|
||||||
|
try:
|
||||||
|
open_session(OpenReq(model="gemma4", ttl=3600))
|
||||||
|
assert False, "open must 409 when its lease vanished mid-load"
|
||||||
|
except HTTPException as e:
|
||||||
|
assert e.status_code == 409
|
||||||
|
_start_subprocess = real_start
|
||||||
|
assert active() is None
|
||||||
|
|
||||||
# in-process model: close must POST /unload to the worker so its resident VRAM is freed.
|
# in-process model: close must POST /unload to the worker so its resident VRAM is freed.
|
||||||
calls = []
|
calls = []
|
||||||
requests.post = lambda url, **kw: calls.append(url) or type("R", (), {"status_code": 200})()
|
requests.post = lambda url, **kw: calls.append(url) or type("R", (), {"status_code": 200})()
|
||||||
|
|||||||
+32
-3
@@ -70,20 +70,32 @@ def build_scene(data: SceneInput):
|
|||||||
|
|
||||||
# Speaker attribution is the vision/dialogue model's job now (it reads bubble tails + turn-taking).
|
# Speaker attribution is the vision/dialogue model's job now (it reads bubble tails + turn-taking).
|
||||||
# speaker is a local_id -> map to character_id via identity; None for narration/sfx/off-panel.
|
# speaker is a local_id -> map to character_id via identity; None for narration/sfx/off-panel.
|
||||||
|
# The orchestrator's normalize_dialogue already rewrote `speaker` to a character_id and put the
|
||||||
|
# typed answer in `speaker_ref`, so the local-id map no longer matches it. Read speaker_ref FIRST;
|
||||||
|
# every lookup used to miss and every line narrated as "Someone".
|
||||||
dialogue = []
|
dialogue = []
|
||||||
for d in data.vision_result.get("dialogue", []):
|
for d in data.vision_result.get("dialogue", []):
|
||||||
raw = d.get("speaker")
|
raw = d.get("speaker")
|
||||||
|
ref = d.get("speaker_ref") or {}
|
||||||
# "unknown" is a DELIBERATE off-panel/indeterminate speaker (narrated as "someone"); it maps
|
# "unknown" is a DELIBERATE off-panel/indeterminate speaker (narrated as "someone"); it maps
|
||||||
# to None just like narration. id_by_local.get already yields None for it.
|
# to None just like narration. id_by_local.get already yields None for it.
|
||||||
dialogue.append({"speaker": id_by_local.get(raw), "text": d.get("text", ""),
|
if ref.get("kind") == "character_id" and ref.get("value"):
|
||||||
|
cid = ref["value"]
|
||||||
|
else:
|
||||||
|
cid = id_by_local.get(raw) or (raw if raw in name_by_id else None)
|
||||||
|
dialogue.append({"speaker": cid, "text": d.get("text", ""),
|
||||||
"type": d.get("type", "speech"),
|
"type": d.get("type", "speech"),
|
||||||
"confidence": d.get("confidence"),
|
"confidence": d.get("confidence"),
|
||||||
"speaker_method": d.get("speaker_method", "unknown")})
|
"speaker_method": d.get("speaker_method", "unknown")})
|
||||||
|
|
||||||
vchars = data.vision_result.get("characters", [])
|
vchars = data.vision_result.get("characters", [])
|
||||||
action = "; ".join(c.get("action", "") for c in vchars if c.get("action")) or ""
|
# `actions` is the list the orchestrator's beat builder reads for verifier evidence; `action` is the
|
||||||
|
# joined string the script prompt renders. Emitting only the string left verification blind.
|
||||||
|
actions = [c["action"].strip() for c in vchars if str(c.get("action") or "").strip()]
|
||||||
|
action = "; ".join(actions)
|
||||||
return {"panel_id": data.panel_id, "characters": characters, "dialogue": dialogue,
|
return {"panel_id": data.panel_id, "characters": characters, "dialogue": dialogue,
|
||||||
"action": action, "entities": data.vision_result.get("entities", []),
|
"action": action, "actions": actions,
|
||||||
|
"entities": data.vision_result.get("entities", []),
|
||||||
"camera": data.vision_result.get("camera", {}), # #8 direction passthrough -> render
|
"camera": data.vision_result.get("camera", {}), # #8 direction passthrough -> render
|
||||||
"transition": data.vision_result.get("transition", "cut")} # #6 transition out
|
"transition": data.vision_result.get("transition", "cut")} # #6 transition out
|
||||||
|
|
||||||
@@ -167,4 +179,21 @@ if __name__ == "__main__":
|
|||||||
))
|
))
|
||||||
assert out5["dialogue"][0]["confidence"] == 0.3
|
assert out5["dialogue"][0]["confidence"] == 0.3
|
||||||
assert out5["dialogue"][0]["speaker_method"] == "turn_taking"
|
assert out5["dialogue"][0]["speaker_method"] == "turn_taking"
|
||||||
|
|
||||||
|
# orchestrator-normalized rows: `speaker` is already a character_id and `speaker_ref` is typed.
|
||||||
|
# the local-id map cannot resolve either — speaker_ref must win, or everything narrates as Someone.
|
||||||
|
out6 = build_scene(SceneInput(
|
||||||
|
panel_id="p007",
|
||||||
|
vision_result={"characters": [{"local_id": "person_1", "bbox": [0, 0, 10, 10]}],
|
||||||
|
"dialogue": [{"speaker": "c1", "speaker_ref": {"kind": "character_id", "value": "c1"},
|
||||||
|
"type": "speech", "text": "Hey."},
|
||||||
|
{"speaker": None, "speaker_ref": {"kind": "unknown", "value": None},
|
||||||
|
"type": "speech", "text": "..."}]},
|
||||||
|
identity_assignments=[{"local_id": "person_1", "character_id": "c1"}],
|
||||||
|
characters_registry=[{"character_id": "c1", "name": "Teto"}],
|
||||||
|
))
|
||||||
|
assert out6["dialogue"][0]["speaker"] == "c1", out6["dialogue"][0]
|
||||||
|
assert out6["dialogue"][1]["speaker"] is None, out6["dialogue"][1]
|
||||||
|
# actions survive as a list for the verifier, not only as the joined prompt string
|
||||||
|
assert out["actions"] == ["waving"] and out["action"] == "waving"
|
||||||
print("worker_scene self-check ok")
|
print("worker_scene self-check ok")
|
||||||
|
|||||||
+48
-6
@@ -21,6 +21,10 @@ class ScriptInput(BaseModel):
|
|||||||
recent: list = [] # #1: last few panels' narration text (local flow, no repeats)
|
recent: list = [] # #1: last few panels' narration text (local flow, no repeats)
|
||||||
introduced: list = [] # #2: names already narrated -> refer by name/pronoun, don't re-describe
|
introduced: list = [] # #2: names already narrated -> refer by name/pronoun, don't re-describe
|
||||||
panel_count: int = 1 # scene-level narration: #panels in this beat -> scales the length budget
|
panel_count: int = 1 # scene-level narration: #panels in this beat -> scales the length budget
|
||||||
|
# the orchestrator has been sending both of these on a failed-script retry since the verifier
|
||||||
|
# landed; pydantic dropped them silently, so the retry was another blind roll of the dice.
|
||||||
|
beat: dict = {} # {beat_id, panels:[{panel_id, dialogue:[{text}], actions}]} — the evidence
|
||||||
|
verifier_feedback: list = [] # [{kind, values|quote}] from verify_script on the previous attempt
|
||||||
|
|
||||||
|
|
||||||
class SummaryInput(BaseModel):
|
class SummaryInput(BaseModel):
|
||||||
@@ -80,9 +84,33 @@ def _chars_line(name_by_id, genders_by_id):
|
|||||||
return ", ".join(out) or "none"
|
return ", ".join(out) or "none"
|
||||||
|
|
||||||
|
|
||||||
|
def _feedback_block(feedback, beat) -> str:
|
||||||
|
"""Render the previous attempt's verifier failures as concrete corrections. Without this the
|
||||||
|
retry is just another sample at the same temperature — the model never learns what was wrong."""
|
||||||
|
if not feedback:
|
||||||
|
return ""
|
||||||
|
lines = []
|
||||||
|
for f in feedback:
|
||||||
|
kind = f.get("kind")
|
||||||
|
if kind == "unsupported-proper-noun":
|
||||||
|
lines.append("- You used name(s) that are not in this beat: "
|
||||||
|
+ ", ".join(str(v) for v in f.get("values", []))
|
||||||
|
+ ". Use only the names listed under 'Characters present', or a pronoun.")
|
||||||
|
elif kind == "misquote":
|
||||||
|
lines.append(f'- Your quote "{f.get("quote","")}" is not what the panel says. '
|
||||||
|
"Quote the exact words below, or drop the quotation marks.")
|
||||||
|
else:
|
||||||
|
lines.append(f"- {kind}: {json.dumps(f, ensure_ascii=False)[:200]}")
|
||||||
|
quotes = [d.get("text", "") for p in (beat or {}).get("panels", [])
|
||||||
|
for d in p.get("dialogue", []) if d.get("text")]
|
||||||
|
exact = ("Exact lines you may quote from:\n" + "\n".join(f'- "{q}"' for q in quotes) + "\n") if quotes else ""
|
||||||
|
return ("Your previous attempt was REJECTED. Fix exactly these problems and rewrite it:\n"
|
||||||
|
+ "\n".join(lines) + "\n" + exact + "\n")
|
||||||
|
|
||||||
|
|
||||||
def build_prompt(sg: dict, chapter_context: str, names_by_id=None,
|
def build_prompt(sg: dict, chapter_context: str, names_by_id=None,
|
||||||
brief: str = "", recent=None, introduced=None, genders_by_id=None,
|
brief: str = "", recent=None, introduced=None, genders_by_id=None,
|
||||||
panel_count: int = 1) -> str:
|
panel_count: int = 1, beat=None, verifier_feedback=None) -> str:
|
||||||
name_by_id = _name_map(sg.get("characters", []), names_by_id)
|
name_by_id = _name_map(sg.get("characters", []), names_by_id)
|
||||||
chars = _chars_line(name_by_id, genders_by_id)
|
chars = _chars_line(name_by_id, genders_by_id)
|
||||||
dialogue = "\n".join(_render_line(d, name_by_id) for d in sg.get("dialogue", [])) or "none"
|
dialogue = "\n".join(_render_line(d, name_by_id) for d in sg.get("dialogue", [])) or "none"
|
||||||
@@ -120,7 +148,7 @@ def build_prompt(sg: dict, chapter_context: str, names_by_id=None,
|
|||||||
else:
|
else:
|
||||||
unit, budget, thin = "this manga panel", "1-2 short sentences, max ~25 words", "panel"
|
unit, budget, thin = "this manga panel", "1-2 short sentences, max ~25 words", "panel"
|
||||||
return (
|
return (
|
||||||
ov + rec + ctx + intro +
|
_feedback_block(verifier_feedback, beat) + ov + rec + ctx + intro +
|
||||||
f"Write TIGHT recap narration for {unit}: {budget}, "
|
f"Write TIGHT recap narration for {unit}: {budget}, "
|
||||||
"present tense. Tell it like you're recapping to a friend — natural, with momentum. Output "
|
"present tense. Tell it like you're recapping to a friend — natural, with momentum. Output "
|
||||||
"only the narration — no preamble, no notes, no alternatives, never ask for more info. If the "
|
"only the narration — no preamble, no notes, no alternatives, never ask for more info. If the "
|
||||||
@@ -182,7 +210,7 @@ async def summary(data: SummaryInput):
|
|||||||
async def script(data: ScriptInput):
|
async def script(data: ScriptInput):
|
||||||
text = call_gemma4(build_prompt(data.scene_graph, data.chapter_context, data.names_by_id,
|
text = call_gemma4(build_prompt(data.scene_graph, data.chapter_context, data.names_by_id,
|
||||||
data.brief, data.recent, data.introduced, data.genders_by_id,
|
data.brief, data.recent, data.introduced, data.genders_by_id,
|
||||||
data.panel_count))
|
data.panel_count, data.beat, data.verifier_feedback))
|
||||||
return {"panel_id": data.panel_id, "text": text}
|
return {"panel_id": data.panel_id, "text": text}
|
||||||
|
|
||||||
|
|
||||||
@@ -199,10 +227,16 @@ class NormalizeInput(BaseModel):
|
|||||||
|
|
||||||
def _extract_json(raw: str) -> dict:
|
def _extract_json(raw: str) -> dict:
|
||||||
text = _strip_thought(raw)
|
text = _strip_thought(raw)
|
||||||
m = re.search(r"\{.*\}", text, re.DOTALL)
|
i = text.find("{")
|
||||||
if not m:
|
if i < 0:
|
||||||
raise ValueError(f"no json: {text[:200]}")
|
raise ValueError(f"no json: {text[:200]}")
|
||||||
return json.loads(m.group(0))
|
# raw_decode stops at the end of the FIRST object. The old greedy `\{.*\}` swallowed any trailing
|
||||||
|
# brace in prose after it and turned a parseable reply into a wasted repair call.
|
||||||
|
try:
|
||||||
|
obj, _ = json.JSONDecoder().raw_decode(text[i:])
|
||||||
|
except json.JSONDecodeError as e:
|
||||||
|
raise ValueError(f"bad json: {text[:200]}") from e
|
||||||
|
return obj
|
||||||
|
|
||||||
|
|
||||||
def build_normalize_prompt(names, entities) -> str:
|
def build_normalize_prompt(names, entities) -> str:
|
||||||
@@ -297,4 +331,12 @@ if __name__ == "__main__":
|
|||||||
nr = _extract_json('{"entities":[{"name":"Everyday","kind":"shop"}],'
|
nr = _extract_json('{"entities":[{"name":"Everyday","kind":"shop"}],'
|
||||||
'"name_fixes":{"CHOI HAESEON":"Choi Haeseon"}}')
|
'"name_fixes":{"CHOI HAESEON":"Choi Haeseon"}}')
|
||||||
assert nr["entities"][0]["name"] == "Everyday" and nr["name_fixes"]["CHOI HAESEON"] == "Choi Haeseon"
|
assert nr["entities"][0]["name"] == "Everyday" and nr["name_fixes"]["CHOI HAESEON"] == "Choi Haeseon"
|
||||||
|
# trailing braced prose after the object must not break the parse (was a wasted repair call)
|
||||||
|
assert _extract_json('{"a":1}\nnote: {not json}')["a"] == 1
|
||||||
|
# verifier feedback reaches the retry prompt with the exact allowed quotes; absent by default
|
||||||
|
assert "REJECTED" not in p
|
||||||
|
beat = {"beat_id": "b1", "panels": [{"panel_id": "p1", "dialogue": [{"text": "stand proud"}]}]}
|
||||||
|
pf = build_prompt(sg, "s", beat=beat, verifier_feedback=[
|
||||||
|
{"kind": "unsupported-proper-noun", "values": ["Kyoto"]}, {"kind": "misquote", "quote": "be proud"}])
|
||||||
|
assert "REJECTED" in pf and "Kyoto" in pf and "be proud" in pf and '- "stand proud"' in pf
|
||||||
print("worker_script self-check ok")
|
print("worker_script self-check ok")
|
||||||
|
|||||||
+72
-23
@@ -19,7 +19,9 @@ GEMMA4_URL = os.environ.get("GEMMA4_URL", "http://127.0.0.1:8090")
|
|||||||
|
|
||||||
# set-of-mark speaker attribution: detect text regions with comic-text-detector (ONNX, CPU) and draw
|
# set-of-mark speaker attribution: detect text regions with comic-text-detector (ONNX, CPU) and draw
|
||||||
# numbered boxes on the panel so gemma transcribes/attributes grounded regions instead of eyeballing
|
# numbered boxes on the panel so gemma transcribes/attributes grounded regions instead of eyeballing
|
||||||
# the whole image. Off by default — flip SOM_ATTRIBUTION=1 once tuned per title (see bubble_detect.py).
|
# the whole image. ON by default: face->identity pairing is now gated on containment, so an unmatched
|
||||||
|
# face is labelled "unknown" instead of borrowing the nearest name (see _pair_faces_to_present).
|
||||||
|
# SOM_ATTRIBUTION=0 falls back to the holistic path (see bubble_detect.py).
|
||||||
try:
|
try:
|
||||||
import bubble_detect
|
import bubble_detect
|
||||||
except Exception: # onnxruntime/model absent -> feature simply stays unavailable
|
except Exception: # onnxruntime/model absent -> feature simply stays unavailable
|
||||||
@@ -31,29 +33,45 @@ except Exception:
|
|||||||
SOM = os.environ.get("SOM_ATTRIBUTION", "1") == "1"
|
SOM = os.environ.get("SOM_ATTRIBUTION", "1") == "1"
|
||||||
|
|
||||||
|
|
||||||
def _pair_faces_to_present(det_faces: list, present: list) -> list:
|
def _pair_faces_to_present(det_faces: list, present: list, margin: float = 0.25) -> list:
|
||||||
"""det_faces = real detector boxes (grounded but identity-less); present = characters gemma placed
|
"""det_faces = real detector boxes (grounded but identity-less); present = characters gemma placed
|
||||||
in the panel (identity + a coarse, imprecise bbox). Assign each detected face the identity of the
|
in the panel (identity + a coarse, imprecise bbox). Pair a face with a present character ONLY when
|
||||||
present character whose gemma-bbox centre falls closest to it, so a green box carries a real name.
|
the face centre falls inside that character's bbox grown by `margin` of its size — gemma's boxes are
|
||||||
A face with no nearby present char stays unknown; a present char is used at most once."""
|
imprecise but not arbitrary. An unpaired face stays unknown instead of borrowing whoever happens to
|
||||||
|
be nearest; the label it carries becomes `speaker_method="som_face"`, the highest-trust provenance
|
||||||
|
the pipeline has, so an ungated guess used to launder itself into evidence.
|
||||||
|
Pairs are taken globally shortest-first, so the first face processed cannot claim a character that
|
||||||
|
fits a later face far better. Each face and each character is used at most once."""
|
||||||
def cx_cy(b): return ((b[0] + b[2]) / 2, (b[1] + b[3]) / 2)
|
def cx_cy(b): return ((b[0] + b[2]) / 2, (b[1] + b[3]) / 2)
|
||||||
used = set()
|
|
||||||
out = []
|
def contains(face_box, char_box) -> bool:
|
||||||
for i, f in enumerate(det_faces, 1):
|
w, h = char_box[2] - char_box[0], char_box[3] - char_box[1]
|
||||||
|
if w <= 0 or h <= 0:
|
||||||
|
return False
|
||||||
|
fx, fy = cx_cy(face_box)
|
||||||
|
return (char_box[0] - margin * w <= fx <= char_box[2] + margin * w
|
||||||
|
and char_box[1] - margin * h <= fy <= char_box[3] + margin * h)
|
||||||
|
|
||||||
|
pairs = []
|
||||||
|
for i, f in enumerate(det_faces):
|
||||||
fx, fy = cx_cy(f["bbox"])
|
fx, fy = cx_cy(f["bbox"])
|
||||||
best, bestd = None, 1e18
|
|
||||||
for j, c in enumerate(present):
|
for j, c in enumerate(present):
|
||||||
b = c.get("bbox")
|
b = c.get("bbox")
|
||||||
if not b or j in used:
|
if not b or len(b) != 4 or not contains(f["bbox"], b):
|
||||||
continue
|
continue
|
||||||
px, py = cx_cy(b)
|
px, py = cx_cy(b)
|
||||||
d = (px - fx) ** 2 + (py - fy) ** 2
|
pairs.append(((px - fx) ** 2 + (py - fy) ** 2, i, j))
|
||||||
if d < bestd:
|
taken_f, taken_c, match = set(), set(), {}
|
||||||
best, bestd = j, d
|
for _, i, j in sorted(pairs):
|
||||||
c = present[best] if best is not None else {}
|
if i in taken_f or j in taken_c:
|
||||||
if best is not None:
|
continue
|
||||||
used.add(best)
|
taken_f.add(i)
|
||||||
out.append({"label": f"P{i}", "bbox": f["bbox"], "local_id": c.get("local_id"),
|
taken_c.add(j)
|
||||||
|
match[i] = j
|
||||||
|
out = []
|
||||||
|
for i, f in enumerate(det_faces):
|
||||||
|
c = present[match[i]] if i in match else {}
|
||||||
|
out.append({"label": f"P{i + 1}", "bbox": f["bbox"], "local_id": c.get("local_id"),
|
||||||
"who": c.get("name") or c.get("desc") or "unknown", "gender": c.get("gender")})
|
"who": c.get("name") or c.get("desc") or "unknown", "gender": c.get("gender")})
|
||||||
return out
|
return out
|
||||||
|
|
||||||
@@ -127,10 +145,17 @@ def _strip_thought(text: str) -> str:
|
|||||||
def _extract_json(raw: str) -> dict:
|
def _extract_json(raw: str) -> dict:
|
||||||
"""strip gemma4 thought, pull the first JSON object, parse it."""
|
"""strip gemma4 thought, pull the first JSON object, parse it."""
|
||||||
text = _strip_thought(raw)
|
text = _strip_thought(raw)
|
||||||
m = re.search(r"\{.*\}", text, re.DOTALL)
|
i = text.find("{")
|
||||||
if not m:
|
if i < 0:
|
||||||
raise ValueError(f"no json in response: {text[:200]}")
|
raise ValueError(f"no json in response: {text[:200]}")
|
||||||
return json.loads(m.group(0))
|
# raw_decode stops at the end of the FIRST object. The old greedy `\{.*\}` ran to the LAST brace in
|
||||||
|
# the reply, so a second object or any trailing braced prose produced an unparseable span and burned
|
||||||
|
# a repair call on a response that was already fine.
|
||||||
|
try:
|
||||||
|
obj, _ = json.JSONDecoder().raw_decode(text[i:])
|
||||||
|
except json.JSONDecodeError as e:
|
||||||
|
raise ValueError(f"bad json in response: {text[:200]}") from e
|
||||||
|
return obj
|
||||||
|
|
||||||
|
|
||||||
def _img_part(image_path: str) -> dict:
|
def _img_part(image_path: str) -> dict:
|
||||||
@@ -893,10 +918,18 @@ async def vision_resolve(data: ResolveInput):
|
|||||||
os.remove(crop)
|
os.remove(crop)
|
||||||
for _, path in refs:
|
for _, path in refs:
|
||||||
os.remove(path)
|
os.remove(path)
|
||||||
# map gemma's 1-based choice back to a character_id; 0 / out-of-range -> NONE (new character).
|
# map gemma's 1-based choice back to a character_id. An explicit 0 means NONE -> a new character.
|
||||||
|
# An OUT-OF-RANGE index is a hallucination, not an answer: it must be `unresolved` like a parse
|
||||||
|
# failure, or a bad index silently mints a brand new character in the permanent registry.
|
||||||
choice = result.get("choice", 0)
|
choice = result.get("choice", 0)
|
||||||
cid = data.candidates[choice - 1]["character_id"] if isinstance(choice, int) and 1 <= choice <= len(data.candidates) else None
|
in_range = isinstance(choice, int) and 1 <= choice <= len(data.candidates)
|
||||||
state = "known" if cid else ("unresolved" if result.get("reason") == "parse_failed" else "new")
|
cid = data.candidates[choice - 1]["character_id"] if in_range else None
|
||||||
|
if in_range:
|
||||||
|
state = "known"
|
||||||
|
elif choice == 0 and result.get("reason") != "parse_failed":
|
||||||
|
state = "new"
|
||||||
|
else:
|
||||||
|
state = "unresolved"
|
||||||
return {"character_id": cid, "state": state, "confidence": float(result.get("confidence", 0.0)),
|
return {"character_id": cid, "state": state, "confidence": float(result.get("confidence", 0.0)),
|
||||||
"reason": result.get("reason", "")}
|
"reason": result.get("reason", "")}
|
||||||
|
|
||||||
@@ -1005,4 +1038,20 @@ if __name__ == "__main__":
|
|||||||
assert False
|
assert False
|
||||||
except ValueError:
|
except ValueError:
|
||||||
pass
|
pass
|
||||||
|
# trailing braced prose after a complete object parses (used to burn a repair call)
|
||||||
|
assert _extract_json('{"skip":false}\nnote {see above}')["skip"] is False
|
||||||
|
|
||||||
|
# face->identity pairing is GATED on containment: a face outside every gemma bbox stays unknown.
|
||||||
|
faces = [{"bbox": [10, 10, 30, 30]}, {"bbox": [900, 900, 920, 920]}]
|
||||||
|
present = [{"local_id": "person_1", "name": "Teto", "bbox": [0, 0, 100, 200]}]
|
||||||
|
paired = _pair_faces_to_present(faces, present)
|
||||||
|
assert paired[0]["local_id"] == "person_1" and paired[0]["who"] == "Teto"
|
||||||
|
assert paired[1]["local_id"] is None and paired[1]["who"] == "unknown", paired[1]
|
||||||
|
# globally shortest-first: the first face must not claim a character that fits the second better.
|
||||||
|
faces2 = [{"bbox": [95, 95, 105, 105]}, {"bbox": [8, 8, 12, 12]}]
|
||||||
|
present2 = [{"local_id": "a", "bbox": [0, 0, 20, 20]}, {"local_id": "b", "bbox": [80, 80, 120, 120]}]
|
||||||
|
p2 = _pair_faces_to_present(faces2, present2)
|
||||||
|
assert [f["local_id"] for f in p2] == ["b", "a"], p2
|
||||||
|
# no present characters at all -> every face unknown, never a phantom identity
|
||||||
|
assert _pair_faces_to_present(faces, [])[0]["local_id"] is None
|
||||||
print("worker_vision self-check ok")
|
print("worker_vision self-check ok")
|
||||||
|
|||||||
Reference in New Issue
Block a user