Files

T

kami bec9411af3 Put every S3 URI in one place, and add a lint gate

Five workers built output URIs with inline f-strings, so the bucket-per-artifact
layout was spread across worker_tts, worker_identity, worker_crop, worker_layers
and worker_render. Moving a class between buckets meant a grep. They are now
templates in transport.py, formatted at each call site.

Three of those workers also each reimplemented the same parse to recover
manga_id and chapter_id from an input uri, because the orchestrator does not
send them. That is transport.ids_from_uri now, and it raises on a uri too short
to carry the ids rather than returning a wrong pair.

ruff.toml makes `ruff check .` exit 0, so CI can gate on it and a new finding
means a new defect. Fixed: an implicit Optional in 8 signatures, an unparenthesized
implicit concatenation in the ASS filter list, 5 subprocess.run calls now saying
check=False out loud, an unused import, a duplicate exception handler and a
non-executable shebang. Every rule left off carries its reason in ruff.toml.

The ASYNC rules are off because ffmpeg on the event loop is real and already
recorded at caveats/audit-open.md#blocking-event-loop. It needs a refactor per
handler, not a lint fix.

Checked: transport, collage, bubble_detect, test_vision_parse, worker_crop,
worker_scene, worker_script, worker_identity, worker_tts, session_manager,
worker_vision and worker_render self-checks all pass. worker_layers still fails
on a missing legacy/qwen_layered_workflow.json, which predates this branch.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

2026-08-13 23:02:06 +04:00

4.4 KiB

Raw Permalink Blame History

CLAUDE.md

Goal, invariants, and working rules. Read this first.

file	holds
`NEXT.md`	the current state and the live plan
`ROADMAP.md`	the ordered outcomes past the current one
`JOURNAL.md`	what was run and when, append-only
`decisions/`	every settled question, indexed in `decisions/CLAUDE.md`
`caveats/`	every known limit and its revisit trigger, indexed in `caveats/CLAUDE.md`
`AGENTS.md`	commands, with the traps beside them
`AUDIT.md`	the 2026-08-11 pipeline audit, the source of the roadmap
`ARCHITECTURE.md`	the target shape of the pipeline, and what exists against it today
`spec-v3.md`	current quality and look work, marked DONE/TODO per item

Do not restate a finding here. Point at the decision.

The goal

This is the workpc compute half of a manga to narrated-video pipeline. It holds stateless GPU and CPU workers only. State, job scheduling, and stage orchestration live in a separate homesrv orchestrator repo (/mnt/server/home/kami/docker-apps/manga-infra/orchestrator/). The two talk over a fixed HTTP contract. The hard part is that one GPU serves every model, so the schedule, not the model, sets the wall time.

Machine split: workers run on workpc (RX 7900 GRE, ROCm). MinIO and the orchestrator run on homesrv (192.168.1.104, CPU-only). /mnt/server/home/kami/ is an SSHFS mount of homesrv.

Invariants

No durable state in a worker. No sqlite, no cross-request memory. A worker pulls inputs from MinIO by URI, does one stage, pushes outputs back, returns URIs. A /dev/shm cache is allowed because it may be dropped at any time.
One warm model at a time. Every GPU stage takes a lease from session_manager.py on 8095. A 409 is a queue signal, never a stale lease (decisions/audit-phase1.md#no-lease-stealing).
Never hold _lock across a model load or a health wait in session_manager.py (decisions/audit-phase1.md#unlocked-model-load).
A dialogue row's speaker is speaker_ref. The flat speaker field is a compatibility value and holds a character_id, not a panel-local id (decisions/audit-phase1.md#speaker-ref-is-canonical).
A stage never raises after setting awaiting_review. The pipeline's catch-all overwrites it with failed (decisions/audit-phase1.md#flag-resolution).
Never mint a character from an unparseable or out-of-range model answer. That is unresolved (decisions/audit-phase1.md#hallucinated-index).
The HTTP contract with the orchestrator is load-bearing. Changing a worker's request or response shape means reconciling the orchestrator in the same session. Neither repo's self-checks can catch a contract break, because each asserts its own side.
ponytail: comments mark deliberate simplifications and name the upgrade path. Respect them.
Nothing that positions an ffmpeg filter may use format=duration. It reports max(video, audio). It hides A/V drift, and it walks xfade offsets past the end of their input. ffmpeg then discards clips and still exits 0 (decisions/chapter-assembly.md#offsets-from-min-stream).

Working rules

./start_workers.sh              # dev: session_manager + 9 workers, each a uvicorn in a tmux window
tmux attach -t manga-workers    # per-worker logs
sudo systemd/install.sh         # production: one systemd unit per process (User=kami)

.venv/bin/python worker_scene.py    # every module has an assert-based __main__ self-check
.venv/bin/python test_vision_parse.py
ruff check .                        # must exit 0; every ignore in ruff.toml carries its reason

cd /mnt/server/home/kami/docker-apps/manga-infra/orchestrator && pytest -q --ignore=test_api.py

There is no build step. .venv is the ROCm torch env. Workers import transport by module name. Ports: crop 8000, vision 8002, identity 8003, scene 8004, script 8005, tts 8006, layers 8007, render 8008, session_manager 8095.

Every output S3 URI is a template in transport.py, not an f-string in a worker. Add one there when a new artifact class appears.

Any non-trivial logic gets ONE runnable check in __main__, assert-based, no framework. Run the file to verify it.
Editing a worker's request or response shape means editing the orchestrator too, in the same session.
Update NEXT.md alongside any change that moves the plan, and append to JOURNAL.md after a run.
Do not run GPU work or a full pipeline without asking.

4.4 KiB Raw Permalink Blame History

CLAUDE.md

The goal

Invariants

Working rules

4.4 KiB

Raw Permalink Blame History