kami
63f7918a3e
Adds decision entries for the unpaired set-of-mark label, the interjection verifier false positive, and the vision-blob clearing bug, plus the per-run speaker audit script used to measure the chapter. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
3.6 KiB
storage-layout
Settled questions about which S3 bucket holds what, and about the MinIO replacement.
One bucket per artifact class
State: closed. 2026-08-11.
Six buckets were created on 2026-07-04 (manga raw panels audio layers video). Only manga ever
received an object, because minio_layout.py hardcoded BUCKET = "manga" and four workers built
their own keys as literal s3://manga/.... The other five sat empty for five weeks.
Artifacts now split by class. The key under the bucket is unchanged, so only the leading segment moved:
| artifact | bucket |
|---|---|
| fetched pages | raw |
| panel crops | panels |
| tts wavs | audio |
| layer pngs | layers |
clips and chapter.mp4 |
video |
| vision, identity, scene, script json, character registry | manga |
Orchestrator: minio_layout.py gained BUCKET_RAW/BUCKET_PANELS/BUCKET_AUDIO/BUCKET_LAYERS/
BUCKET_VIDEO and a BUCKETS tuple. parse_key accepts any of them and rejects anything else.
service.py:_s3_delete_prefix takes a <bucket>/<prefix> pair instead of assuming one bucket, and
_stage_s3_prefixes stops slicing the bucket off. Workers: worker_crop.py, worker_tts.py,
worker_layers.py, worker_render.py.
Every S3 URI is s3://<bucket>/<manga_id>/<chapter_id>/... and every consumer already derives the
bucket from the first path segment, so no reader needed a change.
What this forbids: writing an artifact under a bucket that is not in BUCKETS. parse_key returns
{} for one, and stage clearing would then silently delete nothing.
Objects written before this date stay under manga/ at their old keys. Nothing reads them any more:
they are the rollback for the 2026-07-17 run, not live data.
Evidence: test_minio_layout.py (31 tests), and the 2026-08-11 chapter run, which put pages in raw
and 116 panel crops in panels.
The orchestrator creates missing buckets at startup
State: closed. 2026-08-11.
Workers create a bucket on first write (transport.py:115), but the orchestrator uploads pages before
any worker runs and boto3 will not auto-create. service.py:_ensure_buckets runs in the FastAPI
lifespan and creates whatever is missing. A storage backend that is down at boot logs a warning
instead. The check is not worth a failed start.
RustFS is staged, not adopted
State: open. 2026-08-11.
rustfs holds all six buckets on 127.0.0.1:9010/9011, all empty. MinIO still serves every read and
write. Nothing is repointed. Two things still block a cutover, and neither is settled. RustFS is
1.0.0-beta.12, labeled build-type=prerelease. Swapping storage also adds a variable to the run
meant to produce the baseline. Task [#116].
Clearing a stage strips the vision blob it wrote
Closed. 2026-08-11. dialogue and direct have no output table. They write onto the per-panel
vision blob, and _STAGE_TABLES had no entry for either, so /stage/clear dialogue deleted nothing and
still returned {"ok": true}. run_stage_dialogue then saw "dialogue" in vision and skipped all 116
panels.
Evidence: the first clear reported scripts: 46, scene_graphs: 116 and no vision counts. After the fix
the same call reported vision_results.dialogue: 116 and vision_results.direct: 75, all of which the
first clear had left in place. A whole rerun was wasted on stale data before this was found.
_STAGE_VISION_KEYS in db.py names the keys each stage owns, and clear_stage_data strips them.
Forbids: adding a stage that writes onto a shared blob without listing its keys there.
Check: pytest test_db.py, TestClearStageData.