= infra-artifacts-cas

== purpose

Content-addressable storage (CAS) for binary artifact blobs, backed by append-only segment files on disk. Provides write-once, read-by-hash semantics with integrity verification (CRC32C + BLAKE3), automatic segment rotation, tail recovery after crash, compaction of dead data, and eviction of unreferenced segments. The index is maintained in a SQLite database.

== responsibilities

* Store binary blobs addressed by BLAKE3 hash (content-addressable)
* Append records to immutable segment files with header (length, CRC32C, BLAKE3 hash)
* Look up and read records by hash via a SQLite index
* Detect and recover partially-written tail records after process crash
* Compact fragmented segments: defragment live records into new segments, delete old ones
* Evict dead segments (zero live bytes) when total storage exceeds `maxTotalBytes`
* Materialize `FileWrittenPayload` to the filesystem for artifact writing (sandbox-aware)
* Coordinate flush ordering with the event store to maintain consistency

== non-responsibilities

* Does not manage artifact lifecycle (creation, validation, archival) — that belongs to `core:artifacts`
* Does not store event data — that belongs to `infra:persistence`
* Does not enforce access control — that belongs to `infra:security`
* Does not provide a queryable artifact index beyond hash lookup

== key types

=== CasArtifactStore
* **kind**: class
* **purpose**: Main entry point implementing `ArtifactStore`. Manages a `SegmentWriter` and `SegmentReader`, coordinates compaction and eviction with mutexes, and provides crash recovery via `TailScanner`.
* **fields**: `config`, `index`, `segmentsDir`, `writer`, `reader`

=== CasConfig
* **kind**: data class
* **purpose**: Configuration for the CAS store.
* **fields**: `rootDir` (filesystem root), `maxSegmentBytes` (default 64MB), `maxTotalBytes` (default `Long.MAX_VALUE`)

=== ArtifactIndex (interface)
* **kind**: interface
* **purpose**: Contract for the hash-to-location index. Supports lookup, insert, update, and segment-scoped iteration/deletion.

=== SqliteArtifactIndex
* **kind**: class
* **purpose**: SQLite-backed implementation of `ArtifactIndex`. Uses WAL mode and `INSERT OR IGNORE` / `ON CONFLICT DO UPDATE` semantics.

=== SegmentLayout
* **kind**: object
* **purpose**: Binary layout constants and encoding/decoding utilities. Record header = 4 bytes payload length + 4 bytes CRC32C + 32 bytes BLAKE3 hash.
* **constants**: `HASH_SIZE = 32`, `HEADER_SIZE = 40`

=== SegmentWriter
* **kind**: class
* **purpose**: Writes records to an append-only segment file. Rotates to a new file when the current segment exceeds `maxSegmentBytes`. Supports `fsync()`.

=== SegmentReader
* **kind**: class
* **purpose**: Reads a record from a segment file by `Location`, verifying CRC32C and BLAKE3 hash on read. Throws `CorruptRecordException` on mismatch.

=== Location
* **kind**: data class
* **purpose**: Pointer to a record within a segment file.
* **fields**: `segmentId`, `offset`, `length`

=== TailScanner
* **kind**: class
* **purpose**: After a crash, scans the active segment from the last known tail offset, recovers intact records, and truncates the file at the first corrupted byte.
* **fields**: `segmentsDir`, `index`

=== LivenessScanner
* **kind**: class
* **purpose**: Scans the event store for all referenced artifact IDs (`InferenceStartedEvent.promptArtifactId`, `InferenceCompletedEvent.responseArtifactId`) and computes per-segment live-byte statistics.

=== Compactor
* **kind**: class
* **purpose**: Selects segments below a live-ratio threshold (default 0.5), reads live entries, writes them to a new segment, atomically swaps index entries, and deletes old segment files.
* **fields**: `config`, `index`, `liveness`, `storeMutex`

=== Evictor
* **kind**: class
* **purpose**: When total storage exceeds `maxTotalBytes`, deletes segments that have zero live bytes (not the active segment).
* **fields**: `config`, `index`, `liveness`

=== DefaultMaterializingArtifactWriter
* **kind**: class
* **purpose**: Implements `MaterializingArtifactWriter` by writing `FileWrittenPayload` to the filesystem under a sandbox root, computing the BLAKE3 hash, and returning a `FileWrittenArtifact`. Prevents path traversal.

== event flow

*Inbound:*
* `InferenceStartedEvent` / `InferenceCompletedEvent` — consumed by `LivenessScanner` to determine which artifacts are live (referenced by the event log)

*Outbound:* None. The CAS store does not emit events.

== integration points

* `core:artifactstore` — `ArtifactStore` (interface implemented by `CasArtifactStore`)
* `core:artifacts` — `MaterializingArtifactWriter`, `MaterializationResult`, `FileWrittenArtifact`, `FileWrittenPayload`
* `core:events` — `StoredEvent`, `InferenceStartedEvent`, `InferenceCompletedEvent`, `ArtifactId`
* `infra:persistence` — `EventStore` (consumed by `LivenessScanner`)

== invariants

* Records are immutable once written — updates are not possible (new hash = new record)
* Each record is verified on read: CRC32C and BLAKE3 must match the header
* The `Compactor` holds `storeMutex` during the atomic swap to prevent concurrent puts from observing stale locations
* Compaction and eviction are mutually exclusive (`maintenanceMutex`)
* `TailScanner` truncates the active segment at the first corrupt record before the writer resumes
* The `flushBefore` protocol coordinates event store and CAS flushing to maintain write ordering

== PlantUML diagram

[plantuml, infra-artifacts-cas, "png"]
----
include::../../diagrams/infra-artifacts-cas.puml[]
----




== known issues

* `LivenessScanner.collectLiveIds()` iterates all events in the store — O(n) per compaction/eviction cycle, may be slow for large event logs
* `Evictor` only evicts segments with zero live bytes — partially-dead segments are left for compaction
* `DefaultMaterializingArtifactWriter.materialize()` computes BLAKE3 from a fresh byte array read after writing — could compute during write instead
* No garbage collection for the SQLite index — deleted segment entries leave behind index rows that are only removed via `deleteEntriesIn`

== open questions

* Should liveness scanning support incremental updates via event subscription rather than full replay?
* Should segment files be encrypted at rest?
* Should there be a maximum segment count to bound file descriptor usage?
