docs: add module documentation in AsciiDoc with PlantUML diagrams
Generated by per-group subagents covering all 52 modules across: core (18), infrastructure (10), apps (5), interfaces (3), plugins (7), testing (9) Documentation format: - AsciiDoc (.adoc) files in docs/modules/<group>/ - PlantUML (.puml) files in docs/diagrams/ - .adoc files reference diagrams via include:: directives Each doc covers: purpose, responsibilities, non-responsibilities, key types, event flow, integration points, invariants, PlantUML diagram, known issues, and open questions.
This commit is contained in:
@@ -0,0 +1,130 @@
|
||||
= infra-artifacts-cas
|
||||
|
||||
== purpose
|
||||
|
||||
Content-addressable storage (CAS) for binary artifact blobs, backed by append-only segment files on disk. Provides write-once, read-by-hash semantics with integrity verification (CRC32C + BLAKE3), automatic segment rotation, tail recovery after crash, compaction of dead data, and eviction of unreferenced segments. The index is maintained in a SQLite database.
|
||||
|
||||
== responsibilities
|
||||
|
||||
* Store binary blobs addressed by BLAKE3 hash (content-addressable)
|
||||
* Append records to immutable segment files with header (length, CRC32C, BLAKE3 hash)
|
||||
* Look up and read records by hash via a SQLite index
|
||||
* Detect and recover partially-written tail records after process crash
|
||||
* Compact fragmented segments: defragment live records into new segments, delete old ones
|
||||
* Evict dead segments (zero live bytes) when total storage exceeds `maxTotalBytes`
|
||||
* Materialize `FileWrittenPayload` to the filesystem for artifact writing (sandbox-aware)
|
||||
* Coordinate flush ordering with the event store to maintain consistency
|
||||
|
||||
== non-responsibilities
|
||||
|
||||
* Does not manage artifact lifecycle (creation, validation, archival) — that belongs to `core:artifacts`
|
||||
* Does not store event data — that belongs to `infra:persistence`
|
||||
* Does not enforce access control — that belongs to `infra:security`
|
||||
* Does not provide a queryable artifact index beyond hash lookup
|
||||
|
||||
== key types
|
||||
|
||||
=== CasArtifactStore
|
||||
* **kind**: class
|
||||
* **purpose**: Main entry point implementing `ArtifactStore`. Manages a `SegmentWriter` and `SegmentReader`, coordinates compaction and eviction with mutexes, and provides crash recovery via `TailScanner`.
|
||||
* **fields**: `config`, `index`, `segmentsDir`, `writer`, `reader`
|
||||
|
||||
=== CasConfig
|
||||
* **kind**: data class
|
||||
* **purpose**: Configuration for the CAS store.
|
||||
* **fields**: `rootDir` (filesystem root), `maxSegmentBytes` (default 64MB), `maxTotalBytes` (default `Long.MAX_VALUE`)
|
||||
|
||||
=== ArtifactIndex (interface)
|
||||
* **kind**: interface
|
||||
* **purpose**: Contract for the hash-to-location index. Supports lookup, insert, update, and segment-scoped iteration/deletion.
|
||||
|
||||
=== SqliteArtifactIndex
|
||||
* **kind**: class
|
||||
* **purpose**: SQLite-backed implementation of `ArtifactIndex`. Uses WAL mode and `INSERT OR IGNORE` / `ON CONFLICT DO UPDATE` semantics.
|
||||
|
||||
=== SegmentLayout
|
||||
* **kind**: object
|
||||
* **purpose**: Binary layout constants and encoding/decoding utilities. Record header = 4 bytes payload length + 4 bytes CRC32C + 32 bytes BLAKE3 hash.
|
||||
* **constants**: `HASH_SIZE = 32`, `HEADER_SIZE = 40`
|
||||
|
||||
=== SegmentWriter
|
||||
* **kind**: class
|
||||
* **purpose**: Writes records to an append-only segment file. Rotates to a new file when the current segment exceeds `maxSegmentBytes`. Supports `fsync()`.
|
||||
|
||||
=== SegmentReader
|
||||
* **kind**: class
|
||||
* **purpose**: Reads a record from a segment file by `Location`, verifying CRC32C and BLAKE3 hash on read. Throws `CorruptRecordException` on mismatch.
|
||||
|
||||
=== Location
|
||||
* **kind**: data class
|
||||
* **purpose**: Pointer to a record within a segment file.
|
||||
* **fields**: `segmentId`, `offset`, `length`
|
||||
|
||||
=== TailScanner
|
||||
* **kind**: class
|
||||
* **purpose**: After a crash, scans the active segment from the last known tail offset, recovers intact records, and truncates the file at the first corrupted byte.
|
||||
* **fields**: `segmentsDir`, `index`
|
||||
|
||||
=== LivenessScanner
|
||||
* **kind**: class
|
||||
* **purpose**: Scans the event store for all referenced artifact IDs (`InferenceStartedEvent.promptArtifactId`, `InferenceCompletedEvent.responseArtifactId`) and computes per-segment live-byte statistics.
|
||||
|
||||
=== Compactor
|
||||
* **kind**: class
|
||||
* **purpose**: Selects segments below a live-ratio threshold (default 0.5), reads live entries, writes them to a new segment, atomically swaps index entries, and deletes old segment files.
|
||||
* **fields**: `config`, `index`, `liveness`, `storeMutex`
|
||||
|
||||
=== Evictor
|
||||
* **kind**: class
|
||||
* **purpose**: When total storage exceeds `maxTotalBytes`, deletes segments that have zero live bytes (not the active segment).
|
||||
* **fields**: `config`, `index`, `liveness`
|
||||
|
||||
=== DefaultMaterializingArtifactWriter
|
||||
* **kind**: class
|
||||
* **purpose**: Implements `MaterializingArtifactWriter` by writing `FileWrittenPayload` to the filesystem under a sandbox root, computing the BLAKE3 hash, and returning a `FileWrittenArtifact`. Prevents path traversal.
|
||||
|
||||
== event flow
|
||||
|
||||
*Inbound:*
|
||||
* `InferenceStartedEvent` / `InferenceCompletedEvent` — consumed by `LivenessScanner` to determine which artifacts are live (referenced by the event log)
|
||||
|
||||
*Outbound:* None. The CAS store does not emit events.
|
||||
|
||||
== integration points
|
||||
|
||||
* `core:artifactstore` — `ArtifactStore` (interface implemented by `CasArtifactStore`)
|
||||
* `core:artifacts` — `MaterializingArtifactWriter`, `MaterializationResult`, `FileWrittenArtifact`, `FileWrittenPayload`
|
||||
* `core:events` — `StoredEvent`, `InferenceStartedEvent`, `InferenceCompletedEvent`, `ArtifactId`
|
||||
* `infra:persistence` — `EventStore` (consumed by `LivenessScanner`)
|
||||
|
||||
== invariants
|
||||
|
||||
* Records are immutable once written — updates are not possible (new hash = new record)
|
||||
* Each record is verified on read: CRC32C and BLAKE3 must match the header
|
||||
* The `Compactor` holds `storeMutex` during the atomic swap to prevent concurrent puts from observing stale locations
|
||||
* Compaction and eviction are mutually exclusive (`maintenanceMutex`)
|
||||
* `TailScanner` truncates the active segment at the first corrupt record before the writer resumes
|
||||
* The `flushBefore` protocol coordinates event store and CAS flushing to maintain write ordering
|
||||
|
||||
== PlantUML diagram
|
||||
|
||||
[plantuml, infra-artifacts-cas, "png"]
|
||||
----
|
||||
include::../../diagrams/infra-artifacts-cas.puml[]
|
||||
----
|
||||
|
||||
|
||||
|
||||
|
||||
== known issues
|
||||
|
||||
* `LivenessScanner.collectLiveIds()` iterates all events in the store — O(n) per compaction/eviction cycle, may be slow for large event logs
|
||||
* `Evictor` only evicts segments with zero live bytes — partially-dead segments are left for compaction
|
||||
* `DefaultMaterializingArtifactWriter.materialize()` computes BLAKE3 from a fresh byte array read after writing — could compute during write instead
|
||||
* No garbage collection for the SQLite index — deleted segment entries leave behind index rows that are only removed via `deleteEntriesIn`
|
||||
|
||||
== open questions
|
||||
|
||||
* Should liveness scanning support incremental updates via event subscription rather than full replay?
|
||||
* Should segment files be encrypted at rest?
|
||||
* Should there be a maximum segment count to bound file descriptor usage?
|
||||
@@ -0,0 +1,78 @@
|
||||
= infra-inference-commons
|
||||
|
||||
== purpose
|
||||
|
||||
Defines shared abstractions and types for inference provider lifecycle management. Provides the `ModelManager` interface for loading/unloading models, the `ManagedInferenceProvider` wrapper that ties an `InferenceProvider` to a model lifecycle, and supporting types (`ModelDescriptor`, `ResidencyMode`, `ModelLoadException`).
|
||||
|
||||
== responsibilities
|
||||
|
||||
* Define the contract for model lifecycle management (load, unload, health check)
|
||||
* Provide a managed wrapper around `InferenceProvider` that supports model replacement
|
||||
* Define residency modes that govern when models are unloaded (persistent, dynamic, ephemeral)
|
||||
* Describe model configuration via `ModelDescriptor`
|
||||
|
||||
== non-responsibilities
|
||||
|
||||
* Does not implement any inference provider — that belongs to `infra:inference:llama-cpp` and similar
|
||||
* Does not handle provider routing or selection
|
||||
* Does not define the core inference contract — that belongs to `core:inference`
|
||||
|
||||
== key types
|
||||
|
||||
=== ModelManager
|
||||
* **kind**: interface
|
||||
* **purpose**: Contract for loading, unloading, and health-checking inference models. Enforces single-model-at-a-time.
|
||||
* **methods**: `load(descriptor)`, `unload(modelId)`, `currentModel()`, `healthCheck()`
|
||||
|
||||
=== ManagedInferenceProvider
|
||||
* **kind**: interface
|
||||
* **purpose**: Extends `InferenceProvider` with lifecycle management (unload, isLoaded, load a different model).
|
||||
* **methods**: `unload()`, `isLoaded()`, `load(newDescriptor)`
|
||||
|
||||
=== ModelDescriptor
|
||||
* **kind**: data class
|
||||
* **purpose**: Describes a model instance with its path, residency mode, context size, and capabilities.
|
||||
* **fields**: `modelId`, `modelPath`, `residencyMode`, `idleTimeoutMs` (default 60000), `contextSize` (default 8192), `capabilities`
|
||||
|
||||
=== ResidencyMode
|
||||
* **kind**: enum class
|
||||
* **purpose**: Governs model unloading behavior.
|
||||
* **variants**: `PERSISTENT` (never unload), `DYNAMIC` (unload after idle timeout), `EPHEMERAL` (unload immediately after inference)
|
||||
|
||||
=== ModelLoadException
|
||||
* **kind**: class
|
||||
* **purpose**: Exception thrown when model loading or unloading fails. Carries `modelId` and optional cause.
|
||||
* **fields**: `message`, `modelId`, `cause`
|
||||
|
||||
== event flow
|
||||
|
||||
*None.* The commons layer defines abstractions only; events are emitted by implementations.
|
||||
|
||||
== integration points
|
||||
|
||||
* `core:inference` — `InferenceProvider`, `ProviderHealth`, `CapabilityScore`
|
||||
|
||||
== invariants
|
||||
|
||||
* `ModelManager` enforces single-model-at-a-time via mutex in implementations
|
||||
* Model loading must succeed through health checks before the provider is returned
|
||||
* `ManagedInferenceProvider.load()` replaces the current model entirely
|
||||
|
||||
== PlantUML diagram
|
||||
|
||||
[plantuml, infra-inference-commons, "png"]
|
||||
----
|
||||
include::../../diagrams/infra-inference-commons.puml[]
|
||||
----
|
||||
|
||||
|
||||
|
||||
|
||||
== known issues
|
||||
|
||||
* `ManagedInferenceProvider.load()` uses `as? ManagedInferenceProvider` safe cast — returns null then throws explicit `ModelLoadException`, not `ClassCastException`
|
||||
|
||||
== open questions
|
||||
|
||||
* Should `ModelManager` support multiple concurrent models for different sessions?
|
||||
* Should `ResidencyMode.DYNAMIC` have a configurable timeout per-model or per-manager?
|
||||
@@ -0,0 +1,97 @@
|
||||
= infra-inference-llama-cpp
|
||||
|
||||
== purpose
|
||||
|
||||
Implements the inference provider contracts for llama.cpp. Manages the lifecycle of a llama.cpp server subprocess, communicates with it via HTTP (Ktor client), and provides tokenization, grammar-constrained generation, and model swapping. Converts `JsonSchema` definitions to GBNF grammar for structured output.
|
||||
|
||||
== responsibilities
|
||||
|
||||
* Start, health-check, and stop a `llama-server` subprocess per model
|
||||
* Expose an `InferenceProvider` that sends chat completion requests via the OpenAI-compatible HTTP API
|
||||
* Provide tokenization by delegating to the llama.cpp `/tokenize` endpoint
|
||||
* Convert JSON Schema to GBNF grammar strings for structured output constraints
|
||||
* Emit `ModelLoadedEvent` / `ModelUnloadedEvent` on lifecycle transitions
|
||||
* Enforce single-model-at-a-time via mutex
|
||||
|
||||
== non-responsibilities
|
||||
|
||||
* Does not define inference contracts — that belongs to `core:inference` and `infra:inference:commons`
|
||||
* Does not manage GPU residency or scheduling
|
||||
* Does not implement provider routing or fallback
|
||||
* Does not cache inference results
|
||||
|
||||
== key types
|
||||
|
||||
=== LlamaCppInferenceProvider
|
||||
* **kind**: class
|
||||
* **purpose**: HTTP client-based `InferenceProvider` that sends chat completion requests to a llama.cpp server. Builds messages from `ContextPack`, applies GBNF grammar for JSON responses, and returns `InferenceResponse`.
|
||||
* **fields**: `descriptor` (model config), `baseUrl` (default `http://127.0.0.1:10000`), `httpClient` (Ktor CIO)
|
||||
|
||||
=== DefaultModelManager
|
||||
* **kind**: class
|
||||
* **purpose**: `ModelManager` implementation that spawns and manages a `llama-server` subprocess. Thread-safe via `Mutex`. Emits lifecycle events to the event store. Supports health-check polling and process restart for model swaps.
|
||||
* **fields**: `llamaServerBin` (default `"llama-server"`), `host`, `port`, `healthTimeoutMs` (default 30000), `eventStore`, `httpClient`, `eventDispatcher`
|
||||
|
||||
=== DefaultManagedInferenceProvider
|
||||
* **kind**: class
|
||||
* **purpose**: Wraps a `LlamaCppInferenceProvider` and delegates lifecycle calls to a `ModelManager`. Implements `ManagedInferenceProvider` by delegation.
|
||||
|
||||
=== LlamaProcess
|
||||
* **kind**: class
|
||||
* **purpose**: Manages the OS process lifecycle for the `llama-server` binary. Supports `start()` and `stop()` with I/O redirection to a log file.
|
||||
* **fields**: `command` (process arguments), `logFile` (stdout/stderr destination)
|
||||
|
||||
=== LlamaCppTokenizer
|
||||
* **kind**: class
|
||||
* **purpose**: `Tokenizer` implementation that calls the llama.cpp `/tokenize` HTTP endpoint.
|
||||
|
||||
=== GbnfGrammarConverter
|
||||
* **kind**: internal object
|
||||
* **purpose**: Converts a `JsonSchema` (object type) to a GBNF grammar string used by llama.cpp for constrained generation. Supports required and optional keys.
|
||||
|
||||
=== LlamaCppApiModels
|
||||
* **kind**: file-level serializable classes
|
||||
* **purpose**: DTOs for the OpenAI-compatible chat completion API: `ChatCompletionRequest`, `ChatMessage`, `ChatCompletionResponse`, `Choice`, `Usage`, `TokenizeRequest`, `TokenizeResponse`.
|
||||
|
||||
== event flow
|
||||
|
||||
*Inbound:* None. Inference requests come through `InferenceProvider.infer(request)` which is called by the core inference layer.
|
||||
|
||||
*Outbound:*
|
||||
* `ModelLoadedEvent` — emitted when a model is successfully loaded and health-checked
|
||||
* `ModelUnloadedEvent` — emitted when a model is unloaded
|
||||
|
||||
== integration points
|
||||
|
||||
* `core:inference` — `InferenceProvider`, `InferenceRequest`, `InferenceResponse`, `ProviderHealth`, `Tokenizer`, `CapabilityScore`, `ResponseFormat`, `FinishReason`, `ToolCallRequest`, `ToolDefinition`
|
||||
* `core:events` — `ModelLoadedEvent`, `ModelUnloadedEvent`, `ProviderId`, `SessionId`
|
||||
* `core:tools` — `ToolDefinition`
|
||||
* `infra:inference:commons` — `ModelManager`, `ManagedInferenceProvider`, `ModelDescriptor`, `ModelLoadException`, `ResidencyMode`
|
||||
|
||||
== invariants
|
||||
|
||||
* `DefaultModelManager` ensures single-model-at-a-time — load replaces the current model by killing and restarting the subprocess
|
||||
* Health check must pass before a new model is considered loaded
|
||||
* Model ID mismatch on unload throws `ModelLoadException`
|
||||
* `GbnfGrammarConverter` only supports `type: "object"` schemas — other types will throw
|
||||
|
||||
== PlantUML diagram
|
||||
|
||||
[plantuml, infra-inference-llama-cpp, "png"]
|
||||
----
|
||||
include::../../diagrams/infra-inference-llama-cpp.puml[]
|
||||
----
|
||||
|
||||
|
||||
|
||||
|
||||
== known issues
|
||||
|
||||
* `GbnfGrammarConverter` only supports object-type schemas with string/number properties
|
||||
* `DefaultManagedInferenceProvider.load()` uses `as? ManagedInferenceProvider` safe cast — returns null then throws explicit `ModelLoadException`, not `ClassCastException`
|
||||
* `DefaultModelManager` has an unused `eventStore` constructor parameter (used only for `LivenessScanner` in CAS, not relevant here)
|
||||
|
||||
== open questions
|
||||
|
||||
* Should the `llama-server` binary path be configurable per-environment?
|
||||
* Should process stdout/stderr be exposed for debugging beyond file logging?
|
||||
@@ -0,0 +1,84 @@
|
||||
= infra-persistence
|
||||
|
||||
== purpose
|
||||
|
||||
Provides concrete `EventStore` implementations for persisting and replaying the event log. Supports in-memory storage (for testing and ephemeral sessions) and SQLite-backed durable storage. Also provides a live artifact repository that subscribes to events and maintains an up-to-date in-memory view of artifact state per session.
|
||||
|
||||
== responsibilities
|
||||
|
||||
* Implement the `EventStore` interface for both transient and durable storage
|
||||
* Enforce duplicate event ID detection and sequence ordering
|
||||
* Provide reactive subscriptions via `Flow<StoredEvent>` per session and globally
|
||||
* Maintain an artifact state projection that stays current via event subscription
|
||||
* Support transactional batch appends for atomic event persistence
|
||||
|
||||
== non-responsibilities
|
||||
|
||||
* Does not define event serialization — uses `JsonEventSerializer` from `core:events`
|
||||
* Does not implement CAS content-addressable storage — that belongs to `infra:artifacts-cas`
|
||||
* Does not handle event compaction or retention
|
||||
* Does not implement cross-session querying beyond `allSessionIds()`
|
||||
|
||||
== key types
|
||||
|
||||
=== InMemoryEventStore
|
||||
* **kind**: class
|
||||
* **purpose**: Thread-safe, in-memory event store backed by `ConcurrentHashMap`. Supports duplicate detection via `seenEventIds` set and session-scoped sequence numbers.
|
||||
* **fields**: `streams` (per-session event lists), `sequences` (per-session sequence counters), `globalSeq` (global counter), `seenEventIds` (dedup set)
|
||||
|
||||
=== SqliteEventStore
|
||||
* **kind**: class
|
||||
* **purpose**: Durable event store backed by a SQLite database with WAL mode. Uses JDBC directly. Emits events to reactive flows after append.
|
||||
* **fields**: `connection` (JDBC SQLite connection), `jsonSerializer` (serialization bridge), `artifactStore` (used for `flushBefore` coordination)
|
||||
|
||||
=== LiveArtifactRepository
|
||||
* **kind**: class
|
||||
* **purpose**: Subscribes to an `EventStore` per session and maintains an up-to-date in-memory `ConcurrentHashMap` of `ArtifactState` values. Supports lookup by session, stage, and artifact ID.
|
||||
* **fields**: `artifactCache` (session → artifactId → state), `stageIndex` (session → artifactId → stageId), `subscriptions` (session → coroutine Job), `projector` (reduces events to state)
|
||||
|
||||
=== JDBCHelper
|
||||
* **kind**: object
|
||||
* **purpose**: Utility providing `Connection.transaction()` inline helper with automatic commit/rollback.
|
||||
|
||||
== event flow
|
||||
|
||||
*Inbound:*
|
||||
* `ArtifactCreatedEvent`, `ArtifactValidatingEvent`, `ArtifactValidatedEvent`, `ArtifactRejectedEvent`, `ArtifactSupersededEvent`, `ArtifactArchivedEvent`, `ArtifactRelationshipAddedEvent` — consumed by `LiveArtifactRepository` to maintain artifact projections
|
||||
* `NewEvent` — appended to event stores via `append()` / `appendAll()`
|
||||
|
||||
*Outbound:*
|
||||
* `StoredEvent` — emitted via `Flow<StoredEvent>` subscriptions after each append
|
||||
|
||||
== integration points
|
||||
|
||||
* `core:events` — `EventStore`, `StoredEvent`, `NewEvent`, all artifact event payloads, `EventId`, `SessionId`
|
||||
* `core:artifacts` — `ArtifactProjector`, `ArtifactReducer`, `ArtifactState`, `ArtifactRepository`
|
||||
* `core:artifactstore` — `ArtifactStore` (used by `SqliteEventStore` for flush coordination)
|
||||
|
||||
== invariants
|
||||
|
||||
* Duplicate `eventId` values are rejected with an error
|
||||
* Session sequence numbers are strictly monotonic — violations throw `error()`
|
||||
* `InMemoryEventStore.append()` is synchronized per session for thread safety
|
||||
* `SqliteEventStore` wraps appends in a JDBC transaction and coordinates with `ArtifactStore.flushBefore()`
|
||||
* `LiveArtifactRepository` lazily subscribes on first query and never unsubscribes
|
||||
|
||||
== PlantUML diagram
|
||||
|
||||
[plantuml, infra-persistence, "png"]
|
||||
----
|
||||
include::../../diagrams/infra-persistence.puml[]
|
||||
----
|
||||
|
||||
|
||||
|
||||
|
||||
== known issues
|
||||
|
||||
* `SqliteEventStore` holds an open JDBC `Connection` for the lifetime of the application
|
||||
* No connection pooling or retry logic for SQLite busy conditions
|
||||
|
||||
== open questions
|
||||
|
||||
* Should event stores support snapshot-based recovery for long event streams?
|
||||
* Should there be a read-replica event store for projections to avoid write-contention?
|
||||
@@ -0,0 +1,54 @@
|
||||
= infra-scheduler
|
||||
|
||||
== purpose
|
||||
|
||||
Placeholder module reserved for task scheduling infrastructure — delayed execution, cron-like triggers, and queue management. Currently contains only a `Module` marker object with no implementation.
|
||||
|
||||
== responsibilities
|
||||
|
||||
* Intended: schedule delayed or periodic task execution
|
||||
* Intended: manage execution queues and worker pools
|
||||
* Intended: persist scheduled tasks across restarts
|
||||
|
||||
== non-responsibilities
|
||||
|
||||
* Does not define scheduling contracts — that belongs to `core`
|
||||
* Does not implement workflow orchestration — that belongs to `core:kernel`
|
||||
* Does not provide inference routing — that belongs to `core:inference`
|
||||
|
||||
== key types
|
||||
|
||||
=== Module
|
||||
* **kind**: object
|
||||
* **purpose**: Module marker for the Gradle subproject; no behavior.
|
||||
|
||||
== event flow
|
||||
|
||||
*None.* No events are consumed or emitted. This is a placeholder.
|
||||
|
||||
== integration points
|
||||
|
||||
No integration points. The module has no dependencies beyond `core:events`.
|
||||
|
||||
== invariants
|
||||
|
||||
None — module is empty.
|
||||
|
||||
== PlantUML diagram
|
||||
|
||||
[plantuml, infra-scheduler, "png"]
|
||||
----
|
||||
include::../../diagrams/infra-scheduler.puml[]
|
||||
----
|
||||
|
||||
|
||||
|
||||
|
||||
== known issues
|
||||
|
||||
Module is a placeholder — no scheduler infrastructure exists.
|
||||
|
||||
== open questions
|
||||
|
||||
* Will scheduling be event-driven (e.g., timer events) or time-based (cron)?
|
||||
* Should scheduling be embedded (coroutine-based) or backed by an external queue (Kafka, RabbitMQ)?
|
||||
@@ -0,0 +1,54 @@
|
||||
= infra-security
|
||||
|
||||
== purpose
|
||||
|
||||
Placeholder module reserved for security infrastructure — authentication, authorization, credential management, and secure communication. Currently contains only a `Module` marker object with no implementation.
|
||||
|
||||
== responsibilities
|
||||
|
||||
* Intended: provide authentication adapters (API keys, OAuth, etc.)
|
||||
* Intended: enforce authorization checks on tool execution and API access
|
||||
* Intended: manage credential storage and rotation
|
||||
|
||||
== non-responsibilities
|
||||
|
||||
* Does not define security contracts — that belongs to `core`
|
||||
* Does not implement approval gates — that belongs to `core:approvals`
|
||||
* Does not provide sandboxing — that belongs to `infra:tools`
|
||||
|
||||
== key types
|
||||
|
||||
=== Module
|
||||
* **kind**: object
|
||||
* **purpose**: Module marker for the Gradle subproject; no behavior.
|
||||
|
||||
== event flow
|
||||
|
||||
*None.* No events are consumed or emitted. This is a placeholder.
|
||||
|
||||
== integration points
|
||||
|
||||
No integration points. The module has no dependencies beyond `core:events`.
|
||||
|
||||
== invariants
|
||||
|
||||
None — module is empty.
|
||||
|
||||
== PlantUML diagram
|
||||
|
||||
[plantuml, infra-security, "png"]
|
||||
----
|
||||
include::../../diagrams/infra-security.puml[]
|
||||
----
|
||||
|
||||
|
||||
|
||||
|
||||
== known issues
|
||||
|
||||
Module is a placeholder — no security infrastructure exists.
|
||||
|
||||
== open questions
|
||||
|
||||
* What authentication mechanism(s) will be supported (API key, JWT, mTLS)?
|
||||
* Will credentials be stored in the event store or in an external secrets manager?
|
||||
@@ -0,0 +1,55 @@
|
||||
= infra-telemetry
|
||||
|
||||
== purpose
|
||||
|
||||
Placeholder module reserved for telemetry infrastructure — metrics collection, structured logging, tracing export, and monitoring integration. Currently contains only a `Module` marker object with no implementation.
|
||||
|
||||
== responsibilities
|
||||
|
||||
* Intended: implement metrics collection (counters, gauges, histograms)
|
||||
* Intended: provide log shipping and structured log formatting
|
||||
* Intended: export distributed tracing data
|
||||
* Intended: integrate with monitoring backends (Prometheus, OpenTelemetry, etc.)
|
||||
|
||||
== non-responsibilities
|
||||
|
||||
* Does not define observability contracts — that belongs to `core:observability`
|
||||
* Does not implement application-level instrumentation — that belongs to individual modules
|
||||
* Does not store or query telemetry data long-term
|
||||
|
||||
== key types
|
||||
|
||||
=== Module
|
||||
* **kind**: object
|
||||
* **purpose**: Module marker for the Gradle subproject; no behavior.
|
||||
|
||||
== event flow
|
||||
|
||||
*None.* No events are consumed or emitted. This is a placeholder.
|
||||
|
||||
== integration points
|
||||
|
||||
No integration points. The module has no dependencies beyond `core:events`.
|
||||
|
||||
== invariants
|
||||
|
||||
None — module is empty.
|
||||
|
||||
== PlantUML diagram
|
||||
|
||||
[plantuml, infra-telemetry, "png"]
|
||||
----
|
||||
include::../../diagrams/infra-telemetry.puml[]
|
||||
----
|
||||
|
||||
|
||||
|
||||
|
||||
== known issues
|
||||
|
||||
Module is a placeholder — no telemetry infrastructure exists.
|
||||
|
||||
== open questions
|
||||
|
||||
* Which telemetry backend will be used (Prometheus, OpenTelemetry, Datadog)?
|
||||
* Will telemetry be synchronous (side-channel) or event-sourced?
|
||||
@@ -0,0 +1,82 @@
|
||||
= infra-tools-filesystem
|
||||
|
||||
== purpose
|
||||
|
||||
Provides three filesystem tool implementations for the Correx tool system: `FileReadTool` (read file content with optional line range), `FileWriteTool` (write or delete files with artifact capture via CAS), and `FileEditTool` (append, replace, or patch file content). All tools enforce path allowlisting and validate parameters before execution.
|
||||
|
||||
== responsibilities
|
||||
|
||||
* Read file contents with optional 1-indexed line range slicing
|
||||
* Write content to files, optionally capturing artifacts to the CAS store
|
||||
* Delete files on disk
|
||||
* Edit files via three modes: append (add to end), replace (exact-target string replacement), patch (apply unified diff via external `patch` command)
|
||||
* Validate paths against configured allowlists
|
||||
* Report affected paths for sandbox backup/diff tracking
|
||||
|
||||
== non-responsibilities
|
||||
|
||||
* Do not implement sandboxed execution — that belongs to `infra:tools` (`SandboxedToolExecutor`)
|
||||
* Do not define tool contracts — that belongs to `core:tools`
|
||||
* Do not manage file permissions beyond optional POSIX mode setting
|
||||
* Do not perform approval gating
|
||||
|
||||
== key types
|
||||
|
||||
=== FileReadTool
|
||||
* **kind**: class
|
||||
* **purpose**: Reads a file from disk within allowed paths, with optional start/end line range. Tier T1.
|
||||
* **fields**: `allowedPaths` (set of permitted root directories)
|
||||
|
||||
=== FileWriteTool
|
||||
* **kind**: class
|
||||
* **purpose**: Writes content to a file or deletes a file, validates path against allowlist, optionally captures a `FileWrittenArtifact` to the CAS store. Tier T2.
|
||||
* **fields**: `allowedPaths`, `artifactStore`, `materializingWriter`, `sandboxRoot`, `workingDir`
|
||||
|
||||
=== FileEditTool
|
||||
* **kind**: class
|
||||
* **purpose**: Edits a file via three operations: `append`, `replace` (exact single-occurrence string replacement), or `patch` (applies unified diff via external `patch` binary). Tier T3.
|
||||
* **fields**: `allowedPaths`, `workingDir`
|
||||
|
||||
== event flow
|
||||
|
||||
*Inbound:* `ToolRequest` with parameters (`path`, `content`, `operation`, etc.)
|
||||
|
||||
*Outbound:* None directly. Event emission is handled by the wrapping `SandboxedToolExecutor`. The tools return `ToolResult.Success` or `ToolResult.Failure` synchronously.
|
||||
|
||||
== integration points
|
||||
|
||||
* `core:tools.contract` — `Tool`, `ToolExecutor`, `ToolResult`, `FileAffectingTool`, `ToolCapability`, `ValidationResult`
|
||||
* `core:approvals` — `Tier`
|
||||
* `core:artifactstore` — `ArtifactStore`
|
||||
* `core:artifacts` — `MaterializingArtifactWriter`, `MaterializationResult`, `FileWrittenArtifact`, `FileWrittenPayload`
|
||||
* `infra:artifacts-cas` — used by `FileWriteTool` for CAS artifact storage (via writer interface)
|
||||
|
||||
== invariants
|
||||
|
||||
* All tools validate path against allowed paths — if `allowedPaths` is empty, validation fails ("No paths are allowed")
|
||||
* `FileWriteTool` prevents directory traversal via path normalization
|
||||
* `FileEditTool.replace()` requires exactly one occurrence of the target string — zero or multiple occurrences produce a failure
|
||||
* `FileEditTool.patch()` requires the external `patch` utility to be available
|
||||
* All tools run on `Dispatchers.IO` for blocking filesystem operations
|
||||
|
||||
== PlantUML diagram
|
||||
|
||||
[plantuml, infra-tools-filesystem, "png"]
|
||||
----
|
||||
include::../../diagrams/infra-tools-filesystem.puml[]
|
||||
----
|
||||
|
||||
|
||||
|
||||
|
||||
== known issues
|
||||
|
||||
* `FileEditTool.patch()` shells out to the `patch` binary — not available on all platforms; no error if `patch` is missing beyond a failed `ProcessBuilder`
|
||||
* `FileWriteTool.storeArtifact()` silently logs on materialization failure rather than failing the tool call
|
||||
* `FileEditTool.replace()` reads the entire file into memory — unsuitable for very large files
|
||||
* `FileWriteTool` and `FileEditTool` have duplicated path resolution logic
|
||||
|
||||
== open questions
|
||||
|
||||
* Should `FileEditTool` support a pure-Kotlin patching implementation to remove `patch` dependency?
|
||||
* Should line-range edits (insert before/after line N) be added?
|
||||
@@ -0,0 +1,98 @@
|
||||
= infra-tools
|
||||
|
||||
== purpose
|
||||
|
||||
Provides the concrete tool execution infrastructure: a sandboxed executor that wraps tool calls with file backup/restore, diff computation, and event emission; a dispatching executor that routes tool requests to registered tools; a default tool registry; a shell command tool; and a tool configuration system that maps config to enabled tool instances.
|
||||
|
||||
== responsibilities
|
||||
|
||||
* Wrap tool execution with sandbox directory management and file backup/restore
|
||||
* Compute unified diffs for file-affecting tool results
|
||||
* Emit `ToolExecutionStartedEvent`, `ToolExecutionCompletedEvent`, and `ToolExecutionFailedEvent`
|
||||
* Dispatch tool requests by name to registered tool implementations
|
||||
* Register tools by name via `DefaultToolRegistry`
|
||||
* Execute shell commands with timeout, stdout/stderr capture, and executable allowlisting
|
||||
* Convert `ToolConfig` to a list of `Tool` implementations via `buildTools()`
|
||||
|
||||
== non-responsibilities
|
||||
|
||||
* Does not define tool contracts — that belongs to `core:tools`
|
||||
* Does not implement filesystem tools — that belongs to `infra:tools:filesystem`
|
||||
* Does not enforce approval gating — that belongs to `core:approvals`
|
||||
* Does not perform workflow-level tool routing
|
||||
|
||||
== key types
|
||||
|
||||
=== SandboxedToolExecutor
|
||||
* **kind**: class
|
||||
* **purpose**: Decorator around a `ToolExecutor` that handles per-invocation working directories, file backup/restore for file-affecting tools, diff computation, and event emission. Operates on `Dispatchers.IO`.
|
||||
* **fields**: `delegate` (inner executor), `registry`, `eventDispatcher`, `workDir` (default `/tmp/correx-sandbox`)
|
||||
|
||||
=== DispatchingToolExecutor
|
||||
* **kind**: class
|
||||
* **purpose**: Routes a `ToolRequest` to the appropriate `ToolExecutor` by resolving the tool name from a `ToolRegistry`.
|
||||
|
||||
=== DefaultToolRegistry
|
||||
* **kind**: class
|
||||
* **purpose**: Simple map-based `ToolRegistry` built from a vararg or list of `Tool` instances. Keyed by `tool.name`.
|
||||
|
||||
=== ShellTool
|
||||
* **kind**: class
|
||||
* **purpose**: Implements both `Tool` and `ToolExecutor` for shell command execution. Supports executable allowlisting, configurable timeout (default 30s), and working directory. Validates that `argv` is non-empty and the executable is allowed. Tier T2.
|
||||
* **fields**: `allowedExecutables`, `timeoutMs` (default 30000), `workingDir`
|
||||
|
||||
=== DiffUtil
|
||||
* **kind**: object
|
||||
* **purpose**: Computes unified-diff strings from old and new file contents using LCS-based edit detection. Used by `SandboxedToolExecutor` to record changes in tool receipts.
|
||||
|
||||
=== ToolConfig
|
||||
* **kind**: data class
|
||||
* **purpose**: Configuration structure for enabling and parameterizing tools. Contains sub-configs for shell, file_read, file_write, and file_edit tools.
|
||||
* **fields**: `shell`, `fileRead`, `fileWrite`, `fileEdit`
|
||||
|
||||
=== ToolConfig.buildTools()
|
||||
* **kind**: extension function
|
||||
* **purpose**: Converts a `ToolConfig` into a `List<Tool>` by creating instances for each enabled sub-config.
|
||||
|
||||
== event flow
|
||||
|
||||
*Inbound:* `ToolRequest` (from core tools layer)
|
||||
|
||||
*Outbound:*
|
||||
* `ToolExecutionStartedEvent` — emitted before delegate execution begins
|
||||
* `ToolExecutionCompletedEvent` — includes `ToolReceipt` with exit code, output, metadata, affected entities, duration, tier, and diff
|
||||
* `ToolExecutionFailedEvent` — emitted on execution failure
|
||||
|
||||
== integration points
|
||||
|
||||
* `core:events` — `ToolExecutionStartedEvent`, `ToolExecutionCompletedEvent`, `ToolExecutionFailedEvent`, `ToolReceipt`, `ToolRequest`, `ToolInvocationId`, `SessionId`
|
||||
* `core:tools` — `Tool`, `ToolExecutor`, `ToolResult`, `FileAffectingTool`, `ToolRegistry`, `ToolCapability`, `ValidationResult`
|
||||
* `core:approvals` — `Tier`
|
||||
* `infra:tools:filesystem` — `FileReadTool`, `FileWriteTool`, `FileEditTool`
|
||||
|
||||
== invariants
|
||||
|
||||
* `SandboxedToolExecutor` computes `affectedPaths` once (before delegate execution) to avoid double-invocation divergence
|
||||
* File backup is performed before delegate execution; on failure, originals are restored; on success, backups are deleted
|
||||
* `ShellTool` rejects executables not in the allowed set with `ValidationResult.Invalid`
|
||||
* All tools default to `enabled = false` in configuration
|
||||
|
||||
== PlantUML diagram
|
||||
|
||||
[plantuml, infra-tools, "png"]
|
||||
----
|
||||
include::../../diagrams/infra-tools.puml[]
|
||||
----
|
||||
|
||||
|
||||
|
||||
|
||||
== known issues
|
||||
|
||||
* `SandboxedToolExecutor` cleanup (`cleanWorkingDir`, `restoreOrClean`) is best-effort and may leave files on IO errors
|
||||
* `FileEditTool.patch()` spawns an external `patch` process — not a pure-Kotlin implementation
|
||||
|
||||
== open questions
|
||||
|
||||
* Should `SandboxedToolExecutor` support configurable sandbox root resolution?
|
||||
* Should tool timeout be configurable per-tool-type rather than per-call?
|
||||
@@ -0,0 +1,95 @@
|
||||
= infra-workflow
|
||||
|
||||
== purpose
|
||||
|
||||
Loads workflow definitions from TOML files and from the filesystem, converting them into the core `WorkflowGraph` and providing prompt content loading. The `TomlWorkflowLoader` parses a TOML workflow specification (stages, transitions, conditions) and validates structural correctness. The `FileSystemPromptLoader` resolves prompt file paths locally and from a config directory.
|
||||
|
||||
== responsibilities
|
||||
|
||||
* Parse TOML workflow definitions into `WorkflowGraph` objects
|
||||
* Validate workflow structure: declared start stage exists, all stage/transition references resolve, artifact dependencies are satisfiable
|
||||
* Convert transition condition specs from TOML to `TransitionCondition` objects (always_true, artifact_present/absent/validated, variable_equals, all_of, any_of, not)
|
||||
* Load prompt content from filesystem paths (local path first, then config directory)
|
||||
|
||||
== non-responsibilities
|
||||
|
||||
* Does not orchestrate workflow execution — that belongs to `core:kernel`
|
||||
* Does not define `WorkflowGraph`, `StageConfig`, or `TransitionEdge` types — those come from `core:transitions`
|
||||
* Does not render or template prompt content
|
||||
|
||||
== key types
|
||||
|
||||
=== WorkflowLoader
|
||||
* **kind**: interface
|
||||
* **purpose**: Contract for loading a `WorkflowGraph` from a filesystem path.
|
||||
|
||||
=== TomlWorkflowLoader
|
||||
* **kind**: class
|
||||
* **purpose**: Parses TOML workflow files using Jackson's `TomlMapper` and validates the resulting graph structure.
|
||||
* **fields**: `registry` (`ArtifactKindRegistry` for resolving artifact kind names)
|
||||
|
||||
=== ConditionSpec
|
||||
* **kind**: data class
|
||||
* **purpose**: Intermediate deserialization model for transition conditions from TOML. Converted to `TransitionCondition` via `ConditionFactory`.
|
||||
* **fields**: `type`, `artifactId`, `key`, `value`, `conditions` (nested), `condition` (single nested for `not`)
|
||||
|
||||
=== ConditionFactory (file-level `toCondition()` extension)
|
||||
* **kind**: extension functions
|
||||
* **purpose**: Maps `ConditionSpec.type` strings to `TransitionCondition` implementations. Supports all_of, any_of, not (composite), always_true, artifact_present, artifact_absent, artifact_validated, and variable_equals.
|
||||
|
||||
=== PromptLoader
|
||||
* **kind**: interface
|
||||
* **purpose**: Contract for loading prompt content by path string.
|
||||
|
||||
=== FileSystemPromptLoader
|
||||
* **kind**: class
|
||||
* **purpose**: Resolves prompt files: tries the given path directly, then falls back to `~/.config/correx/prompts/<path>`.
|
||||
* **fields**: `configDir` (default `~/.config/correx/prompts`)
|
||||
|
||||
=== PromptNotFoundException
|
||||
* **kind**: class
|
||||
* **purpose**: Thrown when a prompt file cannot be found at any searched location.
|
||||
|
||||
=== WorkflowValidationException
|
||||
* **kind**: class
|
||||
* **purpose**: Thrown on workflow structural validation failures (missing stages, duplicate transitions, unsatisfied artifact dependencies).
|
||||
|
||||
== event flow
|
||||
|
||||
*None.* The workflow loader is invoked at session start to load configuration — it does not interact with the event store.
|
||||
|
||||
== integration points
|
||||
|
||||
* `core:transitions.graph` — `WorkflowGraph`, `StageConfig`, `TransitionEdge`, `TransitionCondition`
|
||||
* `core:transitions.conditions` — `AlwaysTrue`, `AllOf`, `AnyOf`, `Not`, `ArtifactPresent`, `ArtifactAbsent`, `ArtifactValidated`, `VariableEquals`
|
||||
* `core:artifacts.kind` — `ArtifactKindRegistry`, `DefaultArtifactKindRegistry`, `TypedArtifactSlot`
|
||||
* `core:events.types` — `StageId`, `TransitionId`, `ArtifactId`
|
||||
|
||||
== invariants
|
||||
|
||||
* Start stage must be declared in the stages list
|
||||
* Every transition `from` and `to` reference must name a declared stage (except `to = "done"` which is the terminal sentinel)
|
||||
* Transition IDs must be unique within a workflow file
|
||||
* Every artifact listed in a stage's `needs` must be produced by some stage's `produces`
|
||||
* Unknown condition types in TOML produce a `WorkflowValidationException`
|
||||
* Prompt resolution searches the literal path first, then the config directory
|
||||
|
||||
== PlantUML diagram
|
||||
|
||||
[plantuml, infra-workflow, "png"]
|
||||
----
|
||||
include::../../diagrams/infra-workflow.puml[]
|
||||
----
|
||||
|
||||
|
||||
|
||||
|
||||
== known issues
|
||||
|
||||
* `ConditionSpec` `conditions` and `condition` fields are populated by the TOML parser for their respective condition types (`all_of`/`any_of` use `conditions`; `not` uses `condition`) and consumed by the `toCondition()` extension in `ConditionFactory`. The spec's recursive schema is exercised in tests but the TOML parser only populates flat fields — nested specs are constructed programmatically.
|
||||
* `TomlWorkflowLoader` uses Jackson `TomlMapper` with `FAIL_ON_UNKNOWN_PROPERTIES` disabled, so typos in workflow files may silently produce empty defaults.
|
||||
|
||||
== open questions
|
||||
|
||||
* Should composite conditions (`all_of`, `any_of`, `not`) actually be parseable from TOML, or are they for programmatic use only?
|
||||
* Should workflow validation include cycle detection in the transition graph?
|
||||
Reference in New Issue
Block a user