epic-12: after epic audit and init commit
This commit is contained in:
@@ -0,0 +1,195 @@
|
||||
# Epic 9 — Inference Abstraction (:core:inference)
|
||||
|
||||
## completed deliverables
|
||||
|
||||
### 1. core value types
|
||||
|
||||
introduced the full set of inference-domain value types as the semantic foundation of the module.
|
||||
|
||||
final structures:
|
||||
|
||||
* `ModelCapability` — sealed class (Coding, ToolCalling, Reasoning, Summarization, General)
|
||||
* `CapabilityScore` — capability + score (0.0–1.0); used for provider selection
|
||||
* `FinishReason` — sealed: Stop, Length, Timeout, Cancelled, Error(message)
|
||||
* `ProviderHealth` — sealed: Healthy, Degraded(reason), Unavailable(reason)
|
||||
* `CancellationReason` — sealed: UserRequested, StageTimeout, SessionCancelled, ProviderEvicted
|
||||
* `InferenceTimeout` — value class wrapping Duration
|
||||
* `TokenUsage` — promptTokens, completionTokens, totalTokens (derived)
|
||||
* `GenerationConfig` — fully serializable; temperature, topP, maxTokens, stopSequences, seed
|
||||
|
||||
key properties:
|
||||
|
||||
* `GenerationConfig` and `TokenUsage` are `@Serializable` — replay requirement
|
||||
* `ProviderHealth` and `CancellationReason` are runtime-only sealed classes — not serialized directly
|
||||
* events carry string labels for cancellation context, not sealed class instances
|
||||
|
||||
---
|
||||
|
||||
### 2. inference request / response model
|
||||
|
||||
introduced the canonical data contract for all inference calls in the system.
|
||||
|
||||
final structures:
|
||||
|
||||
* `InferenceRequestId` — opaque value class; caller-generated, provider treats as pass-through
|
||||
* `InferenceRequest` — requestId + sessionId + stageId + contextPack + generationConfig + optional timeout
|
||||
* `InferenceResponse` — requestId + text + finishReason + tokensUsed + latencyMs
|
||||
|
||||
both are `@Serializable` for replay correctness.
|
||||
|
||||
`requestId` is always caller-generated — consistent with identity ownership across sessions, artifacts, and approvals modules.
|
||||
|
||||
---
|
||||
|
||||
### 3. tokenizer interface
|
||||
|
||||
introduced a provider-owned tokenizer abstraction, separated from the provider interface itself.
|
||||
|
||||
final structures:
|
||||
|
||||
* `Token` — value class wrapping Int
|
||||
* `Tokenizer` — interface: `tokenize(text): List<Token>`, `countTokens(text): Int`
|
||||
|
||||
tokenizer is a property of `InferenceProvider`, not a method — makes it clear that tokenization is a static capability of the model, not a per-call operation. two providers declaring the same `ModelCapability` may not share a tokenizer.
|
||||
|
||||
---
|
||||
|
||||
### 4. provider interface and registry
|
||||
|
||||
introduced the core provider abstraction and its lookup contract.
|
||||
|
||||
final structures:
|
||||
|
||||
* `ProviderId` — value class
|
||||
* `InferenceProvider` — interface: id, name, tokenizer, infer, healthCheck, capabilities
|
||||
* `ProviderRegistry` — interface: register, resolve(capability), listAll, healthCheckAll
|
||||
|
||||
`resolve(capability)` returns an ordered list (score descending) — routing strategy picks from this list; registry does not select.
|
||||
|
||||
---
|
||||
|
||||
### 5. routing contracts
|
||||
|
||||
introduced pluggable routing as a first-class contract, not hardcoded selection logic.
|
||||
|
||||
final structures:
|
||||
|
||||
* `RoutingStrategy` — interface: `select(candidates, requiredCapabilities): InferenceProvider`
|
||||
* `InferenceRouter` — interface: `route(stageId, requiredCapabilities): InferenceProvider`
|
||||
* `NoEligibleProviderException` — typed failure; no nullable returns
|
||||
|
||||
routing is pure — no I/O, operates on a snapshot of available providers. decoupling `RoutingStrategy` from `InferenceRouter` allows selection logic (best score, round-robin, fallback chain) to be swapped without touching the router contract.
|
||||
|
||||
---
|
||||
|
||||
### 6. cancellation semantics
|
||||
|
||||
introduced the cancellation contract and established the coroutine cooperation rules for all provider implementations.
|
||||
|
||||
final structure:
|
||||
|
||||
* `InferenceCancellationToken` — interface: isCancelled, cancel(reason)
|
||||
|
||||
coroutine contract (documented as KDoc on the interface):
|
||||
|
||||
* `infer()` must call `ensureActive()` before the socket write, after each streamed chunk, and before parsing the final response
|
||||
* blocking calls must be wrapped with `withContext(Dispatchers.IO)`
|
||||
* `CancellationException` must never be swallowed
|
||||
|
||||
contract is intentionally documentation-only at this layer — enforcement lives in implementations (epic 11).
|
||||
|
||||
---
|
||||
|
||||
### 7. inference events and serialization module
|
||||
|
||||
introduced the full set of inference lifecycle events and registered them in the polymorphic serialization system.
|
||||
|
||||
events:
|
||||
|
||||
* `InferenceStartedEvent` — requestId, sessionId, stageId, providerId
|
||||
* `InferenceCompletedEvent` — requestId, sessionId, stageId, providerId, tokensUsed, latencyMs
|
||||
* `InferenceFailedEvent` — requestId, sessionId, stageId, providerId, reason (string)
|
||||
* `InferenceTimeoutEvent` — requestId, sessionId, stageId, providerId, timeoutMs
|
||||
* `ModelLoadedEvent` — providerId, sessionId
|
||||
* `ModelUnloadedEvent` — providerId, sessionId, cancellationReason (string, optional)
|
||||
|
||||
all events carry `requestId` for causation tracing back to the originating call.
|
||||
|
||||
`inferenceModule: SerializersModule` registers all payloads — follows the same pattern as `eventModule`, `artifactModule`, `contextModule`.
|
||||
|
||||
---
|
||||
|
||||
### 8. mock provider and contract test infrastructure
|
||||
|
||||
introduced a deterministic fake provider for contract validation and test isolation.
|
||||
|
||||
final structures:
|
||||
|
||||
* `MockTokenizer` — character-based approximation (1 token ≈ 4 chars); deterministic; not model-accurate
|
||||
* `MockInferenceProvider` — configurable: fixed response, artificial delay, forced failure; exposes `inferCallCount` for assertion
|
||||
* `InferenceProviderContractTest` — abstract contract test class; same shape as `EventStoreContractTest`
|
||||
|
||||
contract tests cover:
|
||||
|
||||
* response carries matching requestId
|
||||
* text is non-blank
|
||||
* token usage is tracked and positive
|
||||
* latency is non-negative
|
||||
* healthCheck returns non-null
|
||||
* capabilities non-empty with scores in 0.0–1.0
|
||||
* coroutine cancellation is respected
|
||||
* tokenizer count is consistent with tokenize
|
||||
|
||||
---
|
||||
|
||||
# final architecture after Epic 9
|
||||
|
||||
```text
|
||||
InferenceRouter (capability-driven routing)
|
||||
↓
|
||||
RoutingStrategy (pluggable selection logic)
|
||||
↓
|
||||
ProviderRegistry (capability → provider resolution)
|
||||
↓
|
||||
InferenceProvider (stateless compute contract)
|
||||
├── tokenizer: Tokenizer
|
||||
├── infer(InferenceRequest): InferenceResponse
|
||||
└── capabilities(): Set<CapabilityScore>
|
||||
↓
|
||||
InferenceEvents → EventStore (via core:events)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# major architectural outcomes
|
||||
|
||||
Epic 9 established:
|
||||
|
||||
* stable inference provider abstraction — models are interchangeable, infrastructure-independent
|
||||
* capability-based routing with pluggable selection strategy
|
||||
* provider-owned tokenization — no shared tokenizer assumption across model families
|
||||
* caller-generated request identity — consistent with system-wide identity ownership model
|
||||
* cooperative cancellation contract — enforced at implementation layer, documented at contract layer
|
||||
* full inference lifecycle event coverage with causation tracing via requestId
|
||||
* replay-safe serialization for all request/response/config types
|
||||
* mock provider and abstract contract tests for deterministic validation in all future epics
|
||||
|
||||
---
|
||||
|
||||
# what Epic 9 intentionally does NOT include
|
||||
|
||||
not implemented:
|
||||
|
||||
* inference state projection / reducer / repository — deferred to Epic 10 where the orchestrator provides a concrete consumer
|
||||
* actual provider implementations (llama.cpp, ollama) — infrastructure concern, Epic 11
|
||||
* streaming response handling — deferred; contract is text-final only for now
|
||||
* GPU residency scheduling — infrastructure concern, Epic 11
|
||||
* router context isolation (separate L2 memory per provider) — deferred to Epic 13
|
||||
|
||||
---
|
||||
|
||||
# final state
|
||||
|
||||
`:core:inference` provides:
|
||||
|
||||
> a fully specified, infrastructure-independent inference abstraction layer with capability-based routing, cooperative cancellation semantics, replay-safe request/response contracts, and a deterministic mock provider — ready to be composed by the orchestration kernel in Epic 10.
|
||||
Reference in New Issue
Block a user