# Epic 9 — Inference Abstraction (:core:inference)

## completed deliverables

### 1. core value types

introduced the full set of inference-domain value types as the semantic foundation of the module.

final structures:

* `ModelCapability` — sealed class (Coding, ToolCalling, Reasoning, Summarization, General)
* `CapabilityScore` — capability + score (0.0–1.0); used for provider selection
* `FinishReason` — sealed: Stop, Length, Timeout, Cancelled, Error(message)
* `ProviderHealth` — sealed: Healthy, Degraded(reason), Unavailable(reason)
* `CancellationReason` — sealed: UserRequested, StageTimeout, SessionCancelled, ProviderEvicted
* `InferenceTimeout` — value class wrapping Duration
* `TokenUsage` — promptTokens, completionTokens, totalTokens (derived)
* `GenerationConfig` — fully serializable; temperature, topP, maxTokens, stopSequences, seed

key properties:

* `GenerationConfig` and `TokenUsage` are `@Serializable` — replay requirement
* `ProviderHealth` and `CancellationReason` are runtime-only sealed classes — not serialized directly
* events carry string labels for cancellation context, not sealed class instances

---

### 2. inference request / response model

introduced the canonical data contract for all inference calls in the system.

final structures:

* `InferenceRequestId` — opaque value class; caller-generated, provider treats as pass-through
* `InferenceRequest` — requestId + sessionId + stageId + contextPack + generationConfig + optional timeout
* `InferenceResponse` — requestId + text + finishReason + tokensUsed + latencyMs

both are `@Serializable` for replay correctness.

`requestId` is always caller-generated — consistent with identity ownership across sessions, artifacts, and approvals modules.

---

### 3. tokenizer interface

introduced a provider-owned tokenizer abstraction, separated from the provider interface itself.

final structures:

* `Token` — value class wrapping Int
* `Tokenizer` — interface: `tokenize(text): List<Token>`, `countTokens(text): Int`

tokenizer is a property of `InferenceProvider`, not a method — makes it clear that tokenization is a static capability of the model, not a per-call operation. two providers declaring the same `ModelCapability` may not share a tokenizer.

---

### 4. provider interface and registry

introduced the core provider abstraction and its lookup contract.

final structures:

* `ProviderId` — value class
* `InferenceProvider` — interface: id, name, tokenizer, infer, healthCheck, capabilities
* `ProviderRegistry` — interface: register, resolve(capability), listAll, healthCheckAll

`resolve(capability)` returns an ordered list (score descending) — routing strategy picks from this list; registry does not select.

---

### 5. routing contracts

introduced pluggable routing as a first-class contract, not hardcoded selection logic.

final structures:

* `RoutingStrategy` — interface: `select(candidates, requiredCapabilities): InferenceProvider`
* `InferenceRouter` — interface: `route(stageId, requiredCapabilities): InferenceProvider`
* `NoEligibleProviderException` — typed failure; no nullable returns

routing is pure — no I/O, operates on a snapshot of available providers. decoupling `RoutingStrategy` from `InferenceRouter` allows selection logic (best score, round-robin, fallback chain) to be swapped without touching the router contract.

---

### 6. cancellation semantics

introduced the cancellation contract and established the coroutine cooperation rules for all provider implementations.

final structure:

* `InferenceCancellationToken` — interface: isCancelled, cancel(reason)

coroutine contract (documented as KDoc on the interface):

* `infer()` must call `ensureActive()` before the socket write, after each streamed chunk, and before parsing the final response
* blocking calls must be wrapped with `withContext(Dispatchers.IO)`
* `CancellationException` must never be swallowed

contract is intentionally documentation-only at this layer — enforcement lives in implementations (epic 11).

---

### 7. inference events and serialization module

introduced the full set of inference lifecycle events and registered them in the polymorphic serialization system.

events:

* `InferenceStartedEvent` — requestId, sessionId, stageId, providerId
* `InferenceCompletedEvent` — requestId, sessionId, stageId, providerId, tokensUsed, latencyMs
* `InferenceFailedEvent` — requestId, sessionId, stageId, providerId, reason (string)
* `InferenceTimeoutEvent` — requestId, sessionId, stageId, providerId, timeoutMs
* `ModelLoadedEvent` — providerId, sessionId
* `ModelUnloadedEvent` — providerId, sessionId, cancellationReason (string, optional)

all events carry `requestId` for causation tracing back to the originating call.

`inferenceModule: SerializersModule` registers all payloads — follows the same pattern as `eventModule`, `artifactModule`, `contextModule`.

---

### 8. mock provider and contract test infrastructure

introduced a deterministic fake provider for contract validation and test isolation.

final structures:

* `MockTokenizer` — character-based approximation (1 token ≈ 4 chars); deterministic; not model-accurate
* `MockInferenceProvider` — configurable: fixed response, artificial delay, forced failure; exposes `inferCallCount` for assertion
* `InferenceProviderContractTest` — abstract contract test class; same shape as `EventStoreContractTest`

contract tests cover:

* response carries matching requestId
* text is non-blank
* token usage is tracked and positive
* latency is non-negative
* healthCheck returns non-null
* capabilities non-empty with scores in 0.0–1.0
* coroutine cancellation is respected
* tokenizer count is consistent with tokenize

---

# final architecture after Epic 9

```text
InferenceRouter (capability-driven routing)
    ↓
RoutingStrategy (pluggable selection logic)
    ↓
ProviderRegistry (capability → provider resolution)
    ↓
InferenceProvider (stateless compute contract)
 ├── tokenizer: Tokenizer
 ├── infer(InferenceRequest): InferenceResponse
 └── capabilities(): Set<CapabilityScore>
    ↓
InferenceEvents → EventStore (via core:events)
```

---

# major architectural outcomes

Epic 9 established:

* stable inference provider abstraction — models are interchangeable, infrastructure-independent
* capability-based routing with pluggable selection strategy
* provider-owned tokenization — no shared tokenizer assumption across model families
* caller-generated request identity — consistent with system-wide identity ownership model
* cooperative cancellation contract — enforced at implementation layer, documented at contract layer
* full inference lifecycle event coverage with causation tracing via requestId
* replay-safe serialization for all request/response/config types
* mock provider and abstract contract tests for deterministic validation in all future epics

---

# what Epic 9 intentionally does NOT include

not implemented:

* inference state projection / reducer / repository — deferred to Epic 10 where the orchestrator provides a concrete consumer
* actual provider implementations (llama.cpp, ollama) — infrastructure concern, Epic 11
* streaming response handling — deferred; contract is text-final only for now
* GPU residency scheduling — infrastructure concern, Epic 11
* router context isolation (separate L2 memory per provider) — deferred to Epic 13

---

# final state

`:core:inference` provides:

> a fully specified, infrastructure-independent inference abstraction layer with capability-based routing, cooperative cancellation semantics, replay-safe request/response contracts, and a deterministic mock provider — ready to be composed by the orchestration kernel in Epic 10.