correx/docs/epics/epic-9-resolution.md

Epic 9 — Inference Abstraction (:core:inference)

completed deliverables

1. core value types

introduced the full set of inference-domain value types as the semantic foundation of the module.

final structures:

• ModelCapability — sealed class (Coding, ToolCalling, Reasoning, Summarization, General)
• CapabilityScore — capability + score (0.0–1.0); used for provider selection
• FinishReason — sealed: Stop, Length, Timeout, Cancelled, Error(message)
• ProviderHealth — sealed: Healthy, Degraded(reason), Unavailable(reason)
• CancellationReason — sealed: UserRequested, StageTimeout, SessionCancelled, ProviderEvicted
• InferenceTimeout — value class wrapping Duration
• TokenUsage — promptTokens, completionTokens, totalTokens (derived)
• GenerationConfig — fully serializable; temperature, topP, maxTokens, stopSequences, seed

key properties:

• GenerationConfig and TokenUsage are @Serializable — replay requirement
• ProviderHealth and CancellationReason are runtime-only sealed classes — not serialized directly
• events carry string labels for cancellation context, not sealed class instances

---

2. inference request / response model

introduced the canonical data contract for all inference calls in the system.

final structures:

• InferenceRequestId — opaque value class; caller-generated, provider treats as pass-through
• InferenceRequest — requestId + sessionId + stageId + contextPack + generationConfig + optional timeout
• InferenceResponse — requestId + text + finishReason + tokensUsed + latencyMs

both are @Serializable for replay correctness.

requestId is always caller-generated — consistent with identity ownership across sessions, artifacts, and approvals modules.

---

3. tokenizer interface

introduced a provider-owned tokenizer abstraction, separated from the provider interface itself.

final structures:

• Token — value class wrapping Int
• Tokenizer — interface: tokenize(text): List<Token>, countTokens(text): Int

tokenizer is a property of InferenceProvider, not a method — makes it clear that tokenization is a static capability of the model, not a per-call operation. two providers declaring the same ModelCapability may not share a tokenizer.

---

4. provider interface and registry

introduced the core provider abstraction and its lookup contract.

final structures:

• ProviderId — value class
• InferenceProvider — interface: id, name, tokenizer, infer, healthCheck, capabilities
• ProviderRegistry — interface: register, resolve(capability), listAll, healthCheckAll

resolve(capability) returns an ordered list (score descending) — routing strategy picks from this list; registry does not select.

---

5. routing contracts

introduced pluggable routing as a first-class contract, not hardcoded selection logic.

final structures:

• RoutingStrategy — interface: select(candidates, requiredCapabilities): InferenceProvider
• InferenceRouter — interface: route(stageId, requiredCapabilities): InferenceProvider
• NoEligibleProviderException — typed failure; no nullable returns

routing is pure — no I/O, operates on a snapshot of available providers. decoupling RoutingStrategy from InferenceRouter allows selection logic (best score, round-robin, fallback chain) to be swapped without touching the router contract.

---

6. cancellation semantics

introduced the cancellation contract and established the coroutine cooperation rules for all provider implementations.

final structure:

• InferenceCancellationToken — interface: isCancelled, cancel(reason)

coroutine contract (documented as KDoc on the interface):

• infer() must call ensureActive() before the socket write, after each streamed chunk, and before parsing the final response
• blocking calls must be wrapped with withContext(Dispatchers.IO)
• CancellationException must never be swallowed

contract is intentionally documentation-only at this layer — enforcement lives in implementations (epic 11).

---

7. inference events and serialization module

introduced the full set of inference lifecycle events and registered them in the polymorphic serialization system.

events:

• InferenceStartedEvent — requestId, sessionId, stageId, providerId
• InferenceCompletedEvent — requestId, sessionId, stageId, providerId, tokensUsed, latencyMs
• InferenceFailedEvent — requestId, sessionId, stageId, providerId, reason (string)
• InferenceTimeoutEvent — requestId, sessionId, stageId, providerId, timeoutMs
• ModelLoadedEvent — providerId, sessionId
• ModelUnloadedEvent — providerId, sessionId, cancellationReason (string, optional)

all events carry requestId for causation tracing back to the originating call.

inferenceModule: SerializersModule registers all payloads — follows the same pattern as eventModule, artifactModule, contextModule.

---

8. mock provider and contract test infrastructure

introduced a deterministic fake provider for contract validation and test isolation.

final structures:

• MockTokenizer — character-based approximation (1 token ≈ 4 chars); deterministic; not model-accurate
• MockInferenceProvider — configurable: fixed response, artificial delay, forced failure; exposes inferCallCount for assertion
• InferenceProviderContractTest — abstract contract test class; same shape as EventStoreContractTest

contract tests cover:

• response carries matching requestId
• text is non-blank
• token usage is tracked and positive
• latency is non-negative
• healthCheck returns non-null
• capabilities non-empty with scores in 0.0–1.0
• coroutine cancellation is respected
• tokenizer count is consistent with tokenize

---

final architecture after Epic 9

InferenceRouter (capability-driven routing)
    ↓
RoutingStrategy (pluggable selection logic)
    ↓
ProviderRegistry (capability → provider resolution)
    ↓
InferenceProvider (stateless compute contract)
 ├── tokenizer: Tokenizer
 ├── infer(InferenceRequest): InferenceResponse
 └── capabilities(): Set<CapabilityScore>
    ↓
InferenceEvents → EventStore (via core:events)

---

major architectural outcomes

Epic 9 established:

• stable inference provider abstraction — models are interchangeable, infrastructure-independent
• capability-based routing with pluggable selection strategy
• provider-owned tokenization — no shared tokenizer assumption across model families
• caller-generated request identity — consistent with system-wide identity ownership model
• cooperative cancellation contract — enforced at implementation layer, documented at contract layer
• full inference lifecycle event coverage with causation tracing via requestId
• replay-safe serialization for all request/response/config types
• mock provider and abstract contract tests for deterministic validation in all future epics

---

what Epic 9 intentionally does NOT include

not implemented:

• inference state projection / reducer / repository — deferred to Epic 10 where the orchestrator provides a concrete consumer
• actual provider implementations (llama.cpp, ollama) — infrastructure concern, Epic 11
• streaming response handling — deferred; contract is text-final only for now
• GPU residency scheduling — infrastructure concern, Epic 11
• router context isolation (separate L2 memory per provider) — deferred to Epic 13

---

final state

:core:inference provides:

a fully specified, infrastructure-independent inference abstraction layer with capability-based routing, cooperative cancellation semantics, replay-safe request/response contracts, and a deterministic mock provider — ready to be composed by the orchestration kernel in Epic 10.