epic-12: after epic audit and init commit

2026-05-16 11:42:00 +04:00
commit c77277af0b
461 changed files with 28958 additions and 0 deletions
@@ -0,0 +1,234 @@
+---
+name: "Core Inference Submodule Spec"
+description: "Specification for :core:inference – model provider abstraction"
+depth: 2
+links: ["../index.md", "./core-module-spec.md", "./core-context-submodule-spec.md"]
+---
+
+# :core:inference module specification
+
+version: 0.1-draft
+status: foundational specification
+
+---
+
+# 1. purpose
+
+`:core:inference` abstracts model execution behind a stable contract. It owns:
+
+* inference provider interfaces
+* inference request/response model
+* model capability model
+* model lifecycle management contracts
+* GPU residency hints
+* inference isolation
+* retry/detokenization handling
+
+Models are treated as stateless compute engines; this module ensures they are interchangeable, locally or remotely, without affecting orchestration.
+
+---
+
+# 2. responsibilities
+
+`:core:inference` owns:
+
+* `InferenceProvider` contract
+* model capability definitions
+* inference request schema (context pack + generation config)
+* inference response schema (raw text + finish reason)
+* provider registry interface
+* model scheduling (which model for which stage)
+* inference event definitions
+* token accounting (prompt/completion tokens)
+* timeout and cancellation contracts
+
+---
+
+# 3. non-responsibilities
+
+`:core:inference` MUST NOT own:
+
+* actual provider implementations (llama.cpp, ollama, etc.)
+* context pack construction
+* artifact parsing (that's orchestration/validation)
+* model weight management (infrastructure handles storage)
+* UI rendering of inference progress
+
+---
+
+# 4. architectural role
+
+`:core:inference` acts as:
+
+* the only point of contact for models
+* the provider abstraction layer
+* the enforcer of concurrency limits and GPU residency policies
+
+---
+
+# 5. design principles
+
+## 5.1 models are stateless
+
+Every inference call receives a full context pack. No state is retained on the model side between calls.
+
+## 5.2 capability‑based routing
+
+Stages request capabilities (e.g., coding, tool_calling). The inference module selects the best available model from the registry, considering local GPU budget and fallback providers.
+
+## 5.3 deterministic‑enough configuration
+
+Inference parameters (temperature, top_p, etc.) are part of the stage config and must be reproduced identically for replay.
+
+## 5.4 isolation
+
+Inference runs in a separate coroutine with strict time limits. Models may be forcibly unloaded after a swap timeout.
+
+---
+
+# 6. model provider contract
+
+```kotlin
+interface InferenceProvider {
+    val name: String
+    suspend fun infer(request: InferenceRequest): InferenceResponse
+    suspend fun healthCheck(): ProviderHealth
+    fun capabilities(): Set<ModelCapability>
+    fun tokenize(text: String): List<Token>
+}
+```
+
+---
+
+# 7. inference request/response
+
+```kotlin
+data class InferenceRequest(
+    val contextPack: ContextPack,
+    val generationConfig: GenerationConfig,
+    val stageId: StageId,
+    val sessionId: SessionId
+)
+
+data class InferenceResponse(
+    val text: String,
+    val finishReason: FinishReason,
+    val tokensUsed: TokenUsage,
+    val latencyMs: Long
+)
+```
+
+---
+
+# 8. capability negotiation
+
+Models declare capabilities with optional scores. The inference module uses a configurable strategy (e.g., highest score first, then fallback to remote if local below threshold) to select the model for a stage.
+
+---
+
+# 9. model lifecycle management
+
+Inference module defines contracts for:
+
+* loading a model (provider specific)
+* unloading
+* health checks
+* swap scheduling (ephemeral, dynamic, persistent residency modes)
+
+It does not implement these; it exposes interfaces implemented by infrastructure providers.
+
+---
+
+# 10. inference events
+
+`:core:inference` emits:
+
+* `InferenceStarted`
+* `InferenceCompleted`
+* `InferenceFailed`
+* `InferenceTimeout`
+* `ModelLoaded` / `ModelUnloaded`
+
+All carry causation back to the stage scheduling event.
+
+---
+
+# 11. consumed events
+
+Consumes:
+
+* `StageScheduled` (to know which capability to invoke)
+* `ContextPackBuilt` (to receive the actual pack)
+* system resource events (GPU availability)
+
+---
+
+# 12. invariants
+
+* Every inference call is idempotent from the harness perspective (same context → same response… eventually, but replay uses recorded artifacts).
+* Token usage always tracked.
+* No cross‑session model state leakage.
+
+---
+
+# 13. replay semantics
+
+During replay, inference can be skipped. Recorded `InferenceCompleted` events containing the artifact text are used instead. If inference is re‑played (e.g., for evaluation), the same config and context pack must be used.
+
+---
+
+# 14. threading/concurrency
+
+Inference calls are suspending and may run on a dedicated dispatcher. Concurrency is limited by model-level `max_parallel_sessions`; the inference module enforces this limit using semaphores.
+
+---
+
+# 15. failure semantics
+
+Inference failures (timeout, model crash) emit `InferenceFailed` with a typed reason. The transition engine may then retry with a different provider or fail the stage.
+
+---
+
+# 16. observable requirements
+
+* inference latency
+* token consumption
+* model residency time
+* fallback event count
+* GPU utilization (via provider)
+
+---
+
+# 17. security boundaries
+
+Models are untrusted. Inference runs sandboxed: no filesystem access, no network access unless explicitly configured. Artifacts are treated as untrusted text.
+
+---
+
+# 18. extension model
+
+New providers are added by implementing `InferenceProvider` and registering with the provider registry. Plugins cannot alter the core inference contract.
+
+---
+
+# 19. forbidden patterns
+
+* direct model state reuse between calls
+* capability bypass (using a model directly without capability check)
+* hidden fallback that circumvents audit
+* inference without context pack
+
+---
+
+# 20. testing requirements
+
+* mock provider for deterministic tests
+* capability routing logic
+* cancel/timeout behavior
+* replay with recorded artifacts
+
+---
+
+# 21. philosophy
+
+Inference is a utility, not a partner. This module treats language models like an accelerator card: it submits a well‑formed job, waits for a result, and never assumes the result is anything but a candidate.