= infra-inference-commons

== purpose

Defines shared abstractions and types for inference provider lifecycle management. Provides the `ModelManager` interface for loading/unloading models, the `ManagedInferenceProvider` wrapper that ties an `InferenceProvider` to a model lifecycle, and supporting types (`ModelDescriptor`, `ResidencyMode`, `ModelLoadException`).

== responsibilities

* Define the contract for model lifecycle management (load, unload, health check)
* Provide a managed wrapper around `InferenceProvider` that supports model replacement
* Define residency modes that govern when models are unloaded (persistent, dynamic, ephemeral)
* Describe model configuration via `ModelDescriptor`

== non-responsibilities

* Does not implement any inference provider — that belongs to `infra:inference:llama-cpp` and similar
* Does not handle provider routing or selection
* Does not define the core inference contract — that belongs to `core:inference`

== key types

=== ModelManager
* **kind**: interface
* **purpose**: Contract for loading, unloading, and health-checking inference models. Enforces single-model-at-a-time.
* **methods**: `load(descriptor)`, `unload(modelId)`, `currentModel()`, `healthCheck()`

=== ManagedInferenceProvider
* **kind**: interface
* **purpose**: Extends `InferenceProvider` with lifecycle management (unload, isLoaded, load a different model).
* **methods**: `unload()`, `isLoaded()`, `load(newDescriptor)`

=== ModelDescriptor
* **kind**: data class
* **purpose**: Describes a model instance with its path, residency mode, context size, and capabilities.
* **fields**: `modelId`, `modelPath`, `residencyMode`, `idleTimeoutMs` (default 60000), `contextSize` (default 8192), `capabilities`

=== ResidencyMode
* **kind**: enum class
* **purpose**: Governs model unloading behavior.
* **variants**: `PERSISTENT` (never unload), `DYNAMIC` (unload after idle timeout), `EPHEMERAL` (unload immediately after inference)

=== ModelLoadException
* **kind**: class
* **purpose**: Exception thrown when model loading or unloading fails. Carries `modelId` and optional cause.
* **fields**: `message`, `modelId`, `cause`

== event flow

*None.* The commons layer defines abstractions only; events are emitted by implementations.

== integration points

* `core:inference` — `InferenceProvider`, `ProviderHealth`, `CapabilityScore`

== invariants

* `ModelManager` enforces single-model-at-a-time via mutex in implementations
* Model loading must succeed through health checks before the provider is returned
* `ManagedInferenceProvider.load()` replaces the current model entirely

== PlantUML diagram

[plantuml, infra-inference-commons, "png"]
----
include::../../diagrams/infra-inference-commons.puml[]
----




== known issues

* `ManagedInferenceProvider.load()` uses `as? ManagedInferenceProvider` safe cast — returns null then throws explicit `ModelLoadException`, not `ClassCastException`

== open questions

* Should `ModelManager` support multiple concurrent models for different sessions?
* Should `ResidencyMode.DYNAMIC` have a configurable timeout per-model or per-manager?