kami
d26f20c316
Root Child DOX Index assembled, plus a per-module AGENTS.md across the tree (core/*, infrastructure/*, apps/*, testing/*, and docs/examples/frontend/etc), each following the DOX section shape: Purpose, Ownership, Local Contracts, Work Guidance, Verification, Child DOX Index.
1.9 KiB
1.9 KiB
infrastructure/inference/
Purpose
Inference adapter layer. The root module provides DefaultProviderRegistry (registers InferenceProvider instances) and FirstAvailableRoutingStrategy. Submodules cover shared HTTP client infrastructure (commons/) and the llama.cpp server adapter (llama_cpp/).
Ownership
Adapter for LLM inference backends. Implements core:inference interfaces. No dependency on other infrastructure:* modules.
Local Contracts
DefaultProviderRegistryimplementsProviderRegistryfromcore:inference.FirstAvailableRoutingStrategyis the default routing policy; extend only incore:inference, not here.- All LLM responses are proposals — they must be validated by the core before affecting state (invariant #7). Adapters return raw responses; they do not validate.
- Network calls to the llama.cpp server are environment observations; results must be recorded as events by callers if replay must reproduce them (invariant #9).
- Model lifecycle (spawn/own the llama-server process) lives in
llama_cpp/; the autonomous scheduler is intentionally unbuilt (see root CLAUDE.md).
Work Guidance
Standard adapter rules apply (see parent AGENTS.md). HTTP client code uses Ktor CIO; blocking I/O in withContext(Dispatchers.IO).
Verification
./gradlew :infrastructure:inference:test --rerun-tasks
./gradlew :infrastructure:inference:commons:test --rerun-tasks
./gradlew :infrastructure:inference:llama_cpp:test --rerun-tasks
Child DOX Index
commons/— sharedManagedInferenceProvider,ModelManager,ResourceProbe(Nvidia/AMD),ResidencyMode; no separate AGENTS.md (sub-leaf, covered by this doc)llama_cpp/—LlamaCppInferenceProvider,LlamaProcess(spawns/owns llama-server),LlamaCppEmbedder,LlamaCppTokenizer,GbnfGrammarConverter; no separate AGENTS.md (sub-leaf, covered by this doc)