feat(qa): remote NIM provider + headless-QA robustness
Enable autonomous QA through a remote OpenAI-compatible provider (NVIDIA NIM) and harden the tool/approval path so unattended multi-stage runs complete. - inference: add openai_compat provider (Bearer chat-completions for NIM/OpenAI), dispatched by provider type "nim"/"openai"; key via api_key/api_key_env. - server: bind configured [server] host/port instead of a hardcoded 8080; POST /sessions accepts an optional `intent` (WS parity) for intent-driven workflows. - kernel: thread the bound operator profile's approval_mode into per-tool gating so auto/yolo enable unattended approval (engine still consulted; policy/plane-2 BLOCK stays terminal); on a recoverable tool failure feed the tool's arg-schema back into context so the model self-corrects instead of repeating a malformed call. - tools: split deletion out of file_write into a separate, explicitly-named file_delete tool — a model can no longer delete a file by getting a write-mode parameter wrong. - server: add GET /metrics/tool-reliability — per-model tool-call validity from the event log (measurement groundwork for capability-aware routing). - docs: update AGENTS.md across kernel, tools, server, inference. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -47,6 +47,7 @@ import com.correx.infrastructure.inference.commons.ResidencyMode
|
||||
import com.correx.infrastructure.inference.llama.cpp.DefaultModelManager
|
||||
import com.correx.infrastructure.inference.llama.cpp.LlamaCppEmbedder
|
||||
import com.correx.infrastructure.inference.llama.cpp.LlamaCppInferenceProvider
|
||||
import com.correx.infrastructure.inference.openai.OpenAiCompatInferenceProvider
|
||||
import io.ktor.client.HttpClient
|
||||
import io.ktor.client.engine.cio.CIO
|
||||
import com.correx.infrastructure.router.turbovec.TurboVecL3MemoryStore
|
||||
@@ -127,6 +128,20 @@ object InfrastructureModule {
|
||||
baseUrl = baseUrl,
|
||||
)
|
||||
|
||||
fun createOpenAiCompatProvider(
|
||||
modelId: String,
|
||||
baseUrl: String,
|
||||
apiKey: String,
|
||||
idPrefix: String = "openai",
|
||||
capabilities: Set<CapabilityScore> = DEFAULT_LLAMA_CAPABILITIES,
|
||||
): OpenAiCompatInferenceProvider = OpenAiCompatInferenceProvider(
|
||||
modelId = modelId,
|
||||
baseUrl = baseUrl,
|
||||
apiKey = apiKey,
|
||||
idPrefix = idPrefix,
|
||||
capabilities = capabilities,
|
||||
)
|
||||
|
||||
fun createModelManager(
|
||||
settings: ModelsSettings,
|
||||
eventStore: EventStore,
|
||||
|
||||
Reference in New Issue
Block a user