feat(qa): remote NIM provider + headless-QA robustness

Enable autonomous QA through a remote OpenAI-compatible provider (NVIDIA NIM)
and harden the tool/approval path so unattended multi-stage runs complete.

- inference: add openai_compat provider (Bearer chat-completions for NIM/OpenAI),
  dispatched by provider type "nim"/"openai"; key via api_key/api_key_env.
- server: bind configured [server] host/port instead of a hardcoded 8080;
  POST /sessions accepts an optional `intent` (WS parity) for intent-driven workflows.
- kernel: thread the bound operator profile's approval_mode into per-tool gating so
  auto/yolo enable unattended approval (engine still consulted; policy/plane-2 BLOCK
  stays terminal); on a recoverable tool failure feed the tool's arg-schema back into
  context so the model self-corrects instead of repeating a malformed call.
- tools: split deletion out of file_write into a separate, explicitly-named file_delete
  tool — a model can no longer delete a file by getting a write-mode parameter wrong.
- server: add GET /metrics/tool-reliability — per-model tool-call validity from the
  event log (measurement groundwork for capability-aware routing).
- docs: update AGENTS.md across kernel, tools, server, inference.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-29 10:50:16 +00:00
parent d26f20c316
commit 238d353653
27 changed files with 924 additions and 268 deletions
+2 -1
View File
@@ -13,7 +13,8 @@ CORREX kernel team. This is the integration point for all other `core/` modules.
- `SessionOrchestrator` / `DefaultSessionOrchestrator` — primary entry point for launching and advancing sessions through workflow stages.
- `OrchestrationState` / `OrchestrationReducer` (`DefaultOrchestrationReducer`) / `OrchestrationProjector` / `OrchestrationRepository` — standard event-sourcing stack for orchestration state.
- `RetryCoordinator` / `DefaultRetryCoordinator` — manages retry logic per `RetryPolicy`.
- `ApprovalGateway` — kernel-side approval bridge; calls `core:approvals` engine before executing gated operations.
- On a recoverable tool failure `dispatchToolCalls` feeds the failing tool's argument schema back into context alongside the error (`toolArgsHint`), so the model self-corrects a malformed call instead of repeating it — the contract stays strict; the feedback is what loosens.
- `ApprovalGateway` — kernel-side approval bridge; calls `core:approvals` engine before executing gated operations. Per-tool gating in `dispatchToolCalls` builds the `ApprovalContext` mode from the session's bound operator profile (`boundProfile.approvalMode`, mapped by `approvalModeFor`): unset/`prompt` keeps a human in the loop (default), `auto` auto-approves up to T2, `yolo` all tiers, `deny` blocks above T0. The engine is always consulted (Invariant #4 holds); policy/plane-2 BLOCK stays terminal regardless of mode.
- `ReplayOrchestrator` / `ReplayInferenceProvider` / `ReplayStrategy` — deterministic replay of a session from its event log. `ReplayInferenceProvider` returns recorded responses — no live LLM (Hard Invariant #8).
- `SubagentRunner` / `InSessionSubagentRunner` — runs sub-agent invocations within an active session.
- `StaticAnalysisRunner` / `ProcessStaticAnalysisRunner` — runs static analysis tools and records results as events.
@@ -538,7 +538,8 @@ abstract class SessionOrchestrator(
continue
}
val toolEntries = dispatchToolCalls(sessionId, stageId,
inferenceResult.response.toolCalls, stageConfig, effectives)
inferenceResult.response.toolCalls, stageConfig, effectives,
approvalModeFor(session.state.boundProfile?.approvalMode))
val fatalEntry = toolEntries.firstOrNull { it.content.startsWith("FATAL:") }
if (fatalEntry != null) {
emitProcessResultEvents(sessionId, stageId, stageConfig)
@@ -631,12 +632,33 @@ abstract class SessionOrchestrator(
}
@Suppress("CyclomaticComplexMethod")
// Maps the operator profile's free-text approval_mode onto the engine's [ApprovalMode].
// Unset/unknown falls back to PROMPT so an absent profile keeps the human-in-the-loop
// default; only an explicit auto/yolo opts a session into unattended approval.
private fun approvalModeFor(profileMode: String?): ApprovalMode =
when (profileMode?.trim()?.lowercase()) {
"deny" -> ApprovalMode.DENY
"auto" -> ApprovalMode.AUTO
"yolo" -> ApprovalMode.YOLO
else -> ApprovalMode.PROMPT
}
// On a recoverable tool failure, append the tool's argument schema to the error fed back to the
// model so it can self-correct its next attempt (it already sees its own malformed call in the
// assistant entry above) rather than repeating the same mistake. Kept compact to bound context.
private fun toolArgsHint(tool: Tool?): String =
tool?.let {
"\nThe '${it.name}' tool requires arguments matching this JSON schema — re-issue the " +
"call with corrected arguments: ${it.parametersSchema}"
}.orEmpty()
private suspend fun dispatchToolCalls(
sessionId: SessionId,
stageId: StageId,
toolCalls: List<ToolCallRequest>,
stageConfig: StageConfig,
effectives: RunEffectives,
approvalMode: ApprovalMode,
): List<ContextEntry> {
val executor = effectives.executor ?: return emptyList()
val processResultSlots = stageConfig.produces.filter { it.kind.id == "process_result" }
@@ -765,7 +787,7 @@ abstract class SessionOrchestrator(
val projectId = effectives.policy?.workspaceRoot?.let { ProjectIdentity.of(it.toString()) }
val approvalCtx = ApprovalContext(
identity = ApprovalScopeIdentity(sessionId, stageId, projectId = projectId),
mode = ApprovalMode.PROMPT,
mode = approvalMode,
)
val requestId = ApprovalRequestId(UUID.randomUUID().toString())
val toolPreview = computeToolPreview(toolCall.function.name, parameters)
@@ -938,7 +960,7 @@ abstract class SessionOrchestrator(
?: result.output
is ToolResult.Failure -> {
if (!result.recoverable) "FATAL: ${result.reason}"
else "ERROR: ${result.reason}"
else "ERROR: ${result.reason}${toolArgsHint(tool)}"
}
}
val resultEntry = ContextEntry(