feat(inference): capability-aware routing for tool-heavy stages

Add CapabilityAwareRoutingStrategy — it hard-filters to providers that declare every
required capability (like FirstAvailable) but ranks the matches by summed required-capability
score; ties keep list order, so it is a strict superset of first-available. The server now
wires it as the routing policy.

The orchestrator augments a stage's required capabilities with ModelCapability.ToolCalling
when the stage grants tools, so tool-heavy stages route to the best tool-calling model rather
than whichever healthy provider comes first.

Scores come from each provider's declared capabilities(); observed per-model reliability
(GET /metrics/tool-reliability) can feed in later as an additional weight.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-29 11:06:20 +00:00
parent 238d353653
commit 82674e85c4
6 changed files with 122 additions and 4 deletions
+1
View File
@@ -14,6 +14,7 @@ CORREX kernel team. This is the integration point for all other `core/` modules.
- `OrchestrationState` / `OrchestrationReducer` (`DefaultOrchestrationReducer`) / `OrchestrationProjector` / `OrchestrationRepository` — standard event-sourcing stack for orchestration state.
- `RetryCoordinator` / `DefaultRetryCoordinator` — manages retry logic per `RetryPolicy`.
- On a recoverable tool failure `dispatchToolCalls` feeds the failing tool's argument schema back into context alongside the error (`toolArgsHint`), so the model self-corrects a malformed call instead of repeating it — the contract stays strict; the feedback is what loosens.
- Stages that grant tools (`allowedTools` non-empty) request `ModelCapability.ToolCalling` on top of their declared capabilities when routing, so the capability-aware strategy steers them to the best tool-calling model.
- `ApprovalGateway` — kernel-side approval bridge; calls `core:approvals` engine before executing gated operations. Per-tool gating in `dispatchToolCalls` builds the `ApprovalContext` mode from the session's bound operator profile (`boundProfile.approvalMode`, mapped by `approvalModeFor`): unset/`prompt` keeps a human in the loop (default), `auto` auto-approves up to T2, `yolo` all tiers, `deny` blocks above T0. The engine is always consulted (Invariant #4 holds); policy/plane-2 BLOCK stays terminal regardless of mode.
- `ReplayOrchestrator` / `ReplayInferenceProvider` / `ReplayStrategy` — deterministic replay of a session from its event log. `ReplayInferenceProvider` returns recorded responses — no live LLM (Hard Invariant #8).
- `SubagentRunner` / `InSessionSubagentRunner` — runs sub-agent invocations within an active session.
@@ -106,6 +106,7 @@ import com.correx.core.inference.ResponseFormat
import com.correx.core.inference.Tokenizer
import com.correx.core.inference.ToolCallRequest
import com.correx.core.inference.ToolDefinition
import com.correx.core.inference.ModelCapability
import com.correx.core.sessions.ApprovalMode
import com.correx.core.inference.ToolFunction
import com.correx.core.kernel.execution.WorkflowResult
@@ -1753,7 +1754,12 @@ abstract class SessionOrchestrator(
responseFormat: ResponseFormat = ResponseFormat.Text,
effectives: RunEffectives = RunEffectives(toolRegistry, toolExecutor, workspacePolicy),
): InferenceResult {
val provider = inferenceRouter.route(stageId, stageConfig.requiredCapabilities, stageConfig.modelId)
// A stage that grants tools needs a tool-calling model, so request ToolCalling on top of any
// declared capabilities — the capability-aware strategy then ranks eligible providers by their
// ToolCalling score and routes the stage to the best tool-caller.
val requiredCapabilities = stageConfig.requiredCapabilities +
if (stageConfig.allowedTools.isNotEmpty()) setOf(ModelCapability.ToolCalling) else emptySet()
val provider = inferenceRouter.route(stageId, requiredCapabilities, stageConfig.modelId)
log.debug(
"[Orchestrator] inference session={} stage={} provider={} timeoutMs={}",
sessionId.value, stageId.value, provider.id.value, timeoutMs,