feat(inference): capability-aware routing for tool-heavy stages
Add CapabilityAwareRoutingStrategy — it hard-filters to providers that declare every required capability (like FirstAvailable) but ranks the matches by summed required-capability score; ties keep list order, so it is a strict superset of first-available. The server now wires it as the routing policy. The orchestrator augments a stage's required capabilities with ModelCapability.ToolCalling when the stage grants tools, so tool-heavy stages route to the best tool-calling model rather than whichever healthy provider comes first. Scores come from each provider's declared capabilities(); observed per-model reliability (GET /metrics/tool-reliability) can feed in later as an additional weight. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -14,6 +14,7 @@ CORREX kernel team. This is the integration point for all other `core/` modules.
|
||||
- `OrchestrationState` / `OrchestrationReducer` (`DefaultOrchestrationReducer`) / `OrchestrationProjector` / `OrchestrationRepository` — standard event-sourcing stack for orchestration state.
|
||||
- `RetryCoordinator` / `DefaultRetryCoordinator` — manages retry logic per `RetryPolicy`.
|
||||
- On a recoverable tool failure `dispatchToolCalls` feeds the failing tool's argument schema back into context alongside the error (`toolArgsHint`), so the model self-corrects a malformed call instead of repeating it — the contract stays strict; the feedback is what loosens.
|
||||
- Stages that grant tools (`allowedTools` non-empty) request `ModelCapability.ToolCalling` on top of their declared capabilities when routing, so the capability-aware strategy steers them to the best tool-calling model.
|
||||
- `ApprovalGateway` — kernel-side approval bridge; calls `core:approvals` engine before executing gated operations. Per-tool gating in `dispatchToolCalls` builds the `ApprovalContext` mode from the session's bound operator profile (`boundProfile.approvalMode`, mapped by `approvalModeFor`): unset/`prompt` keeps a human in the loop (default), `auto` auto-approves up to T2, `yolo` all tiers, `deny` blocks above T0. The engine is always consulted (Invariant #4 holds); policy/plane-2 BLOCK stays terminal regardless of mode.
|
||||
- `ReplayOrchestrator` / `ReplayInferenceProvider` / `ReplayStrategy` — deterministic replay of a session from its event log. `ReplayInferenceProvider` returns recorded responses — no live LLM (Hard Invariant #8).
|
||||
- `SubagentRunner` / `InSessionSubagentRunner` — runs sub-agent invocations within an active session.
|
||||
|
||||
+7
-1
@@ -106,6 +106,7 @@ import com.correx.core.inference.ResponseFormat
|
||||
import com.correx.core.inference.Tokenizer
|
||||
import com.correx.core.inference.ToolCallRequest
|
||||
import com.correx.core.inference.ToolDefinition
|
||||
import com.correx.core.inference.ModelCapability
|
||||
import com.correx.core.sessions.ApprovalMode
|
||||
import com.correx.core.inference.ToolFunction
|
||||
import com.correx.core.kernel.execution.WorkflowResult
|
||||
@@ -1753,7 +1754,12 @@ abstract class SessionOrchestrator(
|
||||
responseFormat: ResponseFormat = ResponseFormat.Text,
|
||||
effectives: RunEffectives = RunEffectives(toolRegistry, toolExecutor, workspacePolicy),
|
||||
): InferenceResult {
|
||||
val provider = inferenceRouter.route(stageId, stageConfig.requiredCapabilities, stageConfig.modelId)
|
||||
// A stage that grants tools needs a tool-calling model, so request ToolCalling on top of any
|
||||
// declared capabilities — the capability-aware strategy then ranks eligible providers by their
|
||||
// ToolCalling score and routes the stage to the best tool-caller.
|
||||
val requiredCapabilities = stageConfig.requiredCapabilities +
|
||||
if (stageConfig.allowedTools.isNotEmpty()) setOf(ModelCapability.ToolCalling) else emptySet()
|
||||
val provider = inferenceRouter.route(stageId, requiredCapabilities, stageConfig.modelId)
|
||||
log.debug(
|
||||
"[Orchestrator] inference session={} stage={} provider={} timeoutMs={}",
|
||||
sessionId.value, stageId.value, provider.id.value, timeoutMs,
|
||||
|
||||
Reference in New Issue
Block a user