feat(health): gate routing event-driven on connection drop, not just periodic poll (#300)
HealthMonitor's periodic poll took ~18s to notice a dead provider — long after a retry had already re-selected it and died. Health needs to GATE routing, not just log reactively. - InferenceRouter.reportFailure(providerId, reason): new default-no-op method; DefaultInferenceRouter implements it by writing Unavailable straight into its health cache, bypassing healthCheck()/TTL entirely. - SessionOrchestrator.runInference: on a connection-level exception (ConnectException/SocketException/IOException or a message matching "prematurely closed"/"connection refused"/"connection reset"), call reportFailure immediately so the very next route() call — the retry driven by #299 — sees the provider as down right away instead of re-selecting it and dying again. Pairs with #299: routing now (1) reacts to a connection drop instantly and (2) waits/backs off for the capability to recover before declaring terminal. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GeyGFXczJb8RUWGBKmkm6G
This commit is contained in:
@@ -59,6 +59,16 @@ class DefaultInferenceRouter(
|
||||
fresh
|
||||
}
|
||||
|
||||
// Bypasses healthCheck()/TTL entirely — writes Unavailable straight into the cache so the very
|
||||
// next route() call gates on it, closing the ~18s reactive-poll lag (#300). A later cache-TTL
|
||||
// expiry or the bounded-wait re-check in route() will naturally pick the provider back up once
|
||||
// its own healthCheck() reports healthy again.
|
||||
override suspend fun reportFailure(providerId: ProviderId, reason: String) {
|
||||
lockFor(providerId).withLock {
|
||||
mapMutex.withLock { cache[providerId] = HealthEntry(ProviderHealth.Unavailable(reason), timeSource.markNow()) }
|
||||
}
|
||||
}
|
||||
|
||||
override suspend fun route(stageId: StageId, requiredCapabilities: Set<ModelCapability>): InferenceProvider {
|
||||
val candidates = requiredCapabilities
|
||||
.flatMap { registry.resolve(it) }
|
||||
|
||||
@@ -39,6 +39,14 @@ interface InferenceRouter {
|
||||
requiredCapabilities: Set<ModelCapability>,
|
||||
modelId: String?,
|
||||
): InferenceProvider = route(stageId, requiredCapabilities)
|
||||
|
||||
/**
|
||||
* Event-driven health gate: called the moment a connection-level failure is observed on
|
||||
* [providerId] (e.g. mid-request connection drop), so the NEXT route() call sees it as
|
||||
* unavailable immediately instead of waiting for the next periodic health poll/cache TTL to
|
||||
* catch up. Default no-op for routers that don't cache health.
|
||||
*/
|
||||
suspend fun reportFailure(providerId: com.correx.core.events.types.ProviderId, reason: String) = Unit
|
||||
}
|
||||
|
||||
class NoEligibleProviderException(
|
||||
|
||||
Reference in New Issue
Block a user