feat(health): gate routing event-driven on connection drop, not just periodic poll (#300)

HealthMonitor's periodic poll took ~18s to notice a dead provider — long after a retry had already re-selected it and died. Health needs to GATE routing, not just log reactively. - InferenceRouter.reportFailure(providerId, reason): new default-no-op method; DefaultInferenceRouter implements it by writing Unavailable straight into its health cache, bypassing healthCheck()/TTL entirely. - SessionOrchestrator.runInference: on a connection-level exception (ConnectException/SocketException/IOException or a message matching "prematurely closed"/"connection refused"/"connection reset"), call reportFailure immediately so the very next route() call — the retry driven by #299 — sees the provider as down right away instead of re-selecting it and dying again. Pairs with #299: routing now (1) reacts to a connection drop instantly and (2) waits/backs off for the capability to recover before declaring terminal. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GeyGFXczJb8RUWGBKmkm6G
2026-07-21 02:01:35 +04:00
parent c5289420a1
commit bf36252736
5 changed files with 107 additions and 0 deletions
@@ -213,4 +213,28 @@ class DefaultInferenceRouterTest {
            router.route(stage, setOf(ModelCapability.ToolCalling))
        }
    }
+
+    // ── event-driven health gating (#300) ─────────────────────────────────────
+
+    @Test
+    fun `reportFailure gates the very next route call without waiting for a health poll`(): Unit = runBlocking {
+        var healthCheckCount = 0
+        val alwaysClaimsHealthy = object : InferenceProvider by provider("a", ModelCapability.General) {
+            override suspend fun healthCheck(): ProviderHealth {
+                healthCheckCount++
+                return ProviderHealth.Healthy // the provider's own health probe hasn't caught up yet
+            }
+        }
+        val backup = provider("b", ModelCapability.General)
+        val router = DefaultInferenceRouter(registryOf(alwaysClaimsHealthy, backup), firstStrategy())
+
+        // Sanity: before reportFailure, routes to the first (still "healthy") provider.
+        assertSame(alwaysClaimsHealthy, router.route(stage, setOf(ModelCapability.General)))
+
+        router.reportFailure(alwaysClaimsHealthy.id, "connection reset")
+
+        // Immediately after — no delay, no waiting for the next poll — routing must avoid it.
+        val result = router.route(stage, setOf(ModelCapability.General))
+        assertSame(backup, result)
+    }
 }