feat(server): llama-server health probe

Second HealthProbe (after disk-watermark): liveness via GET /health on the
configured llama base URL + tokens/sec degradation watch derived from recorded
InferenceCompletedEvents (windowed average vs configurable threshold, opt-in via
llamaTpsWarnBelow). Plugs into the existing HealthMonitor edge-detection loop;
no new events. Observability spec §4.
This commit is contained in:
2026-06-14 18:26:09 +04:00
parent b1c0f88dee
commit aaf896d8de
4 changed files with 381 additions and 0 deletions
@@ -24,6 +24,11 @@ data class CorrexConfig(
* the on-disk footprint every [intervalMs] and records a degraded/restored event on the edge.
* [diskWarnBytes] is the absolute event-log + CAS size that trips a warning; [diskWarnGrowthBytesPerMin]
* is the growth rate that warns log compaction is becoming non-theoretical. Defaults suit a homelab.
*
* [llamaLivenessTimeoutMs] bounds the GET /health ping to the llama-server; a connection error or
* timeout within this window is reported as DEGRADED (liveness). [llamaTpsWarnBelow] is the
* tokens/sec floor below which the probe reports degradation (KV-cache pressure, thermal throttle);
* 0.0 disables the degradation dimension (no baseline to compare against on a fresh install).
*/
@Serializable
data class HealthConfig(
@@ -31,6 +36,8 @@ data class HealthConfig(
val intervalMs: Long = 30_000,
val diskWarnBytes: Long = 5_000_000_000,
val diskWarnGrowthBytesPerMin: Long = 50_000_000,
val llamaLivenessTimeoutMs: Long = 5_000,
val llamaTpsWarnBelow: Double = 0.0,
)
/**