feat(server): llama-server health probe
Second HealthProbe (after disk-watermark): liveness via GET /health on the configured llama base URL + tokens/sec degradation watch derived from recorded InferenceCompletedEvents (windowed average vs configurable threshold, opt-in via llamaTpsWarnBelow). Plugs into the existing HealthMonitor edge-detection loop; no new events. Observability spec §4.
This commit is contained in:
@@ -24,6 +24,11 @@ data class CorrexConfig(
|
||||
* the on-disk footprint every [intervalMs] and records a degraded/restored event on the edge.
|
||||
* [diskWarnBytes] is the absolute event-log + CAS size that trips a warning; [diskWarnGrowthBytesPerMin]
|
||||
* is the growth rate that warns log compaction is becoming non-theoretical. Defaults suit a homelab.
|
||||
*
|
||||
* [llamaLivenessTimeoutMs] bounds the GET /health ping to the llama-server; a connection error or
|
||||
* timeout within this window is reported as DEGRADED (liveness). [llamaTpsWarnBelow] is the
|
||||
* tokens/sec floor below which the probe reports degradation (KV-cache pressure, thermal throttle);
|
||||
* 0.0 disables the degradation dimension (no baseline to compare against on a fresh install).
|
||||
*/
|
||||
@Serializable
|
||||
data class HealthConfig(
|
||||
@@ -31,6 +36,8 @@ data class HealthConfig(
|
||||
val intervalMs: Long = 30_000,
|
||||
val diskWarnBytes: Long = 5_000_000_000,
|
||||
val diskWarnGrowthBytesPerMin: Long = 50_000_000,
|
||||
val llamaLivenessTimeoutMs: Long = 5_000,
|
||||
val llamaTpsWarnBelow: Double = 0.0,
|
||||
)
|
||||
|
||||
/**
|
||||
|
||||
Reference in New Issue
Block a user