kami
12ff2e97b0
Six QA plans (docs/qa/QA-*.md) pinning observable evidence for the shipped tracks: research egress + batch source-approval (D), architect contradiction (B§4), llama health probe (A§4), idea promotion (E), reviewer static-first filter (B§5, partial), and the brief-echo gate ARM-IT plan (C-A1) — the latter is the go/no-go for arming the gate in production (it breaks the planner if the model can't emit a parseable brief_echo). Plus the env: docs/qa/ENV.md runbook + docs/qa/README.md index, and scripts/qa/ (sync-config.sh, searxng-up/down.sh with the JSON-format gotcha, README). Scripts are set -euo pipefail, syntax-checked, executable. Authored from the repo build/config (server :apps:server:run, CLI :apps:cli:run, tui-go GOTOOLCHAIN=auto, sample-config.toml). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
3.4 KiB
3.4 KiB
QA Plan: LLAMA_SERVER health probe — 266bbf0 64a90e7 9eef936
Status: DRAFT
Run date / operator:
BACKLOG item: §A-§4 "LLAMA_SERVER health probe" — probe 266bbf0, registration 64a90e7, managed-path coverage 9eef936.
Preconditions
- server build/branch:
feat/backlog-burndown. - llama-server + model: a real llama-server reachable at the configured URL. The probe registers when either a static
[[providers]](typellamacpp,url) or a managed[[models]]entry exists (managed URL =[models].host:[models].port). Verify your config has one. - external deps: none.
- config synced:
[health] enabled = true; note[health] interval_ms(the probe cadence — you wait one interval to see an edge). - fixtures/seed: ability to stop/start the llama-server process (managed: it dies with the server; for the kill/restore test use the static
[[providers]]path so you can stop llama-server independently of correx).
Acceptance gate (one sentence)
The
LLAMA_SERVERsubject is monitored only when a model is configured, reports HEALTHY while the server answers/health, and emits a liveness DEGRADED edge when the server stops — and a restored edge when it returns.
Checks
| # | Action | Expected observable evidence | Result |
|---|---|---|---|
| 1 | Start correx with a llama-server configured and reachable. Wait one interval_ms. |
correx health (and the health projection) lists LLAMA_SERVER as HEALTHY. No spurious DEGRADED. |
|
| 2 | Stop the llama-server (static-provider path: kill the external process). Wait one interval_ms. |
A health degraded edge event for subject LLAMA_SERVER, metric = "liveness", detail "llama server unreachable" appears in the system-health log (correx events on the SYSTEM session / correx health). |
|
| 3 | Restart the llama-server. Wait one interval_ms. |
A restored edge event for LLAMA_SERVER; correx health shows HEALTHY again. Edges are recorded once (edge-triggered, not per-tick). |
|
| 4 | Start correx on a box with no [[providers]] and no [[models]]. |
LLAMA_SERVER is absent from the health subjects — the probe is not registered (no spurious DEGRADED). Absence is the evidence. |
|
| 5 | correx replay the SYSTEM health session. |
The degraded/restored edges replay deterministically from the log (the rolling window is live-only per Invariant #8 and does not affect replay). |
Evidence sources: correx health CLI, the health projection events, server logs.
Out of scope (explicitly NOT covered this pass)
- tokens/sec degradation — the
/healthendpoint is reachability-only;HttpLlamaLivenessClientnever populatestokensPerSecond, so the throughput-collapse branch ofLlamaServerHealthProbeis dormant until a tps telemetry feed exists. Liveness only this pass. (Refiled in BACKLOG §A-§4 remainder.) - The live health TUI pane (deferred; system-health events are not yet on the TUI wire).
Disposition
- PASS → move the §A-§4 LLAMA_SERVER bullet to
RETRO.md(liveness verified) with this run date; keep the tokens/sec remainder open in BACKLOG. Status: PASSED. - FAIL → numbered findings. A likely miss: the managed
[[models]]path not registering — confirm9eef936is in the build. Status: FAILED until green.