Every shape of wrong credential gets a case: no header, wrong token, a prefix
of the token, the token with no scheme, and Basic. Plus the two the allowlist
exists for, /slots and its save action, and the caps.
The readiness test now posts to /v1/chat/completions. The allowlist sits in
front of the readiness check and answers 405 to a method mavgpud never
serves, so the old GET measured the allowlist rather than the 503.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ESv8hqNPseYt1CnotZpqDz
mavgpud reverse-proxied every path to llama-server with no authentication on
a LAN port. Any client on the network could submit model work, hold the model
resident by touching the idle clock, and read /slots, which returns the
prompts of whoever else was using the card.
It now reads a bearer token from token_file and requires it on every request,
/health included: /health reports whether the card is loaded and free, which
is what someone deciding to take it would ask. A listen address reachable
from the network with no token is a startup failure rather than a downgrade
to loopback. homesrv is the client and it is on the LAN, so a loopback
default would look safe and take the model arm down.
Beyond the token: an allowlist of the five paths Maven calls, so a leaked
token buys the model API and not llama-server's admin surface; a body cap and
an in-flight cap on the proxy; and header and idle timeouts on the server.
No read or write timeout — a completion on this card legitimately takes
minutes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ESv8hqNPseYt1CnotZpqDz
CW2 is a ROCm process, so it registers on the KFD like any contender. Running
it as its own systemd unit made mavgpud yield llama-server to it every few
seconds. The gemma-4-12b arm was down for eight minutes on 2026-08-09 and
routing had silently fallen back to the resident model.
So mavgpud takes an `stt` block and runs the transcriber itself. `foreign` now
excludes every child rather than one pid, which is the fix. Yielding is all or
nothing, because a job that wants the card wants all of it. Idle unloading
stays llama-server's alone: CW2 holds 1.6GB and unloading it would only send
the next voice turn to the homesrv floor.
Maven still talks to the transcriber directly on 8081. There is no proxy,
because with no idle timer there is nothing for one to measure.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
llama-server aborts inside its own static teardown on SIGTERM — the
handler calls exit(), stream_session_manager's destructor throws, and the
process dies "signal: aborted (core dumped)". mavgpud sends that signal on
every eviction, so a routine yield wrote a multi-gigabyte core into
systemd-coredump and logged the same line a real crash would.
LimitCORE=0 in the unit stops the disk cost. A yielding flag, set by stop
and cleared by start, makes the log distinguish the two: only an exit we
did not ask for is still reported as an exit.
Not filed upstream. Searched ggml-org/llama.cpp for
"ggml_uncaught_exception" with SIGTERM and for stream_session_manager and
found nothing matching, so the issue still wants writing — by someone with
an account on that tracker, which is why it is not in this commit.
The fixtures are the live numbers sampled from the box on 02-08-2026, where the
CPT run held 12.8GB of 16 as proc/478104/vram_35881.
The cases that matter are the ones where a mistake is silent: our own
llama-server counting as a contender, an unreadable card reading as free, and
/health hanging or proxying into a closed port instead of answering 503.
The lifecycle rule from Vikunja #488. Not on demand, because a 7-14B takes tens
of seconds to load and a world question would meet a gap every time the card
had been quiet. Not always on, because that is what holds the card.
/health is answered locally and always, so Maven's prober costs nothing and
works while the model is down. Everything else is reverse-proxied to
llama-server, which is what makes the idle window measurable at all.
Yielding is checked before starting, and both transitions are damped by a poll
streak so a short-lived rocm process cannot evict the model.
The workstation cannot keep a 7-14B resident: it would hold 16GB against the
owner's CPT runs, Correx and the manga-recap pipeline. So the process that
stays up costs no VRAM and the model comes and goes under it.
Contention is detected by presence on the KFD, not by a VRAM threshold. A ROCm
process registers under /sys/class/kfd/kfd/proc when it initialises HIP, well
before it allocates, so we see a contender during its startup instead of after
it has already lost an allocation race. rocm-smi is not installed on that box
and a per-second subprocess would get tuned down until useless, so this reads
sysfs and forks nothing.
Free VRAM is read only to decide whether to start. It is never a reason to
stop: by the time free VRAM has dropped, the other job has already failed.
claude