mavgpud reverse-proxied every path to llama-server with no authentication on
a LAN port. Any client on the network could submit model work, hold the model
resident by touching the idle clock, and read /slots, which returns the
prompts of whoever else was using the card.
It now reads a bearer token from token_file and requires it on every request,
/health included: /health reports whether the card is loaded and free, which
is what someone deciding to take it would ask. A listen address reachable
from the network with no token is a startup failure rather than a downgrade
to loopback. homesrv is the client and it is on the LAN, so a loopback
default would look safe and take the model arm down.
Beyond the token: an allowlist of the five paths Maven calls, so a leaked
token buys the model API and not llama-server's admin surface; a body cap and
an in-flight cap on the proxy; and header and idle timeouts on the server.
No read or write timeout — a completion on this card legitimately takes
minutes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ESv8hqNPseYt1CnotZpqDz
Owner's call. E4B is 4.2GB against 6.7GB plus a 0.86GB draft, so with CW2
resident the card holds 5.8GB of 16GB instead of 9.2GB.
Measured against a same-session 12B control on the 96-case fixture: 83.3% full
against 84.4%, 89.6% intent-only against 91.7%, destination 19/33 against
23/33, p50 294ms against 344ms. Destination is the column that moved. E4B names
nothing where the 12B names recall or calendar, which walks the whole chain
rather than answering wrong.
MTP is gone with the 12B and cannot come back. It is a separate gguf of
architecture gemma4-assistant with nextn_predict_layers=4, and the only one on
disk is trained against the 12B's hidden states. Neither target gguf carries
nextn tensors, so neither self-speculates.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
serve.py lived only on workpc, which was fine while systemd launched it and is
not fine now that mavgpud does. Two endpoints and no framework: /health answers
503 until the model is loaded, /transcribe takes raw PCM and returns
{"text","confidence"}.
The unit carries CW2_TOKEN through EnvironmentFile and the child inherits it,
so the token is never a flag value.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ptwopxyo3Z2kwFckHkLvN
Owner's call, 02-08-2026. Without it the 12B spends the reply budget on
reasoning tokens and answers empty at low max_tokens. Verified on the box:
"Столица Франции?" now answers "Париж" with no reasoning_content.
make build now catches a broken supervisor on homesrv. deploy/mavgpud.json
carries the owner's gemma-4-12b line with the MTP draft model, passed to
llama-server untouched. The unit is a systemd user unit because sudo on the
workstation wants a password; lingering is the one command left to the owner.
claude