llama-server holds 7.9GB RSS for a 1.1GB model, and its startup log goes nowhere #103

2026-08-03T21:20:29+02:00

claude commented

2026-08-03 21:20:29 +02:00

Closes Vikunja #499.

Acceptance criteria and quality gate are in TASK.md on this branch.
Review the review, not the diff — leave comments and the agent will apply them via task start 499.

Closes Vikunja #499. Acceptance criteria and quality gate are in `TASK.md` on this branch. Review the review, not the diff — leave comments and the agent will apply them via `task start 499`.

claude added 2 commits 2026-08-03 21:20:29 +02:00

phraser: forward llama-server's output to mavend's log (V-499) f229795cea

mavend scraped the child's stderr for the listen line and threw every
other line away, and never piped its stdout at all. Nothing about the
resident model's memory was diagnosable from a running box: no buffer
sizes, no KV-cache layout, no offload lines, no prompt-cache limit.

Both streams now share one pipe and every line lands in mavend's log
with a `llama:` prefix. The last 12 startup lines are also kept and go
into the error when the server dies before it listens, because bare
"EOF" never named which allocation it choked on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

phraser: cap llama-server's prompt cache at 512 MiB (V-499) f9b2391a8b

The forwarded log named the cause in one line: the prompt cache limit
defaults to 8192 MiB. llama-server saves the full KV state of every idle
slot it evicts, 112 kiB per token, so RSS climbed about 170MB per
distinct prompt until the deployed server held 7.9GB for a 1.1GB model.

Measured on homesrv today, uncapped versus `--cache-ram 512`: RSS
plateaus at 932MB from the fourth distinct prompt instead of climbing.
The task's leading guess was wrong. `-ngl 99` costs almost no RSS,
because RADV keeps device memory outside the process. Numbers and method
in docs/evals/2026-08-03-llama-prompt-cache.md.

`-c 4096` is untouched. The knob is `phraser.cache_ram_mib`, unset means
512, negative passes no flag for a llama-server too old to know it.

The deploy still runs the old image, so the box keeps its 8 GiB default
until mavend is rebuilt.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

kami added 1 commit 2026-08-03 21:28:39 +02:00

docs: record the #499 deploy (V-499) 58051b5af1

kami merged commit 767748720a into task/495-bug-x-escapes-the-personal-boundary-and

2026-08-03 21:30:37 +02:00

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: kami/Maven#103