Task/495 bug x escapes the personal boundary and #104

2026-08-03T21:30:53+02:00

kami commented

2026-08-03 21:30:53 +02:00

No description provided.

kami added 4 commits 2026-08-03 21:30:54 +02:00

phraser: forward llama-server's output to mavend's log (V-499) f229795cea

mavend scraped the child's stderr for the listen line and threw every
other line away, and never piped its stdout at all. Nothing about the
resident model's memory was diagnosable from a running box: no buffer
sizes, no KV-cache layout, no offload lines, no prompt-cache limit.

Both streams now share one pipe and every line lands in mavend's log
with a `llama:` prefix. The last 12 startup lines are also kept and go
into the error when the server dies before it listens, because bare
"EOF" never named which allocation it choked on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

phraser: cap llama-server's prompt cache at 512 MiB (V-499) f9b2391a8b

The forwarded log named the cause in one line: the prompt cache limit
defaults to 8192 MiB. llama-server saves the full KV state of every idle
slot it evicts, 112 kiB per token, so RSS climbed about 170MB per
distinct prompt until the deployed server held 7.9GB for a 1.1GB model.

Measured on homesrv today, uncapped versus `--cache-ram 512`: RSS
plateaus at 932MB from the fourth distinct prompt instead of climbing.
The task's leading guess was wrong. `-ngl 99` costs almost no RSS,
because RADV keeps device memory outside the process. Numbers and method
in docs/evals/2026-08-03-llama-prompt-cache.md.

`-c 4096` is untouched. The knob is `phraser.cache_ram_mib`, unset means
512, negative passes no flag for a llama-server too old to know it.

The deploy still runs the old image, so the box keeps its 8 GiB default
until mavend is rebuilt.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

docs: record the #499 deploy (V-499) 58051b5af1

Merge pull request 'llama-server holds 7.9GB RSS for a 1.1GB model, and its startup log goes nowhere' (#103 ) from task/499-llama-server-holds-7-9gb-rss-for-a-1-1gb into task/495-bug-x-escapes-the-personal-boundary-and 767748720a

Reviewed-on: #103

kami merged commit 197897516e into master

2026-08-03 21:30:57 +02:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: kami/Maven#104