2b97bac51e01f87486ec5b43a90beac0eb03e4da
The lifecycle rule from Vikunja #488. Not on demand, because a 7-14B takes tens of seconds to load and a world question would meet a gap every time the card had been quiet. Not always on, because that is what holds the card. /health is answered locally and always, so Maven's prober costs nothing and works while the model is down. Everything else is reverse-proxied to llama-server, which is what makes the idle window measurable at all. Yielding is checked before starting, and both transitions are damped by a poll streak so a short-lived rocm process cannot evict the model.
Description
No description provided
Languages
Go
97.1%
HTML
0.9%
Shell
0.6%
CSS
0.5%
Makefile
0.3%
Other
0.6%