memeval: five minutes again, now that the gate keeps a turn from waiting

The budget was cut to 60s because a five-minute evaluation held the single
llama-server slot, and a voice turn arriving mid-evaluation waited behind it.
That collision is now solved where it belongs: the background client yields the
slot while a turn is in flight.

With the gate in place the short budget only truncates a Thinking model
mid-synthesis, which costs an observation and saves no latency on any real turn.
Kami's call, 2026-08-02.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BFaeSbLMEVG5ey8tejU3y2
This commit is contained in:
kami
2026-08-02 02:16:45 +04:00
parent a654b0126f
commit a103708a08
+5 -5
View File
@@ -29,11 +29,11 @@ import (
// minutes of evaluation was five minutes of a mute assistant.
//
// The background client now yields the slot while a turn is in flight, so the
// collision is handled where it belongs and this is a prompt budget again.
// Sixty seconds is long enough for a Thinking model here, and an evaluation cut
// off costs nothing, because it is retried at the next interval. Raise it if
// observations start truncating.
const memoryEvalTimeout = 60 * time.Second
// collision is solved where it belongs and this is a prompt budget again. Five
// minutes is safe once more, and it is back: 60s truncated a Thinking model
// mid-synthesis, which costs an observation for no latency saved. The gate, not
// this number, is what keeps a voice turn from waiting.
const memoryEvalTimeout = 5 * time.Minute
// memoryEvalWorker — ticker + evaluator.
type memoryEvalWorker struct {