memeval: five minutes again, now that the gate keeps a turn from waiting
The budget was cut to 60s because a five-minute evaluation held the single llama-server slot, and a voice turn arriving mid-evaluation waited behind it. That collision is now solved where it belongs: the background client yields the slot while a turn is in flight. With the gate in place the short budget only truncates a Thinking model mid-synthesis, which costs an observation and saves no latency on any real turn. Kami's call, 2026-08-02. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BFaeSbLMEVG5ey8tejU3y2
This commit is contained in:
@@ -29,11 +29,11 @@ import (
|
||||
// minutes of evaluation was five minutes of a mute assistant.
|
||||
//
|
||||
// The background client now yields the slot while a turn is in flight, so the
|
||||
// collision is handled where it belongs and this is a prompt budget again.
|
||||
// Sixty seconds is long enough for a Thinking model here, and an evaluation cut
|
||||
// off costs nothing, because it is retried at the next interval. Raise it if
|
||||
// observations start truncating.
|
||||
const memoryEvalTimeout = 60 * time.Second
|
||||
// collision is solved where it belongs and this is a prompt budget again. Five
|
||||
// minutes is safe once more, and it is back: 60s truncated a Thinking model
|
||||
// mid-synthesis, which costs an observation for no latency saved. The gate, not
|
||||
// this number, is what keeps a voice turn from waiting.
|
||||
const memoryEvalTimeout = 5 * time.Minute
|
||||
|
||||
// memoryEvalWorker — ticker + evaluator.
|
||||
type memoryEvalWorker struct {
|
||||
|
||||
Reference in New Issue
Block a user