A hung workstation no longer costs the resident model its budget (V-581)
Pair.Complete handed the caller's context to the workstation unchanged, so a remote that accepted the connection and then hung spent the whole turn budget. The fallback then ran on an expired context and the floor returned the deadline error instead of an answer, which broke the turn on the workstation being slow. docs/offload.md rules that out. The remote now gets at most half of a deadline that exists, and a context without a deadline is left to the configured workstation timeout. Pair.Stop and worker.Server.Close both closed their channel after a check-then-close, so two concurrent callers could race and the second close panics. Both are sync.Once now, which is what the doc comments already claimed. config.Load names the environment variables it could not resolve. An unset variable still expands to the empty string, because every block reads that as not configured and CI parses deploy/mavend.json with no secrets present. What was missing is the line telling the operator which capability a forgotten env file just turned off. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
+33
-7
@@ -5,6 +5,7 @@ import (
|
||||
"errors"
|
||||
"log"
|
||||
"net/http"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
)
|
||||
@@ -46,6 +47,7 @@ type Pair struct {
|
||||
interval time.Duration
|
||||
http *http.Client
|
||||
stop chan struct{}
|
||||
stopOnce sync.Once
|
||||
}
|
||||
|
||||
// ErrRemoteUnavailable — the workstation model was required and is not
|
||||
@@ -107,13 +109,11 @@ func (p *Pair) Start(ctx context.Context) {
|
||||
}()
|
||||
}
|
||||
|
||||
// Stop ends the prober. Idempotent.
|
||||
// Stop ends the prober. Idempotent, and safe from two goroutines at once. The
|
||||
// check-then-close it replaced let both callers see an open channel and the
|
||||
// second close panicked, which turned a shutdown race into a crash.
|
||||
func (p *Pair) Stop() {
|
||||
select {
|
||||
case <-p.stop:
|
||||
default:
|
||||
close(p.stop)
|
||||
}
|
||||
p.stopOnce.Do(func() { close(p.stop) })
|
||||
}
|
||||
|
||||
// Available reports whether the workstation will take work right now. It reads
|
||||
@@ -170,7 +170,9 @@ func (p *Pair) Complete(ctx context.Context, r Req) (string, error) {
|
||||
}
|
||||
why := "workstation down"
|
||||
if p.Available() {
|
||||
out, err := p.remote.Complete(ctx, r)
|
||||
rctx, cancel := remoteBudget(ctx)
|
||||
out, err := p.remote.Complete(rctx, r)
|
||||
cancel()
|
||||
if err == nil {
|
||||
log.Print("llm: served by the workstation model")
|
||||
return out, nil
|
||||
@@ -184,6 +186,30 @@ func (p *Pair) Complete(ctx context.Context, r Req) (string, error) {
|
||||
return p.floor.Complete(ctx, r)
|
||||
}
|
||||
|
||||
// remoteBudget bounds the workstation attempt so the floor still has time to
|
||||
// answer. A turn carrying a deadline used to hand the whole of it to the
|
||||
// remote, so a workstation that accepted the connection and then hung ate the
|
||||
// budget and the fallback ran on an already-expired context: the floor
|
||||
// returned the deadline error and the turn broke on the workstation being
|
||||
// slow, which docs/offload.md says must never happen. Half is the split
|
||||
// because both halves have to be able to finish, and there is no reason to
|
||||
// prefer either one when the remote is the part that failed.
|
||||
//
|
||||
// A context with no deadline is left alone. The remote client's own timeout
|
||||
// (workstation.timeout, 90s by default) bounds it there, and shortening that
|
||||
// silently would change the configured budget.
|
||||
func remoteBudget(ctx context.Context) (context.Context, context.CancelFunc) {
|
||||
dl, ok := ctx.Deadline()
|
||||
if !ok {
|
||||
return ctx, func() {}
|
||||
}
|
||||
left := time.Until(dl)
|
||||
if left <= 0 {
|
||||
return ctx, func() {}
|
||||
}
|
||||
return context.WithTimeout(ctx, left/2)
|
||||
}
|
||||
|
||||
// CompleteRemote runs r on the workstation or refuses. It never falls back,
|
||||
// because for a world question the resident 1.7B does not answer worse, it
|
||||
// invents. Callers turn ErrRemoteUnavailable into a named gap.
|
||||
|
||||
Reference in New Issue
Block a user