the seam names which model served the turn (V-483)
The transition lines said the card was free at 11:27. They did not say which side answered the turn at 13:24, so an offloaded turn and a floor turn read the same in the log, and QA verifying the offload had nothing to read. One line per model call, naming the side, and naming why when it was the floor: the workstation was down, or it accepted and then failed mid-request. Two lines per turn, since routing and phrasing are separate calls. Silent still means silent to him. He is not told which model phrased his reply.
This commit is contained in:
+10
-2
@@ -159,21 +159,28 @@ func (p *Pair) set(up bool) {
|
||||
// admission answer is a cache and can be one interval out of date, so an error
|
||||
// here is expected rather than exceptional.
|
||||
//
|
||||
// This is the silent half of the degradation rule. It must be indistinguishable
|
||||
// from today's behaviour when the workstation is down.
|
||||
// This is the silent half of the degradation rule: he is not told which model
|
||||
// phrased his reply. The log is told, one line per call, because the seam was
|
||||
// otherwise unreadable after the fact — the transition lines say the card was
|
||||
// free at 11:27, not which side answered the turn at 13:24. QA had no way to
|
||||
// tell an offloaded turn from a floor one.
|
||||
func (p *Pair) Complete(ctx context.Context, r Req) (string, error) {
|
||||
if p.floor == nil {
|
||||
return "", ErrNoFloor
|
||||
}
|
||||
why := "workstation down"
|
||||
if p.Available() {
|
||||
out, err := p.remote.Complete(ctx, r)
|
||||
if err == nil {
|
||||
log.Print("llm: served by the workstation model")
|
||||
return out, nil
|
||||
}
|
||||
// The cached answer was wrong. Correct it now rather than sending the
|
||||
// next request into the same hole, then fall back.
|
||||
p.set(false)
|
||||
why = "workstation failed mid-request"
|
||||
}
|
||||
log.Printf("llm: served by the resident model (%s)", why)
|
||||
return p.floor.Complete(ctx, r)
|
||||
}
|
||||
|
||||
@@ -189,5 +196,6 @@ func (p *Pair) CompleteRemote(ctx context.Context, r Req) (string, error) {
|
||||
p.set(false)
|
||||
return "", errors.Join(ErrRemoteUnavailable, err)
|
||||
}
|
||||
log.Print("llm: served by the workstation model, no floor for this caller")
|
||||
return out, nil
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user