Two halves of the same failure, live on run 5.
F38: the phase brief named .orchestra/research.json and described its contents
in prose, never its schema. The agent guessed dead_ends as strings where the
decoder wants {tried, why_failed} objects. The brief now carries the shape, and
a test decodes each documented shape with the same function the worker uses, so
a struct change that is not mirrored fails the build.
F39: the local artifact check refused the request through recordError alone.
answerRefusedPhase only ran on a coordinator 409, so a decode failure told the
agent nothing. The session sat at a boundary rewriting nothing, which is the
silent-loop shape the comment above that block warns about, reached by the one
path with no delivery. Both local refusals now reach the agent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011xsXyr5J1RACo71YeKG3Pu
Two gaps F33 left behind.
F34: the renewal gate exempts a lease with no baseline, so a pane that opened
and never started got a full free renewal period. That is the exact case the
gate exists to catch, and it happened live at 01:33:37: the stuck run 5 lease
renewed to 22:03 on a pane that had not moved since 01:13. Baseline the
progress hash at launch, where the pane is already being read.
F35: nothing could re-poke a live pane. Orchestra can put text in an editor and
be wrong about whether it landed, and the only recovery was to destroy the
lease and wait out expiry, roughly an hour. The new "resubmit" action presses
Enter on text Orchestra itself submitted. It changes no lifecycle state so it
appends no event, and it is fenced like an approval: a live worker-owned pane
at the capture revision the operator was looking at.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011xsXyr5J1RACo71YeKG3Pu
A release transaction that never reached anchor_pushed has no artifact: tx.Ref
is empty and no successor can pick anything up. The event handler still kept
its session mapping alive on TaskReleased/TaskBlocked/TaskFailed, so once the
pane was gone the mapping was immortal. health() reports ActiveTask straight
out of w.sessions, so the coordinator saw the harness as permanently busy and
never leased to it again. It produced no log line at all.
Found live on workpc-claude, stuck at phase "prepared" behind a rejected
handoff artifact. Freeing it needed hand surgery on the worker's state file.
Keep the mapping only while an anchor actually exists. Drop the transaction
with it, since nothing can advance it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011xsXyr5J1RACo71YeKG3Pu
F25. rotationTick returned early for claude before reaching federatedTurn,
which has one call site below that return. On the harness both burn-in runs
used, no phase request could ever be read and every human decision recorded
against a live session went undelivered. Claude still skips the occupancy
state machine below, because it owns its context rollover through the
installed hook. A turn boundary is not a rotation.
F26. The phase brief listed every domain-legal target, so run 4's frame
session read "research, implement" and asked for implement, which the
project's path refuses. The path is Orchestra's to know: the brief now names
one step and says a wrong target comes back with the right one.
F27. A refused request only reached recordError, leaving the agent to rewrite
the same rejected file forever with nothing telling it why. federation.
StatusError makes a 409 classifiable, and the refusal is delivered through
sendPrompt under the F20 guarantee. A transport failure is not an answer: the
request survives and the agent is told nothing.
The F25 regression test fails against the unfixed rotationTick.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011xsXyr5J1RACo71YeKG3Pu
The brief told the agent to ask for a phase change and never carried the
asking. The agent asked in prose, no code represented the request, and the
session idled until its lease expired. That is what failed run 3.
F21. The agent asks with .orchestra/phase-request.json, and seals
research.json or plan.json where the phase it is leaving produces one. At a
verified turn boundary the worker checks the phase belief, the transition and
the artifact, then calls the coordinator with its lease epoch and a derived
operation id. AdvanceWorkPhase is unchanged, so a request cannot reach a move
the operator surface could not also make. Redelivery is idempotent.
F22. A session now records the phase it was launched to run. One that no
longer matches its task rotates with reason phase_changed, whether this worker
asked for the change or an operator made it.
F20. CLIAdapter.prompt sent handoff and rotation prompts without confirming
them, which is the failure F20 exists to catch. Fixed at the shared call site.
F23 needed no change. Issue comments already become decisions with no
submission, through Reconciler.Reconcile at PreLease and at every turn
boundary. The earlier finding searched internal/operations alone and was
wrong. Tests now cover the boundary it turns on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011xsXyr5J1RACo71YeKG3Pu
F20. Only the launch confirmed its submit. A decision notice at a turn
boundary, and /clear or @HANDOFF.md during a context reset, were
fire-and-forget through the same transport that loses an Enter often enough
that the launch needed three resubmits. A lost Enter on the context-reset path
is the worst of them: it strands the session mid-rollover and nothing retries
it. LaunchConfirmer is therefore InputConfirmer, ConfirmLaunch is ConfirmInput,
and sendPrompt and sendLine both go through it.
Orchestra does not try to guarantee delivery of input it did not originate.
But it must never read that input as work, which is the F16 half. Burn-in run
3 stalled with an unexplained "go ahead and implement it" in the editor, and
the renewal check hashed the whole capture, so those keystrokes read as
progress and the lease kept renewing around an idle agent. PaneProgress drops
input lines from the capture, which the -J join makes exact: a wrapped input
block is one line beginning with the prompt marker.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
F16 shipped inert. renewLeases refuses to renew when the agent is idle and the
pane is unchanged since the hash recorded at the previous renewal, but
reconcileLeases rebuilt every lease from the coordinator's task list on each
tick and constructed a fresh struct, dropping ProgressSHA. The check therefore
took its no-baseline branch forever and renewed unconditionally.
Observed live: task 06G44JZB80MZBEY97196EZN8EC renewed at 09:31, 09:52 and
10:12 while its agent had been idle since 09:13, with a byte-identical pane
capture and no busy marker anywhere in it.
UsageBaseline and PickupAcknowledged were being wiped the same way, so a usage
receipt lost its baseline and a pickup could be re-acknowledged. The
coordinator is authoritative for the lease, not for what this worker observed
under it, so those three carry across a reconcile while the epoch is unchanged
and reset when it is not.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
renewLeases renewed whenever a session existed and PaneCapture succeeded,
so a pane that opened and never accepted a prompt held its lease forever.
That is the mechanism behind the July stuck task: the launch failed and
nothing ever let go.
Renewal now needs the agent to be busy, or the pane capture to differ from
the one recorded at the previous renewal. The first renewal has no baseline,
so it records one and passes; the next must show movement.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Burn-in run 2 recorded TaskLaunchAcknowledged, opened a pane, and ran nothing
for fifteen minutes. The launch instruction sat in Claude Code's input editor
as "[Pasted text #1 +66 lines]" at zero tokens and zero elapsed. Two separate
bugs produced that.
The transport was wrong for the harness. TmuxBackend.Prompt writes the whole
instruction with send-keys -l and then sends Enter, and the TUI coalesces the
fast multi-line write into a paste that absorbs the following Enter. Launch
transport is now a backend property rather than one universal prompt format:
claude on tmux submits a single line pointing at .orchestra/launch.md, every
other harness keeps the inline path it was verified on. agentctx is unchanged
and the file still holds the exact bytes Orchestra rendered, so what the agent
receives is identical either way. Under the file transport a failed write is
now a failed launch, because there the file is the instruction.
The acknowledgement was also wrong. It meant "Prompt returned nil", not "the
harness accepted the prompt". Backends may now implement ConfirmLaunch, and
the tmux one polls until the input editor clears and the agent is observably
busy, blocked on approval, or at least no longer holding the text. An editor
that still holds the prompt at the deadline is a definite failure. The worker
kills the pane, drops the session so the retry starts clean, and returns
ErrPromptNotSubmitted, which classifies as prompt_not_submitted rather than
launch_uncertain. That class already falls through to TaskReleased, so the
existing retry path takes it and no lease is held on a launch that never
happened.
The confirmation bound is tunable because how fast a terminal harness reacts
is a property of the host. It is not a sleep before the submit: the submit is
deterministic, and this waits for the harness to visibly react to it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The burn-in pairs a coordinator with a worker, and step one of every run is
checking that both report the burn-in commit. That was only visible behind the
operator login, at /v1/admin/diagnostics and /v1/federation/workers, and the
worker never logged its own revision at all. Both now print it on startup, so
`docker logs orchestra-api` and `journalctl -u orchestra-worker` are enough.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The v3 stack, previously an uncommitted working tree, plus this session's two
units and the burn-in instrument. This commit is the burn-in build identity:
coordinator and worker must both report this revision before a task is created.
Workflow (earlier sessions, uncommitted until now): human decision events and
reduction, source cursors and reconcile-before-launch, turn-boundary
reconciliation, internal/agentctx as the single renderer, ace-fca phases with
sealed artifacts, the trajectory gate, bounded grilling, independent review,
task pr enforcement, and human review reflection.
Capability restrictions at the agent boundary: an authz.Agent surface at
GatedWrite may ask and may not act. It also fixes two bugs the unit exposed --
gated surfaces could not reach the two endpoints written for them, and
RequestHumanDecision would block an unowned task while rejecting a question
from the session that did own it.
Turn-boundary reconcile-failure escalation: a streak of consecutive failures
asks the session to hand off, fenced on the lease epoch, with reconcile_failure
as a real handoff reason. The worker was dropping the coordinator's verdict on
the floor; it now acts on it.
Burn-in: herdr.WriteLaunchContext dumps the exact agentctx.Build result to
<worktree>/.orchestra/launch.md at every launch, local and federated. BURNIN.md
is the runbook. deploy/build.sh stamps both binaries from one commit.
go build, go vet and go test ./... pass, 20 packages.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Introduces the browser-facing surface and the worker-side protocol that
backs it:
- internal/ui: joined read model plus per-task lifecycle and approval
controls, kept separate from the raw endpoints workers and harnesses
depend on.
- internal/webui + web/: Vite/React app, build output embedded via
go:embed and served as an SPA fallback.
- federation: per-(worker, task) captures with a monotonic revision that
advances only when pane text actually changes, and a command queue
restricted to grant_approval / deny_approval, each bound to the capture
revision the operator acted on.
- orchestra-worker: publishes captures and executes commands only after
re-reading the pane and confirming the revision still matches. Sends
keystrokes only for a visible y/n prompt or OpenCode's fully labelled
selector, and refuses to deny through that selector rather than guess
at unobservable navigation.
This is the ownership boundary AUDIT.md's B14 and B17 call for: approval
becomes an explicit, revision-bound operation executed by the worker that
owns the pane, instead of a side effect of prompting over a
coordinator-driven remote socket.
Also ignores the web build inputs and outputs. node_modules ships vendored
Go packages, so go build and go test walk into it if it is merely
untracked; both node_modules and .node_modules are excluded.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01535A3Y8RtkAi8wYuWhtkEd
kami
kami