Give the phase brief a protocol, and end the session it advances
The brief told the agent to ask for a phase change and never carried the asking. The agent asked in prose, no code represented the request, and the session idled until its lease expired. That is what failed run 3. F21. The agent asks with .orchestra/phase-request.json, and seals research.json or plan.json where the phase it is leaving produces one. At a verified turn boundary the worker checks the phase belief, the transition and the artifact, then calls the coordinator with its lease epoch and a derived operation id. AdvanceWorkPhase is unchanged, so a request cannot reach a move the operator surface could not also make. Redelivery is idempotent. F22. A session now records the phase it was launched to run. One that no longer matches its task rotates with reason phase_changed, whether this worker asked for the change or an operator made it. F20. CLIAdapter.prompt sent handoff and rotation prompts without confirming them, which is the failure F20 exists to catch. Fixed at the shared call site. F23 needed no change. Issue comments already become decisions with no submission, through Reconciler.Reconcile at PreLease and at every turn boundary. The earlier finding searched internal/operations alone and was wrong. Tests now cover the boundary it turns on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011xsXyr5J1RACo71YeKG3Pu
This commit is contained in:
@@ -610,3 +610,324 @@ was.
|
||||
Both halves report `edff021`, built from a detached worktree of that revision.
|
||||
Worker sha256 `68ac265455acb0e007bfb0e898a91b317cd7289d7f2bf5da6df1224e1d6706a5`
|
||||
at `/usr/local/bin/orchestra-worker`. Coordinator image built 13:08:30 +0400.
|
||||
|
||||
## Run 3: autonomous expiry and relaunch, 2026-08-27 11:02 UTC
|
||||
|
||||
Task `06G44JZB80MZBEY97196EZN8EC`, issue `kami/test-e2e#3`. Observed live, no
|
||||
operator action in the window. The stranded epoch was
|
||||
`06G44ZBZX4YH6PRN7Y4ZH8GG3W`.
|
||||
|
||||
| seq | at (UTC) | event | epoch |
|
||||
|---|---|---|---|
|
||||
| 358 | 10:32:09 | `TaskLeaseRenewed` v15 | `06G44ZBZX4YH6PRN7Y4ZH8GG3W` |
|
||||
| 359 | 11:02:16 | `TaskReleased` v16, `reason: lease_expired` | `06G44ZBZX4YH6PRN7Y4ZH8GG3W` |
|
||||
| 360 | 11:03:17 | `TaskLeased` v17 | `06G45RVKAW20H1D22NHBRYKMN8` |
|
||||
| 361 | 11:03:22 | `TaskLaunchAcknowledged` v18, `lifecycle_phase: started` | `06G45RVKAW20H1D22NHBRYKMN8` |
|
||||
|
||||
Expiry to acknowledged launch: 66 seconds, autonomous. Every event from seq 353
|
||||
onward carries `surface: system`. The last `tui` event is the `TaskCorrected` at
|
||||
09:11:55, before the window.
|
||||
|
||||
Launch receipt, worker journal, 15:03:22 +04:
|
||||
|
||||
```
|
||||
launch 06G44JZB80MZBEY97196EZN8EC confirmed: confirmation=editor_cleared submit_attempts=2 first_submit_at=2026-08-27T11:03:22.203859355Z confirmed_at=2026-08-27T11:03:22.717657436Z
|
||||
```
|
||||
|
||||
### F16 is partial, not closed
|
||||
|
||||
The hard predicate passed: no renewal carried the stranded epoch after
|
||||
10:45:26. The run did not exercise the progress comparison. Worker health
|
||||
records why the renewal was skipped:
|
||||
|
||||
```
|
||||
"last_error": "validate lease 06G44JZB80MZBEY97196EZN8EC: tmux capture-pane -p -J -S -200 -t =orchestra-06g44jzb80mzbey97196ezn8ec-e53607ea:1.0: no server running on /tmp/tmux-1000/orchestra: exit status 1",
|
||||
"error_at": "2026-08-27T11:02:16.213318366Z"
|
||||
```
|
||||
|
||||
That is the `paneProgress` error path at `cmd/orchestra-worker/main.go:973`, not
|
||||
the `default` branch at line 992. The tmux server was gone, so no pane text
|
||||
could be hashed.
|
||||
|
||||
- **Proven:** dead or missing pane, no renewal, lease expires, autonomous
|
||||
re-lease and relaunch.
|
||||
- **Unproven:** live pane with an idle agent and unchanged output, renewal
|
||||
refused.
|
||||
|
||||
### Epoch reset, stated precisely
|
||||
|
||||
The new epoch and its acknowledged launch establish the reset structurally.
|
||||
`leases` in `/var/lib/orchestra/worker-state/workpc-claude.json` holds only
|
||||
`{epoch, version, until}`, with `progress_sha` absent under `omitempty`. No
|
||||
runtime value of `ProgressSHA`, `UsageBaseline` or `PickupAcknowledged` was
|
||||
observed.
|
||||
|
||||
### F6 closed
|
||||
|
||||
Worker health reports `active_task_id: 06G44JZB80MZBEY97196EZN8EC` with
|
||||
`active_pane_id` set, against a leased task.
|
||||
|
||||
### F18 sharpened, non-blocking
|
||||
|
||||
`recordError` (`cmd/orchestra-worker/main.go:70`) keeps one slot, so the earlier
|
||||
renewal decision near 10:52 was overwritten and cannot be read. One
|
||||
`last_error` slot cannot preserve a causal sequence. Later replacement: a
|
||||
bounded recent-error ring or event-backed observations, roughly the last 8
|
||||
`{at, class, message}` entries. Do not land this during a live run.
|
||||
|
||||
### Ledger after this checkpoint
|
||||
|
||||
```
|
||||
F6 closed
|
||||
F14 closed
|
||||
F15 closed by detection, transport fix pending live proof
|
||||
F16 partial: missing-pane branch proven, static-live-pane branch unproven
|
||||
F17 fixed, isolated live proof
|
||||
F18 open observability
|
||||
F19 fixed
|
||||
F20 awaiting first post-frame confirmed input
|
||||
```
|
||||
|
||||
### Next decisive checkpoint, F20
|
||||
|
||||
```
|
||||
frame completes
|
||||
→ Orchestra emits the next input
|
||||
→ worker journal must contain: input to <pane> confirmed: confirmation=<kind> submit_attempts=<n>
|
||||
→ only then may the agent continue
|
||||
```
|
||||
|
||||
Receipt appears and the model proceeds: let the run continue through phase
|
||||
transitions. Input lands with no receipt: F20 fails. No input attempted after
|
||||
`frame`: a different lifecycle or phase-advance bug.
|
||||
|
||||
## Run 3 conformance result, 2026-08-27 12:00 UTC
|
||||
|
||||
```text
|
||||
run 3 conformance result: failed
|
||||
cause: no autonomous phase-advance path
|
||||
```
|
||||
|
||||
### F16 closed
|
||||
|
||||
```text
|
||||
missing-pane branch: proven
|
||||
live-pane idle + unchanged output branch: proven
|
||||
external editor input ignored as progress: proven
|
||||
```
|
||||
|
||||
Second branch evidence, worker health at 11:53:46 with the pane alive and
|
||||
`herdr_status: reachable`:
|
||||
|
||||
```
|
||||
"last_error": "lease 06G44JZB80MZBEY97196EZN8EC not renewed: agent status idle and pane unchanged since the last renewal",
|
||||
"error_at": "2026-08-27T11:53:46.195553513Z"
|
||||
```
|
||||
|
||||
That is the `default` branch at `cmd/orchestra-worker/main.go:992`.
|
||||
|
||||
### F21, lifecycle: the phase brief promises a protocol that does not exist
|
||||
|
||||
`.orchestra/launch.md:62` tells the agent `Orchestra decides when this phase
|
||||
ends. Ask for a phase change, do not declare one.` The agent complied and
|
||||
printed `Nothing blocks. Ready for a phase change to implement.`
|
||||
|
||||
```text
|
||||
phase brief: "ask for a phase change"
|
||||
agent: asks
|
||||
worker: has no representation of that request
|
||||
coordinator: AdvanceWorkPhase exists, but nothing invokes it autonomously
|
||||
```
|
||||
|
||||
`operations.AdvanceWorkPhase` is reachable only from the HTTP handler at
|
||||
`cmd/orchestra/main.go:883` and from `internal/operations/review.go:101`. The
|
||||
worker's only post-frame send is a decision notice at
|
||||
`cmd/orchestra-worker/main.go:1639`, gated on `len(answer.Decisions) == 0`
|
||||
returning early.
|
||||
|
||||
The fix is a bounded agent intent, not pane-text matching:
|
||||
|
||||
```go
|
||||
type PhaseAdvanceRequest struct {
|
||||
From WorkPhase
|
||||
To WorkPhase
|
||||
}
|
||||
```
|
||||
|
||||
```text
|
||||
verified turn boundary
|
||||
→ obtain bounded phase intent from harness
|
||||
→ validate requested transition
|
||||
→ submit to coordinator
|
||||
→ operations.AdvanceWorkPhase
|
||||
→ if transition requires sealed artifact:
|
||||
collect/seal artifact first
|
||||
→ rotate/start successor if phase policy requires fresh context
|
||||
```
|
||||
|
||||
`frame` needs no artifact. `research` and `plan` take the same path but must
|
||||
seal their artifact before `AdvanceWorkPhase` accepts them.
|
||||
|
||||
### F22, lifecycle: nothing reacts to `WorkPhaseChanged` at runtime
|
||||
|
||||
`t.WorkPhase` is read only when a launch context is built, at
|
||||
`cmd/orchestra-worker/main.go:369` and
|
||||
`internal/orchestrator/orchestrator.go:1218`. No code path rotates, relaunches
|
||||
or notifies a live session when the phase changes. A manual advance therefore
|
||||
takes effect only at the next launch, and leaves the current idle pane idle.
|
||||
|
||||
### F23, lifecycle: a decision cannot exist before a submission
|
||||
|
||||
`answer.Decisions` is the only input `sendPrompt` ever carries post-frame.
|
||||
Decisions come from `operations.ReflectSubmission`, and
|
||||
`human.PullRequestState.FeedbackAfter` returns input only strictly after a
|
||||
submission. A task still in `frame` has no submission, so no decision can be
|
||||
recorded for it.
|
||||
|
||||
Consequence: **F20 cannot be exercised on this task in its current state.** A
|
||||
manual phase advance does not produce a post-frame input, because nothing
|
||||
reacts to the phase change and no decision can exist yet.
|
||||
|
||||
### Observation kept separate from F20
|
||||
|
||||
`❯ go ahead and implement it` appeared in the editor again on attempt 2, cursor
|
||||
at `cursor_x=2`, never submitted. The agent transcript at
|
||||
`~/.claude/projects/-tmp-test-e2e-worktrees-06G44JZB80MZBEY97196EZN8EC/97b30e4d-90a7-49bc-a9b8-7d9e450831e4.jsonl`
|
||||
holds exactly one user message, the launch prompt at 11:55:52.236Z. No worker
|
||||
send and no receipt exist for it. Origin unknown, external to Orchestra. F20
|
||||
does not absorb it.
|
||||
|
||||
### Ledger after this checkpoint
|
||||
|
||||
```
|
||||
F6 closed
|
||||
F14 closed
|
||||
F15 closed by detection, transport fix pending live proof
|
||||
F16 closed, both branches live-proven
|
||||
F17 fixed, isolated live proof
|
||||
F18 open observability
|
||||
F19 fixed
|
||||
F20 blocked, not merely awaiting: see F23
|
||||
F21 open lifecycle, no autonomous phase-advance path
|
||||
F22 open lifecycle, no runtime reaction to WorkPhaseChanged
|
||||
F23 open lifecycle, no decision path before a submission
|
||||
```
|
||||
|
||||
## F21, F22, F23 implemented, 2026-08-27
|
||||
|
||||
Run 3 was not shepherded further. Attempt 3 was left unspent: F16 is closed on
|
||||
both branches, and another idle phase would have proven nothing new.
|
||||
|
||||
### F23 was already implemented, and the earlier entry was wrong
|
||||
|
||||
`human.Reconciler.Reconcile` imports issue comments from `task.Source` and
|
||||
records `HumanDecisionRecorded` with no submission involved. It is wired at
|
||||
two points in `cmd/orchestra/main.go`: `Store.PreLease`, and
|
||||
`Coordinator.ReconcileHumanInput`, which `RemoteTurn` calls at every verified
|
||||
turn boundary. `provider.GiteaComments.FetchAfter` reads the task's own issue.
|
||||
|
||||
The earlier F23 entry traced `EventHumanDecisionRecorded` through
|
||||
`internal/operations` only and concluded decisions required a submission. That
|
||||
was a scoping error in the search, not a gap in the code. `ReflectSubmission`
|
||||
is the pull-request path and stays narrow; it was never the general decision
|
||||
source.
|
||||
|
||||
Consequence for the record: **F20 had a legitimate post-launch send path
|
||||
throughout run 3.** A comment on `kami/test-e2e#3` would have produced a
|
||||
decision, a `DecisionNotice` at the next boundary, and a receipt.
|
||||
|
||||
F23 is closed as already-implemented, with tests added for the boundary it
|
||||
turns on.
|
||||
|
||||
### F21, the phase-request protocol
|
||||
|
||||
The agent asks with a bounded file. Prose is not a protocol, so nothing
|
||||
matches on pane text.
|
||||
|
||||
```
|
||||
.orchestra/phase-request.json {"from": "frame", "to": "research"}
|
||||
.orchestra/research.json sealed before leaving research
|
||||
.orchestra/plan.json sealed before leaving plan
|
||||
```
|
||||
|
||||
At a verified turn boundary the worker validates what it can see locally: the
|
||||
phase the agent believes it is in, the legality of the transition, and the
|
||||
presence and decode of the artifact the phase must seal. It then calls
|
||||
`POST /v1/federation/phase` with the lease epoch and a derived operation id.
|
||||
The coordinator calls the existing `operations.AdvanceWorkPhase`.
|
||||
|
||||
- `internal/operations/workphase.go`: `RequestWorkPhase`, `ErrPhaseRequest`.
|
||||
Fences on lease epoch, refuses a stale phase belief, refuses any target but
|
||||
the project's next phase, idempotent per operation id.
|
||||
- `internal/federation/client.go`: `Client.AdvancePhase`.
|
||||
- `cmd/orchestra/main.go`: the `/v1/federation/phase` route.
|
||||
- `cmd/orchestra-worker/main.go`: `requestPhase`, `phaseArtifact`,
|
||||
`phaseRequestFile`.
|
||||
- `internal/agentctx/agentctx.go`: `phaseRequestBrief` renders the protocol
|
||||
under the sentence that used to promise it.
|
||||
|
||||
The operation id is derived, not random: `phase:<task>:<epoch>:<from>:<to>`. A
|
||||
redelivery after a lost response carries the same id, so the coordinator
|
||||
returns the first event instead of advancing twice.
|
||||
|
||||
### F22, a phase change ends that cognitive session
|
||||
|
||||
`herdr.Session` now carries `Phase`, the phase the session was launched to
|
||||
run. At a turn boundary a session whose task has moved on is rotated with
|
||||
reason `phase_changed`, which is added to the closed handoff-reason set in
|
||||
`internal/herdr/adapter.go` and given its own wording in
|
||||
`RequestHandoffReason`.
|
||||
|
||||
One comparison covers both cases: a change this worker requested, and one an
|
||||
operator made through `POST /v1/tasks/{id}/phase`. Both leave the same
|
||||
evidence, a session built for a phase that is no longer current.
|
||||
|
||||
### F20 hole found and closed while implementing F22
|
||||
|
||||
`CLIAdapter.prompt` called `backend.Prompt` and returned. Every rotation and
|
||||
handoff prompt therefore went out unconfirmed, which is precisely the failure
|
||||
F20 exists to catch. `RequestHandoffReason` is one of its callers, so the
|
||||
phase rotation would have inherited it.
|
||||
|
||||
Fixed at the shared call site rather than per caller: `prompt` now routes
|
||||
through `InputConfirmer` and logs the same `input to <pane> confirmed:`
|
||||
receipt the worker logs.
|
||||
|
||||
### Tests
|
||||
|
||||
`go build ./...`, `go vet ./...` and `go test ./...` all pass.
|
||||
|
||||
- `internal/operations/phase_request_test.go`: full path with each artifact
|
||||
sealed, skipped phase refused, stale phase belief refused, stale epoch
|
||||
refused, missing operation id refused, redelivery idempotent, operation id
|
||||
recorded.
|
||||
- `cmd/orchestra-worker/phase_test.go`: request accepted then session rotates,
|
||||
external phase change rotates, unsealed artifact refused locally, malformed
|
||||
artifact refused locally, sealed artifact travels with the request, refused
|
||||
request retried, decision notice stays undelivered until confirmed.
|
||||
- `internal/integration/phase_protocol_test.go`: a pre-submission issue
|
||||
comment steers a live leased session and is not re-sent once delivered;
|
||||
the phase-request path seals and fences at the coordinator.
|
||||
- `internal/human/pullrequest_window_test.go`: pull-request feedback ignores
|
||||
anything not strictly after the submission, and applies trust.
|
||||
- `internal/agentctx/agentctx_test.go`: the brief names the request file, the
|
||||
request shape, and the artifact to seal.
|
||||
|
||||
### Ledger
|
||||
|
||||
```
|
||||
F6 closed
|
||||
F14 closed
|
||||
F15 closed by detection, transport fix pending live proof
|
||||
F16 closed, both branches live-proven
|
||||
F17 fixed, isolated live proof
|
||||
F18 open observability, non-blocking
|
||||
F19 fixed
|
||||
F20 fixed, adapter hole closed; live proof pending run 4
|
||||
F21 fixed, tests only
|
||||
F22 fixed, tests only
|
||||
F23 closed, already implemented; tests added
|
||||
```
|
||||
|
||||
Run 4 starts from here. Both halves must be rebuilt and redeployed before it
|
||||
begins, and the deployment boundary recorded as usual.
|
||||
|
||||
Reference in New Issue
Block a user