v3 workflow: intent, phases, review, submission, enforcement, burn-in
The v3 stack, previously an uncommitted working tree, plus this session's two units and the burn-in instrument. This commit is the burn-in build identity: coordinator and worker must both report this revision before a task is created. Workflow (earlier sessions, uncommitted until now): human decision events and reduction, source cursors and reconcile-before-launch, turn-boundary reconciliation, internal/agentctx as the single renderer, ace-fca phases with sealed artifacts, the trajectory gate, bounded grilling, independent review, task pr enforcement, and human review reflection. Capability restrictions at the agent boundary: an authz.Agent surface at GatedWrite may ask and may not act. It also fixes two bugs the unit exposed -- gated surfaces could not reach the two endpoints written for them, and RequestHumanDecision would block an unowned task while rejecting a question from the session that did own it. Turn-boundary reconcile-failure escalation: a streak of consecutive failures asks the session to hand off, fenced on the lease epoch, with reconcile_failure as a real handoff reason. The worker was dropping the coordinator's verdict on the floor; it now acts on it. Burn-in: herdr.WriteLaunchContext dumps the exact agentctx.Build result to <worktree>/.orchestra/launch.md at every launch, local and federated. BURNIN.md is the runbook. deploy/build.sh stamps both binaries from one commit. go build, go vet and go test ./... pass, 20 packages. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -1,5 +1,36 @@
|
||||
# Orchestra audit — handoff first
|
||||
|
||||
## 2026-08-11 — tmux backend and Claude in-place context rollover
|
||||
|
||||
- `orchestra-worker` now selects `ORCHESTRA_WORKER_BACKEND=herdr|tmux`.
|
||||
The tmux backend is intentionally Claude-only: it creates one detached,
|
||||
isolated tmux session per task, resolves configured tmux pane indexes rather
|
||||
than assuming `0.0`, starts Claude in the task worktree, handles the exact
|
||||
workspace-trust prompt, captures the pane, forwards explicit input, reports
|
||||
health/status, and retains/kills the session at the existing lifecycle
|
||||
boundaries. Codex and OpenCode still use herdr.
|
||||
- Registry entries accept `"backend":"tmux"`. Those entries are worker-owned
|
||||
even on the coordinator machine, so the Dockerized coordinator neither
|
||||
probes nor tries to operate a host tmux socket. Their router health remains
|
||||
gated by a fresh worker heartbeat and local backend check.
|
||||
- Claude rotation no longer asks the agent to duplicate the workpc hook's
|
||||
handoff in `.orchestra-handoff-report.md`. A changed `HANDOFF.md`, observed
|
||||
after Claude is idle, drives the persisted in-place sequence `/clear` +
|
||||
`ENTER`, then `@HANDOFF.md` + `ENTER`; the exhausted transcript identity is
|
||||
discarded so the next observation resolves the new session. The canonical
|
||||
cross-worker release protocol remains separate and unchanged.
|
||||
- The portable workpc Claude configuration (`settings*.json`, `CLAUDE.md`,
|
||||
status line, hooks, and helper scripts) was checksum-verified after copying
|
||||
to homesrv. The prior homesrv files are backed up under
|
||||
`/home/kami/.claude/backups/workpc-import-20260811T120000`; machine/session
|
||||
state was excluded and local-only hooks were preserved.
|
||||
- Verified in the source tree with the real isolated-tmux lifecycle test,
|
||||
focused backend/worker tests, `go build ./...`, `go vet ./...`,
|
||||
`go test ./...`, `go test -race ./...`, and the web lint/test/production
|
||||
build. No worker binary, registry file, service, container, or live pane was
|
||||
deployed/restarted/touched; live Claude/tmux behavior remains a deployment
|
||||
QA item rather than a completed claim.
|
||||
|
||||
Audited 2026-07-30 against the working tree, spec, deployed coordinator,
|
||||
workpc worker, event log, and live herdr (read-only).
|
||||
|
||||
@@ -168,3 +199,137 @@ verified live, which is why deleting it costs nothing.
|
||||
completion, and late-recovery paths on OpenCode, Claude, and Codex without
|
||||
manual intervention for safe repository work. Follow the QA handoff above
|
||||
and attach the resulting event ranges/artifacts before clearing this gate.
|
||||
|
||||
## Capability restrictions at the agent boundary (2026-08-26)
|
||||
|
||||
The rule now enforced: an agent may perform work and *request* lifecycle
|
||||
changes; it may never perform one.
|
||||
|
||||
- **New `authz.Agent` surface**, `GatedWrite`. `AuthorizeEvent` refuses every
|
||||
event type Orchestra owns (`WorkPhaseChanged`, `ReviewRecorded`,
|
||||
`TaskSubmitted`, `TaskCompleted`, `TaskLeased`, `ApprovalGranted`, …), and the
|
||||
HTTP gate refuses their endpoints. Credential is `ORCHESTRA_AGENT_TOKEN`, the
|
||||
only Orchestra credential that may enter an agent pane.
|
||||
- **Gated surfaces could not reach the two endpoints written for them.** The
|
||||
middleware admitted only paths ending in `/approval`, so `decision-request`
|
||||
and `deferred` were 403 before their handlers ran, and the handlers' own
|
||||
`AuthorizeEvent("ApprovalRequested")` was dead code. The allowlist is now
|
||||
`authz.GatedWritePath`, and the gate keys off `CapabilityFor(s) ==
|
||||
GatedWrite` rather than naming MCP and Maven.
|
||||
- **`/v1/harness/turn` was unreachable.** An unlabelled request defaults to the
|
||||
Web surface, `sessions` is always non-nil, so every harness call returned 401
|
||||
in any deployment with web credentials. It is now exempt from the surface gate
|
||||
and authenticates its own bearer token in the handler, like federation does.
|
||||
- **`RequestHumanDecision` now requires ownership and fences on the lease.** It
|
||||
blocked any task in any state, and its `TaskBlocked` event carried no
|
||||
`harness_id`/`lease_epoch`. That is backwards twice over: an agent credential
|
||||
was a way to block a queued task nobody was working on, and a question from
|
||||
the session that *did* own the task was rejected by `Store.Append`'s fence.
|
||||
Both fixed; six tests that had been exercising the unleased state now lease
|
||||
first.
|
||||
|
||||
Not built, deliberately: a `Capability` vocabulary, `CapabilitySet`, per-role
|
||||
`SessionAuthority`, or role-to-capability project policy. One surface at one
|
||||
capability level expresses "may ask, may not act", and the store already fences
|
||||
every lifecycle event on `(harness_id, lease_epoch)`. Add roles when two agent
|
||||
roles actually need different API rights — today the difference between an
|
||||
implementer and a reviewer is what `agentctx` renders, not what the API allows.
|
||||
|
||||
Still open, and it is the real enforcement:
|
||||
|
||||
- **Execution capabilities are unenforced.** Network, secrets, paths outside
|
||||
the worktree and destructive commands have no policy check; agents inherit
|
||||
the harness pane's environment, and nothing in this repo sets it. `git push`
|
||||
and a direct Gitea call succeed or fail purely on whatever credentials that
|
||||
environment happens to hold. Credential isolation in the pane is an operator
|
||||
task today, not a code path.
|
||||
- **A reviewing session still cannot seal its own review**, and should not:
|
||||
the worker seals on its behalf. That answers question 1 of the handoff.
|
||||
|
||||
## Turn-boundary reconcile-failure escalation (2026-08-26)
|
||||
|
||||
The gap the 2026-08-26 handoff deliberately left open. A failed reconcile at a
|
||||
verified turn boundary was recorded and the turn continued, forever. One
|
||||
failure is transient, so continuing is right. A streak means Orchestra can no
|
||||
longer promise that the newest human input outranks the session's intent, and
|
||||
continuing silently is exactly the failure shape this repo keeps producing.
|
||||
|
||||
- `Coordinator.ReconcileFailureHandoff` (default 3, `defaultReconcileFailureHandoff`)
|
||||
is the number of *consecutive* failures that escalate. Env:
|
||||
`ORCHESTRA_RECONCILE_FAILURE_HANDOFF`.
|
||||
- `Coordinator.noteReconcileResult` holds the streak per task, **keyed on the
|
||||
lease epoch**. A successor never inherits its predecessor's count, so no
|
||||
release path needs a cleanup hook. A success deletes the entry. The count is
|
||||
written into `MonitorHealth.Sessions[id].LastError` as
|
||||
`reconcile human input (N consecutive): ...`.
|
||||
- On escalation `TurnDecision` asks for a handoff with reason
|
||||
`reconcile_failure` and answers `prepare_handoff`. `RemoteTurn` answers the
|
||||
same for a worker-owned session, on the same threshold.
|
||||
- **The escalation only fires where rotation had no reason of its own.** It sits
|
||||
on the `continue` branch, so an existing rotate, refuse, or prepare_handoff
|
||||
keeps its own reason rather than having a second one manufactured for it.
|
||||
- `reconcile_failure` is a real handoff reason: added to `continuity.reasons`
|
||||
(or the artifact it produces would fail validation), to `herdr.handoffReason`,
|
||||
and to the new `orchestrator.bypassReason` — which replaces the
|
||||
`manual || milestone || thrash` comparison that was duplicated in `rotate`
|
||||
and `TurnDecision`. Release therefore runs through the existing bypass path,
|
||||
with no new state, blocker or recovery protocol.
|
||||
- `CLIAdapter.RequestHandoffReason` explains the reason to the agent and says
|
||||
explicitly that it is not a judgement about its work.
|
||||
- **The worker ignored the coordinator's verdict entirely.** `federatedTurn`
|
||||
read `answer.Decisions` and dropped `answer.Verdict` on the floor, so a
|
||||
federated session could never be asked to hand off for any coordinator-side
|
||||
reason. It now requests the handoff and records it on the session, which is
|
||||
what the release loop watches for.
|
||||
|
||||
Then the existing machinery does the useful part: the successor's
|
||||
`Store.PreLease` reconcile fails closed while the source is down, so the task
|
||||
waits in the queue instead of resuming from an older authority.
|
||||
|
||||
Proofs: `internal/orchestrator/reconcile_escalation_test.go` (streak, reset,
|
||||
rotation-wins, release-through-bypass, no-source, delivery-failure-is-not-a-
|
||||
reconcile-failure, federated parity), `internal/integration/reconcile_escalation_test.go`
|
||||
(the full loop: two turns continue, third hands off, successor refused while the
|
||||
source is down, correction reconciled on recovery), and
|
||||
`cmd/orchestra-worker/main_test.go:TestFederatedTurnActsOnPrepareHandoffVerdict`.
|
||||
|
||||
Deviation from the requested shape: the threshold is one coordinator field, not
|
||||
per-project `human_reconcile.turn_failure_handoff_after`. `TurnDecision` has no
|
||||
project in scope, and `Soft`/`Hard`/`Thrash` are already coordinator-level for
|
||||
the same reason. Making it per-project means plumbing the registry into the
|
||||
coordinator, which is worth doing when a second project actually needs a
|
||||
different number.
|
||||
|
||||
## Burn-in instrumentation and readiness probe (2026-08-26)
|
||||
|
||||
Feature work stops here. `BURNIN.md` is the runbook: evidence per run, the five
|
||||
flows, the failure classification, and the pane credential cleanup.
|
||||
|
||||
One addition, because the burn-in's main inspection was impossible without it:
|
||||
`herdr.WriteLaunchContext` dumps the exact `agentctx.Build` result to
|
||||
`<worktree>/.orchestra/launch.md` at every launch, local (`Coordinator.Start`)
|
||||
and federated (`orchestra-worker`). Reading it back from pane scrollback is not
|
||||
equivalent, because the harness reflows and truncates. A write failure is
|
||||
recorded, never fatal. Proof:
|
||||
`internal/integration/reconcile_launch_test.go:TestLaunchWritesTheContextItSent`
|
||||
compares the file against the instruction the adapter actually received.
|
||||
|
||||
Probed live state, and the burn-in is blocked on deployment, not on code:
|
||||
|
||||
- The API is up on homesrv (`/readyz` ready, gitea and jsonl configured).
|
||||
- The workpc worker is up (pid 741, restarted 2026-08-26 11:39, no errors),
|
||||
serving `workpc-claude` on a tmux backend and `workpc-opencode` on a herdr
|
||||
unix socket.
|
||||
- **The deployed worker is built from 97a9c65 (2026-07-30).** Every v3 unit from
|
||||
the last two sessions is an uncommitted working-tree change, so neither the
|
||||
container nor the worker has phases, review, submission, the agent surface, or
|
||||
the reconcile escalation. Commit and redeploy both before any run.
|
||||
- **Codex has no entry in `/etc/orchestra/harnesses.json`**, and no worker runs
|
||||
on homesrv, so three of the four target harnesses cannot be exercised at all
|
||||
today.
|
||||
- Correction to an earlier assumption in this file: the live `config.jsonc` sets
|
||||
no `backend` and no `address` on any of its six herdrs, so all six resolve to
|
||||
`<machine>:9245` via `registry.defaultHerdrPort`. Both ports are closed, and
|
||||
that is *not* evidence about the workpc harnesses, which use a tmux socket and
|
||||
a unix socket. Federated reachability defers to worker heartbeat. Do not
|
||||
diagnose harness availability from a TCP probe of 9245.
|
||||
|
||||
Reference in New Issue
Block a user