orchestra/orchestra-spec (1).md

Orchestra — autonomous handoff protocol

Version: 2 (binding target) Revised: 2026-07-30 Implementation language: Go

MUST, MUST NOT, SHOULD, and MAY are normative. Defaults marked [D] are tunable. AUDIT.md records implementation gaps; it does not weaken this contract.

0. Objective

Maximize useful unattended runtime across opaque CLI harnesses: Claude Code, Codex, OpenCode, and future adapters. Orchestra drives processes through herdr; it does not call model APIs.

Optimize, in order:

1. No lost or duplicated work.
2. One valid owner per task.
3. Recovery without operator reconstruction.
4. Useful autonomous progress.
5. Throughput and resource efficiency.

Cost-per-turn is secondary. UI breadth, provider breadth, and delegation are secondary until the handoff protocol passes §13.

1. Invariants

1. Append-only truth. The coordinator event log is canonical. Events are never edited or deleted; corrections append compensation events.
2. Durable before visible. An event is appended and fsynced before any projection changes or success is acknowledged.
3. Plane-owned lifecycle. Agents may write code and bounded intent artifacts. Only Orchestra emits lifecycle events.
4. Artifact references. Events carry content hashes, never embedded reports, task specifications, diffs, or handoffs.
5. Strict input. Event and artifact schemas reject unknown fields, invalid transitions, oversized values, and unbounded instruction text.
6. Ground-truth pickup. A successor validates Git, the immutable task artifact, and the handoff artifact before starting work.
7. One owner. Every lease has an unguessable lease_id and monotonically increasing lease_epoch. Stale owners cannot renew, release, complete, or mutate task lifecycle.
8. Self-fencing workers. A worker that cannot renew before its local lease deadline parks its agent before the lease can be reassigned.
9. Git crosses machines. Remote workers exchange repository state only through verified Git refs plus CAS artifacts. The coordinator never validates a remote checkout.
10. Split, validate, then close. A predecessor remains recoverable until the successor has validated pickup and started the replacement lease.
11. Idempotent control. Every cross-process mutation has an operation ID. Retries return the original result; ambiguous UI-changing herdr calls are reconciled, not blindly repeated.
12. Observable refusal. No lifecycle-critical error is a bare continue. Phase, error class, retry time, owner, pane, and anchor are queryable.
13. Liveness is live. Pane/agent status comes from herdr or process exit, not event replay.
14. No preemption. Importance orders queued work only; it never interrupts a valid running lease.

2. Ownership and topology

Component	Owner	Authority
Event log, CAS, projections, router	coordinator on homesrv	canonical
Git checkout, worktree, herdr socket, pane	worker on that machine	local execution
Git remote	homesrv	cross-machine repository transport
Browser/TUI/notifications	clients	projections and authorized commands

2.1 Federation

• Remote execution is worker-pull only. A coordinator MUST NOT call a remote herdr socket or inspect a remote filesystem.
• One worker represents one configured herdr/harness capacity pool.
• Worker identity and herdr registry identity MUST match.
• Registration declares protocol version, immutable build revision, capabilities, concurrency, supported projects, and local health.
• Registration does not imply eligibility. Eligibility requires a recent heartbeat and recent local-herdr protocol check.
• Coordinator and worker protocol versions MUST be compatible. Production release gates require the same build revision unless a rolling-upgrade compatibility test exists.
• Heartbeat loss marks a worker ineligible; it MUST NOT release a lease early. Lease expiry or explicit relinquishment is authoritative.

Direct coordinator-to-remote-herdr operation (“Design A”) is forbidden.

2.2 Project configuration

Each project declares:

id
machine_affinity[]          # hard by default
worker-local repo
worker-local worktree_root
git remote
quality_gate
safe_action_policy
capabilities

Paths are local to the worker that advertises the project. A worker MUST NOT accept a project lacking a complete local configuration.

Machine affinity is hard [D]: an unavailable preferred machine leaves the task queued instead of running it elsewhere.

3. Durable substrate

3.1 Event envelope

{
  "schema_version": 3,
  "seq": 123,
  "id": "event-ulid",
  "type": "TaskLeased",
  "task_id": "task-ulid",
  "task_version": 7,
  "at": "RFC3339Nano",
  "surface": "system|web|tui|mcp|maven",
  "actor": "stable-actor-id",
  "operation_id": "idempotency-id",
  "payload": {}
}

• The append boundary assigns seq, task_version, and at.
• seq is globally monotonic; task_version is monotonic per task.
• A producer MAY supply expected_task_version for optimistic concurrency.
• Worker ownership is checked with lease_id + lease_epoch, not task version. Unrelated approvals cannot invalidate a lease.
• Reusing operation_id with the same request returns the original event. Reusing it with different input is rejected.
• Global projections use a named aggregate ID; they do not fake task versions.

3.2 Append transaction

For each accepted event:

1. Authenticate surface and actor.
2. Strict-decode payload.
3. Validate artifact references.
4. Validate legal state transition.
5. Validate lease owner/epoch when required.
6. Allocate sequence and task version.
7. Append and fsync the event.
8. Apply in-memory projections.
9. Acknowledge the caller.
10. Write snapshots asynchronously and atomically.

Snapshot failure MUST NOT make a durable append appear failed. Restart rebuilds from the last valid snapshot plus the complete event tail.

3.3 CAS

• Address: lowercase SHA-256 of exact bytes.
• Write: temporary file → fsync → atomic rename → directory fsync.
• Existing objects are re-hashed before reuse.
• Read always re-hashes.
• Corruption is a hard, observable error; a corrupt object is never trusted.

3.4 Worker state

Worker state is a durable local journal containing:

event cursor
leases and local monotonic deadlines
sessions and exact pane/agent identities
handoff transaction phases
pending coordinator operations
last health/error observations

Writes are temporary file → fsync → atomic rename. Missing state starts empty; corrupt state fails closed and requires reconciliation. It MUST NOT silently become an empty map.

4. Task and lease model

4.1 Immutable task

TaskCreated references a strict, immutable task artifact rendered as TASK.md:

task_id, source, external_id, project
title, instructions, acceptance[]
capabilities[], quality_gate

The artifact hash is task_ref. Scheduling metadata—priority, due date, estimate, parent—may be amended. Instructions, acceptance, and gate are not amended in place; a scope change supersedes the task with a new task artifact.

4.2 Projected states

State	Meaning	Lease
queued	eligible now or at next_retry_at	none
leased	launching, running, or preparing handoff	required
needs_attention	recoverable pane awaits approval/operator action	retained when live
completed	gate, commit, push, and report verified	none
failed	retry policy exhausted or non-recoverable failure	none

needs_attention is not terminal. The same fenced owner may renew, resume, or complete it. A task with no recoverable session returns to queued through an explicit retry/release event or becomes failed.

Terminal state changes require an explicit compensation event naming the event being corrected.

A transition to failed must include evidence that any live pane was parked or retired. Terminal state is never used as a substitute for cleanup.

4.3 Lease

{
  "lease_id": "unguessable-id",
  "lease_epoch": 4,
  "worker_id": "workpc-opencode",
  "expires_at": "RFC3339Nano",
  "attempt": 2
}

• Epoch increases on every new lease.
• Renewal preserves ID and epoch and advances expiry.
• Every worker release, handoff, attention, approval execution, and completion includes ID and epoch.
• A worker derives a conservative local monotonic deadline from each lease response. It parks the agent before deadline if renewal cannot complete.
• Self-fencing means no process may keep editing after expiry. If an agent is still busy inside the safety margin and releasing its binding does not stop execution, the worker closes the exact pane while retaining the worktree and local recovery journal. Before closing, it best-effort commits current local state to an immutable orphan ref.
• A resumed worker reconciles ownership before reading, prompting, or starting any persisted session.
• If ownership was lost, the worker never resumes or auto-merges. It pushes the orphan ref when connectivity returns and publishes observation evidence for operator or explicit recovery.
• Capturing a pane is not sufficient proof for renewal; pane identity, agent identity, lease ownership, and local-herdr health must all match.

4.4 Retry

Failures are typed:

transient_transport
worker_unhealthy
launch_failed
lease_expired
handoff_invalid
pickup_invalid
quality_gate_failed
git_push_failed
protocol_incompatible
operator_required

attempt, failure_class, last_failure_ref, and next_retry_at are durable projection fields. Healthy rotations do not increment attempts. Retry policy is exponential with bounded jitter [D]; the default maximum is three failed attempts.

5. Event contract

Event	Required payload	Transition / reaction
TaskCreated	task_ref, project, source, external ID, scheduling fields	new → queued; dedup (source, external_id)
TaskAmended	scheduling fields only	reproject queued ordering
TaskLeased	lease, task_ref, handoff_ref?	queued → leased; worker validates offer
TaskLeaseStarted	lease token, pane/session evidence	leased → leased; launch ACK
TaskLeaseRenewed	lease token, new expiry	leased/needs_attention → same
TaskLeaseFailed	lease token, class, evidence ref, retry time	leased → queued/failed
TaskLeaseExpired	lease token, attempt, retry time	leased/needs_attention → queued/failed
TaskOrphanCheckpointed	old lease token, evidence ref, Git ref/SHA	observation only; never restores ownership
TaskNeedsAttention	lease token, class, evidence ref, pane evidence	leased → needs_attention
TaskResumed	lease token, resolution event	needs_attention → leased
ApprovalRequested	lease token, subject ref, bounded options	leased → needs_attention
ApprovalGranted/Denied	subject ref, actor, capture revision	command result; may resume
TaskHandoffPrepared	transaction ID, lease token, handoff ref, Git anchor/ref, reason	leased → leased
TaskReleased	transaction ID, lease token, handoff ref, anchor	leased → queued; predecessor parked
TaskPickupValidated	new lease token, transaction ID, handoff ref, anchor	leased → leased
TaskHandoffRetired	transaction ID, predecessor pane result	observation after successor start
TaskCompleted	lease token, report ref, result SHA/ref, receipt	leased/needs_attention → completed
TaskFailed	class, attempts, evidence ref	queued/leased/needs_attention → failed
TaskCorrected	corrected event ID, bounded compensation fields	explicit compensation only
QuotaReported	worker/harness, window, consumed, observed at	quota projection

Events not listed here are invalid until this contract is versioned.

Per-turn transcripts, pane liveness, and raw token events are not lifecycle events.

6. Worker launch and pickup

6.1 Initial launch

1. Verify current lease token and worker/project eligibility.
2. Fetch/verify task_ref; write exact immutable TASK.md.
3. Sync the configured base checkout by fast-forward only.
4. Create/open the task worktree and branch.
5. Ask local herdr to open the worktree and start the configured harness.
6. Confirm the exact pane contains the exact attached agent.
7. Persist session identity.
8. Deliver one bounded launch prompt.
9. Emit TaskLeaseStarted.

Before step 7, a partial pane creation failure is still reconciled and cleaned or retained as evidence. A launch failure emits TaskLeaseFailed; it does not hold capacity until TTL by default.

6.2 Handoff pickup

Before starting a successor agent:

1. Strict-decode and validate the handoff artifact.
2. Fetch the exact remote handoff ref.
3. Create a worktree at anchor.git_sha.
4. Verify HEAD == anchor.git_sha.
5. Verify the remote ref resolves to the same SHA.
6. Verify exact TASK.md hash equals task_ref.
7. Verify transaction ID, task ID, handoff ref, and new lease token.
8. Emit TaskPickupValidated.
9. Start/confirm the successor agent.
10. Deliver the bounded bootstrap prompt.
11. Emit TaskLeaseStarted.

Any failure emits TaskLeaseFailed(class=pickup_invalid) with evidence. The predecessor remains parked and recoverable.

7. Rotation and handoff

Rotation runs on the worker that owns the checkout. Coordinator-local rotation logic MUST NOT be a separate implementation.

7.1 Triggers

Trigger	Default	Action
soft occupancy	55%	request handoff at a natural boundary
hard occupancy	75%	prepare and rotate at the next verified boundary
milestone	successful coherent checkpoint	rotate
thrash	repeated failures/edits/calls	rotate with dead ends
manual	authorized operator/agent intent	rotate

Occupancy is current-turn context input, not cumulative token cost. Each harness adapter binds usage data to the exact session/pane; “newest file globally” is invalid when identity is ambiguous.

Harness	Exact source	Occupancy numerator
Claude Code	attached session transcript, last assistant usage	input + cache read + cache creation
Codex	rollout path bound to the attached thread/session	last input + last cached input
OpenCode	message/status record bound to the attached session	input + cache read

Cumulative session totals are receipts, never occupancy.

Unknown occupancy is observable. After two missed samples [D], request a conservative handoff at the next verified boundary; never interrupt a busy turn.

7.2 Canonical handoff artifact

The agent writes only bounded labelled answers. The worker derives protocol facts and seals strict JSON:

{
  "schema_version": 2,
  "handoff_id": "id",
  "transaction_id": "id",
  "task_id": "id",
  "lease_id": "id",
  "reason": "threshold|milestone|thrash|manual",
  "rotation_index": 1,
  "anchor": {
    "git_sha": "40-hex",
    "remote_ref": "refs/orchestra/tasks/<task>/<rotation>",
    "task_sha256": "64-hex"
  },
  "next": {
    "action": "one bounded line",
    "why": "one bounded line",
    "command": "optional literal command",
    "files": ["relative/path"]
  },
  "remaining": ["bounded item"],
  "dead_ends": [{"tried": "bounded", "why_failed": "bounded"}],
  "open_questions": ["bounded item"],
  "learned": ["bounded invariant"]
}

Goal and done criteria live only in immutable TASK.md. Completed work is derived from Git. The agent does not supply SHA, branch/ref, task hash, lease identity, completion claims, or a prose report.

Authored handoff fields are treated as bounded data. Orchestra never executes next.command automatically or promotes handoff text into control input.

7.3 Checkpoint algorithm

1. Verify lease and exact turn boundary.
2. Obtain and strict-parse bounded handoff answers.
3. Verify immutable TASK.md.
4. Remove/exclude transient protocol markers.
5. git add -A; staged, unstaged, untracked, renamed, and deleted paths count.
6. Commit to an immutable per-rotation ref if the index differs from HEAD.
7. Select current HEAD even when the work was already clean and committed.
8. Push exact SHA to refs/orchestra/tasks/<task>/<rotation>.
9. Verify ls-remote returns that exact SHA.
10. Seal/upload the handoff artifact.
11. Persist local transaction phase prepared.
12. Emit idempotent TaskHandoffPrepared.

No handoff may reference an object that is only present in a worker-local object database.

7.4 Release transaction

1. After TaskHandoffPrepared is durable, persist phase prepared_acked.
2. Ask herdr to pane.release_agent; retain pane, worktree, report, artifact, and transaction state.
3. Reconcile ambiguous herdr results by inspecting the exact pane/agent.
4. Persist phase predecessor_parked.
5. Emit idempotent TaskReleased.
6. On lost response, pull events by transaction ID and resume the known phase.
7. Successor completes §6.2 and emits TaskLeaseStarted.
8. Coordinator queues an idempotent retire command to the predecessor worker.
9. Predecessor closes the exact old pane, cleans protocol markers, persists retired, and emits TaskHandoffRetired.

The predecessor MUST NOT be closed at step 5. It MUST NOT resume work after release without receiving a new lease epoch.

8. Completion

The agent may create an empty, bounded completion-intent marker. The worker does not trust a narrative completion report.

At a verified idle boundary:

1. Verify lease token and immutable TASK.md.
2. If no completion, handoff, or blocker intent exists, issue one bounded continuation prompt requiring exactly one of: continue work, completion intent, handoff answers, or blocker data.
3. On completion intent, run the configured worker-owned quality gate.
4. Reject a gate that modifies files unless policy explicitly permits it and the gate is rerun on the resulting tree.
5. Stage all intended changes, commit if required, and obtain result SHA.
6. Push an immutable result ref.
7. Verify remote ref → exact result SHA.
8. Build a worker-derived report artifact containing gate command/exit, result ref/SHA, task hash, timestamps, and pane/session evidence.
9. Record per-lease usage delta.
10. Emit idempotent TaskCompleted with the lease token.
11. Only after durable acknowledgement, close the pane and clean the worktree according to retention policy.

A late completion from needs_attention is valid only for its retained lease token. A stale or superseded lease can never complete.

9. Herdr contract

• Transport is raw JSON-RPC over TCP or Unix socket, not HTTP.
• Every request includes params; use {} for parameterless methods.
• Protocol is checked with ping; protocol 17 is a JSON number.
• Relevant real methods include worktree.create/open, agent.start/get, agent.prompt, pane.get/read, pane.send_text/keys, pane.release_agent, and pane.close.
• pane.release, pane.kill, pane.rotation_signal, and pane.status do not exist.
• Herdr never creates a handoff or returns a handoff reference.
• pane.release_agent({pane_id,source,agent}) drops a binding; pane.close({pane_id}) retires the pane.
• UI-changing calls are not retried after an ambiguous post-write transport failure. The worker reconciles state first.
• Orchestra never prompts a blocked pane or a visible permission dialog.
• Approval input is bound to an exact pane and capture revision and uses only a recognized explicit control.

10. Routing, quota, and performance

10.1 Scheduling

Router evaluation occurs when:

• a task becomes queued;
• retry time arrives;
• a worker becomes eligible;
• capacity or quota changes.

Candidate order:

1. hard project affinity;
2. supported local project;
3. fresh worker and local-herdr health;
4. capability match;
5. free concurrency;
6. quota headroom;
7. derived importance;
8. creation sequence, then task ID for deterministic ties.

The scheduler uses one immutable task/worker/health snapshot per pass. It MUST NOT perform network probes or full event-log scans inside the task-candidate loop.

10.2 Quota

Quota and context occupancy are separate:

• occupancy: current session window; drives rotation;
• quota: subscription usage across sessions; drives eligibility.

Maintain indexed 5-hour and 7-day projections per worker/harness. Receipts are per-lease deltas and sum across rotations. Unknown or stale quota makes a bounded pool unavailable [D]; 80% reported usage is treated as full [D].

10.3 Performance requirements

• Active leases, external task keys, worker eligibility, and quota windows have indexed projections.
• Event APIs are cursor-paginated and bounded.
• Slow notification/provider consumers cannot block append or scheduling.
• Snapshots are periodic/checkpointed, not rewritten synchronously per event.
• Benchmarks cover 1k and 10k queued tasks and publish append/assignment p50, p95, allocation count, and projection rebuild time.
• Default target [D]: schedule 10k tasks against 100 worker slots in ≤250 ms p95 on the deployment-class coordinator, excluding Git/herdr work.

11. Authorization and approvals

Authorization is enforced at the append/command bus:

Surface	Rights
system worker/coordinator	scoped protocol operations
web/TUI	full operator control
MCP/Maven	read plus approval-gated writes
ntfy/telegram	notification only

Workers are authorized only for their own lease tokens, captures, and command queue. “System” cannot be selected by an HTTP header.

Approval policy is project-scoped and audited:

• MAY pre-authorize safe reads, edits, tests, formatting, and Git operations inside the isolated worktree;
• MUST gate destructive operations, secret access, arbitrary network access, privilege changes, and paths outside the worktree;
• has no automatic timeout;
• records requested option, capture revision, actor/policy, worker ACK, and resulting lifecycle event.

12. Providers, reflection, and operations

• Providers ingest by stable (source, external_id) and reflect terminal or attention state without becoming task truth.
• Ingestion is idempotent; write-back cannot create an ingestion loop.
• The JSONL provider is the baseline. Gitea/Vikunja adapters use the same port.
• Notifications are asynchronous subscribers with durable cursors.
• The brief is a read-only windowed projection of completions, failures, attention, approvals, quota, worker health, and Git refs.
• Mid-day handoff and overnight batch are the same lifecycle. The morning brief is the overnight window; workpc pulls verified result refs on wake.
• Parent links project task → epic → roadmap. Sprint is a queued-task selection; estimates retain author and confidence.
• Standups are advisory projections and require the same approval path as other control changes.
• Shared AGENTS.md, CLAUDE.md, and vocabulary files are agent-editable repository state. Workers hash them at lease time and notify adjacent live sessions when the hash changes; pickup always rereads them.
• Deterministic quality-gate residue that is outside the current task creates a deduplicated follow-up task instead of relying on an agent promise.
• Health exposes coordinator revision/protocol, worker revision/protocol, local-herdr status, active lease/epoch, pane, phase, last error, retry time, occupancy source, quota freshness, and event lag.
• Investigative tooling uses read-only herdr calls. Destructive pane operations require explicit operator intent outside the protocol owner.

13. Release gate

Every production revision MUST pass:

go build ./...
go vet ./...
go test ./...
go test -race ./...
git diff --check

Required automated coverage:

1. Strict schema, transition, authorization, and lease-fencing tests.
2. Fuzz/property tests for replay and invalid event/artifact input.
3. Crash injection before/after every append, CAS, worker-state, Git push, herdr release, release event, pickup validation, and retire phase.
4. Lost request and lost response tests for every worker mutation.
5. Worker partition, lease self-fence, expiry, reassignment, and stale completion tests.
6. Staged, unstaged, untracked, renamed, deleted, and clean-committed handoff tests across distinct repositories/machines.
7. Invalid/missing remote ref, task hash, handoff, and pickup tests.
8. Coordinator and worker restart at every handoff phase.
9. Soft, hard, milestone, thrash, manual, completion, attention, approval, and late-completion paths.
10. Performance benchmarks from §10.3.

Required controlled live proof on each harness:

lease → start → work → prepare → release → pickup validate
→ successor start → predecessor retire → quality gate → push → complete

The proof records event IDs, lease epochs, transaction ID, pane IDs, task hash, handoff hash, Git refs/SHAs, usage receipt, build revisions, and checksums. Safe repository work completes without manual approvals under the configured project policy.

14. Migration and defaults

• Existing schema v1/v2 events remain immutable and replay through tolerant legacy readers.
• New protocol behavior emits schema v3 events only.
• Migration rebuilds projections; it never rewrites the log.
• Ambiguous legacy blocked/session state is needs_attention, never guessed active or completed.
• Raw events are retained indefinitely [D].
• No approval timeout [D].
• Hard affinity [D].
• Three failed attempts [D].
• Unknown bounded quota is unavailable [D].
• Unknown occupancy requests a conservative boundary handoff [D].

This specification defines the target. Passing isolated package tests does not establish conformance; only the live owner path and §13 do.