orchestra/AGENTS.md

Orchestra

A Go implementation of orchestra-spec (1).md — an unattended multi-agent task orchestrator that leases coding tasks to CLI harnesses (Claude Code, Codex, opencode) running inside herdr-managed panes, rotates them across context-window limits, and hands off work via a git-anchored continuity protocol.

Layout: internal/{domain,store,provider,registry,router,herdr,orchestrator, continuity,federation,delivery,authz,operations,admin} + cmd/orchestra/main.go.

Ground truth over documentation

This repo has a documented history of code that looks wired but isn't — packages with tests that pass in isolation while the live call path silently no-ops (bare continue on error, discarded return values). See AUDIT.md for the full audit and progress.md for a running log. Before trusting a claim in progress.md that something "works" or "is fixed," check the actual call site — the file is written by past sessions of this same assistant and has previously overstated completion.

The single most reliable way to verify herdr-adapter code is right: don't read internal/herdr/adapter.go and assume the method names are real. Ping the live herdr instance and check.

herdr protocol — verified against a live instance, 2026-07-27

• herdr speaks JSON-RPC over a raw TCP (or unix-socket) connection — not HTTP. internal/herdr/herdr.go's Client.Call is the only correct way to talk to it; a bare curl to the port returns nothing.
• Request shape: {"id":"<n>","method":"<name>","params":<object>}. Herdr's Rust JSON-RPC decoder requires params to be present and rejects a bare null — always send {} for parameterless calls (the client does this automatically).
• Full real method list is committed at deploy/herdr-schema.json, captured live from 192.168.1.105:9245 (the workpc herdr) since no local herdr CLI is available in this sandbox — the schema was reconstructed by sending an unknown method name and reading the unknown variant ... expected one of ... error, then probing each method of interest with params:{} / params:{pane_id:"nonexistent"} to read Rust serde's missing field <x> errors for its param shape.
• Confirmed invented (do not use, they don't exist): pane.release, pane.kill, pane.rotation_signal, pane.status. If you see these anywhere, it's a bug, not a valid call.
• Real replacements: pane.close({pane_id}) for kill; pane.release_agent({pane_id, source, agent}) for release (structurally different — does not return a handoff_ref, see below). No replacement exists for rotation_signal — herdr has no concept of Orchestra rotation.
• Architectural point that's easy to get wrong: herdr never produces a handoff. The agent writes the handoff artifact (§6.1 of the spec); herdr's role in "release" is only to drop its own claim on the pane/agent binding. Any adapter code that expects herdr to hand back a handoff_ref is wrong by construction, independent of whether the method name is right.
• Protocol version is returned as a JSON number (17), not a string, even though config.jsonc declares "protocol": "17" as a string. CheckProtocol's raw-bytes fallback happens to make this compare correctly today — don't "clean up" that code without checking this note first, or it might start doing a real numeric-vs-string comparison and break.

Deployment topology (as of 2026-07-27)

• Runs as orchestra.service on homesrv (this machine's own systemd — journalctl -u orchestra.service for logs; sudo isn't available in this sandbox environment, but plain journalctl without sudo works here).
• Config: .orchestra-config/orchestra.env (env vars) + .orchestra-config/config.jsonc (projects/machines/herdrs registry). ORCHESTRA_CONFIG=/etc/orchestra/config.jsonc — the deployed copy, not the repo's deploy/config.example.jsonc.
• Two machines in the registry: homesrv (192.168.1.104) and workpc (192.168.1.105), each nominally running 3 herdrs (claude/codex/opencode). In practice homesrv has no local herdr running (connection refused on 9245) — only workpc's herdr is live and reachable. main.go only logs herdr connection failures at startup, never successes, so "no log line" for a herdr does not mean it's down — check reachability directly.
• There was a real, live, stuck task as of 2026-07-27: workspace wA, task id 06FT6CKD9Y98AZRX6X8K3QXFZG, opencode harness, pane wA:p1, agent_status: "blocked". Likely stuck because rotation/release could never reach it (B2/B5). Check whether it's still stuck before assuming fixes here have taken effect operationally — code fixes don't retroactively unstick an already-orphaned pane; that needs a manual kill/restart once the release path is trustworthy.

The federation fork — read before touching anything cross-machine

Two incompatible designs coexist. Design A ("drive the remote socket", currently deployed via clients/herdr-bridge.go) has homesrv call worktree.create/agent.start etc. directly on workpc's herdr over TCP as if it were local — meaning anchor validation (git rev-parse HEAD) run by the coordinator executes on the wrong machine relative to the actual checkout. Design B ("workers pull tasks", /v1/federation/*) is fully built server-side but has zero clients — no worker binary exists. Decision (AUDIT.md, 2026-07-27): keep Design A through Phase 5, commit to Design B in Phase 6, with two guardrails landed immediately (refuse to rotate/cleanup a lease held by a non-local herdr rather than validate against the wrong checkout). Don't build on top of Design A's cross-machine calls without reading that section first.

Working conventions

• workpc worker deployment target: copy the built worker binary to workpc:~/orchestra-deploy/orchestra-worker (that is, /home/kami/orchestra-deploy/orchestra-worker), not directly to /usr/local/bin. The workpc deployment process installs from this staging path. Verify the remote checksum and Go build revision before restart.
• go build ./..., go vet ./..., and go test ./... must all pass — go vet was broken for a while (duplicate JSON struct tags) and nobody noticed because only build/test were being checked. Always run all three.
• Silent continue-on-error is the recurring bug pattern in this codebase (adapter lookups, rotation, expiry). When touching internal/orchestrator or internal/herdr, prefer a recorded/observable failure (MonitorHealth fields) over a bare continue — that's literally what turned B1/B2 invisible for as long as they were.
• Don't invoke destructive herdr calls (pane.close, pane.release_agent) against a real pane from an investigative/audit session without asking first — there is live operator state on the other end (see the stuck-task note above).