orchestra/progress.md

Orchestra progress

Updated: 2026-07-26

Current state

This is a working Go implementation of orchestra-spec (1).md's Layer 1–3 (substrate, harness/rotation, continuity) plus a first cut of Layer 4 (surfaces). go build ./... and go test ./... both pass. The codebase is small (~4.6k lines across internal/{domain,store,provider,registry,router, herdr,orchestrator,continuity,federation,delivery,authz,operations,admin} and cmd/orchestra/main.go).

Earlier revisions of this file accumulated a long, self-contradictory chronological log — gaps were listed as open in one section and then claimed closed in a later section, sometimes inaccurately. This revision replaces that log with one audited snapshot. Treat prior git history of this file as session notes, not as ground truth.

Verified fixed this pass

• Rotation emitted an invalid TaskReleased (the previously reported highest-priority defect) — now fixed. internal/orchestrator.Coordinator.rotate built the release payload as {"handoff_ref","reason"}, omitting the anchor_sha the spec (§4, §6.2) and domain.ValidatePayload require whenever handoff_ref is present. store.Append would reject it, the error was discarded (if c.Store.Append(e) == nil), and the lease/session silently never rotated — the coordinator would just retry next tick with no visible failure. Fixed by adding herdr.HeadSHA(worktree) and having rotate populate anchor_sha from the real worktree HEAD before appending; if the anchor can't be read, rotation now correctly skips that tick (leaving the lease intact for TTL/next-tick reclaim) instead of emitting a payload guaranteed to fail validation. Covered by internal/orchestrator/rotation_test.go (TestRotationEmitsValidReleaseWithAnchorSHA), which drives the real Coordinator.Monitor loop against an actual git worktree and asserts the emitted event passes domain.ValidatePayload with the correct SHA — the previous end-to-end test masked this bug by manually crafting a replacement TaskReleased event after observing the (silently failed) adapter-side release.
• The federation worker release endpoint had the same gap. The /v1/federation/workers/{id}/release handler (cmd/orchestra/main.go) built TaskReleased from a request body with only handoff_ref, no anchor_sha. Since a remote worker is the only party with the actual checkout (§2.1: "validate against the local checkout wherever the harness runs"), the endpoint now requires and forwards a 40-hex-char anchor_sha in the request body, rejecting the call with 400 otherwise.

Multi-repo Gitea ingestion (new)

• provider.Gitea gained an optional Project field and SourceName() ("gitea" if unset, "gitea:<project>" if set) — the namespaced source doubles as the (source,external_id) dedup key, so issue #7 in two different repos never collides, and as the reflection dispatch key.
• New provider.MultiGitea{Sources map[string]Gitea} implements TaskReflector by looking up task.Source and forwarding to the matching Gitea instance — lets several Gitea repos (one per project) share one ReflectingSink.
• New provider.GiteaSourceConfig + LoadGiteaConfigs(path) load a JSON array of {project,base_url,owner,repo,token,webhook_secret}. main.go reads this from ORCHESTRA_GITEA_CONFIG if set; each source gets its own poll supervisor (gitea:<project>) and webhook path (/v1/providers/gitea/webhook/<project>).
• The legacy single-repo env vars (ORCHESTRA_GITEA_URL/TOKEN/OWNER/REPO/ WEBHOOK_SECRET) still work unchanged when ORCHESTRA_GITEA_CONFIG is unset — same unprefixed webhook path, same project = ORCHESTRA_GITEA_REPO tagging, same dedup source "gitea" — so existing deployments and already-configured Gitea webhooks need no changes.
• Added internal/provider/gitea_test.go — previously there were zero tests exercising the Gitea provider at all despite progress.md's prior claim of Gitea webhook/poll test coverage; that claim was not accurate. New tests cover source-name namespacing, webhook signature verification/rejection, project tagging, MultiGitea dispatch-by-source (via two httptest.Servers, asserting only the right one is hit), and LoadGiteaConfigs validation/duplicate-project rejection.

Per-project repos (new)

• registry.Project gained optional repo/worktree_root fields. Each project can now resolve its own git checkout rather than every project sharing one global ORCHESTRA_REPO/ORCHESTRA_WORKTREE_ROOT — matches spec §2.2 ("projects are first-class and extensible... the binding is a field + a config entry, not a schema change"). main.go builds a orchestrator.PerProjectGitWorktrees from the registry, falling back to the global default for any project that omits these fields, so single-repo deployments are unaffected. Covered by internal/orchestrator/worktrees_test.go.

Closed this pass (were open gaps as of the last snapshot)

• Bus-level authorization. authz.AuthorizeEvent is now enforced inside store.Append itself — the single choke point every event passes through (HTTP handlers, router, coordinator/rotation, providers, federation relay) — not just at HTTP handlers. Event schema bumped to v2, which requires every event to declare a Surface; a new authz.System surface (full control) covers internal emitters (router leases/failures, coordinator releases/blocks, standup advisory/apply). Schema v1 events on disk still replay (tolerant reader). Covered by internal/store/store_test.go and internal/router/router_test.go additions asserting a non-HTTP append with no/wrong surface is rejected.
• Dual quota windows. router.QuotaAvailability now tracks a 5-hour rolling window and a 7-day weekly window independently per harness (QuotaWindowLimits{FiveHour, Weekly}), applying the conservative 80% rule to each separately — a harness over threshold on either window is unavailable. Replaces the old single-Window field. Covered by new router_test.go cases for weekly-only and 5h-only exhaustion.
• Turn-boundary detection made observable, not silently optional. Rotation still can't force a harness adapter to implement TurnBoundary Face B, but an adapter that fails to answer it now blocks that tick's release (never treats a failed check as "safe to proceed"), and any adapter without the capability — or one whose check errors — increments MonitorHealth.TurnBoundaryDegraded, exposed via the coordinator's health endpoint so degraded-safety operation is visible, not silent.
• Cross-machine lease correctness has a real test. internal/integration/federation_lease_test.go (TestCrossMachineLeaseAnchorAndQuotaArePerHost) exercises a lease claimed through the federation worker HTTP API, validates the anchor against that worker's own local checkout (not the router's), and asserts quota is accounted per-host. Spec §9 item 8 said "prove on the first federated run" — this is that proof for the primitives that exist today (registration, heartbeat, lease-claim); it does not yet run against two real physical machines.
• Fuzz coverage for lifecycle payload validation. internal/domain/fuzz_test.go adds FuzzValidatePayload and FuzzValidateEvent covering all event types (including malformed nested receipt/knowledge shapes) — asserts no panic and always a typed error on adversarial input.

Believed accurate from prior sessions (spot-checked, not exhaustively re-verified)

• Event log: append-only JSONL, versioned envelope (schema v1), snapshot load/replay, CAS with content-hash verification at append.
• domain.ValidatePayload enforces required fields per event type, including expected_version/ttl on TaskLeased, anchor_sha on TaskReleased (now correctly emitted, see above), report_ref+receipt on TaskCompleted, and blocker on TaskBlocked.
• Router: project→affinity→machine resolution, capability match, quota availability at conservative 80% threshold, derived-importance ordering, retry-then-TaskFailed.
• herdr adapters (Claude/Codex/opencode) with native occupancy readers, optional TurnBoundary/RotationSignal/PaneExit capability interfaces, bootstrap/lease/release/kill.
• Continuity: strict handoff schema/validation, CAS save/load, pickup validation (HEAD match, dirty-file hashes, immutable TASK.md hash), scratch-branch commit/push/pull helpers.
• Provider layer: JSONL watcher, Gitea webhook+poll with HMAC auth, idempotent (source,external_id) dedup, terminal-state reflection, supervised restart with backoff.
• Federation: worker registration, heartbeat/TTL offline detection, event cursor polling/ack, lease claim endpoint.
• Authorization: bus-level capability table (notify-only / full / gated) is applied to lifecycle and approval writes via AuthorizeEvent.
• Delivery: Telegram/ntfy fan-out for completion/failure/block/approval events.
• /readyz, /v1/brief, /v1/providers/health, /v1/standup exist and return real state (not stubs).

Known open gaps (named, not silently assumed done)

• Cross-machine lease correctness is proven at the primitive level, not on real hardware. TestCrossMachineLeaseAnchorAndQuotaArePerHost exercises the federation worker HTTP API (registration, lease-claim, anchor validation against the worker's own checkout, per-host quota) inside one test process. Spec §9 item 8 says "prove on the first federated run" — that means an actual homesrv/workpc pair over the real mesh, which this repo cannot exercise by itself. Named here as the one item that needs a live two-machine run to fully close, not more code.
• Turn-boundary Face B still degrades to occupancy-only for adapters that don't implement it, by design — the spec's Face B is per-harness native session state (Stop hook / rollout tail / SSE), which this repo can only wire against a real running herdr+harness pair. The degradation is now observable (MonitorHealth.TurnBoundaryDegraded) and blocks-on-failure rather than silently proceeding, but whether Claude/Codex/opencode's native hooks are wired in a live deployment is a deployment-config fact, not something provable from source alone.

Everything else named as open in the previous snapshot (bus-level authorization, dual 5h/weekly quota windows, fuzz coverage of lifecycle payload validation) is now closed — see "Closed this pass" above. Broader areas (provider layer, continuity, router matching, delivery, federation registration) were spot-checked against the code and their tests and matched their described behavior.