orchestra/progress.md

# Orchestra progress

Updated: 2026-07-26

## Current state

This is a working Go implementation of `orchestra-spec (1).md`'s Layer 1–3
(substrate, harness/rotation, continuity) plus a first cut of Layer 4
(surfaces). `go build ./...` and `go test ./...` both pass. The codebase is
small (~4.6k lines across `internal/{domain,store,provider,registry,router,
herdr,orchestrator,continuity,federation,delivery,authz,operations,admin}`
and `cmd/orchestra/main.go`).

Earlier revisions of this file accumulated a long, self-contradictory
chronological log — gaps were listed as open in one section and then claimed
closed in a later section, sometimes inaccurately. This revision replaces
that log with one audited snapshot. Treat prior git history of this file as
session notes, not as ground truth.

### Verified fixed this pass

- **Rotation emitted an invalid `TaskReleased` (the previously reported
  highest-priority defect) — now fixed.** `internal/orchestrator.Coordinator.rotate`
  built the release payload as `{"handoff_ref","reason"}`, omitting the
  `anchor_sha` the spec (§4, §6.2) and `domain.ValidatePayload` require
  whenever `handoff_ref` is present. `store.Append` would reject it, the
  error was discarded (`if c.Store.Append(e) == nil`), and the lease/session
  silently never rotated — the coordinator would just retry next tick with
  no visible failure. Fixed by adding `herdr.HeadSHA(worktree)` and having
  `rotate` populate `anchor_sha` from the real worktree HEAD before
  appending; if the anchor can't be read, rotation now correctly skips that
  tick (leaving the lease intact for TTL/next-tick reclaim) instead of
  emitting a payload guaranteed to fail validation.
  Covered by `internal/orchestrator/rotation_test.go`
  (`TestRotationEmitsValidReleaseWithAnchorSHA`), which drives the real
  `Coordinator.Monitor` loop against an actual git worktree and asserts the
  emitted event passes `domain.ValidatePayload` with the correct SHA — the
  previous end-to-end test masked this bug by manually crafting a
  replacement `TaskReleased` event after observing the (silently failed)
  adapter-side release.
- **The federation worker release endpoint had the same gap.** The
  `/v1/federation/workers/{id}/release` handler (cmd/orchestra/main.go)
  built `TaskReleased` from a request body with only `handoff_ref`, no
  `anchor_sha`. Since a remote worker is the only party with the actual
  checkout (§2.1: "validate against the local checkout wherever the harness
  runs"), the endpoint now requires and forwards a 40-hex-char `anchor_sha`
  in the request body, rejecting the call with 400 otherwise.

### Multi-repo Gitea ingestion (new)

- `provider.Gitea` gained an optional `Project` field and `SourceName()`
  (`"gitea"` if unset, `"gitea:<project>"` if set) — the namespaced source
  doubles as the `(source,external_id)` dedup key, so issue #7 in two
  different repos never collides, and as the reflection dispatch key.
- New `provider.MultiGitea{Sources map[string]Gitea}` implements
  `TaskReflector` by looking up `task.Source` and forwarding to the matching
  Gitea instance — lets several Gitea repos (one per project) share one
  `ReflectingSink`.
- New `provider.GiteaSourceConfig` + `LoadGiteaConfigs(path)` load a JSON
  array of `{project,base_url,owner,repo,token,webhook_secret}`.
  `main.go` reads this from `ORCHESTRA_GITEA_CONFIG` if set; each source
  gets its own poll supervisor (`gitea:<project>`) and webhook path
  (`/v1/providers/gitea/webhook/<project>`).
- The legacy single-repo env vars (`ORCHESTRA_GITEA_URL/TOKEN/OWNER/REPO/
  WEBHOOK_SECRET`) still work unchanged when `ORCHESTRA_GITEA_CONFIG` is
  unset — same unprefixed webhook path, same `project = ORCHESTRA_GITEA_REPO`
  tagging, same dedup source `"gitea"` — so existing deployments and
  already-configured Gitea webhooks need no changes.
- Added `internal/provider/gitea_test.go` — previously **there were zero
  tests exercising the Gitea provider at all** despite progress.md's prior
  claim of Gitea webhook/poll test coverage; that claim was not accurate.
  New tests cover source-name namespacing, webhook signature
  verification/rejection, project tagging, `MultiGitea` dispatch-by-source
  (via two `httptest.Server`s, asserting only the right one is hit), and
  `LoadGiteaConfigs` validation/duplicate-project rejection.

### Per-project repos (new)

- `registry.Project` gained optional `repo`/`worktree_root` fields. Each
  project can now resolve its own git checkout rather than every project
  sharing one global `ORCHESTRA_REPO`/`ORCHESTRA_WORKTREE_ROOT` — matches
  spec §2.2 ("projects are first-class and extensible... the binding is a
  field + a config entry, not a schema change"). `main.go` builds a
  `orchestrator.PerProjectGitWorktrees` from the registry, falling back to
  the global default for any project that omits these fields, so
  single-repo deployments are unaffected. Covered by
  `internal/orchestrator/worktrees_test.go`.

### Closed this pass (were open gaps as of the last snapshot)

- **Bus-level authorization.** `authz.AuthorizeEvent` is now enforced inside
  `store.Append` itself — the single choke point every event passes through
  (HTTP handlers, router, coordinator/rotation, providers, federation relay)
  — not just at HTTP handlers. Event schema bumped to v2, which requires
  every event to declare a `Surface`; a new `authz.System` surface (full
  control) covers internal emitters (router leases/failures, coordinator
  releases/blocks, standup advisory/apply). Schema v1 events on disk still
  replay (tolerant reader). Covered by `internal/store/store_test.go` and
  `internal/router/router_test.go` additions asserting a non-HTTP append
  with no/wrong surface is rejected.
- **Dual quota windows.** `router.QuotaAvailability` now tracks a 5-hour
  rolling window and a 7-day weekly window independently per harness
  (`QuotaWindowLimits{FiveHour, Weekly}`), applying the conservative 80%
  rule to each separately — a harness over threshold on either window is
  unavailable. Replaces the old single-`Window` field. Covered by new
  `router_test.go` cases for weekly-only and 5h-only exhaustion.
- **Turn-boundary detection made observable, not silently optional.**
  Rotation still can't force a harness adapter to implement `TurnBoundary`
  Face B, but an adapter that fails to answer it now blocks that tick's
  release (never treats a failed check as "safe to proceed"), and any
  adapter without the capability — or one whose check errors — increments
  `MonitorHealth.TurnBoundaryDegraded`, exposed via the coordinator's health
  endpoint so degraded-safety operation is visible, not silent.
- **Cross-machine lease correctness has a real test.**
  `internal/integration/federation_lease_test.go`
  (`TestCrossMachineLeaseAnchorAndQuotaArePerHost`) exercises a lease
  claimed through the federation worker HTTP API, validates the anchor
  against that worker's own local checkout (not the router's), and asserts
  quota is accounted per-host. Spec §9 item 8 said "prove on the first
  federated run" — this is that proof for the primitives that exist today
  (registration, heartbeat, lease-claim); it does not yet run against two
  real physical machines.
- **Fuzz coverage for lifecycle payload validation.**
  `internal/domain/fuzz_test.go` adds `FuzzValidatePayload` and
  `FuzzValidateEvent` covering all event types (including malformed nested
  `receipt`/`knowledge` shapes) — asserts no panic and always a typed error
  on adversarial input.

### Believed accurate from prior sessions (spot-checked, not exhaustively re-verified)

- Event log: append-only JSONL, versioned envelope (schema v1), snapshot
  load/replay, CAS with content-hash verification at append.
- `domain.ValidatePayload` enforces required fields per event type,
  including `expected_version`/`ttl` on `TaskLeased`, `anchor_sha` on
  `TaskReleased` (now correctly emitted, see above), `report_ref`+`receipt`
  on `TaskCompleted`, and `blocker` on `TaskBlocked`.
- Router: project→affinity→machine resolution, capability match, quota
  availability at conservative 80% threshold, derived-importance ordering,
  retry-then-`TaskFailed`.
- herdr adapters (Claude/Codex/opencode) with native occupancy readers,
  optional `TurnBoundary`/`RotationSignal`/`PaneExit` capability interfaces,
  bootstrap/lease/release/kill.
- Continuity: strict handoff schema/validation, CAS save/load, pickup
  validation (HEAD match, dirty-file hashes, immutable `TASK.md` hash),
  scratch-branch commit/push/pull helpers.
- Provider layer: JSONL watcher, Gitea webhook+poll with HMAC auth,
  idempotent `(source,external_id)` dedup, terminal-state reflection,
  supervised restart with backoff.
- Federation: worker registration, heartbeat/TTL offline detection, event
  cursor polling/ack, lease claim endpoint.
- Authorization: bus-level capability table (notify-only / full / gated) is
  applied to lifecycle and approval writes via `AuthorizeEvent`.
- Delivery: Telegram/ntfy fan-out for completion/failure/block/approval
  events.
- `/readyz`, `/v1/brief`, `/v1/providers/health`, `/v1/standup` exist and
  return real state (not stubs).

## Known open gaps (named, not silently assumed done)

- **Cross-machine lease correctness is proven at the primitive level, not on
  real hardware.** `TestCrossMachineLeaseAnchorAndQuotaArePerHost` exercises
  the federation worker HTTP API (registration, lease-claim, anchor
  validation against the worker's own checkout, per-host quota) inside one
  test process. Spec §9 item 8 says "prove on the first federated run" —
  that means an actual homesrv/workpc pair over the real mesh, which this
  repo cannot exercise by itself. Named here as the one item that needs a
  live two-machine run to fully close, not more code.
- **Turn-boundary Face B still degrades to occupancy-only for adapters that
  don't implement it**, by design — the spec's Face B is per-harness native
  session state (Stop hook / rollout tail / SSE), which this repo can only
  wire against a real running herdr+harness pair. The degradation is now
  observable (`MonitorHealth.TurnBoundaryDegraded`) and blocks-on-failure
  rather than silently proceeding, but whether Claude/Codex/opencode's
  native hooks are wired in a live deployment is a deployment-config fact,
  not something provable from source alone.

Everything else named as open in the previous snapshot (bus-level
authorization, dual 5h/weekly quota windows, fuzz coverage of lifecycle
payload validation) is now closed — see "Closed this pass" above. Broader
areas (provider layer, continuity, router matching, delivery, federation
registration) were spot-checked against the code and their tests and
matched their described behavior.