orchestra/progress.md

# Orchestra progress

Updated: 2026-07-27

## AUDIT.md remediation — in progress

Working through `AUDIT.md`'s blocking/secondary defects in order of the
"suggested order of attack." Each item below is landed, tested, and
committed individually; see the git log for the exact commits.

Fixed so far:
- **B2** — adapters were looked up by `session.Harness` (the harness kind,
  e.g. `"claude"`) in `Reconcile`/`expire`/`rotate`, but `AdapterFactory.Herdrs`
  is keyed by herdr instance id (e.g. `"homesrv-claude"`). Every one of those
  call sites silently no-opped. Added `Coordinator.adapterFor`, routed all
  four call sites through it. Regression test registers an adapter under a
  herdr-id key distinct from the harness kind and asserts rotation fires.
- **B1** — `CLIAdapter.Occupancy` called `a.Usage(s.PaneID)`, but the usage
  readers want a filesystem path to session state, not a herdr pane id.
  Added `herdr.Session.SessionFile` and per-harness resolution
  (`ClaudeSessionFile` by newest-mtime under Claude Code's own project
  directory; codex via the existing `CodexActiveUsage` sqlite discovery;
  opencode refuses loudly — needs a live session id, not resolvable from the
  worktree alone). A missing/unreadable session file is now a hard error,
  surfaced via new `SessionHealth.Occupancy`/`OccupancyError` fields on
  `GET /v1/tasks/{id}/health`, not a silent zero. **Still needs live
  verification against a real Claude Code session** (the spec's own
  acceptance bar for this phase) — not possible from this sandbox.
- **B4** — the router counted every `TaskReleased` (including rotation,
  which *is* a `TaskReleased` carrying a valid `handoff_ref`) against
  `MaxAttempts`, and double-counted by also incrementing on every
  subsequent lease. A task that rotated twice hit the default
  `MaxAttempts=3` and was killed. Now only a release without a
  `handoff_ref` (expiry/crash) advances the counter.
- **B8** — `X-Orchestra-Surface: system` was reachable from an HTTP request
  header in both `authz.HTTP` and `main.go`'s `surface` closure (the one
  every handler actually calls). Since no deployment sets
  `ORCHESTRA_SYSTEM_TOKEN`, this was an unauthenticated full-control bypass
  reachable from any LAN caller. Both call sites now downgrade `system` to
  `web` before doing anything else with it.
- **S1** — `Brief.From`/`To` and `GitSync.Branch`/`Head`/`Status` all shared
  one JSON tag each (Go only honors the first `json:"..."` tag on a
  combined field declaration). `go vet ./...` now passes clean.
- **S5** — `Store.Lease`/`ExpireLeases` set `Event.ID` to the task id, so
  every lease of a task produced colliding event IDs. Now `domain.NewID()`.
- **S6** — the ingest dedup path returned `nil` (success) without
  appending; `main.go` then returned an unrelated event with `201`. Added
  `domain.ErrDuplicate` and `Store.TaskBySource`; `POST /v1/tasks` now
  returns the existing task with `200` on a duplicate. Updated every other
  `Append` caller (Gitea poll/webhook, JSONL ingest) to treat
  `ErrDuplicate` as expected rather than a failure — without that, Gitea
  polling would error out of its scan loop on the first already-ingested
  issue in every batch.

Not yet started: B3 (no `TaskCompleted` producer / no Stop hook), B5/B6
(herdr protocol verification, Layer 3 wiring), B7 (quota projection has no
producer), S2–S4, S7–S11, and the Phase 0 live-herdr verification step that
several of the above still need. See `AUDIT.md` for the full plan.

## Current state

This is a working Go implementation of `orchestra-spec (1).md`'s Layer 1–3
(substrate, harness/rotation, continuity) plus a first cut of Layer 4
(surfaces). `go build ./...` and `go test ./...` both pass. The codebase is
small (~4.6k lines across `internal/{domain,store,provider,registry,router,
herdr,orchestrator,continuity,federation,delivery,authz,operations,admin}`
and `cmd/orchestra/main.go`).

Earlier revisions of this file accumulated a long, self-contradictory
chronological log — gaps were listed as open in one section and then claimed
closed in a later section, sometimes inaccurately. This revision replaces
that log with one audited snapshot. Treat prior git history of this file as
session notes, not as ground truth.

### Verified fixed this pass

- **Rotation emitted an invalid `TaskReleased` (the previously reported
  highest-priority defect) — now fixed.** `internal/orchestrator.Coordinator.rotate`
  built the release payload as `{"handoff_ref","reason"}`, omitting the
  `anchor_sha` the spec (§4, §6.2) and `domain.ValidatePayload` require
  whenever `handoff_ref` is present. `store.Append` would reject it, the
  error was discarded (`if c.Store.Append(e) == nil`), and the lease/session
  silently never rotated — the coordinator would just retry next tick with
  no visible failure. Fixed by adding `herdr.HeadSHA(worktree)` and having
  `rotate` populate `anchor_sha` from the real worktree HEAD before
  appending; if the anchor can't be read, rotation now correctly skips that
  tick (leaving the lease intact for TTL/next-tick reclaim) instead of
  emitting a payload guaranteed to fail validation.
  Covered by `internal/orchestrator/rotation_test.go`
  (`TestRotationEmitsValidReleaseWithAnchorSHA`), which drives the real
  `Coordinator.Monitor` loop against an actual git worktree and asserts the
  emitted event passes `domain.ValidatePayload` with the correct SHA — the
  previous end-to-end test masked this bug by manually crafting a
  replacement `TaskReleased` event after observing the (silently failed)
  adapter-side release.
- **The federation worker release endpoint had the same gap.** The
  `/v1/federation/workers/{id}/release` handler (cmd/orchestra/main.go)
  built `TaskReleased` from a request body with only `handoff_ref`, no
  `anchor_sha`. Since a remote worker is the only party with the actual
  checkout (§2.1: "validate against the local checkout wherever the harness
  runs"), the endpoint now requires and forwards a 40-hex-char `anchor_sha`
  in the request body, rejecting the call with 400 otherwise.

### Multi-repo Gitea ingestion (new)

- `provider.Gitea` gained an optional `Project` field and `SourceName()`
  (`"gitea"` if unset, `"gitea:<project>"` if set) — the namespaced source
  doubles as the `(source,external_id)` dedup key, so issue #7 in two
  different repos never collides, and as the reflection dispatch key.
- New `provider.MultiGitea{Sources map[string]Gitea}` implements
  `TaskReflector` by looking up `task.Source` and forwarding to the matching
  Gitea instance — lets several Gitea repos (one per project) share one
  `ReflectingSink`.
- New `provider.GiteaSourceConfig` + `LoadGiteaConfigs(path)` load a JSON
  array of `{project,base_url,owner,repo,token,webhook_secret}`.
  `main.go` reads this from `ORCHESTRA_GITEA_CONFIG` if set; each source
  gets its own poll supervisor (`gitea:<project>`) and webhook path
  (`/v1/providers/gitea/webhook/<project>`).
- The legacy single-repo env vars (`ORCHESTRA_GITEA_URL/TOKEN/OWNER/REPO/
  WEBHOOK_SECRET`) still work unchanged when `ORCHESTRA_GITEA_CONFIG` is
  unset — same unprefixed webhook path, same `project = ORCHESTRA_GITEA_REPO`
  tagging, same dedup source `"gitea"` — so existing deployments and
  already-configured Gitea webhooks need no changes.
- Added `internal/provider/gitea_test.go` — previously **there were zero
  tests exercising the Gitea provider at all** despite progress.md's prior
  claim of Gitea webhook/poll test coverage; that claim was not accurate.
  New tests cover source-name namespacing, webhook signature
  verification/rejection, project tagging, `MultiGitea` dispatch-by-source
  (via two `httptest.Server`s, asserting only the right one is hit), and
  `LoadGiteaConfigs` validation/duplicate-project rejection.

### Per-project repos (new)

- `registry.Project` gained optional `repo`/`worktree_root` fields. Each
  project can now resolve its own git checkout rather than every project
  sharing one global `ORCHESTRA_REPO`/`ORCHESTRA_WORKTREE_ROOT` — matches
  spec §2.2 ("projects are first-class and extensible... the binding is a
  field + a config entry, not a schema change"). `main.go` builds a
  `orchestrator.PerProjectGitWorktrees` from the registry, falling back to
  the global default for any project that omits these fields, so
  single-repo deployments are unaffected. Covered by
  `internal/orchestrator/worktrees_test.go`.

### Closed this pass (were open gaps as of the last snapshot)

- **Bus-level authorization.** `authz.AuthorizeEvent` is now enforced inside
  `store.Append` itself — the single choke point every event passes through
  (HTTP handlers, router, coordinator/rotation, providers, federation relay)
  — not just at HTTP handlers. Event schema bumped to v2, which requires
  every event to declare a `Surface`; a new `authz.System` surface (full
  control) covers internal emitters (router leases/failures, coordinator
  releases/blocks, standup advisory/apply). Schema v1 events on disk still
  replay (tolerant reader). Covered by `internal/store/store_test.go` and
  `internal/router/router_test.go` additions asserting a non-HTTP append
  with no/wrong surface is rejected.
- **Dual quota windows.** `router.QuotaAvailability` now tracks a 5-hour
  rolling window and a 7-day weekly window independently per harness
  (`QuotaWindowLimits{FiveHour, Weekly}`), applying the conservative 80%
  rule to each separately — a harness over threshold on either window is
  unavailable. Replaces the old single-`Window` field. Covered by new
  `router_test.go` cases for weekly-only and 5h-only exhaustion.
- **Turn-boundary detection made observable, not silently optional.**
  Rotation still can't force a harness adapter to implement `TurnBoundary`
  Face B, but an adapter that fails to answer it now blocks that tick's
  release (never treats a failed check as "safe to proceed"), and any
  adapter without the capability — or one whose check errors — increments
  `MonitorHealth.TurnBoundaryDegraded`, exposed via the coordinator's health
  endpoint so degraded-safety operation is visible, not silent.
- **Cross-machine lease correctness has a real test.**
  `internal/integration/federation_lease_test.go`
  (`TestCrossMachineLeaseAnchorAndQuotaArePerHost`) exercises a lease
  claimed through the federation worker HTTP API, validates the anchor
  against that worker's own local checkout (not the router's), and asserts
  quota is accounted per-host. Spec §9 item 8 said "prove on the first
  federated run" — this is that proof for the primitives that exist today
  (registration, heartbeat, lease-claim); it does not yet run against two
  real physical machines.
- **Fuzz coverage for lifecycle payload validation.**
  `internal/domain/fuzz_test.go` adds `FuzzValidatePayload` and
  `FuzzValidateEvent` covering all event types (including malformed nested
  `receipt`/`knowledge` shapes) — asserts no panic and always a typed error
  on adversarial input.

### Believed accurate from prior sessions (spot-checked, not exhaustively re-verified)

- Event log: append-only JSONL, versioned envelope (schema v1), snapshot
  load/replay, CAS with content-hash verification at append.
- `domain.ValidatePayload` enforces required fields per event type,
  including `expected_version`/`ttl` on `TaskLeased`, `anchor_sha` on
  `TaskReleased` (now correctly emitted, see above), `report_ref`+`receipt`
  on `TaskCompleted`, and `blocker` on `TaskBlocked`.
- Router: project→affinity→machine resolution, capability match, quota
  availability at conservative 80% threshold, derived-importance ordering,
  retry-then-`TaskFailed`.
- herdr adapters (Claude/Codex/opencode) with native occupancy readers,
  optional `TurnBoundary`/`RotationSignal`/`PaneExit` capability interfaces,
  bootstrap/lease/release/kill.
- Continuity: strict handoff schema/validation, CAS save/load, pickup
  validation (HEAD match, dirty-file hashes, immutable `TASK.md` hash),
  scratch-branch commit/push/pull helpers.
- Provider layer: JSONL watcher, Gitea webhook+poll with HMAC auth,
  idempotent `(source,external_id)` dedup, terminal-state reflection,
  supervised restart with backoff.
- Federation: worker registration, heartbeat/TTL offline detection, event
  cursor polling/ack, lease claim endpoint.
- Authorization: bus-level capability table (notify-only / full / gated) is
  applied to lifecycle and approval writes via `AuthorizeEvent`.
- Delivery: Telegram/ntfy fan-out for completion/failure/block/approval
  events.
- `/readyz`, `/v1/brief`, `/v1/providers/health`, `/v1/standup` exist and
  return real state (not stubs).

## Known open gaps (named, not silently assumed done)

- **Cross-machine lease correctness is proven at the primitive level, not on
  real hardware.** `TestCrossMachineLeaseAnchorAndQuotaArePerHost` exercises
  the federation worker HTTP API (registration, lease-claim, anchor
  validation against the worker's own checkout, per-host quota) inside one
  test process. Spec §9 item 8 says "prove on the first federated run" —
  that means an actual homesrv/workpc pair over the real mesh, which this
  repo cannot exercise by itself. Named here as the one item that needs a
  live two-machine run to fully close, not more code.
- **Turn-boundary Face B still degrades to occupancy-only for adapters that
  don't implement it**, by design — the spec's Face B is per-harness native
  session state (Stop hook / rollout tail / SSE), which this repo can only
  wire against a real running herdr+harness pair. The degradation is now
  observable (`MonitorHealth.TurnBoundaryDegraded`) and blocks-on-failure
  rather than silently proceeding, but whether Claude/Codex/opencode's
  native hooks are wired in a live deployment is a deployment-config fact,
  not something provable from source alone.

Everything else named as open in the previous snapshot (bus-level
authorization, dual 5h/weekly quota windows, fuzz coverage of lifecycle
payload validation) is now closed — see "Closed this pass" above. Broader
areas (provider layer, continuity, router matching, delivery, federation
registration) were spot-checked against the code and their tests and
matched their described behavior.