118ac9fbcb
Not my work. These nine files sat uncommitted in the shared checkout while another session worked on them, and the UI redesign that follows rewrites web/src/main.tsx and web/src/style.css. Committing first means that work is recoverable rather than overwritten. Contents, by inspection rather than by authorship: whitespace normalisation and edits across main.tsx, 568 added lines of style.css, client and client test changes, the orchestra-user line in build.sh, and docs updates to AGENTS.md, AUDIT.md, DEPLOYMENT.md and the env example. Committed at the operator's explicit instruction.
428 lines
27 KiB
Markdown
428 lines
27 KiB
Markdown
# Orchestra audit — handoff first
|
|
|
|
## 2026-08-11 — tmux backend and Claude in-place context rollover
|
|
|
|
- `orchestra-worker` now selects `ORCHESTRA_WORKER_BACKEND=herdr|tmux`.
|
|
The tmux backend is intentionally Claude-only: it creates one detached,
|
|
isolated tmux session per task, resolves configured tmux pane indexes rather
|
|
than assuming `0.0`, starts Claude in the task worktree, handles the exact
|
|
workspace-trust prompt, captures the pane, forwards explicit input, reports
|
|
health/status, and retains/kills the session at the existing lifecycle
|
|
boundaries. Codex and OpenCode still use herdr.
|
|
- Registry entries accept `"backend":"tmux"`. Those entries are worker-owned
|
|
even on the coordinator machine, so the Dockerized coordinator neither
|
|
probes nor tries to operate a host tmux socket. Their router health remains
|
|
gated by a fresh worker heartbeat and local backend check.
|
|
- Claude rotation no longer asks the agent to duplicate the workpc hook's
|
|
handoff in `.orchestra-handoff-report.md`. A changed `HANDOFF.md`, observed
|
|
after Claude is idle, drives the persisted in-place sequence `/clear` +
|
|
`ENTER`, then `@HANDOFF.md` + `ENTER`; the exhausted transcript identity is
|
|
discarded so the next observation resolves the new session. The canonical
|
|
cross-worker release protocol remains separate and unchanged.
|
|
- The portable workpc Claude configuration (`settings*.json`, `CLAUDE.md`,
|
|
status line, hooks, and helper scripts) was checksum-verified after copying
|
|
to homesrv. The prior homesrv files are backed up under
|
|
`/home/kami/.claude/backups/workpc-import-20260811T120000`; machine/session
|
|
state was excluded and local-only hooks were preserved.
|
|
- Verified in the source tree with the real isolated-tmux lifecycle test,
|
|
focused backend/worker tests, `go build ./...`, `go vet ./...`,
|
|
`go test ./...`, `go test -race ./...`, and the web lint/test/production
|
|
build. No worker binary, registry file, service, container, or live pane was
|
|
deployed/restarted/touched; live Claude/tmux behavior remains a deployment
|
|
QA item rather than a completed claim.
|
|
|
|
Audited 2026-07-30 against the working tree, spec, deployed coordinator,
|
|
workpc worker, event log, and live herdr (read-only).
|
|
|
|
**Verdict:** the source-level P0/P1/P2 fixes are in place, the live
|
|
coordinator can replay its repaired event log, and the current OpenCode worker
|
|
is connected. The system is not safe to run unattended until the controlled
|
|
QA matrix has passed for all three harnesses.
|
|
|
|
## Evidence
|
|
|
|
- `go build ./...`, `go vet ./...`, `go test ./...`, and `go test -race ./...`: pass.
|
|
- Live B17: release `seq=251`, re-lease `252`, completion `262`; the simple
|
|
probe needed six approvals, logged a `409 lease version conflict`, and
|
|
recorded `consumed:0`.
|
|
- Live follow-up: Docker owns the coordinator; the old systemd unit is
|
|
inactive. The pre-v2 repeated-`seq=1` event prefix was migrated with a
|
|
backup-preserving, explicit tool before the current coordinator replayed it.
|
|
- Deployment follow-up: coordinator and installed workpc OpenCode worker are
|
|
clean revision `d6cab133b56666f81f569f4c1c3c9a6f104088d1`. The worker service
|
|
restarted at 2026-07-30 16:07 +04, has emitted no federation failures since,
|
|
and its configured Unix-socket herdr answered `ping` with protocol `17`.
|
|
|
|
## Federation design status (updated 2026-07-30)
|
|
|
|
Design B ("workers pull tasks", `/v1/federation/*`) is the **live** design. It
|
|
is no longer clientless: `cmd/orchestra-worker/main.go` (~1,131 lines, tests in
|
|
`cmd/orchestra-worker/main_test.go`) is the deployed worker, and the workpc
|
|
OpenCode worker runs it. Any earlier statement here or in `CLAUDE.md` that
|
|
Design B "has zero clients — no worker binary exists" is obsolete.
|
|
|
|
The Design A guardrail from the 2026-07-27 decision has landed:
|
|
`Coordinator.adapterFor` (`internal/orchestrator/orchestrator.go`) refuses to
|
|
resolve an adapter for a session owned by a non-local herdr, so rotation and
|
|
cleanup can no longer validate a git anchor against the wrong machine's
|
|
checkout.
|
|
|
|
Design A is **deleted as of 2026-07-31**, superseding the 2026-07-27 "retain
|
|
through Phase 5" decision: the operator confirmed the bridge is undeployed now
|
|
that workers carry cross-machine work, so `clients/` was removed rather than
|
|
tracked. The Phase 6 cutover is therefore already done on this axis.
|
|
|
|
The legacy harness-hook completion path was removed in the same pass, once it
|
|
was confirmed that nothing calls it. `orchestra-worker` owns completion — it
|
|
watches for `.orchestra/done`, confirms via `AgentStatus` that the agent is not
|
|
busy, then posts through `/v1/federation/*` with the lease epoch and expected
|
|
version. Deleted: the `/v1/harness/complete` route (a `410` stub), its unmounted
|
|
`harnessCompletion` handler, that handler's test (green against unreachable
|
|
code — the pattern this audit exists to catch), and the three `deploy/hooks/`
|
|
scripts, which still used the older `.orchestra-report.md` marker and would
|
|
have failed against the `410`. `/v1/harness/turn` is unaffected and still live.
|
|
|
|
Note for the QA matrix: the OpenCode run completed through the worker path, so
|
|
no hook script was exercised. Nothing about hook-based completion was ever
|
|
verified live, which is why deleting it costs nothing.
|
|
|
|
## Remaining release blockers
|
|
|
|
- **Only OpenCode capacity is ready.** Workpc's `workpc-opencode` worker has
|
|
a configured project file and a reachable local herdr Unix socket. Homesrv
|
|
has no reachable herdr, and no Claude/Codex worker/herdr pair has been
|
|
verified, so the full three-harness matrix cannot begin yet.
|
|
- **B17 needs a fresh controlled run.** The historical probe's six approvals,
|
|
one `409 lease version conflict`, and `consumed:0` receipt came from the
|
|
old worker. They cannot be treated as evidence for the current worker until
|
|
a live OpenCode run is repeated; Claude and Codex require their own runs.
|
|
|
|
## QA handoff — next agent
|
|
|
|
1. **Preflight before creating work.** Read `GET /v1/federation/workers` and
|
|
coordinator diagnostics through an authenticated operator session. Confirm
|
|
each target worker reports revision `d6cab13`, supported `test-e2e`, and
|
|
fresh `herdr_status: reachable`; raw-ping its configured local Unix socket
|
|
with `params:{}` and confirm protocol 17. Confirm no pre-existing agents
|
|
or leased task on the target harness.
|
|
2. **OpenCode controlled continuity run.** Submit one new disposable
|
|
`test-e2e` task that makes a deterministic marker, releases at a clear turn
|
|
boundary, validates pickup from the resulting anchor, then completes.
|
|
Record event sequence, handoff ref, anchor SHA, transaction id, lease epoch,
|
|
native session evidence, quality-gate result, remote SHA, and a receipt
|
|
with known non-zero (or explicitly explained known-zero) usage. Do not use
|
|
destructive herdr calls against unrelated panes.
|
|
3. **Exercise rotation and recovery.** In separate disposable tasks, trigger
|
|
soft, hard, milestone, thrash, coordinator restart, worker restart/lost
|
|
response, stale completion, and corrupted-worker-state paths. Verify each
|
|
result is a fenced lifecycle event or durable `needs_attention`, never a
|
|
silent retry or orphaned pane. Preserve the predecessor until matching
|
|
pickup validation.
|
|
4. **Repeat on Claude and Codex only after provisioning their own reachable
|
|
worker/herdr pairs.** Do not treat OpenCode evidence as cross-harness
|
|
proof. After all runs, compare worker/coordinator revisions and checksums,
|
|
attach the artifacts/event ranges to this audit, and only then clear the
|
|
live release gate.
|
|
|
|
## P0 — correctness
|
|
|
|
| ID | Current failure | Required fix |
|
|
|---|---|---|
|
|
| H1 | **Closed 2026-07-30.** The checkout-owning worker and coordinator turn path now use `RotationStateMachine`. Workers persist harness-native identity (Claude/Codex transcript, OpenCode SQLite session id), apply soft/milestone/thrash/hard-boundary decisions, and record unknown activity/occupancy/boundary as degraded health rather than zero usage. | Verified by `go test -race ./...`; the existing turn-policy coverage now exercises the shared state machine. |
|
|
| H2 | **Closed 2026-07-30.** `PrepareRelease` verifies immutable `TASK.md`, checkpoints all repository work except protocol markers, always pushes the per-task project's scratch anchor, verifies it with `ls-remote`, and only then seals the CAS handoff. | `TestScratchCommitCapturesAllGitStatesExceptProtocolMarkers` covers staged, deleted, renamed, untracked, and protocol-marker cases; release uses the configured project remote. |
|
|
| H3 | **Closed 2026-07-30.** Worker state persists idempotent release transactions through `prepared → anchor_pushed → event_committed → pickup_validated → predecessor_retired`. Release/pickup endpoints bind transaction, anchor, and lease version; a predecessor remains mapped and is retired only after matching pickup validation. | `TestReleaseTransactionSurvivesReLeaseUntilMatchingPickup` covers transaction propagation and pickup epoch binding; full race suite passes. |
|
|
| H4 | **Closed 2026-07-30.** Every new lease carries an opaque durable `lease_epoch`; renew/release/pickup/complete validate the exact harness owner and epoch at the store boundary and federation API. Offline heartbeats retain leases until expiry, new workers require a fresh reachable local-herdr probe, and local/worker ownership loss stops or durably quarantines the old pane before its mapping is dropped. | `TestLeaseEpochFencesStaleOwnerLifecycleWrites`, `TestAvailableRequiresFreshReachableLocalHerdrHealth`, plus the full race suite cover stale re-lease/completion and health admission. |
|
|
| H5 | **Closed 2026-07-30.** `Store.Append` validates legal state/owner/epoch transitions, fsyncs the event before applying its projection, and replays projections solely from `events.jsonl` (snapshots are disposable caches). CAS, worker/federation/coordinator state use temp-file + fsync + rename; corrupt worker state aborts startup. The legacy `/v1/harness/complete` route and its handler were deleted outright on 2026-07-31 (previously a 410 stub plus an unmounted, separately-fenced handler). | `TestOpenRebuildsOnlyFromLogAndIgnoresCorruptSnapshot`, `TestWorkerRefusesCorruptDurableState`, and `go test -race ./...` pass. |
|
|
|
|
## P1 — autonomy and recovery
|
|
|
|
- **Recovery:** **Closed 2026-07-30.** Launch/recovery faults now emit
|
|
`TaskNeedsAttention`, retaining the durable harness owner and lease epoch.
|
|
Renew, release, expiry, and a late reconciled completion accept that same
|
|
fenced lease; worker state advances its expected aggregate version without
|
|
dropping the live session. `TaskBlocked` remains terminal for an explicit
|
|
operator block. `TestNeedsAttentionRetainsFencedLeaseForLateCompletion`
|
|
covers the durable recovery path.
|
|
- **Retries:** **Closed 2026-07-30.** Hand-off-less `TaskReleased` is the
|
|
single durable reclaim transition. It projects exponential `attempt`,
|
|
`next_retry_at`, and `failure_class`; router assignment reads those fields,
|
|
so coordinator restarts cannot reset a backoff or retry limit.
|
|
`TestReclaimPersistsAttemptAndBackoffAcrossReopen` covers replay.
|
|
- **Launch:** **Closed 2026-07-30.** Workers emit a fenced
|
|
`TaskLaunchAcknowledged` only after a local start/prompt is persisted.
|
|
Typed NACKs immediately reclaim transient unusable capacity, terminally
|
|
block invalid handoffs, and retain uncertain live panes for reconciliation.
|
|
- **Completion:** **Closed 2026-07-30.** `.orchestra/done` is explicit
|
|
intent only; the worker also requires native non-busy identity, runs its
|
|
quality gate, verifies immutable `TASK.md`, commits, pushes, and checks
|
|
the remote SHA before it emits completion.
|
|
- **Quota:** **Closed 2026-07-30.** Completion receipts contain native
|
|
per-lease deltas plus a known/unknown marker. Five-hour and weekly
|
|
projections are published from the same receipts; any bounded harness
|
|
without fresh known usage fails routing closed.
|
|
- **Approvals:** **Closed 2026-07-30.** Projects have a validated audited
|
|
`safe_operations` policy limited to worktree-local read/edit/test/Git.
|
|
Workers inject it into the task prompt; network, secrets, destructive
|
|
actions, and paths outside the worktree remain operator-gated.
|
|
- **Observability:** **Closed 2026-07-30.** Task projections now retain
|
|
lifecycle phase, last error, retry time/failure class, lease epoch, pane
|
|
state, and anchor. Release/anchor certification faults enter durable
|
|
`needs_attention` instead of disappearing through retry `continue` paths.
|
|
|
|
## P2 — performance
|
|
|
|
- **Closed 2026-07-30.** Each scheduling pass takes one atomic task/lease
|
|
snapshot, batches cached (TTL) reachability probes concurrently, and
|
|
evaluates candidate availability once. It no longer probes candidates or
|
|
scans active tasks once per queued task.
|
|
- **Closed 2026-07-30.** Active leases and per-harness, time-ordered quota
|
|
receipts are projection indexes. Availability uses indexed rolling-window
|
|
sums rather than decoding the event log; the task snapshot is a one-time
|
|
disposable compatibility cache rather than a full rewrite on every append.
|
|
- **Closed 2026-07-30.** `BenchmarkAssignPending{1K,10K}` and
|
|
`BenchmarkAppend{1K,10K}` report and enforce p95 budgets, with durable
|
|
fsync cost included in their respective paths.
|
|
|
|
## Delivery order
|
|
|
|
1. Durable event transitions + lease fencing.
|
|
2. Idempotent checkpoint/release/pickup transaction.
|
|
3. Worker-local rotation, completion, quota, and typed recovery.
|
|
4. Approval policy and performance indexes.
|
|
5. Only then: ingestion/UI expansion.
|
|
|
|
## Release gate
|
|
|
|
- **Pass:** build, vet, test, and race checks pass; unit/integration coverage
|
|
includes the defined fault and cross-machine cases.
|
|
- **Pending QA:** run the controlled soft, hard, milestone, thrash,
|
|
completion, and late-recovery paths on OpenCode, Claude, and Codex without
|
|
manual intervention for safe repository work. Follow the QA handoff above
|
|
and attach the resulting event ranges/artifacts before clearing this gate.
|
|
|
|
## Capability restrictions at the agent boundary (2026-08-26)
|
|
|
|
The rule now enforced: an agent may perform work and *request* lifecycle
|
|
changes; it may never perform one.
|
|
|
|
- **New `authz.Agent` surface**, `GatedWrite`. `AuthorizeEvent` refuses every
|
|
event type Orchestra owns (`WorkPhaseChanged`, `ReviewRecorded`,
|
|
`TaskSubmitted`, `TaskCompleted`, `TaskLeased`, `ApprovalGranted`, …), and the
|
|
HTTP gate refuses their endpoints. Credential is `ORCHESTRA_AGENT_TOKEN`, the
|
|
only Orchestra credential that may enter an agent pane.
|
|
- **Gated surfaces could not reach the two endpoints written for them.** The
|
|
middleware admitted only paths ending in `/approval`, so `decision-request`
|
|
and `deferred` were 403 before their handlers ran, and the handlers' own
|
|
`AuthorizeEvent("ApprovalRequested")` was dead code. The allowlist is now
|
|
`authz.GatedWritePath`, and the gate keys off `CapabilityFor(s) ==
|
|
GatedWrite` rather than naming MCP and Maven.
|
|
- **`/v1/harness/turn` was unreachable.** An unlabelled request defaults to the
|
|
Web surface, `sessions` is always non-nil, so every harness call returned 401
|
|
in any deployment with web credentials. It is now exempt from the surface gate
|
|
and authenticates its own bearer token in the handler, like federation does.
|
|
- **`RequestHumanDecision` now requires ownership and fences on the lease.** It
|
|
blocked any task in any state, and its `TaskBlocked` event carried no
|
|
`harness_id`/`lease_epoch`. That is backwards twice over: an agent credential
|
|
was a way to block a queued task nobody was working on, and a question from
|
|
the session that *did* own the task was rejected by `Store.Append`'s fence.
|
|
Both fixed; six tests that had been exercising the unleased state now lease
|
|
first.
|
|
|
|
Not built, deliberately: a `Capability` vocabulary, `CapabilitySet`, per-role
|
|
`SessionAuthority`, or role-to-capability project policy. One surface at one
|
|
capability level expresses "may ask, may not act", and the store already fences
|
|
every lifecycle event on `(harness_id, lease_epoch)`. Add roles when two agent
|
|
roles actually need different API rights — today the difference between an
|
|
implementer and a reviewer is what `agentctx` renders, not what the API allows.
|
|
|
|
Still open, and it is the real enforcement:
|
|
|
|
- **Execution capabilities are unenforced.** Network, secrets, paths outside
|
|
the worktree and destructive commands have no policy check; agents inherit
|
|
the harness pane's environment, and nothing in this repo sets it. `git push`
|
|
and a direct Gitea call succeed or fail purely on whatever credentials that
|
|
environment happens to hold. Credential isolation in the pane is an operator
|
|
task today, not a code path.
|
|
- **A reviewing session still cannot seal its own review**, and should not:
|
|
the worker seals on its behalf. That answers question 1 of the handoff.
|
|
|
|
## Turn-boundary reconcile-failure escalation (2026-08-26)
|
|
|
|
The gap the 2026-08-26 handoff deliberately left open. A failed reconcile at a
|
|
verified turn boundary was recorded and the turn continued, forever. One
|
|
failure is transient, so continuing is right. A streak means Orchestra can no
|
|
longer promise that the newest human input outranks the session's intent, and
|
|
continuing silently is exactly the failure shape this repo keeps producing.
|
|
|
|
- `Coordinator.ReconcileFailureHandoff` (default 3, `defaultReconcileFailureHandoff`)
|
|
is the number of *consecutive* failures that escalate. Env:
|
|
`ORCHESTRA_RECONCILE_FAILURE_HANDOFF`.
|
|
- `Coordinator.noteReconcileResult` holds the streak per task, **keyed on the
|
|
lease epoch**. A successor never inherits its predecessor's count, so no
|
|
release path needs a cleanup hook. A success deletes the entry. The count is
|
|
written into `MonitorHealth.Sessions[id].LastError` as
|
|
`reconcile human input (N consecutive): ...`.
|
|
- On escalation `TurnDecision` asks for a handoff with reason
|
|
`reconcile_failure` and answers `prepare_handoff`. `RemoteTurn` answers the
|
|
same for a worker-owned session, on the same threshold.
|
|
- **The escalation only fires where rotation had no reason of its own.** It sits
|
|
on the `continue` branch, so an existing rotate, refuse, or prepare_handoff
|
|
keeps its own reason rather than having a second one manufactured for it.
|
|
- `reconcile_failure` is a real handoff reason: added to `continuity.reasons`
|
|
(or the artifact it produces would fail validation), to `herdr.handoffReason`,
|
|
and to the new `orchestrator.bypassReason` — which replaces the
|
|
`manual || milestone || thrash` comparison that was duplicated in `rotate`
|
|
and `TurnDecision`. Release therefore runs through the existing bypass path,
|
|
with no new state, blocker or recovery protocol.
|
|
- `CLIAdapter.RequestHandoffReason` explains the reason to the agent and says
|
|
explicitly that it is not a judgement about its work.
|
|
- **The worker ignored the coordinator's verdict entirely.** `federatedTurn`
|
|
read `answer.Decisions` and dropped `answer.Verdict` on the floor, so a
|
|
federated session could never be asked to hand off for any coordinator-side
|
|
reason. It now requests the handoff and records it on the session, which is
|
|
what the release loop watches for.
|
|
|
|
Then the existing machinery does the useful part: the successor's
|
|
`Store.PreLease` reconcile fails closed while the source is down, so the task
|
|
waits in the queue instead of resuming from an older authority.
|
|
|
|
Proofs: `internal/orchestrator/reconcile_escalation_test.go` (streak, reset,
|
|
rotation-wins, release-through-bypass, no-source, delivery-failure-is-not-a-
|
|
reconcile-failure, federated parity), `internal/integration/reconcile_escalation_test.go`
|
|
(the full loop: two turns continue, third hands off, successor refused while the
|
|
source is down, correction reconciled on recovery), and
|
|
`cmd/orchestra-worker/main_test.go:TestFederatedTurnActsOnPrepareHandoffVerdict`.
|
|
|
|
Deviation from the requested shape: the threshold is one coordinator field, not
|
|
per-project `human_reconcile.turn_failure_handoff_after`. `TurnDecision` has no
|
|
project in scope, and `Soft`/`Hard`/`Thrash` are already coordinator-level for
|
|
the same reason. Making it per-project means plumbing the registry into the
|
|
coordinator, which is worth doing when a second project actually needs a
|
|
different number.
|
|
|
|
## Burn-in instrumentation and readiness probe (2026-08-26)
|
|
|
|
Feature work stops here. `BURNIN.md` is the runbook: evidence per run, the five
|
|
flows, the failure classification, and the pane credential cleanup.
|
|
|
|
One addition, because the burn-in's main inspection was impossible without it:
|
|
`herdr.WriteLaunchContext` dumps the exact `agentctx.Build` result to
|
|
`<worktree>/.orchestra/launch.md` at every launch, local (`Coordinator.Start`)
|
|
and federated (`orchestra-worker`). Reading it back from pane scrollback is not
|
|
equivalent, because the harness reflows and truncates. A write failure is
|
|
recorded, never fatal. Proof:
|
|
`internal/integration/reconcile_launch_test.go:TestLaunchWritesTheContextItSent`
|
|
compares the file against the instruction the adapter actually received.
|
|
|
|
Probed live state, and the burn-in is blocked on deployment, not on code:
|
|
|
|
- The API is up on homesrv (`/readyz` ready, gitea and jsonl configured).
|
|
- The workpc worker is up (pid 741, restarted 2026-08-26 11:39, no errors),
|
|
serving `workpc-claude` on a tmux backend and `workpc-opencode` on a herdr
|
|
unix socket.
|
|
- **The deployed worker is built from 97a9c65 (2026-07-30).** Every v3 unit from
|
|
the last two sessions is an uncommitted working-tree change, so neither the
|
|
container nor the worker has phases, review, submission, the agent surface, or
|
|
the reconcile escalation. Commit and redeploy both before any run.
|
|
- **Codex has no entry in `/etc/orchestra/harnesses.json`**, and no worker runs
|
|
on homesrv, so three of the four target harnesses cannot be exercised at all
|
|
today.
|
|
- Correction to an earlier assumption in this file: the live `config.jsonc` sets
|
|
no `backend` and no `address` on any of its six herdrs, so all six resolve to
|
|
`<machine>:9245` via `registry.defaultHerdrPort`. Both ports are closed, and
|
|
that is *not* evidence about the workpc harnesses, which use a tmux socket and
|
|
a unix socket. Federated reachability defers to worker heartbeat. Do not
|
|
diagnose harness availability from a TCP probe of 9245.
|
|
|
|
## Burn-in deployment, 2026-08-26 18:35
|
|
|
|
Burn-in build identity: `6f9300b549362c4c5788f8845b56aaff9672d993`. The v3 stack
|
|
is committed (`7f12c7f`), then startup revision logging (`86b67d9`), then this
|
|
record. Both halves are built from `6f9300b` so the identity is HEAD, and a
|
|
later `deploy/build.sh` cannot silently produce a different revision than the
|
|
one deployed.
|
|
|
|
- **Coordinator deployed.** Rebuilt on homesrv with `--build-arg
|
|
BUILD_REVISION/BUILD_TIME/BUILD_DIRTY`, container recreated, `/readyz` ready.
|
|
It now logs `orchestra revision 6f9300b... dirty false` at startup.
|
|
- **Worker staged, not installed.** `~/orchestra-deploy/orchestra-worker`,
|
|
sha256 `2b1c430...`. `install` and `systemctl restart` need root, which this
|
|
sandbox does not have, so the running worker is still the 2026-07-30 build.
|
|
Until it is installed the pair is mismatched and no task should be created.
|
|
- **Observability fixed before proceeding**, per the requirement that deployed
|
|
identity be evidence. Revision was previously visible only behind the operator
|
|
login, and the worker never logged its own build at all. Both now print it at
|
|
startup, so `docker logs orchestra-api` and `journalctl -u orchestra-worker`
|
|
are sufficient.
|
|
- **`deploy/build.sh`** stamps both binaries from one commit and refuses a dirty
|
|
tree, so a burn-in run cannot pair a new coordinator with an old worker.
|
|
- The deployed coordinator confirms the transport split directly:
|
|
`herdr workpc-opencode is worker-owned on workpc; coordinator probe skipped`,
|
|
while the three `homesrv-*` herdrs report `dial tcp 192.168.1.104:9245:
|
|
connect: connection refused`.
|
|
|
|
Not done, and both need root: the `/etc/orchestra/worker.env` scrub (mode 0600,
|
|
root-owned) and its in-pane verification. No agent should run before that.
|
|
|
|
## Pane environment and the opencode backend, probed 2026-08-26 18:45
|
|
|
|
Three findings, all blocking flow 1, none of them code defects in this session's
|
|
work.
|
|
|
|
- **herdr is not running on workpc.** `herdr status server` reports `not
|
|
running`; the socket refuses connections and its log stops at 2026-07-30.
|
|
`workpc-opencode` cannot start a pane. Note that the worker logs `serving
|
|
harness workpc-opencode (opencode) on herdr backend` at startup **without
|
|
touching the socket**, so that line is not evidence of reachability. Same
|
|
shape as the older note about the coordinator never logging a herdr success:
|
|
absence of an error is not evidence here either.
|
|
- **The two workpc harnesses inherit different environments.** An opencode pane
|
|
is created by the herdr daemon and inherits *herdr's* environment, so scrubbing
|
|
`/etc/orchestra/worker.env` does not affect it. A claude pane comes from
|
|
`TmuxBackend.StartAgent`, which runs `tmux new-session` via
|
|
`exec.CommandContext` with no `Env` set, so the tmux server inherits the
|
|
worker's full environment and every pane under it does too.
|
|
- **The scrub alone cannot close the tmux path.** The worker needs
|
|
`ORCHESTRA_WORKER_TOKEN*` and `ORCHESTRA_FEDERATION_ADMIT_TOKEN` to function,
|
|
and the pane inherits exactly those. Closing it needs a filtered `cmd.Env` in
|
|
the backend, or a tmux server started separately with a clean environment.
|
|
Deliberately not built now: flow 1 is opencode only, and the burn-in order
|
|
puts claude at step 6.
|
|
|
|
Sudo is not available in this sandbox, so the worker install, the restart, and
|
|
the `worker.env` scrub remain operator steps.
|
|
|
|
## Browser operator database and UI refresh (2026-08-26)
|
|
|
|
The browser login no longer depends on an operator copying a bcrypt hash into
|
|
deployment configuration. The live startup path in `cmd/orchestra/main.go`
|
|
opens `$ORCHESTRA_DATA/auth.db` through `internal/authn`, refuses to serve with
|
|
an empty operator database, and registers the database-backed session and
|
|
account handlers before wrapping the mux with `authz.HTTPWithSessions`.
|
|
|
|
- `auth.db` is an embedded bbolt database created mode 0600. Passwords are
|
|
bcrypt-hashed before the record is written; login also performs bcrypt for an
|
|
unknown username to avoid an account-existence timing shortcut.
|
|
- `orchestra-user set -data DIR -username NAME` reads and confirms a password
|
|
from the terminal, creates the first operator, and resets an existing one.
|
|
The Docker API image includes this helper. The authenticated Settings screen
|
|
changes the current username/password and revokes every session for that
|
|
identity.
|
|
- An existing `ORCHESTRA_WEB_USERNAME`/`ORCHESTRA_WEB_PASSWORD_HASH` pair is
|
|
imported once if and only if the database has no users. Once a user exists,
|
|
those variables are ignored with an explicit startup log, so an old `.env`
|
|
cannot overwrite a database credential.
|
|
- The browser now gets its actual username from `GET /v1/ui/session`, renders
|
|
it in the shell, and has a dedicated account page. The login view was rebuilt
|
|
as a responsive desktop/mobile entry experience.
|
|
- Frontend state drift was fixed at the same time: `needs_attention` and
|
|
`in_review`, plus the three newer block reasons, are in the TypeScript model,
|
|
board lanes, status colors, diagnosis copy, and filtering. The seven-state
|
|
"All" board now has an explicit layout instead of falling back to one column.
|
|
|
|
Verified from the working tree after rebuilding the embedded assets:
|
|
`go build ./...`, `go vet ./...`, and `go test ./...` all pass (21 test
|
|
packages). The frontend TypeScript build passes, all five API-client tests
|
|
pass, and Vite's production build emits the assets embedded by
|
|
`internal/webui`.
|