Commit Graph

23 Commits

Author SHA1 Message Date
kami a221502356 Let Orchestra establish plan progress instead of the implementer asserting it
A detailed plan that nothing enforces is a document. This makes the phases
executable: the implementer may write exactly one status, and every other
status is a conclusion Orchestra reaches by running the plan's own commands.

    agent may request:  ready_for_verification
    agent may not assert: verified, awaiting_manual_verification, failed, skipped

The worker resolves commands from the coordinator, never from the request, so a
request cannot smuggle in a command the planner did not write. They run as argv
through exec with Dir set to the worktree, which is the quality gate's existing
envelope and not a weaker one. There is no shell, so a pipe is a literal
argument.

Project policy decides executable reach. registry.Project.Verification matches
argv positionally, and an absent policy refuses everything: a plan command is
agent-authored, so inheriting the operator-authored gate's reach by default
would be the wrong direction to fail in. A refused command is refused before
anything runs, and the refusal names the project and the command so the planner
learns its real reach.

Two bindings make the record mean something later. PlanRef, so progress earned
under plan A cannot survive into plan B. AtSHA, so "verified" does not outlive
the code that made it true: a record whose commit has moved is retained as
provenance and rendered as stale, never as a claim about the current tree.
Both are the same failure this codebase already fixed for reviews, which bind
to the commit they examined.

Manual steps hold a phase at awaiting_manual_verification. The sign-off is an
ordinary human decision whose subject carries the plan ref and the phase id, so
a later "looks good" on an unrelated thread cannot satisfy a gate nobody was
discussing.

A plan sealed before plan.md declares no executable unit, and says so: the
implement context states that phase progress is unavailable and the work
continues under the old semantics. Inventing phases it never had would be worse
than admitting it has none.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CVbaKucEYBjMqVeUgJUsc1
2026-08-28 11:59:39 +04:00
kami dea56e4bcd Let systemd track the tmux server it restarts
Type=forking with no PIDFile left MainPID=0, so systemd read a crashed tmux
server as a clean exit and Restart=on-failure never fired. tmux -D runs the
server in the foreground, which gives systemd the real pid.

-D also turns exit-empty off, so the synthetic orchestra-runtime session is no
longer needed to hold the server open past its last agent pane. Nothing else
in the tree referenced that session.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011xsXyr5J1RACo71YeKG3Pu
2026-08-27 18:05:57 +04:00
kami fc4c29ff3f Match the worker unit's User to the installed one
The socket is /tmp/tmux-$UID, so orchestra-tmux.service and
orchestra-worker.service must run as the same user or the worker talks to a
different server. The installed unit has run as kami since before the split;
the repo copy still said orchestra.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011xsXyr5J1RACo71YeKG3Pu
2026-08-27 17:55:17 +04:00
kami daa5d20d9b Own the tmux execution runtime as its own service
F28. The worker spawns the tmux server on its first command, so the server and
every agent pane sit in the worker unit's cgroup. Restarting the worker
destroyed the sessions it was restarting to manage, and F16's missing-pane
branch has been firing on deployment rather than on real execution loss.

KillMode is not the fix. Under mixed systemd still SIGKILLs the cgroup
remainder once the main process exits, and process only encodes accidental
orphaning. The runtime becomes its own service instead.

The worker gains After= and Wants= on it, ordering only: a worker that finds
the runtime missing must report that rather than be stopped by it. The unit
holds an idle session so the server outlives its last agent pane.

User must match between the units, since the socket lives under /tmp/tmux-$UID.
The installed worker on workpc runs as kami while this file still says
orchestra; the staged copy is set to kami to match.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011xsXyr5J1RACo71YeKG3Pu
2026-08-27 17:48:28 +04:00
kami 2f7b209b62 Build stamped binaries in a throwaway worktree of HEAD
This checkout is shared with another session. Its uncommitted Go changes must
neither be compiled into a binary stamped with a commit revision nor block a
deploy, and a dirty-tree refusal does both jobs badly. deploy/build.sh now
builds in a detached worktree of the revision it stamps, and the container image
is built the same way rather than from the live checkout.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 23:50:02 +04:00
kami 7f12c7fc37 v3 workflow: intent, phases, review, submission, enforcement, burn-in
The v3 stack, previously an uncommitted working tree, plus this session's two
units and the burn-in instrument. This commit is the burn-in build identity:
coordinator and worker must both report this revision before a task is created.

Workflow (earlier sessions, uncommitted until now): human decision events and
reduction, source cursors and reconcile-before-launch, turn-boundary
reconciliation, internal/agentctx as the single renderer, ace-fca phases with
sealed artifacts, the trajectory gate, bounded grilling, independent review,
task pr enforcement, and human review reflection.

Capability restrictions at the agent boundary: an authz.Agent surface at
GatedWrite may ask and may not act. It also fixes two bugs the unit exposed --
gated surfaces could not reach the two endpoints written for them, and
RequestHumanDecision would block an unowned task while rejecting a question
from the session that did own it.

Turn-boundary reconcile-failure escalation: a streak of consecutive failures
asks the session to hand off, fenced on the lease epoch, with reconcile_failure
as a real handoff reason. The worker was dropping the coordinator's verdict on
the floor; it now acts on it.

Burn-in: herdr.WriteLaunchContext dumps the exact agentctx.Build result to
<worktree>/.orchestra/launch.md at every launch, local and federated. BURNIN.md
is the runbook. deploy/build.sh stamps both binaries from one commit.

go build, go vet and go test ./... pass, 20 packages.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 18:31:20 +04:00
kami 97a9c65302 Delete Design A, the harness-hook completion path, and retired deploy files
Acts on the seven review comments on PR #1.

Design A is gone (comment 4). clients/ deleted rather than tracked: with
workers carrying cross-machine work the bridge is undeployed, which supersedes
the 2026-07-27 "keep through Phase 5" decision. CLAUDE.md, AGENTS.md and
AUDIT.md updated from "retained" to "deleted".

The harness-hook completion path is gone (comment 10). Investigation of the
live OpenCode QA run showed orchestra-worker owns completion end to end: it
watches for .orchestra/done, confirms via AgentStatus that the agent is not
busy, then posts through /v1/federation/* with both lease epoch and expected
version. The hook scripts used a different, older convention
(.orchestra-report.md) and posted to /v1/harness/complete, which had already
been reduced to a 410 stub - so that path could not have completed a task.
Nothing exercised it, because the live run never used it. Deleted: the three
deploy/hooks scripts, the 410 route, the unmounted harnessCompletion handler,
and its test. That test passed against a handler no mux routed to, which is
the exact "looks wired but isn't" pattern CLAUDE.md warns about; the
constant-time token compare added to it earlier today goes with it, having
never been reachable. /v1/harness/turn is untouched and still live.

Retired deployment files (comments 8, 12, 14): deploy/orchestra.service and
deploy/redeploy.sh (which sudo-installed to /usr/local/bin and restarted that
unit), plus deploy/docker-api-entrypoint.sh. The entrypoint was safe to remove
once its premise was checked: env vars reach the container through
`env_file: .env` in compose.yaml, not by sourcing /etc/orchestra/orchestra.env
- only config.jsonc is bind-mounted there - and Dockerfile.api's line 17
already sets ORCHESTRA_DATA/ORCHESTRA_PORT. Dockerfile.api now execs
/app/orchestra directly. orchestra-worker.service is a different, current unit
and is kept.

deploy/config.example.json deleted as a duplicate (comment 6); the annotated
.jsonc is the one registry.go points at, and its header no longer tells the
reader to copy the file that just went away.

Documentation corrected beyond the deletions:
- CLAUDE.md's deployment section claimed the container bind-mounts
  /etc/orchestra:ro and its entrypoint sources the env file. Both wrong.
- AGENTS.md still described a systemd deployment on homesrv as of 2026-07-27.
- AUDIT.md's H5 row still described a "retained compatibility handler".
- deploy/DEPLOYMENT.md still named redeploy.sh as the deployment path.
- deploy/orchestra.env.example still cited EnvironmentFile=.

TOKEN_MINIMAL_WORKFLOW_PLAN.md (comment 2) is untouched: it and WEB_UI_PLAN.md
were both missed by REVIEW.md's documentation sweep, and reconciling a 534-line
forward-looking plan against AUDIT.md is its own task, not a review fixup.

Verified: go build ./..., go vet ./..., go test ./... all pass after the
deletions, and go list ./... has no node_modules entry. No live herdr or pane
was touched; nothing was deployed. The running image still predates this
commit until compose is rebuilt.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GEugbHVYfAXFpTqDYbByEB
2026-07-31 00:29:42 +04:00
kami 56f5aac582 Reconcile docs with reality; fix module graph, token compare, health
Acts on the 2026-07-30 senior review (REVIEW.md findings 1, 2, 4, 5, 7).

Docs (finding 1): CLAUDE.md and AGENTS.md both claimed Design B "has zero
clients - no worker binary exists". cmd/orchestra-worker/main.go is the
deployed worker, and the non-local-herdr guardrail has landed in
Coordinator.adapterFor. Both sections rewritten; AUDIT.md gains a matching
federation-status record. The Phase 5 retention / Phase 6 deletion decision
for Design A is preserved, not flattened.

clients/ un-ignored and tracked, including the .service unit and README:
deployed code belongs in version control. Design A is NOT deleted here.

progress.md (finding 2): the file was deleted after 636ed8a, yet CLAUDE.md
instructed every session to cross-check against it. References removed from
CLAUDE.md, AGENTS.md, internal/orchestrator/rotation_test.go (comment only)
and deploy/hooks/orchestra-codex-poll.sh; AUDIT.md now carries the log role.

web/go.mod (finding 4): a module stub ends the parent package graph at the
directory boundary, so go list ./... no longer yields
web/node_modules/flatted/golang/pkg/flatted. A build tag cannot work - the
package is in the package list before tags are evaluated. Local/CI-only
breakage: Dockerfile.api builds ./cmd/orchestra by explicit path and
.dockerignore already excluded node_modules.

orchestra-worker (finding 5): untracked (8.9MB, mode 100755, still on disk);
both binaries now gitignored.

Token compare (finding 7): cmd/orchestra/main.go:139,582 use
subtle.ConstantTimeCompare, matching the authz.go idiom. The token != ""
guard stays first, so an empty configured token still means auth-disabled
rather than auth-bypass. Three further plain != secret compares remain in
internal/federation/federation.go:343,346,368 - tracked, not fixed here.

Also included from the review pass: orchestrator.go records adapter-resolution
failures in SessionHealth.LastError instead of dropping them on a bare
continue, plus an Observed flag so lease-seeded health is not mistaken for a
live reading, with a covering test. GET /v1/tasks/<id>/health now returns a
record with last_error where it previously returned a bare 404.

REVIEW.md's own second pass claimed every checkable fact held up; four did
not. AUDIT.md never contained the false Design B claim (AGENTS.md was the
second copy), the guardrail is at orchestrator.go:312 not :309, the
progress.md site list missed the codex-poll hook, and only orchestra-worker
was tracked. Verified: go build, go vet, go test, and
go list ./... | grep node_modules all clean with every change applied
together. No live herdr or pane was touched; nothing was deployed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GEugbHVYfAXFpTqDYbByEB
2026-07-30 22:51:54 +04:00
kami 524d93d7cd ops: migrate legacy event sequences safely 2026-07-30 15:47:43 +04:00
kami e8fadfc998 Complete autonomous recovery controls 2026-07-30 14:57:25 +04:00
kami f6ee0e3060 Harden lease lifecycle durability 2026-07-30 14:34:29 +04:00
kami 1ff0af2e69 fix: make worker handoff rotation durable 2026-07-30 01:30:59 +04:00
kami 1ca9d64e89 Harden worker federation and operator UI 2026-07-29 13:31:31 +04:00
kami 95454afa72 Close B19-B21 and S12-S13, and fix the flaky router test
All five defects filed while implementing B18, plus the router flake that
predated them. None of this has run on the deployed instance: the service
is stopped and /usr/local/bin/orchestra predates every change here.

B20 is the one that could silently defeat approvals. The capture revision
was UnixNano, so it changed on every read and said nothing about whether
the pane had changed; it is now an FNV-1a hash of the pane text, changing
iff the text does. The worse half was precedence: capture() preferred the
coordinator over a published worker capture, handing Queue a timestamp the
owning worker's staleness check could never match, so every federated
approval resolved "stale" and the keystroke never happened. Worker captures
now win — their existence means a registered worker owns that pane — and
capturePane follows the same precedence via Capture.Source rather than
guessing.

B19 was filed as "federated approvals emit no event", which overstated it:
the resolution half already existed, and correctly fires only on an
acknowledged worker report. The missing half was the request. Server.action
now appends ApprovalRequested at queue time, subject_ref set to the command
ID the later resolution carries. If that append fails the queued command is
resolved "rejected" — a keystroke that left no audit trail must not run.

B21 bounds the command list: resolved commands prune after 30 minutes on
both Queue and Commands, pending ones never at any age, since dropping one
would discard an operator decision. The persistence half stays open and is
recorded as such — captures and commands are still in-memory only.

S12 splits ORCHESTRA_NTFY_TOKEN, which was both the secret handed to the
ntfy server and a valid inbound credential for the ntfy surface; the latter
is now ORCHESTRA_NTFY_SURFACE_TOKEN. Breaking: a deployment relying on the
old dual use has no inbound gate until it sets the new variable. S13
deletes the dead auth() copy of the authorization policy.

The router flake was in the test, not in assignment. Store.Tasks() ranges a
map, and the assertion indexed two separate Tasks() calls, failing whenever
the orderings disagreed; instrumenting it showed a valid TaskLeased and a
genuinely leased task on every "failing" run. It now snapshots once and
asserts that exactly one task is leased, and passes at -count=60.

AUDIT.md records what is still not done: the deployed env and binary, the
live re-verification B13-B17 has always lacked, and two operational faults
found in the journal that block it — all six herdrs are refusing
connections, and ntfy delivery is failing 403 on every send.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01535A3Y8RtkAi8wYuWhtkEd
2026-07-29 01:33:59 +04:00
kami b57894b183 Add web UI and worker capture/approval command channel
Introduces the browser-facing surface and the worker-side protocol that
backs it:

- internal/ui: joined read model plus per-task lifecycle and approval
  controls, kept separate from the raw endpoints workers and harnesses
  depend on.
- internal/webui + web/: Vite/React app, build output embedded via
  go:embed and served as an SPA fallback.
- federation: per-(worker, task) captures with a monotonic revision that
  advances only when pane text actually changes, and a command queue
  restricted to grant_approval / deny_approval, each bound to the capture
  revision the operator acted on.
- orchestra-worker: publishes captures and executes commands only after
  re-reading the pane and confirming the revision still matches. Sends
  keystrokes only for a visible y/n prompt or OpenCode's fully labelled
  selector, and refuses to deny through that selector rather than guess
  at unobservable navigation.

This is the ownership boundary AUDIT.md's B14 and B17 call for: approval
becomes an explicit, revision-bound operation executed by the worker that
owns the pane, instead of a side effect of prompting over a
coordinator-driven remote socket.

Also ignores the web build inputs and outputs. node_modules ships vendored
Go packages, so go build and go test walk into it if it is merely
untracked; both node_modules and .node_modules are excluded.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01535A3Y8RtkAi8wYuWhtkEd
2026-07-28 23:14:16 +04:00
kami 2cecbc4015 Add federation worker and canonical handoffs 2026-07-28 16:17:18 +04:00
kami d678959d65 feat(hooks): add Codex/opencode Stop-hook-equivalent poll scripts
Neither harness has a native Stop hook, so both poll for the newest
session/rollout file and post to /v1/harness/{complete,turn} the same way
Claude's Stop hook does.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1rkJ2hBMybnJctPbcy4tT
2026-07-28 00:09:48 +04:00
kami 72f6230b4a feat(harness): dispatch /v1/harness/complete by harness kind (codex/opencode)
Adds an optional "harness" field so codex/opencode completions route to
their own Usage readers instead of always assuming Claude's transcript
format; unblocks the server side named as open in progress.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1rkJ2hBMybnJctPbcy4tT
2026-07-27 23:31:10 +04:00
kami 972845bd98 feat(orchestrator): wire turn-decision endpoint and QuotaReported producer
Closes Phase 2 items 1-2 (AUDIT.md): Coordinator.TurnDecision evaluates
occupancy/turn-boundary/handoff state synchronously per turn and returns
continue/prepare_handoff/rotate_now/refuse, exposed via POST
/v1/harness/turn. The Claude Stop hook now calls it on ordinary turn
boundaries instead of no-op'ing, and exits 2 on refuse.

Also closes B7's post-hoc producer: /v1/harness/complete now appends a
QuotaReported event from the completing lease's harness usage, so the
router's quota-availability filter and the brief's quota_consumed stop
evaluating against a permanent zero. Live per-harness push producers
(Claude statusline, Codex rollout tail) remain unbuilt — investigation
recorded in AUDIT.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1rkJ2hBMybnJctPbcy4tT
2026-07-27 23:17:21 +04:00
kami f8a397a14c fix(harness): add automatic TaskCompleted producer (B3, partial)
POST /v1/harness/complete lets a Claude Code Stop hook report task
completion instead of relying on a human hitting the manual endpoint.
The hook only fires on an explicit .orchestra-report.md marker (not
every turn boundary); the server builds the receipt itself from the
real transcript via herdr.ClaudeUsage rather than trusting a
self-reported number. Codex/opencode producers and the turn-decision
endpoint are still unbuilt — see AUDIT.md/progress.md for scope.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1rkJ2hBMybnJctPbcy4tT
2026-07-27 21:24:17 +04:00
kami 63cda5557e fix(herdr): B5 — replace invented pane.kill/release/rotation_signal with real methods
Verified against a live herdr instance (192.168.1.105:9245) that pane.kill,
pane.release, and pane.rotation_signal never existed in the protocol, as
AUDIT.md's B5 suspected. Real method list captured in deploy/herdr-schema.json.

- Kill now calls the real pane.close({pane_id}).
- RotationSignal interface/method/call-site deleted; no real equivalent exists.
- Release now refuses loudly instead of calling a nonexistent method — the
  real pane.release_agent can't return a handoff_ref either way (herdr
  doesn't write handoffs, the agent does), so a real fix needs Phase 4
  handoff production first.

Also documents Phase 0 findings in AUDIT.md/progress.md, and adds
CLAUDE.md/AGENTS.md with project-specific knowledge (herdr protocol facts,
deployment topology, a currently-stuck live task, the federation fork) for
future sessions.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1rkJ2hBMybnJctPbcy4tT
2026-07-27 21:05:53 +04:00
kami ce6f02f9e6 checkpoint: multi-repo Gitea ingestion, per-project repos, rotation anchor_sha fix
Pre-existing uncommitted work found at session start: rotation now emits
anchor_sha on TaskReleased (previously silently dropped by store.Append
validation), multi-repo Gitea provider support, per-project git worktree
roots, and associated test coverage. Committing as a checkpoint before
starting remediation work tracked in AUDIT.md.
2026-07-27 18:15:02 +04:00
kami 8822e028bb Implement projections and operations 2026-07-26 19:14:46 +04:00