Close B19-B21 and S12-S13, and fix the flaky router test

All five defects filed while implementing B18, plus the router flake that
predated them. None of this has run on the deployed instance: the service
is stopped and /usr/local/bin/orchestra predates every change here.

B20 is the one that could silently defeat approvals. The capture revision
was UnixNano, so it changed on every read and said nothing about whether
the pane had changed; it is now an FNV-1a hash of the pane text, changing
iff the text does. The worse half was precedence: capture() preferred the
coordinator over a published worker capture, handing Queue a timestamp the
owning worker's staleness check could never match, so every federated
approval resolved "stale" and the keystroke never happened. Worker captures
now win — their existence means a registered worker owns that pane — and
capturePane follows the same precedence via Capture.Source rather than
guessing.

B19 was filed as "federated approvals emit no event", which overstated it:
the resolution half already existed, and correctly fires only on an
acknowledged worker report. The missing half was the request. Server.action
now appends ApprovalRequested at queue time, subject_ref set to the command
ID the later resolution carries. If that append fails the queued command is
resolved "rejected" — a keystroke that left no audit trail must not run.

B21 bounds the command list: resolved commands prune after 30 minutes on
both Queue and Commands, pending ones never at any age, since dropping one
would discard an operator decision. The persistence half stays open and is
recorded as such — captures and commands are still in-memory only.

S12 splits ORCHESTRA_NTFY_TOKEN, which was both the secret handed to the
ntfy server and a valid inbound credential for the ntfy surface; the latter
is now ORCHESTRA_NTFY_SURFACE_TOKEN. Breaking: a deployment relying on the
old dual use has no inbound gate until it sets the new variable. S13
deletes the dead auth() copy of the authorization policy.

The router flake was in the test, not in assignment. Store.Tasks() ranges a
map, and the assertion indexed two separate Tasks() calls, failing whenever
the orderings disagreed; instrumenting it showed a valid TaskLeased and a
genuinely leased task on every "failing" run. It now snapshots once and
asserts that exactly one task is leased, and passes at -count=60.

AUDIT.md records what is still not done: the deployed env and binary, the
live re-verification B13-B17 has always lacked, and two operational faults
found in the journal that block it — all six herdrs are refusing
connections, and ntfy delivery is failing 403 on every send.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01535A3Y8RtkAi8wYuWhtkEd
This commit is contained in:
kami
2026-07-29 01:33:59 +04:00
parent 0b4b52ac45
commit 95454afa72
8 changed files with 349 additions and 89 deletions
+101 -43
View File
@@ -88,11 +88,18 @@ path only and remains open for Design A. B18 (the unauthenticated web
surface) is closed in code but **requires an env change before the service
will start** — see B18 and "What's next".
Nothing is currently blocking in the sense B12/B13 were. The open defects
B19B21 and S12S13, all found while implementing B18, are correctness and
hygiene gaps in the new command channel rather than things that stop a task
from running. The real remaining risk is evidential: B13 through B17 were
each found live and none has been re-verified live.
Nothing is currently blocking in the sense B12/B13 were. B19B21 and S12S13
— the correctness and hygiene gaps found while implementing B18 are now
fixed in code and covered by tests (2026-07-28); each is marked closed in
its own section below. The flaky router test is also fixed, and the flake
was in the test, not in assignment.
The real remaining risk is entirely evidential now: B13 through B17 were
each found live and **none has been re-verified live**, and none of the
2026-07-28 fixes has run on the deployed instance either — the service is
stopped, `/usr/local/bin/orchestra` predates all of them, and installing a
new binary or editing `/etc/orchestra/orchestra.env` needs privileges this
sandbox does not have.
### B18 — the web UI surface is unauthenticated (found by audit 2026-07-28, uncommitted working tree) — closed (code fix; requires an env change before restart)
@@ -621,12 +628,13 @@ individually below — **B19** (federated approvals emit no event), **B20**
as build output; there is no documented step tying `web/` sources to a
rebuild of `internal/webui/assets`, so the two can silently diverge.
## Found while implementing B18 (2026-07-28) — not yet fixed
## Found while implementing B18 (2026-07-28) — all fixed the same day
These were noticed while wiring the session gate. None is fixed; all were
verified against the code rather than inferred.
These were noticed while wiring the session gate, and all five were fixed in
the following pass. Each section states the original defect first, then how
it was closed.
### B19 — federated approvals leave no audit trail
### B19 — federated approvals leave no audit trail — closed (2026-07-28)
`Server.action`'s `grant_approval` / `deny_approval` branch
(`internal/ui/ui.go:301-341`) has two exits that are **not** symmetric. The
@@ -648,7 +656,21 @@ already carries `acknowledged` / `rejected` / `stale`). Emitting
`ApprovalGranted` at queue time would claim an outcome that has not
happened yet.
### B20 — the local capture revision is a timestamp, not a change counter
**Fix.** The resolution half already existed and this entry understated it:
`/v1/federation/commands/` in `cmd/orchestra/main.go` appends
`ApprovalGranted`/`ApprovalDenied` — but only on `status == "acknowledged"`,
so a queued click never claims an outcome. What was missing was the request
half. `Server.action` now appends `ApprovalRequested` at queue time, with
`subject_ref` set to the command ID that the later resolution event carries,
plus `decision`, `worker`, `pane_id` and `capture_revision`. If that append
fails the queued command is immediately resolved `rejected`, so the worker
can never execute a keystroke that left no audit trail. `rejected`/`stale`
outcomes still emit no resolution event; the `ApprovalRequested` stays
unresolved and surfaces in `operations.NeedsAttention`, which is the honest
reading. Covered by
`TestFederatedApprovalIsQueuedAtTheWorkerRevisionAndAudited`.
### B20 — the local capture revision is a timestamp, not a change counter — closed (2026-07-28)
`Server.capture` (`ui.go:80`) fabricates `Revision:
uint64(time.Now().UnixNano())` for the coordinator path. The federation
@@ -670,7 +692,7 @@ Two consequences:
never happens. The precedence between the two capture sources needs to be
explicit rather than incidental.
### B21 — `Registry.commands` is append-only
### B21 — `Registry.commands` is append-only — closed (2026-07-28)
`r.commands[worker]` is only ever appended to (`federation.go:119`);
`CompleteCommand` flips a status in place and nothing is ever deleted.
@@ -679,7 +701,16 @@ rescans the entire history on every worker poll. Combined with the
in-memory-only storage already noted, the lifecycle is wrong at both ends:
it forgets across restarts and never forgets within one.
### S12 — `ORCHESTRA_NTFY_TOKEN` serves two unrelated purposes
**Fix (within-process half only).** `Registry.pruneCommands` drops resolved
commands older than `CommandRetention` (30 minutes), running on both `Queue`
and `Commands`, so the per-worker list is bounded and the poll path no
longer rescans unbounded history. Pending commands are never pruned, at any
age — dropping one would silently discard an operator decision. Covered by
`TestResolvedCommandsArePrunedButPendingOnesSurvive`. **The persistence half
of this defect is still open:** captures and commands remain in-memory only,
so a coordinator restart still drops pending approvals silently.
### S12 — `ORCHESTRA_NTFY_TOKEN` serves two unrelated purposes — closed (2026-07-28)
The same env var is the ntfy *server* credential (`main.go:1153`,
`delivery.Ntfy{Token: ...}`) and the authz *surface* credential for
@@ -688,7 +719,13 @@ trust boundaries: one is handed to a third-party notification server, the
other authenticates callers to Orchestra. Setting the former silently makes
it a valid inbound credential. They need separate variables.
### S13 — dead `auth()` middleware in `cmd/orchestra/main.go`
**Fix.** The `authz.Ntfy` surface token is now `ORCHESTRA_NTFY_SURFACE_TOKEN`;
`ORCHESTRA_NTFY_TOKEN` is once again only the credential handed to the ntfy
server. Documented in `deploy/orchestra.env.example`. **Operator action:**
any deployment that relied on the old dual use must now set the new variable
explicitly, or the ntfy surface reverts to having no inbound gate.
### S13 — dead `auth()` middleware in `cmd/orchestra/main.go` — closed (2026-07-28)
`func auth(next http.Handler)` (`main.go:1212`) is defined and never
referenced; Go does not flag unused functions, so `go vet` stays quiet. It
@@ -710,28 +747,41 @@ bypass is closed by construction rather than by the header check alone.
In dependency order, not importance order:
1. **Apply the B18 env change** before any restart —
`.orchestra-config/orchestra.env` needs `ORCHESTRA_WEB_TOKEN` (plus
`ORCHESTRA_UI_INSECURE_COOKIE=1` for plain HTTP), and every existing
unauthenticated `/v1/` client of this instance needs the token too. The
service will not start otherwise. Nothing else can be tested live until
this is done.
2. **S13, then S12** — deleting dead policy code and splitting the conflated
ntfy token are both small, local, and reduce the chance of the next
auth change being made against the wrong copy.
3. **B19** — decide the queued-approval event model and give federated
approvals an audit trail. This is the largest correctness gap in the new
command channel.
4. **B20 and B21** — make the capture revision mean one thing, make the
coordinator/worker precedence explicit, and give commands a retention
policy. B20 in particular can silently defeat approvals.
5. **Diagnose the flaky router test** (see open gaps). A nondeterministic
assignment test is exactly the failure shape this codebase has hidden
real defects behind before; do not paper over it with a retry.
6. **Then, and only then, the live re-verification** that B13B17 all still
lack. Each was found live and closed on paper; the code fixes are
unproven against a real cross-machine run, and that remains the single
biggest gap between this document and reality.
1. **Finish applying the B18 env change on the deployed instance.** The
repo-side `.orchestra-config/orchestra.env` now sets a generated
`ORCHESTRA_WEB_TOKEN` and `ORCHESTRA_UI_INSECURE_COOKIE=1`, but the unit
reads `/etc/orchestra/orchestra.env`, which is `orchestra:orchestra 0600`
and unreadable from this sandbox — whether it carries the token could not
be confirmed. It was last modified 2026-07-28 14:30 and the service did
start at 16:17 with the B18 build absent, so this is unverified either
way. Also newly required: `ORCHESTRA_NTFY_SURFACE_TOKEN` (S12) if the
ntfy surface should stay gated, and the token for every existing
unauthenticated `/v1/` client, which now defaults to the Web surface.
2. **Install the new binary and restart.** `orchestra.service` has been
stopped since 2026-07-28 19:46 and `/usr/local/bin/orchestra` predates
every fix in this section. Needs privileges the sandbox lacks:
`go build -o /tmp/orchestra ./cmd/orchestra && sudo install /tmp/orchestra
/usr/local/bin/orchestra && sudo systemctl restart orchestra.service`.
3. **The live re-verification** that B13B17 all still lack, plus a first
live exercise of B18B21. Each of B13B17 was found live and closed on
paper; the code fixes are unproven against a real cross-machine run, and
that remains the single biggest gap between this document and reality.
Note that no herdr is currently reachable at all — the 16:17 startup logs
`connection refused` for **all six**, including workpc's, which was live
on 2026-07-27 — so a live run needs a herdr brought up first.
4. **Two operational faults visible in the journal**, unrelated to this
audit's defects but blocking a clean live run: every herdr is refusing
connections (above), and ntfy delivery is failing `403 Forbidden` on
every send (16:43 and 16:46), i.e. the ntfy server credential is wrong or
expired.
5. **The persistence half of B21** — captures and commands are in-memory
only, so a restart still silently drops pending approvals. The retention
fix bounds growth within a process; it does not make the lifecycle
durable.
6. **`RespondApproval` (coordinator path) still has no test**, unlike its
worker counterpart. With B20 fixed its revision is now meaningful, but
its text-comparison guard remains the thing actually binding a decision
to what the operator saw, and that guard is untested.
Deliberately *not* next: building further on Design A's cross-machine calls,
and closing B17 for the coordinator path. Both wait on the federation-fork
@@ -1227,16 +1277,24 @@ Numbered defects are not repeated here — B19, B20, B21, S12 and S13 are in
in "What's next". This list is the unnumbered residue: conditions that are
known, accepted, or not actionable as a single fix.
- **B18's env change is not yet applied.** The code now refuses to start
without `ORCHESTRA_WEB_TOKEN`, but `.orchestra-config/orchestra.env` does
not set one — `orchestra.service` will fail to start until it does.
- **B18's env change is applied repo-side only.** `.orchestra-config/
orchestra.env` now sets `ORCHESTRA_WEB_TOKEN`, but the unit loads
`/etc/orchestra/orchestra.env`, which is not readable or writable from
this sandbox. See "What's next" item 1.
- **The listener still binds all interfaces**; loopback-by-default was
considered and not taken (B18).
- **`TestAssignsByAffinityCapabilityAndConcurrency` is flaky** (`no task
leased`, roughly 1 run in 10, reproduced with `-count=10`). Predates these
changes — confirmed by stashing them — and is unrelated to authz. A
nondeterministic router assignment test is exactly the kind of thing this
codebase has been bitten by before; it should be diagnosed, not retried.
- ~~**`TestAssignsByAffinityCapabilityAndConcurrency` is flaky**~~ —
diagnosed and fixed 2026-07-28. The flake was in the test, not in
assignment. `Store.Tasks()` ranges a map, so its order is randomized per
call, and the assertion indexed **two separate `Tasks()` calls**
(`s.Tasks()[0] ... && s.Tasks()[1] ...`); it failed whenever the two
orderings disagreed. Instrumenting the failure showed a valid `TaskLeased`
event and a genuinely leased task on every "failing" run. It now snapshots
once and asserts the real invariant — concurrency 1 means exactly one of
the two tasks is leased — and passes at `-count=60`. Worth recording that
*which* of two equal-priority tasks wins is still nondeterministic, since
`sort.SliceStable` is applied to a randomly ordered slice; that is a real
property of the router, not a test artifact.
- **B14/B15/B16 are code-fixed but not re-verified live**, and B17 is closed
only for tasks that run through a worker. Each was found live, so a code
fix plus unit tests is weaker evidence than the failure that produced it.
+5 -13
View File
@@ -1205,19 +1205,11 @@ func main() {
tokens := map[authz.Surface]string{
authz.TUI: os.Getenv("ORCHESTRA_TUI_TOKEN"), authz.Web: os.Getenv("ORCHESTRA_WEB_TOKEN"),
authz.MCP: os.Getenv("ORCHESTRA_MCP_TOKEN"), authz.Maven: os.Getenv("ORCHESTRA_MAVEN_TOKEN"),
authz.Telegram: os.Getenv("ORCHESTRA_TELEGRAM_TOKEN"), authz.Ntfy: os.Getenv("ORCHESTRA_NTFY_TOKEN"),
// S12: this is the credential callers present *to* Orchestra on the
// ntfy surface. ORCHESTRA_NTFY_TOKEN is a different secret entirely —
// it is handed out to the third-party ntfy server (see the sender
// above) and must never be accepted as an inbound credential.
authz.Telegram: os.Getenv("ORCHESTRA_TELEGRAM_TOKEN"), authz.Ntfy: os.Getenv("ORCHESTRA_NTFY_SURFACE_TOKEN"),
}
log.Fatal(http.ListenAndServe(":"+port, authz.HTTPWithSessions(tokens, sessions, mux)))
}
func auth(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
surface := strings.ToLower(r.Header.Get("X-Orchestra-Surface"))
if surface == "telegram" || surface == "ntfy" {
if r.Method != "GET" {
http.Error(w, "notify-only surface", 403)
return
}
}
next.ServeHTTP(w, r)
})
}
+15 -5
View File
@@ -64,7 +64,9 @@ ORCHESTRA_OCCUPANCY_HARD=0.75
# Telegram: both required together.
#ORCHESTRA_TELEGRAM_BOT_TOKEN=
#ORCHESTRA_TELEGRAM_CHAT_ID=
# ntfy: topic required, token/url optional (self-hosted ntfy).
# ntfy: topic required, token/url optional (self-hosted ntfy). This token is
# handed *out* to the ntfy server; it is not an inbound credential — see
# ORCHESTRA_NTFY_SURFACE_TOKEN below.
#ORCHESTRA_NTFY_TOPIC=
#ORCHESTRA_NTFY_TOKEN=
#ORCHESTRA_NTFY_URL=https://ntfy.sh
@@ -72,10 +74,18 @@ ORCHESTRA_OCCUPANCY_HARD=0.75
# --- Bus authorization tokens (bearer auth per surface; a surface with no
# token set has no auth requirement — set these once you have real clients) ---
#ORCHESTRA_TUI_TOKEN=
#ORCHESTRA_WEB_TOKEN=
# Required: the service refuses to start without it. It gates the web UI's
# task, lifecycle and approval controls, and it is also the token for any
# /v1/ caller that does not declare a surface (they default to Web).
ORCHESTRA_WEB_TOKEN=
# Set when the UI is served over plain HTTP, so the session cookie can be
# sent without Secure. Leave unset behind TLS.
#ORCHESTRA_UI_INSECURE_COOKIE=1
#ORCHESTRA_MCP_TOKEN=
#ORCHESTRA_MAVEN_TOKEN=
# Telegram/ntfy tokens above double as their surface auth tokens
# (ORCHESTRA_TELEGRAM_TOKEN is the inbound bearer token if you also expose an
# endpoint they poll, separate from the bot token used to send messages).
# Inbound bearer tokens for the notify-only surfaces, separate from the
# credentials used to *send* (ORCHESTRA_TELEGRAM_BOT_TOKEN, ORCHESTRA_NTFY_TOKEN).
# S12: ORCHESTRA_NTFY_TOKEN used to serve both roles, so configuring ntfy
# delivery silently minted a valid inbound credential.
#ORCHESTRA_TELEGRAM_TOKEN=
#ORCHESTRA_NTFY_SURFACE_TOKEN=
+26
View File
@@ -117,8 +117,33 @@ func (r *Registry) Queue(worker string, c Command) (Command, error) {
c.CreatedAt = time.Now().UTC()
c.Status = "pending"
r.commands[worker] = append(r.commands[worker], c)
r.pruneCommands(worker)
return c, nil
}
// CommandRetention is how long a resolved command stays queryable so a worker
// that retries a completion, or an operator reading the UI, still sees its
// outcome. Pending commands are never pruned.
const CommandRetention = 30 * time.Minute
// pruneCommands drops resolved commands past CommandRetention. B21: this list
// was append-only, so resolved commands accumulated for the process lifetime
// and every worker poll rescanned the entire history. Callers hold r.mu.
func (r *Registry) pruneCommands(worker string) {
cutoff := time.Now().UTC().Add(-CommandRetention)
in := r.commands[worker]
out := in[:0]
for _, c := range in {
if c.Status == "pending" || c.CreatedAt.After(cutoff) {
out = append(out, c)
}
}
if len(out) == 0 {
delete(r.commands, worker)
return
}
r.commands[worker] = out
}
func (r *Registry) Commands(worker string) ([]Command, error) {
r.mu.Lock()
defer r.mu.Unlock()
@@ -126,6 +151,7 @@ func (r *Registry) Commands(worker string) ([]Command, error) {
if _, ok := r.workers[worker]; !ok {
return nil, ErrUnknownWorker
}
r.pruneCommands(worker)
var out []Command
for _, c := range r.commands[worker] {
if c.Status == "pending" {
+33
View File
@@ -105,3 +105,36 @@ func TestCaptureRevisionAndCommandQueue(t *testing.T) {
t.Fatalf("pending=%#v", commands)
}
}
func TestResolvedCommandsArePrunedButPendingOnesSurvive(t *testing.T) {
r := &Registry{}
if err := r.Register(Worker{ID: "w", Token: "t"}, ""); err != nil {
t.Fatal(err)
}
old, err := r.Queue("w", Command{TaskID: "task", Kind: "grant_approval", PaneID: "pane", CaptureRevision: 1})
if err != nil {
t.Fatal(err)
}
if err := r.CompleteCommand("w", old.ID, "acknowledged", ""); err != nil {
t.Fatal(err)
}
pending, err := r.Queue("w", Command{TaskID: "task", Kind: "deny_approval", PaneID: "pane", CaptureRevision: 2})
if err != nil {
t.Fatal(err)
}
// Age both past the retention window; only the resolved one may go.
r.mu.Lock()
for i := range r.commands["w"] {
r.commands["w"][i].CreatedAt = time.Now().UTC().Add(-2 * CommandRetention)
}
r.mu.Unlock()
if _, err := r.Commands("w"); err != nil {
t.Fatal(err)
}
if _, ok := r.Command("w", old.ID); ok {
t.Fatal("resolved command past retention was not pruned")
}
if c, ok := r.Command("w", pending.ID); !ok || c.Status != "pending" {
t.Fatalf("pending command was pruned: %#v ok=%v", c, ok)
}
}
+13 -2
View File
@@ -40,8 +40,19 @@ func TestAssignsByAffinityCapabilityAndConcurrency(t *testing.T) {
if err != nil || len(got) != 1 {
t.Fatalf("assigned %d events, err=%v", len(got), err)
}
if s.Tasks()[0].State != domain.StateLeased && s.Tasks()[1].State != domain.StateLeased {
t.Fatal("no task leased")
// Store.Tasks() ranges a map, so its order is randomized per call. This
// assertion used to index two *separate* Tasks() calls, and failed
// whenever the two orderings disagreed — the flake was in the test, not
// in assignment. Snapshot once, and assert the actual invariant:
// concurrency 1 means exactly one of the two tasks is leased.
leased := 0
for _, tk := range s.Tasks() {
if tk.State == domain.StateLeased {
leased++
}
}
if leased != 1 {
t.Fatalf("leased %d tasks, want exactly 1 (concurrency 1); got=%+v", leased, got)
}
}
+69 -26
View File
@@ -5,6 +5,7 @@ import (
"context"
"encoding/json"
"fmt"
"hash/fnv"
"net/http"
"orchestra/internal/authz"
"orchestra/internal/domain"
@@ -74,15 +75,21 @@ func (s Server) capture(ctx context.Context, t domain.Task) (*Capture, error) {
if t.State != domain.StateLeased || t.Lease == nil {
return nil, nil
}
// B20: precedence is deliberate, not incidental. A published worker
// capture means a registered worker owns that pane, and its revision is
// the counter the worker's own staleness check compares against — so it
// must win. Preferring the coordinator here (as this once did) handed
// Queue a revision the worker could never match, and every such approval
// resolved "stale" and silently never happened.
if s.Workers != nil {
if c, ok := s.Workers.Capture(t.Lease.HarnessID, t.ID); ok {
return &Capture{TaskID: t.ID, Source: "worker", Text: c.Text, Revision: c.Revision, At: c.At}, nil
}
}
if s.Coordinator != nil {
text, err := s.Coordinator.Capture(ctx, t.ID, "recent")
if err == nil {
return &Capture{TaskID: t.ID, Source: "recent", Text: text, Revision: uint64(time.Now().UnixNano()), At: time.Now().UTC()}, nil
}
}
if s.Workers != nil {
if c, ok := s.Workers.Capture(t.Lease.HarnessID, t.ID); ok {
return &Capture{TaskID: t.ID, Source: "recent", Text: c.Text, Revision: c.Revision, At: c.At}, nil
return &Capture{TaskID: t.ID, Source: "recent", Text: text, Revision: captureRevision(t.Lease.HarnessID, text), At: time.Now().UTC()}, nil
}
}
return nil, fmt.Errorf("capture unavailable from owning worker")
@@ -152,7 +159,7 @@ func (s Server) detail(ctx context.Context, id string) (TaskDetail, error) {
session := &Session{HarnessID: t.Lease.HarnessID, LeaseUntil: &t.Lease.Until}
if c, err := s.capture(ctx, t); err == nil && c != nil {
session.Capture = c
session.PaneID = capturePane(s, t.ID, c)
session.PaneID = capturePane(s, t, c)
session.Approval = ParsePendingApproval(c.Text, session.PaneID, c.Revision, c.At)
} else if err != nil {
session.Blocker = "capture unavailable: " + err.Error()
@@ -161,17 +168,35 @@ func (s Server) detail(ctx context.Context, id string) (TaskDetail, error) {
}
return d, nil
}
func capturePane(s Server, taskID string, c *Capture) string {
if s.Coordinator != nil {
if session, ok := s.Coordinator.Session(taskID); ok {
return session.PaneID
}
// captureRevision identifies *what the operator saw*, not when they saw it.
// B20: this was UnixNano, so it changed on every read and said nothing about
// whether the pane had changed. A content hash changes if and only if the
// text does, which is the only property any consumer of a revision wants.
func captureRevision(harness, text string) uint64 {
h := fnv.New64a()
_, _ = h.Write([]byte(harness))
_, _ = h.Write([]byte{0})
_, _ = h.Write([]byte(text))
// 0 means "no revision" to federation.Queue; never collide with it.
if v := h.Sum64(); v != 0 {
return v
}
if s.Workers != nil {
for _, w := range s.Workers.Snapshot() {
if remote, ok := s.Workers.Capture(w.ID, taskID); ok && remote.Revision == c.Revision {
return remote.PaneID
}
return 1
}
// capturePane resolves the pane the capture came from, following the same
// source precedence capture() used — asking the coordinator about a pane a
// worker owns (or vice versa) can return a pane the text never came from.
func capturePane(s Server, t domain.Task, c *Capture) string {
if c.Source == "worker" && s.Workers != nil && t.Lease != nil {
if remote, ok := s.Workers.Capture(t.Lease.HarnessID, t.ID); ok && remote.Revision == c.Revision {
return remote.PaneID
}
return ""
}
if s.Coordinator != nil {
if session, ok := s.Coordinator.Session(t.ID); ok {
return session.PaneID
}
}
return ""
@@ -304,22 +329,40 @@ func (s Server) action(w http.ResponseWriter, r *http.Request, id, action string
http.Error(w, "current capture unavailable", 503)
return
}
pane := capturePane(s, id, c)
pane := capturePane(s, t, c)
approval := ParsePendingApproval(c.Text, pane, c.Revision, c.At)
if approval == nil || approval.Kind == "unknown" || pane == "" {
http.Error(w, "current prompt is not safely actionable", 409)
return
}
if s.Workers != nil {
if _, remote := s.Workers.Capture(t.Lease.HarnessID, t.ID); remote {
command, err := s.Workers.Queue(t.Lease.HarnessID, federation.Command{TaskID: id, Kind: action, PaneID: pane, CaptureRevision: c.Revision})
if err != nil {
http.Error(w, err.Error(), 409)
return
}
writeJSON(w, map[string]any{"command": command, "task": t})
if c.Source == "worker" {
command, err := s.Workers.Queue(t.Lease.HarnessID, federation.Command{TaskID: id, Kind: action, PaneID: pane, CaptureRevision: c.Revision})
if err != nil {
http.Error(w, err.Error(), 409)
return
}
// B19: the federated path used to return here, leaving the most
// safety-critical operation in the system invisible in the event
// log exactly when it crossed a machine boundary. The honest
// event at this point is a *request* — the keystroke has only
// been queued. The matching ApprovalGranted/ApprovalDenied is
// appended when the worker reports the input was acknowledged
// (see /v1/federation/commands/ in cmd/orchestra).
payload, _ := json.Marshal(map[string]any{
"subject_ref": command.ID, "options": []string{"grant_approval", "deny_approval"},
"decision": action, "task_id": id, "worker": t.Lease.HarnessID,
"pane_id": pane, "capture_revision": c.Revision,
})
e := domain.Event{ID: domain.NewID(), Type: "ApprovalRequested", TaskID: id, Version: t.Version + 1, Payload: payload, Surface: string(authz.Web)}
if err := s.Store.Append(e); err != nil {
// No audit trail, no approval. Resolve the queued command so
// the worker never executes an unrecorded keystroke.
_ = s.Workers.CompleteCommand(t.Lease.HarnessID, command.ID, "rejected", "audit append failed: "+err.Error())
http.Error(w, "approval not recorded: "+err.Error(), 409)
return
}
writeJSON(w, map[string]any{"command": command, "task": t})
return
}
if s.Coordinator == nil {
http.Error(w, "approval executor unavailable", 503)
+87
View File
@@ -2,8 +2,11 @@ package ui
import (
"bytes"
"encoding/json"
"net/http"
"net/http/httptest"
"orchestra/internal/domain"
"orchestra/internal/federation"
"orchestra/internal/store"
"strings"
"testing"
@@ -43,3 +46,87 @@ func TestParsePendingApprovalDoesNotInventActionablePrompt(t *testing.T) {
t.Fatalf("got %#v", p)
}
}
// TestFederatedApprovalIsQueuedAtTheWorkerRevisionAndAudited guards B19 and
// B20 together: the queued command must carry the worker's own capture
// counter (not a coordinator timestamp the worker can never match, which
// resolved every such approval "stale"), and queueing must leave an
// ApprovalRequested event behind — the grant/deny is appended later, when
// the worker reports the keystroke was acknowledged.
func TestFederatedApprovalIsQueuedAtTheWorkerRevisionAndAudited(t *testing.T) {
s, err := store.Open(t.TempDir())
if err != nil {
t.Fatal(err)
}
workers := &federation.Registry{}
if err := workers.Register(federation.Worker{ID: "wpc-claude", Token: "tok"}, ""); err != nil {
t.Fatal(err)
}
h := Server{Store: s, Workers: workers}.Handler()
req := httptest.NewRequest(http.MethodPost, "/v1/ui/tasks", bytes.NewBufferString(`{"source":"web","external_id":"fed","project":"demo","title":"Federated"}`))
r := httptest.NewRecorder()
h.ServeHTTP(r, req)
if r.Code != http.StatusOK {
t.Fatalf("create status=%d body=%s", r.Code, r.Body.String())
}
id := s.Tasks()[0].ID
if _, err := s.Lease(id, "wpc-claude", time.Hour); err != nil {
t.Fatal(err)
}
capture, err := workers.PutCapture("wpc-claude", federation.Capture{TaskID: id, PaneID: "wA:p1", Text: "Permission required\n$ rm -rf build"})
if err != nil {
t.Fatal(err)
}
req = httptest.NewRequest(http.MethodPost, "/v1/ui/tasks/"+id+"/actions/grant_approval", bytes.NewBufferString(`{}`))
r = httptest.NewRecorder()
h.ServeHTTP(r, req)
if r.Code != http.StatusOK {
t.Fatalf("grant status=%d body=%s", r.Code, r.Body.String())
}
pending, err := workers.Commands("wpc-claude")
if err != nil || len(pending) != 1 {
t.Fatalf("commands=%#v err=%v", pending, err)
}
if pending[0].CaptureRevision != capture.Revision || pending[0].PaneID != "wA:p1" {
t.Fatalf("queued at revision %d pane %q, worker published %d wA:p1", pending[0].CaptureRevision, pending[0].PaneID, capture.Revision)
}
var requested *domain.Event
for _, e := range s.Events(0) {
if e.Type == "ApprovalRequested" && e.TaskID == id {
x := e
requested = &x
}
if e.Type == "ApprovalGranted" {
t.Fatal("queueing a command must not claim an outcome the worker has not reported")
}
}
if requested == nil {
t.Fatal("federated approval left no audit trail")
}
var p struct {
SubjectRef string `json:"subject_ref"`
Decision string `json:"decision"`
}
if err := json.Unmarshal(requested.Payload, &p); err != nil {
t.Fatal(err)
}
if p.SubjectRef != pending[0].ID || p.Decision != "grant_approval" {
t.Fatalf("audit event does not identify the command: %s", requested.Payload)
}
}
// TestCoordinatorCaptureRevisionTracksContentNotTime guards B20: the local
// revision used to be UnixNano, so it changed on every read and told a
// caller nothing about whether the pane had changed.
func TestCoordinatorCaptureRevisionTracksContentNotTime(t *testing.T) {
a := captureRevision("claude", "Permission required\n$ ls")
if a != captureRevision("claude", "Permission required\n$ ls") {
t.Fatal("revision changed without the pane text changing")
}
if a == captureRevision("claude", "Permission required\n$ rm -rf /") {
t.Fatal("revision did not change when the pane text changed")
}
if a == 0 {
t.Fatal("revision 0 means \"no revision\" to federation.Queue")
}
}