diff --git a/HANDOFF-2026-08-26-burnin.md b/HANDOFF-2026-08-26-burnin.md new file mode 100644 index 0000000..8a334a9 --- /dev/null +++ b/HANDOFF-2026-08-26-burnin.md @@ -0,0 +1,135 @@ +# Handoff — burn-in, run 1 closed, run 2 not started + +Written 2026-08-26, late. Read with `BURNIN.md` (the runbook and the findings +ledger), `AUDIT.md`, and `CLAUDE.md`. This session stopped adding workflow +features and started running real tasks. Everything below was observed live, not +inferred. + +## Deployed state + +| Half | Revision | Evidence | +|---|---|---| +| Coordinator, homesrv container | `77a2b32` | `docker logs orchestra-api` prints it at startup, `/readyz` ready | +| Worker, workpc systemd | `77a2b32` | `journalctl -u orchestra-worker`, both harnesses serving | + +Burn-in identity is `77a2b32`. Both halves report it, which was the precondition +for a conformance run. `deploy/build.sh [outdir] [rev]` builds both from a +detached worktree of that revision, so a shared checkout cannot leak +uncommitted work into a stamped binary. + +Deploying the coordinator: + +```sh +ssh kami@192.168.1.104 +repo=/home/kami/apps/orchestra; rev=$(git -C $repo rev-parse HEAD) +bt=$(git -C $repo show -s --format=%cI $rev); tree=/tmp/orchestra-build-$rev +git -C $repo worktree add --detach --quiet $tree $rev +docker build --build-arg BUILD_REVISION=$rev --build-arg BUILD_TIME=$bt \ + --build-arg BUILD_DIRTY=false -f $tree/Dockerfile.api \ + -t orchestra-web-ui-orchestra-api $tree +git -C $repo worktree remove --force $tree +cd /home/kami/docker-apps/orchestra-web-ui +docker compose up -d --no-deps --force-recreate orchestra-api +``` + +Do not use `docker compose build`: its context is the live checkout, which is +shared. Installing the worker needs root, so it is always the operator's step. + +## Another session owns 18 uncommitted paths + +An auth and frontend session worked in this same checkout and has finished, but +its work is **uncommitted**: `internal/authn/*`, `web/*`, +`internal/webui/assets/*`, `AGENTS.md`, `deploy/DEPLOYMENT.md`, +`deploy/orchestra.env.example`, `AUDIT.md`, and an extension to +`deploy/build.sh` that also builds `cmd/orchestra-user`. Do not commit those on +its behalf, and do not use `git add -A`. Commit by path. + +The deployed coordinator does not contain that work. Deploying it means +committing it first. + +One thing to check: this session's first commit, `7f12c7f`, predates that +discipline and used `git add -A`. It swept in whatever was uncommitted then, +including `web/src` and `internal/authn`. + +## Where the burn-in stands + +**Run 1 is closed as a diagnostic, not a conformance run.** Task +`06G3YR34117MAYT6KEAC9RJHD0` from issue `kami/test-e2e#1` needed three manual +leases, and its authority was rendered before the issue-body fix. It was +released with reason `abandoned diagnostic run` and sits at `attempt: 3`, so the +router will fail it. + +What it proved anyway: `.orchestra/launch.md` renders authority order, the +ambiguity ladder, the phase brief, the standing decision set and a verified git +state; an opencode agent respected `frame` and changed no code; and a bad base +checkout nacked cleanly back to `queued` with no orphan pane. + +**Run 2 has not started.** Every precondition is met except creating the task. + +## Do this first + +1. Create one ordinary issue in `kami/test-e2e`, with a real body. The body is + the goal and acceptance; the title alone is not enough. +2. **Do not lease it by hand.** Autonomous assignment is the thing under test. +3. Watch, in this order: + - `GET /v1/router/health` — if the task is not assigned within a minute, this + names the gate that refused it. That is the finding; stop there. + - `journalctl -u orchestra-worker -f` — launch, nack or error. + - `cat /tmp/test-e2e-worktrees//.orchestra/launch.md` — the highest + value artifact. Ask only whether the agent knows what the task wants, what + was most recently decided, which phase it is in, what is merely history, + and what to do next. +4. Compare that against what the model does. Classify before changing anything: + authority, context-selection, lifecycle, adapter/harness, model-following, or + operator-policy. A prompt change is only ever the answer to a + model-following failure. + +Success criteria for run 2, in order: issue ingested, router assigns +`workpc-opencode` on its own, `Store.Lease`, pre-lease reconcile against +`gitea:test-e2e` only, agent launches, `launch.md` carries a body-derived goal +and acceptance, `frame` behaves, and the phase advances with no manual +lifecycle intervention. + +## Findings ledger + +Fixed this session, all verified by test and most seen live: + +- **F1**, authority: a federated worker could not read `GET /v1/tasks//intent`, so every federated launch died with a 401. `2753a8d`. +- **F2**, authority: `Gitea.event` dropped the issue body, so every Gitea task ran on its title with `Acceptance: Not stated.` `09e572f`. +- **F3**, hygiene: the launch dump left `.orchestra/` untracked in a worktree whose own context claimed a clean tree. `a0209a2`. +- **F5**, lifecycle: every router eligibility gate now records a `router.Rejection` at `GET /v1/router/health`, including tasks skipped for retry backoff. No gate was weakened. `0ead6d2`, `4fbf3ac`. +- **F7**, security: an unset token means no check, so a full-control surface without a credential was an open control plane. Startup now refuses. `ORCHESTRA_TUI_TOKEN` is set. `0ead6d2`. +- **F8**, correctness: reconciliation is bound to `task.Source`, so a comment on one repository can no longer become an authoritative decision for a same-numbered task in another. `0ead6d2`. +- **F11**, outage: `ValidateEvent` compared `until_ns` against `time.Now()`, so an expired lease event failed validation and `store.Open` refused the log tail. The API was in a restart loop. Validation is time-independent now. `77a2b32`. + +Open, in the order this session would take them: + +- **F6**, observability: worker health reports `active_task: null` while the store shows the task leased and herdr shows a live agent in its worktree. +- **F4**, adapter quality: `harness "opencode" has no session-file resolver`, so activity and thrash triggers are permanently degraded on opencode. Acceptable for a boring-success baseline; milestone and occupancy paths still work. +- **F9**, lifecycle: releasing a leased task through `POST /v1/tasks//release` requires supplying `harness_id` and `lease_epoch`, because `Store.Append` fences the event and the endpoint passes the body through. There is no operator escape hatch. +- **F10**, hygiene: a coordinator-side release under a live worker leaves the worker renewing a lease it no longer holds. +- **Unset gated tokens**: `ORCHESTRA_MCP_TOKEN`, `ORCHESTRA_MAVEN_TOKEN` and `ORCHESTRA_AGENT_TOKEN` are all empty, so those surfaces need no credential. Bounded by capability to reads plus the three request endpoints, which is why startup does not refuse. Worth closing. + +## Things that will bite + +- **`snapshot.json` lags the event log.** It showed 30 tasks while `/v1/tasks` + showed 34. Read the API or `events.jsonl`, never the snapshot, when checking + live state. +- **The MCP surface is a read-only diagnostic today.** `curl -H + 'X-Orchestra-Surface: mcp' .../v1/tasks` works with no credential, which is + how this session read live state without the operator login. Mutations need + the TUI token from `.env` on homesrv. +- **`/tmp` is tmpfs.** `/tmp/test-e2e` (a **non-bare** clone, origin + `ssh://git@gitea.kvmx.ru:2222`) and `/tmp/test-e2e-worktrees` do not survive a + reboot, and `/tmp` has a 10-day sweep. A bare clone fails with `fast-forward + base checkout: fatal: this operation must be run in a work tree`. +- **The three queued `correx` tasks will never lease.** No worker declares that + project. `/v1/router/health` says so plainly now. +- **23 blocked test-e2e tasks are July leftovers.** They have been inert across + hours of `ResumeAnsweredBlockers`, so they are not a hazard, only noise. +- **A pane can push.** Panes run as `kami`, and pushing uses kami's registered + SSH identity over port 2222. Isolation needs a separate unix user, not an + environment scrub. +- **Ingest is now `kami/test-e2e`, not `correx`.** `ORCHESTRA_GITEA_REPO` was + switched for the burn-in; the previous `.env` is kept as + `.env.pre-burnin-20260826`. The project id must equal the repo name.