136 lines
8.0 KiB
Markdown
136 lines
8.0 KiB
Markdown
# Handoff — burn-in, run 1 closed, run 2 not started
|
|
|
|
Written 2026-08-26, late. Read with `BURNIN.md` (the runbook and the findings
|
|
ledger), `AUDIT.md`, and `CLAUDE.md`. This session stopped adding workflow
|
|
features and started running real tasks. Everything below was observed live, not
|
|
inferred.
|
|
|
|
## Deployed state
|
|
|
|
| Half | Revision | Evidence |
|
|
|---|---|---|
|
|
| Coordinator, homesrv container | `77a2b32` | `docker logs orchestra-api` prints it at startup, `/readyz` ready |
|
|
| Worker, workpc systemd | `77a2b32` | `journalctl -u orchestra-worker`, both harnesses serving |
|
|
|
|
Burn-in identity is `77a2b32`. Both halves report it, which was the precondition
|
|
for a conformance run. `deploy/build.sh [outdir] [rev]` builds both from a
|
|
detached worktree of that revision, so a shared checkout cannot leak
|
|
uncommitted work into a stamped binary.
|
|
|
|
Deploying the coordinator:
|
|
|
|
```sh
|
|
ssh kami@192.168.1.104
|
|
repo=/home/kami/apps/orchestra; rev=$(git -C $repo rev-parse HEAD)
|
|
bt=$(git -C $repo show -s --format=%cI $rev); tree=/tmp/orchestra-build-$rev
|
|
git -C $repo worktree add --detach --quiet $tree $rev
|
|
docker build --build-arg BUILD_REVISION=$rev --build-arg BUILD_TIME=$bt \
|
|
--build-arg BUILD_DIRTY=false -f $tree/Dockerfile.api \
|
|
-t orchestra-web-ui-orchestra-api $tree
|
|
git -C $repo worktree remove --force $tree
|
|
cd /home/kami/docker-apps/orchestra-web-ui
|
|
docker compose up -d --no-deps --force-recreate orchestra-api
|
|
```
|
|
|
|
Do not use `docker compose build`: its context is the live checkout, which is
|
|
shared. Installing the worker needs root, so it is always the operator's step.
|
|
|
|
## Another session owns 18 uncommitted paths
|
|
|
|
An auth and frontend session worked in this same checkout and has finished, but
|
|
its work is **uncommitted**: `internal/authn/*`, `web/*`,
|
|
`internal/webui/assets/*`, `AGENTS.md`, `deploy/DEPLOYMENT.md`,
|
|
`deploy/orchestra.env.example`, `AUDIT.md`, and an extension to
|
|
`deploy/build.sh` that also builds `cmd/orchestra-user`. Do not commit those on
|
|
its behalf, and do not use `git add -A`. Commit by path.
|
|
|
|
The deployed coordinator does not contain that work. Deploying it means
|
|
committing it first.
|
|
|
|
One thing to check: this session's first commit, `7f12c7f`, predates that
|
|
discipline and used `git add -A`. It swept in whatever was uncommitted then,
|
|
including `web/src` and `internal/authn`.
|
|
|
|
## Where the burn-in stands
|
|
|
|
**Run 1 is closed as a diagnostic, not a conformance run.** Task
|
|
`06G3YR34117MAYT6KEAC9RJHD0` from issue `kami/test-e2e#1` needed three manual
|
|
leases, and its authority was rendered before the issue-body fix. It was
|
|
released with reason `abandoned diagnostic run` and sits at `attempt: 3`, so the
|
|
router will fail it.
|
|
|
|
What it proved anyway: `.orchestra/launch.md` renders authority order, the
|
|
ambiguity ladder, the phase brief, the standing decision set and a verified git
|
|
state; an opencode agent respected `frame` and changed no code; and a bad base
|
|
checkout nacked cleanly back to `queued` with no orphan pane.
|
|
|
|
**Run 2 has not started.** Every precondition is met except creating the task.
|
|
|
|
## Do this first
|
|
|
|
1. Create one ordinary issue in `kami/test-e2e`, with a real body. The body is
|
|
the goal and acceptance; the title alone is not enough.
|
|
2. **Do not lease it by hand.** Autonomous assignment is the thing under test.
|
|
3. Watch, in this order:
|
|
- `GET /v1/router/health` — if the task is not assigned within a minute, this
|
|
names the gate that refused it. That is the finding; stop there.
|
|
- `journalctl -u orchestra-worker -f` — launch, nack or error.
|
|
- `cat /tmp/test-e2e-worktrees/<task-id>/.orchestra/launch.md` — the highest
|
|
value artifact. Ask only whether the agent knows what the task wants, what
|
|
was most recently decided, which phase it is in, what is merely history,
|
|
and what to do next.
|
|
4. Compare that against what the model does. Classify before changing anything:
|
|
authority, context-selection, lifecycle, adapter/harness, model-following, or
|
|
operator-policy. A prompt change is only ever the answer to a
|
|
model-following failure.
|
|
|
|
Success criteria for run 2, in order: issue ingested, router assigns
|
|
`workpc-opencode` on its own, `Store.Lease`, pre-lease reconcile against
|
|
`gitea:test-e2e` only, agent launches, `launch.md` carries a body-derived goal
|
|
and acceptance, `frame` behaves, and the phase advances with no manual
|
|
lifecycle intervention.
|
|
|
|
## Findings ledger
|
|
|
|
Fixed this session, all verified by test and most seen live:
|
|
|
|
- **F1**, authority: a federated worker could not read `GET /v1/tasks/<id>/intent`, so every federated launch died with a 401. `2753a8d`.
|
|
- **F2**, authority: `Gitea.event` dropped the issue body, so every Gitea task ran on its title with `Acceptance: Not stated.` `09e572f`.
|
|
- **F3**, hygiene: the launch dump left `.orchestra/` untracked in a worktree whose own context claimed a clean tree. `a0209a2`.
|
|
- **F5**, lifecycle: every router eligibility gate now records a `router.Rejection` at `GET /v1/router/health`, including tasks skipped for retry backoff. No gate was weakened. `0ead6d2`, `4fbf3ac`.
|
|
- **F7**, security: an unset token means no check, so a full-control surface without a credential was an open control plane. Startup now refuses. `ORCHESTRA_TUI_TOKEN` is set. `0ead6d2`.
|
|
- **F8**, correctness: reconciliation is bound to `task.Source`, so a comment on one repository can no longer become an authoritative decision for a same-numbered task in another. `0ead6d2`.
|
|
- **F11**, outage: `ValidateEvent` compared `until_ns` against `time.Now()`, so an expired lease event failed validation and `store.Open` refused the log tail. The API was in a restart loop. Validation is time-independent now. `77a2b32`.
|
|
|
|
Open, in the order this session would take them:
|
|
|
|
- **F6**, observability: worker health reports `active_task: null` while the store shows the task leased and herdr shows a live agent in its worktree.
|
|
- **F4**, adapter quality: `harness "opencode" has no session-file resolver`, so activity and thrash triggers are permanently degraded on opencode. Acceptable for a boring-success baseline; milestone and occupancy paths still work.
|
|
- **F9**, lifecycle: releasing a leased task through `POST /v1/tasks/<id>/release` requires supplying `harness_id` and `lease_epoch`, because `Store.Append` fences the event and the endpoint passes the body through. There is no operator escape hatch.
|
|
- **F10**, hygiene: a coordinator-side release under a live worker leaves the worker renewing a lease it no longer holds.
|
|
- **Unset gated tokens**: `ORCHESTRA_MCP_TOKEN`, `ORCHESTRA_MAVEN_TOKEN` and `ORCHESTRA_AGENT_TOKEN` are all empty, so those surfaces need no credential. Bounded by capability to reads plus the three request endpoints, which is why startup does not refuse. Worth closing.
|
|
|
|
## Things that will bite
|
|
|
|
- **`snapshot.json` lags the event log.** It showed 30 tasks while `/v1/tasks`
|
|
showed 34. Read the API or `events.jsonl`, never the snapshot, when checking
|
|
live state.
|
|
- **The MCP surface is a read-only diagnostic today.** `curl -H
|
|
'X-Orchestra-Surface: mcp' .../v1/tasks` works with no credential, which is
|
|
how this session read live state without the operator login. Mutations need
|
|
the TUI token from `.env` on homesrv.
|
|
- **`/tmp` is tmpfs.** `/tmp/test-e2e` (a **non-bare** clone, origin
|
|
`ssh://git@gitea.kvmx.ru:2222`) and `/tmp/test-e2e-worktrees` do not survive a
|
|
reboot, and `/tmp` has a 10-day sweep. A bare clone fails with `fast-forward
|
|
base checkout: fatal: this operation must be run in a work tree`.
|
|
- **The three queued `correx` tasks will never lease.** No worker declares that
|
|
project. `/v1/router/health` says so plainly now.
|
|
- **23 blocked test-e2e tasks are July leftovers.** They have been inert across
|
|
hours of `ResumeAnsweredBlockers`, so they are not a hazard, only noise.
|
|
- **A pane can push.** Panes run as `kami`, and pushing uses kami's registered
|
|
SSH identity over port 2222. Isolation needs a separate unix user, not an
|
|
environment scrub.
|
|
- **Ingest is now `kami/test-e2e`, not `correx`.** `ORCHESTRA_GITEA_REPO` was
|
|
switched for the burn-in; the previous `.env` is kept as
|
|
`.env.pre-burnin-20260826`. The project id must equal the repo name.
|