1 changed files with 167 additions and 0 deletions
@@ -0,0 +1,167 @@
+# Handoff — run 3 relaunch pending, two hard predicates open
+
+Written 2026-08-27, 11:00 UTC, mid-observation. Read with
+`HANDOFF-2026-08-27-burnin.md` (the session before this one), `BURNIN.md`
+(the ledger, now current through run 3), `AUDIT.md` and `CLAUDE.md`.
+
+Everything below was observed live unless it says otherwise.
+
+## Evidence standard, set by the operator
+
+Use this hierarchy and do not mix the tiers:
+
+```
+event log       → lifecycle truth
+worker journal  → worker-local observations and confirmer receipts
+pane capture    → harness evidence only
+code-path argument → supporting evidence, not live proof
+```
+
+This exists because this session reported "F16 passes its first branch live"
+on the strength of three `TaskLeaseRenewed` events. The predicate was inert.
+A code-path argument was presented as live proof, and the bug survived the
+report. Do not repeat that.
+
+## Deployed state
+
+| Half | Revision | Evidence |
+|---|---|---|
+| Coordinator, homesrv container | `edbe98f` | `docker logs orchestra-api`, 10:45:46 UTC |
+| Worker, workpc systemd | `edbe98f` | `journalctl -u orchestra-worker`, 14:45:26 +04 |
+
+Branch `webui-and-audit-reconciliation`, HEAD `edbe98f`. Worker sha256
+`e94ab4ddbbd755796ca68e273a55029895ec930ea88405be2f6af65be9bdc494`, staged at
+`~/orchestra-deploy/orchestra-worker.edbe98f`.
+
+Both halves match. Do not start a new run until they do again after any change.
+
+## What is being watched right now
+
+Task `06G44JZB80MZBEY97196EZN8EC`, issue `kami/test-e2e#3`, is `leased` at v15
+under **stranded epoch `06G44ZBZX4YH6PRN7Y4ZH8GG3W`**. Its pane is gone and its
+agent has been idle since 09:13. The last renewal was seq 358 at 10:32:09, by
+the old worker. The lease runs to roughly 11:02 UTC.
+
+Nobody should touch it. The expiry is the test.
+
+Expected chain, autonomous, no operator action:
+
+```
+renewal boundary reached
+→ progress observation fails, pane is gone
+→ worker does not renew
+→ lease expires
+→ TaskReleased(reason=lease_expired)
+→ queued, attempt 1
+→ router selects the worker
+→ new lease, new epoch, new pane
+→ confirmed input submission
+→ TaskLaunchAcknowledged
+```
+
+### Two hard failure predicates
+
+```
+old epoch 06G44ZBZX4YH6PRN7Y4ZH8GG3W renewed after 10:45:26  → F16 fail
+first orchestra-originated post-frame input has no confirmer receipt → F20 fail
+```
+
+The F20 receipt is strictly this line in the worker journal:
+
+```
+input to <pane> confirmed: confirmation=<kind> submit_attempts=<n> ...
+```
+
+Pane movement, text leaving the editor, and later model output are all
+insufficient.
+
+### Checks to capture, in order
+
+1. No `TaskLeaseRenewed` carrying the stranded epoch after 10:45:26.
+2. Release reason is literally `lease_expired`, not a restart or reconcile
+   fallback.
+3. New lease epoch differs from `06G44ZBZX4YH6PRN7Y4ZH8GG3W`.
+4. `ProgressSHA`, `UsageBaseline` and `PickupAcknowledged` start fresh for the
+   new epoch. State this precisely: the reset is **structurally established by
+   the reconciler**, and the first successful renewal under the new epoch shows
+   the reconstructed lease stayed usable. Those fields are not exposed, so do
+   not claim their runtime values were observed.
+5. Launch confirmation evidence carries `confirmation=` and `submit_attempts=`.
+6. No operator action between expiry and relaunch.
+7. Then the F20 boundary: the first Orchestra-originated input after `frame`
+   completes.
+
+If the task reaches real implementation after that with no manual
+intervention, this stops being a diagnostic run.
+
+## Fixed since the last handoff
+
+Each landed in the revision named. `BURNIN.md` carries the full entries.
+
+- **F16**, lifecycle, `1888d42` then `770cc6a` then `edbe98f`. Renewal requires
+  observable progress. It shipped **inert**: `reconcileLeases` rebuilt every
+  lease from the coordinator's task list each tick and dropped `ProgressSHA`,
+  so the check took its no-baseline branch forever. `UsageBaseline` and
+  `PickupAcknowledged` were wiped the same way. Progress is now measured with
+  input lines removed, so keystrokes are not work.
+- **F17**, adapter, `f54fb00`. `pendingInput` scanned every line with the
+  prompt marker, but queued and accepted input renders identically. Only the
+  editor owning the cursor is unsubmitted. `ConfirmLaunch` became an active
+  submit protocol: resend Enter while the live editor holds exactly what was
+  submitted, at most 3 times, then observe. Verified live:
+  `submit_attempts=2` in production against 1 in six isolated probes.
+- **F19**, lifecycle, `edff021`. `POST /v1/tasks/{id}/retry`, full-control
+  surfaces only, for a failed and unleased task whose failure reason is
+  `retry_limit`. Requires `operation_id` and is idempotent. Verified live: the
+  task leased itself one second after the correction.
+- **F20**, adapter, `edbe98f`. Every Orchestra-originated pane write confirms.
+  `LaunchConfirmer` is `InputConfirmer`, `ConfirmLaunch` is `ConfirmInput`, and
+  `sendPrompt` and `sendLine` both route through it. Tests only so far.
+- **F12**, **F13**, **F14** landed earlier and are described in `BURNIN.md`.
+  F12 and F13 were verified live, F14 on run 3's ingest.
+
+## Open
+
+- **F20 live**, unproven. This is the next boundary.
+- **F18**, observability. Nothing besides `inputState` distinguishes the active
+  editor from history when parsing pane text, so the same class of error can
+  recur wherever `PaneCapture` output is matched.
+- **F9**, unchanged. An operator cannot release a lease someone else owns
+  without supplying that owner's `harness_id` and `lease_epoch`, read out of
+  the event log first. F19 did not address this.
+- **Approval input is unconfirmed on purpose.** `main.go:1078` writes `y`, `n`
+  and `ENTER` to a y/n dialog, not an editor, so the editor predicate would
+  report nonsense. It needs its own predicate if it is ever covered.
+- **F6**, observability. Worker health reported `active_task: null` against a
+  leased task. Recheck now that launches confirm.
+- **F4**, adapter quality. `harness "opencode" has no session-file resolver`.
+- **F10**, hygiene. A coordinator-side release under a live worker leaves the
+  worker renewing a lease it no longer holds.
+- Unset gated tokens: `ORCHESTRA_MCP_TOKEN`, `ORCHESTRA_MAVEN_TOKEN`,
+  `ORCHESTRA_AGENT_TOKEN`.
+- `gofmt -l` fails on `internal/provider/provider.go`,
+  `internal/router/router.go` and `internal/webui/webui.go`. Untouched by these
+  runs, and `go vet` passes, which is why nobody noticed.
+
+## Things that will bite
+
+- **Another session still owns 18 uncommitted paths**, including `AUDIT.md`.
+  Commit by path. Never `git add -A`. The burn-in ledger lives in `BURNIN.md`
+  for exactly this reason.
+- **An unexplained line was typed into the run-3 pane.** The operator confirms
+  it was not them, and the worker journal shows no send. Treat unexplained pane
+  input as possible, which is why progress must ignore input entirely.
+- **Do not use `docker compose build`.** Its context is the shared live
+  checkout. Build from a detached worktree, as `deploy/build.sh` does and as
+  the coordinator image command in `HANDOFF-2026-08-26-burnin.md` shows.
+- **Installing the worker needs root**, so it is always the operator's step.
+  Mutations through the API need the TUI token from the compose `.env`, which
+  `secret-guard` blocks from being read. Hand the operator the command.
+- **This machine is workpc**, `hostname` is `bugmachine`. homesrv is
+  `kami@192.168.1.104` with `/usr/bin/ssh`, not the `ssh` on PATH.
+- **`/tmp` is tmpfs with a 10-day sweep.** Check `/tmp/test-e2e` and
+  `/tmp/test-e2e-worktrees` exist before every run.
+- **The MCP surface is read-only and needs no credential.**
+  `curl -H 'X-Orchestra-Surface: mcp' http://192.168.1.104:9145/v1/tasks`.
+- **Three queued `correx` tasks will never lease.** No worker declares that
+  project. Router health says so.