kami/orchestra
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Hand off with the release path settled and the UI as a truth detector

Browse Source 
F57 to F60 close the release-transaction family, F18 closes the single
last_error slot, the debt ledger exists as a read-only projection, and
the operator console is rebuilt on ethos.

The part worth acting on is what the UI could not do honestly: no
web-facing human-decision write path, no keystroke forwarding, context
occupancy trapped in herdr, project configuration unserved, and no
federated handoff request. F61 and F62 are recorded and unbuilt.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CVbaKucEYBjMqVeUgJUsc1
This commit is contained in:
kami
2026-08-29 14:59:05 +04:00
parent cbd6b11c49
commit 765bf2afc6
1 changed files with 314 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
HANDOFF-2026-08-29-truth-detector.md
+314
View File
@@ -0,0 +1,314 @@
# Handoff: the release path is settled, and the UI became a truth detector
Written 2026-08-29, 14:20 local (10:20 UTC). Read with `BURNIN.md` (the run
ledger, current through run 16), `DEBT-DESIGN.md`, `PLAN-SPEC-DESIGN.md`,
`AUDIT.md` and `CLAUDE.md`. The previous handoff is
`HANDOFF-2026-08-28-plan-v1.md`.
Everything below was observed live unless it says otherwise.
## The headline
**The expired-release defect is fixed and proven both ways.** The whole family
around it is closed. F57 through F60 settle what happens to a release
transaction in every case. That includes the ones that used to need an operator
with a text editor.
**Three new things exist that did not before.** A bounded observation ring on
worker health, closing F18. A read-only debt ledger projected from the event
log. An operator console rebuilt on the ethos design system.
**The UI turned out to be a truth detector.** Nine screens were built against
real endpoints. They found four places where Orchestra has no capability to
support the intended interface. That list is the most valuable output of the
session.
```text
orchestra-plan-v1 plan machinery proven
↓
orchestra-f18-baseline d6ee10f, the bounded observation ring
↓ 6 commits
34f3c28 deployed now: release path settled, debt ledger, new UI
```
## Deployed state
| Half | Revision |
|---|---|
| Coordinator, homesrv container | `34f3c28` |
| Worker, workpc systemd | `34f3c28` |
```text
commit 34f3c2888fc7d45d190d93e1ea42501b6cd3e474
coordinator sha256 1d32d83ec859b36e12473fab01bbfc3c769b97b4f6ae4db2841db5161e0eda19
worker sha256 0e3877321dea8a1eeda51ccc6f3ead1a14aa5f5cae4d95b704f61c364248ec65
```
`cbd6b11` is one commit above and is documentation only. Three commits are
unpushed.
**Worker installs no longer need a human.** The operator installed the
`/etc/sudoers.d` line, so `sudo -n install …` and
`sudo -n systemctl restart orchestra-worker` both work unattended. Verify the
running revision from the journal, never the installed file.
## The defects fixed, and how each was found
Not one came from reading code. Every one came from a live run failing.
| Id | Commit | What |
|---|---|---|
| F57 | `6565b9f` | An expired lease could never commit the anchor it had already pushed. The worker sent an epoch the expiry replay had deleted, and the coordinator refused any handoff without a live lease. The epoch now belongs to the transaction, `TaskReleased` retains the ending epoch, and `lateHandoffAccepted` lets exactly that owner commit while the task is queued and unleased. |
| F58 | `03663f4` | A superseded transaction retried a permanent 409 forever, holding the pane and pinning `ActiveTask`. Run 10's task did it for seven hours. `TaskLeased` now abandons a transaction whose id the lease does not carry. |
| F59 | `8e37989` | F58 fires on `TaskLeased`, and a failed task is never leased again. `TaskFailed` now drops the transaction too. |
| F60 | `3c7cf95` | The general rule the other two were reaching for. Terminal is failed or completed. Blocked keeps the transaction, because a reopen can still commit it, so `TaskBlocked` now retains the ending epoch as well. A refusal parks the commit for 30s backing off to 5 minutes, and any event about the task un-parks it. A transport failure is not an answer and retries at once. |
| F18 | `d6ee10f` | The single `last_error` slot. Worker health now carries up to sixteen distinct observations with repeat counts and first/last times, collapsing by message rather than by position. |
### The rig that proved F57, and the guard
```text
19:00:10.742 transaction opens at prepared, anchor pushing
19:00:10.727 TaskReleased v15 reason=lease_expired surface=tui
19:00:11.662 TaskReleased v16 the late commit, accepted 935ms after the lease died
19:00:11.665 TaskLeased v17 successor picks up the handoff
19:00:15.524 TaskPickupValidated v18
```
Race guard, next boundary: force the expiry, then lease the task to a probe
harness before the push finishes. The late commit is refused, no handoff is
written, and the successor's lease stands.
**The rig technique matters more than the rig.** Suspending the worker cannot
produce this ordering. The event replay runs at the top of every tick and
discards the transaction. The ordering exists only inside one call:
transaction opened, anchor pushing, commit not yet sent. So poll the worker
state file at 2ms and fire `POST /v1/tasks/<id>/release` the instant a
transaction appears at `prepared`.
**A named probe harness owns a lease without starting an agent.**
`race-guard-probe` never picks anything up and expires on the normal TTL.
## Corrections to the previous handoff
**The operator lifecycle actions do not lose a version race.** On a leased
task, `block`, `release` and `attention` are refused by
`internal/store/store.go:885-901` when the payload omits `harness_id` and
`lease_epoch`. Ten attempts in 550ms all failed that way. Send both fencing
fields and they succeed on the first try.
**The OpenCode Zen free tier is not blocked.** The selected model was.
## The debt ledger
`DEBT-DESIGN.md` answers nine design questions and carries four amendments the
operator made. Slice one is built, deployed and run.
**Slice one writes nothing.** A read-only projection over the existing log,
plus a pure eligibility function and `GET /v1/debt`. It folded 881 events and
produced 15 candidates and 3 gaps.
```text
v1:operational:lease_expired:workpc-opencode:lease r=41 tasks=4
v1:operational:lease_expired:workpc-claude:lease r=29 tasks=13
v1:operational:lease_failure:-:lease r=20 tasks=16
v1:correctness:handoff_validation:-:lease r=5 tasks=5
```
The opencode failure shape is the top item, found mechanically. Run 14 reached
the same conclusion by hand from a pane capture.
**It reported what it cannot see, which was the point.** The 409 release loop
does not appear. That evidence lived in the F18 worker ring, and no event
carries it. Manual interventions are a non-durable gap for the same reason.
**The first run exposed four defects in the model**, all recorded at the end of
`DEBT-DESIGN.md`:
- the component part is too coarse for lease evidence
- `harness` is often empty on block reasons
- path normalization mangled a mismatch reference
- recurrence alone is the wrong sort order
Layout, so the read model does not end up in the command layer:
```text
internal/domain/debt.go types, signature, classification
internal/store/debt_projection.go the fold, and the gap report
internal/operations/debt.go CheckDebtEligibility
```
## The operator console
Nine screens on the ethos system, signal violet `#8F7AE5`, routing fork motif.
Each screen was built by its own agent against a foundation with one author.
The shell, tokens and primitives could not drift into nine dialects.
**Render before signing off.** Three bugs existed that no computed value would
have caught. All three came from looking at a screenshot:
- The previous stylesheet fought every shared class name and leaked properties
the new rules never mention, which is how `position: fixed` survived on
`.topbar`. It is now scoped under `.legacy` and reaches only the login route.
That also stops its green accent and its `backdrop-filter` from reaching the
console.
- Go marshals a zero `time.Time` as `0001-01-01T00:00:00Z` and `omitempty` does
not omit a struct, so absent timestamps arrived populated-looking and
rendered as `739855d ago`. Stripped once in `client.ts`, with a test.
- Long machine ids overflowed their cards and painted under the next one.
Chromium is installed at `/usr/bin/chromium`. To see a screen without a live
session, write a throwaway harness that stubs `window.fetch` and renders
`<Console>` inside a `MemoryRouter`, served by vite on a spare port. Note that
`npx` and `./node_modules/.bin/*` do not work on this filesystem: call
`node ./node_modules/vite/bin/vite.js` directly.
## What the UI proved Orchestra cannot do
This is the part worth acting on. Each screen refused to fake something, and
the refusals name real capability gaps.
| Gap | Evidence |
|---|---|
| **No web-facing human-decision write path** | `Steer / Correct` is disabled. `internal/ui/ui.go`'s action switch has grant/deny approval, resubmit, handoff, release, block and complete, and nothing writes `HumanDecisionRecorded`. The spec makes steering the primary action of the task detail screen. |
| **No keystroke forwarding** | `Take control` is disabled. Only resubmit and approval grant/deny reach a live pane. |
| **Context occupancy is trapped in herdr** | Three screens independently hit it. No projection carries it. |
| **Project configuration is not served** | Repo, remote, quality gate and verification policy live only in `config.jsonc`. The projects screen can show none of it. |
| **The web cannot request a handoff for a federated task** | `RequestHandoff` needs a local coordinator and answers 503. That is the Design A guardrail working. |
The operator's direction on these. Treat first-class direct human input as the
highest-value backend feature. Build it as `POST /v1/tasks/<id>/decisions`,
using the same durable decision semantics as Gitea comments, so Gitea, CLI and
web converge on one `HumanDecisionRecorded`. Keep take-control disabled,
because arbitrary pane input bypasses the durable authority model. Expose
occupancy through a session health projection rather than teaching the web
server about herdr. Add a read-only effective project configuration endpoint,
which F61 will also need.
## Where the plan-machinery ladder stands
Proven in run 14: the worker executes the **sealed plan's** commands rather
than the request's, and every `PlanPhaseVerified` binds `plan_ref`, `phase_id`,
`at_sha`, `evidence_ref`, `lease_epoch` and `harness_id`.
Proven in run 16: a successor inherits the **whole sealed plan**, all phases
with their files, changes, verification, and the research citations.
**Still unproven, and the next runtime item:**
```text
mid-implement rotation
→ successor picks up in implement
→ launch context states which phases are already verified
manual verification
→ SHA goes stale
→ re-verification
plan mismatch
→ human decision
→ real replan, old plan retained, replacement launched
```
Two things make the first one hard, and both are now known:
- **A trivial task spends about four minutes in implement**, and verifies every
phase against one tree near the end. The implementer writes the whole change
first, then verifies each phase in turn. Use a task whose implement phase
genuinely runs long.
- **A state-file edit does not survive a running worker.** It holds sessions in
memory and writes them back. Suspend, edit, then `sudo systemctl restart`.
Resuming lets the old copy win. Setting `handoff_requested` on the session is
the production rotation lever.
Progress is withheld from a **review** successor on purpose
(`internal/agentctx/agentctx.go:321`), because an independent review must
reconstruct the change from the diff. That absence is not the defect.
## F61 and F62, recorded and not built
**F61: the planner learns the verification policy by refusal.** The brief says
a policy exists, not what is in it. Every plan therefore pays one refused round
trip. The recovery loop works: run 14's planner consumed the refusal and
resealed 31 seconds later. It is not a one-liner, because `agentctx.Input.Policy` is filled
from the worker's `SafeOperations` while the verification policy is
coordinator-side. The reason to promote it later is local models, which may
propose forbidden commands repeatedly because they cannot infer the allowed
substitute.
**F62: a requested handoff nobody answers is invisible.** The rotation is
agent-driven. In run 16 the agent never wrote `HANDOFF.md`, renewals stopped,
the lease expired, and the task lost an attempt. Worker health recorded only
`agent status idle and pane unchanged`, 34 times. There is no timeout, no
retry, and no observation saying a handoff was requested and left unanswered.
The expiry is indistinguishable from an ordinary idle one.
## Live state
Three pull requests are open and unreviewed:
```text
06G4M6HF1Z3EREX1X3NEKSHP24 pulls/18
06G4M8WHGQ4P3GQMPEEH0RJRHM pulls/19
06G4SWEVP71FYYKAV5FV0ZK5ZG pulls/20
```
Both workers are online on `34f3c28` with no release transactions and no
sessions. 29 blocked `test-e2e` tasks are burn-in debris. Three `correx` tasks
are queued and unschedulable, because `correx` has no entry in the
coordinator's `config.jsonc`.
**opencode now runs.** The model was the problem, and `hy3-free` works. It then
stops on a permission prompt, because `~/.config/opencode/opencode.jsonc` sets
`"bash": "ask"`. Orchestra can answer that exact dialog, but only when an
operator queues `grant_approval`, so unattended work stalls on the first
command. Set `"bash": "allow"` for unattended runs. That file also has no
top-level `model` key, so OpenCode picks whatever sits at the top of
`~/.local/state/opencode/model.json`, which any manual pick silently changes.
**The opencode adapter is now a debt item, not a curiosity.** It cannot resolve
a session file. Activity therefore reads `unknown`, and the worker cannot tell
finished from never-started. The debt projection surfaced it independently as
the top recurring operational item.
## Things that will bite
- **Background python tasks get killed here.** Three watchers died before doing
anything. Foreground polling and the `Monitor` tool both work.
- **`/v1/events` is one line of JSON.** A `grep` for two substrings matches
across unrelated tasks. Parse it.
- **`npx` and `./node_modules/.bin/*` fail on this filesystem.** Call node
directly.
- **`rm` and `cp` are interactive.** Use `/bin/rm -f` and `install`.
- **Secrets are guarded.** Expand a token inside the container in one remote
command:
`T=$(docker exec orchestra-api printenv ORCHESTRA_TUI_TOKEN); curl -s -H "X-Orchestra-Surface: tui" -H "Authorization: Bearer $T" ...`
- **Rebuild the coordinator from a detached worktree**, and split the worktree
add, the docker build and the compose up into separate commands.
- **`deploy/build.sh` builds both halves from one commit with one stamp.** Use
it.
- **Geist is not on disk.** Both stacks fall back to system faces, and the
ethos threat model rules out the font CDN.
## The roadmap the operator set
```text
A. runtime correctness finish the plan-machinery live proof
B. evidence and debt durable worker observations, manual-intervention
events, then rerun the projection
C. operator surface first-class HumanDecision write API,
effective project-config read API,
session and context health projection
D. adapter fix opencode activity and session resolution
E. UI wire capabilities as backend support becomes real
```
The framing that ties them together is the thing to keep. The UI says what
Orchestra cannot expose or control. The debt ledger says which of those
shortcomings repeatedly costs something. The burn-in says which runtime
semantics are reliable. Those three decide what gets built next.
One design question to settle before B's slice two writes any code. The worker
ring is bounded and lossy by construction. Ingesting it durably means deciding
whether the coordinator stores every observation as an event, or only
transitions. Storing every heartbeat's ring would write the same 41-count
observation hundreds of times.