Hand off with the lifecycle frozen and the plan machinery unproven
The tag orchestra-conformance-v1 marks the full original lifecycle, live-proven by run 6. Both deployed halves sit there deliberately. The four plan-machinery commits above it pass build, vet and test, and have never run against a live agent. The handoff records what they need before they can: a verification policy on test-e2e, both halves rebuilt together, and F18 fixed first, since the burn-in will generate the retry loops it exists to make visible. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CVbaKucEYBjMqVeUgJUsc1
This commit is contained in:
@@ -1900,3 +1900,58 @@ Configuration this needed, for the record: `quality_gate` on `test-e2e` in both
|
||||
`config.jsonc` (coordinator) and `/etc/orchestra/worker-projects.json` (worker,
|
||||
which is the copy `finalize` actually reads), plus `ORCHESTRA_REVIEW_ACTORS=kami`
|
||||
in the compose `.env`.
|
||||
|
||||
## Plan machinery, written 2026-08-28, unproven live
|
||||
|
||||
Four commits above `orchestra-conformance-v1`. Nothing deployed. Full detail is
|
||||
in `HANDOFF-2026-08-28-plan-machinery.md`; this is the ledger entry.
|
||||
|
||||
| Commit | What |
|
||||
|---|---|
|
||||
| `822f086` | Research findings gain id and confidence. The advertised schema becomes true. |
|
||||
| `57c028f` | `plan.md` replaces the four-bullet-list plan artifact. Renders verbatim into the implement launch. |
|
||||
| `a221502` | Plan phase progress Orchestra establishes, plus the project verification allowlist. |
|
||||
| `c76112a` | Typed plan mismatch and the Orchestra-owned reopen. |
|
||||
|
||||
### The burn-in it needs
|
||||
|
||||
Configuration first: `test-e2e` needs a `verification` block in the
|
||||
coordinator's `config.jsonc`. Absent policy refuses every plan command, so
|
||||
without it a plan seals and no phase can ever verify. Both halves must be
|
||||
rebuilt and redeployed together, because the worker calls two new federation
|
||||
routes.
|
||||
|
||||
Then the ladder:
|
||||
|
||||
```text
|
||||
1. plan seals three detailed phases
|
||||
-> grep the rendered launch.md for the phase-three verification command
|
||||
2. phase 1 requests verification
|
||||
-> worker runs the exact plan command
|
||||
-> durable state says verified
|
||||
3. rotation
|
||||
-> successor gets the complete original plan
|
||||
-> phase 1 verified, phase 2 explicitly current
|
||||
4. human correction lands above the plan
|
||||
5. phase 2 contradiction
|
||||
-> PlanMismatchRecorded, no improvisation
|
||||
-> old plan retained until the revised one seals
|
||||
-> fresh launch carries the revised plan in full
|
||||
6. a plan command outside project policy
|
||||
-> refused at seal time, on the planner
|
||||
7. a legacy plan
|
||||
-> "phase progress unavailable", task still completes
|
||||
```
|
||||
|
||||
Steps 1 and 3 are the rungs that matter for smaller local models. Steps 6 and 7
|
||||
were added from what this session's implementation surfaced.
|
||||
|
||||
Freeze as `orchestra-plan-v1` if it passes.
|
||||
|
||||
### F18 is now blocking
|
||||
|
||||
Three more instances landed with this change: every refusal in the new plan
|
||||
machinery records into the same single `last_error` slot. That is the fourth,
|
||||
fifth and sixth instance. The plan burn-in will produce exactly the five-second
|
||||
retry loops the bounded observation ring exists to make visible, so fix F18
|
||||
before running it.
|
||||
|
||||
@@ -0,0 +1,286 @@
|
||||
# Handoff: the lifecycle is proven and frozen, the plan machinery is written and unproven
|
||||
|
||||
Written 2026-08-28, 12:22 local (08:22 UTC). Read with `BURNIN.md` (the ledger,
|
||||
current through F46 and run 6), `PLAN-SPEC-DESIGN.md` (the design this
|
||||
implements), `HANDOFF-2026-08-28-burnin-5.md` (the session before this one),
|
||||
`AUDIT.md` and `CLAUDE.md`.
|
||||
|
||||
Everything below was observed live unless it says otherwise.
|
||||
|
||||
## The headline
|
||||
|
||||
Two things happened, in this order, deliberately.
|
||||
|
||||
**The original Orchestra lifecycle is proven end to end and tagged.** Run 6
|
||||
carried issue `kami/test-e2e#7` from creation to `TaskCompleted`. The chain
|
||||
covered a real pull request, a trusted human rejection, a resubmission onto the
|
||||
same pull request, and a merge. No manual lifecycle intervention.
|
||||
|
||||
**The plan machinery is written, tested, and deployed nowhere.** Four commits
|
||||
sit above the tag. `go build`, `go vet` and `go test ./...` pass across 21
|
||||
packages, and not one line of it has run against a live agent.
|
||||
|
||||
```text
|
||||
orchestra-conformance-v1 full lifecycle proven, both halves deployed here
|
||||
↓ four commits
|
||||
HEAD plan.md + verified phase progress
|
||||
+ policy-bounded verification + typed mismatch
|
||||
```
|
||||
|
||||
## Deployed state
|
||||
|
||||
| Half | Revision | Evidence |
|
||||
|---|---|---|
|
||||
| Worker, workpc systemd | `063a3ab` | PID 3840261, sha256 `51321b93…`, verified from `/proc/<pid>/exe` |
|
||||
| Coordinator, homesrv container | `4af9880` | `docker logs orchestra-api` prints the revision at startup |
|
||||
|
||||
Both are the tag. `063a3ab` contains `4af9880`: the two halves are one linear
|
||||
history, and the later commits are worker-only.
|
||||
|
||||
**Verify the running worker, not the installed file.** This bit two sessions
|
||||
ago, twice in one night. `sudo install` lands the new binary while the old
|
||||
process keeps serving from the unlinked inode. Always
|
||||
`sha256sum /proc/$(systemctl show -p MainPID --value orchestra-worker)/exe`.
|
||||
|
||||
`sudo` needs the operator in this sandbox. The worker deploy staging path is
|
||||
`~/orchestra-deploy/orchestra-worker`, and it currently holds the same
|
||||
`51321b93…` that is running.
|
||||
|
||||
## What run 6 proved
|
||||
|
||||
Task `06G4E6F69AKP2PA00S28D7ASBC`, issue `kami/test-e2e#7`, `workpc-claude`.
|
||||
|
||||
```text
|
||||
06:41:14 TaskCreated issue #7 polled
|
||||
06:41:37 frame -> research
|
||||
06:42:57 research -> plan
|
||||
06:45:12 plan -> implement
|
||||
06:46:27 implement -> review
|
||||
06:47:48 ReviewRecorded blocking 0, sha 41dee802
|
||||
06:47:49 TaskSubmitted PR 8 opened
|
||||
06:55:11 TaskChangesRequested trusted comment by kami
|
||||
06:55:11 review -> implement
|
||||
06:57:47 implement -> review
|
||||
07:00:09 ReviewRecorded blocking 0, sha 0d1585df
|
||||
07:00:09 TaskSubmitted PR 8 reused
|
||||
07:02:07 merge by kami
|
||||
07:02:11 TaskCompleted receipt binds merge f48e363d
|
||||
```
|
||||
|
||||
Gate sha, review sha, submitted sha and the pull request head are one commit,
|
||||
`0d1585df`. Each was checked against its artifact rather than its log line. The
|
||||
full acceptance table is in `BURNIN.md` under "Run 6".
|
||||
|
||||
## The premise that was wrong
|
||||
|
||||
The previous handoff said submission, human review and merge needed
|
||||
configuration rather than code. The event log held zero `ReviewRecorded` and
|
||||
zero `TaskSubmitted` across its whole history. Nothing had ever called either
|
||||
endpoint. Four defects, all live-proven and fixed:
|
||||
|
||||
- **F43**: the reviewed change had no path to the human. The worker completed
|
||||
directly, and review, submission and the publisher had no live caller.
|
||||
`e8d04d7`
|
||||
- **F44**: the Gitea source ingested Orchestra's own pull request as a new task,
|
||||
one per cycle, forever. `6ccc755`
|
||||
- **F45**: the submission reflection loop was dead code, guarded on a map filled
|
||||
600 lines later. A merge could never complete a task. `4af9880`
|
||||
- **F46**: stale `review.json` survives a changes-requested round trip.
|
||||
`063a3ab`
|
||||
|
||||
Plus `c11bf0e`, delivering a refusal to a reviewer whose findings file is
|
||||
missing rather than retrying every five seconds into worker health.
|
||||
|
||||
The lesson worth carrying: **a documented endpoint is not a live call path.**
|
||||
Grep for the caller before believing a feature exists. This is the third time
|
||||
this repo has produced that failure.
|
||||
|
||||
## What the four new commits do
|
||||
|
||||
### `822f086`, research findings are citable
|
||||
|
||||
`Finding` gains `ID` and `Confidence` (`fact|inference|assumption`). The brief
|
||||
at `agentctx.go` had advertised both for months while the struct dropped them
|
||||
silently on every seal.
|
||||
|
||||
`DecodeStoredResearch` reads what is already in the CAS and backfills both,
|
||||
labelling a backfilled finding `inference`. `DecodeResearch` stays strict for
|
||||
new seals.
|
||||
|
||||
### `57c028f`, `plan.md`
|
||||
|
||||
Markdown, 128 KiB, no per-line cap, same seal path and same `PlanRef`. The old
|
||||
artifact capped every entry at 500 single-line characters, which makes a
|
||||
specification impossible.
|
||||
|
||||
The parser enforces required sections, contiguous phase numbering from 1,
|
||||
`Files`/`Changes`/`Verification` per phase, and at least one check per phase.
|
||||
`run:` lines are JSON argv arrays. Headings inside fenced blocks are content,
|
||||
so a plan can show markdown without parsing its own example.
|
||||
|
||||
Citations resolve on the coordinator at seal time, the only party holding
|
||||
`ResearchRef`.
|
||||
|
||||
**The property this exists for**: the plan renders byte for byte into the
|
||||
implement launch. A rotated successor receives the same complete document.
|
||||
`TestAcceptedPlanRendersVerbatim` and `TestRotatedSuccessorReceivesTheWholePlan`
|
||||
guard it.
|
||||
|
||||
`DecodeStoredPlan` renders a legacy JSON plan into the same type, carrying zero
|
||||
phases. The implement brief then says "legacy accepted plan, phase progress
|
||||
unavailable" and the task continues under old semantics.
|
||||
|
||||
### `a221502`, progress Orchestra establishes
|
||||
|
||||
```text
|
||||
agent may request: ready_for_verification
|
||||
agent may not assert: verified, awaiting_manual_verification, failed, skipped
|
||||
```
|
||||
|
||||
The worker resolves commands from the coordinator, never from the request. They
|
||||
run as argv through `exec` with `Dir` set to the worktree, the quality gate's
|
||||
existing envelope, with no shell.
|
||||
|
||||
`registry.Project.Verification` matches argv positionally. An absent policy
|
||||
refuses everything: a plan command is agent-authored and must not inherit the
|
||||
operator-authored gate's reach.
|
||||
|
||||
`PlanPhaseRecord` binds `plan_ref` and `at_sha`. Progress earned under plan A
|
||||
cannot survive into plan B. A verified phase whose commit has moved renders as
|
||||
stale rather than as a claim about the current tree.
|
||||
|
||||
Manual steps hold a phase at `awaiting_manual_verification`. The sign-off is an
|
||||
ordinary human decision whose subject is
|
||||
`plan_phase_verification:<plan_ref>:<phase_id>`, so a later "looks good" on an
|
||||
unrelated thread cannot satisfy it.
|
||||
|
||||
### `c76112a`, typed mismatch
|
||||
|
||||
The report carries an observation and never a replacement plan.
|
||||
`requested_action` is advisory.
|
||||
|
||||
The worker fills `plan_ref` and `at_sha` from what it can verify, so a stale
|
||||
report is refused rather than replayed.
|
||||
|
||||
The distinction that matters in the reducer:
|
||||
|
||||
```text
|
||||
mismatch recorded != plan superseded
|
||||
```
|
||||
|
||||
A plan stops being accepted only when a replacement is sealed. An abandoned
|
||||
replan leaves the accepted plan and its progress intact. On a real re-seal the
|
||||
old ref moves to `PlanHistory` and its verification stays in the log as
|
||||
provenance.
|
||||
|
||||
`human_decision` never reopens. It blocks with a packet, and a human answer can
|
||||
resolve the contradiction without resealing anything.
|
||||
|
||||
`CanReopenPhase` is separate from `CanTransitionPhase`. Every path validating an
|
||||
agent request uses the latter, so `phase-request.json` still refuses a backward
|
||||
move.
|
||||
|
||||
## What the plan machinery needs before it can run
|
||||
|
||||
**Configuration, in two places.** The dual-config trap from run 6 applies here
|
||||
too. It is worse for verification, because the worker is the half that
|
||||
executes.
|
||||
|
||||
The coordinator's `config.jsonc` on homesrv
|
||||
(`~/docker-apps/orchestra-web-ui/config.jsonc`) needs a `verification` block on
|
||||
`test-e2e`:
|
||||
|
||||
```json
|
||||
"verification": { "allowed": [["bash", "-n", "*"], ["bash", "scripts/orchestra_e2e_healthcheck.sh"]] }
|
||||
```
|
||||
|
||||
`deploy/config.example.jsonc` documents the matcher rules. Absent policy refuses
|
||||
every plan command, so without this a plan seals and then no phase can ever
|
||||
verify.
|
||||
|
||||
**Operational invariant worth recording**: the worker-side project policy is
|
||||
authoritative for execution, because the worker runs the verification. The
|
||||
coordinator's copy is what `PlanPhaseCommands` authorises against. Today the
|
||||
policy lives on `registry.Project`, which is the coordinator's copy only. The
|
||||
worker's `/etc/orchestra/worker-projects.json` carries `quality_gate` and no
|
||||
verification field. It does not need one: the worker asks the coordinator which
|
||||
commands to run. Do not "fix" that by duplicating the policy.
|
||||
|
||||
**Both halves need rebuilding and redeploying together.** The four commits touch
|
||||
`internal/domain`, `internal/store`, `internal/agentctx`, `internal/operations`,
|
||||
`internal/registry`, `internal/federation`, `internal/workphase`, and both
|
||||
binaries. A split deployment would have a worker calling
|
||||
`/v1/federation/workers/<id>/plan-phase` against a coordinator that returns 404.
|
||||
|
||||
## The burn-in this needs
|
||||
|
||||
The proof ladder from `PLAN-SPEC-DESIGN.md` section 7, unchanged:
|
||||
|
||||
1. Plan seals three detailed phases. Implement launch contains the plan
|
||||
verbatim. **Grep the rendered `launch.md` for the phase-three verification
|
||||
command.** It is there or it is not.
|
||||
2. Phase 1 requests verification. Worker runs the exact plan command. Durable
|
||||
state says phase 1 verified.
|
||||
3. Rotation. Successor receives the complete original plan, phase 1 verified,
|
||||
phase 2 explicitly current.
|
||||
4. Human correction lands above the plan.
|
||||
5. Phase 2 hits a contradiction. Mismatch recorded, no improvisation, old plan
|
||||
retained until the revised one seals, fresh launch carries the revised plan
|
||||
in full.
|
||||
|
||||
Steps 1 and 3 are the rungs that matter for smaller local models.
|
||||
|
||||
Two more worth adding, from what this session found:
|
||||
|
||||
6. A plan whose commands fall outside project policy. The refusal must reach the
|
||||
planner at seal time, not the implementer at verification time.
|
||||
7. A legacy plan. The implement context must say phase progress is unavailable
|
||||
and the task must still complete.
|
||||
|
||||
## Things that will bite
|
||||
|
||||
- **Verify the running worker binary, not the installed one.** `/proc/<pid>/exe`.
|
||||
- **A documented endpoint is not a live call path.** Grep for the caller. F43,
|
||||
F45, and the two before them were all this.
|
||||
- **Gitea returns 422 for a review on your own pull request.** Orchestra opens
|
||||
the pull request as the same forge user the operator reviews as. That makes
|
||||
`REQUEST_CHANGES` unreachable in this deployment. The comment path works and
|
||||
is what run 6 exercised. A separate bot account for `ORCHESTRA_GITEA_TOKEN`
|
||||
would restore it.
|
||||
- **`quality_gate` lives in two files.** `config.jsonc` on the coordinator and
|
||||
`/etc/orchestra/worker-projects.json` on the worker. `finalize` reads the
|
||||
worker copy. Verification policy is coordinator-only, deliberately.
|
||||
- **Task `06G4E83E4KRXM8DS90M2648MGM` is blocked, deliberately.** It is the
|
||||
task F44 spawned from Orchestra's own pull request. Its worktree and branch
|
||||
are cleaned up. The task is left blocked on `operator_required` because it is
|
||||
not real work. Fail it or leave it.
|
||||
- **`rm` and `cp` are interactive in this shell.** A plain `rm` prompts, appears
|
||||
to succeed, and leaves the file. Use `/bin/rm -f` and `install`.
|
||||
- **Another session owns 13 uncommitted paths**, including `AUDIT.md`,
|
||||
`deploy/build.sh`, `web/` and `AGENTS.md`. Commit by path. Never `git add -A`
|
||||
without a pathspec.
|
||||
- **Secrets are guarded.** Reading `.env` or `printenv` of a token through the
|
||||
shell is blocked. The working recipe is expanding the token inside the
|
||||
container in a single remote command.
|
||||
- **`/tmp` is tmpfs with a 10-day sweep.** Check `/tmp/test-e2e` and
|
||||
`/tmp/test-e2e-worktrees` exist before every run. Eight worktrees are there
|
||||
now, most from completed tasks.
|
||||
- **Rebuild the coordinator from a detached worktree, never
|
||||
`docker compose build`.** The recipe is in `HANDOFF-2026-08-28-burnin-5.md`
|
||||
and was used four times today without trouble.
|
||||
|
||||
## What I would do next, in order
|
||||
|
||||
1. **Fix F18.** Three more instances landed today: every refusal in the new plan
|
||||
machinery records into the same single `last_error` slot. The bounded
|
||||
observation ring is overdue. The plan burn-in will generate exactly the kind
|
||||
of five-second retry loop it exists to make visible.
|
||||
2. **Configure `test-e2e` verification policy** in the coordinator config.
|
||||
3. **Rebuild and redeploy both halves at HEAD**, together.
|
||||
4. **Run the seven-step ladder above.** Freeze the result as
|
||||
`orchestra-plan-v1` if it passes.
|
||||
5. Only then consider the Gitea bot account and the duplicated `quality_gate`
|
||||
configuration. Both are real, neither blocks anything.
|
||||
|
||||
Do step 1 first. Steps 3 and 4 will produce failures whose only trace is
|
||||
otherwise the journal.
|
||||
Reference in New Issue
Block a user