Record run 19: the manual verification seam, and F63

The keyed seam holds. Automated checks alone leave a phase awaiting, a
generic approving comment satisfies nothing, and the keyed form verifies
exactly the phase it names under the plan ref Orchestra supplies. A
re-verification request reruns the plan's own commands at the new tree and
writes a record bound to it.

F63 is what the rig found. A manual sign-off is consulted by existence
alone, so it survives every later tree the human never saw, while the
automated half is re-established at each new SHA. The proposed fix stamps
the satisfied record's sha on the decision and requires it to match.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CVbaKucEYBjMqVeUgJUsc1
This commit is contained in:
2026-08-29 19:25:21 +04:00
parent 99b209ba10
commit bbf3335857
+88
View File
@@ -2760,3 +2760,91 @@ observed without being asked for.
worker's tick save can clobber the edit. It clobbered run 17's attempt and
survived run 18's. Stop, edit, start is the reliable sequence, and it needs
`systemctl stop` and `start` in the unattended sudo rule.
## Run 19, 2026-08-29: the manual verification seam, and F63
Two tasks, both ingested from Gitea so that comments reconcile:
`06G4W2TMZ9MW9SEBM5MGTM5288` (issue 24, plan `500464b9`) and
`06G4W63545T3RV8SSKGKN11B3G` (issue 26, plan `53f7918b`).
A burn-in task cannot test this rung. Comments reconcile only for the source
the task came from, so a task created with `source: "burnin"` has no comment
stream at all.
### Proven: the keyed seam
```text
15:09:15 PlanPhaseVerified phase-1 awaiting_manual_verification at 80728967
comment: "Looks good to me. Nice work on this one, ship it."
17:10:03 HumanDecisionRecorded subject=operator_instruction
phase-1 unchanged, still awaiting_manual_verification
comment: "orchestra verify phase-1"
17:10:25 HumanDecisionRecorded subject=plan_phase_verification:500464b9…:phase-1
phase-1 verified
```
All five phases reached `awaiting_manual_verification` on their automated
checks alone. A generic approval satisfied nothing. The keyed form flipped
exactly the phase it named, under the plan ref Orchestra supplied rather than
one the commenter chose.
### Proven: re-verification at a new SHA
On issue 26, inside one live implement lease, with the worker stopped for the
whole edit:
```text
15:23:00 PlanPhaseVerified phase-1 awaiting_manual_verification at f5d8f90b
operator commit moves the tree f5d8f90b -> 1f07b89c
.orchestra/plan-progress.json requests phase-1 again
15:23:17 PlanPhaseVerified phase-1 awaiting_manual_verification at 1f07b89c
17:23:16 HumanDecisionRecorded subject=plan_phase_verification:53f7918b…:phase-1
phase-1 verified at 1f07b89c
```
The rerun ran the plan's own commands at the new tree and wrote a fresh record
bound to it. The worker never reads a command out of the request: the request
names a phase and nothing else about it is trusted.
### F63: a manual sign-off outlives the tree it was given against
The second half of the same rig, with no new comment:
```text
operator commit moves the tree 1f07b89c -> 28af84ff
.orchestra/plan-progress.json requests phase-1 again
15:24:02 PlanPhaseVerified phase-1 verified at 28af84ff
```
`RecordPlanPhaseVerification` consults `manuallySignedOff`
(`internal/operations/planprogress.go:127,152`), which asks only whether a
decision with that subject exists. It does, forever, for that plan ref and
phase id. So the automated half of a phase is re-established at every new SHA
while the manual half is asserted from a tree the human never saw.
This is the same class of bug `AtSHA` exists to prevent, applied to the half
that `AtSHA` does not cover. A manual check on this project is a human reading
printed output; an edit between the sign-off and the rerun can change exactly
that output.
Only a replan clears it, because `PlanPhases()` drops records whose plan ref is
not the accepted one.
Proposed shape, not built: when the reducer flips a record on a keyed decision
(`internal/store/store.go:273`), it knows the record it satisfied and that
record's `AtSHA`. Stamp that sha on the decision, and have `manuallySignedOff`
require it to match the run being recorded. A sign-off that arrives before any
run has no sha and keeps today's behaviour, so the ordinary ordering is
unchanged.
### Also established
- `review -> implement` is legal in `legalPhaseTransitions`, but
`AdvanceWorkPhase` only walks the project's path forward. A task that has
left implement can return only through a review that requests changes, not
through the operator phase endpoint.
- The agent's re-verification request is a file an operator can write:
`.orchestra/plan-progress.json`, `{"phase": "...", "status":
"ready_for_verification"}`. It is the whole lever for this rung.
- Ingest turns every issue label into a required capability, so a rig issue
must carry no labels or it will not schedule.