Record run 19: the manual verification seam, and F63
The keyed seam holds. Automated checks alone leave a phase awaiting, a generic approving comment satisfies nothing, and the keyed form verifies exactly the phase it names under the plan ref Orchestra supplies. A re-verification request reruns the plan's own commands at the new tree and writes a record bound to it. F63 is what the rig found. A manual sign-off is consulted by existence alone, so it survives every later tree the human never saw, while the automated half is re-established at each new SHA. The proposed fix stamps the satisfied record's sha on the decision and requires it to match. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CVbaKucEYBjMqVeUgJUsc1
This commit is contained in:
@@ -2760,3 +2760,91 @@ observed without being asked for.
|
||||
worker's tick save can clobber the edit. It clobbered run 17's attempt and
|
||||
survived run 18's. Stop, edit, start is the reliable sequence, and it needs
|
||||
`systemctl stop` and `start` in the unattended sudo rule.
|
||||
|
||||
## Run 19, 2026-08-29: the manual verification seam, and F63
|
||||
|
||||
Two tasks, both ingested from Gitea so that comments reconcile:
|
||||
`06G4W2TMZ9MW9SEBM5MGTM5288` (issue 24, plan `500464b9`) and
|
||||
`06G4W63545T3RV8SSKGKN11B3G` (issue 26, plan `53f7918b`).
|
||||
|
||||
A burn-in task cannot test this rung. Comments reconcile only for the source
|
||||
the task came from, so a task created with `source: "burnin"` has no comment
|
||||
stream at all.
|
||||
|
||||
### Proven: the keyed seam
|
||||
|
||||
```text
|
||||
15:09:15 PlanPhaseVerified phase-1 awaiting_manual_verification at 80728967
|
||||
comment: "Looks good to me. Nice work on this one, ship it."
|
||||
17:10:03 HumanDecisionRecorded subject=operator_instruction
|
||||
phase-1 unchanged, still awaiting_manual_verification
|
||||
comment: "orchestra verify phase-1"
|
||||
17:10:25 HumanDecisionRecorded subject=plan_phase_verification:500464b9…:phase-1
|
||||
phase-1 verified
|
||||
```
|
||||
|
||||
All five phases reached `awaiting_manual_verification` on their automated
|
||||
checks alone. A generic approval satisfied nothing. The keyed form flipped
|
||||
exactly the phase it named, under the plan ref Orchestra supplied rather than
|
||||
one the commenter chose.
|
||||
|
||||
### Proven: re-verification at a new SHA
|
||||
|
||||
On issue 26, inside one live implement lease, with the worker stopped for the
|
||||
whole edit:
|
||||
|
||||
```text
|
||||
15:23:00 PlanPhaseVerified phase-1 awaiting_manual_verification at f5d8f90b
|
||||
operator commit moves the tree f5d8f90b -> 1f07b89c
|
||||
.orchestra/plan-progress.json requests phase-1 again
|
||||
15:23:17 PlanPhaseVerified phase-1 awaiting_manual_verification at 1f07b89c
|
||||
17:23:16 HumanDecisionRecorded subject=plan_phase_verification:53f7918b…:phase-1
|
||||
phase-1 verified at 1f07b89c
|
||||
```
|
||||
|
||||
The rerun ran the plan's own commands at the new tree and wrote a fresh record
|
||||
bound to it. The worker never reads a command out of the request: the request
|
||||
names a phase and nothing else about it is trusted.
|
||||
|
||||
### F63: a manual sign-off outlives the tree it was given against
|
||||
|
||||
The second half of the same rig, with no new comment:
|
||||
|
||||
```text
|
||||
operator commit moves the tree 1f07b89c -> 28af84ff
|
||||
.orchestra/plan-progress.json requests phase-1 again
|
||||
15:24:02 PlanPhaseVerified phase-1 verified at 28af84ff
|
||||
```
|
||||
|
||||
`RecordPlanPhaseVerification` consults `manuallySignedOff`
|
||||
(`internal/operations/planprogress.go:127,152`), which asks only whether a
|
||||
decision with that subject exists. It does, forever, for that plan ref and
|
||||
phase id. So the automated half of a phase is re-established at every new SHA
|
||||
while the manual half is asserted from a tree the human never saw.
|
||||
|
||||
This is the same class of bug `AtSHA` exists to prevent, applied to the half
|
||||
that `AtSHA` does not cover. A manual check on this project is a human reading
|
||||
printed output; an edit between the sign-off and the rerun can change exactly
|
||||
that output.
|
||||
|
||||
Only a replan clears it, because `PlanPhases()` drops records whose plan ref is
|
||||
not the accepted one.
|
||||
|
||||
Proposed shape, not built: when the reducer flips a record on a keyed decision
|
||||
(`internal/store/store.go:273`), it knows the record it satisfied and that
|
||||
record's `AtSHA`. Stamp that sha on the decision, and have `manuallySignedOff`
|
||||
require it to match the run being recorded. A sign-off that arrives before any
|
||||
run has no sha and keeps today's behaviour, so the ordinary ordering is
|
||||
unchanged.
|
||||
|
||||
### Also established
|
||||
|
||||
- `review -> implement` is legal in `legalPhaseTransitions`, but
|
||||
`AdvanceWorkPhase` only walks the project's path forward. A task that has
|
||||
left implement can return only through a review that requests changes, not
|
||||
through the operator phase endpoint.
|
||||
- The agent's re-verification request is a file an operator can write:
|
||||
`.orchestra/plan-progress.json`, `{"phase": "...", "status":
|
||||
"ready_for_verification"}`. It is the whole lever for this rung.
|
||||
- Ingest turns every issue label into a required capability, so a rig issue
|
||||
must carry no labels or it will not schedule.
|
||||
|
||||
Reference in New Issue
Block a user