From bbf33358576204812dac3144f9ec66cc4d44ab8d Mon Sep 17 00:00:00 2001 From: kami Date: Sat, 29 Aug 2026 19:25:21 +0400 Subject: [PATCH] Record run 19: the manual verification seam, and F63 The keyed seam holds. Automated checks alone leave a phase awaiting, a generic approving comment satisfies nothing, and the keyed form verifies exactly the phase it names under the plan ref Orchestra supplies. A re-verification request reruns the plan's own commands at the new tree and writes a record bound to it. F63 is what the rig found. A manual sign-off is consulted by existence alone, so it survives every later tree the human never saw, while the automated half is re-established at each new SHA. The proposed fix stamps the satisfied record's sha on the decision and requires it to match. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01CVbaKucEYBjMqVeUgJUsc1 --- BURNIN.md | 88 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) diff --git a/BURNIN.md b/BURNIN.md index 50cbad0..f154018 100644 --- a/BURNIN.md +++ b/BURNIN.md @@ -2760,3 +2760,91 @@ observed without being asked for. worker's tick save can clobber the edit. It clobbered run 17's attempt and survived run 18's. Stop, edit, start is the reliable sequence, and it needs `systemctl stop` and `start` in the unattended sudo rule. + +## Run 19, 2026-08-29: the manual verification seam, and F63 + +Two tasks, both ingested from Gitea so that comments reconcile: +`06G4W2TMZ9MW9SEBM5MGTM5288` (issue 24, plan `500464b9`) and +`06G4W63545T3RV8SSKGKN11B3G` (issue 26, plan `53f7918b`). + +A burn-in task cannot test this rung. Comments reconcile only for the source +the task came from, so a task created with `source: "burnin"` has no comment +stream at all. + +### Proven: the keyed seam + +```text +15:09:15 PlanPhaseVerified phase-1 awaiting_manual_verification at 80728967 + comment: "Looks good to me. Nice work on this one, ship it." +17:10:03 HumanDecisionRecorded subject=operator_instruction + phase-1 unchanged, still awaiting_manual_verification + comment: "orchestra verify phase-1" +17:10:25 HumanDecisionRecorded subject=plan_phase_verification:500464b9…:phase-1 + phase-1 verified +``` + +All five phases reached `awaiting_manual_verification` on their automated +checks alone. A generic approval satisfied nothing. The keyed form flipped +exactly the phase it named, under the plan ref Orchestra supplied rather than +one the commenter chose. + +### Proven: re-verification at a new SHA + +On issue 26, inside one live implement lease, with the worker stopped for the +whole edit: + +```text +15:23:00 PlanPhaseVerified phase-1 awaiting_manual_verification at f5d8f90b + operator commit moves the tree f5d8f90b -> 1f07b89c + .orchestra/plan-progress.json requests phase-1 again +15:23:17 PlanPhaseVerified phase-1 awaiting_manual_verification at 1f07b89c +17:23:16 HumanDecisionRecorded subject=plan_phase_verification:53f7918b…:phase-1 + phase-1 verified at 1f07b89c +``` + +The rerun ran the plan's own commands at the new tree and wrote a fresh record +bound to it. The worker never reads a command out of the request: the request +names a phase and nothing else about it is trusted. + +### F63: a manual sign-off outlives the tree it was given against + +The second half of the same rig, with no new comment: + +```text + operator commit moves the tree 1f07b89c -> 28af84ff + .orchestra/plan-progress.json requests phase-1 again +15:24:02 PlanPhaseVerified phase-1 verified at 28af84ff +``` + +`RecordPlanPhaseVerification` consults `manuallySignedOff` +(`internal/operations/planprogress.go:127,152`), which asks only whether a +decision with that subject exists. It does, forever, for that plan ref and +phase id. So the automated half of a phase is re-established at every new SHA +while the manual half is asserted from a tree the human never saw. + +This is the same class of bug `AtSHA` exists to prevent, applied to the half +that `AtSHA` does not cover. A manual check on this project is a human reading +printed output; an edit between the sign-off and the rerun can change exactly +that output. + +Only a replan clears it, because `PlanPhases()` drops records whose plan ref is +not the accepted one. + +Proposed shape, not built: when the reducer flips a record on a keyed decision +(`internal/store/store.go:273`), it knows the record it satisfied and that +record's `AtSHA`. Stamp that sha on the decision, and have `manuallySignedOff` +require it to match the run being recorded. A sign-off that arrives before any +run has no sha and keeps today's behaviour, so the ordinary ordering is +unchanged. + +### Also established + +- `review -> implement` is legal in `legalPhaseTransitions`, but + `AdvanceWorkPhase` only walks the project's path forward. A task that has + left implement can return only through a review that requests changes, not + through the operator phase endpoint. +- The agent's re-verification request is a file an operator can write: + `.orchestra/plan-progress.json`, `{"phase": "...", "status": + "ready_for_verification"}`. It is the whole lever for this rung. +- Ingest turns every issue label into a required capability, so a rig issue + must carry no labels or it will not schedule.