From 0797432d6f04b3af5295c1f0c4811835cfd16127 Mon Sep 17 00:00:00 2001 From: kami Date: Fri, 28 Aug 2026 23:55:20 +0400 Subject: [PATCH] Record run 13: F60 settled, and F18 populated under a real failure Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01CVbaKucEYBjMqVeUgJUsc1 --- BURNIN.md | 83 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) diff --git a/BURNIN.md b/BURNIN.md index 8059448..df0112e 100644 --- a/BURNIN.md +++ b/BURNIN.md @@ -2331,3 +2331,86 @@ worker sha256 fe7d2188b12369fb7682d3424fa11e2db47abed9c55313291a4b467b5972 `REQUEST_CHANGES` needs a separate bot account for `ORCHESTRA_GITEA_TOKEN`. - **A blocked task that is never reopened can still loop** on a dead release transaction. F59 covers failed, not blocked. + +## Run 13, 2026-08-28: F60, and the ring proven under a real failure + +### F60: a transaction must settle deterministically, in every case + +`3c7cf95`. F58 and F59 each fixed one case of the same rule. F60 states the +rule and covers the rest: + +> a transaction that can no longer be legitimately committed must be abandoned +> deterministically, and must never spin on an answer that cannot change. + +- **Terminal is failed or completed.** Both drop the transaction and free the + session. Completed was the gap F59 left. +- **Blocked keeps the transaction**, because a reopen returns the task to the + queue and that exact owner can still commit. +- **`TaskBlocked` now retains the ending epoch**, as `TaskReleased` already did. + Without it a reopened task has no epoch for the late-handoff path to fence + against, so keeping the transaction would be a lie. +- **A refusal parks the commit**, 30s backing off to a 5 minute cap. Any event + about the task un-parks it. A reopen arrives as `TaskCorrected`, so the rule + cannot be a list of event types. +- **A transport failure is not an answer** and retries at once. The park keys on + a 4xx `StatusError`, never on any error. + +### Proven live, and the contrast with run 10 + +Task `06G4KVSHAK9B8M9K8HENAF23CG`, same race-guard rig: force the expiry inside +the push, then hand the task to `race-guard-probe`. + +```text +19:54:10.888 transaction 06G4KVYT91GQH5ZYY6CNG6FWZ8 opens at prepared +19:54:10.907 forced expiry accepted +19:54:10.916 successor lease to race-guard-probe accepted +19:54:11.776 commit refused once, 409 lease not owned, parked +19:54:15.741 superseded by lease 06G4KVYSXJH053K3DAG6FKHD20, transaction abandoned +``` + +Run 10's equivalent ran roughly 5,000 retries over seven hours and needed an +operator to clear the state file by hand. This one asked once and settled +itself in four seconds. Worker state afterwards: no sessions, no leases, no +transactions, and `tmux -L orchestra ls` empty. + +### F18 populated under a real failure + +The gap the previous section recorded is closed. The same run produced this +health payload, two distinct observations rather than one overwritten slot: + +```json +"observations": [ + {"message": "release 06G4KVSHAK9B8M9K8HENAF23CG commit: federation: 409 Conflict: lease not owned", + "count": 1, "first": "2026-08-28T19:54:11.776815524Z", "last": "2026-08-28T19:54:11.776815524Z"}, + {"message": "release 06G4KVSHAK9B8M9K8HENAF23CG superseded by lease 06G4KVYSXJH053K3DAG6FKHD20: abandoning transaction 06G4KVYT91GQH5ZYY6CNG6FWZ8", + "count": 1, "first": "2026-08-28T19:54:15.74116014Z", "last": "2026-08-28T19:54:15.74116014Z"} +] +``` + +Under the old single slot the abandonment message would have erased the 409 +that caused it, and the causal chain would have been unreadable. Both counts +are 1, which is itself the F60 evidence: nothing looped. + +### Deployed state + +| Half | Revision | +|---|---| +| Coordinator, homesrv container | `3c7cf95` | +| Worker, workpc systemd | `3c7cf95` | + +```text +commit 3c7cf95d8cdaaff5dcec460c5d7874f7f4f0dbea +coordinator sha256 3fec1e99a638da851f5c970abaf80a006dc87425936f71fe33f10830f4bac573 +worker sha256 a373445f167f0a321dfc3866a1d2ed703b670b0fb3c188a8d00ad8099586dfdf +``` + +`orchestra-f18-baseline` tags `d6ee10f`, the post-F18 deployed baseline, and is +pushed. The branch is pushed too: 91 commits, because nothing had gone up since +`97a9c65` on 2026-07-31. + +### Still open + +Unchanged and all operational rather than runtime: 29 blocked `test-e2e` tasks, +three unschedulable `correx` tasks, and the Gitea 422 on reviewing your own +pull request. The rig task above will requeue when the probe lease expires and +start fresh, which is correct: its anchor was abandoned, not committed.