docs: hand off live QA matrix

This commit is contained in:
kami
2026-07-30 16:09:23 +04:00
parent d6cab133b5
commit 682155c5fe
+43 -18
View File
@@ -3,9 +3,10 @@
Audited 2026-07-30 against the working tree, spec, deployed coordinator,
workpc worker, event log, and live herdr (read-only).
**Verdict:** the source-level P0/P1/P2 fixes are in place and the live
coordinator can replay its repaired event log, but the system is not safe to
run unattended until live harness capacity and the worker rollout are restored.
**Verdict:** the source-level P0/P1/P2 fixes are in place, the live
coordinator can replay its repaired event log, and the current OpenCode worker
is connected. The system is not safe to run unattended until the controlled
QA matrix has passed for all three harnesses.
## Evidence
@@ -16,23 +17,49 @@ run unattended until live harness capacity and the worker rollout are restored.
- Live follow-up: Docker owns the coordinator; the old systemd unit is
inactive. The pre-v2 repeated-`seq=1` event prefix was migrated with a
backup-preserving, explicit tool before the current coordinator replayed it.
- Deployment follow-up: coordinator and installed workpc OpenCode worker are
clean revision `d6cab133b56666f81f569f4c1c3c9a6f104088d1`. The worker service
restarted at 2026-07-30 16:07 +04, has emitted no federation failures since,
and its configured Unix-socket herdr answered `ping` with protocol `17`.
## Remaining release blockers
- **No live harness capacity.** Homesrv has no reachable herdr. Workpc has no
listener on any expected herdr port, so no safe live rotation, handoff,
completion, or late-recovery path can be exercised on Claude, Codex, or
OpenCode.
- **Worker rollout needs an administrator.** Workpc is still running the old
dirty `1ca9d64` `/usr/local/bin/orchestra-worker`. The current worker binary
is staged at `~/orchestra-deploy/orchestra-worker`, but this account cannot
install it or restart the systemd service. Its project configuration must
also declare supported projects before routing can admit a task.
- **Only OpenCode capacity is ready.** Workpc's `workpc-opencode` worker has
a configured project file and a reachable local herdr Unix socket. Homesrv
has no reachable herdr, and no Claude/Codex worker/herdr pair has been
verified, so the full three-harness matrix cannot begin yet.
- **B17 needs a fresh controlled run.** The historical probe's six approvals,
one `409 lease version conflict`, and `consumed:0` receipt came from the
old worker. They cannot be treated as evidence for the current worker until
a live OpenCode run is repeated; Claude and Codex require their own runs.
## QA handoff — next agent
1. **Preflight before creating work.** Read `GET /v1/federation/workers` and
coordinator diagnostics through an authenticated operator session. Confirm
each target worker reports revision `d6cab13`, supported `test-e2e`, and
fresh `herdr_status: reachable`; raw-ping its configured local Unix socket
with `params:{}` and confirm protocol 17. Confirm no pre-existing agents
or leased task on the target harness.
2. **OpenCode controlled continuity run.** Submit one new disposable
`test-e2e` task that makes a deterministic marker, releases at a clear turn
boundary, validates pickup from the resulting anchor, then completes.
Record event sequence, handoff ref, anchor SHA, transaction id, lease epoch,
native session evidence, quality-gate result, remote SHA, and a receipt
with known non-zero (or explicitly explained known-zero) usage. Do not use
destructive herdr calls against unrelated panes.
3. **Exercise rotation and recovery.** In separate disposable tasks, trigger
soft, hard, milestone, thrash, coordinator restart, worker restart/lost
response, stale completion, and corrupted-worker-state paths. Verify each
result is a fenced lifecycle event or durable `needs_attention`, never a
silent retry or orphaned pane. Preserve the predecessor until matching
pickup validation.
4. **Repeat on Claude and Codex only after provisioning their own reachable
worker/herdr pairs.** Do not treat OpenCode evidence as cross-harness
proof. After all runs, compare worker/coordinator revisions and checksums,
attach the artifacts/event ranges to this audit, and only then clear the
live release gate.
## P0 — correctness
| ID | Current failure | Required fix |
@@ -104,9 +131,7 @@ run unattended until live harness capacity and the worker rollout are restored.
- **Pass:** build, vet, test, and race checks pass; unit/integration coverage
includes the defined fault and cross-machine cases.
- **Pending live capacity:** run the controlled soft, hard, milestone, thrash,
completion, and late-recovery paths on each harness without manual
intervention for safe repository work.
- **Pending worker installation:** install the staged worker, restart it after
herdr is restored, and verify its checksum and immutable build revision
against the coordinator diagnostics.
- **Pending QA:** run the controlled soft, hard, milestone, thrash,
completion, and late-recovery paths on OpenCode, Claude, and Codex without
manual intervention for safe repository work. Follow the QA handoff above
and attach the resulting event ranges/artifacts before clearing this gate.