docs: hand off live QA matrix
This commit is contained in:
@@ -3,9 +3,10 @@
|
|||||||
Audited 2026-07-30 against the working tree, spec, deployed coordinator,
|
Audited 2026-07-30 against the working tree, spec, deployed coordinator,
|
||||||
workpc worker, event log, and live herdr (read-only).
|
workpc worker, event log, and live herdr (read-only).
|
||||||
|
|
||||||
**Verdict:** the source-level P0/P1/P2 fixes are in place and the live
|
**Verdict:** the source-level P0/P1/P2 fixes are in place, the live
|
||||||
coordinator can replay its repaired event log, but the system is not safe to
|
coordinator can replay its repaired event log, and the current OpenCode worker
|
||||||
run unattended until live harness capacity and the worker rollout are restored.
|
is connected. The system is not safe to run unattended until the controlled
|
||||||
|
QA matrix has passed for all three harnesses.
|
||||||
|
|
||||||
## Evidence
|
## Evidence
|
||||||
|
|
||||||
@@ -16,23 +17,49 @@ run unattended until live harness capacity and the worker rollout are restored.
|
|||||||
- Live follow-up: Docker owns the coordinator; the old systemd unit is
|
- Live follow-up: Docker owns the coordinator; the old systemd unit is
|
||||||
inactive. The pre-v2 repeated-`seq=1` event prefix was migrated with a
|
inactive. The pre-v2 repeated-`seq=1` event prefix was migrated with a
|
||||||
backup-preserving, explicit tool before the current coordinator replayed it.
|
backup-preserving, explicit tool before the current coordinator replayed it.
|
||||||
|
- Deployment follow-up: coordinator and installed workpc OpenCode worker are
|
||||||
|
clean revision `d6cab133b56666f81f569f4c1c3c9a6f104088d1`. The worker service
|
||||||
|
restarted at 2026-07-30 16:07 +04, has emitted no federation failures since,
|
||||||
|
and its configured Unix-socket herdr answered `ping` with protocol `17`.
|
||||||
|
|
||||||
## Remaining release blockers
|
## Remaining release blockers
|
||||||
|
|
||||||
- **No live harness capacity.** Homesrv has no reachable herdr. Workpc has no
|
- **Only OpenCode capacity is ready.** Workpc's `workpc-opencode` worker has
|
||||||
listener on any expected herdr port, so no safe live rotation, handoff,
|
a configured project file and a reachable local herdr Unix socket. Homesrv
|
||||||
completion, or late-recovery path can be exercised on Claude, Codex, or
|
has no reachable herdr, and no Claude/Codex worker/herdr pair has been
|
||||||
OpenCode.
|
verified, so the full three-harness matrix cannot begin yet.
|
||||||
- **Worker rollout needs an administrator.** Workpc is still running the old
|
|
||||||
dirty `1ca9d64` `/usr/local/bin/orchestra-worker`. The current worker binary
|
|
||||||
is staged at `~/orchestra-deploy/orchestra-worker`, but this account cannot
|
|
||||||
install it or restart the systemd service. Its project configuration must
|
|
||||||
also declare supported projects before routing can admit a task.
|
|
||||||
- **B17 needs a fresh controlled run.** The historical probe's six approvals,
|
- **B17 needs a fresh controlled run.** The historical probe's six approvals,
|
||||||
one `409 lease version conflict`, and `consumed:0` receipt came from the
|
one `409 lease version conflict`, and `consumed:0` receipt came from the
|
||||||
old worker. They cannot be treated as evidence for the current worker until
|
old worker. They cannot be treated as evidence for the current worker until
|
||||||
a live OpenCode run is repeated; Claude and Codex require their own runs.
|
a live OpenCode run is repeated; Claude and Codex require their own runs.
|
||||||
|
|
||||||
|
## QA handoff — next agent
|
||||||
|
|
||||||
|
1. **Preflight before creating work.** Read `GET /v1/federation/workers` and
|
||||||
|
coordinator diagnostics through an authenticated operator session. Confirm
|
||||||
|
each target worker reports revision `d6cab13`, supported `test-e2e`, and
|
||||||
|
fresh `herdr_status: reachable`; raw-ping its configured local Unix socket
|
||||||
|
with `params:{}` and confirm protocol 17. Confirm no pre-existing agents
|
||||||
|
or leased task on the target harness.
|
||||||
|
2. **OpenCode controlled continuity run.** Submit one new disposable
|
||||||
|
`test-e2e` task that makes a deterministic marker, releases at a clear turn
|
||||||
|
boundary, validates pickup from the resulting anchor, then completes.
|
||||||
|
Record event sequence, handoff ref, anchor SHA, transaction id, lease epoch,
|
||||||
|
native session evidence, quality-gate result, remote SHA, and a receipt
|
||||||
|
with known non-zero (or explicitly explained known-zero) usage. Do not use
|
||||||
|
destructive herdr calls against unrelated panes.
|
||||||
|
3. **Exercise rotation and recovery.** In separate disposable tasks, trigger
|
||||||
|
soft, hard, milestone, thrash, coordinator restart, worker restart/lost
|
||||||
|
response, stale completion, and corrupted-worker-state paths. Verify each
|
||||||
|
result is a fenced lifecycle event or durable `needs_attention`, never a
|
||||||
|
silent retry or orphaned pane. Preserve the predecessor until matching
|
||||||
|
pickup validation.
|
||||||
|
4. **Repeat on Claude and Codex only after provisioning their own reachable
|
||||||
|
worker/herdr pairs.** Do not treat OpenCode evidence as cross-harness
|
||||||
|
proof. After all runs, compare worker/coordinator revisions and checksums,
|
||||||
|
attach the artifacts/event ranges to this audit, and only then clear the
|
||||||
|
live release gate.
|
||||||
|
|
||||||
## P0 — correctness
|
## P0 — correctness
|
||||||
|
|
||||||
| ID | Current failure | Required fix |
|
| ID | Current failure | Required fix |
|
||||||
@@ -104,9 +131,7 @@ run unattended until live harness capacity and the worker rollout are restored.
|
|||||||
|
|
||||||
- **Pass:** build, vet, test, and race checks pass; unit/integration coverage
|
- **Pass:** build, vet, test, and race checks pass; unit/integration coverage
|
||||||
includes the defined fault and cross-machine cases.
|
includes the defined fault and cross-machine cases.
|
||||||
- **Pending live capacity:** run the controlled soft, hard, milestone, thrash,
|
- **Pending QA:** run the controlled soft, hard, milestone, thrash,
|
||||||
completion, and late-recovery paths on each harness without manual
|
completion, and late-recovery paths on OpenCode, Claude, and Codex without
|
||||||
intervention for safe repository work.
|
manual intervention for safe repository work. Follow the QA handoff above
|
||||||
- **Pending worker installation:** install the staged worker, restart it after
|
and attach the resulting event ranges/artifacts before clearing this gate.
|
||||||
herdr is restored, and verify its checksum and immutable build revision
|
|
||||||
against the coordinator diagnostics.
|
|
||||||
|
|||||||
Reference in New Issue
Block a user