Compare commits

...

9 Commits

3 changed files with 91 additions and 52 deletions
+30 -42
View File
@@ -1,64 +1,52 @@
# Task 06G4FT0MB733H2ZCZ237WFHFD8
# Task 06G4V20T528ZTER7KZBVGNAYXC
Give the healthcheck a self-test and a machine-readable mode
Give the healthcheck a --timing flag, in five sequential phases
- Project: test-e2e
- Source: gitea:test-e2e/10
- Source: burnin/run17-implement-successor-and-f62
- Priority: 0
## Instructions
Three pieces of work, in order. Plan them as exactly three phases in that order.
Add a --timing flag to scripts/orchestra_e2e_healthcheck.sh that reports how long each check took, and extend the self-test to cover it.
## Phase 1 — a self-test for the healthcheck
Plan this as FIVE phases. Each phase is independently verifiable and must be implemented and verified on its own:
New file `scripts/orchestra_e2e_selftest.sh`, executable, `#!/usr/bin/env bash`.
1. Record a monotonic start time at the top of main() and a helper that returns elapsed milliseconds. No change to any existing output or exit status.
2. Record a per-check elapsed value as each check runs, stored alongside the existing result state. Still no output change.
3. Add the --timing flag. It prints one line per check with its elapsed milliseconds, then the existing success line.
4. Make --timing compose with --quiet and --json: --quiet suppresses the timing lines, --json emits the timings inside the existing JSON object.
5. Update the usage text in --help and extend scripts/test_healthcheck.sh with assertions for --timing, --timing --quiet, and --timing --json.
It runs `scripts/orchestra_e2e_healthcheck.sh`, then asserts:
## Work one phase at a time
- exit status is 0
- stdout contains `OK`
This is a hard requirement of this task, not a style preference. Implement phase N, request its verification, and only then begin phase N+1. Do not write the whole change first and verify the phases afterwards.
On failure it prints which assertion failed and exits 1. On success it prints `selftest: ok` and exits 0.
## Verification policy for this project
Verification for this phase is automated only. The one check is:
Only two commands may appear on a run: line, exactly:
- run: ["bash", "-n", "scripts/orchestra_e2e_selftest.sh"]
- ["bash", "-n", "<path>"]
- ["bash", "scripts/orchestra_e2e_healthcheck.sh"]
Do not add a manual step to this phase.
## Phase 2 — `--json` on the healthcheck
`scripts/orchestra_e2e_healthcheck.sh --json` prints exactly one line:
{"status":"ok","checks":1}
and exits 0. Without `--json` the existing human output is unchanged, and `--help` keeps working. `--json` and `--help` together behave as `--help`.
Verification for this phase must include both an automated check and a manual step. The JSON shape is a judgement a human confirms by reading it, so state the manual step as the exact command a human runs and the exact line they should see.
## Phase 3 — teach the self-test about `--json`
Extend `scripts/orchestra_e2e_selftest.sh` to run the healthcheck twice: once plain and once with `--json`. Assert the plain run still contains `OK`, and assert the `--json` run prints one line beginning with `{"status":`.
Verification for this phase is automated only:
- run: ["bash", "-n", "scripts/orchestra_e2e_selftest.sh"]
- run: ["bash", "scripts/orchestra_e2e_healthcheck.sh"]
## Constraints
The project's verification policy allows exactly two command shapes:
["bash", "-n", "<one file>"]
["bash", "scripts/orchestra_e2e_healthcheck.sh"]
A `run:` line outside those is refused when the plan seals, not later.
Anything else is refused when you seal the plan. Give every phase at least one automated check from that list, and add a manual check where a human should confirm the printed text.
## Research first
Before planning, establish at least three findings and label their confidence honestly: at least one `fact`, one `inference`, and one `assumption`. The plan must cite at least two of them in References.
Establish at least three findings with distinct confidence: at least one fact, one inference, one assumption.
## Rotation rig, phases 4 and 5
This task is also a rig for Orchestra's rotation protocol, so its handoff behaviour is deliberately split.
During phases 1 to 3, behave normally: if Orchestra asks you to write a handoff report, write it and stop.
From the moment phase 4 begins, do the opposite. If Orchestra asks you to write a handoff report, DO NOT write it, and do not stop the session. Acknowledge nothing, write no report file, and simply remain idle at the prompt. Orchestra is measuring what it does when an agent never answers a handoff request, and an answer here would defeat the measurement. This instruction applies only to phases 4 and 5 of this task.
## Quality gate
bash -n scripts/*.sh && bash scripts/orchestra_e2e_healthcheck.sh
## Completion
+53 -7
View File
@@ -1,28 +1,74 @@
#!/usr/bin/env bash
readonly VERSION=1.0.0
# emit <format> <template> [args...]: the single output path.
# format: text or json print; none suppresses (--quiet). Explicit argument,
# never read from an enclosing variable.
emit() {
local format=$1
shift
[ "$format" = none ] || printf "$@"
}
# now_us: current time in microseconds. Strips any locale decimal separator.
now_us() { printf '%s' "${EPOCHREALTIME//[!0-9]/}"; }
# elapsed_ms <start-us>: whole milliseconds since <start-us>.
elapsed_ms() { printf '%s' "$(( ($(now_us) - $1) / 1000 ))"; }
main() {
local start_us
start_us=$(now_us)
local quiet=
local json=
local timing=
for arg in "$@"; do
if [ "$arg" = --help ]; then
printf 'Usage: %s [--quiet]\n\n' "${0##*/}"
printf 'Healthcheck script for Orchestra E2E tests.\n'
printf 'Exits 0 with a success message if all checks pass.\n'
printf ' --quiet Suppress the success message; still exits 0.\n'
emit text 'Usage: %s [--quiet] [--json] [--timing]\n\n' "${0##*/}"
emit text 'Healthcheck script for Orchestra E2E tests.\n'
emit text 'Exits 0 with a success message if all checks pass.\n'
emit text ' --quiet Suppress the success message; still exits 0.\n'
emit text ' --timing Print elapsed milliseconds for each check.\n'
emit text 'Version: %s\n' "$VERSION"
exit 0
elif [ "$arg" = --quiet ]; then
quiet=1
elif [ "$arg" = --json ]; then
json=1
elif [ "$arg" = --timing ]; then
timing=1
fi
done
if [ -n "$json" ]; then
printf '{"status":"ok","checks":1}\n'
local format=text
[ -n "$quiet" ] && format=none
[ -n "$json" ] && format=json
local -a TIMINGS=()
TIMINGS+=("healthchecks=$(elapsed_ms "$start_us")")
if [ "$format" = json ]; then
local timings=
if [ -n "$timing" ]; then
local sep=
for t in "${TIMINGS[@]}"; do
timings+="$sep\"${t%%=*}\":${t#*=}"
sep=,
done
timings=",\"timings\":{$timings}"
fi
emit "$format" '{"status":"ok","checks":%d%s}\n' "${#TIMINGS[@]}" "$timings"
exit 0
fi
[ -n "$quiet" ] || printf 'OK - all healthchecks passed\n'
if [ -n "$timing" ]; then
for t in "${TIMINGS[@]}"; do
emit "$format" '%s: %sms\n' "${t%%=*}" "${t#*=}"
done
fi
emit "$format" 'OK - all healthchecks passed\n'
exit 0
}
+8 -3
View File
@@ -3,14 +3,16 @@
HC="$(dirname "$0")/orchestra_e2e_healthcheck.sh"
OK='OK - all healthchecks passed'
USAGE="$(printf 'Usage: orchestra_e2e_healthcheck.sh [--quiet]\n\nHealthcheck script for Orchestra E2E tests.\nExits 0 with a success message if all checks pass.\n --quiet Suppress the success message; still exits 0.')"
USAGE="$(printf 'Usage: orchestra_e2e_healthcheck.sh [--quiet] [--json] [--timing]\n\nHealthcheck script for Orchestra E2E tests.\nExits 0 with a success message if all checks pass.\n --quiet Suppress the success message; still exits 0.\n --timing Print elapsed milliseconds for each check.\nVersion: 1.0.0')"
fails=0
check() { # check <expected-stdout> <label> [args...]
local want=$1 label=$2 got status
local want=$1 label=$2 got status raw
shift 2
got=$(bash "$HC" "$@" 2>/dev/null)
raw=$(bash "$HC" "$@" 2>/dev/null)
status=$?
# Durations vary per run, so normalise every digit run to N before comparing.
got=$(printf '%s' "$raw" | sed -E 's/[0-9]+ms/Nms/g; s/("healthchecks":)[0-9]+/\1N/g')
if [ "$got" != "$want" ] || [ "$status" -ne 0 ]; then
printf 'FAIL: %s (exit=%s)\n---got---\n%s\n---want---\n%s\n' "$label" "$status" "$got" "$want"
fails=$((fails + 1))
@@ -23,6 +25,9 @@ check "$USAGE" '--help' --help
check "$USAGE" '--help --quiet' --help --quiet
check "$USAGE" '--quiet --help' --quiet --help
check "$OK" 'unknown flag ignored' --bogus
check "$(printf 'healthchecks: Nms\n%s' "$OK")" '--timing' --timing
check '' '--timing --quiet' --timing --quiet
check '{"status":"ok","checks":1,"timings":{"healthchecks":N}}' '--timing --json' --timing --json
[ "$fails" -eq 0 ] || exit 1
printf 'all checks passed\n'