orchestra/BURNIN.md

Burn-in: live conformance, not more architecture

Written 2026-08-26. The workflow is feature-complete enough to exercise. What remains is empirical: run the same real task shape through each harness and classify what breaks. Isolated tests are insufficient by design here, and only the live owner path establishes conformance.

Do not add workflow features while this is running. Findings decide the next implementation work.

Burn-in build identity

6f9300b549362c4c5788f8845b56aaff9672d993

Both halves must report exactly this revision before a task is created. Neither needs a credential now: the coordinator prints it in docker logs orchestra-api and the worker in journalctl -u orchestra-worker. The same object is at GET /v1/admin/diagnostics and GET /v1/federation/workers behind the operator login.

Build both with deploy/build.sh <outdir>, which refuses a dirty tree. Later documentation-only commits do not change this identity, so a rebuild either passes this revision explicitly or accepts the new one and redeploys both halves. Never one half.

Deployment state, 2026-08-26 18:35

Half	State
Coordinator (homesrv)	Deployed at 6f9300b. Rebuilt with --build-arg BUILD_REVISION, recreated, /readyz ready, self-reporting the revision in its log
Worker (workpc)	Staged, not installed. ~/orchestra-deploy/orchestra-worker sha256 2b1c43071eaf6b5c15b110de39f204038a9629897e0ce3dacf45be96a6e6529e. Installed binary is still the 2026-07-30 build

Two operator steps remain, both needing root:

sudo install -m 0755 /home/kami/orchestra-deploy/orchestra-worker /usr/local/bin/orchestra-worker
sudo systemctl restart orchestra-worker
journalctl -u orchestra-worker -n 5 --no-pager   # must print revision 6f9300b...

Then scrub the pane environment before letting an agent run.

Baseline: one harness, one path

workpc-opencode only. Codex is a coverage gap, not a prerequisite, and no worker runs on homesrv. Prove one real path first:

real vikunja task
→ workpc-opencode
→ frame/research/plan
→ trajectory gate if configured
→ implement
→ independent review
→ task pr
→ human merge
→ completed

Then raise difficulty in this order:

1. opencode boring success
2. opencode mid-session human correction
3. opencode forced rotation
4. opencode pr rejection -> fix -> merge
5. opencode reconcile outage/recovery
6. representative cases on claude
7. add the codex adapter/config
8. cross-harness rotation: opencode -> claude/codex

Herdr transports, so nobody misdiagnoses again

No entry in the live config.jsonc sets backend or address, so all six resolve to <machine>:9245 (registry.defaultHerdrPort). Both ports are closed. That says nothing about the workpc harnesses: they are worker-owned, and the coordinator logs worker-owned on workpc; coordinator probe skipped for each. The worker reaches workpc-claude over a tmux socket and workpc-opencode over /home/kami/.config/herdr/herdr.sock. Do not diagnose harness availability from a TCP probe of 9245.

Evidence to record per run

One row per run. The launch instruction is now dumped to <worktree>/.orchestra/launch.md (herdr.LaunchContextFile) at every launch, local and federated, so the context is auditable without reading pane scrollback.

harness            (claude | codex | opencode)
task id
lease epoch        (every epoch, if the task rotates)
session ids        (pane ids per epoch)
human decision ids
launch context     (sha256 of .orchestra/launch.md, per launch)
handoff ref
git sha            (at every boundary: launch, each handoff, review, submit, merge)
review ref
submission ref
pr id
completion receipt

The five flows

1. Boring success. task → research → plan → implement → review → pr → merge.
2. Mid-session correction. Agent is doing A, the human says B, the next verified boundary delivers B, and A becomes history rather than a competing instruction.
3. Rotation. Agent A hands off, agent B resumes. Switch harnesses across the boundary if both are up.
4. Human rejection. Submit sha A, comment on the pull request, task reopens, fix at sha B, a fresh review, the same pull request, merge.
5. Failure path. Human source goes down, the reconcile streak escalates, the session hands off, the successor lease is refused, the source recovers, and a corrected successor starts.

What to inspect by hand

For every launch, read .orchestra/launch.md and ask only:

does this agent know
- what the task actually wants?
- what was most recently decided?
- what phase it is in?
- what earlier material is merely historical?
- what to do next?

Then compare that against what the model did.

Classify before fixing

Every failure gets exactly one label before any code is written:

authority bug            the wrong thing outranked the right thing
context-selection bug    the renderer showed or hid the wrong material
lifecycle bug            state, lease, phase or event handling is wrong
adapter/harness bug       the pane, occupancy, boundary or prompt path is wrong
model-following failure  the context was right and the model ignored it
operator-policy gap      credentials, deployment or configuration

This matters because the cheap response to every failure is another prompt rule, and prompt rules accumulated to compensate for lifecycle or adapter bugs are how the enforcement boundary rots. A model-following failure is the only class a prompt change should ever answer.

Pane credential cleanup (operator, in parallel)

The agent pane inherits the worker's environment. On workpc the worker is orchestra-worker.service, User=kami, EnvironmentFile=/etc/orchestra/worker.env (mode 0600, root-owned, deliberately unreadable here). Remove from that file, and from anything else the pane inherits:

gitea mutation token
vikunja mutation credentials
orchestra operator / system token

Retain only ORCHESTRA_AGENT_TOKEN and credentials a specific task genuinely needs. If an agent needs a forge operation, it asks Orchestra to perform it.

Then verify from inside a real pane:

git push                 # must fail unless Orchestra supplied the auth
curl .../v1/tasks/<id>/submission   # must be 403 for the agent surface
tea pr create            # must have no usable mutation credential

Note what is not enforced by code: nothing in this repo scrubs the pane environment, and a fake git earlier in $PATH is not enforcement because /usr/bin/git bypasses it. Credential isolation is the enforcement. Confinement of network, filesystem and destructive commands belongs to whatever launches the process (herdr, a container, systemd, bubblewrap), not to Orchestra, which supplies identity and policy inputs.

Exit criterion

Roughly 10 to 20 real tasks, with the five flows covered on each harness that is actually up. Then read the classified failures and decide the next implementation unit from them.