kami
7f12c7fc37
The v3 stack, previously an uncommitted working tree, plus this session's two units and the burn-in instrument. This commit is the burn-in build identity: coordinator and worker must both report this revision before a task is created. Workflow (earlier sessions, uncommitted until now): human decision events and reduction, source cursors and reconcile-before-launch, turn-boundary reconciliation, internal/agentctx as the single renderer, ace-fca phases with sealed artifacts, the trajectory gate, bounded grilling, independent review, task pr enforcement, and human review reflection. Capability restrictions at the agent boundary: an authz.Agent surface at GatedWrite may ask and may not act. It also fixes two bugs the unit exposed -- gated surfaces could not reach the two endpoints written for them, and RequestHumanDecision would block an unowned task while rejecting a question from the session that did own it. Turn-boundary reconcile-failure escalation: a streak of consecutive failures asks the session to hand off, fenced on the lease epoch, with reconcile_failure as a real handoff reason. The worker was dropping the coordinator's verdict on the floor; it now acts on it. Burn-in: herdr.WriteLaunchContext dumps the exact agentctx.Build result to <worktree>/.orchestra/launch.md at every launch, local and federated. BURNIN.md is the runbook. deploy/build.sh stamps both binaries from one commit. go build, go vet and go test ./... pass, 20 packages. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
150 lines
6.5 KiB
Markdown
150 lines
6.5 KiB
Markdown
# Burn-in: live conformance, not more architecture
|
|
|
|
Written 2026-08-26. The workflow is feature-complete enough to exercise. What
|
|
remains is empirical: run the same real task shape through each harness and
|
|
classify what breaks. Isolated tests are insufficient by design here, and only
|
|
the live owner path establishes conformance.
|
|
|
|
Do not add workflow features while this is running. Findings decide the next
|
|
implementation work.
|
|
|
|
## Deployment state, probed 2026-08-26
|
|
|
|
Burn-in cannot start yet. Two blockers, one config gap.
|
|
|
|
| Fact | Evidence |
|
|
|---|---|
|
|
| The API is up on homesrv | `GET 192.168.1.104:9145/readyz` → `ready:true`, store/router/gitea/jsonl all ready |
|
|
| The workpc worker is up and serving two harnesses | `orchestra-worker` pid 741, restarted 2026-08-26 11:39, no errors since |
|
|
| **The deployed worker predates every v3 unit** | `go version -m /usr/local/bin/orchestra-worker` → `vcs.revision=97a9c65`, `vcs.time=2026-07-30` |
|
|
| **The v3 work is uncommitted** | Both sessions' units are working-tree changes on `webui-and-audit-reconciliation` |
|
|
| Codex has no harness entry at all | `/etc/orchestra/harnesses.json` declares only `workpc-claude` (tmux) and `workpc-opencode` (herdr) |
|
|
| No homesrv worker is running | Only the API container runs there; the three `homesrv-*` herdrs have no worker |
|
|
|
|
Consequences:
|
|
|
|
1. **Commit, then rebuild and redeploy both artifacts.** The API container
|
|
(`docker compose -f compose.yaml -f compose.live.yaml up -d --build` in
|
|
`/mnt/server/home/kami/docker-apps/orchestra-web-ui`) and the worker binary
|
|
(staged at `workpc:~/orchestra-deploy/orchestra-worker`, then installed to
|
|
`/usr/local/bin`, then `systemctl restart orchestra-worker`). Confirm the API
|
|
revision at `GET /v1/admin/diagnostics` and the worker's at
|
|
`go version -m`. Neither currently has phases, review, submission, the agent
|
|
surface, or the reconcile escalation.
|
|
2. **Codex cannot be burned in until it has a harnesses.json entry.** Three of
|
|
four target harnesses are unreachable today: codex is unconfigured, and both
|
|
`homesrv-claude`/`homesrv-codex`/`homesrv-opencode` have no worker.
|
|
3. The registry is misleading about herdr transports. No entry in the live
|
|
`config.jsonc` sets `backend` or `address`, so all six resolve to
|
|
`<machine>:9245` (`registry.defaultHerdrPort`), and both ports are closed.
|
|
The workpc worker actually uses a tmux socket and a unix socket
|
|
(`/home/kami/.config/herdr/herdr.sock`). Federated reachability defers to
|
|
worker heartbeat, so this does not block leasing, but a TCP probe of 9245 is
|
|
not evidence about any of these harnesses. Do not diagnose from it.
|
|
|
|
## Evidence to record per run
|
|
|
|
One row per run. The launch instruction is now dumped to
|
|
`<worktree>/.orchestra/launch.md` (`herdr.LaunchContextFile`) at every launch,
|
|
local and federated, so the context is auditable without reading pane
|
|
scrollback.
|
|
|
|
```
|
|
harness (claude | codex | opencode)
|
|
task id
|
|
lease epoch (every epoch, if the task rotates)
|
|
session ids (pane ids per epoch)
|
|
human decision ids
|
|
launch context (sha256 of .orchestra/launch.md, per launch)
|
|
handoff ref
|
|
git sha (at every boundary: launch, each handoff, review, submit, merge)
|
|
review ref
|
|
submission ref
|
|
pr id
|
|
completion receipt
|
|
```
|
|
|
|
## The five flows
|
|
|
|
1. **Boring success.** `task → research → plan → implement → review → pr → merge`.
|
|
2. **Mid-session correction.** Agent is doing A, the human says B, the next
|
|
verified boundary delivers B, and A becomes history rather than a competing
|
|
instruction.
|
|
3. **Rotation.** Agent A hands off, agent B resumes. Switch harnesses across the
|
|
boundary if both are up.
|
|
4. **Human rejection.** Submit sha A, comment on the pull request, task reopens,
|
|
fix at sha B, a fresh review, the same pull request, merge.
|
|
5. **Failure path.** Human source goes down, the reconcile streak escalates, the
|
|
session hands off, the successor lease is refused, the source recovers, and a
|
|
corrected successor starts.
|
|
|
|
## What to inspect by hand
|
|
|
|
For every launch, read `.orchestra/launch.md` and ask only:
|
|
|
|
```
|
|
does this agent know
|
|
- what the task actually wants?
|
|
- what was most recently decided?
|
|
- what phase it is in?
|
|
- what earlier material is merely historical?
|
|
- what to do next?
|
|
```
|
|
|
|
Then compare that against what the model did.
|
|
|
|
## Classify before fixing
|
|
|
|
Every failure gets exactly one label before any code is written:
|
|
|
|
```
|
|
authority bug the wrong thing outranked the right thing
|
|
context-selection bug the renderer showed or hid the wrong material
|
|
lifecycle bug state, lease, phase or event handling is wrong
|
|
adapter/harness bug the pane, occupancy, boundary or prompt path is wrong
|
|
model-following failure the context was right and the model ignored it
|
|
operator-policy gap credentials, deployment or configuration
|
|
```
|
|
|
|
This matters because the cheap response to every failure is another prompt
|
|
rule, and prompt rules accumulated to compensate for lifecycle or adapter bugs
|
|
are how the enforcement boundary rots. A model-following failure is the only
|
|
class a prompt change should ever answer.
|
|
|
|
## Pane credential cleanup (operator, in parallel)
|
|
|
|
The agent pane inherits the worker's environment. On workpc the worker is
|
|
`orchestra-worker.service`, `User=kami`, `EnvironmentFile=/etc/orchestra/worker.env`
|
|
(mode 0600, root-owned, deliberately unreadable here). Remove from that file, and
|
|
from anything else the pane inherits:
|
|
|
|
```
|
|
gitea mutation token
|
|
vikunja mutation credentials
|
|
orchestra operator / system token
|
|
```
|
|
|
|
Retain only `ORCHESTRA_AGENT_TOKEN` and credentials a specific task genuinely
|
|
needs. If an agent needs a forge operation, it asks Orchestra to perform it.
|
|
|
|
Then verify from inside a real pane:
|
|
|
|
```bash
|
|
git push # must fail unless Orchestra supplied the auth
|
|
curl .../v1/tasks/<id>/submission # must be 403 for the agent surface
|
|
tea pr create # must have no usable mutation credential
|
|
```
|
|
|
|
Note what is *not* enforced by code: nothing in this repo scrubs the pane
|
|
environment, and a fake `git` earlier in `$PATH` is not enforcement because
|
|
`/usr/bin/git` bypasses it. Credential isolation is the enforcement.
|
|
Confinement of network, filesystem and destructive commands belongs to whatever
|
|
launches the process (herdr, a container, systemd, bubblewrap), not to
|
|
Orchestra, which supplies identity and policy inputs.
|
|
|
|
## Exit criterion
|
|
|
|
Roughly 10 to 20 real tasks, with the five flows covered on each harness that
|
|
is actually up. Then read the classified failures and decide the next
|
|
implementation unit from them.
|