orchestra/PLAN-SPEC-DESIGN.md

# Locked design: plan.md, durable plan state, verified progress

Written 2026-08-28 while run 5's completion tail is still open. **No
implementation until run 5 reaches a terminal outcome.** This file is a note,
not code, and is deliberately uncommitted.

Approved by the operator with three amendments folded in. Markdown plan with
structural validation, stable research finding ids, and progress that Orchestra
establishes rather than the implementer asserting.

The project-level verification allowlist was added on the second pass.

## Ground truth this design starts from

Verified in the tree at `2417a39`, not assumed.

| Concern | Where | Behaviour today |
|---|---|---|
| Research artifact | `internal/workphase/workphase.go` | `Findings{Claim,Evidence}`, 64 items, 500 chars, single line |
| Plan artifact | same | `Changes{Target,Intent}`, `Verification[]`, `Risks[]`, `DecisionsNeeded[]`, same bounds |
| Advertised research schema | `internal/agentctx/agentctx.go:167` | promises `findings[].id` and `findings[].confidence`; struct has neither, both dropped silently |
| Seal path | `cmd/orchestra-worker/main.go:1833` | worker decodes, refuses locally through `answerRefusedPhase` (F39), then `AdvancePhase` |
| Storage | `internal/store/store.go:961`, `cmd/orchestra/main.go:467` | content-addressed CAS, format-agnostic, 4 MiB body cap |
| Refs | `internal/store/store.go:282` | reducer sets `t.ResearchRef` / `t.PlanRef` |
| Implement launch | `internal/agentctx/agentctx.go:418` | `renderSealed` flattens every field through `collapse()` to one bullet line |
| Plan progress | none | no state exists; `workphase` doc points at `continuity.Handoff` |
| Rotation progress | `internal/continuity/continuity.go:113` | `Remaining[]`, `Action`, `Learned[]`, 200 chars per line, unlinked to any plan item |
| Phase graph | `internal/domain/workphase.go:35` | `implement → review` only; no route back |
| Quality gate execution | `cmd/orchestra-worker/main.go:955` | `exec.CommandContext(ctx, "sh", "-c", gateCommand)`, `Dir = worktree`, operator-authored string |

The last row is the security baseline. The gate command is operator-authored,
from the project registry or the task. A plan-authored command is agent-authored,
so it must not inherit that envelope unchanged.

## 1. Sealed specification: `plan.md`

- Markdown, sealed through the existing path, referenced by the existing
  `t.PlanRef`. No new storage, no new event type for sealing.
- Bound: 128 KiB total. No per-line cap. The 500-character single-line rule is
  what makes a specification impossible today.
- `phaseSealFile[plan]` and `phaseArtifact[plan]` become `plan.md`.
- `DecodePlan` keeps the JSON path for refs already in the CAS. New seals must be
  markdown.
- `phaseSealSchema[plan]` states the required outline. Run 5 proved the planner
  follows that block (F38), so it is the delivery mechanism for the structure.

Required sections, validated by presence and shape:

```
# <name> implementation plan
## Overview
## Current state
## Desired end state
## Non-goals
## Approach
## Phase <n>: <name>          (one or more)
### Files
### Changes
### Verification
#### Automated
- run: ["go", "test", "./internal/foo/..."]
#### Manual
- <human-testable step>
## Testing strategy
## Risks and edge cases
## Migration
## References
```

Validation rules:

- At least one phase block. Phases numbered from 1, contiguous.
- Every phase carries `### Files`, `### Changes`, `### Verification`.
- Every phase carries at least one entry under `#### Automated` or
  `#### Manual`. A phase with neither cannot be verified and is refused.
- Automated entries are argv arrays, JSON-parsed from the `run:` line. Not
  shell strings. See section 4.
- `## References` entries cite research ids. An id that does not resolve against
  the sealed research fails the seal.

## 2. Research ids

- `Finding{ID, Claim, Evidence, Confidence}`.
- `ID` required, unique within the artifact, matching `^r[0-9]+$` or a stable
  slug.
- `Confidence` limited to `fact|inference|assumption`.
- The advertised schema at `agentctx.go:167` becomes true rather than
  aspirational.
- `renderSealed` prints the id with each finding, so a plan can cite
  `research:r12` and a reader can resolve it.
- Plan seal validation resolves every reference. The coordinator holds
  `ResearchRef`, so the cross-check costs one CAS read.

## 3. Progress: request, then Orchestra verifies

The implementer may write exactly one status.

```json
{ "phase": "phase-2", "status": "ready_for_verification" }
```

- Any other status value is refused, and the refusal is delivered to the live
  session through the F39 path. `verified` is not writable by an agent.
- Worker reads that phase's `#### Automated` entries from the accepted plan, not
  from the request.
- Worker runs each argv, in order, in the worktree, capturing exit code and
  output.
- All pass, manual steps exist: phase becomes `awaiting_manual_verification`.
- All pass, no manual steps: phase becomes `verified`.
- Any fail: phase stays `in_progress`, and the failing command plus its output
  reaches the implementer.

Durable record:

- New event `PlanPhaseVerified{phase, commands, exit_codes, at_sha,
  evidence_ref}`.
- Reduced into a task field `PlanProgress`, so state is derived from events
  rather than from a worker cache.
- Manual sign-off arrives as a human decision, on the surface decisions already
  use. No new approval mechanism.

Rendering, in `agentctx`:

- Plan state sits above continuity and below human decisions.
- It names verified phases, the current phase, and any awaiting-manual gate.
- The plan document itself renders verbatim. `collapse()` stays for research
  bullets only.

## 4. Verification-command allowlist

The rule, operator's wording:

```
accepted plan command
  ∩ existing quality-gate execution envelope
  ∩ project verification policy
= runnable
```

Properties this must hold:

- No weaker sandbox than the existing quality gate.
- No implicit `sh -c`. Commands execute as argv through `exec.CommandContext`,
  with `Dir` set to the worktree, exactly as the gate sets it.
- Shell operators, pipes, and redirections are unavailable by construction,
  since there is no shell. A project that genuinely needs one declares it in
  policy, explicitly.
- Refusal is observable and delivered to the implementer.
- A rejected verification never becomes `verified`.
- Project policy, not the plan, determines executable reach.

Policy shape, on `registry.Project`:

```json
"verification": {
  "allowed": [
    ["go", "test", "./..."],
    ["go", "test", "./internal/..."],
    ["go", "vet", "./..."],
    ["npm", "test", "--", "*"]
  ]
}
```

Matcher, argv-aware:

- Match is positional. `argv[0]` must match exactly.
- A literal pattern element matches that element exactly.
- `*` matches exactly one element, any value.
- A pattern element ending in `/...` matches a path argument under that prefix.
- A pattern shorter than the command does not match unless its last element is
  `*`. No prefix-match by accident.
- An empty or absent policy means no plan command is runnable. Absence is a
  refusal, not a default-allow.

Refusal text names the command and the project, so the planner learns its real
reach. A plan whose commands are all refused fails at seal time, not at
verification time. The failure lands on the planner rather than the
implementer.

## 5. Mismatch as a typed request

```json
{
  "plan_phase": "phase-2",
  "observed": "...",
  "contradicts": "...",
  "evidence": ["..."],
  "requested_action": "replan | research | human_decision"
}
```

- Written to `.orchestra/plan-mismatch.json`. Bounded lines, same validator
  style as the handoff.
- Worker validates and posts. Coordinator records `PlanMismatchRecorded`, then
  decides the move.
- Orchestra performs the reopen. The agent never asks for a backward phase, so
  `phase-request.json` still refuses one.
- The superseded plan moves to a history list on the task. `PlanRef` always
  names the currently accepted plan, so no ref is ever ambiguous.
- A fresh implement session launches against the newly sealed plan. The mismatch
  is retained as provenance.

This keeps the phase machine from becoming freely reversible navigation. The
edge exists for Orchestra, not for the agent.

## 6. Tests

Unit:

- Plan with every required section validates. One missing section fails, and the
  error names the section.
- Phase with no automated and no manual verification fails.
- `run:` line that is not a JSON argv array fails.
- Reference to an unknown research id fails, with the id named.
- Duplicate research finding id fails. Confidence outside the enum fails.
- `renderSealed` contains the plan document byte-for-byte, fences and multi-line
  blocks included.
- Two `Build` calls from the same refs with different handoffs both contain the
  complete plan.
- `status: "verified"` written by an agent is refused, and the refusal reaches
  the delivery path.
- Allowlist matcher: exact match, `*` single-element, `/...` path prefix,
  shorter-pattern rejection, empty policy rejects everything.
- A failed verification leaves the phase unverified and records the exit code.

## 7. Live proof ladder

1. Plan seals three detailed phases. Implement launch contains the plan
   verbatim.
2. Phase 1 requests verification. Worker runs the exact plan command. Durable
   state says phase 1 verified.
3. Rotation. Successor receives the complete original plan, phase 1 verified,
   phase 2 explicitly current.
4. Human correction lands above the plan. Plan and state stay subordinate.
5. Phase 2 hits a contradiction. Mismatch recorded, no silent implementation,
   old plan retained, revised plan sealed, fresh launch carries the revised plan
   in full.

Step 1 and step 3 are the two that matter for smaller local models. A grep of
the rendered `launch.md` either finds the phase-three specification and its
verification command, or it does not.

## Not in scope

Lifecycle, leases, review, submission, auth, federation, and the burn-in fixes
stay untouched. One adjacent change is needed: the Orchestra-only reopen edge in
section 5. It is reached through a typed request, not through a new
agent-askable transition.