orchestra/DEBT-DESIGN.md

# Debt ledger: design from what the tree already supports

Written 2026-08-29 against `3c7cf95`. Read with `BURNIN.md` (the run ledger),
`PLAN-SPEC-DESIGN.md` and `AUDIT.md`.

This is a design, not an implementation. Nothing here has been built.

## The problem being solved

AI raises implementation throughput faster than it raises integration, cleanup,
observability, consolidation and architectural understanding. The residue is
fixes, compatibility paths, operational workarounds, duplicate config, adapter
gaps and one-off patches. Some are justified. Some become permanent because
nobody remembers why they were added or what they have cost since.

The target is a first-class debt ledger driven by evidence from real tasks, not
a second TODO list.

## Classes

Four, and no more. A fifth class invented at runtime makes the priority model
meaningless.

| Class | Meaning |
|---|---|
| correctness | Known behaviour is wrong or violates an invariant. |
| operational | The system works, but diagnosis, recovery, deployment, observability or operation repeatedly costs time. |
| structural | Duplication or architecture is demonstrably raising the cost of future changes. |
| polish | Cleanup or consistency work with no demonstrated cost yet. |

Correctness and operational debt gain priority quickly. Structural debt needs
evidence that it is causing repeated work. Polish never becomes work on its own.

## 1. What Orchestra already has

The event log is the durable spine, and it already carries most of what a debt
ledger needs. Every row below is mechanically countable today, with no new
instrumentation.

| Evidence | Source | Carries |
|---|---|---|
| Failure shape | `TaskFailed`, `TaskBlocked` | 12 typed `block_reason` values, 6 `failure_class` values |
| Retry cost | `Task.Attempt`, `NextRetryAt`, `FailureClass` | how many leases a task burned, durable in the projection |
| Review findings | `ReviewRecorded` | `{ID, Severity, File, Line, Claim, Evidence}`, bound to a `ResultSHA` |
| Out-of-scope discoveries | `DeferredFindingRecorded` | `{Summary, Why}`, already recorded outside agent context |
| Plan contradicted by code | `PlanMismatchRecorded` | `{PlanRef, PhaseID, AtSHA, Observed, Contradicts, Evidence[]}` |
| Verification history | `PlanPhaseVerified` | `{Commands, ExitCodes, AtSHA, EvidenceRef}` per phase |
| Rework rounds | `TaskChangesRequested`, `TaskSubmitted` | how many times a change went back |
| Provenance | `TaskCreated` | `Source`, `ExternalID`, `Parent` |

Three existing patterns matter more than the data.

**Global-subject events already work.** `QuotaReported` and `StandupAdvisory`
use `TaskID: "system"` and are whitelisted in `internal/domain/domain.go:279`.
A debt item needs no new subject mechanism.

**`GenerateStandupAdvisory` is the precedent for the shape.** It is a
persisted, read-only recommendation, and applying it is a separate
approval-gated operation. A debt ledger is that pattern with better inputs.

**`CheckSubmission` is the precedent for eligibility.** It is a pure function
of task, commit and gate run, returning `{Eligible bool, Reasons []string}`.
Debt eligibility should have the same shape.

## 2. What evidence is missing

**Worker observations are not durable.** This is the largest gap. The F18 ring
lives in worker memory and reaches the coordinator inside `WorkerHealth` on
heartbeat. `Registry.persistedState` holds captures, commands and workers only,
with no health. A coordinator restart erases every observation.

Operational debt is exactly what that ring holds. Repeated 409s, unrenewed
leases, adapter gaps.

**No operator-intervention record.** Every manual repair in run 12 and run 13
left no trace in Orchestra. State-file edits, worker restarts, two manual
transaction cleanups. "Required manual recovery" is the strongest priority
signal available, and it is currently unrecorded.

**No diagnosis-cost signal.** Time spent diagnosing is not measured. The
closest proxy is wall time in `blocked` or `needs_attention`.

**No component dimension.** Findings carry file paths. Tasks carry none.
Breadth across components has to be derived from paths.

**No link from a repair commit back to what it repaired.** The Fxx-to-commit
mapping exists only in `BURNIN.md` prose.

**Deferred findings are too thin.** `{Summary, Why}` has no class, no severity,
no paths and no evidence refs.

## 3. Minimal durable data model

No new subsystem. Two event types on the existing spine, one projection beside
`Task`.

```go
// DebtObservation is one piece of counted evidence, always pointing at the
// event that produced it. Provenance is an event id, never prose.
type DebtObservation struct {
    // Exactly one of EventID and LegacyRef. See section 8.
    EventID   string    `json:"event_id,omitempty"`
    LegacyRef string    `json:"legacy_ref,omitempty"`
    TaskID    string    `json:"task_id,omitempty"`
    Kind      string    `json:"kind"`      // block_reason, failure_class, review_finding,
                                           // plan_mismatch, manual_intervention, deferred_finding
    Signature string    `json:"signature"` // the mechanical dedup key, see section 4
    Detail    string    `json:"detail"`
    Paths     []string  `json:"paths,omitempty"`
    At        time.Time `json:"at"`
}

type DebtItem struct {
    ID             string            `json:"id"`     // continues the Fxx namespace
    Class          DebtClass         `json:"class"`  // correctness, operational, structural, polish
    Status         DebtStatus        `json:"status"` // observed, eligible, scheduled, repaired, withdrawn
    Symptom        string            `json:"symptom"`
    Consequence    string            `json:"consequence"`
    Severity       Severity          `json:"severity"` // reuse review.Severity
    Paths          []string          `json:"paths,omitempty"`
    Signatures     []string          `json:"signatures"` // every key that attaches here
    Observations   []DebtObservation `json:"observations"`
    IntroducedIn   string            `json:"introduced_in,omitempty"` // task id or commit
    RepairBoundary string            `json:"repair_boundary,omitempty"`
    RepairTask     string            `json:"repair_task,omitempty"`
    RepairCommit   string            `json:"repair_commit,omitempty"`
}
```

Every counted field the priority model needs is derived, never stored:

```text
recurrence            len(Observations)
blocked_tasks         distinct TaskID where Kind is block_reason
manual_interventions  count of Kind == manual_intervention
breadth               distinct components derived from Paths
```

A stored count invites drift from the log. A derived one cannot drift.

Two events carry it. `DebtObserved` appends one observation. `DebtItemUpdated`
records class, status, severity or a merge. Both need adding to the `allowed`
map and a validator, which is the work every existing event type already did.

## 4. Deduplication without corrupting provenance

**A signature is computed from typed facts, never from prose.**

```text
operational   class + block_reason + harness + normalized component
correctness   class + failure_class + normalized component
structural    class + finding severity + normalized component
```

Exact signature match attaches automatically. That is the only automatic path.

**Signatures are versioned, in the string itself.**

```text
v1:operational:lease_not_renewed:workpc-opencode:federation
```

Normalization rules will change. Without a version, changing them silently
regroups every historical observation, and the recurrence counts that drive
eligibility move underneath the items that already used them. A `v2` signature
never matches a `v1` one, so old evidence keeps the grouping it was counted
under.

**A model may propose, never merge.** Clustering emits `DebtMergeSuggested`
carrying both item ids and its reasoning. The merge is an operator action or a
policy threshold, recorded as `DebtItemUpdated`.

**Merges are additive.** The surviving item gains the other's signatures and
observations. The absorbed item becomes `withdrawn` with a pointer, and its
observations keep their original event ids. Nothing is rewritten, so a bad
merge is reversible by reading the log.

**Every observation names its event.** A debt item's evidence is always
checkable against the log that produced it. That rule is what stops a fuzzy
proposal from becoming an unverifiable claim.

## 5. Eligibility policy

A pure function, mirroring `CheckSubmission`.

```go
func CheckDebtEligibility(item DebtItem) DebtCheck // {Eligible bool, Reasons []string}
```

| Class | Becomes eligible when |
|---|---|
| correctness | first confirmed observation, always |
| operational | recurrence >= 3 across >= 2 distinct tasks, or >= 1 manual intervention |
| structural | >= 3 review findings, or >= 2 tasks blocked or reworked in the same component |
| polish | never automatically, operator promotion only |

`Reasons` lists what fired and what did not. "Not eligible" alone sends an
operator reading code, which is the mistake `SubmissionCheck` already documents.

Eligible means a task may be created. It does not create one.

## 6. How maintenance and consolidation fit the existing lifecycle

A maintenance task is an ordinary task. Source `debt`, external id the debt item
id, phases as usual.

```text
research → plan → implement → verification → review → submission
```

Nothing in the lifecycle changes. One extension is needed: the task's brief must
carry the debt item's evidence. That is the mechanism research and plan
artifacts already use, a CAS ref on the task.

A consolidation task is the same thing with a research brief that asks:

- which temporary paths are still necessary?
- which fixes now duplicate each other?
- which compatibility branches are obsolete?
- which config or state is represented twice?
- which abstractions exist only because of defects that have since disappeared?
- what can now be deleted safely?

Its first output is a deletion plan, never an automatic refactor. Verification
uses the project's existing policy. Review is the normal independent review.

**The repair gets no discount.** Classification as debt changes what gets
scheduled, never what gets checked.

### Success metrics for maintenance work

Lines of code are not a metric. Prefer evidence that can be checked:

- behaviour preserved
- tests preserved or strengthened
- branches removed
- obsolete types removed
- compatibility code removed
- duplicate config removed
- manual recovery paths eliminated

## 7. What must not be automated

- No autonomous cleanup agent, and no repo-wide sweep.
- No LLM score treated as authoritative. Counted facts decide, models propose.
- No automatic merge of debt items.
- No maintenance task from polish without an operator.
- No deletion without an independent review.
- No auto-closing an item because a commit touched the file. Closure needs a
  repair task or a stated run of clean evidence.
- No new classes invented at runtime.

## 8. Migration, so there is one track and not two

`BURNIN.md` holds 58 Fxx entries in the form `| F52 | commit | prose |`.
`AUDIT.md` holds the older narrative.

One-time import, by hand, with a class assigned per row. Each becomes a
`DebtItem` with `Status: repaired`, its commit as `RepairCommit`, and one
observation citing the ledger.

**Imported observations carry legacy provenance, never a fabricated event id.**
The Fxx history predates the capability that would have produced an event, and
inventing one would break the provenance rule the ledger exists to enforce.

```go
EventID   string `json:"event_id,omitempty"`
LegacyRef string `json:"legacy_ref,omitempty"` // "BURNIN.md:F18"
```

Exactly one is required. A reader can then tell a counted fact from an imported
claim at a glance, and the counts that drive eligibility can exclude imported
evidence if that turns out to matter.

**Keep the namespace.** New items continue at F62. Two numbering schemes would
be the first structural debt the ledger itself creates.

After import, `BURNIN.md` stops being the item ledger and remains the run
narrative. The defect table becomes generated output from the projection.

### The history that tests the model

These are not special cases. They are the check on whether the model can
represent the history this project already has.

| History | Expected class | Why |
|---|---|---|
| F18, the single `last_error` slot | operational | Repeatedly destroyed causal evidence before it was fixed. |
| F43 to F46, the submission path | correctness | Apparently implemented, actually unreachable. |
| F61, planner learns policy by refusal | operational | Costs a round trip per plan, and becomes model compatibility on weaker planners. |
| Duplicated `quality_gate` config | structural | Only if it keeps causing drift or operator mistakes. |
| Old blocked burn-in tasks | neither | Hygiene, unless one exposes a runtime defect. |
| The blocked release loop, F57 to F60 | correctness and operational | Wrong behaviour, and it required manual cleanup twice. |

## 9. Smallest slice to live-prove first

**A read-only projection over the existing log. No new events, no writes, no
schema commitment.**

```text
internal/domain/debt.go            DebtItem, DebtObservation, signature, classification
internal/store/debt_projection.go  the fold over s.Events(0), and the gap report
internal/operations/debt.go        CheckDebtEligibility, and later the actions
GET /v1/debt                       read-only, tui surface
```

The projected type stays out of `operations`. Baking a read model into the
command layer in the first slice is the mistake that would be hardest to undo
later. `operations` owns actions: `CheckDebtEligibility` now, and
`SuggestDebtMerge`, `PromoteDebtItem` and `ScheduleDebtRepair` when they exist.

The proof is what the projection can and cannot recover from canonical history.
Section 2 already says worker observations are not durable and operator
interventions are unrecorded, so demanding their reconstruction would be asking
the projection to invent evidence.

The first burn-in is therefore three requirements, not one:

1. Recover every debt signal that canonical history actually encodes.
2. Report, explicitly and per kind, where known debt cannot be reconstructed.
3. Never infer a missing observation from `BURNIN.md`.

Incompleteness is part of the result, not a failure. The 409 loop lived in the
F18 worker ring rather than in an event, so the ledger should say it has no
durable evidence for that shape. That statement is what makes slice two
necessary, measurably rather than by assertion.

What the projection should recover from the log alone:

- repeated `lease_expired` and `retry_limit` on one harness, which is the
  opencode failure shape across four tasks
- the retry-idleness dynamic, attached to the tasks that expired
- review findings grouped by component, if any repeat

A model that cannot represent debt already known is wrong, and no schema has
been committed to yet. That is the cheapest place to find out.

**Slice two** makes worker observations durable, because that is the input the
projection will most visibly lack.

**Slice three** adds `DebtObserved` and manual-intervention recording.

## Scope boundary

This design changes no part of the task lifecycle, phases, leases, review,
submission, federation or the plan machinery. The only extensions it needs are
two event types, one projection, one pure eligibility function and one
read-only endpoint.

---

# Slice one, run against real history

Built and deployed as `a757cff`. `GET /v1/debt` folded 881 events from the
live log, wrote nothing, and produced 15 candidate items and 3 gaps.

## What it recovered

```text
v1:operational:lease_expired:workpc-opencode:lease      r=41  tasks=4
v1:operational:lease_expired:workpc-claude:lease        r=29  tasks=13
v1:operational:lease_failure:-:lease                    r=20  tasks=16
v1:operational:system_error:-:lease                     r=11  tasks=7
v1:correctness:handoff_validation:-:lease               r=5   tasks=5
v1:correctness:plan_mismatch:-:...                      r=1   tasks=1
v1:structural:minor:-:scripts/orchestra_e2e_healthcheck.sh  r=1  tasks=1
```

The opencode failure shape is the top item, found mechanically. Run 14
diagnosed the same thing by hand from a pane capture. The retry-idleness
dynamic is the second item, and it is now a number: 29 expiries across 13
tasks on one harness.

Seven items are eligible under the stated policy. Polish and the single
structural finding correctly are not.

## What it reported that it could not see

```text
durable=false  manual_intervention  no event type records an operator repair
durable=false  worker_observation   worker health is not persisted
durable=true   deferred_finding     carried by the log, this history has none
```

The 409 release loop does not appear, and it should not. That evidence lived in
the F18 worker ring, which no event carries. The ledger says so rather than
inferring it, which is the result slice two exists to change.

## Four defects the first run exposed

**The component part is too coarse for lease evidence.** Everything
lease-related normalizes to `lease`, so `lease_expired:workpc-claude` is one
bucket holding idle agents, rig interference and real failures. Recurrence 29
is true and the item is not a defect.

**`harness` is often empty on block reasons.** `TaskBlocked` payloads do not
always carry `harness_id`, so `lease_failure:-` mixes harnesses that should be
separate items.

**Path normalization mangled a mismatch reference.**
`scripts/orchestra_e2e_healthcheck.sh:12` became
`scripts/orchestra_e2e_healthcheck.sh_12`, because the signature sanitizer
replaces `:` and the mismatch evidence field holds prose, not clean paths.

**Recurrence alone is the wrong sort.** 41 occurrences on 4 tasks currently
outranks 29 on 13 tasks. Breadth is in the design and not yet in the ordering,
which is the priority function slice one deliberately omitted.

None of these required a schema commitment to discover. That was the point of
making the first slice read-only.

## The invariant slice B was written against

**A projection must never manufacture provenance to make evidence easier to
classify.** `task=None` is better than a confident lie.

Both defects the first live run of durable observations exposed were failures
of exactly this rule, and both looked perfectly reasonable in code:

- **False attribution.** The worker's ring outlives the work it describes, so
  binding its entries to whatever task the worker is running now produced a
  clean, well-formed, wrong association between an old failure and an unrelated
  task. The task is read from the observation itself, and only an observation
  that names no task belongs to the current lease.
- **Manufactured recurrence.** Treating "still present in the ring" as "it
  happened again" turned one failure into four incidents. Presence is not
  occurrence: an incident opens only when the entry advances past what has
  already been accounted for, and that high-water mark survives the close.

Either one would have corrupted the ledger while every number in it stayed
plausible, which is the specific way this repo's bugs have always presented.

## Signatures are frozen at write time

The signature is computed when an incident is recorded and stored in the event.
Re-projecting an old log with a newer normalizer therefore changes nothing:
running one history through the build before and after the pane-name fix
produced identical output.

That is the intended behavior. An event must not silently change meaning
because normalization code changed. Repairing historical signatures is an
explicit migration or reclassification event, never a different projection
result from the same log.

## What counts as a manual intervention

An intervention is an operator action required to recover, repair, unblock or
correct behaviour that should otherwise have proceeded autonomously. The
ledger measures what the system costs to keep running, so routine operation
does not belong in it.

Counts:

```text
transaction_cleanup
forced_release
state_repair
manual_requeue          when recovery failed and a human had to requeue
manual_phase_recovery
worker_restart          only when restarting is itself the repair
```

Does not count:

```text
deploy restart
planned upgrade
configuration rollout
normal shutdown and start
deliberate burn-in setup
```

The distinction is policy rather than schema. `worker_restart` is the one kind
that spans both sides, and it stays a single kind until someone actually
misuses it; a field added before the confusion exists is a guess about how it
will be misread.

The first live consequence: the deploy restart of `79d2053` was not recorded,
and `manual_intervention` stayed in the gap list afterwards. That is the
correct result. Orchestra can record the evidence and this history contains
none, which is an honest gap rather than synthetic evidence.

## Slice B status, 2026-08-30

```text
worker observation durability         proven live
incident open/close semantics         proven live
high-water, no phantom recurrence     proven live
task attribution                      proven live
frozen historical provenance          proven live

eligibility calculation               proven against real data
eligibility transition                unexercised
manual intervention recording         implemented
manual intervention provenance        unexercised
```

Neither unexercised branch will be manufactured. A second task hitting the
same failure exercises the first, and a genuine repair exercises the second.

The proof that matters from the live run:

```text
recurrence 5, intensity 10, distinct tasks 1, interventions 0  ->  eligible=false
```

A noisy failure inside one task does not become system-level maintenance debt.
The three counts stay separate on purpose: recurrence is how often the incident
happened, intensity is how repetitive each incident became, and breadth is how
many independent tasks paid for it. Operational debt requires breadth unless an
operator had to intervene.

### The reasons check is part of the design, not a formality

The aggregate numbers moved in a way that looked like a promotion, and reading
them that way was wrong: the eligible count rose because of an unrelated item.
The pure eligibility explanation contradicted that inference with counted
facts. That is the argument for the ledger resting on mechanically derived
evidence rather than on anyone's reading of what looks recurring, including an
agent's.