Design the debt ledger from what the event log already carries
Nine questions answered against the tree at 3c7cf95, no implementation.
The ledger is two event types, one projection beside Task, one pure
eligibility function and one read-only endpoint. Nothing in the task
lifecycle changes.
The first slice writes nothing: a projection over the existing log that
must reproduce the release loop, the opencode adapter gap and the
retry-idleness dynamic from runs 10 to 14. A model that cannot represent
debt already known is wrong before any schema is committed to.
Also corrects run 14: the OpenCode Zen free tier is not blocked, the
selected model is.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CVbaKucEYBjMqVeUgJUsc1
This commit is contained in:
@@ -2479,7 +2479,8 @@ stops being latency and becomes model compatibility.
|
||||
### opencode is not running, and the reason was only in the pane
|
||||
|
||||
Task `06G4M6HH5BAM235AC4PWYXF1HM` failed at attempt 3 having never left
|
||||
`frame`. Its worktree held only `launch.md`. The pane said why:
|
||||
`frame`. Its worktree held only `launch.md`. The pane said why. The blocked
|
||||
thing is the selected model, not the OpenCode Zen free tier, which works:
|
||||
|
||||
```text
|
||||
This model is not available in your country.
|
||||
@@ -2499,7 +2500,9 @@ fail. Two things are worth keeping.
|
||||
not Orchestra's job; surfacing the capture is.
|
||||
|
||||
**Do not route to `workpc-opencode` until its model is reachable.** Every task
|
||||
sent there burns three attempts and then fails.
|
||||
sent there burns three attempts and then fails. `~/.config/opencode/opencode.jsonc`
|
||||
sets no top-level `model`, so OpenCode picks its own default. The file already
|
||||
declares reachable `nvidia-nim` and `llama-cpp` providers.
|
||||
|
||||
### The retry budget counts idleness, not just failure
|
||||
|
||||
|
||||
+301
@@ -0,0 +1,301 @@
|
||||
# Debt ledger: design from what the tree already supports
|
||||
|
||||
Written 2026-08-29 against `3c7cf95`. Read with `BURNIN.md` (the run ledger),
|
||||
`PLAN-SPEC-DESIGN.md` and `AUDIT.md`.
|
||||
|
||||
This is a design, not an implementation. Nothing here has been built.
|
||||
|
||||
## The problem being solved
|
||||
|
||||
AI raises implementation throughput faster than it raises integration, cleanup,
|
||||
observability, consolidation and architectural understanding. The residue is
|
||||
fixes, compatibility paths, operational workarounds, duplicate config, adapter
|
||||
gaps and one-off patches. Some are justified. Some become permanent because
|
||||
nobody remembers why they were added or what they have cost since.
|
||||
|
||||
The target is a first-class debt ledger driven by evidence from real tasks, not
|
||||
a second TODO list.
|
||||
|
||||
## Classes
|
||||
|
||||
Four, and no more. A fifth class invented at runtime makes the priority model
|
||||
meaningless.
|
||||
|
||||
| Class | Meaning |
|
||||
|---|---|
|
||||
| correctness | Known behaviour is wrong or violates an invariant. |
|
||||
| operational | The system works, but diagnosis, recovery, deployment, observability or operation repeatedly costs time. |
|
||||
| structural | Duplication or architecture is demonstrably raising the cost of future changes. |
|
||||
| polish | Cleanup or consistency work with no demonstrated cost yet. |
|
||||
|
||||
Correctness and operational debt gain priority quickly. Structural debt needs
|
||||
evidence that it is causing repeated work. Polish never becomes work on its own.
|
||||
|
||||
## 1. What Orchestra already has
|
||||
|
||||
The event log is the durable spine, and it already carries most of what a debt
|
||||
ledger needs. Every row below is mechanically countable today, with no new
|
||||
instrumentation.
|
||||
|
||||
| Evidence | Source | Carries |
|
||||
|---|---|---|
|
||||
| Failure shape | `TaskFailed`, `TaskBlocked` | 12 typed `block_reason` values, 6 `failure_class` values |
|
||||
| Retry cost | `Task.Attempt`, `NextRetryAt`, `FailureClass` | how many leases a task burned, durable in the projection |
|
||||
| Review findings | `ReviewRecorded` | `{ID, Severity, File, Line, Claim, Evidence}`, bound to a `ResultSHA` |
|
||||
| Out-of-scope discoveries | `DeferredFindingRecorded` | `{Summary, Why}`, already recorded outside agent context |
|
||||
| Plan contradicted by code | `PlanMismatchRecorded` | `{PlanRef, PhaseID, AtSHA, Observed, Contradicts, Evidence[]}` |
|
||||
| Verification history | `PlanPhaseVerified` | `{Commands, ExitCodes, AtSHA, EvidenceRef}` per phase |
|
||||
| Rework rounds | `TaskChangesRequested`, `TaskSubmitted` | how many times a change went back |
|
||||
| Provenance | `TaskCreated` | `Source`, `ExternalID`, `Parent` |
|
||||
|
||||
Three existing patterns matter more than the data.
|
||||
|
||||
**Global-subject events already work.** `QuotaReported` and `StandupAdvisory`
|
||||
use `TaskID: "system"` and are whitelisted in `internal/domain/domain.go:279`.
|
||||
A debt item needs no new subject mechanism.
|
||||
|
||||
**`GenerateStandupAdvisory` is the precedent for the shape.** It is a
|
||||
persisted, read-only recommendation, and applying it is a separate
|
||||
approval-gated operation. A debt ledger is that pattern with better inputs.
|
||||
|
||||
**`CheckSubmission` is the precedent for eligibility.** It is a pure function
|
||||
of task, commit and gate run, returning `{Eligible bool, Reasons []string}`.
|
||||
Debt eligibility should have the same shape.
|
||||
|
||||
## 2. What evidence is missing
|
||||
|
||||
**Worker observations are not durable.** This is the largest gap. The F18 ring
|
||||
lives in worker memory and reaches the coordinator inside `WorkerHealth` on
|
||||
heartbeat. `Registry.persistedState` holds captures, commands and workers only,
|
||||
with no health. A coordinator restart erases every observation.
|
||||
|
||||
Operational debt is exactly what that ring holds. Repeated 409s, unrenewed
|
||||
leases, adapter gaps.
|
||||
|
||||
**No operator-intervention record.** Every manual repair in run 12 and run 13
|
||||
left no trace in Orchestra. State-file edits, worker restarts, two manual
|
||||
transaction cleanups. "Required manual recovery" is the strongest priority
|
||||
signal available, and it is currently unrecorded.
|
||||
|
||||
**No diagnosis-cost signal.** Time spent diagnosing is not measured. The
|
||||
closest proxy is wall time in `blocked` or `needs_attention`.
|
||||
|
||||
**No component dimension.** Findings carry file paths. Tasks carry none.
|
||||
Breadth across components has to be derived from paths.
|
||||
|
||||
**No link from a repair commit back to what it repaired.** The Fxx-to-commit
|
||||
mapping exists only in `BURNIN.md` prose.
|
||||
|
||||
**Deferred findings are too thin.** `{Summary, Why}` has no class, no severity,
|
||||
no paths and no evidence refs.
|
||||
|
||||
## 3. Minimal durable data model
|
||||
|
||||
No new subsystem. Two event types on the existing spine, one projection beside
|
||||
`Task`.
|
||||
|
||||
```go
|
||||
// DebtObservation is one piece of counted evidence, always pointing at the
|
||||
// event that produced it. Provenance is an event id, never prose.
|
||||
type DebtObservation struct {
|
||||
EventID string `json:"event_id"`
|
||||
TaskID string `json:"task_id,omitempty"`
|
||||
Kind string `json:"kind"` // block_reason, failure_class, review_finding,
|
||||
// plan_mismatch, manual_intervention, deferred_finding
|
||||
Signature string `json:"signature"` // the mechanical dedup key, see section 4
|
||||
Detail string `json:"detail"`
|
||||
Paths []string `json:"paths,omitempty"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
type DebtItem struct {
|
||||
ID string `json:"id"` // continues the Fxx namespace
|
||||
Class DebtClass `json:"class"` // correctness, operational, structural, polish
|
||||
Status DebtStatus `json:"status"` // observed, eligible, scheduled, repaired, withdrawn
|
||||
Symptom string `json:"symptom"`
|
||||
Consequence string `json:"consequence"`
|
||||
Severity Severity `json:"severity"` // reuse review.Severity
|
||||
Paths []string `json:"paths,omitempty"`
|
||||
Signatures []string `json:"signatures"` // every key that attaches here
|
||||
Observations []DebtObservation `json:"observations"`
|
||||
IntroducedIn string `json:"introduced_in,omitempty"` // task id or commit
|
||||
RepairBoundary string `json:"repair_boundary,omitempty"`
|
||||
RepairTask string `json:"repair_task,omitempty"`
|
||||
RepairCommit string `json:"repair_commit,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
Every counted field the priority model needs is derived, never stored:
|
||||
|
||||
```text
|
||||
recurrence len(Observations)
|
||||
blocked_tasks distinct TaskID where Kind is block_reason
|
||||
manual_interventions count of Kind == manual_intervention
|
||||
breadth distinct components derived from Paths
|
||||
```
|
||||
|
||||
A stored count invites drift from the log. A derived one cannot drift.
|
||||
|
||||
Two events carry it. `DebtObserved` appends one observation. `DebtItemUpdated`
|
||||
records class, status, severity or a merge. Both need adding to the `allowed`
|
||||
map and a validator, which is the work every existing event type already did.
|
||||
|
||||
## 4. Deduplication without corrupting provenance
|
||||
|
||||
**A signature is computed from typed facts, never from prose.**
|
||||
|
||||
```text
|
||||
operational class + block_reason + harness + normalized component
|
||||
correctness class + failure_class + normalized component
|
||||
structural class + finding severity + normalized component
|
||||
```
|
||||
|
||||
Exact signature match attaches automatically. That is the only automatic path.
|
||||
|
||||
**A model may propose, never merge.** Clustering emits `DebtMergeSuggested`
|
||||
carrying both item ids and its reasoning. The merge is an operator action or a
|
||||
policy threshold, recorded as `DebtItemUpdated`.
|
||||
|
||||
**Merges are additive.** The surviving item gains the other's signatures and
|
||||
observations. The absorbed item becomes `withdrawn` with a pointer, and its
|
||||
observations keep their original event ids. Nothing is rewritten, so a bad
|
||||
merge is reversible by reading the log.
|
||||
|
||||
**Every observation names its event.** A debt item's evidence is always
|
||||
checkable against the log that produced it. That rule is what stops a fuzzy
|
||||
proposal from becoming an unverifiable claim.
|
||||
|
||||
## 5. Eligibility policy
|
||||
|
||||
A pure function, mirroring `CheckSubmission`.
|
||||
|
||||
```go
|
||||
func CheckDebtEligibility(item DebtItem) DebtCheck // {Eligible bool, Reasons []string}
|
||||
```
|
||||
|
||||
| Class | Becomes eligible when |
|
||||
|---|---|
|
||||
| correctness | first confirmed observation, always |
|
||||
| operational | recurrence >= 3 across >= 2 distinct tasks, or >= 1 manual intervention |
|
||||
| structural | >= 3 review findings, or >= 2 tasks blocked or reworked in the same component |
|
||||
| polish | never automatically, operator promotion only |
|
||||
|
||||
`Reasons` lists what fired and what did not. "Not eligible" alone sends an
|
||||
operator reading code, which is the mistake `SubmissionCheck` already documents.
|
||||
|
||||
Eligible means a task may be created. It does not create one.
|
||||
|
||||
## 6. How maintenance and consolidation fit the existing lifecycle
|
||||
|
||||
A maintenance task is an ordinary task. Source `debt`, external id the debt item
|
||||
id, phases as usual.
|
||||
|
||||
```text
|
||||
research → plan → implement → verification → review → submission
|
||||
```
|
||||
|
||||
Nothing in the lifecycle changes. One extension is needed: the task's brief must
|
||||
carry the debt item's evidence. That is the mechanism research and plan
|
||||
artifacts already use, a CAS ref on the task.
|
||||
|
||||
A consolidation task is the same thing with a research brief that asks:
|
||||
|
||||
- which temporary paths are still necessary?
|
||||
- which fixes now duplicate each other?
|
||||
- which compatibility branches are obsolete?
|
||||
- which config or state is represented twice?
|
||||
- which abstractions exist only because of defects that have since disappeared?
|
||||
- what can now be deleted safely?
|
||||
|
||||
Its first output is a deletion plan, never an automatic refactor. Verification
|
||||
uses the project's existing policy. Review is the normal independent review.
|
||||
|
||||
**The repair gets no discount.** Classification as debt changes what gets
|
||||
scheduled, never what gets checked.
|
||||
|
||||
### Success metrics for maintenance work
|
||||
|
||||
Lines of code are not a metric. Prefer evidence that can be checked:
|
||||
|
||||
- behaviour preserved
|
||||
- tests preserved or strengthened
|
||||
- branches removed
|
||||
- obsolete types removed
|
||||
- compatibility code removed
|
||||
- duplicate config removed
|
||||
- manual recovery paths eliminated
|
||||
|
||||
## 7. What must not be automated
|
||||
|
||||
- No autonomous cleanup agent, and no repo-wide sweep.
|
||||
- No LLM score treated as authoritative. Counted facts decide, models propose.
|
||||
- No automatic merge of debt items.
|
||||
- No maintenance task from polish without an operator.
|
||||
- No deletion without an independent review.
|
||||
- No auto-closing an item because a commit touched the file. Closure needs a
|
||||
repair task or a stated run of clean evidence.
|
||||
- No new classes invented at runtime.
|
||||
|
||||
## 8. Migration, so there is one track and not two
|
||||
|
||||
`BURNIN.md` holds 58 Fxx entries in the form `| F52 | commit | prose |`.
|
||||
`AUDIT.md` holds the older narrative.
|
||||
|
||||
One-time import, by hand, with a class assigned per row. Each becomes a
|
||||
`DebtItem` with `Status: repaired`, its commit as `RepairCommit`, and one
|
||||
observation citing the ledger.
|
||||
|
||||
**Keep the namespace.** New items continue at F62. Two numbering schemes would
|
||||
be the first structural debt the ledger itself creates.
|
||||
|
||||
After import, `BURNIN.md` stops being the item ledger and remains the run
|
||||
narrative. The defect table becomes generated output from the projection.
|
||||
|
||||
### The history that tests the model
|
||||
|
||||
These are not special cases. They are the check on whether the model can
|
||||
represent the history this project already has.
|
||||
|
||||
| History | Expected class | Why |
|
||||
|---|---|---|
|
||||
| F18, the single `last_error` slot | operational | Repeatedly destroyed causal evidence before it was fixed. |
|
||||
| F43 to F46, the submission path | correctness | Apparently implemented, actually unreachable. |
|
||||
| F61, planner learns policy by refusal | operational | Costs a round trip per plan, and becomes model compatibility on weaker planners. |
|
||||
| Duplicated `quality_gate` config | structural | Only if it keeps causing drift or operator mistakes. |
|
||||
| Old blocked burn-in tasks | neither | Hygiene, unless one exposes a runtime defect. |
|
||||
| The blocked release loop, F57 to F60 | correctness and operational | Wrong behaviour, and it required manual cleanup twice. |
|
||||
|
||||
## 9. Smallest slice to live-prove first
|
||||
|
||||
**A read-only projection over the existing log. No new events, no writes, no
|
||||
schema commitment.**
|
||||
|
||||
```text
|
||||
internal/operations/debt.go DebtItem, signature, projection over s.Events(0)
|
||||
CheckDebtEligibility
|
||||
GET /v1/debt read-only, tui surface
|
||||
```
|
||||
|
||||
The proof is that the current log already contains runs 10 to 14. The projection
|
||||
must independently surface, from history alone:
|
||||
|
||||
- the release loop as operational debt, with its 409 recurrence and two manual
|
||||
cleanups
|
||||
- the opencode adapter gap as operational debt, across three failed attempts on
|
||||
one harness
|
||||
- the retry-idleness dynamic, attached to the tasks that expired
|
||||
|
||||
A model that cannot reproduce debt already known is wrong, and no schema has
|
||||
been committed to yet. That is the cheapest place to find out.
|
||||
|
||||
**Slice two** makes worker observations durable, because that is the input the
|
||||
projection will most visibly lack.
|
||||
|
||||
**Slice three** adds `DebtObserved` and manual-intervention recording.
|
||||
|
||||
## Scope boundary
|
||||
|
||||
This design changes no part of the task lifecycle, phases, leases, review,
|
||||
submission, federation or the plan machinery. The only extensions it needs are
|
||||
two event types, one projection, one pure eligibility function and one
|
||||
read-only endpoint.
|
||||
Reference in New Issue
Block a user