Reconcile docs with reality; fix module graph, token compare, health #1
@@ -2479,7 +2479,8 @@ stops being latency and becomes model compatibility.
|
||||
### opencode is not running, and the reason was only in the pane
|
||||
|
||||
Task `06G4M6HH5BAM235AC4PWYXF1HM` failed at attempt 3 having never left
|
||||
`frame`. Its worktree held only `launch.md`. The pane said why:
|
||||
`frame`. Its worktree held only `launch.md`. The pane said why. The blocked
|
||||
thing is the selected model, not the OpenCode Zen free tier, which works:
|
||||
|
||||
```text
|
||||
This model is not available in your country.
|
||||
@@ -2499,7 +2500,9 @@ fail. Two things are worth keeping.
|
||||
not Orchestra's job; surfacing the capture is.
|
||||
|
||||
**Do not route to `workpc-opencode` until its model is reachable.** Every task
|
||||
sent there burns three attempts and then fails.
|
||||
sent there burns three attempts and then fails. `~/.config/opencode/opencode.jsonc`
|
||||
sets no top-level `model`, so OpenCode picks its own default. The file already
|
||||
declares reachable `nvidia-nim` and `llama-cpp` providers.
|
||||
|
||||
### The retry budget counts idleness, not just failure
|
||||
|
||||
|
||||
+301
@@ -0,0 +1,301 @@
|
||||
# Debt ledger: design from what the tree already supports
|
||||
|
||||
Written 2026-08-29 against `3c7cf95`. Read with `BURNIN.md` (the run ledger),
|
||||
`PLAN-SPEC-DESIGN.md` and `AUDIT.md`.
|
||||
|
||||
This is a design, not an implementation. Nothing here has been built.
|
||||
|
||||
## The problem being solved
|
||||
|
||||
AI raises implementation throughput faster than it raises integration, cleanup,
|
||||
observability, consolidation and architectural understanding. The residue is
|
||||
fixes, compatibility paths, operational workarounds, duplicate config, adapter
|
||||
gaps and one-off patches. Some are justified. Some become permanent because
|
||||
nobody remembers why they were added or what they have cost since.
|
||||
|
||||
The target is a first-class debt ledger driven by evidence from real tasks, not
|
||||
a second TODO list.
|
||||
|
||||
## Classes
|
||||
|
||||
Four, and no more. A fifth class invented at runtime makes the priority model
|
||||
meaningless.
|
||||
|
||||
| Class | Meaning |
|
||||
|---|---|
|
||||
| correctness | Known behaviour is wrong or violates an invariant. |
|
||||
| operational | The system works, but diagnosis, recovery, deployment, observability or operation repeatedly costs time. |
|
||||
| structural | Duplication or architecture is demonstrably raising the cost of future changes. |
|
||||
| polish | Cleanup or consistency work with no demonstrated cost yet. |
|
||||
|
||||
Correctness and operational debt gain priority quickly. Structural debt needs
|
||||
evidence that it is causing repeated work. Polish never becomes work on its own.
|
||||
|
||||
## 1. What Orchestra already has
|
||||
|
||||
The event log is the durable spine, and it already carries most of what a debt
|
||||
ledger needs. Every row below is mechanically countable today, with no new
|
||||
instrumentation.
|
||||
|
||||
| Evidence | Source | Carries |
|
||||
|---|---|---|
|
||||
| Failure shape | `TaskFailed`, `TaskBlocked` | 12 typed `block_reason` values, 6 `failure_class` values |
|
||||
| Retry cost | `Task.Attempt`, `NextRetryAt`, `FailureClass` | how many leases a task burned, durable in the projection |
|
||||
| Review findings | `ReviewRecorded` | `{ID, Severity, File, Line, Claim, Evidence}`, bound to a `ResultSHA` |
|
||||
| Out-of-scope discoveries | `DeferredFindingRecorded` | `{Summary, Why}`, already recorded outside agent context |
|
||||
| Plan contradicted by code | `PlanMismatchRecorded` | `{PlanRef, PhaseID, AtSHA, Observed, Contradicts, Evidence[]}` |
|
||||
| Verification history | `PlanPhaseVerified` | `{Commands, ExitCodes, AtSHA, EvidenceRef}` per phase |
|
||||
| Rework rounds | `TaskChangesRequested`, `TaskSubmitted` | how many times a change went back |
|
||||
| Provenance | `TaskCreated` | `Source`, `ExternalID`, `Parent` |
|
||||
|
||||
Three existing patterns matter more than the data.
|
||||
|
||||
**Global-subject events already work.** `QuotaReported` and `StandupAdvisory`
|
||||
use `TaskID: "system"` and are whitelisted in `internal/domain/domain.go:279`.
|
||||
A debt item needs no new subject mechanism.
|
||||
|
||||
**`GenerateStandupAdvisory` is the precedent for the shape.** It is a
|
||||
persisted, read-only recommendation, and applying it is a separate
|
||||
approval-gated operation. A debt ledger is that pattern with better inputs.
|
||||
|
||||
**`CheckSubmission` is the precedent for eligibility.** It is a pure function
|
||||
of task, commit and gate run, returning `{Eligible bool, Reasons []string}`.
|
||||
Debt eligibility should have the same shape.
|
||||
|
||||
## 2. What evidence is missing
|
||||
|
||||
**Worker observations are not durable.** This is the largest gap. The F18 ring
|
||||
lives in worker memory and reaches the coordinator inside `WorkerHealth` on
|
||||
heartbeat. `Registry.persistedState` holds captures, commands and workers only,
|
||||
with no health. A coordinator restart erases every observation.
|
||||
|
||||
Operational debt is exactly what that ring holds. Repeated 409s, unrenewed
|
||||
leases, adapter gaps.
|
||||
|
||||
**No operator-intervention record.** Every manual repair in run 12 and run 13
|
||||
left no trace in Orchestra. State-file edits, worker restarts, two manual
|
||||
transaction cleanups. "Required manual recovery" is the strongest priority
|
||||
signal available, and it is currently unrecorded.
|
||||
|
||||
**No diagnosis-cost signal.** Time spent diagnosing is not measured. The
|
||||
closest proxy is wall time in `blocked` or `needs_attention`.
|
||||
|
||||
**No component dimension.** Findings carry file paths. Tasks carry none.
|
||||
Breadth across components has to be derived from paths.
|
||||
|
||||
**No link from a repair commit back to what it repaired.** The Fxx-to-commit
|
||||
mapping exists only in `BURNIN.md` prose.
|
||||
|
||||
**Deferred findings are too thin.** `{Summary, Why}` has no class, no severity,
|
||||
no paths and no evidence refs.
|
||||
|
||||
## 3. Minimal durable data model
|
||||
|
||||
No new subsystem. Two event types on the existing spine, one projection beside
|
||||
`Task`.
|
||||
|
||||
```go
|
||||
// DebtObservation is one piece of counted evidence, always pointing at the
|
||||
// event that produced it. Provenance is an event id, never prose.
|
||||
type DebtObservation struct {
|
||||
EventID string `json:"event_id"`
|
||||
TaskID string `json:"task_id,omitempty"`
|
||||
Kind string `json:"kind"` // block_reason, failure_class, review_finding,
|
||||
// plan_mismatch, manual_intervention, deferred_finding
|
||||
Signature string `json:"signature"` // the mechanical dedup key, see section 4
|
||||
Detail string `json:"detail"`
|
||||
Paths []string `json:"paths,omitempty"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
type DebtItem struct {
|
||||
ID string `json:"id"` // continues the Fxx namespace
|
||||
Class DebtClass `json:"class"` // correctness, operational, structural, polish
|
||||
Status DebtStatus `json:"status"` // observed, eligible, scheduled, repaired, withdrawn
|
||||
Symptom string `json:"symptom"`
|
||||
Consequence string `json:"consequence"`
|
||||
Severity Severity `json:"severity"` // reuse review.Severity
|
||||
Paths []string `json:"paths,omitempty"`
|
||||
Signatures []string `json:"signatures"` // every key that attaches here
|
||||
Observations []DebtObservation `json:"observations"`
|
||||
IntroducedIn string `json:"introduced_in,omitempty"` // task id or commit
|
||||
RepairBoundary string `json:"repair_boundary,omitempty"`
|
||||
RepairTask string `json:"repair_task,omitempty"`
|
||||
RepairCommit string `json:"repair_commit,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
Every counted field the priority model needs is derived, never stored:
|
||||
|
||||
```text
|
||||
recurrence len(Observations)
|
||||
blocked_tasks distinct TaskID where Kind is block_reason
|
||||
manual_interventions count of Kind == manual_intervention
|
||||
breadth distinct components derived from Paths
|
||||
```
|
||||
|
||||
A stored count invites drift from the log. A derived one cannot drift.
|
||||
|
||||
Two events carry it. `DebtObserved` appends one observation. `DebtItemUpdated`
|
||||
records class, status, severity or a merge. Both need adding to the `allowed`
|
||||
map and a validator, which is the work every existing event type already did.
|
||||
|
||||
## 4. Deduplication without corrupting provenance
|
||||
|
||||
**A signature is computed from typed facts, never from prose.**
|
||||
|
||||
```text
|
||||
operational class + block_reason + harness + normalized component
|
||||
correctness class + failure_class + normalized component
|
||||
structural class + finding severity + normalized component
|
||||
```
|
||||
|
||||
Exact signature match attaches automatically. That is the only automatic path.
|
||||
|
||||
**A model may propose, never merge.** Clustering emits `DebtMergeSuggested`
|
||||
carrying both item ids and its reasoning. The merge is an operator action or a
|
||||
policy threshold, recorded as `DebtItemUpdated`.
|
||||
|
||||
**Merges are additive.** The surviving item gains the other's signatures and
|
||||
observations. The absorbed item becomes `withdrawn` with a pointer, and its
|
||||
observations keep their original event ids. Nothing is rewritten, so a bad
|
||||
merge is reversible by reading the log.
|
||||
|
||||
**Every observation names its event.** A debt item's evidence is always
|
||||
checkable against the log that produced it. That rule is what stops a fuzzy
|
||||
proposal from becoming an unverifiable claim.
|
||||
|
||||
## 5. Eligibility policy
|
||||
|
||||
A pure function, mirroring `CheckSubmission`.
|
||||
|
||||
```go
|
||||
func CheckDebtEligibility(item DebtItem) DebtCheck // {Eligible bool, Reasons []string}
|
||||
```
|
||||
|
||||
| Class | Becomes eligible when |
|
||||
|---|---|
|
||||
| correctness | first confirmed observation, always |
|
||||
| operational | recurrence >= 3 across >= 2 distinct tasks, or >= 1 manual intervention |
|
||||
| structural | >= 3 review findings, or >= 2 tasks blocked or reworked in the same component |
|
||||
| polish | never automatically, operator promotion only |
|
||||
|
||||
`Reasons` lists what fired and what did not. "Not eligible" alone sends an
|
||||
operator reading code, which is the mistake `SubmissionCheck` already documents.
|
||||
|
||||
Eligible means a task may be created. It does not create one.
|
||||
|
||||
## 6. How maintenance and consolidation fit the existing lifecycle
|
||||
|
||||
A maintenance task is an ordinary task. Source `debt`, external id the debt item
|
||||
id, phases as usual.
|
||||
|
||||
```text
|
||||
research → plan → implement → verification → review → submission
|
||||
```
|
||||
|
||||
Nothing in the lifecycle changes. One extension is needed: the task's brief must
|
||||
carry the debt item's evidence. That is the mechanism research and plan
|
||||
artifacts already use, a CAS ref on the task.
|
||||
|
||||
A consolidation task is the same thing with a research brief that asks:
|
||||
|
||||
- which temporary paths are still necessary?
|
||||
- which fixes now duplicate each other?
|
||||
- which compatibility branches are obsolete?
|
||||
- which config or state is represented twice?
|
||||
- which abstractions exist only because of defects that have since disappeared?
|
||||
- what can now be deleted safely?
|
||||
|
||||
Its first output is a deletion plan, never an automatic refactor. Verification
|
||||
uses the project's existing policy. Review is the normal independent review.
|
||||
|
||||
**The repair gets no discount.** Classification as debt changes what gets
|
||||
scheduled, never what gets checked.
|
||||
|
||||
### Success metrics for maintenance work
|
||||
|
||||
Lines of code are not a metric. Prefer evidence that can be checked:
|
||||
|
||||
- behaviour preserved
|
||||
- tests preserved or strengthened
|
||||
- branches removed
|
||||
- obsolete types removed
|
||||
- compatibility code removed
|
||||
- duplicate config removed
|
||||
- manual recovery paths eliminated
|
||||
|
||||
## 7. What must not be automated
|
||||
|
||||
- No autonomous cleanup agent, and no repo-wide sweep.
|
||||
- No LLM score treated as authoritative. Counted facts decide, models propose.
|
||||
- No automatic merge of debt items.
|
||||
- No maintenance task from polish without an operator.
|
||||
- No deletion without an independent review.
|
||||
- No auto-closing an item because a commit touched the file. Closure needs a
|
||||
repair task or a stated run of clean evidence.
|
||||
- No new classes invented at runtime.
|
||||
|
||||
## 8. Migration, so there is one track and not two
|
||||
|
||||
`BURNIN.md` holds 58 Fxx entries in the form `| F52 | commit | prose |`.
|
||||
`AUDIT.md` holds the older narrative.
|
||||
|
||||
One-time import, by hand, with a class assigned per row. Each becomes a
|
||||
`DebtItem` with `Status: repaired`, its commit as `RepairCommit`, and one
|
||||
observation citing the ledger.
|
||||
|
||||
**Keep the namespace.** New items continue at F62. Two numbering schemes would
|
||||
be the first structural debt the ledger itself creates.
|
||||
|
||||
After import, `BURNIN.md` stops being the item ledger and remains the run
|
||||
narrative. The defect table becomes generated output from the projection.
|
||||
|
||||
### The history that tests the model
|
||||
|
||||
These are not special cases. They are the check on whether the model can
|
||||
represent the history this project already has.
|
||||
|
||||
| History | Expected class | Why |
|
||||
|---|---|---|
|
||||
| F18, the single `last_error` slot | operational | Repeatedly destroyed causal evidence before it was fixed. |
|
||||
| F43 to F46, the submission path | correctness | Apparently implemented, actually unreachable. |
|
||||
| F61, planner learns policy by refusal | operational | Costs a round trip per plan, and becomes model compatibility on weaker planners. |
|
||||
| Duplicated `quality_gate` config | structural | Only if it keeps causing drift or operator mistakes. |
|
||||
| Old blocked burn-in tasks | neither | Hygiene, unless one exposes a runtime defect. |
|
||||
| The blocked release loop, F57 to F60 | correctness and operational | Wrong behaviour, and it required manual cleanup twice. |
|
||||
|
||||
## 9. Smallest slice to live-prove first
|
||||
|
||||
**A read-only projection over the existing log. No new events, no writes, no
|
||||
schema commitment.**
|
||||
|
||||
```text
|
||||
internal/operations/debt.go DebtItem, signature, projection over s.Events(0)
|
||||
CheckDebtEligibility
|
||||
GET /v1/debt read-only, tui surface
|
||||
```
|
||||
|
||||
The proof is that the current log already contains runs 10 to 14. The projection
|
||||
must independently surface, from history alone:
|
||||
|
||||
- the release loop as operational debt, with its 409 recurrence and two manual
|
||||
cleanups
|
||||
- the opencode adapter gap as operational debt, across three failed attempts on
|
||||
one harness
|
||||
- the retry-idleness dynamic, attached to the tasks that expired
|
||||
|
||||
A model that cannot reproduce debt already known is wrong, and no schema has
|
||||
been committed to yet. That is the cheapest place to find out.
|
||||
|
||||
**Slice two** makes worker observations durable, because that is the input the
|
||||
projection will most visibly lack.
|
||||
|
||||
**Slice three** adds `DebtObserved` and manual-intervention recording.
|
||||
|
||||
## Scope boundary
|
||||
|
||||
This design changes no part of the task lifecycle, phases, leases, review,
|
||||
submission, federation or the plan machinery. The only extensions it needs are
|
||||
two event types, one projection, one pure eligibility function and one
|
||||
read-only endpoint.
|
||||
Reference in New Issue
Block a user