Compare commits
3 Commits
595a1d3533
...
2d28f7b462
| Author | SHA1 | Date | |
|---|---|---|---|
| 2d28f7b462 | |||
| a757cffc78 | |||
| 2f61a99986 |
@@ -2479,7 +2479,8 @@ stops being latency and becomes model compatibility.
|
||||
### opencode is not running, and the reason was only in the pane
|
||||
|
||||
Task `06G4M6HH5BAM235AC4PWYXF1HM` failed at attempt 3 having never left
|
||||
`frame`. Its worktree held only `launch.md`. The pane said why:
|
||||
`frame`. Its worktree held only `launch.md`. The pane said why. The blocked
|
||||
thing is the selected model, not the OpenCode Zen free tier, which works:
|
||||
|
||||
```text
|
||||
This model is not available in your country.
|
||||
@@ -2499,7 +2500,9 @@ fail. Two things are worth keeping.
|
||||
not Orchestra's job; surfacing the capture is.
|
||||
|
||||
**Do not route to `workpc-opencode` until its model is reachable.** Every task
|
||||
sent there burns three attempts and then fails.
|
||||
sent there burns three attempts and then fails. `~/.config/opencode/opencode.jsonc`
|
||||
sets no top-level `model`, so OpenCode picks its own default. The file already
|
||||
declares reachable `nvidia-nim` and `llama-cpp` providers.
|
||||
|
||||
### The retry budget counts idleness, not just failure
|
||||
|
||||
|
||||
+410
@@ -0,0 +1,410 @@
|
||||
# Debt ledger: design from what the tree already supports
|
||||
|
||||
Written 2026-08-29 against `3c7cf95`. Read with `BURNIN.md` (the run ledger),
|
||||
`PLAN-SPEC-DESIGN.md` and `AUDIT.md`.
|
||||
|
||||
This is a design, not an implementation. Nothing here has been built.
|
||||
|
||||
## The problem being solved
|
||||
|
||||
AI raises implementation throughput faster than it raises integration, cleanup,
|
||||
observability, consolidation and architectural understanding. The residue is
|
||||
fixes, compatibility paths, operational workarounds, duplicate config, adapter
|
||||
gaps and one-off patches. Some are justified. Some become permanent because
|
||||
nobody remembers why they were added or what they have cost since.
|
||||
|
||||
The target is a first-class debt ledger driven by evidence from real tasks, not
|
||||
a second TODO list.
|
||||
|
||||
## Classes
|
||||
|
||||
Four, and no more. A fifth class invented at runtime makes the priority model
|
||||
meaningless.
|
||||
|
||||
| Class | Meaning |
|
||||
|---|---|
|
||||
| correctness | Known behaviour is wrong or violates an invariant. |
|
||||
| operational | The system works, but diagnosis, recovery, deployment, observability or operation repeatedly costs time. |
|
||||
| structural | Duplication or architecture is demonstrably raising the cost of future changes. |
|
||||
| polish | Cleanup or consistency work with no demonstrated cost yet. |
|
||||
|
||||
Correctness and operational debt gain priority quickly. Structural debt needs
|
||||
evidence that it is causing repeated work. Polish never becomes work on its own.
|
||||
|
||||
## 1. What Orchestra already has
|
||||
|
||||
The event log is the durable spine, and it already carries most of what a debt
|
||||
ledger needs. Every row below is mechanically countable today, with no new
|
||||
instrumentation.
|
||||
|
||||
| Evidence | Source | Carries |
|
||||
|---|---|---|
|
||||
| Failure shape | `TaskFailed`, `TaskBlocked` | 12 typed `block_reason` values, 6 `failure_class` values |
|
||||
| Retry cost | `Task.Attempt`, `NextRetryAt`, `FailureClass` | how many leases a task burned, durable in the projection |
|
||||
| Review findings | `ReviewRecorded` | `{ID, Severity, File, Line, Claim, Evidence}`, bound to a `ResultSHA` |
|
||||
| Out-of-scope discoveries | `DeferredFindingRecorded` | `{Summary, Why}`, already recorded outside agent context |
|
||||
| Plan contradicted by code | `PlanMismatchRecorded` | `{PlanRef, PhaseID, AtSHA, Observed, Contradicts, Evidence[]}` |
|
||||
| Verification history | `PlanPhaseVerified` | `{Commands, ExitCodes, AtSHA, EvidenceRef}` per phase |
|
||||
| Rework rounds | `TaskChangesRequested`, `TaskSubmitted` | how many times a change went back |
|
||||
| Provenance | `TaskCreated` | `Source`, `ExternalID`, `Parent` |
|
||||
|
||||
Three existing patterns matter more than the data.
|
||||
|
||||
**Global-subject events already work.** `QuotaReported` and `StandupAdvisory`
|
||||
use `TaskID: "system"` and are whitelisted in `internal/domain/domain.go:279`.
|
||||
A debt item needs no new subject mechanism.
|
||||
|
||||
**`GenerateStandupAdvisory` is the precedent for the shape.** It is a
|
||||
persisted, read-only recommendation, and applying it is a separate
|
||||
approval-gated operation. A debt ledger is that pattern with better inputs.
|
||||
|
||||
**`CheckSubmission` is the precedent for eligibility.** It is a pure function
|
||||
of task, commit and gate run, returning `{Eligible bool, Reasons []string}`.
|
||||
Debt eligibility should have the same shape.
|
||||
|
||||
## 2. What evidence is missing
|
||||
|
||||
**Worker observations are not durable.** This is the largest gap. The F18 ring
|
||||
lives in worker memory and reaches the coordinator inside `WorkerHealth` on
|
||||
heartbeat. `Registry.persistedState` holds captures, commands and workers only,
|
||||
with no health. A coordinator restart erases every observation.
|
||||
|
||||
Operational debt is exactly what that ring holds. Repeated 409s, unrenewed
|
||||
leases, adapter gaps.
|
||||
|
||||
**No operator-intervention record.** Every manual repair in run 12 and run 13
|
||||
left no trace in Orchestra. State-file edits, worker restarts, two manual
|
||||
transaction cleanups. "Required manual recovery" is the strongest priority
|
||||
signal available, and it is currently unrecorded.
|
||||
|
||||
**No diagnosis-cost signal.** Time spent diagnosing is not measured. The
|
||||
closest proxy is wall time in `blocked` or `needs_attention`.
|
||||
|
||||
**No component dimension.** Findings carry file paths. Tasks carry none.
|
||||
Breadth across components has to be derived from paths.
|
||||
|
||||
**No link from a repair commit back to what it repaired.** The Fxx-to-commit
|
||||
mapping exists only in `BURNIN.md` prose.
|
||||
|
||||
**Deferred findings are too thin.** `{Summary, Why}` has no class, no severity,
|
||||
no paths and no evidence refs.
|
||||
|
||||
## 3. Minimal durable data model
|
||||
|
||||
No new subsystem. Two event types on the existing spine, one projection beside
|
||||
`Task`.
|
||||
|
||||
```go
|
||||
// DebtObservation is one piece of counted evidence, always pointing at the
|
||||
// event that produced it. Provenance is an event id, never prose.
|
||||
type DebtObservation struct {
|
||||
// Exactly one of EventID and LegacyRef. See section 8.
|
||||
EventID string `json:"event_id,omitempty"`
|
||||
LegacyRef string `json:"legacy_ref,omitempty"`
|
||||
TaskID string `json:"task_id,omitempty"`
|
||||
Kind string `json:"kind"` // block_reason, failure_class, review_finding,
|
||||
// plan_mismatch, manual_intervention, deferred_finding
|
||||
Signature string `json:"signature"` // the mechanical dedup key, see section 4
|
||||
Detail string `json:"detail"`
|
||||
Paths []string `json:"paths,omitempty"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
type DebtItem struct {
|
||||
ID string `json:"id"` // continues the Fxx namespace
|
||||
Class DebtClass `json:"class"` // correctness, operational, structural, polish
|
||||
Status DebtStatus `json:"status"` // observed, eligible, scheduled, repaired, withdrawn
|
||||
Symptom string `json:"symptom"`
|
||||
Consequence string `json:"consequence"`
|
||||
Severity Severity `json:"severity"` // reuse review.Severity
|
||||
Paths []string `json:"paths,omitempty"`
|
||||
Signatures []string `json:"signatures"` // every key that attaches here
|
||||
Observations []DebtObservation `json:"observations"`
|
||||
IntroducedIn string `json:"introduced_in,omitempty"` // task id or commit
|
||||
RepairBoundary string `json:"repair_boundary,omitempty"`
|
||||
RepairTask string `json:"repair_task,omitempty"`
|
||||
RepairCommit string `json:"repair_commit,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
Every counted field the priority model needs is derived, never stored:
|
||||
|
||||
```text
|
||||
recurrence len(Observations)
|
||||
blocked_tasks distinct TaskID where Kind is block_reason
|
||||
manual_interventions count of Kind == manual_intervention
|
||||
breadth distinct components derived from Paths
|
||||
```
|
||||
|
||||
A stored count invites drift from the log. A derived one cannot drift.
|
||||
|
||||
Two events carry it. `DebtObserved` appends one observation. `DebtItemUpdated`
|
||||
records class, status, severity or a merge. Both need adding to the `allowed`
|
||||
map and a validator, which is the work every existing event type already did.
|
||||
|
||||
## 4. Deduplication without corrupting provenance
|
||||
|
||||
**A signature is computed from typed facts, never from prose.**
|
||||
|
||||
```text
|
||||
operational class + block_reason + harness + normalized component
|
||||
correctness class + failure_class + normalized component
|
||||
structural class + finding severity + normalized component
|
||||
```
|
||||
|
||||
Exact signature match attaches automatically. That is the only automatic path.
|
||||
|
||||
**Signatures are versioned, in the string itself.**
|
||||
|
||||
```text
|
||||
v1:operational:lease_not_renewed:workpc-opencode:federation
|
||||
```
|
||||
|
||||
Normalization rules will change. Without a version, changing them silently
|
||||
regroups every historical observation, and the recurrence counts that drive
|
||||
eligibility move underneath the items that already used them. A `v2` signature
|
||||
never matches a `v1` one, so old evidence keeps the grouping it was counted
|
||||
under.
|
||||
|
||||
**A model may propose, never merge.** Clustering emits `DebtMergeSuggested`
|
||||
carrying both item ids and its reasoning. The merge is an operator action or a
|
||||
policy threshold, recorded as `DebtItemUpdated`.
|
||||
|
||||
**Merges are additive.** The surviving item gains the other's signatures and
|
||||
observations. The absorbed item becomes `withdrawn` with a pointer, and its
|
||||
observations keep their original event ids. Nothing is rewritten, so a bad
|
||||
merge is reversible by reading the log.
|
||||
|
||||
**Every observation names its event.** A debt item's evidence is always
|
||||
checkable against the log that produced it. That rule is what stops a fuzzy
|
||||
proposal from becoming an unverifiable claim.
|
||||
|
||||
## 5. Eligibility policy
|
||||
|
||||
A pure function, mirroring `CheckSubmission`.
|
||||
|
||||
```go
|
||||
func CheckDebtEligibility(item DebtItem) DebtCheck // {Eligible bool, Reasons []string}
|
||||
```
|
||||
|
||||
| Class | Becomes eligible when |
|
||||
|---|---|
|
||||
| correctness | first confirmed observation, always |
|
||||
| operational | recurrence >= 3 across >= 2 distinct tasks, or >= 1 manual intervention |
|
||||
| structural | >= 3 review findings, or >= 2 tasks blocked or reworked in the same component |
|
||||
| polish | never automatically, operator promotion only |
|
||||
|
||||
`Reasons` lists what fired and what did not. "Not eligible" alone sends an
|
||||
operator reading code, which is the mistake `SubmissionCheck` already documents.
|
||||
|
||||
Eligible means a task may be created. It does not create one.
|
||||
|
||||
## 6. How maintenance and consolidation fit the existing lifecycle
|
||||
|
||||
A maintenance task is an ordinary task. Source `debt`, external id the debt item
|
||||
id, phases as usual.
|
||||
|
||||
```text
|
||||
research → plan → implement → verification → review → submission
|
||||
```
|
||||
|
||||
Nothing in the lifecycle changes. One extension is needed: the task's brief must
|
||||
carry the debt item's evidence. That is the mechanism research and plan
|
||||
artifacts already use, a CAS ref on the task.
|
||||
|
||||
A consolidation task is the same thing with a research brief that asks:
|
||||
|
||||
- which temporary paths are still necessary?
|
||||
- which fixes now duplicate each other?
|
||||
- which compatibility branches are obsolete?
|
||||
- which config or state is represented twice?
|
||||
- which abstractions exist only because of defects that have since disappeared?
|
||||
- what can now be deleted safely?
|
||||
|
||||
Its first output is a deletion plan, never an automatic refactor. Verification
|
||||
uses the project's existing policy. Review is the normal independent review.
|
||||
|
||||
**The repair gets no discount.** Classification as debt changes what gets
|
||||
scheduled, never what gets checked.
|
||||
|
||||
### Success metrics for maintenance work
|
||||
|
||||
Lines of code are not a metric. Prefer evidence that can be checked:
|
||||
|
||||
- behaviour preserved
|
||||
- tests preserved or strengthened
|
||||
- branches removed
|
||||
- obsolete types removed
|
||||
- compatibility code removed
|
||||
- duplicate config removed
|
||||
- manual recovery paths eliminated
|
||||
|
||||
## 7. What must not be automated
|
||||
|
||||
- No autonomous cleanup agent, and no repo-wide sweep.
|
||||
- No LLM score treated as authoritative. Counted facts decide, models propose.
|
||||
- No automatic merge of debt items.
|
||||
- No maintenance task from polish without an operator.
|
||||
- No deletion without an independent review.
|
||||
- No auto-closing an item because a commit touched the file. Closure needs a
|
||||
repair task or a stated run of clean evidence.
|
||||
- No new classes invented at runtime.
|
||||
|
||||
## 8. Migration, so there is one track and not two
|
||||
|
||||
`BURNIN.md` holds 58 Fxx entries in the form `| F52 | commit | prose |`.
|
||||
`AUDIT.md` holds the older narrative.
|
||||
|
||||
One-time import, by hand, with a class assigned per row. Each becomes a
|
||||
`DebtItem` with `Status: repaired`, its commit as `RepairCommit`, and one
|
||||
observation citing the ledger.
|
||||
|
||||
**Imported observations carry legacy provenance, never a fabricated event id.**
|
||||
The Fxx history predates the capability that would have produced an event, and
|
||||
inventing one would break the provenance rule the ledger exists to enforce.
|
||||
|
||||
```go
|
||||
EventID string `json:"event_id,omitempty"`
|
||||
LegacyRef string `json:"legacy_ref,omitempty"` // "BURNIN.md:F18"
|
||||
```
|
||||
|
||||
Exactly one is required. A reader can then tell a counted fact from an imported
|
||||
claim at a glance, and the counts that drive eligibility can exclude imported
|
||||
evidence if that turns out to matter.
|
||||
|
||||
**Keep the namespace.** New items continue at F62. Two numbering schemes would
|
||||
be the first structural debt the ledger itself creates.
|
||||
|
||||
After import, `BURNIN.md` stops being the item ledger and remains the run
|
||||
narrative. The defect table becomes generated output from the projection.
|
||||
|
||||
### The history that tests the model
|
||||
|
||||
These are not special cases. They are the check on whether the model can
|
||||
represent the history this project already has.
|
||||
|
||||
| History | Expected class | Why |
|
||||
|---|---|---|
|
||||
| F18, the single `last_error` slot | operational | Repeatedly destroyed causal evidence before it was fixed. |
|
||||
| F43 to F46, the submission path | correctness | Apparently implemented, actually unreachable. |
|
||||
| F61, planner learns policy by refusal | operational | Costs a round trip per plan, and becomes model compatibility on weaker planners. |
|
||||
| Duplicated `quality_gate` config | structural | Only if it keeps causing drift or operator mistakes. |
|
||||
| Old blocked burn-in tasks | neither | Hygiene, unless one exposes a runtime defect. |
|
||||
| The blocked release loop, F57 to F60 | correctness and operational | Wrong behaviour, and it required manual cleanup twice. |
|
||||
|
||||
## 9. Smallest slice to live-prove first
|
||||
|
||||
**A read-only projection over the existing log. No new events, no writes, no
|
||||
schema commitment.**
|
||||
|
||||
```text
|
||||
internal/domain/debt.go DebtItem, DebtObservation, signature, classification
|
||||
internal/store/debt_projection.go the fold over s.Events(0), and the gap report
|
||||
internal/operations/debt.go CheckDebtEligibility, and later the actions
|
||||
GET /v1/debt read-only, tui surface
|
||||
```
|
||||
|
||||
The projected type stays out of `operations`. Baking a read model into the
|
||||
command layer in the first slice is the mistake that would be hardest to undo
|
||||
later. `operations` owns actions: `CheckDebtEligibility` now, and
|
||||
`SuggestDebtMerge`, `PromoteDebtItem` and `ScheduleDebtRepair` when they exist.
|
||||
|
||||
The proof is what the projection can and cannot recover from canonical history.
|
||||
Section 2 already says worker observations are not durable and operator
|
||||
interventions are unrecorded, so demanding their reconstruction would be asking
|
||||
the projection to invent evidence.
|
||||
|
||||
The first burn-in is therefore three requirements, not one:
|
||||
|
||||
1. Recover every debt signal that canonical history actually encodes.
|
||||
2. Report, explicitly and per kind, where known debt cannot be reconstructed.
|
||||
3. Never infer a missing observation from `BURNIN.md`.
|
||||
|
||||
Incompleteness is part of the result, not a failure. The 409 loop lived in the
|
||||
F18 worker ring rather than in an event, so the ledger should say it has no
|
||||
durable evidence for that shape. That statement is what makes slice two
|
||||
necessary, measurably rather than by assertion.
|
||||
|
||||
What the projection should recover from the log alone:
|
||||
|
||||
- repeated `lease_expired` and `retry_limit` on one harness, which is the
|
||||
opencode failure shape across four tasks
|
||||
- the retry-idleness dynamic, attached to the tasks that expired
|
||||
- review findings grouped by component, if any repeat
|
||||
|
||||
A model that cannot represent debt already known is wrong, and no schema has
|
||||
been committed to yet. That is the cheapest place to find out.
|
||||
|
||||
**Slice two** makes worker observations durable, because that is the input the
|
||||
projection will most visibly lack.
|
||||
|
||||
**Slice three** adds `DebtObserved` and manual-intervention recording.
|
||||
|
||||
## Scope boundary
|
||||
|
||||
This design changes no part of the task lifecycle, phases, leases, review,
|
||||
submission, federation or the plan machinery. The only extensions it needs are
|
||||
two event types, one projection, one pure eligibility function and one
|
||||
read-only endpoint.
|
||||
|
||||
---
|
||||
|
||||
# Slice one, run against real history
|
||||
|
||||
Built and deployed as `a757cff`. `GET /v1/debt` folded 881 events from the
|
||||
live log, wrote nothing, and produced 15 candidate items and 3 gaps.
|
||||
|
||||
## What it recovered
|
||||
|
||||
```text
|
||||
v1:operational:lease_expired:workpc-opencode:lease r=41 tasks=4
|
||||
v1:operational:lease_expired:workpc-claude:lease r=29 tasks=13
|
||||
v1:operational:lease_failure:-:lease r=20 tasks=16
|
||||
v1:operational:system_error:-:lease r=11 tasks=7
|
||||
v1:correctness:handoff_validation:-:lease r=5 tasks=5
|
||||
v1:correctness:plan_mismatch:-:... r=1 tasks=1
|
||||
v1:structural:minor:-:scripts/orchestra_e2e_healthcheck.sh r=1 tasks=1
|
||||
```
|
||||
|
||||
The opencode failure shape is the top item, found mechanically. Run 14
|
||||
diagnosed the same thing by hand from a pane capture. The retry-idleness
|
||||
dynamic is the second item, and it is now a number: 29 expiries across 13
|
||||
tasks on one harness.
|
||||
|
||||
Seven items are eligible under the stated policy. Polish and the single
|
||||
structural finding correctly are not.
|
||||
|
||||
## What it reported that it could not see
|
||||
|
||||
```text
|
||||
durable=false manual_intervention no event type records an operator repair
|
||||
durable=false worker_observation worker health is not persisted
|
||||
durable=true deferred_finding carried by the log, this history has none
|
||||
```
|
||||
|
||||
The 409 release loop does not appear, and it should not. That evidence lived in
|
||||
the F18 worker ring, which no event carries. The ledger says so rather than
|
||||
inferring it, which is the result slice two exists to change.
|
||||
|
||||
## Four defects the first run exposed
|
||||
|
||||
**The component part is too coarse for lease evidence.** Everything
|
||||
lease-related normalizes to `lease`, so `lease_expired:workpc-claude` is one
|
||||
bucket holding idle agents, rig interference and real failures. Recurrence 29
|
||||
is true and the item is not a defect.
|
||||
|
||||
**`harness` is often empty on block reasons.** `TaskBlocked` payloads do not
|
||||
always carry `harness_id`, so `lease_failure:-` mixes harnesses that should be
|
||||
separate items.
|
||||
|
||||
**Path normalization mangled a mismatch reference.**
|
||||
`scripts/orchestra_e2e_healthcheck.sh:12` became
|
||||
`scripts/orchestra_e2e_healthcheck.sh_12`, because the signature sanitizer
|
||||
replaces `:` and the mismatch evidence field holds prose, not clean paths.
|
||||
|
||||
**Recurrence alone is the wrong sort.** 41 occurrences on 4 tasks currently
|
||||
outranks 29 on 13 tasks. Breadth is in the design and not yet in the ordering,
|
||||
which is the priority function slice one deliberately omitted.
|
||||
|
||||
None of these required a schema commitment to discover. That was the point of
|
||||
making the first slice read-only.
|
||||
@@ -445,6 +445,17 @@ func main() {
|
||||
}
|
||||
json.NewEncoder(w).Encode(s.Events(n))
|
||||
})
|
||||
mux.HandleFunc("/v1/debt", func(w http.ResponseWriter, r *http.Request) {
|
||||
// Read-only, and deliberately so. This projection is evidence about
|
||||
// history, not a new kind of truth: nothing here writes an event,
|
||||
// schedules work, or decides that something is worth repairing.
|
||||
ledger := s.DebtLedger()
|
||||
out := struct {
|
||||
domain.DebtLedger
|
||||
Eligible []operations.DebtCandidate `json:"eligible"`
|
||||
}{ledger, operations.EligibleDebt(ledger)}
|
||||
json.NewEncoder(w).Encode(out)
|
||||
})
|
||||
mux.HandleFunc("/v1/handoffs", func(w http.ResponseWriter, r *http.Request) {
|
||||
out := make([]domain.Event, 0)
|
||||
for _, e := range s.Events(0) {
|
||||
|
||||
@@ -0,0 +1,260 @@
|
||||
package domain
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"orchestra/internal/review"
|
||||
)
|
||||
|
||||
// Debt is the projected read model for accumulated cost: the defects,
|
||||
// workarounds and operational burdens that real tasks keep paying for. It is
|
||||
// derived from the event log rather than written by hand, so a claim in the
|
||||
// ledger can always be checked against the events that produced it.
|
||||
//
|
||||
// Four classes, and no more. A fifth invented at runtime makes the priority
|
||||
// model meaningless, because the thresholds in operations.CheckDebtEligibility
|
||||
// are stated per class.
|
||||
type DebtClass string
|
||||
|
||||
const (
|
||||
// DebtCorrectness is behaviour that is wrong or violates an invariant.
|
||||
DebtCorrectness DebtClass = "correctness"
|
||||
// DebtOperational works, but repeatedly costs time to diagnose, recover,
|
||||
// deploy, observe or operate.
|
||||
DebtOperational DebtClass = "operational"
|
||||
// DebtStructural is duplication or architecture demonstrably raising the
|
||||
// cost of future changes.
|
||||
DebtStructural DebtClass = "structural"
|
||||
// DebtPolish has no demonstrated cost yet, and never promotes itself.
|
||||
DebtPolish DebtClass = "polish"
|
||||
)
|
||||
|
||||
func (c DebtClass) Valid() bool {
|
||||
switch c {
|
||||
case DebtCorrectness, DebtOperational, DebtStructural, DebtPolish:
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
type DebtStatus string
|
||||
|
||||
const (
|
||||
DebtObserved DebtStatus = "observed"
|
||||
DebtEligible DebtStatus = "eligible"
|
||||
DebtScheduled DebtStatus = "scheduled"
|
||||
DebtRepaired DebtStatus = "repaired"
|
||||
DebtWithdrawn DebtStatus = "withdrawn"
|
||||
)
|
||||
|
||||
// ObservationKind names where one piece of evidence came from. It is the kind
|
||||
// of the source fact, never an interpretation of it.
|
||||
type ObservationKind string
|
||||
|
||||
const (
|
||||
ObservationBlockReason ObservationKind = "block_reason"
|
||||
ObservationFailureClass ObservationKind = "failure_class"
|
||||
ObservationReviewFinding ObservationKind = "review_finding"
|
||||
ObservationPlanMismatch ObservationKind = "plan_mismatch"
|
||||
ObservationDeferredFinding ObservationKind = "deferred_finding"
|
||||
ObservationManualIntervention ObservationKind = "manual_intervention"
|
||||
ObservationWorkerFailure ObservationKind = "worker_observation"
|
||||
)
|
||||
|
||||
// SignatureVersion prefixes every signature this build produces. Normalization
|
||||
// rules will change, and without a version a change silently regroups every
|
||||
// historical observation, moving the recurrence counts that eligibility was
|
||||
// already decided on. A v2 signature never matches a v1 one, so old evidence
|
||||
// keeps the grouping it was counted under.
|
||||
const SignatureVersion = "v1"
|
||||
|
||||
// DebtSignature is the mechanical dedup key. Every part is a typed fact, never
|
||||
// prose, because exact signature match is the only path that attaches evidence
|
||||
// without a human. An empty part becomes "-" so the arity never varies.
|
||||
func DebtSignature(class DebtClass, reason, scope, component string) string {
|
||||
part := func(s string) string {
|
||||
s = strings.TrimSpace(strings.ToLower(s))
|
||||
s = strings.ReplaceAll(s, ":", "_")
|
||||
if s == "" {
|
||||
return "-"
|
||||
}
|
||||
return s
|
||||
}
|
||||
return strings.Join([]string{SignatureVersion, part(string(class)), part(reason), part(scope), part(component)}, ":")
|
||||
}
|
||||
|
||||
// DebtComponent reduces a repository path to the unit that owns it. Two
|
||||
// segments is the whole rule: it keeps internal/store distinct from
|
||||
// internal/herdr without splitting one package across files.
|
||||
func DebtComponent(path string) string {
|
||||
path = strings.TrimSpace(strings.Trim(path, "/"))
|
||||
if path == "" {
|
||||
return ""
|
||||
}
|
||||
parts := strings.Split(path, "/")
|
||||
if len(parts) == 1 {
|
||||
return parts[0]
|
||||
}
|
||||
return parts[0] + "/" + parts[1]
|
||||
}
|
||||
|
||||
// DebtObservation is one piece of counted evidence. Provenance is required and
|
||||
// is either an event id or, for history imported from a markdown ledger that
|
||||
// predates this projection, an explicit legacy reference. Exactly one, because
|
||||
// a fabricated event id would break the rule the ledger exists to enforce.
|
||||
type DebtObservation struct {
|
||||
EventID string `json:"event_id,omitempty"`
|
||||
LegacyRef string `json:"legacy_ref,omitempty"`
|
||||
TaskID string `json:"task_id,omitempty"`
|
||||
Kind ObservationKind `json:"kind"`
|
||||
Signature string `json:"signature"`
|
||||
Detail string `json:"detail,omitempty"`
|
||||
Paths []string `json:"paths,omitempty"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
func (o DebtObservation) Validate() error {
|
||||
if (o.EventID == "") == (o.LegacyRef == "") {
|
||||
return fmt.Errorf("observation needs exactly one of event_id and legacy_ref")
|
||||
}
|
||||
if o.Kind == "" {
|
||||
return fmt.Errorf("observation kind required")
|
||||
}
|
||||
if !strings.HasPrefix(o.Signature, SignatureVersion+":") {
|
||||
return fmt.Errorf("observation signature %q is not %s", o.Signature, SignatureVersion)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// DebtItem groups observations that share a signature. Every count the
|
||||
// priority model needs is derived from Observations rather than stored, so a
|
||||
// count can never drift from the log that justifies it.
|
||||
type DebtItem struct {
|
||||
ID string `json:"id"`
|
||||
Class DebtClass `json:"class"`
|
||||
Status DebtStatus `json:"status"`
|
||||
Symptom string `json:"symptom"`
|
||||
Consequence string `json:"consequence,omitempty"`
|
||||
Severity review.Severity `json:"severity,omitempty"`
|
||||
Signatures []string `json:"signatures"`
|
||||
Paths []string `json:"paths,omitempty"`
|
||||
Observations []DebtObservation `json:"observations"`
|
||||
IntroducedIn string `json:"introduced_in,omitempty"`
|
||||
RepairBoundary string `json:"repair_boundary,omitempty"`
|
||||
RepairTask string `json:"repair_task,omitempty"`
|
||||
RepairCommit string `json:"repair_commit,omitempty"`
|
||||
}
|
||||
|
||||
// Recurrence is how many times this shape has been seen.
|
||||
func (d DebtItem) Recurrence() int { return len(d.Observations) }
|
||||
|
||||
// AffectedTasks is how broadly the shape has spread. Distinct tasks, because
|
||||
// one task failing ten times is weaker evidence than ten tasks failing once.
|
||||
func (d DebtItem) AffectedTasks() int {
|
||||
seen := map[string]bool{}
|
||||
for _, o := range d.Observations {
|
||||
if o.TaskID != "" {
|
||||
seen[o.TaskID] = true
|
||||
}
|
||||
}
|
||||
return len(seen)
|
||||
}
|
||||
|
||||
// BlockedTasks counts distinct tasks this shape actually stopped.
|
||||
func (d DebtItem) BlockedTasks() int {
|
||||
seen := map[string]bool{}
|
||||
for _, o := range d.Observations {
|
||||
if o.Kind == ObservationBlockReason && o.TaskID != "" {
|
||||
seen[o.TaskID] = true
|
||||
}
|
||||
}
|
||||
return len(seen)
|
||||
}
|
||||
|
||||
// ManualInterventions counts recorded operator repairs. It reads zero on any
|
||||
// log written before that recording exists, which is why the ledger reports it
|
||||
// as an evidence gap rather than as an absence of operator cost.
|
||||
func (d DebtItem) ManualInterventions() int {
|
||||
n := 0
|
||||
for _, o := range d.Observations {
|
||||
if o.Kind == ObservationManualIntervention {
|
||||
n++
|
||||
}
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// Components is the breadth of the shape across the tree.
|
||||
func (d DebtItem) Components() []string {
|
||||
seen := map[string]bool{}
|
||||
out := []string{}
|
||||
for _, o := range d.Observations {
|
||||
for _, p := range o.Paths {
|
||||
if c := DebtComponent(p); c != "" && !seen[c] {
|
||||
seen[c] = true
|
||||
out = append(out, c)
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Imported reports whether any evidence here came from a markdown ledger
|
||||
// rather than from an event. A reader can then tell a counted fact from an
|
||||
// imported claim without opening the observations.
|
||||
func (d DebtItem) Imported() bool {
|
||||
for _, o := range d.Observations {
|
||||
if o.LegacyRef != "" {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// EvidenceGap is a debt signal the ledger knows it cannot see. Reporting the
|
||||
// gap is the point: a ledger that silently omits what it cannot record reads
|
||||
// as "no operator cost" when it means "operator cost is not recorded".
|
||||
type EvidenceGap struct {
|
||||
Kind ObservationKind `json:"kind"`
|
||||
Reason string `json:"reason"`
|
||||
// Durable is false when no event type carries this evidence at all. It is
|
||||
// true when the log could carry it and this particular log does not.
|
||||
Durable bool `json:"durable"`
|
||||
}
|
||||
|
||||
// DebtLedger is the whole read model: what history establishes, and what it
|
||||
// cannot.
|
||||
type DebtLedger struct {
|
||||
Items []DebtItem `json:"items"`
|
||||
Gaps []EvidenceGap `json:"gaps"`
|
||||
// Events is how many log entries the projection folded, so a caller can
|
||||
// tell an empty ledger from an unread log.
|
||||
Events int `json:"events"`
|
||||
}
|
||||
|
||||
// DebtClassForBlockReason maps a typed block reason to a class. A reason that
|
||||
// is a normal lifecycle stop rather than a cost returns false: waiting for a
|
||||
// human decision is the system working, not debt.
|
||||
func DebtClassForBlockReason(r BlockReason) (DebtClass, bool) {
|
||||
switch r {
|
||||
case BlockReasonLeaseFailure, BlockReasonWorkerOffline, BlockReasonLeaseExpired, BlockReasonSystem:
|
||||
return DebtOperational, true
|
||||
case BlockReasonHandoffValidation, BlockReasonPlanMismatch:
|
||||
return DebtCorrectness, true
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
|
||||
// DebtClassForFailureClass maps a worker failure class to a class. Only the
|
||||
// classes a worker actually emits are listed; an unknown one is not guessed at.
|
||||
func DebtClassForFailureClass(f string) (DebtClass, bool) {
|
||||
switch f {
|
||||
case "retry_limit", "launch_failed", "launch_transient", "launch_uncertain", "prompt_not_submitted", "lease_expired":
|
||||
return DebtOperational, true
|
||||
case "invalid_handoff":
|
||||
return DebtCorrectness, true
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
package domain
|
||||
|
||||
import "testing"
|
||||
|
||||
// Normalization rules will change. A v2 signature must never match a v1 one,
|
||||
// or changing them silently regroups history and moves the recurrence counts
|
||||
// eligibility was already decided on.
|
||||
func TestSignatureCarriesItsVersion(t *testing.T) {
|
||||
sig := DebtSignature(DebtOperational, "lease_expired", "workpc-opencode", "internal/herdr")
|
||||
if sig != "v1:operational:lease_expired:workpc-opencode:internal/herdr" {
|
||||
t.Fatalf("signature %q", sig)
|
||||
}
|
||||
// Arity never varies, so a missing part cannot shift the fields left.
|
||||
if got := DebtSignature(DebtPolish, "deferred_finding", "", ""); got != "v1:polish:deferred_finding:-:-" {
|
||||
t.Fatalf("empty parts not padded: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestComponentIsTwoSegments(t *testing.T) {
|
||||
for path, want := range map[string]string{
|
||||
"internal/store/store.go": "internal/store",
|
||||
"internal/herdr/adapter.go": "internal/herdr",
|
||||
"cmd/orchestra-worker/main.go": "cmd/orchestra-worker",
|
||||
"BURNIN.md": "BURNIN.md",
|
||||
"": "",
|
||||
} {
|
||||
if got := DebtComponent(path); got != want {
|
||||
t.Fatalf("component(%q) = %q, want %q", path, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Imported history predates the events that would have justified it. A
|
||||
// fabricated event id would break the provenance rule the ledger enforces.
|
||||
func TestObservationNeedsExactlyOneProvenance(t *testing.T) {
|
||||
sig := DebtSignature(DebtOperational, "x", "", "")
|
||||
both := DebtObservation{EventID: "e1", LegacyRef: "BURNIN.md:F18", Kind: ObservationBlockReason, Signature: sig}
|
||||
neither := DebtObservation{Kind: ObservationBlockReason, Signature: sig}
|
||||
if both.Validate() == nil || neither.Validate() == nil {
|
||||
t.Fatal("exactly one of event_id and legacy_ref must be required")
|
||||
}
|
||||
for _, ok := range []DebtObservation{
|
||||
{EventID: "e1", Kind: ObservationBlockReason, Signature: sig},
|
||||
{LegacyRef: "BURNIN.md:F18", Kind: ObservationBlockReason, Signature: sig},
|
||||
} {
|
||||
if err := ok.Validate(); err != nil {
|
||||
t.Fatalf("valid observation refused: %v", err)
|
||||
}
|
||||
}
|
||||
stale := DebtObservation{EventID: "e1", Kind: ObservationBlockReason, Signature: "v0:operational:x:-:-"}
|
||||
if stale.Validate() == nil {
|
||||
t.Fatal("a signature from another version must be refused")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
package operations
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"orchestra/internal/domain"
|
||||
)
|
||||
|
||||
// DebtCheck is why a debt item may or may not become work. Reasons are listed
|
||||
// rather than summarised, for the same reason SubmissionCheck lists them: "not
|
||||
// eligible" alone sends an operator reading code.
|
||||
type DebtCheck struct {
|
||||
Eligible bool `json:"eligible"`
|
||||
Reasons []string `json:"reasons,omitempty"`
|
||||
}
|
||||
|
||||
// CheckDebtEligibility is the whole promotion rule, as one pure function of a
|
||||
// projected item. It decides only whether a maintenance task may be created.
|
||||
// It never creates one, and it never changes what that task must then pass:
|
||||
// classification as debt changes what gets scheduled, never what gets checked.
|
||||
//
|
||||
// The thresholds differ by class on purpose. Correctness and operational debt
|
||||
// have already cost something measurable. Structural debt needs evidence that
|
||||
// it is causing repeated work rather than merely offending taste. Polish never
|
||||
// promotes itself, or the ledger becomes a permanent cleanup generator.
|
||||
func CheckDebtEligibility(item domain.DebtItem) DebtCheck {
|
||||
recurrence, tasks := item.Recurrence(), item.AffectedTasks()
|
||||
blocked, manual := item.BlockedTasks(), item.ManualInterventions()
|
||||
switch item.Class {
|
||||
case domain.DebtCorrectness:
|
||||
if recurrence >= 1 {
|
||||
return DebtCheck{true, []string{fmt.Sprintf("correctness debt is eligible on first confirmed observation, and has %d", recurrence)}}
|
||||
}
|
||||
return DebtCheck{false, []string{"no confirmed observation"}}
|
||||
case domain.DebtOperational:
|
||||
var why []string
|
||||
if manual >= 1 {
|
||||
why = append(why, fmt.Sprintf("%d manual intervention(s) recorded", manual))
|
||||
}
|
||||
if recurrence >= 3 && tasks >= 2 {
|
||||
why = append(why, fmt.Sprintf("recurred %d times across %d tasks", recurrence, tasks))
|
||||
}
|
||||
if len(why) > 0 {
|
||||
return DebtCheck{true, why}
|
||||
}
|
||||
return DebtCheck{false, []string{
|
||||
fmt.Sprintf("needs 3 occurrences across 2 tasks, or 1 manual intervention; has %d across %d tasks with %d interventions", recurrence, tasks, manual),
|
||||
"manual interventions are not recorded by any event type, so that count reads 0 on every current log",
|
||||
}}
|
||||
case domain.DebtStructural:
|
||||
if recurrence >= 3 {
|
||||
return DebtCheck{true, []string{fmt.Sprintf("%d review findings in this component", recurrence)}}
|
||||
}
|
||||
if blocked >= 2 {
|
||||
return DebtCheck{true, []string{fmt.Sprintf("blocked %d distinct tasks in this component", blocked)}}
|
||||
}
|
||||
return DebtCheck{false, []string{
|
||||
fmt.Sprintf("needs 3 review findings or 2 blocked tasks in one component; has %d findings and %d blocked", recurrence, blocked),
|
||||
}}
|
||||
case domain.DebtPolish:
|
||||
return DebtCheck{false, []string{"polish never promotes itself, an operator promotes it explicitly"}}
|
||||
}
|
||||
return DebtCheck{false, []string{"unknown debt class " + string(item.Class)}}
|
||||
}
|
||||
|
||||
// EligibleDebt filters a projected ledger to what policy would allow to become
|
||||
// work. It returns the check alongside each item so the reasons stay visible.
|
||||
func EligibleDebt(ledger domain.DebtLedger) []DebtCandidate {
|
||||
out := []DebtCandidate{}
|
||||
for _, item := range ledger.Items {
|
||||
if check := CheckDebtEligibility(item); check.Eligible {
|
||||
out = append(out, DebtCandidate{Item: item, Check: check})
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
type DebtCandidate struct {
|
||||
Item domain.DebtItem `json:"item"`
|
||||
Check DebtCheck `json:"check"`
|
||||
}
|
||||
@@ -0,0 +1,68 @@
|
||||
package operations
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"orchestra/internal/domain"
|
||||
)
|
||||
|
||||
func debtItem(class domain.DebtClass, kinds ...struct {
|
||||
kind domain.ObservationKind
|
||||
task string
|
||||
}) domain.DebtItem {
|
||||
item := domain.DebtItem{Class: class}
|
||||
for _, k := range kinds {
|
||||
item.Observations = append(item.Observations, domain.DebtObservation{Kind: k.kind, TaskID: k.task})
|
||||
}
|
||||
return item
|
||||
}
|
||||
|
||||
type obs = struct {
|
||||
kind domain.ObservationKind
|
||||
task string
|
||||
}
|
||||
|
||||
// The thresholds are the whole policy, so each class gets its own case. Polish
|
||||
// is the one that must never pass, or the ledger becomes a cleanup generator.
|
||||
func TestDebtEligibilityPerClass(t *testing.T) {
|
||||
fail := obs{domain.ObservationFailureClass, "t1"}
|
||||
if !CheckDebtEligibility(debtItem(domain.DebtCorrectness, fail)).Eligible {
|
||||
t.Fatal("correctness debt is eligible on first observation")
|
||||
}
|
||||
twice := debtItem(domain.DebtOperational, fail, obs{domain.ObservationFailureClass, "t1"})
|
||||
if CheckDebtEligibility(twice).Eligible {
|
||||
t.Fatal("two occurrences on one task must not qualify as operational debt")
|
||||
}
|
||||
spread := debtItem(domain.DebtOperational, fail,
|
||||
obs{domain.ObservationFailureClass, "t2"}, obs{domain.ObservationFailureClass, "t3"})
|
||||
if !CheckDebtEligibility(spread).Eligible {
|
||||
t.Fatal("three occurrences across three tasks must qualify")
|
||||
}
|
||||
manual := debtItem(domain.DebtOperational, obs{domain.ObservationManualIntervention, "t1"})
|
||||
if !CheckDebtEligibility(manual).Eligible {
|
||||
t.Fatal("one manual intervention must qualify on its own")
|
||||
}
|
||||
blocked := debtItem(domain.DebtStructural,
|
||||
obs{domain.ObservationBlockReason, "t1"}, obs{domain.ObservationBlockReason, "t2"})
|
||||
if !CheckDebtEligibility(blocked).Eligible {
|
||||
t.Fatal("two blocked tasks in one component must qualify as structural")
|
||||
}
|
||||
polish := debtItem(domain.DebtPolish, fail, fail, fail, fail, fail)
|
||||
if CheckDebtEligibility(polish).Eligible {
|
||||
t.Fatal("polish must never promote itself, at any recurrence")
|
||||
}
|
||||
}
|
||||
|
||||
// A refusal has to say what is missing. "Not eligible" alone sends an operator
|
||||
// reading code, which is the mistake SubmissionCheck already documents.
|
||||
func TestDebtRefusalNamesTheMissingEvidence(t *testing.T) {
|
||||
check := CheckDebtEligibility(debtItem(domain.DebtOperational, obs{domain.ObservationFailureClass, "t1"}))
|
||||
if check.Eligible || len(check.Reasons) == 0 {
|
||||
t.Fatalf("want a refusal with reasons, got %+v", check)
|
||||
}
|
||||
joined := strings.Join(check.Reasons, " ")
|
||||
if !strings.Contains(joined, "manual interventions are not recorded") {
|
||||
t.Fatalf("the refusal must say the intervention count is structurally zero: %v", check.Reasons)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,227 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"orchestra/internal/domain"
|
||||
"orchestra/internal/review"
|
||||
)
|
||||
|
||||
// ProjectDebt folds canonical history into a debt ledger. It is a pure
|
||||
// function of the events so it can be tested against a real log, and it writes
|
||||
// nothing: this projection is a read model, not a new kind of truth.
|
||||
//
|
||||
// It deliberately reports what it cannot see. Worker observations and operator
|
||||
// interventions are not carried by any event type today, so a ledger that
|
||||
// stayed silent about them would read as "no operational cost" when it means
|
||||
// "operational cost is unrecorded".
|
||||
func ProjectDebt(events []domain.Event) domain.DebtLedger {
|
||||
return projectDebt(events, nil)
|
||||
}
|
||||
|
||||
// ProjectDebtWithArtifacts is ProjectDebt plus the CAS reads that review
|
||||
// findings need. Findings are sealed as an artifact rather than inlined in the
|
||||
// event, and they are the only evidence structural debt has, so a projection
|
||||
// without them can never produce a structural item at all.
|
||||
func ProjectDebtWithArtifacts(events []domain.Event, read func(ref string) ([]byte, error)) domain.DebtLedger {
|
||||
return projectDebt(events, read)
|
||||
}
|
||||
|
||||
func projectDebt(events []domain.Event, readArtifact func(string) ([]byte, error)) domain.DebtLedger {
|
||||
bySignature := map[string]*domain.DebtItem{}
|
||||
order := []string{}
|
||||
seenKinds := map[domain.ObservationKind]bool{}
|
||||
|
||||
add := func(class domain.DebtClass, sig string, o domain.DebtObservation, symptom string, sev review.Severity) {
|
||||
o.Signature = sig
|
||||
if o.Validate() != nil {
|
||||
return
|
||||
}
|
||||
seenKinds[o.Kind] = true
|
||||
item, ok := bySignature[sig]
|
||||
if !ok {
|
||||
item = &domain.DebtItem{
|
||||
// The id is the signature, not an Fxx number. This projection
|
||||
// produces candidates from history; adopting a curated id is a
|
||||
// later, deliberate step, and minting one here would collide
|
||||
// with the hand-written ledger.
|
||||
ID: sig, Class: class, Status: domain.DebtObserved,
|
||||
Symptom: symptom, Signatures: []string{sig}, Severity: sev,
|
||||
}
|
||||
bySignature[sig] = item
|
||||
order = append(order, sig)
|
||||
}
|
||||
item.Observations = append(item.Observations, o)
|
||||
for _, p := range o.Paths {
|
||||
if !contains(item.Paths, p) {
|
||||
item.Paths = append(item.Paths, p)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for _, e := range events {
|
||||
var p map[string]any
|
||||
if len(e.Payload) > 0 && json.Unmarshal(e.Payload, &p) != nil {
|
||||
continue
|
||||
}
|
||||
base := domain.DebtObservation{EventID: e.ID, TaskID: e.TaskID, At: e.At}
|
||||
harness, _ := p["harness_id"].(string)
|
||||
switch e.Type {
|
||||
case "TaskBlocked":
|
||||
reason, _ := p["block_reason"].(string)
|
||||
if reason == "" {
|
||||
reason = string(domain.InferBlockReason(str(p["blocker"])))
|
||||
}
|
||||
class, ok := domain.DebtClassForBlockReason(domain.BlockReason(reason))
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
o := base
|
||||
o.Kind = domain.ObservationBlockReason
|
||||
o.Detail = str(p["blocker"])
|
||||
add(class, domain.DebtSignature(class, reason, harness, "lease"), o,
|
||||
"tasks are blocked with "+reason, review.Important)
|
||||
case "TaskFailed", "TaskReleased":
|
||||
// A release carries a failure class only when it is a reclaim. A
|
||||
// handoff release is ordinary progress and never debt.
|
||||
failure, _ := p["failure_class"].(string)
|
||||
if failure == "" {
|
||||
failure, _ = p["reason"].(string)
|
||||
}
|
||||
if e.Type == "TaskReleased" && str(p["handoff_ref"]) != "" {
|
||||
continue
|
||||
}
|
||||
class, ok := domain.DebtClassForFailureClass(failure)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
o := base
|
||||
o.Kind = domain.ObservationFailureClass
|
||||
o.Detail = str(p["last_error"])
|
||||
add(class, domain.DebtSignature(class, failure, harness, "lease"), o,
|
||||
"tasks end in "+failure, review.Important)
|
||||
case domain.EventPlanMismatchRecorded:
|
||||
o := base
|
||||
o.Kind = domain.ObservationPlanMismatch
|
||||
o.Detail = str(p["observed"])
|
||||
o.Paths = strList(p["evidence"])
|
||||
component := firstComponent(o.Paths)
|
||||
add(domain.DebtCorrectness,
|
||||
domain.DebtSignature(domain.DebtCorrectness, "plan_mismatch", "", component), o,
|
||||
"the repository contradicts sealed plans", review.Important)
|
||||
case domain.EventReviewRecorded:
|
||||
// Repeated findings in one component are the only mechanical
|
||||
// evidence that structure is costing future changes. One finding
|
||||
// is a review doing its job.
|
||||
if readArtifact == nil {
|
||||
break
|
||||
}
|
||||
raw, err := readArtifact(str(p["artifact_ref"]))
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
result, err := review.Decode(raw)
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
for _, f := range result.Findings {
|
||||
component := domain.DebtComponent(f.File)
|
||||
o := base
|
||||
o.Kind = domain.ObservationReviewFinding
|
||||
o.Detail = f.Claim
|
||||
if f.File != "" {
|
||||
o.Paths = []string{f.File}
|
||||
}
|
||||
add(domain.DebtStructural,
|
||||
domain.DebtSignature(domain.DebtStructural, string(f.Severity), "", component), o,
|
||||
"review keeps finding "+string(f.Severity)+" issues in "+component, f.Severity)
|
||||
}
|
||||
case domain.EventDeferredFindingRecorded:
|
||||
o := base
|
||||
o.Kind = domain.ObservationDeferredFinding
|
||||
o.Detail = str(p["summary"])
|
||||
// Polish until something demonstrates a cost. A discovery nobody
|
||||
// has paid for yet is not debt with a priority.
|
||||
add(domain.DebtPolish,
|
||||
domain.DebtSignature(domain.DebtPolish, "deferred_finding", "", ""), o,
|
||||
"out-of-scope discoveries recorded and unaddressed", review.Minor)
|
||||
}
|
||||
}
|
||||
|
||||
items := make([]domain.DebtItem, 0, len(order))
|
||||
for _, sig := range order {
|
||||
items = append(items, *bySignature[sig])
|
||||
}
|
||||
sort.SliceStable(items, func(i, j int) bool {
|
||||
if items[i].Recurrence() != items[j].Recurrence() {
|
||||
return items[i].Recurrence() > items[j].Recurrence()
|
||||
}
|
||||
return items[i].ID < items[j].ID
|
||||
})
|
||||
return domain.DebtLedger{Items: items, Gaps: debtGaps(seenKinds), Events: len(events)}
|
||||
}
|
||||
|
||||
// debtGaps separates two different silences. A kind no event type carries is a
|
||||
// hole in the system. A kind the log could carry and does not is a fact about
|
||||
// this history.
|
||||
func debtGaps(seen map[domain.ObservationKind]bool) []domain.EvidenceGap {
|
||||
gaps := []domain.EvidenceGap{
|
||||
{Kind: domain.ObservationManualIntervention, Durable: false,
|
||||
Reason: "no event type records an operator repair, so every manual recovery is invisible to this ledger"},
|
||||
{Kind: domain.ObservationWorkerFailure, Durable: false,
|
||||
Reason: "worker observations live in worker memory and reach the coordinator only inside WorkerHealth, which is not persisted"},
|
||||
}
|
||||
for _, k := range []domain.ObservationKind{
|
||||
domain.ObservationBlockReason, domain.ObservationFailureClass,
|
||||
domain.ObservationReviewFinding, domain.ObservationPlanMismatch,
|
||||
domain.ObservationDeferredFinding,
|
||||
} {
|
||||
if !seen[k] {
|
||||
gaps = append(gaps, domain.EvidenceGap{Kind: k, Durable: true,
|
||||
Reason: "carried by the log, but this history contains none"})
|
||||
}
|
||||
}
|
||||
return gaps
|
||||
}
|
||||
|
||||
// DebtLedger projects the whole log. Read-only, like every other projection
|
||||
// the coordinator serves.
|
||||
func (s *Store) DebtLedger() domain.DebtLedger {
|
||||
return ProjectDebtWithArtifacts(s.Events(0), s.Artifact)
|
||||
}
|
||||
|
||||
func str(v any) string { s, _ := v.(string); return s }
|
||||
|
||||
func strList(v any) []string {
|
||||
list, ok := v.([]any)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
out := make([]string, 0, len(list))
|
||||
for _, item := range list {
|
||||
if s := strings.TrimSpace(str(item)); s != "" {
|
||||
out = append(out, s)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func firstComponent(paths []string) string {
|
||||
for _, p := range paths {
|
||||
if c := domain.DebtComponent(p); c != "" {
|
||||
return c
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func contains(list []string, s string) bool {
|
||||
for _, item := range list {
|
||||
if item == s {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"orchestra/internal/domain"
|
||||
)
|
||||
|
||||
func debtEvent(id, typ, task string, payload map[string]any) domain.Event {
|
||||
b, _ := json.Marshal(payload)
|
||||
return domain.Event{ID: id, Type: typ, TaskID: task, Payload: b, At: time.Now().UTC()}
|
||||
}
|
||||
|
||||
// The opencode shape from run 14: one harness failing the same way across
|
||||
// several tasks. That is the recurrence the ledger exists to notice.
|
||||
func TestProjectDebtGroupsOneFailureShapeAcrossTasks(t *testing.T) {
|
||||
events := []domain.Event{
|
||||
debtEvent("e1", "TaskFailed", "t1", map[string]any{"failure_class": "retry_limit", "harness_id": "workpc-opencode"}),
|
||||
debtEvent("e2", "TaskFailed", "t2", map[string]any{"failure_class": "retry_limit", "harness_id": "workpc-opencode"}),
|
||||
debtEvent("e3", "TaskFailed", "t3", map[string]any{"failure_class": "retry_limit", "harness_id": "workpc-opencode"}),
|
||||
debtEvent("e4", "TaskFailed", "t4", map[string]any{"failure_class": "retry_limit", "harness_id": "workpc-claude"}),
|
||||
// A handoff release is ordinary progress and must not become debt.
|
||||
debtEvent("e5", "TaskReleased", "t5", map[string]any{"handoff_ref": "sha256:abc", "harness_id": "workpc-claude"}),
|
||||
}
|
||||
ledger := ProjectDebt(events)
|
||||
if len(ledger.Items) != 2 {
|
||||
t.Fatalf("want one item per harness, got %d: %+v", len(ledger.Items), ledger.Items)
|
||||
}
|
||||
top := ledger.Items[0]
|
||||
if top.Recurrence() != 3 || top.AffectedTasks() != 3 {
|
||||
t.Fatalf("recurrence %d across %d tasks, want 3 and 3", top.Recurrence(), top.AffectedTasks())
|
||||
}
|
||||
if top.Class != domain.DebtOperational {
|
||||
t.Fatalf("class %q, want operational", top.Class)
|
||||
}
|
||||
if got := top.ID; got != "v1:operational:retry_limit:workpc-opencode:lease" {
|
||||
t.Fatalf("signature %q", got)
|
||||
}
|
||||
for _, o := range top.Observations {
|
||||
if o.EventID == "" || o.LegacyRef != "" {
|
||||
t.Fatalf("observation lost its event provenance: %+v", o)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A lifecycle stop is the system working. Waiting for a human is not debt, and
|
||||
// counting it would drown the real signal.
|
||||
func TestProjectDebtIgnoresOrdinaryLifecycleStops(t *testing.T) {
|
||||
events := []domain.Event{
|
||||
debtEvent("e1", "TaskBlocked", "t1", map[string]any{"block_reason": "human_decision", "blocker": "which route"}),
|
||||
debtEvent("e2", "TaskBlocked", "t2", map[string]any{"block_reason": "trajectory_gate", "blocker": "confirm plan"}),
|
||||
debtEvent("e3", "TaskBlocked", "t3", map[string]any{"block_reason": "lease_expired", "blocker": "lease expired"}),
|
||||
}
|
||||
ledger := ProjectDebt(events)
|
||||
if len(ledger.Items) != 1 || ledger.Items[0].Class != domain.DebtOperational {
|
||||
t.Fatalf("want only the lease_expired item, got %+v", ledger.Items)
|
||||
}
|
||||
}
|
||||
|
||||
// Incompleteness is part of the result. A ledger that stays silent about what
|
||||
// it cannot record reads as "no operator cost" when it means "operator cost is
|
||||
// not recorded anywhere".
|
||||
func TestProjectDebtReportsWhatItCannotSee(t *testing.T) {
|
||||
ledger := ProjectDebt(nil)
|
||||
var manual, worker bool
|
||||
for _, g := range ledger.Gaps {
|
||||
if g.Durable {
|
||||
continue
|
||||
}
|
||||
switch g.Kind {
|
||||
case domain.ObservationManualIntervention:
|
||||
manual = true
|
||||
case domain.ObservationWorkerFailure:
|
||||
worker = true
|
||||
}
|
||||
}
|
||||
if !manual || !worker {
|
||||
t.Fatalf("the two known holes must always be reported: %+v", ledger.Gaps)
|
||||
}
|
||||
for _, g := range ledger.Gaps {
|
||||
if g.Reason == "" {
|
||||
t.Fatalf("gap %q has no reason", g.Kind)
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user