Reconcile docs with reality; fix module graph, token compare, health #1
+58
-11
@@ -98,7 +98,9 @@ No new subsystem. Two event types on the existing spine, one projection beside
|
||||
// DebtObservation is one piece of counted evidence, always pointing at the
|
||||
// event that produced it. Provenance is an event id, never prose.
|
||||
type DebtObservation struct {
|
||||
EventID string `json:"event_id"`
|
||||
// Exactly one of EventID and LegacyRef. See section 8.
|
||||
EventID string `json:"event_id,omitempty"`
|
||||
LegacyRef string `json:"legacy_ref,omitempty"`
|
||||
TaskID string `json:"task_id,omitempty"`
|
||||
Kind string `json:"kind"` // block_reason, failure_class, review_finding,
|
||||
// plan_mismatch, manual_intervention, deferred_finding
|
||||
@@ -152,6 +154,18 @@ structural class + finding severity + normalized component
|
||||
|
||||
Exact signature match attaches automatically. That is the only automatic path.
|
||||
|
||||
**Signatures are versioned, in the string itself.**
|
||||
|
||||
```text
|
||||
v1:operational:lease_not_renewed:workpc-opencode:federation
|
||||
```
|
||||
|
||||
Normalization rules will change. Without a version, changing them silently
|
||||
regroups every historical observation, and the recurrence counts that drive
|
||||
eligibility move underneath the items that already used them. A `v2` signature
|
||||
never matches a `v1` one, so old evidence keeps the grouping it was counted
|
||||
under.
|
||||
|
||||
**A model may propose, never merge.** Clustering emits `DebtMergeSuggested`
|
||||
carrying both item ids and its reasoning. The merge is an operator action or a
|
||||
policy threshold, recorded as `DebtItemUpdated`.
|
||||
@@ -245,6 +259,19 @@ One-time import, by hand, with a class assigned per row. Each becomes a
|
||||
`DebtItem` with `Status: repaired`, its commit as `RepairCommit`, and one
|
||||
observation citing the ledger.
|
||||
|
||||
**Imported observations carry legacy provenance, never a fabricated event id.**
|
||||
The Fxx history predates the capability that would have produced an event, and
|
||||
inventing one would break the provenance rule the ledger exists to enforce.
|
||||
|
||||
```go
|
||||
EventID string `json:"event_id,omitempty"`
|
||||
LegacyRef string `json:"legacy_ref,omitempty"` // "BURNIN.md:F18"
|
||||
```
|
||||
|
||||
Exactly one is required. A reader can then tell a counted fact from an imported
|
||||
claim at a glance, and the counts that drive eligibility can exclude imported
|
||||
evidence if that turns out to matter.
|
||||
|
||||
**Keep the namespace.** New items continue at F62. Two numbering schemes would
|
||||
be the first structural debt the ledger itself creates.
|
||||
|
||||
@@ -271,21 +298,41 @@ represent the history this project already has.
|
||||
schema commitment.**
|
||||
|
||||
```text
|
||||
internal/operations/debt.go DebtItem, signature, projection over s.Events(0)
|
||||
CheckDebtEligibility
|
||||
GET /v1/debt read-only, tui surface
|
||||
internal/domain/debt.go DebtItem, DebtObservation, signature, classification
|
||||
internal/store/debt_projection.go the fold over s.Events(0), and the gap report
|
||||
internal/operations/debt.go CheckDebtEligibility, and later the actions
|
||||
GET /v1/debt read-only, tui surface
|
||||
```
|
||||
|
||||
The proof is that the current log already contains runs 10 to 14. The projection
|
||||
must independently surface, from history alone:
|
||||
The projected type stays out of `operations`. Baking a read model into the
|
||||
command layer in the first slice is the mistake that would be hardest to undo
|
||||
later. `operations` owns actions: `CheckDebtEligibility` now, and
|
||||
`SuggestDebtMerge`, `PromoteDebtItem` and `ScheduleDebtRepair` when they exist.
|
||||
|
||||
- the release loop as operational debt, with its 409 recurrence and two manual
|
||||
cleanups
|
||||
- the opencode adapter gap as operational debt, across three failed attempts on
|
||||
one harness
|
||||
The proof is what the projection can and cannot recover from canonical history.
|
||||
Section 2 already says worker observations are not durable and operator
|
||||
interventions are unrecorded, so demanding their reconstruction would be asking
|
||||
the projection to invent evidence.
|
||||
|
||||
The first burn-in is therefore three requirements, not one:
|
||||
|
||||
1. Recover every debt signal that canonical history actually encodes.
|
||||
2. Report, explicitly and per kind, where known debt cannot be reconstructed.
|
||||
3. Never infer a missing observation from `BURNIN.md`.
|
||||
|
||||
Incompleteness is part of the result, not a failure. The 409 loop lived in the
|
||||
F18 worker ring rather than in an event, so the ledger should say it has no
|
||||
durable evidence for that shape. That statement is what makes slice two
|
||||
necessary, measurably rather than by assertion.
|
||||
|
||||
What the projection should recover from the log alone:
|
||||
|
||||
- repeated `lease_expired` and `retry_limit` on one harness, which is the
|
||||
opencode failure shape across four tasks
|
||||
- the retry-idleness dynamic, attached to the tasks that expired
|
||||
- review findings grouped by component, if any repeat
|
||||
|
||||
A model that cannot reproduce debt already known is wrong, and no schema has
|
||||
A model that cannot represent debt already known is wrong, and no schema has
|
||||
been committed to yet. That is the cheapest place to find out.
|
||||
|
||||
**Slice two** makes worker observations durable, because that is the input the
|
||||
|
||||
@@ -445,6 +445,17 @@ func main() {
|
||||
}
|
||||
json.NewEncoder(w).Encode(s.Events(n))
|
||||
})
|
||||
mux.HandleFunc("/v1/debt", func(w http.ResponseWriter, r *http.Request) {
|
||||
// Read-only, and deliberately so. This projection is evidence about
|
||||
// history, not a new kind of truth: nothing here writes an event,
|
||||
// schedules work, or decides that something is worth repairing.
|
||||
ledger := s.DebtLedger()
|
||||
out := struct {
|
||||
domain.DebtLedger
|
||||
Eligible []operations.DebtCandidate `json:"eligible"`
|
||||
}{ledger, operations.EligibleDebt(ledger)}
|
||||
json.NewEncoder(w).Encode(out)
|
||||
})
|
||||
mux.HandleFunc("/v1/handoffs", func(w http.ResponseWriter, r *http.Request) {
|
||||
out := make([]domain.Event, 0)
|
||||
for _, e := range s.Events(0) {
|
||||
|
||||
@@ -0,0 +1,260 @@
|
||||
package domain
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"orchestra/internal/review"
|
||||
)
|
||||
|
||||
// Debt is the projected read model for accumulated cost: the defects,
|
||||
// workarounds and operational burdens that real tasks keep paying for. It is
|
||||
// derived from the event log rather than written by hand, so a claim in the
|
||||
// ledger can always be checked against the events that produced it.
|
||||
//
|
||||
// Four classes, and no more. A fifth invented at runtime makes the priority
|
||||
// model meaningless, because the thresholds in operations.CheckDebtEligibility
|
||||
// are stated per class.
|
||||
type DebtClass string
|
||||
|
||||
const (
|
||||
// DebtCorrectness is behaviour that is wrong or violates an invariant.
|
||||
DebtCorrectness DebtClass = "correctness"
|
||||
// DebtOperational works, but repeatedly costs time to diagnose, recover,
|
||||
// deploy, observe or operate.
|
||||
DebtOperational DebtClass = "operational"
|
||||
// DebtStructural is duplication or architecture demonstrably raising the
|
||||
// cost of future changes.
|
||||
DebtStructural DebtClass = "structural"
|
||||
// DebtPolish has no demonstrated cost yet, and never promotes itself.
|
||||
DebtPolish DebtClass = "polish"
|
||||
)
|
||||
|
||||
func (c DebtClass) Valid() bool {
|
||||
switch c {
|
||||
case DebtCorrectness, DebtOperational, DebtStructural, DebtPolish:
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
type DebtStatus string
|
||||
|
||||
const (
|
||||
DebtObserved DebtStatus = "observed"
|
||||
DebtEligible DebtStatus = "eligible"
|
||||
DebtScheduled DebtStatus = "scheduled"
|
||||
DebtRepaired DebtStatus = "repaired"
|
||||
DebtWithdrawn DebtStatus = "withdrawn"
|
||||
)
|
||||
|
||||
// ObservationKind names where one piece of evidence came from. It is the kind
|
||||
// of the source fact, never an interpretation of it.
|
||||
type ObservationKind string
|
||||
|
||||
const (
|
||||
ObservationBlockReason ObservationKind = "block_reason"
|
||||
ObservationFailureClass ObservationKind = "failure_class"
|
||||
ObservationReviewFinding ObservationKind = "review_finding"
|
||||
ObservationPlanMismatch ObservationKind = "plan_mismatch"
|
||||
ObservationDeferredFinding ObservationKind = "deferred_finding"
|
||||
ObservationManualIntervention ObservationKind = "manual_intervention"
|
||||
ObservationWorkerFailure ObservationKind = "worker_observation"
|
||||
)
|
||||
|
||||
// SignatureVersion prefixes every signature this build produces. Normalization
|
||||
// rules will change, and without a version a change silently regroups every
|
||||
// historical observation, moving the recurrence counts that eligibility was
|
||||
// already decided on. A v2 signature never matches a v1 one, so old evidence
|
||||
// keeps the grouping it was counted under.
|
||||
const SignatureVersion = "v1"
|
||||
|
||||
// DebtSignature is the mechanical dedup key. Every part is a typed fact, never
|
||||
// prose, because exact signature match is the only path that attaches evidence
|
||||
// without a human. An empty part becomes "-" so the arity never varies.
|
||||
func DebtSignature(class DebtClass, reason, scope, component string) string {
|
||||
part := func(s string) string {
|
||||
s = strings.TrimSpace(strings.ToLower(s))
|
||||
s = strings.ReplaceAll(s, ":", "_")
|
||||
if s == "" {
|
||||
return "-"
|
||||
}
|
||||
return s
|
||||
}
|
||||
return strings.Join([]string{SignatureVersion, part(string(class)), part(reason), part(scope), part(component)}, ":")
|
||||
}
|
||||
|
||||
// DebtComponent reduces a repository path to the unit that owns it. Two
|
||||
// segments is the whole rule: it keeps internal/store distinct from
|
||||
// internal/herdr without splitting one package across files.
|
||||
func DebtComponent(path string) string {
|
||||
path = strings.TrimSpace(strings.Trim(path, "/"))
|
||||
if path == "" {
|
||||
return ""
|
||||
}
|
||||
parts := strings.Split(path, "/")
|
||||
if len(parts) == 1 {
|
||||
return parts[0]
|
||||
}
|
||||
return parts[0] + "/" + parts[1]
|
||||
}
|
||||
|
||||
// DebtObservation is one piece of counted evidence. Provenance is required and
|
||||
// is either an event id or, for history imported from a markdown ledger that
|
||||
// predates this projection, an explicit legacy reference. Exactly one, because
|
||||
// a fabricated event id would break the rule the ledger exists to enforce.
|
||||
type DebtObservation struct {
|
||||
EventID string `json:"event_id,omitempty"`
|
||||
LegacyRef string `json:"legacy_ref,omitempty"`
|
||||
TaskID string `json:"task_id,omitempty"`
|
||||
Kind ObservationKind `json:"kind"`
|
||||
Signature string `json:"signature"`
|
||||
Detail string `json:"detail,omitempty"`
|
||||
Paths []string `json:"paths,omitempty"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
func (o DebtObservation) Validate() error {
|
||||
if (o.EventID == "") == (o.LegacyRef == "") {
|
||||
return fmt.Errorf("observation needs exactly one of event_id and legacy_ref")
|
||||
}
|
||||
if o.Kind == "" {
|
||||
return fmt.Errorf("observation kind required")
|
||||
}
|
||||
if !strings.HasPrefix(o.Signature, SignatureVersion+":") {
|
||||
return fmt.Errorf("observation signature %q is not %s", o.Signature, SignatureVersion)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// DebtItem groups observations that share a signature. Every count the
|
||||
// priority model needs is derived from Observations rather than stored, so a
|
||||
// count can never drift from the log that justifies it.
|
||||
type DebtItem struct {
|
||||
ID string `json:"id"`
|
||||
Class DebtClass `json:"class"`
|
||||
Status DebtStatus `json:"status"`
|
||||
Symptom string `json:"symptom"`
|
||||
Consequence string `json:"consequence,omitempty"`
|
||||
Severity review.Severity `json:"severity,omitempty"`
|
||||
Signatures []string `json:"signatures"`
|
||||
Paths []string `json:"paths,omitempty"`
|
||||
Observations []DebtObservation `json:"observations"`
|
||||
IntroducedIn string `json:"introduced_in,omitempty"`
|
||||
RepairBoundary string `json:"repair_boundary,omitempty"`
|
||||
RepairTask string `json:"repair_task,omitempty"`
|
||||
RepairCommit string `json:"repair_commit,omitempty"`
|
||||
}
|
||||
|
||||
// Recurrence is how many times this shape has been seen.
|
||||
func (d DebtItem) Recurrence() int { return len(d.Observations) }
|
||||
|
||||
// AffectedTasks is how broadly the shape has spread. Distinct tasks, because
|
||||
// one task failing ten times is weaker evidence than ten tasks failing once.
|
||||
func (d DebtItem) AffectedTasks() int {
|
||||
seen := map[string]bool{}
|
||||
for _, o := range d.Observations {
|
||||
if o.TaskID != "" {
|
||||
seen[o.TaskID] = true
|
||||
}
|
||||
}
|
||||
return len(seen)
|
||||
}
|
||||
|
||||
// BlockedTasks counts distinct tasks this shape actually stopped.
|
||||
func (d DebtItem) BlockedTasks() int {
|
||||
seen := map[string]bool{}
|
||||
for _, o := range d.Observations {
|
||||
if o.Kind == ObservationBlockReason && o.TaskID != "" {
|
||||
seen[o.TaskID] = true
|
||||
}
|
||||
}
|
||||
return len(seen)
|
||||
}
|
||||
|
||||
// ManualInterventions counts recorded operator repairs. It reads zero on any
|
||||
// log written before that recording exists, which is why the ledger reports it
|
||||
// as an evidence gap rather than as an absence of operator cost.
|
||||
func (d DebtItem) ManualInterventions() int {
|
||||
n := 0
|
||||
for _, o := range d.Observations {
|
||||
if o.Kind == ObservationManualIntervention {
|
||||
n++
|
||||
}
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// Components is the breadth of the shape across the tree.
|
||||
func (d DebtItem) Components() []string {
|
||||
seen := map[string]bool{}
|
||||
out := []string{}
|
||||
for _, o := range d.Observations {
|
||||
for _, p := range o.Paths {
|
||||
if c := DebtComponent(p); c != "" && !seen[c] {
|
||||
seen[c] = true
|
||||
out = append(out, c)
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Imported reports whether any evidence here came from a markdown ledger
|
||||
// rather than from an event. A reader can then tell a counted fact from an
|
||||
// imported claim without opening the observations.
|
||||
func (d DebtItem) Imported() bool {
|
||||
for _, o := range d.Observations {
|
||||
if o.LegacyRef != "" {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// EvidenceGap is a debt signal the ledger knows it cannot see. Reporting the
|
||||
// gap is the point: a ledger that silently omits what it cannot record reads
|
||||
// as "no operator cost" when it means "operator cost is not recorded".
|
||||
type EvidenceGap struct {
|
||||
Kind ObservationKind `json:"kind"`
|
||||
Reason string `json:"reason"`
|
||||
// Durable is false when no event type carries this evidence at all. It is
|
||||
// true when the log could carry it and this particular log does not.
|
||||
Durable bool `json:"durable"`
|
||||
}
|
||||
|
||||
// DebtLedger is the whole read model: what history establishes, and what it
|
||||
// cannot.
|
||||
type DebtLedger struct {
|
||||
Items []DebtItem `json:"items"`
|
||||
Gaps []EvidenceGap `json:"gaps"`
|
||||
// Events is how many log entries the projection folded, so a caller can
|
||||
// tell an empty ledger from an unread log.
|
||||
Events int `json:"events"`
|
||||
}
|
||||
|
||||
// DebtClassForBlockReason maps a typed block reason to a class. A reason that
|
||||
// is a normal lifecycle stop rather than a cost returns false: waiting for a
|
||||
// human decision is the system working, not debt.
|
||||
func DebtClassForBlockReason(r BlockReason) (DebtClass, bool) {
|
||||
switch r {
|
||||
case BlockReasonLeaseFailure, BlockReasonWorkerOffline, BlockReasonLeaseExpired, BlockReasonSystem:
|
||||
return DebtOperational, true
|
||||
case BlockReasonHandoffValidation, BlockReasonPlanMismatch:
|
||||
return DebtCorrectness, true
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
|
||||
// DebtClassForFailureClass maps a worker failure class to a class. Only the
|
||||
// classes a worker actually emits are listed; an unknown one is not guessed at.
|
||||
func DebtClassForFailureClass(f string) (DebtClass, bool) {
|
||||
switch f {
|
||||
case "retry_limit", "launch_failed", "launch_transient", "launch_uncertain", "prompt_not_submitted", "lease_expired":
|
||||
return DebtOperational, true
|
||||
case "invalid_handoff":
|
||||
return DebtCorrectness, true
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
package domain
|
||||
|
||||
import "testing"
|
||||
|
||||
// Normalization rules will change. A v2 signature must never match a v1 one,
|
||||
// or changing them silently regroups history and moves the recurrence counts
|
||||
// eligibility was already decided on.
|
||||
func TestSignatureCarriesItsVersion(t *testing.T) {
|
||||
sig := DebtSignature(DebtOperational, "lease_expired", "workpc-opencode", "internal/herdr")
|
||||
if sig != "v1:operational:lease_expired:workpc-opencode:internal/herdr" {
|
||||
t.Fatalf("signature %q", sig)
|
||||
}
|
||||
// Arity never varies, so a missing part cannot shift the fields left.
|
||||
if got := DebtSignature(DebtPolish, "deferred_finding", "", ""); got != "v1:polish:deferred_finding:-:-" {
|
||||
t.Fatalf("empty parts not padded: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestComponentIsTwoSegments(t *testing.T) {
|
||||
for path, want := range map[string]string{
|
||||
"internal/store/store.go": "internal/store",
|
||||
"internal/herdr/adapter.go": "internal/herdr",
|
||||
"cmd/orchestra-worker/main.go": "cmd/orchestra-worker",
|
||||
"BURNIN.md": "BURNIN.md",
|
||||
"": "",
|
||||
} {
|
||||
if got := DebtComponent(path); got != want {
|
||||
t.Fatalf("component(%q) = %q, want %q", path, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Imported history predates the events that would have justified it. A
|
||||
// fabricated event id would break the provenance rule the ledger enforces.
|
||||
func TestObservationNeedsExactlyOneProvenance(t *testing.T) {
|
||||
sig := DebtSignature(DebtOperational, "x", "", "")
|
||||
both := DebtObservation{EventID: "e1", LegacyRef: "BURNIN.md:F18", Kind: ObservationBlockReason, Signature: sig}
|
||||
neither := DebtObservation{Kind: ObservationBlockReason, Signature: sig}
|
||||
if both.Validate() == nil || neither.Validate() == nil {
|
||||
t.Fatal("exactly one of event_id and legacy_ref must be required")
|
||||
}
|
||||
for _, ok := range []DebtObservation{
|
||||
{EventID: "e1", Kind: ObservationBlockReason, Signature: sig},
|
||||
{LegacyRef: "BURNIN.md:F18", Kind: ObservationBlockReason, Signature: sig},
|
||||
} {
|
||||
if err := ok.Validate(); err != nil {
|
||||
t.Fatalf("valid observation refused: %v", err)
|
||||
}
|
||||
}
|
||||
stale := DebtObservation{EventID: "e1", Kind: ObservationBlockReason, Signature: "v0:operational:x:-:-"}
|
||||
if stale.Validate() == nil {
|
||||
t.Fatal("a signature from another version must be refused")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
package operations
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"orchestra/internal/domain"
|
||||
)
|
||||
|
||||
// DebtCheck is why a debt item may or may not become work. Reasons are listed
|
||||
// rather than summarised, for the same reason SubmissionCheck lists them: "not
|
||||
// eligible" alone sends an operator reading code.
|
||||
type DebtCheck struct {
|
||||
Eligible bool `json:"eligible"`
|
||||
Reasons []string `json:"reasons,omitempty"`
|
||||
}
|
||||
|
||||
// CheckDebtEligibility is the whole promotion rule, as one pure function of a
|
||||
// projected item. It decides only whether a maintenance task may be created.
|
||||
// It never creates one, and it never changes what that task must then pass:
|
||||
// classification as debt changes what gets scheduled, never what gets checked.
|
||||
//
|
||||
// The thresholds differ by class on purpose. Correctness and operational debt
|
||||
// have already cost something measurable. Structural debt needs evidence that
|
||||
// it is causing repeated work rather than merely offending taste. Polish never
|
||||
// promotes itself, or the ledger becomes a permanent cleanup generator.
|
||||
func CheckDebtEligibility(item domain.DebtItem) DebtCheck {
|
||||
recurrence, tasks := item.Recurrence(), item.AffectedTasks()
|
||||
blocked, manual := item.BlockedTasks(), item.ManualInterventions()
|
||||
switch item.Class {
|
||||
case domain.DebtCorrectness:
|
||||
if recurrence >= 1 {
|
||||
return DebtCheck{true, []string{fmt.Sprintf("correctness debt is eligible on first confirmed observation, and has %d", recurrence)}}
|
||||
}
|
||||
return DebtCheck{false, []string{"no confirmed observation"}}
|
||||
case domain.DebtOperational:
|
||||
var why []string
|
||||
if manual >= 1 {
|
||||
why = append(why, fmt.Sprintf("%d manual intervention(s) recorded", manual))
|
||||
}
|
||||
if recurrence >= 3 && tasks >= 2 {
|
||||
why = append(why, fmt.Sprintf("recurred %d times across %d tasks", recurrence, tasks))
|
||||
}
|
||||
if len(why) > 0 {
|
||||
return DebtCheck{true, why}
|
||||
}
|
||||
return DebtCheck{false, []string{
|
||||
fmt.Sprintf("needs 3 occurrences across 2 tasks, or 1 manual intervention; has %d across %d tasks with %d interventions", recurrence, tasks, manual),
|
||||
"manual interventions are not recorded by any event type, so that count reads 0 on every current log",
|
||||
}}
|
||||
case domain.DebtStructural:
|
||||
if recurrence >= 3 {
|
||||
return DebtCheck{true, []string{fmt.Sprintf("%d review findings in this component", recurrence)}}
|
||||
}
|
||||
if blocked >= 2 {
|
||||
return DebtCheck{true, []string{fmt.Sprintf("blocked %d distinct tasks in this component", blocked)}}
|
||||
}
|
||||
return DebtCheck{false, []string{
|
||||
fmt.Sprintf("needs 3 review findings or 2 blocked tasks in one component; has %d findings and %d blocked", recurrence, blocked),
|
||||
}}
|
||||
case domain.DebtPolish:
|
||||
return DebtCheck{false, []string{"polish never promotes itself, an operator promotes it explicitly"}}
|
||||
}
|
||||
return DebtCheck{false, []string{"unknown debt class " + string(item.Class)}}
|
||||
}
|
||||
|
||||
// EligibleDebt filters a projected ledger to what policy would allow to become
|
||||
// work. It returns the check alongside each item so the reasons stay visible.
|
||||
func EligibleDebt(ledger domain.DebtLedger) []DebtCandidate {
|
||||
out := []DebtCandidate{}
|
||||
for _, item := range ledger.Items {
|
||||
if check := CheckDebtEligibility(item); check.Eligible {
|
||||
out = append(out, DebtCandidate{Item: item, Check: check})
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
type DebtCandidate struct {
|
||||
Item domain.DebtItem `json:"item"`
|
||||
Check DebtCheck `json:"check"`
|
||||
}
|
||||
@@ -0,0 +1,68 @@
|
||||
package operations
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"orchestra/internal/domain"
|
||||
)
|
||||
|
||||
func debtItem(class domain.DebtClass, kinds ...struct {
|
||||
kind domain.ObservationKind
|
||||
task string
|
||||
}) domain.DebtItem {
|
||||
item := domain.DebtItem{Class: class}
|
||||
for _, k := range kinds {
|
||||
item.Observations = append(item.Observations, domain.DebtObservation{Kind: k.kind, TaskID: k.task})
|
||||
}
|
||||
return item
|
||||
}
|
||||
|
||||
type obs = struct {
|
||||
kind domain.ObservationKind
|
||||
task string
|
||||
}
|
||||
|
||||
// The thresholds are the whole policy, so each class gets its own case. Polish
|
||||
// is the one that must never pass, or the ledger becomes a cleanup generator.
|
||||
func TestDebtEligibilityPerClass(t *testing.T) {
|
||||
fail := obs{domain.ObservationFailureClass, "t1"}
|
||||
if !CheckDebtEligibility(debtItem(domain.DebtCorrectness, fail)).Eligible {
|
||||
t.Fatal("correctness debt is eligible on first observation")
|
||||
}
|
||||
twice := debtItem(domain.DebtOperational, fail, obs{domain.ObservationFailureClass, "t1"})
|
||||
if CheckDebtEligibility(twice).Eligible {
|
||||
t.Fatal("two occurrences on one task must not qualify as operational debt")
|
||||
}
|
||||
spread := debtItem(domain.DebtOperational, fail,
|
||||
obs{domain.ObservationFailureClass, "t2"}, obs{domain.ObservationFailureClass, "t3"})
|
||||
if !CheckDebtEligibility(spread).Eligible {
|
||||
t.Fatal("three occurrences across three tasks must qualify")
|
||||
}
|
||||
manual := debtItem(domain.DebtOperational, obs{domain.ObservationManualIntervention, "t1"})
|
||||
if !CheckDebtEligibility(manual).Eligible {
|
||||
t.Fatal("one manual intervention must qualify on its own")
|
||||
}
|
||||
blocked := debtItem(domain.DebtStructural,
|
||||
obs{domain.ObservationBlockReason, "t1"}, obs{domain.ObservationBlockReason, "t2"})
|
||||
if !CheckDebtEligibility(blocked).Eligible {
|
||||
t.Fatal("two blocked tasks in one component must qualify as structural")
|
||||
}
|
||||
polish := debtItem(domain.DebtPolish, fail, fail, fail, fail, fail)
|
||||
if CheckDebtEligibility(polish).Eligible {
|
||||
t.Fatal("polish must never promote itself, at any recurrence")
|
||||
}
|
||||
}
|
||||
|
||||
// A refusal has to say what is missing. "Not eligible" alone sends an operator
|
||||
// reading code, which is the mistake SubmissionCheck already documents.
|
||||
func TestDebtRefusalNamesTheMissingEvidence(t *testing.T) {
|
||||
check := CheckDebtEligibility(debtItem(domain.DebtOperational, obs{domain.ObservationFailureClass, "t1"}))
|
||||
if check.Eligible || len(check.Reasons) == 0 {
|
||||
t.Fatalf("want a refusal with reasons, got %+v", check)
|
||||
}
|
||||
joined := strings.Join(check.Reasons, " ")
|
||||
if !strings.Contains(joined, "manual interventions are not recorded") {
|
||||
t.Fatalf("the refusal must say the intervention count is structurally zero: %v", check.Reasons)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,227 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"orchestra/internal/domain"
|
||||
"orchestra/internal/review"
|
||||
)
|
||||
|
||||
// ProjectDebt folds canonical history into a debt ledger. It is a pure
|
||||
// function of the events so it can be tested against a real log, and it writes
|
||||
// nothing: this projection is a read model, not a new kind of truth.
|
||||
//
|
||||
// It deliberately reports what it cannot see. Worker observations and operator
|
||||
// interventions are not carried by any event type today, so a ledger that
|
||||
// stayed silent about them would read as "no operational cost" when it means
|
||||
// "operational cost is unrecorded".
|
||||
func ProjectDebt(events []domain.Event) domain.DebtLedger {
|
||||
return projectDebt(events, nil)
|
||||
}
|
||||
|
||||
// ProjectDebtWithArtifacts is ProjectDebt plus the CAS reads that review
|
||||
// findings need. Findings are sealed as an artifact rather than inlined in the
|
||||
// event, and they are the only evidence structural debt has, so a projection
|
||||
// without them can never produce a structural item at all.
|
||||
func ProjectDebtWithArtifacts(events []domain.Event, read func(ref string) ([]byte, error)) domain.DebtLedger {
|
||||
return projectDebt(events, read)
|
||||
}
|
||||
|
||||
func projectDebt(events []domain.Event, readArtifact func(string) ([]byte, error)) domain.DebtLedger {
|
||||
bySignature := map[string]*domain.DebtItem{}
|
||||
order := []string{}
|
||||
seenKinds := map[domain.ObservationKind]bool{}
|
||||
|
||||
add := func(class domain.DebtClass, sig string, o domain.DebtObservation, symptom string, sev review.Severity) {
|
||||
o.Signature = sig
|
||||
if o.Validate() != nil {
|
||||
return
|
||||
}
|
||||
seenKinds[o.Kind] = true
|
||||
item, ok := bySignature[sig]
|
||||
if !ok {
|
||||
item = &domain.DebtItem{
|
||||
// The id is the signature, not an Fxx number. This projection
|
||||
// produces candidates from history; adopting a curated id is a
|
||||
// later, deliberate step, and minting one here would collide
|
||||
// with the hand-written ledger.
|
||||
ID: sig, Class: class, Status: domain.DebtObserved,
|
||||
Symptom: symptom, Signatures: []string{sig}, Severity: sev,
|
||||
}
|
||||
bySignature[sig] = item
|
||||
order = append(order, sig)
|
||||
}
|
||||
item.Observations = append(item.Observations, o)
|
||||
for _, p := range o.Paths {
|
||||
if !contains(item.Paths, p) {
|
||||
item.Paths = append(item.Paths, p)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for _, e := range events {
|
||||
var p map[string]any
|
||||
if len(e.Payload) > 0 && json.Unmarshal(e.Payload, &p) != nil {
|
||||
continue
|
||||
}
|
||||
base := domain.DebtObservation{EventID: e.ID, TaskID: e.TaskID, At: e.At}
|
||||
harness, _ := p["harness_id"].(string)
|
||||
switch e.Type {
|
||||
case "TaskBlocked":
|
||||
reason, _ := p["block_reason"].(string)
|
||||
if reason == "" {
|
||||
reason = string(domain.InferBlockReason(str(p["blocker"])))
|
||||
}
|
||||
class, ok := domain.DebtClassForBlockReason(domain.BlockReason(reason))
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
o := base
|
||||
o.Kind = domain.ObservationBlockReason
|
||||
o.Detail = str(p["blocker"])
|
||||
add(class, domain.DebtSignature(class, reason, harness, "lease"), o,
|
||||
"tasks are blocked with "+reason, review.Important)
|
||||
case "TaskFailed", "TaskReleased":
|
||||
// A release carries a failure class only when it is a reclaim. A
|
||||
// handoff release is ordinary progress and never debt.
|
||||
failure, _ := p["failure_class"].(string)
|
||||
if failure == "" {
|
||||
failure, _ = p["reason"].(string)
|
||||
}
|
||||
if e.Type == "TaskReleased" && str(p["handoff_ref"]) != "" {
|
||||
continue
|
||||
}
|
||||
class, ok := domain.DebtClassForFailureClass(failure)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
o := base
|
||||
o.Kind = domain.ObservationFailureClass
|
||||
o.Detail = str(p["last_error"])
|
||||
add(class, domain.DebtSignature(class, failure, harness, "lease"), o,
|
||||
"tasks end in "+failure, review.Important)
|
||||
case domain.EventPlanMismatchRecorded:
|
||||
o := base
|
||||
o.Kind = domain.ObservationPlanMismatch
|
||||
o.Detail = str(p["observed"])
|
||||
o.Paths = strList(p["evidence"])
|
||||
component := firstComponent(o.Paths)
|
||||
add(domain.DebtCorrectness,
|
||||
domain.DebtSignature(domain.DebtCorrectness, "plan_mismatch", "", component), o,
|
||||
"the repository contradicts sealed plans", review.Important)
|
||||
case domain.EventReviewRecorded:
|
||||
// Repeated findings in one component are the only mechanical
|
||||
// evidence that structure is costing future changes. One finding
|
||||
// is a review doing its job.
|
||||
if readArtifact == nil {
|
||||
break
|
||||
}
|
||||
raw, err := readArtifact(str(p["artifact_ref"]))
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
result, err := review.Decode(raw)
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
for _, f := range result.Findings {
|
||||
component := domain.DebtComponent(f.File)
|
||||
o := base
|
||||
o.Kind = domain.ObservationReviewFinding
|
||||
o.Detail = f.Claim
|
||||
if f.File != "" {
|
||||
o.Paths = []string{f.File}
|
||||
}
|
||||
add(domain.DebtStructural,
|
||||
domain.DebtSignature(domain.DebtStructural, string(f.Severity), "", component), o,
|
||||
"review keeps finding "+string(f.Severity)+" issues in "+component, f.Severity)
|
||||
}
|
||||
case domain.EventDeferredFindingRecorded:
|
||||
o := base
|
||||
o.Kind = domain.ObservationDeferredFinding
|
||||
o.Detail = str(p["summary"])
|
||||
// Polish until something demonstrates a cost. A discovery nobody
|
||||
// has paid for yet is not debt with a priority.
|
||||
add(domain.DebtPolish,
|
||||
domain.DebtSignature(domain.DebtPolish, "deferred_finding", "", ""), o,
|
||||
"out-of-scope discoveries recorded and unaddressed", review.Minor)
|
||||
}
|
||||
}
|
||||
|
||||
items := make([]domain.DebtItem, 0, len(order))
|
||||
for _, sig := range order {
|
||||
items = append(items, *bySignature[sig])
|
||||
}
|
||||
sort.SliceStable(items, func(i, j int) bool {
|
||||
if items[i].Recurrence() != items[j].Recurrence() {
|
||||
return items[i].Recurrence() > items[j].Recurrence()
|
||||
}
|
||||
return items[i].ID < items[j].ID
|
||||
})
|
||||
return domain.DebtLedger{Items: items, Gaps: debtGaps(seenKinds), Events: len(events)}
|
||||
}
|
||||
|
||||
// debtGaps separates two different silences. A kind no event type carries is a
|
||||
// hole in the system. A kind the log could carry and does not is a fact about
|
||||
// this history.
|
||||
func debtGaps(seen map[domain.ObservationKind]bool) []domain.EvidenceGap {
|
||||
gaps := []domain.EvidenceGap{
|
||||
{Kind: domain.ObservationManualIntervention, Durable: false,
|
||||
Reason: "no event type records an operator repair, so every manual recovery is invisible to this ledger"},
|
||||
{Kind: domain.ObservationWorkerFailure, Durable: false,
|
||||
Reason: "worker observations live in worker memory and reach the coordinator only inside WorkerHealth, which is not persisted"},
|
||||
}
|
||||
for _, k := range []domain.ObservationKind{
|
||||
domain.ObservationBlockReason, domain.ObservationFailureClass,
|
||||
domain.ObservationReviewFinding, domain.ObservationPlanMismatch,
|
||||
domain.ObservationDeferredFinding,
|
||||
} {
|
||||
if !seen[k] {
|
||||
gaps = append(gaps, domain.EvidenceGap{Kind: k, Durable: true,
|
||||
Reason: "carried by the log, but this history contains none"})
|
||||
}
|
||||
}
|
||||
return gaps
|
||||
}
|
||||
|
||||
// DebtLedger projects the whole log. Read-only, like every other projection
|
||||
// the coordinator serves.
|
||||
func (s *Store) DebtLedger() domain.DebtLedger {
|
||||
return ProjectDebtWithArtifacts(s.Events(0), s.Artifact)
|
||||
}
|
||||
|
||||
func str(v any) string { s, _ := v.(string); return s }
|
||||
|
||||
func strList(v any) []string {
|
||||
list, ok := v.([]any)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
out := make([]string, 0, len(list))
|
||||
for _, item := range list {
|
||||
if s := strings.TrimSpace(str(item)); s != "" {
|
||||
out = append(out, s)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func firstComponent(paths []string) string {
|
||||
for _, p := range paths {
|
||||
if c := domain.DebtComponent(p); c != "" {
|
||||
return c
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func contains(list []string, s string) bool {
|
||||
for _, item := range list {
|
||||
if item == s {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"orchestra/internal/domain"
|
||||
)
|
||||
|
||||
func debtEvent(id, typ, task string, payload map[string]any) domain.Event {
|
||||
b, _ := json.Marshal(payload)
|
||||
return domain.Event{ID: id, Type: typ, TaskID: task, Payload: b, At: time.Now().UTC()}
|
||||
}
|
||||
|
||||
// The opencode shape from run 14: one harness failing the same way across
|
||||
// several tasks. That is the recurrence the ledger exists to notice.
|
||||
func TestProjectDebtGroupsOneFailureShapeAcrossTasks(t *testing.T) {
|
||||
events := []domain.Event{
|
||||
debtEvent("e1", "TaskFailed", "t1", map[string]any{"failure_class": "retry_limit", "harness_id": "workpc-opencode"}),
|
||||
debtEvent("e2", "TaskFailed", "t2", map[string]any{"failure_class": "retry_limit", "harness_id": "workpc-opencode"}),
|
||||
debtEvent("e3", "TaskFailed", "t3", map[string]any{"failure_class": "retry_limit", "harness_id": "workpc-opencode"}),
|
||||
debtEvent("e4", "TaskFailed", "t4", map[string]any{"failure_class": "retry_limit", "harness_id": "workpc-claude"}),
|
||||
// A handoff release is ordinary progress and must not become debt.
|
||||
debtEvent("e5", "TaskReleased", "t5", map[string]any{"handoff_ref": "sha256:abc", "harness_id": "workpc-claude"}),
|
||||
}
|
||||
ledger := ProjectDebt(events)
|
||||
if len(ledger.Items) != 2 {
|
||||
t.Fatalf("want one item per harness, got %d: %+v", len(ledger.Items), ledger.Items)
|
||||
}
|
||||
top := ledger.Items[0]
|
||||
if top.Recurrence() != 3 || top.AffectedTasks() != 3 {
|
||||
t.Fatalf("recurrence %d across %d tasks, want 3 and 3", top.Recurrence(), top.AffectedTasks())
|
||||
}
|
||||
if top.Class != domain.DebtOperational {
|
||||
t.Fatalf("class %q, want operational", top.Class)
|
||||
}
|
||||
if got := top.ID; got != "v1:operational:retry_limit:workpc-opencode:lease" {
|
||||
t.Fatalf("signature %q", got)
|
||||
}
|
||||
for _, o := range top.Observations {
|
||||
if o.EventID == "" || o.LegacyRef != "" {
|
||||
t.Fatalf("observation lost its event provenance: %+v", o)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A lifecycle stop is the system working. Waiting for a human is not debt, and
|
||||
// counting it would drown the real signal.
|
||||
func TestProjectDebtIgnoresOrdinaryLifecycleStops(t *testing.T) {
|
||||
events := []domain.Event{
|
||||
debtEvent("e1", "TaskBlocked", "t1", map[string]any{"block_reason": "human_decision", "blocker": "which route"}),
|
||||
debtEvent("e2", "TaskBlocked", "t2", map[string]any{"block_reason": "trajectory_gate", "blocker": "confirm plan"}),
|
||||
debtEvent("e3", "TaskBlocked", "t3", map[string]any{"block_reason": "lease_expired", "blocker": "lease expired"}),
|
||||
}
|
||||
ledger := ProjectDebt(events)
|
||||
if len(ledger.Items) != 1 || ledger.Items[0].Class != domain.DebtOperational {
|
||||
t.Fatalf("want only the lease_expired item, got %+v", ledger.Items)
|
||||
}
|
||||
}
|
||||
|
||||
// Incompleteness is part of the result. A ledger that stays silent about what
|
||||
// it cannot record reads as "no operator cost" when it means "operator cost is
|
||||
// not recorded anywhere".
|
||||
func TestProjectDebtReportsWhatItCannotSee(t *testing.T) {
|
||||
ledger := ProjectDebt(nil)
|
||||
var manual, worker bool
|
||||
for _, g := range ledger.Gaps {
|
||||
if g.Durable {
|
||||
continue
|
||||
}
|
||||
switch g.Kind {
|
||||
case domain.ObservationManualIntervention:
|
||||
manual = true
|
||||
case domain.ObservationWorkerFailure:
|
||||
worker = true
|
||||
}
|
||||
}
|
||||
if !manual || !worker {
|
||||
t.Fatalf("the two known holes must always be reported: %+v", ledger.Gaps)
|
||||
}
|
||||
for _, g := range ledger.Gaps {
|
||||
if g.Reason == "" {
|
||||
t.Fatalf("gap %q has no reason", g.Kind)
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user