Compare commits
4 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| f25964c18e | |||
| 0aefe021b0 | |||
| 982741fe97 | |||
| 42c5f07844 |
+110
@@ -408,3 +408,113 @@ which is the priority function slice one deliberately omitted.
|
||||
|
||||
None of these required a schema commitment to discover. That was the point of
|
||||
making the first slice read-only.
|
||||
|
||||
## The invariant slice B was written against
|
||||
|
||||
**A projection must never manufacture provenance to make evidence easier to
|
||||
classify.** `task=None` is better than a confident lie.
|
||||
|
||||
Both defects the first live run of durable observations exposed were failures
|
||||
of exactly this rule, and both looked perfectly reasonable in code:
|
||||
|
||||
- **False attribution.** The worker's ring outlives the work it describes, so
|
||||
binding its entries to whatever task the worker is running now produced a
|
||||
clean, well-formed, wrong association between an old failure and an unrelated
|
||||
task. The task is read from the observation itself, and only an observation
|
||||
that names no task belongs to the current lease.
|
||||
- **Manufactured recurrence.** Treating "still present in the ring" as "it
|
||||
happened again" turned one failure into four incidents. Presence is not
|
||||
occurrence: an incident opens only when the entry advances past what has
|
||||
already been accounted for, and that high-water mark survives the close.
|
||||
|
||||
Either one would have corrupted the ledger while every number in it stayed
|
||||
plausible, which is the specific way this repo's bugs have always presented.
|
||||
|
||||
## Signatures are frozen at write time
|
||||
|
||||
The signature is computed when an incident is recorded and stored in the event.
|
||||
Re-projecting an old log with a newer normalizer therefore changes nothing:
|
||||
running one history through the build before and after the pane-name fix
|
||||
produced identical output.
|
||||
|
||||
That is the intended behavior. An event must not silently change meaning
|
||||
because normalization code changed. Repairing historical signatures is an
|
||||
explicit migration or reclassification event, never a different projection
|
||||
result from the same log.
|
||||
|
||||
## What counts as a manual intervention
|
||||
|
||||
An intervention is an operator action required to recover, repair, unblock or
|
||||
correct behaviour that should otherwise have proceeded autonomously. The
|
||||
ledger measures what the system costs to keep running, so routine operation
|
||||
does not belong in it.
|
||||
|
||||
Counts:
|
||||
|
||||
```text
|
||||
transaction_cleanup
|
||||
forced_release
|
||||
state_repair
|
||||
manual_requeue when recovery failed and a human had to requeue
|
||||
manual_phase_recovery
|
||||
worker_restart only when restarting is itself the repair
|
||||
```
|
||||
|
||||
Does not count:
|
||||
|
||||
```text
|
||||
deploy restart
|
||||
planned upgrade
|
||||
configuration rollout
|
||||
normal shutdown and start
|
||||
deliberate burn-in setup
|
||||
```
|
||||
|
||||
The distinction is policy rather than schema. `worker_restart` is the one kind
|
||||
that spans both sides, and it stays a single kind until someone actually
|
||||
misuses it; a field added before the confusion exists is a guess about how it
|
||||
will be misread.
|
||||
|
||||
The first live consequence: the deploy restart of `79d2053` was not recorded,
|
||||
and `manual_intervention` stayed in the gap list afterwards. That is the
|
||||
correct result. Orchestra can record the evidence and this history contains
|
||||
none, which is an honest gap rather than synthetic evidence.
|
||||
|
||||
## Slice B status, 2026-08-30
|
||||
|
||||
```text
|
||||
worker observation durability proven live
|
||||
incident open/close semantics proven live
|
||||
high-water, no phantom recurrence proven live
|
||||
task attribution proven live
|
||||
frozen historical provenance proven live
|
||||
|
||||
eligibility calculation proven against real data
|
||||
eligibility transition unexercised
|
||||
manual intervention recording implemented
|
||||
manual intervention provenance unexercised
|
||||
```
|
||||
|
||||
Neither unexercised branch will be manufactured. A second task hitting the
|
||||
same failure exercises the first, and a genuine repair exercises the second.
|
||||
|
||||
The proof that matters from the live run:
|
||||
|
||||
```text
|
||||
recurrence 5, intensity 10, distinct tasks 1, interventions 0 -> eligible=false
|
||||
```
|
||||
|
||||
A noisy failure inside one task does not become system-level maintenance debt.
|
||||
The three counts stay separate on purpose: recurrence is how often the incident
|
||||
happened, intensity is how repetitive each incident became, and breadth is how
|
||||
many independent tasks paid for it. Operational debt requires breadth unless an
|
||||
operator had to intervene.
|
||||
|
||||
### The reasons check is part of the design, not a formality
|
||||
|
||||
The aggregate numbers moved in a way that looked like a promotion, and reading
|
||||
them that way was wrong: the eligible count rose because of an unrelated item.
|
||||
The pure eligibility explanation contradicted that inference with counted
|
||||
facts. That is the argument for the ledger resting on mechanically derived
|
||||
evidence rather than on anyone's reading of what looks recurring, including an
|
||||
agent's.
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
// Runs the debt projection over a captured event log and prints one row per
|
||||
// item, so two builds can be compared on identical input. The point of the
|
||||
// comparison is the signature fix: cross-task incidents must collapse into one
|
||||
// item while genuinely different failures stay separate.
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"sort"
|
||||
|
||||
"orchestra/internal/domain"
|
||||
"orchestra/internal/operations"
|
||||
"orchestra/internal/store"
|
||||
)
|
||||
|
||||
func main() {
|
||||
b, err := os.ReadFile(os.Args[1])
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
var events []domain.Event
|
||||
if err := json.Unmarshal(b, &events); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
ledger := store.ProjectDebt(events)
|
||||
eligible := map[string]operations.DebtCandidate{}
|
||||
for _, c := range operations.EligibleDebt(ledger) {
|
||||
eligible[c.Item.ID] = c
|
||||
}
|
||||
rows := make([]map[string]any, 0, len(ledger.Items))
|
||||
for _, item := range ledger.Items {
|
||||
tasks := map[string]bool{}
|
||||
intensity := 0
|
||||
for _, o := range item.Observations {
|
||||
if o.TaskID != "" {
|
||||
tasks[o.TaskID] = true
|
||||
}
|
||||
intensity += o.Repeats
|
||||
}
|
||||
row := map[string]any{
|
||||
"signature": item.ID, "class": string(item.Class),
|
||||
"recurrence": len(item.Observations), "intensity": intensity,
|
||||
"tasks": len(tasks), "eligible": false, "reasons": []string{},
|
||||
}
|
||||
check := operations.CheckDebtEligibility(item)
|
||||
row["eligible"] = check.Eligible
|
||||
row["reasons"] = check.Reasons
|
||||
_ = eligible
|
||||
rows = append(rows, row)
|
||||
}
|
||||
sort.Slice(rows, func(i, j int) bool { return rows[i]["signature"].(string) < rows[j]["signature"].(string) })
|
||||
out, _ := json.MarshalIndent(map[string]any{"items": rows, "gaps": ledger.Gaps}, "", " ")
|
||||
fmt.Println(string(out))
|
||||
}
|
||||
@@ -43,9 +43,17 @@ func CheckDebtEligibility(item domain.DebtItem) DebtCheck {
|
||||
if len(why) > 0 {
|
||||
return DebtCheck{true, why}
|
||||
}
|
||||
// Reasons state facts about this item and nothing about the
|
||||
// architecture around it. The second line here used to say manual
|
||||
// interventions were recorded by no event type, which was true when it
|
||||
// was written and false the moment OperatorInterventionRecorded
|
||||
// landed. A decision that is right for a reason that has become a lie
|
||||
// cannot be audited, and why a count is zero is not this function's
|
||||
// business: no intervention happened, none was recorded, or none was
|
||||
// migrated all read the same from here.
|
||||
return DebtCheck{false, []string{
|
||||
fmt.Sprintf("needs 3 occurrences across 2 tasks, or 1 manual intervention; has %d across %d tasks with %d interventions", recurrence, tasks, manual),
|
||||
"manual interventions are not recorded by any event type, so that count reads 0 on every current log",
|
||||
fmt.Sprintf("breadth threshold not met: %d occurrences across %d tasks, needs 3 across 2", recurrence, tasks),
|
||||
fmt.Sprintf("manual intervention threshold not met: %d recorded, needs 1", manual),
|
||||
}}
|
||||
case domain.DebtStructural:
|
||||
if recurrence >= 3 {
|
||||
|
||||
@@ -62,7 +62,17 @@ func TestDebtRefusalNamesTheMissingEvidence(t *testing.T) {
|
||||
t.Fatalf("want a refusal with reasons, got %+v", check)
|
||||
}
|
||||
joined := strings.Join(check.Reasons, " ")
|
||||
if !strings.Contains(joined, "manual interventions are not recorded") {
|
||||
t.Fatalf("the refusal must say the intervention count is structurally zero: %v", check.Reasons)
|
||||
// Every reason is a fact about this item. The old text asserted that no
|
||||
// event type recorded manual interventions, which stopped being true the
|
||||
// day one did, leaving a correct decision explained by a lie.
|
||||
for _, want := range []string{"1 occurrences across 1 tasks", "0 recorded, needs 1"} {
|
||||
if !strings.Contains(joined, want) {
|
||||
t.Fatalf("the refusal does not state %q: %v", want, check.Reasons)
|
||||
}
|
||||
}
|
||||
for _, forbidden := range []string{"event type", "every current log"} {
|
||||
if strings.Contains(joined, forbidden) {
|
||||
t.Fatalf("a reason describes the architecture instead of the item: %v", check.Reasons)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user