feat(recovery): failure-ticket routing + review-gate RECOVER + return-to-sender

Retry-agency invariant: a stage may only retry a gate it has the capability
to change. A write-less stage failing a build/contract/static gate (e.g.
freestyle final_verification with allowedTools=[shell]) can never fix it, so
retrying in place is futile until the budget drains (Vikunja #41).

Instead: open a FailureTicketOpenedEvent (category + requiredCapability derived
deterministically from the gate id, no LLM) and route to a recovery stage that
holds the capability, bounded by a small per-stage route budget.

- Slice 2: SalvageDecision.RECOVER (ternary CONTINUE/RECOVER/FAIL) lets the
  review-gate salvage judge hand off to recovery. Shared routeToRecovery()
  unifies the deterministic agency guard and the judge on one destination;
  decideGateExhaustion now returns StepResult?.
- Return-to-sender: recovery's exit is dynamic (recoveryReturnMove) — it goes
  back to the exact ticket-origin stage to re-run its gate, so a write-less
  gate anywhere in the graph is handled without skipping intervening stages.
  The synthesized recovery->terminal edge is now only a no-ticket fallback.
- Freestyle wiring: ExecutionPlanCompiler injectRecovery flag (Main passes
  true) synthesizes a write-capable recovery stage; ticket evidence reaches it
  via buildRecoveryTicketEntry.
- Read-only tools always present: StageConfig.effectiveAllowedTools adds
  file_read/list_dir to any tool-granting stage (fixes the verifier reaching
  for `shell ls -R` to inspect the tree).

Tests: RecoveryRoutingTest (deterministic gate, no static return edge — proves
dynamic return) + GateRetryBudgetExhaustionTest RECOVER case + two compiler
tests. All green; detekt clean.
This commit is contained in:
2026-07-07 12:05:26 +04:00
parent 597a26d551
commit 879672a47d
13 changed files with 672 additions and 23 deletions
@@ -371,7 +371,11 @@ fun main() {
// Plan-compile gate: wrap the ExecutionPlanCompiler as a post-stage check so a compile-invalid // Plan-compile gate: wrap the ExecutionPlanCompiler as a post-stage check so a compile-invalid
// architect plan is handed back through the retry-feedback loop instead of parking the session in // architect plan is handed back through the retry-feedback loop instead of parking the session in
// ACTIVE (post-planning compile dead-end). Returns null on success, the compiler message on failure. // ACTIVE (post-planning compile dead-end). Returns null on success, the compiler message on failure.
val planCompiler = ExecutionPlanCompiler(artifactKindRegistry, toolRegistry.all().map { it.name }.toSet()) val planCompiler = ExecutionPlanCompiler(
artifactKindRegistry,
toolRegistry.all().map { it.name }.toSet(),
injectRecovery = true,
)
val orchestrator = DefaultSessionOrchestrator( val orchestrator = DefaultSessionOrchestrator(
repositories = repositories, repositories = repositories,
engines = engines.copy( engines = engines.copy(
@@ -69,9 +69,45 @@ data class RetryAttemptedEvent(
val charged: Boolean = true, val charged: Boolean = true,
) : EventPayload ) : EventPayload
/** Outcome of the hybrid-exhaustion salvage judgement for the semantic review gate. */ /**
* Outcome of the hybrid-exhaustion salvage judgement for the semantic review gate.
* CONTINUE resets the gate budget and retries the reviewed stage in place; FAIL is terminal;
* RECOVER hands the failure to the recovery stage (same destination as the deterministic-gate
* agency guard) — safe under invariant #7 because recovery output re-enters verification and so
* cannot corrupt state, it only chooses which bounded remediation to spend budget on.
*/
@Serializable @Serializable
enum class SalvageDecision { CONTINUE, FAIL } enum class SalvageDecision { CONTINUE, FAIL, RECOVER }
/**
* A verification gate failed on a stage that lacks the capability to change the failure condition
* (the retry-agency invariant: recovery/ticket model). Rather than retry the same write-less stage
* in place — which is futile by construction (live repro: freestyle `final_verification` compiled
* with `allowedTools=[shell]` re-ran the identical broken `npm run build` until the budget drained,
* never able to touch a file) — the orchestrator opens a failure ticket and routes to a recovery
* stage that DOES hold the capability. [category] and [requiredCapability] are derived
* deterministically from [gate] (no LLM), so this drives routing without violating invariant #7.
* [evidence] is the gate's captured failure output; it is recorded here at ticket-open time and
* replay reads it back (invariant #9) — the gate command is never re-run on replay.
*/
@Serializable
@SerialName("FailureTicketOpened")
data class FailureTicketOpenedEvent(
val sessionId: SessionId,
// The stage that failed the gate but lacks the capability to fix it (the write-less verifier).
val stageId: StageId,
val gate: String,
// Deterministic from [gate]: "implementation" | "planning" | "environment".
val category: String,
// The capability the failing stage lacks (e.g. "file_write"); the recovery stage must hold it.
val requiredCapability: String,
// The recovery stage the ticket is routed to.
val routeTo: StageId,
// The gate's captured failure output (command/exit/stderr/findings) that recovery must resolve.
val evidence: String,
// 1-based count of times this stage has been routed to recovery (own, small route budget).
val routeAttempt: Int,
) : EventPayload
/** /**
* Recorded when a gate exhausts its retry budget and (per design decision 3) the salvage judge is * Recorded when a gate exhausts its retry budget and (per design decision 3) the salvage judge is
@@ -25,4 +25,10 @@ data class OrchestrationState(
// Gates that have already spent their one hybrid-exhaustion salvage reset (review gate only, // Gates that have already spent their one hybrid-exhaustion salvage reset (review gate only,
// see RetrySalvageDecidedEvent) — a second exhaustion for that gate is terminal. // see RetrySalvageDecidedEvent) — a second exhaustion for that gate is terminal.
val gateSalvageUsed: Set<String> = emptySet(), val gateSalvageUsed: Set<String> = emptySet(),
// Recovery-routing budget (retry-agency / ticket model): how many times each stage id has been
// routed to a recovery stage because it failed a gate it lacked the capability to fix (see
// FailureTicketOpenedEvent). Deliberately NOT reset by TransitionExecutedEvent — the
// verification→recovery→verification loop must stay bounded across those transitions, exactly as
// refinementIterations does for back-edge loops.
val recoveryRoutes: Map<String, Int> = emptyMap(),
) )
@@ -59,6 +59,7 @@ import com.correx.core.events.events.RepoKnowledgeRetrievedEvent
import com.correx.core.events.events.RepoMapComputedEvent import com.correx.core.events.events.RepoMapComputedEvent
import com.correx.core.events.events.RetryAttemptedEvent import com.correx.core.events.events.RetryAttemptedEvent
import com.correx.core.events.events.RetrySalvageDecidedEvent import com.correx.core.events.events.RetrySalvageDecidedEvent
import com.correx.core.events.events.FailureTicketOpenedEvent
import com.correx.core.events.events.WorkspaceStateObservedEvent import com.correx.core.events.events.WorkspaceStateObservedEvent
import com.correx.core.events.events.RiskAssessedEvent import com.correx.core.events.events.RiskAssessedEvent
import com.correx.core.events.events.SourceFetchedEvent import com.correx.core.events.events.SourceFetchedEvent
@@ -146,6 +147,7 @@ val eventModule = SerializersModule {
subclass(WorkflowCompletedEvent::class) subclass(WorkflowCompletedEvent::class)
subclass(RetryAttemptedEvent::class) subclass(RetryAttemptedEvent::class)
subclass(RetrySalvageDecidedEvent::class) subclass(RetrySalvageDecidedEvent::class)
subclass(FailureTicketOpenedEvent::class)
subclass(RefinementIterationEvent::class) subclass(RefinementIterationEvent::class)
subclass(RepoMapComputedEvent::class) subclass(RepoMapComputedEvent::class)
subclass(WorkspaceStateObservedEvent::class) subclass(WorkspaceStateObservedEvent::class)
@@ -8,6 +8,7 @@ import com.correx.core.events.events.RepoKnowledgeHit
import com.correx.core.context.model.ContextEntry import com.correx.core.context.model.ContextEntry
import com.correx.core.context.model.ContextLayer import com.correx.core.context.model.ContextLayer
import com.correx.core.context.model.EntryRole import com.correx.core.context.model.EntryRole
import com.correx.core.events.events.FailureTicketOpenedEvent
import com.correx.core.events.events.RefinementIterationEvent import com.correx.core.events.events.RefinementIterationEvent
import com.correx.core.events.events.RetryAttemptedEvent import com.correx.core.events.events.RetryAttemptedEvent
import com.correx.core.events.events.StoredEvent import com.correx.core.events.events.StoredEvent
@@ -38,6 +39,33 @@ fun buildRetryFeedbackEntry(events: List<StoredEvent>, stageId: StageId): Contex
) )
} }
/**
* Feeds the open failure ticket into the recovery stage's context. The recovery stage was entered
* by kernel routing (not a normal edge) precisely because an upstream write-less stage failed a gate
* it lacked the capability to fix; without the ticket the recovery stage has no idea what to repair.
* Surfaces the failing stage, the gate, and the captured gate output (evidence, recorded at
* ticket-open per invariant #9) so recovery can act on recorded fact, not a re-observation.
*/
fun buildRecoveryTicketEntry(events: List<StoredEvent>, stageId: StageId): ContextEntry? {
val ticket = events
.mapNotNull { it.payload as? FailureTicketOpenedEvent }
.lastOrNull { it.routeTo == stageId } ?: return null
val content = "## Recovery ticket\n" +
"Stage '${ticket.stageId.value}' failed the '${ticket.gate}' gate (${ticket.category}) and " +
"lacks the '${ticket.requiredCapability}' capability to fix it, so the failure was routed to " +
"you. Repair the underlying cause, then control returns to verification.\n" +
"### Gate output\n${ticket.evidence}"
return ContextEntry(
id = ContextEntryId(UUID.randomUUID().toString()),
layer = ContextLayer.L1,
content = content,
sourceType = "recoveryTicket",
sourceId = stageId.value,
tokenEstimate = content.length / 4,
role = EntryRole.SYSTEM,
)
}
fun criticArtifactIds( fun criticArtifactIds(
events: List<StoredEvent>, events: List<StoredEvent>,
graph: WorkflowGraph, graph: WorkflowGraph,
@@ -7,6 +7,7 @@ import com.correx.core.events.events.RefinementIterationEvent
import com.correx.core.events.events.RetryAttemptedEvent import com.correx.core.events.events.RetryAttemptedEvent
import com.correx.core.events.events.RetrySalvageDecidedEvent import com.correx.core.events.events.RetrySalvageDecidedEvent
import com.correx.core.events.events.SalvageDecision import com.correx.core.events.events.SalvageDecision
import com.correx.core.events.events.FailureTicketOpenedEvent
import com.correx.core.events.events.StoredEvent import com.correx.core.events.events.StoredEvent
import com.correx.core.events.events.TransitionExecutedEvent import com.correx.core.events.events.TransitionExecutedEvent
import com.correx.core.events.events.WorkflowCompletedEvent import com.correx.core.events.events.WorkflowCompletedEvent
@@ -94,6 +95,12 @@ class DefaultOrchestrationReducer : OrchestrationReducer {
refinementIterations = state.refinementIterations + (p.cycleKey to p.iteration), refinementIterations = state.refinementIterations + (p.cycleKey to p.iteration),
) )
// Charge the failing stage's recovery-route budget. routeAttempt is the 1-based count the
// orchestrator computed, so fold it directly (idempotent under replay).
is FailureTicketOpenedEvent -> state.copy(
recoveryRoutes = state.recoveryRoutes + (p.stageId.value to p.routeAttempt),
)
else -> state else -> state
} }
} }
@@ -15,6 +15,7 @@ import com.correx.core.events.events.ArtifactContentStoredEvent
import com.correx.core.events.events.ClarificationAnswer import com.correx.core.events.events.ClarificationAnswer
import com.correx.core.events.events.ClarificationAnsweredEvent import com.correx.core.events.events.ClarificationAnsweredEvent
import com.correx.core.events.events.ArtifactValidatedEvent import com.correx.core.events.events.ArtifactValidatedEvent
import com.correx.core.events.events.FailureTicketOpenedEvent
import com.correx.core.events.events.OrchestrationResumedEvent import com.correx.core.events.events.OrchestrationResumedEvent
import com.correx.core.events.events.RefinementIterationEvent import com.correx.core.events.events.RefinementIterationEvent
import com.correx.core.events.events.RetrySalvageDecidedEvent import com.correx.core.events.events.RetrySalvageDecidedEvent
@@ -29,6 +30,7 @@ import com.correx.core.events.types.ClarificationRequestId
import com.correx.core.events.types.ArtifactLifecyclePhase import com.correx.core.events.types.ArtifactLifecyclePhase
import com.correx.core.events.types.SessionId import com.correx.core.events.types.SessionId
import com.correx.core.events.types.StageId import com.correx.core.events.types.StageId
import com.correx.core.events.types.TransitionId
import com.correx.core.kernel.execution.WorkflowResult import com.correx.core.kernel.execution.WorkflowResult
import com.correx.core.kernel.orchestration.subagent.InSessionSubagentRunner import com.correx.core.kernel.orchestration.subagent.InSessionSubagentRunner
import com.correx.core.kernel.orchestration.subagent.SubagentRunRequest import com.correx.core.kernel.orchestration.subagent.SubagentRunRequest
@@ -54,6 +56,25 @@ private val log = LoggerFactory.getLogger(DefaultSessionOrchestrator::class.java
// Fallback refinement-loop cap when a stage does not declare maxRetries. // Fallback refinement-loop cap when a stage does not declare maxRetries.
private const val DEFAULT_MAX_REFINEMENT = 3 private const val DEFAULT_MAX_REFINEMENT = 3
// Own, small budget for the retry-agency recovery route: how many times a single stage may be
// routed to a recovery stage before the run fails terminally. Deliberately smaller than the
// per-gate retry budget — recovery is a purpose-built repair-then-reverify loop, not open-ended.
private const val RECOVERY_ROUTE_BUDGET = 2
// Deterministic gate id → capability the failing stage must hold to change the failure condition.
// A gate not listed here is not eligible for recovery routing (retries in place as before).
private val GATE_REQUIRED_CAPABILITY: Map<String, String> = mapOf(
"execution" to "file_write",
"contract" to "file_write",
"static_analysis" to "file_write",
)
// Deterministic failure category from the gate id (no LLM — keeps routing off the untrusted path).
private fun ticketCategory(gate: String): String = when (gate) {
"plan_compile" -> "planning"
else -> "implementation"
}
class DefaultSessionOrchestrator( class DefaultSessionOrchestrator(
private val repositories: OrchestratorRepositories, private val repositories: OrchestratorRepositories,
engines: OrchestratorEngines, engines: OrchestratorEngines,
@@ -138,6 +159,12 @@ class DefaultSessionOrchestrator(
val resolved = resolveTransition( val resolved = resolveTransition(
enriched.graph, enriched.sessionId, enriched.currentStageId, stageArtifacts, artifactContent, enriched.graph, enriched.sessionId, enriched.currentStageId, stageArtifacts, artifactContent,
) )
// Recovery return-to-sender: a recovery stage is entered by kernel routing from whichever
// write-less stage failed a gate, so its exit must go BACK to that exact stage (re-run its
// gate) — not follow a static edge that would skip intervening stages. Derived from the latest
// failure ticket routed here (replay-deterministic). Overrides the resolved edge; an operator
// preempt below still wins.
val baseDecision = recoveryReturnMove(enriched) ?: resolved
// Preemptive redirect override (freestyle graph re-routing): if the operator confirmed a // Preemptive redirect override (freestyle graph re-routing): if the operator confirmed a
// jump (a recorded, unconsumed PreemptRedirectEvent), it overrides the resolver's edge — // jump (a recorded, unconsumed PreemptRedirectEvent), it overrides the resolver's edge —
// unless the target is unknown or its needs aren't satisfied, in which case it is blocked // unless the target is unknown or its needs aren't satisfied, in which case it is blocked
@@ -161,9 +188,9 @@ class DefaultSessionOrchestrator(
"[Orchestrator] redirect blocked session={} to={} reason={}", "[Orchestrator] redirect blocked session={} to={} reason={}",
enriched.sessionId.value, outcome.event.toStageId.value, outcome.event.reason, enriched.sessionId.value, outcome.event.toStageId.value, outcome.event.reason,
) )
resolved baseDecision
} }
PreemptRedirect.Outcome.None -> resolved PreemptRedirect.Outcome.None -> baseDecision
} }
log.debug( log.debug(
"[Orchestrator] transition session={} stage={} decision={}", "[Orchestrator] transition session={} stage={} decision={}",
@@ -229,6 +256,114 @@ class DefaultSessionOrchestrator(
} }
/**
* Retry-agency guard (ticket + recovery model). When [failure]'s gate requires a capability the
* failing stage does not hold (e.g. a build/contract gate needs `file_write` on a write-less
* verifier), retrying the stage in place can never change the outcome. Instead open a
* [FailureTicketOpenedEvent] and route to the graph's recovery stage — which does hold the
* capability — bounded by the stage's own [RECOVERY_ROUTE_BUDGET].
*
* Returns a [StepResult] when it took over the failure (routed, or budget-exhausted terminal),
* or null to fall through to the normal per-gate retry path. Null in every backward-compatible
* case: gate not capability-gated, stage already has the capability, or no recovery stage exists.
*/
private suspend fun maybeRouteToRecovery(
ctx: EnrichedExecutionContext,
stageId: StageId,
failure: StageExecutionResult.Failure,
state: OrchestrationState,
): StepResult? {
val requiredCapability = GATE_REQUIRED_CAPABILITY[failure.gate] ?: return null
val stageTools = ctx.graph.stages[stageId]?.allowedTools ?: emptySet()
if (requiredCapability in stageTools) return null // stage has agency — retry in place is valid
return routeToRecovery(ctx, stageId, failure.gate, requiredCapability, failure.reason, state)
}
/**
* Open a [FailureTicketOpenedEvent] and route [stageId]'s failure to the graph's recovery stage,
* bounded by [RECOVERY_ROUTE_BUDGET]. Shared by the deterministic agency guard
* ([maybeRouteToRecovery]) and the review-gate salvage judge ([decideGateExhaustion] on
* [SalvageDecision.RECOVER]) so both routing paths reach the same recovery destination.
*
* Returns null when the graph declares no recovery stage (caller falls back to its legacy path);
* a terminal [StepResult] when the route budget is spent; otherwise the recovery stage's result.
*/
private suspend fun routeToRecovery(
ctx: EnrichedExecutionContext,
stageId: StageId,
gate: String,
requiredCapability: String,
reason: String,
state: OrchestrationState,
): StepResult? {
val recoveryId = findRecoveryStage(ctx.graph, stageId) ?: return null // no recovery stage: legacy path
val used = state.recoveryRoutes[stageId.value] ?: 0
if (used >= RECOVERY_ROUTE_BUDGET) {
return StepResult.Terminal(
failWorkflow(
ctx.sessionId,
stageId,
"recovery route budget exhausted for stage ${stageId.value} (gate=$gate): $reason",
retryExhausted = true,
),
)
}
emit(
ctx.sessionId,
FailureTicketOpenedEvent(
sessionId = ctx.sessionId,
stageId = stageId,
gate = gate,
category = ticketCategory(gate),
requiredCapability = requiredCapability,
routeTo = recoveryId,
evidence = reason,
routeAttempt = used + 1,
),
)
log.info(
"[Orchestrator] failure ticket session={} stage={} gate={} lacks={} -> route to recovery={} ({}/{})",
ctx.sessionId.value, stageId.value, gate, requiredCapability,
recoveryId.value, used + 1, RECOVERY_ROUTE_BUDGET,
)
val advancedTo = advanceStage(
ctx.sessionId,
stageId,
TransitionDecision.Move(transitionId = TransitionId("recovery-route"), to = recoveryId),
)
return enterStage(ctx.copy(currentStageId = advancedTo), recoveryId)
}
/**
* The graph's recovery stage: a stage marked `metadata["role"] == "recovery"` (or, as a
* convenience, one whose id is literally "recovery"), other than [failingStageId] itself.
* Null when the graph declares none — recovery routing is opt-in per workflow.
*/
private fun findRecoveryStage(graph: WorkflowGraph, failingStageId: StageId): StageId? =
graph.stages.entries.firstOrNull { (id, cfg) ->
id != failingStageId && (cfg.metadata["role"] == "recovery" || id.value == "recovery")
}?.key
/**
* When the current stage is a recovery stage that has finished, route it back to the stage the
* failure came from — the origin of the latest [FailureTicketOpenedEvent] routed here — so the
* gate that triggered the route is re-run. This makes recovery correct for arbitrary graph
* topology (a write-less gate anywhere routes here and returns to exactly its own stage) rather
* than relying on a static recovery→terminal edge that would skip intervening stages. Returns
* null when the current stage is not a recovery stage or no ticket routed here (then the
* resolver's normal edge — the compiler's fallback recovery→terminal — is used).
*/
private fun recoveryReturnMove(ctx: EnrichedExecutionContext): TransitionDecision.Move? {
val cfg = ctx.graph.stages[ctx.currentStageId] ?: return null
val isRecovery = cfg.metadata["role"] == "recovery" || ctx.currentStageId.value == "recovery"
if (!isRecovery) return null
val origin = repositories.eventStore.read(ctx.sessionId)
.mapNotNull { it.payload as? FailureTicketOpenedEvent }
.lastOrNull { it.routeTo == ctx.currentStageId }
?.stageId ?: return null
return TransitionDecision.Move(transitionId = TransitionId("recovery-return"), to = origin)
}
/** Returns the cached validated artifact content for the given session + artifact id, or null if absent. */ /** Returns the cached validated artifact content for the given session + artifact id, or null if absent. */
fun validatedArtifactContent(sessionId: SessionId, artifactId: ArtifactId): String? = fun validatedArtifactContent(sessionId: SessionId, artifactId: ArtifactId): String? =
artifactContentCache["${sessionId.value}:${artifactId.value}"] artifactContentCache["${sessionId.value}:${artifactId.value}"]
@@ -450,6 +585,10 @@ class DefaultSessionOrchestrator(
return StepResult.Continue(ctx.copy(stageCount = ctx.stageCount + 1)) return StepResult.Continue(ctx.copy(stageCount = ctx.stageCount + 1))
} }
val refreshedState = orchestrationRepository.getState(ctx.sessionId) val refreshedState = orchestrationRepository.getState(ctx.sessionId)
// Retry-agency invariant: a gate this stage lacks the capability to fix must not
// be retried in place (futile). Route to a recovery stage that holds the
// capability, if one exists and the route budget remains.
maybeRouteToRecovery(ctx, stageId, result, refreshedState)?.let { return it }
val gateDecision = retryCoordinator.decide( val gateDecision = retryCoordinator.decide(
sessionId = ctx.sessionId, sessionId = ctx.sessionId,
stageId = stageId, stageId = stageId,
@@ -461,10 +600,9 @@ class DefaultSessionOrchestrator(
when (gateDecision) { when (gateDecision) {
RetryDecision.Retry -> Unit // retry — loop and re-execute RetryDecision.Retry -> Unit // retry — loop and re-execute
RetryDecision.Exhausted -> { RetryDecision.Exhausted -> {
val terminal = decideGateExhaustion( decideGateExhaustion(
ctx.sessionId, stageId, result.gate, result.reason, refreshedState, ctx, stageId, result.gate, result.reason, refreshedState,
) )?.let { return it }
if (terminal != null) return StepResult.Terminal(terminal)
// review-gate salvage said CONTINUE — budget was reset by the reducer; // review-gate salvage said CONTINUE — budget was reset by the reducer;
// loop and re-execute the stage with a fresh budget. // loop and re-execute the stage with a fresh budget.
} }
@@ -482,17 +620,19 @@ class DefaultSessionOrchestrator(
* FAIL is terminal. A gate that already spent its one salvage reset (state.gateSalvageUsed) fails * FAIL is terminal. A gate that already spent its one salvage reset (state.gateSalvageUsed) fails
* immediately on a second exhaustion, regardless of what the judge would say. * immediately on a second exhaustion, regardless of what the judge would say.
* *
* Returns the terminal [WorkflowResult] to fail with, or null to retry (loop and re-execute). * Returns a terminal [StepResult] to fail/route with, the recovery stage's result on RECOVER,
* or null to retry in place (loop and re-execute).
*/ */
private suspend fun decideGateExhaustion( private suspend fun decideGateExhaustion(
sessionId: SessionId, ctx: EnrichedExecutionContext,
stageId: StageId, stageId: StageId,
gate: String, gate: String,
reason: String, reason: String,
state: OrchestrationState, state: OrchestrationState,
): WorkflowResult? { ): StepResult? {
val sessionId = ctx.sessionId
if (gate != "review" || state.gateSalvageUsed.contains(gate)) { if (gate != "review" || state.gateSalvageUsed.contains(gate)) {
return failWorkflow(sessionId, stageId, reason, retryExhausted = true) return StepResult.Terminal(failWorkflow(sessionId, stageId, reason, retryExhausted = true))
} }
// No judge wired: degrade safely with a deterministic allow-one-reset-then-fail policy — // No judge wired: degrade safely with a deterministic allow-one-reset-then-fail policy —
// the gateSalvageUsed check above already ensures this fires at most once per gate. // the gateSalvageUsed check above already ensures this fires at most once per gate.
@@ -504,7 +644,12 @@ class DefaultSessionOrchestrator(
emit(sessionId, RetrySalvageDecidedEvent(sessionId, stageId, gate, judgment.decision, judgment.rationale)) emit(sessionId, RetrySalvageDecidedEvent(sessionId, stageId, gate, judgment.decision, judgment.rationale))
return when (judgment.decision) { return when (judgment.decision) {
SalvageDecision.CONTINUE -> null SalvageDecision.CONTINUE -> null
SalvageDecision.FAIL -> failWorkflow(sessionId, stageId, reason, retryExhausted = true) SalvageDecision.FAIL -> StepResult.Terminal(failWorkflow(sessionId, stageId, reason, retryExhausted = true))
// The judge chose recovery: route to the recovery stage (file_write is the capability it
// provides). Degrade to terminal if the graph declares no recovery stage.
SalvageDecision.RECOVER ->
routeToRecovery(ctx, stageId, gate, "file_write", reason, state)
?: StepResult.Terminal(failWorkflow(sessionId, stageId, reason, retryExhausted = true))
} }
} }
@@ -505,6 +505,8 @@ abstract class SessionOrchestrator(
val agentInstructionsEntries = emptyList<ContextEntry>() val agentInstructionsEntries = emptyList<ContextEntry>()
val retryFeedbackEntries = buildRetryFeedbackEntry(sessionEvents, stageId) val retryFeedbackEntries = buildRetryFeedbackEntry(sessionEvents, stageId)
?.let { listOf(it) } ?: emptyList() ?.let { listOf(it) } ?: emptyList()
val recoveryTicketEntries = buildRecoveryTicketEntry(sessionEvents, stageId)
?.let { listOf(it) } ?: emptyList()
val vocabularyEntries = artifactKindRegistry val vocabularyEntries = artifactKindRegistry
?.takeIf { stageConfig.metadata["injectArtifactKinds"] == "true" } ?.takeIf { stageConfig.metadata["injectArtifactKinds"] == "true" }
?.let { listOf(buildArtifactKindVocabularyEntry(it.list())) } ?: emptyList() ?.let { listOf(buildArtifactKindVocabularyEntry(it.list())) } ?: emptyList()
@@ -532,7 +534,7 @@ abstract class SessionOrchestrator(
systemPrompt + profileEntries + projectProfileEntries + agentInstructionsEntries + systemPrompt + profileEntries + projectProfileEntries + agentInstructionsEntries +
journalEntries + repoMapEntries + claimedTaskEntries + journalEntries + repoMapEntries + claimedTaskEntries +
needsEntries + schemaEntries + vocabularyEntries + promptEntries + steeringEntries + needsEntries + schemaEntries + vocabularyEntries + promptEntries + steeringEntries +
clarificationEntries + retryFeedbackEntries, clarificationEntries + retryFeedbackEntries + recoveryTicketEntries,
) )
val contextPack = runCatching { val contextPack = runCatching {
contextPackBuilder.build( contextPackBuilder.build(
@@ -1021,7 +1023,7 @@ abstract class SessionOrchestrator(
), ),
) )
val deniedByStage = toolCall.function.name != STAGE_COMPLETE_TOOL && val deniedByStage = toolCall.function.name != STAGE_COMPLETE_TOOL &&
toolCall.function.name !in stageConfig.allowedTools toolCall.function.name !in stageConfig.effectiveAllowedTools
val deniedByReadOnly = !deniedByStage && isReadOnlyMode(sessionId) && val deniedByReadOnly = !deniedByStage && isReadOnlyMode(sessionId) &&
(tool?.requiredCapabilities?.contains(ToolCapability.FILE_WRITE) == true) (tool?.requiredCapabilities?.contains(ToolCapability.FILE_WRITE) == true)
if (deniedByStage || deniedByReadOnly) { if (deniedByStage || deniedByReadOnly) {
@@ -2563,7 +2565,7 @@ abstract class SessionOrchestrator(
tools = if (!withTools) { tools = if (!withTools) {
emptyList() emptyList()
} else { } else {
stageConfig.allowedTools stageConfig.effectiveAllowedTools
.mapNotNull { effectives.registry?.resolve(it) } .mapNotNull { effectives.registry?.resolve(it) }
.filter { tool -> .filter { tool ->
// ponytail: filter write tools while read-before-write block is active; restored once a read completes // ponytail: filter write tools while read-before-write block is active; restored once a read completes
@@ -52,4 +52,20 @@ data class StageConfig(
// inference), so a docs-only plan is left untouched. Explicit build_expectation always wins. // inference), so a docs-only plan is left untouched. Explicit build_expectation always wins.
val autoBuildGate: Boolean = false, val autoBuildGate: Boolean = false,
val metadata: Map<String, String> = emptyMap(), val metadata: Map<String, String> = emptyMap(),
) ) {
/**
* The tools actually permitted at run time: the stage's declared [allowedTools] plus the
* always-available read-only tools ([ALWAYS_AVAILABLE_READ_TOOLS]). A stage that grants any tool
* can always inspect the workspace (read a file, list a dir) without the planner having to
* remember to list them — this is why a write-less verifier could `shell ls -R` (flooding its
* own context) instead of a scoped `list_dir`. A stage that declares NO tools stays toolless (a
* pure-reasoning stage is left as designed — read tools would flip it into tool-calling mode).
*/
val effectiveAllowedTools: Set<String>
get() = if (allowedTools.isEmpty()) allowedTools else allowedTools + ALWAYS_AVAILABLE_READ_TOOLS
companion object {
/** Read-only tools every tool-granting stage may call regardless of its declared set. */
val ALWAYS_AVAILABLE_READ_TOOLS: Set<String> = setOf("file_read", "list_dir")
}
}
@@ -15,6 +15,20 @@ import com.fasterxml.jackson.module.kotlin.kotlinModule
import com.fasterxml.jackson.module.kotlin.readValue import com.fasterxml.jackson.module.kotlin.readValue
private const val TERMINAL = "done" private const val TERMINAL = "done"
private const val RECOVERY_STAGE = "recovery"
// Synthesized recovery stage prompt. Freestyle plans are LLM-emitted and never declare a recovery
// stage, but the kernel's retry-agency guard routes a write-less stage's unfixable gate failure to
// one when it exists. So the compiler always injects it (inert unless a gate actually routes): a
// write-capable stage that reads the failure ticket from context, repairs the cause, and hands
// control back to verification.
private const val RECOVERY_PROMPT =
"You are the recovery stage. An upstream stage failed a verification gate it could not fix " +
"(it lacked write access). The failure ticket — the failing stage, the gate, and its exact " +
"output — is in your context under \"## Recovery ticket\". Read the gate output, inspect the " +
"relevant files (file_read / list_dir), and make the minimal edits that fix the reported " +
"cause. Do not re-scaffold or rewrite working code. When done, control returns to the stage " +
"that failed so its gate re-runs."
// Freestyle plans don't specify a per-stage token budget, so without this every compiled stage // Freestyle plans don't specify a per-stage token budget, so without this every compiled stage
// inherits StageConfig's 4096 default — 1/8th of what the static role_pipeline execution stages // inherits StageConfig's 4096 default — 1/8th of what the static role_pipeline execution stages
@@ -32,6 +46,12 @@ class ExecutionPlanCompiler(
// freestyle driver feeds back for a retry). Empty = the caller didn't supply the tool // freestyle driver feeds back for a retry). Empty = the caller didn't supply the tool
// universe, so validation is skipped (preserves the bare `ExecutionPlanCompiler(registry)`). // universe, so validation is skipped (preserves the bare `ExecutionPlanCompiler(registry)`).
private val knownTools: Set<String> = emptySet(), private val knownTools: Set<String> = emptySet(),
// When true, synthesize a write-capable `recovery` stage (+ a recovery->terminal edge) into every
// compiled graph. The kernel's retry-agency guard routes a write-less stage's unfixable gate
// failure there; without it the guard finds no recovery stage and falls back to futile in-place
// retries (Vikunja #41). Off by default so the compiler's own unit tests see only plan stages;
// the server's freestyle path (Main.kt) turns it on.
private val injectRecovery: Boolean = false,
) { ) {
private val mapper = JsonMapper.builder() private val mapper = JsonMapper.builder()
.addModule(kotlinModule()) .addModule(kotlinModule())
@@ -100,6 +120,32 @@ class ExecutionPlanCompiler(
metadata = mapOf("promptInline" to s.prompt), metadata = mapOf("promptInline" to s.prompt),
) )
} }
// Inject a recovery stage unless the plan already declares one. Reached only by the kernel's
// failure-ticket routing (no incoming plan edge), so it is excluded from the reachability
// check below. At run time the kernel returns recovery to the exact stage that failed
// (recoveryReturnMove, from the ticket origin); the recovery->terminal edge synthesized here
// is only a fallback for the impossible case of a recovery stage reached with no ticket, so
// the resolver always has a valid outgoing edge and can never dead-end.
val terminal = terminalStageId(plan)
val recoveryStage: Pair<StageId, StageConfig>? =
if (!injectRecovery || RECOVERY_STAGE in stageMap.keys.map { it.value }) {
null
} else {
StageId(RECOVERY_STAGE) to StageConfig(
allowedTools = setOf("file_write", "shell"),
tokenBudget = DEFAULT_STAGE_TOKEN_BUDGET,
metadata = mapOf("role" to "recovery", "promptInline" to RECOVERY_PROMPT),
)
}
val recoveryEdge: TransitionEdge? = recoveryStage?.let {
TransitionEdge(
id = TransitionId("$RECOVERY_STAGE->$terminal"),
from = StageId(RECOVERY_STAGE),
to = StageId(terminal),
condition = ConditionSpec(type = "always_true").toCondition(),
)
}
val fullStageMap = stageMap + listOfNotNull(recoveryStage)
val declared = stageMap.keys.map { it.value }.toSet() val declared = stageMap.keys.map { it.value }.toSet()
val transitions = plan.edges.map { e -> val transitions = plan.edges.map { e ->
@@ -123,11 +169,13 @@ class ExecutionPlanCompiler(
operator = e.condition.operator, operator = e.condition.operator,
).toCondition(), ).toCondition(),
) )
}.toSet() }.toSet() + listOfNotNull(recoveryEdge)
val start = StageId(plan.stages.first().id) val start = StageId(plan.stages.first().id)
validateReachability(declared, transitions, start) // The recovery stage is reached by kernel routing, not a plan edge, so exclude it from the
return WorkflowGraph(id = workflowId, stages = stageMap, transitions = transitions, start = start) // "unreachable from start" check (it would otherwise always trip it).
validateReachability(declared, transitions, start, excluded = setOfNotNull(recoveryStage?.first?.value))
return WorkflowGraph(id = workflowId, stages = fullStageMap, transitions = transitions, start = start)
} }
/** /**
@@ -174,7 +222,12 @@ class ExecutionPlanCompiler(
private fun isGlob(path: String): Boolean = path.any { it == '*' || it == '?' || it == '[' } private fun isGlob(path: String): Boolean = path.any { it == '*' || it == '?' || it == '[' }
private fun validateReachability(declared: Set<String>, transitions: Set<TransitionEdge>, start: StageId) { private fun validateReachability(
declared: Set<String>,
transitions: Set<TransitionEdge>,
start: StageId,
excluded: Set<String> = emptySet(),
) {
// A declared stage with no path from start can never run — typical LLM topology // A declared stage with no path from start can never run — typical LLM topology
// failure: every stage wired straight to "done" instead of chained, which silently // failure: every stage wired straight to "done" instead of chained, which silently
// truncates the plan to its first stage. // truncates the plan to its first stage.
@@ -187,7 +240,7 @@ class ExecutionPlanCompiler(
.filter { it != TERMINAL && reachable.add(it) } .filter { it != TERMINAL && reachable.add(it) }
.toSet() .toSet()
} }
val unreachable = declared - reachable val unreachable = declared - reachable - excluded
if (unreachable.isNotEmpty()) { if (unreachable.isNotEmpty()) {
throw WorkflowValidationException( throw WorkflowValidationException(
"stages unreachable from start '${start.value}': ${unreachable.sorted().joinToString(", ")}" + "stages unreachable from start '${start.value}': ${unreachable.sorted().joinToString(", ")}" +
@@ -18,6 +18,7 @@ class ExecutionPlanCompilerTest {
it.register(ConfigArtifactKind(id = "patch", schema = trivialSchema, llmEmitted = true)) it.register(ConfigArtifactKind(id = "patch", schema = trivialSchema, llmEmitted = true))
} }
private val compiler = ExecutionPlanCompiler(registry) private val compiler = ExecutionPlanCompiler(registry)
private val recoveringCompiler = ExecutionPlanCompiler(registry, injectRecovery = true)
private val validPlan = """ private val validPlan = """
{ {
@@ -72,6 +73,42 @@ class ExecutionPlanCompilerTest {
assertEquals(setOf("patch"), reviewStage.needs.map { it.value }.toSet()) assertEquals(setOf("patch"), reviewStage.needs.map { it.value }.toSet())
} }
@Test
fun `injectRecovery synthesizes a write-capable recovery stage looping back to the terminal`() {
val graph = recoveringCompiler.compile(validPlan, "recovery-workflow")
// Two plan stages + the synthesized recovery stage; still reachable-valid (recovery is
// excluded from the reachability check because it's entered by kernel routing, not an edge).
assertEquals(3, graph.stages.size)
val recovery = graph.stages[StageId("recovery")]
assertNotNull(recovery)
assertEquals("recovery", recovery.metadata["role"])
assertTrue(recovery.allowedTools.contains("file_write"))
// file_read / list_dir are always available to any tool-granting stage.
assertTrue(recovery.effectiveAllowedTools.contains("file_read"))
// recovery -> review (the plan's terminal stage, the one whose edge points at 'done').
val recoveryEdge = graph.transitions.single { it.from == StageId("recovery") }
assertEquals(StageId("review"), recoveryEdge.to)
}
@Test
fun `injectRecovery is a no-op when the plan already declares a recovery stage`() {
val plan = """
{
"goal": "x",
"stages": [
{ "id": "recovery", "prompt": "fix", "produces": "patch", "needs": [], "tools": ["file_write"] }
],
"edges": [
{ "from": "recovery", "to": "done", "condition": { "type": "always_true" } }
]
}
""".trimIndent()
val graph = recoveringCompiler.compile(plan, "wf")
assertEquals(1, graph.stages.size)
}
@Test @Test
fun `plan-declared writes are derived into the stage write manifest`() { fun `plan-declared writes are derived into the stage write manifest`() {
val plan = """ val plan = """
@@ -14,6 +14,7 @@ import com.correx.core.events.events.ReviewFinding
import com.correx.core.events.events.ReviewFindingsRaisedEvent import com.correx.core.events.events.ReviewFindingsRaisedEvent
import com.correx.core.events.events.ReviewSeverity import com.correx.core.events.events.ReviewSeverity
import com.correx.core.events.events.ReviewVerdict import com.correx.core.events.events.ReviewVerdict
import com.correx.core.events.events.FailureTicketOpenedEvent
import com.correx.core.events.events.RetrySalvageDecidedEvent import com.correx.core.events.events.RetrySalvageDecidedEvent
import com.correx.core.events.events.SalvageDecision import com.correx.core.events.events.SalvageDecision
import com.correx.core.events.events.WorkflowFailedEvent import com.correx.core.events.events.WorkflowFailedEvent
@@ -311,6 +312,61 @@ class GateRetryBudgetExhaustionTest {
assertTrue(blockedReviews >= 2, "expected at least 2 blocked review attempts (pre- and post-salvage)") assertTrue(blockedReviews >= 2, "expected at least 2 blocked review attempts (pre- and post-salvage)")
} }
// Stage A opts into the (always-failing) review gate; a sibling recovery stage holds file_write.
// Recovery has NO outgoing edge — the kernel returns it to the ticket origin (A) dynamically.
private fun recoveryReviewGraph(): WorkflowGraph = WorkflowGraph(
id = "gate-review-recover-test",
stages = mapOf(
StageId("A") to StageConfig(allowedTools = setOf("file_write"), semanticReview = true),
StageId("recovery") to StageConfig(
allowedTools = setOf("file_write"),
metadata = mapOf("role" to "recovery"),
),
),
transitions = setOf(
TransitionEdge(TransitionId("t1"), StageId("A"), StageId("done"), condition = { true }),
),
start = StageId("A"),
)
@Test
fun `review gate exhaustion with RECOVER salvage routes to the recovery stage and exhausts the route budget`(): Unit =
runBlocking {
// Slice 2: the salvage judge chooses RECOVER instead of CONTINUE/FAIL, so a review-gate
// exhaustion hands off to the recovery stage (same destination as the deterministic agency
// guard) rather than retrying A in place or failing outright. Bounded by RECOVERY_ROUTE_BUDGET.
val salvageJudge = com.correx.core.kernel.orchestration.SalvageJudge { _, _, _ ->
SalvageJudgment(SalvageDecision.RECOVER, "hand this to recovery")
}
val (orchestrator, eventStore) = buildOrchestrator(alwaysFailingReviewer, salvageJudge)
val sessionId = SessionId("gate-review-recover")
val config = OrchestrationConfig(
retryPolicy = RetryPolicy(maxAttempts = 3, backoffMs = 0, perGateMaxAttempts = mapOf("review" to 1)),
)
val result = orchestrator.run(sessionId, recoveryReviewGraph(), config)
assertTrue(result is WorkflowResult.Failed, "route budget must exhaust terminally")
assertTrue((result as WorkflowResult.Failed).retryExhausted)
val events = eventStore.read(sessionId)
val tickets = events.mapNotNull { it.payload as? FailureTicketOpenedEvent }
assertEquals(2, tickets.size, "one ticket per route, bounded by RECOVERY_ROUTE_BUDGET")
tickets.forEach {
assertEquals("A", it.stageId.value)
assertEquals("review", it.gate)
assertEquals("implementation", it.category)
assertEquals("file_write", it.requiredCapability)
assertEquals("recovery", it.routeTo.value)
}
assertEquals(listOf(1, 2), tickets.map { it.routeAttempt })
// Every salvage decision on this path is RECOVER (judge consulted afresh each re-entry
// because TransitionExecuted resets gateSalvageUsed).
val salvageDecisions = events.mapNotNull { it.payload as? RetrySalvageDecidedEvent }
assertTrue(salvageDecisions.size >= 2, "judge consulted on each review exhaustion")
assertTrue(salvageDecisions.all { it.decision == SalvageDecision.RECOVER })
}
@Test @Test
fun `review gate exhaustion with FAIL salvage is terminal immediately`(): Unit = runBlocking { fun `review gate exhaustion with FAIL salvage is terminal immediately`(): Unit = runBlocking {
val salvageJudge = com.correx.core.kernel.orchestration.SalvageJudge { _, _, _ -> val salvageJudge = com.correx.core.kernel.orchestration.SalvageJudge { _, _, _ ->
@@ -0,0 +1,257 @@
import com.correx.core.approvals.ApprovalOutcome
import com.correx.core.approvals.ApprovalProjector
import com.correx.core.approvals.ApprovalStatus
import com.correx.core.approvals.DefaultApprovalReducer
import com.correx.core.approvals.DefaultApprovalRepository
import com.correx.core.approvals.domain.ApprovalEngine
import com.correx.core.approvals.model.ApprovalContext
import com.correx.core.approvals.model.ApprovalDecision
import com.correx.core.approvals.model.ApprovalGrant
import com.correx.core.approvals.model.DomainApprovalRequest
import com.correx.core.artifacts.DefaultArtifactReducer
import com.correx.core.events.EventDispatcher
import com.correx.core.events.events.FailureTicketOpenedEvent
import com.correx.core.events.events.WorkflowFailedEvent
import com.correx.core.events.execution.RetryPolicy
import com.correx.core.events.types.ProviderId
import com.correx.core.events.types.SessionId
import com.correx.core.events.types.StageId
import com.correx.core.events.types.TransitionId
import com.correx.core.inference.CapabilityScore
import com.correx.core.inference.FinishReason
import com.correx.core.inference.InferenceProvider
import com.correx.core.inference.InferenceRepository
import com.correx.core.inference.InferenceRequest
import com.correx.core.inference.InferenceResponse
import com.correx.core.inference.InferenceRouter
import com.correx.core.inference.InferenceState
import com.correx.core.inference.ModelCapability
import com.correx.core.inference.ProviderHealth
import com.correx.core.inference.Tokenizer
import com.correx.core.inference.ToolCallFunction
import com.correx.core.inference.ToolCallRequest
import com.correx.core.inference.TokenUsage
import com.correx.core.journal.DecisionJournalProjector
import com.correx.core.journal.DefaultDecisionJournalReducer
import com.correx.core.journal.DefaultDecisionJournalRepository
import com.correx.core.kernel.execution.WorkflowResult
import com.correx.core.kernel.orchestration.DefaultOrchestrationReducer
import com.correx.core.kernel.orchestration.DefaultSessionOrchestrator
import com.correx.core.kernel.orchestration.OrchestrationConfig
import com.correx.core.kernel.orchestration.OrchestrationProjector
import com.correx.core.kernel.orchestration.OrchestrationRepository
import com.correx.core.kernel.orchestration.OrchestratorEngines
import com.correx.core.kernel.orchestration.OrchestratorRepositories
import com.correx.core.kernel.orchestration.StaticAnalysisRunResult
import com.correx.core.kernel.orchestration.StaticAnalysisRunner
import com.correx.core.kernel.retry.DefaultRetryCoordinator
import com.correx.core.risk.DefaultRiskAssessor
import com.correx.core.sessions.DefaultSessionReducer
import com.correx.core.sessions.DefaultSessionRepository
import com.correx.core.sessions.SessionProjector
import com.correx.core.sessions.projections.replay.DefaultEventReplayer
import com.correx.core.sessions.projections.replay.EventReplayer
import com.correx.core.toolintent.WorkspacePolicy
import com.correx.core.tools.contract.Tool
import com.correx.core.tools.registry.ToolRegistry
import com.correx.core.transitions.graph.StageConfig
import com.correx.core.transitions.graph.TransitionEdge
import com.correx.core.transitions.graph.WorkflowGraph
import com.correx.core.transitions.resolution.DefaultTransitionResolver
import com.correx.core.validation.pipeline.ValidationPipeline
import com.correx.infrastructure.persistence.InMemoryEventStore
import com.correx.infrastructure.tools.SandboxedToolExecutor
import com.correx.infrastructure.tools.shell.ShellTool
import com.correx.testing.contracts.fixtures.artifactstore.NoopArtifactStore
import com.correx.testing.fixtures.context.ContextFixtures
import com.correx.testing.fixtures.cyclePolicyMissingValidator
import kotlinx.coroutines.runBlocking
import kotlinx.datetime.Instant
import java.nio.file.Files
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertTrue
/**
* Retry-agency / recovery-routing coverage. A write-less `verify` stage runs `shell true` (so it
* passes the producer gate, exactly as #41's `final_verification` ran shell) then fails a
* `static_analysis` gate (fake runner always non-clean). Because `verify`'s allowedTools lack the
* `file_write` capability that gate requires, the orchestrator must NOT retry it in place (futile) —
* it opens a [FailureTicketOpenedEvent] and routes to the `recovery` stage (marked
* `metadata["role"]="recovery"`), which loops back to `verify`. After RECOVERY_ROUTE_BUDGET routes
* the run fails terminally.
*/
class RecoveryRoutingTest {
private val workspace = Files.createTempDirectory("recovery-routing-test")
/** Odd calls issue a `shell true` tool call; even calls end the round — repeats across re-entries. */
private class ShellThenStopProvider : InferenceProvider {
override val id = ProviderId("shell-caller")
override val name = "shell-caller"
override val tokenizer: Tokenizer = com.correx.testing.fixtures.inference.MockTokenizer()
private var callCount = 0
override suspend fun infer(request: InferenceRequest): InferenceResponse {
callCount++
return if (callCount % 2 == 1) {
InferenceResponse(
requestId = request.requestId,
text = "",
finishReason = FinishReason.ToolCall,
tokensUsed = TokenUsage(10, 5),
latencyMs = 0,
toolCalls = listOf(
ToolCallRequest(
id = "tc-$callCount",
function = ToolCallFunction(name = "shell", arguments = """{"argv":["true"]}"""),
),
),
)
} else {
InferenceResponse(
requestId = request.requestId,
text = "done",
finishReason = FinishReason.Stop,
tokensUsed = TokenUsage(10, 5),
latencyMs = 0,
)
}
}
override suspend fun healthCheck(): ProviderHealth = ProviderHealth.Healthy
override fun capabilities(): Set<CapabilityScore> = setOf(CapabilityScore(ModelCapability.General, 1.0))
}
private class SingleToolRegistry(private val tool: Tool) : ToolRegistry {
override fun resolve(name: String): Tool? = if (name == tool.name) tool else null
override fun all(): List<Tool> = listOf(tool)
}
private fun buildOrchestrator(): Pair<DefaultSessionOrchestrator, InMemoryEventStore> {
val eventStore = InMemoryEventStore()
val shell = ShellTool(allowedExecutables = setOf("true"), workingDir = workspace)
val toolRegistry = SingleToolRegistry(shell)
val executor = SandboxedToolExecutor(
delegate = shell,
registry = toolRegistry,
eventDispatcher = EventDispatcher(eventStore),
workDir = workspace,
artifactStore = NoopArtifactStore(),
)
val provider = ShellThenStopProvider()
val inferenceRouter = object : InferenceRouter {
override suspend fun route(stageId: StageId, requiredCapabilities: Set<ModelCapability>) = provider
}
val failingStaticAnalysis =
StaticAnalysisRunner { _, _ -> StaticAnalysisRunResult(exitCode = 1, output = "TS5023: bad option") }
val repositories = OrchestratorRepositories(
eventStore = eventStore,
inferenceRepository = InferenceRepository(object : EventReplayer<InferenceState> {
override fun rebuild(sessionId: SessionId) = InferenceState()
}),
orchestrationRepository = OrchestrationRepository(
DefaultEventReplayer(eventStore, OrchestrationProjector(DefaultOrchestrationReducer())),
),
sessionRepository = DefaultSessionRepository(
DefaultEventReplayer(eventStore, SessionProjector(DefaultSessionReducer())),
),
artifactRepository = com.correx.infrastructure.persistence.artifact.LiveArtifactRepository(
eventStore, DefaultArtifactReducer(),
),
approvalRepository = DefaultApprovalRepository(
DefaultEventReplayer(eventStore, ApprovalProjector(DefaultApprovalReducer())),
),
)
val engines = OrchestratorEngines(
transitionResolver = DefaultTransitionResolver { _, _ -> true },
contextPackBuilder = ContextFixtures.simpleBuilder(),
inferenceRouter = inferenceRouter,
validationPipeline = ValidationPipeline(validators = listOf(cyclePolicyMissingValidator())),
riskAssessor = DefaultRiskAssessor(),
toolExecutor = executor,
toolRegistry = toolRegistry,
workspacePolicy = WorkspacePolicy(workspace),
staticAnalysisRunner = failingStaticAnalysis,
approvalEngine = object : ApprovalEngine {
override fun evaluate(
request: DomainApprovalRequest,
context: ApprovalContext,
grants: List<ApprovalGrant>,
now: Instant,
) = ApprovalDecision(
id = null,
requestId = request.id,
outcome = ApprovalOutcome.AUTO_APPROVED,
state = ApprovalStatus.COMPLETED,
tier = request.tier,
contextSnapshot = context,
resolutionTimestamp = now,
reason = "test-auto-approve",
)
},
)
val decisionJournalRepository = DefaultDecisionJournalRepository(
DefaultEventReplayer(eventStore, DecisionJournalProjector(DefaultDecisionJournalReducer())),
)
val orchestrator = DefaultSessionOrchestrator(
repositories = repositories,
engines = engines,
retryCoordinator = DefaultRetryCoordinator(eventStore),
artifactStore = NoopArtifactStore(),
decisionJournalRepository = decisionJournalRepository,
)
return orchestrator to eventStore
}
// verify (write-less, failing static gate) --t-verify--> done
// recovery (role=recovery, holds file_write) has NO outgoing edge on purpose: the kernel returns
// it to the ticket's origin stage (recoveryReturnMove), so the recovery->verify loop is driven
// dynamically. If the return-to-sender path regresses, recovery dead-ends and the route count is
// wrong — this graph would fail rather than silently pass on a stale static edge.
private fun recoveryGraph(): WorkflowGraph = WorkflowGraph(
id = "recovery-routing-test",
stages = mapOf(
StageId("verify") to StageConfig(
allowedTools = setOf("shell"),
staticAnalysis = listOf("typecheck"),
),
StageId("recovery") to StageConfig(
allowedTools = setOf("shell", "file_write"),
metadata = mapOf("role" to "recovery"),
),
),
transitions = setOf(
TransitionEdge(TransitionId("t-verify"), StageId("verify"), StageId("done"), condition = { true }),
),
start = StageId("verify"),
)
@Test
fun `write-less gate failure routes to recovery and exhausts its own route budget`(): Unit = runBlocking {
val (orchestrator, eventStore) = buildOrchestrator()
val sessionId = SessionId("recovery-route")
// Generous in-place per-gate budget proves routing pre-empts it: without the agency guard,
// static_analysis would retry verify in place (never able to change the outcome) instead.
val config = OrchestrationConfig(retryPolicy = RetryPolicy(maxAttempts = 10, backoffMs = 0))
val result = orchestrator.run(sessionId, recoveryGraph(), config)
assertTrue(result is WorkflowResult.Failed, "route budget must exhaust terminally")
assertTrue((result as WorkflowResult.Failed).retryExhausted)
val events = eventStore.read(sessionId)
val tickets = events.mapNotNull { it.payload as? FailureTicketOpenedEvent }
assertEquals(2, tickets.size, "one ticket per route, bounded by RECOVERY_ROUTE_BUDGET")
tickets.forEach {
assertEquals("verify", it.stageId.value)
assertEquals("static_analysis", it.gate)
assertEquals("implementation", it.category)
assertEquals("file_write", it.requiredCapability)
assertEquals("recovery", it.routeTo.value)
}
assertEquals(listOf(1, 2), tickets.map { it.routeAttempt })
assertTrue(events.any { it.payload is WorkflowFailedEvent })
}
}