feat(recovery): failure-ticket routing + review-gate RECOVER + return-to-sender
Retry-agency invariant: a stage may only retry a gate it has the capability to change. A write-less stage failing a build/contract/static gate (e.g. freestyle final_verification with allowedTools=[shell]) can never fix it, so retrying in place is futile until the budget drains (Vikunja #41). Instead: open a FailureTicketOpenedEvent (category + requiredCapability derived deterministically from the gate id, no LLM) and route to a recovery stage that holds the capability, bounded by a small per-stage route budget. - Slice 2: SalvageDecision.RECOVER (ternary CONTINUE/RECOVER/FAIL) lets the review-gate salvage judge hand off to recovery. Shared routeToRecovery() unifies the deterministic agency guard and the judge on one destination; decideGateExhaustion now returns StepResult?. - Return-to-sender: recovery's exit is dynamic (recoveryReturnMove) — it goes back to the exact ticket-origin stage to re-run its gate, so a write-less gate anywhere in the graph is handled without skipping intervening stages. The synthesized recovery->terminal edge is now only a no-ticket fallback. - Freestyle wiring: ExecutionPlanCompiler injectRecovery flag (Main passes true) synthesizes a write-capable recovery stage; ticket evidence reaches it via buildRecoveryTicketEntry. - Read-only tools always present: StageConfig.effectiveAllowedTools adds file_read/list_dir to any tool-granting stage (fixes the verifier reaching for `shell ls -R` to inspect the tree). Tests: RecoveryRoutingTest (deterministic gate, no static return edge — proves dynamic return) + GateRetryBudgetExhaustionTest RECOVER case + two compiler tests. All green; detekt clean.
This commit is contained in:
+58
-5
@@ -15,6 +15,20 @@ import com.fasterxml.jackson.module.kotlin.kotlinModule
|
||||
import com.fasterxml.jackson.module.kotlin.readValue
|
||||
|
||||
private const val TERMINAL = "done"
|
||||
private const val RECOVERY_STAGE = "recovery"
|
||||
|
||||
// Synthesized recovery stage prompt. Freestyle plans are LLM-emitted and never declare a recovery
|
||||
// stage, but the kernel's retry-agency guard routes a write-less stage's unfixable gate failure to
|
||||
// one when it exists. So the compiler always injects it (inert unless a gate actually routes): a
|
||||
// write-capable stage that reads the failure ticket from context, repairs the cause, and hands
|
||||
// control back to verification.
|
||||
private const val RECOVERY_PROMPT =
|
||||
"You are the recovery stage. An upstream stage failed a verification gate it could not fix " +
|
||||
"(it lacked write access). The failure ticket — the failing stage, the gate, and its exact " +
|
||||
"output — is in your context under \"## Recovery ticket\". Read the gate output, inspect the " +
|
||||
"relevant files (file_read / list_dir), and make the minimal edits that fix the reported " +
|
||||
"cause. Do not re-scaffold or rewrite working code. When done, control returns to the stage " +
|
||||
"that failed so its gate re-runs."
|
||||
|
||||
// Freestyle plans don't specify a per-stage token budget, so without this every compiled stage
|
||||
// inherits StageConfig's 4096 default — 1/8th of what the static role_pipeline execution stages
|
||||
@@ -32,6 +46,12 @@ class ExecutionPlanCompiler(
|
||||
// freestyle driver feeds back for a retry). Empty = the caller didn't supply the tool
|
||||
// universe, so validation is skipped (preserves the bare `ExecutionPlanCompiler(registry)`).
|
||||
private val knownTools: Set<String> = emptySet(),
|
||||
// When true, synthesize a write-capable `recovery` stage (+ a recovery->terminal edge) into every
|
||||
// compiled graph. The kernel's retry-agency guard routes a write-less stage's unfixable gate
|
||||
// failure there; without it the guard finds no recovery stage and falls back to futile in-place
|
||||
// retries (Vikunja #41). Off by default so the compiler's own unit tests see only plan stages;
|
||||
// the server's freestyle path (Main.kt) turns it on.
|
||||
private val injectRecovery: Boolean = false,
|
||||
) {
|
||||
private val mapper = JsonMapper.builder()
|
||||
.addModule(kotlinModule())
|
||||
@@ -100,6 +120,32 @@ class ExecutionPlanCompiler(
|
||||
metadata = mapOf("promptInline" to s.prompt),
|
||||
)
|
||||
}
|
||||
// Inject a recovery stage unless the plan already declares one. Reached only by the kernel's
|
||||
// failure-ticket routing (no incoming plan edge), so it is excluded from the reachability
|
||||
// check below. At run time the kernel returns recovery to the exact stage that failed
|
||||
// (recoveryReturnMove, from the ticket origin); the recovery->terminal edge synthesized here
|
||||
// is only a fallback for the impossible case of a recovery stage reached with no ticket, so
|
||||
// the resolver always has a valid outgoing edge and can never dead-end.
|
||||
val terminal = terminalStageId(plan)
|
||||
val recoveryStage: Pair<StageId, StageConfig>? =
|
||||
if (!injectRecovery || RECOVERY_STAGE in stageMap.keys.map { it.value }) {
|
||||
null
|
||||
} else {
|
||||
StageId(RECOVERY_STAGE) to StageConfig(
|
||||
allowedTools = setOf("file_write", "shell"),
|
||||
tokenBudget = DEFAULT_STAGE_TOKEN_BUDGET,
|
||||
metadata = mapOf("role" to "recovery", "promptInline" to RECOVERY_PROMPT),
|
||||
)
|
||||
}
|
||||
val recoveryEdge: TransitionEdge? = recoveryStage?.let {
|
||||
TransitionEdge(
|
||||
id = TransitionId("$RECOVERY_STAGE->$terminal"),
|
||||
from = StageId(RECOVERY_STAGE),
|
||||
to = StageId(terminal),
|
||||
condition = ConditionSpec(type = "always_true").toCondition(),
|
||||
)
|
||||
}
|
||||
val fullStageMap = stageMap + listOfNotNull(recoveryStage)
|
||||
val declared = stageMap.keys.map { it.value }.toSet()
|
||||
|
||||
val transitions = plan.edges.map { e ->
|
||||
@@ -123,11 +169,13 @@ class ExecutionPlanCompiler(
|
||||
operator = e.condition.operator,
|
||||
).toCondition(),
|
||||
)
|
||||
}.toSet()
|
||||
}.toSet() + listOfNotNull(recoveryEdge)
|
||||
|
||||
val start = StageId(plan.stages.first().id)
|
||||
validateReachability(declared, transitions, start)
|
||||
return WorkflowGraph(id = workflowId, stages = stageMap, transitions = transitions, start = start)
|
||||
// The recovery stage is reached by kernel routing, not a plan edge, so exclude it from the
|
||||
// "unreachable from start" check (it would otherwise always trip it).
|
||||
validateReachability(declared, transitions, start, excluded = setOfNotNull(recoveryStage?.first?.value))
|
||||
return WorkflowGraph(id = workflowId, stages = fullStageMap, transitions = transitions, start = start)
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -174,7 +222,12 @@ class ExecutionPlanCompiler(
|
||||
|
||||
private fun isGlob(path: String): Boolean = path.any { it == '*' || it == '?' || it == '[' }
|
||||
|
||||
private fun validateReachability(declared: Set<String>, transitions: Set<TransitionEdge>, start: StageId) {
|
||||
private fun validateReachability(
|
||||
declared: Set<String>,
|
||||
transitions: Set<TransitionEdge>,
|
||||
start: StageId,
|
||||
excluded: Set<String> = emptySet(),
|
||||
) {
|
||||
// A declared stage with no path from start can never run — typical LLM topology
|
||||
// failure: every stage wired straight to "done" instead of chained, which silently
|
||||
// truncates the plan to its first stage.
|
||||
@@ -187,7 +240,7 @@ class ExecutionPlanCompiler(
|
||||
.filter { it != TERMINAL && reachable.add(it) }
|
||||
.toSet()
|
||||
}
|
||||
val unreachable = declared - reachable
|
||||
val unreachable = declared - reachable - excluded
|
||||
if (unreachable.isNotEmpty()) {
|
||||
throw WorkflowValidationException(
|
||||
"stages unreachable from start '${start.value}': ${unreachable.sorted().joinToString(", ")} — " +
|
||||
|
||||
+37
@@ -18,6 +18,7 @@ class ExecutionPlanCompilerTest {
|
||||
it.register(ConfigArtifactKind(id = "patch", schema = trivialSchema, llmEmitted = true))
|
||||
}
|
||||
private val compiler = ExecutionPlanCompiler(registry)
|
||||
private val recoveringCompiler = ExecutionPlanCompiler(registry, injectRecovery = true)
|
||||
|
||||
private val validPlan = """
|
||||
{
|
||||
@@ -72,6 +73,42 @@ class ExecutionPlanCompilerTest {
|
||||
assertEquals(setOf("patch"), reviewStage.needs.map { it.value }.toSet())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `injectRecovery synthesizes a write-capable recovery stage looping back to the terminal`() {
|
||||
val graph = recoveringCompiler.compile(validPlan, "recovery-workflow")
|
||||
|
||||
// Two plan stages + the synthesized recovery stage; still reachable-valid (recovery is
|
||||
// excluded from the reachability check because it's entered by kernel routing, not an edge).
|
||||
assertEquals(3, graph.stages.size)
|
||||
val recovery = graph.stages[StageId("recovery")]
|
||||
assertNotNull(recovery)
|
||||
assertEquals("recovery", recovery.metadata["role"])
|
||||
assertTrue(recovery.allowedTools.contains("file_write"))
|
||||
// file_read / list_dir are always available to any tool-granting stage.
|
||||
assertTrue(recovery.effectiveAllowedTools.contains("file_read"))
|
||||
|
||||
// recovery -> review (the plan's terminal stage, the one whose edge points at 'done').
|
||||
val recoveryEdge = graph.transitions.single { it.from == StageId("recovery") }
|
||||
assertEquals(StageId("review"), recoveryEdge.to)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `injectRecovery is a no-op when the plan already declares a recovery stage`() {
|
||||
val plan = """
|
||||
{
|
||||
"goal": "x",
|
||||
"stages": [
|
||||
{ "id": "recovery", "prompt": "fix", "produces": "patch", "needs": [], "tools": ["file_write"] }
|
||||
],
|
||||
"edges": [
|
||||
{ "from": "recovery", "to": "done", "condition": { "type": "always_true" } }
|
||||
]
|
||||
}
|
||||
""".trimIndent()
|
||||
val graph = recoveringCompiler.compile(plan, "wf")
|
||||
assertEquals(1, graph.stages.size)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `plan-declared writes are derived into the stage write manifest`() {
|
||||
val plan = """
|
||||
|
||||
Reference in New Issue
Block a user