fix(events,tools,toolintent): failure attribution, one path normalization rule, file_copy (#713)

Three generic harness fixes from the web-ui postmortem dataset. Nothing here keys
on a language, framework, build tool or task type.

1. Failure attribution. WorkflowFailedEvent carries one primary FailureAttribution
   (AGENT | HARNESS | WORKFLOW | ENVIRONMENT | PROVIDER | OPERATOR | UNKNOWN),
   defaulted to UNKNOWN so pre-field events replay unchanged. FailureAttributor is
   the deterministic reason->layer mapping, used both at emission and when
   classifying history, so the baseline and the live metric are one measurement.
   Emission sites set it: failWorkflow derives from the reason unless the caller
   knows the layer, cancellation is OPERATOR, the server catch-all falls back to
   HARNESS, a grounding-rejected plan is AGENT. Multi-cause chains stay on
   FailureTicketOpened — no second causal structure.

   GET /metrics/failure-attribution (FailureAttributionInspectionService, mirroring
   ToolReliabilityInspectionService) reports counts, share, UNKNOWN share, the
   preserved reasons and the ticket categories from the same sessions. Read-only:
   historical events are classified at READ time and reported as `inferred`, never
   written back over an append-only log.

   Baseline over the local log, 122 terminal failures: AGENT 51 (41.8%),
   OPERATOR 28 (23.0%), WORKFLOW 19 (15.6%), PROVIDER 15 (12.3%), HARNESS 6 (4.9%),
   ENVIRONMENT 3 (2.5%), UNKNOWN 0.

2. The `~` guard bug. ToolPath is now the ONE canonical normalization rule
   (expand a leading `~`/`~/`, keep absolutes, anchor relatives on the session
   working dir). Every filesystem tool, all six plane-2 path rules and the approval
   preview resolve through it, so policy and existence checks inspect the path the
   tool will operate on. `~/.gradle/init.d/offline.gradle` used to resolve to
   `<workspace>/~/.gradle/...`: reported non-existent AND in-workspace, so the
   reference gate called a real file a hallucination and the out-of-workspace prompt
   never fired. Containment and external-read approval behaviour are unchanged —
   the expanded path is simply outside the workspace, where it always belonged.

3. file_copy (#713). A first-class tool with the writer's jail, tier, receipt,
   replay and CAS pre/post images; static and binary assets no longer move through
   the model's token stream. Needed one generic split: ParamRole.SOURCE_PATH marks a
   path a call reads FROM, so containment gates judge both params while write-target
   gates (read-before-write, stale-write, write scope, write manifest) judge the
   mutated one. ReadBeforeWriteRule exempts any call declaring a SOURCE_PATH: its
   content comes from disk, not from memory, and requiring a read of a binary is
   unsatisfiable. Existing tools declare no SOURCE_PATH, so their behaviour is
   byte-identical.

Tests: ToolPathTest (9), FailureAttributionTest (10), PathNormalizationRuleTest (6),
FileCopyToolTest (10), plus a home-relative FileReadTool read. ./gradlew check green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-27 11:57:49 +04:00
parent 519290368f
commit 6a8a7b31c1
36 changed files with 1232 additions and 79 deletions
@@ -7,6 +7,7 @@ import com.correx.core.toolintent.ToolCallAssessmentInput
import com.correx.core.toolintent.ToolCallRule
import com.correx.core.toolintent.maxAction
import com.correx.core.tools.contract.ToolCapability
import com.correx.core.tools.contract.ToolPath
import com.correx.core.validation.model.ValidationIssue
import com.correx.core.validation.model.ValidationSeverity
import java.nio.file.FileSystems
@@ -52,10 +53,8 @@ class ManifestContainmentRule : ToolCallRule {
val observations = mutableListOf<ToolCallObservation>()
var disposition = RiskAction.PROCEED
for (raw in candidatePathStrings(input.paramRoles, input.request.parameters)) {
val candidate = Path.of(raw)
val resolvedInput =
if (candidate.isAbsolute) candidate else input.workspace.workspaceRoot.resolve(candidate)
for (raw in writeTargetPathStrings(input.paramRoles, input.request.parameters)) {
val resolvedInput = ToolPath.resolve(raw, input.workspace.workspaceRoot)
val resolvedReal = input.probe.resolveReal(resolvedInput)
val inWorkspace = resolvedReal.startsWith(workspaceReal)
val relative = if (inWorkspace) workspaceReal.relativize(resolvedReal).toString() else raw
@@ -14,13 +14,31 @@ internal fun extractParamStrings(value: Any?): List<String> = when (value) {
}
/**
* The path-like argument strings of a tool call: the values of params declared
* [ParamRole.PATH], or — when none are declared — any string value that looks like a
* path. Shared by the path-containment and write-manifest rules so both judge exactly
* the same set of targets.
* Every path-like argument of a tool call the call's own targets ([ParamRole.PATH]) and any path
* it merely reads from ([ParamRole.SOURCE_PATH]). Used by the containment/existence gates, which
* must judge both: a source outside the workspace is as much an escape as a destination outside it.
*/
internal fun candidatePathStrings(paramRoles: Map<String, ParamRole>, parameters: Map<String, Any>): List<String> {
val declared = paramRoles.filterValues { it == ParamRole.PATH }.keys
internal fun candidatePathStrings(paramRoles: Map<String, ParamRole>, parameters: Map<String, Any>): List<String> =
pathStringsForRoles(paramRoles, parameters, setOf(ParamRole.PATH, ParamRole.SOURCE_PATH))
/**
* Only the paths a tool call MUTATES ([ParamRole.PATH]). Used by the write-target gates
* (read-before-write, stale-write, write scope, write manifest): blocking a copy because its
* SOURCE was never read, or charging a source against the task's write scope, would be wrong.
*/
internal fun writeTargetPathStrings(paramRoles: Map<String, ParamRole>, parameters: Map<String, Any>): List<String> =
pathStringsForRoles(paramRoles, parameters, setOf(ParamRole.PATH))
/**
* Values of the params declared with one of [roles] or — when the tool declares none of them (e.g.
* `shell`, whose param is an [ParamRole.EXEC_COMMAND]) — any string value that looks like a path.
*/
private fun pathStringsForRoles(
paramRoles: Map<String, ParamRole>,
parameters: Map<String, Any>,
roles: Set<ParamRole>,
): List<String> {
val declared = paramRoles.filterValues { it in roles }.keys
return if (declared.isNotEmpty()) {
declared.flatMap { extractParamStrings(parameters[it]) }
} else {
@@ -7,9 +7,9 @@ import com.correx.core.toolintent.ToolCallAssessmentInput
import com.correx.core.toolintent.ToolCallRule
import com.correx.core.toolintent.maxAction
import com.correx.core.tools.contract.ToolCapability
import com.correx.core.tools.contract.ToolPath
import com.correx.core.validation.model.ValidationIssue
import com.correx.core.validation.model.ValidationSeverity
import java.nio.file.Path
/**
* Effect-based path containment. Dispatches on FILE_READ / FILE_WRITE. For every
@@ -34,9 +34,7 @@ class PathContainmentRule : ToolCallRule {
var disposition = RiskAction.PROCEED
for (raw in candidatePathStrings(input.paramRoles, input.request.parameters)) {
val candidate = Path.of(raw)
val resolvedInput =
if (candidate.isAbsolute) candidate else input.workspace.workspaceRoot.resolve(candidate)
val resolvedInput = ToolPath.resolve(raw, input.workspace.workspaceRoot)
val resolvedReal = input.probe.resolveReal(resolvedInput)
val exists = input.probe.exists(resolvedInput)
val privileged = privilegedReal.any { resolvedReal.startsWith(it) }
@@ -6,7 +6,9 @@ import com.correx.core.toolintent.ToolCallAssessment
import com.correx.core.toolintent.ToolCallAssessmentInput
import com.correx.core.toolintent.ToolCallRule
import com.correx.core.toolintent.maxAction
import com.correx.core.tools.contract.ParamRole
import com.correx.core.tools.contract.ToolCapability
import com.correx.core.tools.contract.ToolPath
import com.correx.core.validation.model.ValidationIssue
import com.correx.core.validation.model.ValidationSeverity
import java.nio.file.Path
@@ -27,6 +29,12 @@ class ReadBeforeWriteRule : ToolCallRule {
ToolCapability.FILE_WRITE in capabilities
override fun assess(input: ToolCallAssessmentInput): ToolCallAssessment {
// A call that declares a SOURCE_PATH takes its content from a file on disk, not from the
// model's memory, so there is nothing for this gate to protect: demanding a file_read of the
// source (or of the target being replaced by it) would force the very bytes-through-context
// round-trip that such tools exist to avoid — and is unsatisfiable for a binary file.
if (input.paramRoles.containsValue(ParamRole.SOURCE_PATH)) return ToolCallAssessment()
val root = input.workspace.workspaceRoot
val readReal = input.session.reads.map { realOf(input, root, it) }.toSet()
@@ -34,7 +42,7 @@ class ReadBeforeWriteRule : ToolCallRule {
val observations = mutableListOf<ToolCallObservation>()
var disposition = RiskAction.PROCEED
for (raw in candidatePathStrings(input.paramRoles, input.request.parameters)) {
for (raw in writeTargetPathStrings(input.paramRoles, input.request.parameters)) {
val resolvedInput = resolveInput(root, raw)
val exists = input.probe.exists(resolvedInput)
val read = input.probe.resolveReal(resolvedInput) in readReal
@@ -58,10 +66,7 @@ class ReadBeforeWriteRule : ToolCallRule {
return ToolCallAssessment(issues = issues, observations = observations, disposition = disposition)
}
private fun resolveInput(root: Path, raw: String): Path {
val candidate = Path.of(raw)
return if (candidate.isAbsolute) candidate else root.resolve(candidate)
}
private fun resolveInput(root: Path, raw: String): Path = ToolPath.resolve(raw, root)
private fun realOf(input: ToolCallAssessmentInput, root: Path, raw: String): Path =
input.probe.resolveReal(resolveInput(root, raw))
@@ -7,9 +7,9 @@ import com.correx.core.toolintent.ToolCallAssessmentInput
import com.correx.core.toolintent.ToolCallRule
import com.correx.core.toolintent.maxAction
import com.correx.core.tools.contract.ToolCapability
import com.correx.core.tools.contract.ToolPath
import com.correx.core.validation.model.ValidationIssue
import com.correx.core.validation.model.ValidationSeverity
import java.nio.file.Path
/**
* Reference-must-exist gate (anti-hallucination). Dispatches on FILE_READ: a read of a path that is
@@ -37,8 +37,7 @@ class ReferenceExistsRule : ToolCallRule {
var disposition = RiskAction.PROCEED
for (raw in candidatePathStrings(input.paramRoles, input.request.parameters)) {
val candidate = Path.of(raw)
val resolvedInput = if (candidate.isAbsolute) candidate else root.resolve(candidate)
val resolvedInput = ToolPath.resolve(raw, root)
val exists = input.probe.exists(resolvedInput)
val inWorkspace = input.probe.resolveReal(resolvedInput).startsWith(workspaceReal)
@@ -7,6 +7,7 @@ import com.correx.core.toolintent.ToolCallAssessmentInput
import com.correx.core.toolintent.ToolCallRule
import com.correx.core.toolintent.maxAction
import com.correx.core.tools.contract.ToolCapability
import com.correx.core.tools.contract.ToolPath
import com.correx.core.validation.model.ValidationIssue
import com.correx.core.validation.model.ValidationSeverity
import java.nio.file.Path
@@ -35,7 +36,7 @@ class StaleWriteRule : ToolCallRule {
val observations = mutableListOf<ToolCallObservation>()
var disposition = RiskAction.PROCEED
for (raw in candidatePathStrings(input.paramRoles, input.request.parameters)) {
for (raw in writeTargetPathStrings(input.paramRoles, input.request.parameters)) {
val resolved = resolveInput(root, raw)
val real = input.probe.resolveReal(resolved)
val recorded = hashByReal[real]
@@ -60,10 +61,7 @@ class StaleWriteRule : ToolCallRule {
return ToolCallAssessment(issues = issues, observations = observations, disposition = disposition)
}
private fun resolveInput(root: Path, raw: String): Path {
val candidate = Path.of(raw)
return if (candidate.isAbsolute) candidate else root.resolve(candidate)
}
private fun resolveInput(root: Path, raw: String): Path = ToolPath.resolve(raw, root)
private fun realOf(input: ToolCallAssessmentInput, root: Path, raw: String): Path =
input.probe.resolveReal(resolveInput(root, raw))
@@ -7,10 +7,10 @@ import com.correx.core.toolintent.ToolCallAssessmentInput
import com.correx.core.toolintent.ToolCallRule
import com.correx.core.toolintent.maxAction
import com.correx.core.tools.contract.ToolCapability
import com.correx.core.tools.contract.ToolPath
import com.correx.core.validation.model.ValidationIssue
import com.correx.core.validation.model.ValidationSeverity
import java.nio.file.FileSystems
import java.nio.file.Path
/**
* Write-scope adherence. When the session has claimed a task that declared affected_paths, an
@@ -37,11 +37,8 @@ class WriteScopeRule : ToolCallRule {
val observations = mutableListOf<ToolCallObservation>()
var disposition = RiskAction.PROCEED
for (raw in candidatePathStrings(input.paramRoles, input.request.parameters)) {
val candidate = Path.of(raw)
val resolvedReal = input.probe.resolveReal(
if (candidate.isAbsolute) candidate else input.workspace.workspaceRoot.resolve(candidate),
)
for (raw in writeTargetPathStrings(input.paramRoles, input.request.parameters)) {
val resolvedReal = input.probe.resolveReal(ToolPath.resolve(raw, input.workspace.workspaceRoot))
if (!resolvedReal.startsWith(workspaceReal)) continue // out of workspace: not this gate
val rel = workspaceReal.relativize(resolvedReal)
val inScope = matchers.any { it.matches(rel) }
@@ -0,0 +1,149 @@
package com.correx.core.toolintent
import com.correx.core.events.events.ToolRequest
import com.correx.core.events.risk.RiskAction
import com.correx.core.events.types.SessionId
import com.correx.core.events.types.StageId
import com.correx.core.events.types.ToolInvocationId
import com.correx.core.toolintent.rules.PathContainmentRule
import com.correx.core.toolintent.rules.ReadBeforeWriteRule
import com.correx.core.toolintent.rules.ReferenceExistsRule
import com.correx.core.toolintent.rules.WriteScopeRule
import com.correx.core.tools.contract.ParamRole
import com.correx.core.tools.contract.ToolCapability
import java.nio.file.Path
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertTrue
/**
* Plane-2 must judge the SAME path the tool will operate on (ToolPath), and must judge a call's
* source path differently from its write target.
*
* The `~` cases pin the harness bug from the 2026-08 web-ui runs: a correct diagnosis of
* `~/.gradle/init.d/offline.gradle` was resolved to `<workspace>/~/.gradle/…`, so the reference gate
* called a real file a hallucination and the out-of-workspace prompt never fired.
*/
class PathNormalizationRuleTest {
private val workspace = Path.of("/work/project")
private val home: String = System.getProperty("user.home")
private val tildeTarget = "~/.gradle/init.d/offline.gradle"
private class FakeProbe(private val existing: Set<Path> = emptySet()) : WorldProbe {
override fun exists(path: Path): Boolean = path.toAbsolutePath().normalize() in existing
override fun resolveReal(path: Path): Path = path.toAbsolutePath().normalize()
}
private fun input(
parameters: Map<String, Any>,
capabilities: Set<ToolCapability>,
probe: WorldProbe,
paramRoles: Map<String, ParamRole> = emptyMap(),
reads: Set<String> = emptySet(),
tool: String = "file_read",
activeTask: ActiveTask? = null,
) = ToolCallAssessmentInput(
request = ToolRequest(ToolInvocationId("i"), SessionId("s"), StageId("st"), tool, parameters),
capabilities = capabilities,
workspace = WorkspacePolicy(workspace, emptyList()),
probe = probe,
paramRoles = paramRoles,
session = SessionContext(reads = reads, activeTask = activeTask),
)
@Test
fun `a tilde path is not mistaken for a non-existent in-workspace file`() {
val real = Path.of(home, ".gradle/init.d/offline.gradle")
val r = ReferenceExistsRule().assess(
input(
mapOf("path" to tildeTarget),
setOf(ToolCapability.FILE_READ),
FakeProbe(existing = setOf(real)),
),
)
assertEquals(RiskAction.PROCEED, r.disposition)
assertTrue(r.issues.isEmpty(), "a real home-directory file must not be reported as a hallucination")
assertEquals("true", r.observations.single().facts["exists"])
assertEquals("false", r.observations.single().facts["inWorkspace"])
}
@Test
fun `a tilde path still prompts as an out-of-workspace read`() {
// Containment behaviour is preserved: expansion moves the path OUT of the workspace, which is
// where it always belonged, so the operator approval fires instead of being silently skipped.
val r = PathContainmentRule().assess(
input(mapOf("path" to tildeTarget), setOf(ToolCapability.FILE_READ), FakeProbe()),
)
assertEquals(RiskAction.PROMPT_USER, r.disposition)
assertEquals("PATH_OUTSIDE_WORKSPACE", r.issues.single().code)
assertEquals(Path.of(home, ".gradle/init.d/offline.gradle").toString(), r.observations.single().facts["resolved"])
}
@Test
fun `containment judges a source path as well as a write target`() {
val r = PathContainmentRule().assess(
input(
mapOf("source" to "/etc/hosts", "dest" to "public/hosts"),
setOf(ToolCapability.FILE_WRITE),
FakeProbe(),
paramRoles = mapOf("source" to ParamRole.SOURCE_PATH, "dest" to ParamRole.PATH),
tool = "file_copy",
),
)
assertEquals(RiskAction.PROMPT_USER, r.disposition)
assertEquals(2, r.observations.size)
}
@Test
fun `the write-scope gate judges the write target only, not the source`() {
// A source path is read, not mutated, so it is not charged against the task's write scope —
// otherwise every copy of an in-repo asset would have to widen affected_paths to include it.
val r = WriteScopeRule().assess(
input(
mapOf("source" to "assets/logo.png", "dest" to "public/logo.png"),
setOf(ToolCapability.FILE_WRITE),
FakeProbe(),
paramRoles = mapOf("source" to ParamRole.SOURCE_PATH, "dest" to ParamRole.PATH),
tool = "file_copy",
activeTask = ActiveTask("42", listOf("public/**")),
),
)
assertEquals(RiskAction.PROCEED, r.disposition)
assertEquals(listOf("public/logo.png"), r.observations.map { it.facts["path"] })
}
@Test
fun `a content-from-disk call is exempt from read-before-write even when the dest exists`() {
// Overwriting an existing binary via a copy must not demand a file_read of it first — that
// read is impossible to satisfy usefully and is exactly the round-trip file_copy removes.
val dest = Path.of("/work/project/public/logo.png")
val r = ReadBeforeWriteRule().assess(
input(
mapOf("source" to "assets/logo.png", "dest" to "public/logo.png"),
setOf(ToolCapability.FILE_WRITE),
FakeProbe(existing = setOf(dest)),
paramRoles = mapOf("source" to ParamRole.SOURCE_PATH, "dest" to ParamRole.PATH),
tool = "file_copy",
),
)
assertEquals(RiskAction.PROCEED, r.disposition)
assertTrue(r.issues.isEmpty())
}
@Test
fun `a model-authored write still requires a prior read`() {
val target = "/work/project/src/A.kt"
val r = ReadBeforeWriteRule().assess(
input(
mapOf("path" to target),
setOf(ToolCapability.FILE_WRITE),
FakeProbe(existing = setOf(Path.of(target))),
paramRoles = mapOf("path" to ParamRole.PATH),
tool = "file_write",
),
)
assertEquals(RiskAction.BLOCK, r.disposition)
assertEquals("READ_BEFORE_WRITE", r.issues.single().code)
}
}