The exemption added in 6a8a7b31 was broader than the invariant it stood on. "Tool
declares a SOURCE_PATH" is a claim about the parameter list; the safe property is
"every byte written derives from an existing source object rather than from
model-supplied content". A future transform or import tool could name a source and
still write model-controlled output, and would have inherited the exemption.
ToolCapability.CONTENT_FROM_SOURCE now carries that provenance claim explicitly.
file_copy declares it; ReadBeforeWriteRule.appliesTo stands down only for calls that
do, so ToolCallAssessor skips the rule rather than the rule skipping itself. The
capability is recorded on the invocation event like every other one, so replay
classifies a call by what it actually claimed instead of re-deriving it from
parameters.
Tool availability is by declared tool name, not capability-set containment, so the
extra capability does not narrow which stages can reach file_copy.
Tests: the exemption is asserted through ToolCallAssessor, plus a source-naming tool
WITHOUT the provenance capability that stays gated. ./gradlew check green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three generic harness fixes from the web-ui postmortem dataset. Nothing here keys
on a language, framework, build tool or task type.
1. Failure attribution. WorkflowFailedEvent carries one primary FailureAttribution
(AGENT | HARNESS | WORKFLOW | ENVIRONMENT | PROVIDER | OPERATOR | UNKNOWN),
defaulted to UNKNOWN so pre-field events replay unchanged. FailureAttributor is
the deterministic reason->layer mapping, used both at emission and when
classifying history, so the baseline and the live metric are one measurement.
Emission sites set it: failWorkflow derives from the reason unless the caller
knows the layer, cancellation is OPERATOR, the server catch-all falls back to
HARNESS, a grounding-rejected plan is AGENT. Multi-cause chains stay on
FailureTicketOpened — no second causal structure.
GET /metrics/failure-attribution (FailureAttributionInspectionService, mirroring
ToolReliabilityInspectionService) reports counts, share, UNKNOWN share, the
preserved reasons and the ticket categories from the same sessions. Read-only:
historical events are classified at READ time and reported as `inferred`, never
written back over an append-only log.
Baseline over the local log, 122 terminal failures: AGENT 51 (41.8%),
OPERATOR 28 (23.0%), WORKFLOW 19 (15.6%), PROVIDER 15 (12.3%), HARNESS 6 (4.9%),
ENVIRONMENT 3 (2.5%), UNKNOWN 0.
2. The `~` guard bug. ToolPath is now the ONE canonical normalization rule
(expand a leading `~`/`~/`, keep absolutes, anchor relatives on the session
working dir). Every filesystem tool, all six plane-2 path rules and the approval
preview resolve through it, so policy and existence checks inspect the path the
tool will operate on. `~/.gradle/init.d/offline.gradle` used to resolve to
`<workspace>/~/.gradle/...`: reported non-existent AND in-workspace, so the
reference gate called a real file a hallucination and the out-of-workspace prompt
never fired. Containment and external-read approval behaviour are unchanged —
the expanded path is simply outside the workspace, where it always belonged.
3. file_copy (#713). A first-class tool with the writer's jail, tier, receipt,
replay and CAS pre/post images; static and binary assets no longer move through
the model's token stream. Needed one generic split: ParamRole.SOURCE_PATH marks a
path a call reads FROM, so containment gates judge both params while write-target
gates (read-before-write, stale-write, write scope, write manifest) judge the
mutated one. ReadBeforeWriteRule exempts any call declaring a SOURCE_PATH: its
content comes from disk, not from memory, and requiring a read of a binary is
unsatisfiable. Existing tools declare no SOURCE_PATH, so their behaviour is
byte-identical.
Tests: ToolPathTest (9), FailureAttributionTest (10), PathNormalizationRuleTest (6),
FileCopyToolTest (10), plus a home-relative FileReadTool read. ./gradlew check green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
file_edit failed en masse for small models on two ergonomics traps:
- replace() rejected calls that sent `content` (the append param) instead of
`replacement` — intent unambiguous; now accepted as an alias.
- exact-string match died on leading-whitespace drift (model can't reproduce
indentation). Added whitespace-flexible line matching + replacement reindent,
wired into both the pre-exec validation gate and replace().
Also made the WRITE_SCOPE / PATH_OUTSIDE_MANIFEST block messages emit a literal
copy-pasteable task_update(id=..., affected_paths=[...]) call and warn against
action=block — the exact wrong turn models kept taking (18x in one session).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rdo9fe7SujNVeyZA8YkpkD
Two failed web-ui freestyle runs dead-ended on the write manifest. The
scaffold_frontend stage declared writes=[frontend/package.json,
frontend/vite.config.ts], so every other file the scaffold produced
(tsconfig, src/main.tsx, index.html, App.tsx) was BLOCKED as
PATH_OUTSIDE_MANIFEST with no agent-facing escape hatch — unlike WRITE_SCOPE,
which advertises task_update. Retry exhausted -> WorkflowFailed.
- ManifestContainmentRule: a write already inside the active task's
affected_paths is allowed even when the stage manifest is narrower. The
task scope is the agent-widenable, recorded authority (see WriteScopeRule);
the stage manifest is a planner hint that defers to it. Block message now
names the remedy (widen affected_paths via task_update).
- architect_freestyle prompt: a scaffold/generator stage must declare its
`writes` as a covering directory glob (frontend/**), not enumerate files,
and use ** not * — frontend/* does not cover frontend/src/main.tsx.
core:toolintent green (78 tests), detekt clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Greenfield discovery/analyst stages thrashed on plane-2 REFERENCE_EXISTS
rejections: the model listed a not-yet-created dir (e.g. frontend/), got
hard-BLOCKED every round, and burned MAX_TOOL_ROUNDS without progress.
- New ToolCapability.DIRECTORY_LIST; ListDirTool declares it alongside
FILE_READ. ReferenceExistsRule now exempts directory enumeration (a
listing of an absent dir truthfully reports empty; only hallucinated
file *reads* still fail loud). Dispatch stays on capability, not name.
- Stage-agnostic rejection-loop breaker in the tool loop: after 3 rounds
where every tool call is rejected (BLOCKED/ERROR) with no success,
force the model to produce its output. Covers JSON-emitting stages the
read-loop breaker (file_written-only) never protected.
Bundles three operator-reliability guardrails (Vikunja #28/#29/#30) plus the
in-flight branch WIP they were built on top of (reasoning_content capture,
operator/project profile editor, write-jail workspaceRoot fix) — the tree is
interdependent (SessionOrchestrator references reasoningArtifactId from the WIP)
and does not compile as separable subsets, so it lands as one commit.
Guardrails:
- #28 mid-stage steering: ClientMessage.SteerSession -> GlobalStreamHandler ->
orchestrator.submitSteering, reusing SteeringNoteAddedEvent + existing context
fold (advisory, non-authoritative; invariants #3/#7). Closes the gap where
steering typed off an approval gate was silently dropped.
- #29 shell-in-file guardrail: ShellInFileContentRule (core:toolintent) blocks a
file_write whose content is a bare shell command (e.g. "mkdir -p ..."); FileWriteTool
description now advertises auto-mkdir of parent dirs. Basename-allowlist so the
extensionless case is caught; scripts/Makefiles/multiline exempt.
- #30 pt1 capability-gap detector: deterministic CapabilityGapDetector maps stage
intent -> implied ToolCapability, compares to granted tools, emits advisory
CapabilityGapDetectedEvent in FreestyleDriver.lockAndRun. Recorded, never fails
the gate and never auto-grants (invariants #3/#4/#5). Reflection rung is pt2.
Verified: ./gradlew check green (whole tree).
Three tool-path correctness fixes:
- Tool-execution events were emitted twice — SandboxedToolExecutor and
SessionOrchestrator.recordToolExecution both emitted Completed/Failed/
FileWritten per call (double TUI rows, double-counted metrics). Split
ownership: orchestrator is the sole recorder of Completed/Failed (truncates
output, drives the read-before-write gate); executor owns Started +
FileWritten (pre/post-image hashes it alone can capture).
- computeToolPreview still gated on operation == "write", but file_write lost
its operation param when delete was split out, so it bailed to raw-args for
every write (approval card showed raw JSON). Drop the dead guard.
- A file_read blocked by REFERENCE_EXISTS can never complete, so it could never
lift read-only mode: a stage writing a NEW file deadlocked (writes filtered,
every read of the not-yet-created target blocked) until MAX_TOOL_ROUNDS.
Lift read-only on a REFERENCE_EXISTS block, and reword the block message to
tell the model to file_write directly. Regression test added.
Block writing a file whose on-disk content changed since the session read it — a
concurrent/external edit the agent's view doesn't reflect. file_read records a
content hash on whole-file reads; the orchestrator's completion path now carries
the tool's structuredOutput (it previously dropped it, unlike SandboxedToolExecutor
— the two paths were inconsistent), so SessionContext.readHashes folds it.
StaleWriteRule compares the current hash (WorldProbe.contentHash) against the
read-time hash and BLOCKs a mismatch, telling the agent to re-read first. Files the
session wrote itself are excluded, so its own edits never look stale; partial reads
set no baseline.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When a session has claimed a task with declared affected_paths, an in-workspace
write outside that scope is BLOCKED — but declarably: the message tells the agent
to widen the task's affected_paths via task_update (which re-records the scope)
and retry. Legitimate, unforeseen changes (a dependency, a caller) are never
trapped, only forced to become a recorded, auditable scope change. Reads the
session's activeTask scope (recorded at claim) — no cross-stream lookup.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claiming a task now emits SessionWorkingTaskEvent{taskId, affectedPaths} into the
session's own stream (via a SessionFactRecorder port + event-store adapter), and
SessionContextProjection folds it into SessionContext.activeTask. So a gate learns
"which task is this session working, and its scope" by reading the session stream
— no cross-stream scan, no kernel->tasks dependency. Re-emitted on an affected_paths
edit by the claimant, so the snapshot scope stays current; latest wins on replay.
This is the data source the write-scope gate needs (next).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the single-purpose sessionReadPaths plumbing with a SessionContext
{reads, writes, activeTask} folded by SessionContextProjection over the session
stream — the read-model every anti-hallucination gate consumes (plane-2 rules via
ToolCallAssessmentInput.session; tool gates via a port, next).
- reads: completions whose recorded capability includes FILE_READ.
- writes: the resolved paths the orchestrator already records on each completion's
receipt.affectedEntities (FileAffectingTool) — no param-parsing reinvention.
- activeTask: null for now; populated once claim records a session fact.
ReadBeforeWriteRule now reads input.session.reads; ReadFilesProjection is retired.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Capability is the canonical signal the plane-2 gates dispatch on, but
ToolInvocationRequestedEvent only carried the tool name — so ReadFilesProjection
classified reads by a hardcoded "file_read" string, bypassing the abstraction and
(worse) re-deriving a semantic fact at replay from whatever the name meant. That
violates the event-sourcing invariant that replay reads recorded facts, not live
state.
Now the tool's requiredCapabilities are recorded on the event at emission, and the
projection classifies by ToolCapability.FILE_READ. To let the lowest module carry
it, ToolCapability moves from core:tools into core:events keeping its package, so
every existing import resolves unchanged and core:tools imports it from below. The
field is defaulted, so pre-existing events deserialize as empty.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A file_read of a path that is inside the workspace but does not exist is now
BLOCKED with "no such path" instead of silently returning empty — so an agent
that hallucinated a filename fails loud and self-corrects (list the directory,
fix the name) rather than proceeding on an absent reference. Out-of-workspace
targets stay PathContainmentRule's concern, so the gates don't double-handle.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An LLM that edits or overwrites a file it never read is acting on hallucinated
contents. This plane-2 ToolCallRule blocks it: dispatching on FILE_WRITE (covers
file_write and file_edit), a write to a path that exists on disk but was not
file_read earlier this session is BLOCKED. A brand-new file passes — you can't
read what doesn't exist, and creating is legitimate.
- ReadFilesProjection folds the session's file_read events (request + completion,
so a failed read doesn't count) into the set of paths read, mirroring
EgressAllowlistProjection; threaded into ToolCallAssessmentInput.sessionReadPaths
via SessionOrchestrator.resolveSessionReadPaths.
- Read paths and the write target are both resolved through the probe (toRealPath),
so spelling differences and symlinks can't dodge the gate.
- The orchestrator already feeds a block's rationale back as a tool result, so the
agent reads the file and retries; the rejected-event reason now carries that
rationale too (specific audit trail instead of a generic string).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ToolCallAssessmentInput gains sessionEgressHosts; SessionOrchestrator resolves it at
the assessment build site by folding EgressHostsGrantedEvents through
EgressAllowlistProjection. NetworkHostRule now enforces session-granted hosts (union
with static), replacing the emptySet() TODO — the allowlist built in 027ff1f is live.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- EgressHostsGrantedEvent (registered + serialization test): additive per-session
egress grants (e.g. an approved research source list)
- EgressAllowlist pure union helper + EgressAllowlistProjection (folds grants per
session); NetworkHostRule delegates to the union, preserving exact/suffix semantics
- rule not yet fed the session set live (ToolCallAssessmentInput has no sessionId);
precise TODO(wiring) left. batch fetch-approval skipped (needs approval-flow surgery)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Scope creep — an implementer writing files it never declared — is the
dominant local-model failure that compiler/tests don't catch. A stage can
now declare a `writes` manifest (workspace-relative globs); a FILE_WRITE
that resolves inside the workspace but outside the declared set is raised
as PATH_OUTSIDE_MANIFEST → BLOCK, so the model gets the violation as tool
feedback and retries within scope (the §2 bounded-retry signal).
Implemented as a plane-2 rule beside PathContainmentRule (same effect-based
dispatch on FILE_WRITE, same symlink-safe WorldProbe resolution, same
recorded observations so replay reads them back — invariant #9). An empty
manifest is unrestricted; out-of-workspace targets stay PathContainmentRule's
concern, so the two rules don't double-flag.
- StageConfig.writeManifest + TomlWorkflowLoader `writes` parsing
- ToolCallAssessmentInput.writeManifest, threaded from the active stage via
SessionOrchestrator.runPlane2Assessment
- new ManifestContainmentRule, registered in the server assessor
- shared candidatePathStrings extracted so both path rules judge the same
target set
- role_pipeline.toml documents the option on the implementer stage
The dedicated ManifestViolationEvent the spec names is not added: the
violation is already recorded replayably in ToolCallAssessedEvent (issue +
observations + BLOCK) and surfaced in the TUI rationale band. A first-class
event is worth adding only once reconciliation consumes it.
Add ParamRole and a defaulted Tool.paramRoles to the tool contract; built-in
tools declare which params carry effects: file_read/write/edit -> path=PATH,
shell -> argv=EXEC_COMMAND. PathContainmentRule / ExecInterpreterRule /
NetworkHostRule now inspect only the declared params when roles are present, and
fall back to the looksLikePath / leading-token / extractHost heuristic only for
un-enriched (e.g. custom) tools - so a custom FILE_WRITE tool is still
containment-checked. Shared extractParamStrings flattens String and List<*>
(argv) values. SessionOrchestrator.runPlane2Assessment resolves the tool once
and passes tool.paramRoles into the assessment input.
This retires the arg-guessing from the first-party path: a slashy edit 'content'
arg is no longer mis-flagged as a path.
Implements the full vertical slice for invariant #9 tool-call assessment:
- core:toolintent — new module with ToolCallRule seam, ToolCallAssessor,
WorldProbe/FileSystemWorldProbe, WorkspacePolicy, PathContainmentRule,
and RiskMapping (assessment → RiskSummary / AssessedIssue)
- core:tools — ToolCallAssessmentRecord + ToolInvocationRecord.assessment
field; DefaultToolReducer handles ToolCallAssessedEvent (replay proof)
- core:config — ToolsConfig gains workspaceRoot + privilegedLocations;
ConfigLoader parses both; DEFAULT_PRIVILEGED_LOCATIONS built-in
- core:kernel — OrchestratorEngines gains toolCallAssessor/workspacePolicy/
worldProbe fields; SessionOrchestrator.dispatchToolCalls runs
runPlane2Assessment before the tier gate: BLOCK → hard-reject without
executing; PROMPT_USER → elevates tier into approval path with plane2Risk
in the ApprovalRequestedEvent
- apps/server — constructs PathContainmentRule + ToolCallAssessor +
WorkspacePolicy from config and wires them into OrchestratorEngines
Assessment is recorded as ToolCallAssessedEvent (environment observed once,
facts stored, replay reads events — invariant #9). Assessor and WorldProbe
are only invoked on the live orchestrator path, never in replay.
claude
kami