Commit Graph

154 Commits

Author SHA1 Message Date
kami 159b3f1eb9 fix(build-gate): close two frontend COMPLETE-lie holes (#277)
Run 771c0b96 marked COMPLETE with a frontend that did not build. Two
verification holes let a broken import through:

1. A MODULE build_expectation delegates to LSP and the execution gate
   returned Success trusting it — but tsserver failed to initialize every
   stage, so empty diagnostics read as clean. runExecutionGate now only
   trusts the MODULE->LSP short-circuit when the LSP run actually ran
   (new lspDiagnosticsSkipped projection); on skip it falls through to the
   real build command.

2. ExecutionPlanCompiler disabled the terminal whole-project auto build gate
   whenever ANY stage declared a build_expectation — so a MODULE (typecheck-
   only) declaration removed the real `npm run build` floor. Now only a real
   whole-project build (PROJECT/TESTS) suppresses the auto gate; MODULE/NONE
   do not. Extracted autoGateStages() helper. Two new compiler tests.

core:kernel + infrastructure:workflow tests green; no new detekt.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 11:13:22 +04:00
kami b43ab77eee feat(server): REST /clarify route for headless clarification answers (#42)
Discovery-stage clarifications could only be answered over WebSocket
(ClientMessage.ClarificationResponse), so the curl-based headless QA
driver parked forever at discovery.

Add POST /sessions/{id}/clarify mirroring approveStageRoute. The server
resolves the live (stageId, requestId) from the session id via
SessionOrchestrator.pendingClarificationFor() — the newest still-live,
unanswered ClarificationRequestedEvent from the event log — so a curl
caller need not know the requestId. Empty answers = free-text skip.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 23:18:47 +04:00
kami 95b16a5047 fix(weak-model-gates): stop three gates from stalling weak stage models
Diagnosed from session 508c8d58 (frontend freestyle run, WorkflowFailed):
three independent weak-model-hostile gates, none a model-capability problem.

- shell: split a collapsed single-string command line (["npm create vite …"])
  into tokens instead of rejecting it as a "collapsed array". The model
  reliably re-emits this shape; rejecting looped bootstrap_frontend until
  stage_loop_break. JSON-escape mangles (quotes/commas in argv[0]) stay rejected.
- recovery: a stage_loop_break route now gets a "Stuck-loop ticket", not the
  "Contract arbitration ticket". The arbitration prompt told the model to read
  and reconcile "the files named below" — but a tool-syntax loop names zero
  files, sending the recovery agent grepping the repo for 40+ turns until the
  repair ladder exhausted.
- plan lint: H3 (unreferenced_prompt_artifact) demoted from hard failure to
  soft finding, and seeds excluded from it. It word-matches artifact IDs in
  free prose and cannot tell a forgotten dep from a descriptive mention, so as
  a hard gate it burned architect retries on words it couldn't reword away
  (analysis/dod). Hard tier is now deterministic graph facts only (H1/H2).

Tests: ShellToolTest, PlanLinterTest, RecoveryRoutingTest green; detekt clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Ly2mMnt9TCZbvhcC1JfuV
2026-07-19 22:33:01 +04:00
kami 68c56b6af6 feat(freestyle): return grounding-rejected plan to architect for a bounded re-run
A plan that failed grounding used to dead-end — every gate rejection in
FreestyleDriver.lockAndRun was terminal, so a legitimate grounding catch
(e.g. a stage declaring a PROJECT build with no manifest) left the run stuck
with no retry.

lockAndRun is now a gate loop: on a grounding rejection with retries left it
re-runs the planning workflow from the architect stage (rerunArchitect), which
emits a corrected plan, then re-gates. Other gate failures — and grounding once
maxGroundingRetries is spent — stay terminal.

- FreestyleDriver: gate loop + rerunArchitect/maxGroundingRetries seams;
  groundPlan returns findings (String?) instead of Boolean; post-grounding
  tail extracted to lockAndRunGrounded.
- DefaultSessionOrchestrator.runFrom(startStage) + emitWorkflowStarted(startStage);
  run() delegates to it. Lets the re-run enter directly at architect.
- buildGroundingFeedbackEntry (ContextFeedback) injects the already-recorded
  PlanGroundingEvaluatedEvent findings into the architect's L1 context on re-run;
  wired in SessionOrchestratorExecution.
- Main: rerunArchitect lambda (rehydrate -> runFrom(architect) -> rehydrate).

The architect stage-entry approval gate already reuses a prior APPROVED decision
(alreadyApproved), so the re-run does not re-prompt the operator — added a
FreestyleApprovalGateTest regression guard proving runFrom(architect) with a
seeded approval emits no second request and runs straight through.

Tests: FreestyleDriverTest retry-then-lock + exhaustion->reject(source=grounding);
FreestyleApprovalGateTest reuse-approval guard.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 03:00:35 +04:00
kami 1b58bc325e wip(freestyle/acr): grounding & edit-tool fixes + ACR-compiler experiment
This branch's uncommitted WIP, committed together (entangled at file level).
Distinct pieces of work:

Freestyle QA fixes (this session):
- FileEditTool: pre-validate replace anchor in validateRequest — reject a
  missing/ambiguous target BEFORE the approval gate, mirroring read/write's
  file-not-found / read-before-write pre-checks. Shared not-found/ambiguous
  messages between validate and execute so they can't drift.
- PlanGrounder: add `scanned` flag; when no RepoMapComputedEvent was recorded,
  repoMapPaths is "unknown" not "empty workspace" — skip scope grounding
  (which proves a path ABSENT) so real paths (apps/server/**) aren't falsely
  rejected. Build-manifest check still runs.
- FreestyleDriver: wire scanned=(repoMap!=null); on plan rejection emit a
  session-terminal WorkflowFailedEvent so a rejected run reads FAILED, not the
  COMPLETED-lie (last verdict was the planning-phase WorkflowCompleted).
- ServerModule: resolve project-memory workspace root from the session's bound
  workspace (sessionWorkspaceRoot) instead of boot-static pm.repoRoot(), fixing
  the workspace-binding divergence (correx vs empty scratch dir). Retire tracked
  in Vikunja #266.
- LaunchRegistrationRaceTest: join registered jobs before asserting launchCount
  — computeIfAbsent returns the Job immediately but the fire-and-forget launch
  body lagged awaitAll (the 49-vs-50 flake).

ACR concept-compiler experiment (pre-existing WIP on this branch):
- ExecutionPlanCompiler/Model/PlanLinter, #264 needs-seam (sessionArtifacts),
  LSP diagnostics subsystem (LspDiagnosticEvents/Runner/Lsp4j), BootWorkspace,
  config surface, workflow prompts/schemas, orchestrator advance-don't-rerun.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 01:20:37 +04:00
kami e25e9e46fd fix(kernel): stop compaction self-deadlocking long runs on re-entrant artifact Mutex
JournalCompactionService and TierContextSummarizer wrapped their event emit
in artifactStore.flushBefore { }. But emit -> SqliteEventStore.append already
calls flushBefore internally, re-acquiring CasArtifactStore's non-reentrant
Mutex -> the coroutine parks forever (no CPU, no thread, no exception, no
terminal event). Only fires once the journal crosses the compaction threshold,
i.e. exactly on long runs.

Emit directly; append()'s own flushBefore still fsyncs artifacts before the
referencing event is persisted, so durability ordering is preserved. Adds a
regression test with a lock-holding fake store (the prior fake took no lock,
which is why the deadlock escaped tests).

Vikunja #244.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LG7sGEbVJQncHJtsbPFJZm
2026-07-17 16:17:56 +04:00
kami 633da3d2df feat(kernel): structured package-404 recovery evidence (#192)
The 2026-07-16 audition run-3 burned 26 inferences in install_dependencies
because a registry 404 for a hallucinated package (@types/vite) surfaced only
as raw shell output: the model retried npm, pinned versions/flags, switched to
Yarn, and probed pnpm/network before rewriting the manifest.

packageNotFoundAdvisory deterministically parses npm/yarn/pnpm 404 output,
names the offending manifest entry, and appends a directive telling the model
to edit the manifest — not retry the installer. Wired into renderToolResult
for both the nonzero-exit Success and recoverable Failure framings. Pure over
already-recorded tool output (no new event, invariant #9 unaffected).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 18:56:21 +04:00
kami 59a0ad3c2b feat(kernel): stage-global repeated-failure loop breaker (#78)
Existing read-loop/rejection-loop breakers key on CONSECUTIVE rounds, so a
single interleaved success resets them — the hole that let the 2026-07-16
audition run-3 thrash npm install against a hallucinated package across 57
inferences. Add repeatedToolFailureLoop: cumulative count per normalized
tool-failure signature within a stage; once a signature hits
stageFailureLoopLimit (default 6) the stage fails with STAGE_LOOP_BREAK_GATE
and the step handler routes straight to recovery (never retried in place).

Pure fold over recorded events (replay-safe, invariants #8/#9);
ToolExecutionFailedEvent lacks stageId so failures correlate via
invocationId -> ToolInvocationRequestedEvent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 18:50:11 +04:00
kami a2b97221d5 chore(prompt): trim performance-neutral stage operating-guidance block
Vikunja #169 audition result. The 5x5 controlled A/B
(docs/qa/QA-stage-prompt-audition-results-2026-07-16.md) found the curated
guidance block performance-neutral: 5/5 build success both variants, cost
differences within heavy-tailed noise. Trimmed to the single sentence the
gates don't already enforce (scaffolding-scope boundary); dropped the
read-before-write / verify-before-complete / use-exact-feedback nudges that
restate gate-enforced behavior. ON run 3 proved prose doesn't stop failure
loops — deterministic defenses do (filed Vikunja #191/#192, unblocked #78).
2026-07-16 18:38:16 +04:00
kami 53d8108189 feat(orchestration): plan grounding, positive-pattern mining, stage-prompt audition rig
Vikunja #167/#168/#169.

#167 PlanGrounder (infrastructure/workflow): build-prereq + touches-scope
existence grounding, emits PlanGroundingEvaluatedEvent. Deterministic fold
over recorded workspace/plan events (invariants #8/#9).

#168 positive-pattern mining (SessionOrchestratorPlanPatterns): mine
SuccessfulPlanShape from cross-session log — a locked plan whose own
workflow later completed — and inject the closest-resembling plan shape as
L0/SYSTEM advisory context for the planning stage. Keyword-Jaccard
resemblance, no LLM/embedder, replays identically. Advisory only (#3).

#169 stage-prompt audit + CORREX_STAGE_GUIDANCE ON/OFF toggle in
SessionOrchestratorExecution. RunCommand gains --intent to seed a
freestyle session over REST.

Also: workspace verification events + concept-compiler wiring. ./gradlew
check green.
2026-07-16 14:55:56 +04:00
kami ed7efb6072 Implement closed-loop workspace follow-ups 2026-07-15 23:48:12 +04:00
kami 3a48ecd24f feat(concept): heuristic concept compiler (ACR fold-in)
Promotes recurring validated failure->fix patterns into L3 as retrieval-on-demand
concepts. Deterministic core: ConceptCompilerProjection clusters
RetryAttempted->StageCompleted pairs by fingerprint (gate-agnostic), promotes at
N=3 cross-session validated fixes, never-contradicted. ConceptPromotedEvent is the
sole authoritative write (idempotent under replay); ConceptCompilerService appends
it + best-effort injects to L3 (non-authoritative, inv #6). Wired live in
ServerModule.start() on StageCompleted.

Design: docs/plans/2026-07-12-acr-concept-compiler.md

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DhFXmKe4WisSSPf9LrmmTg
2026-07-15 13:35:17 +04:00
kami d69cb12ce9 refactor: decomposition WIP (orchestrator/server/execution-plan)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DhFXmKe4WisSSPf9LrmmTg
2026-07-15 13:17:01 +04:00
kami 1f794dad63 refactor(kernel): decompose DefaultSessionOrchestrator into per-concern extensions
Moves the 11 private step/recovery helpers off the concrete orchestrator into
internal extension-fun files (Step: step, executeMove, enterStage,
decideGateExhaustion; Recovery: retry/route/ticket helpers). Class keeps the
override seams (run, cancel, submitApprovalDecision), the cross-module public API
(rehydrate, resume, submitClarification, submitSteering), enrich, and
validatedArtifactContent. File-private helpers the moved funs need are promoted
to internal. Pure relocation; clears TooManyFunctions on the class. kernel +
testing:kernel green; server/cli compile clean.
2026-07-13 12:06:20 +04:00
kami 670e0c4828 refactor(kernel): decompose SessionOrchestrator god-class into per-concern extensions
Splits the 3.7k-line SessionOrchestrator into the state-owning abstract class
(fields + open/abstract seams: run, cancel, runInference, mapValidationOutcome,
estimateTokens) plus behavior-preserving internal extension-fun files grouped by
concern: Artifacts, ToolExec, Workspace, Context, RepoContext, Gates, Gates2,
Workflow, Approval, Preview. Two public members consumed cross-module
(liveClarificationRequestIds, requestPlanApproval) stay on the class. Each file
kept <=10 top-level funs; pure relocation, no logic changes.

Clears LargeClass; adds no new TooManyFunctions. kernel test + detekt green;
server/cli compile clean.
2026-07-13 11:58:44 +04:00
kami 7f820bafe6 refactor(kernel): extract executeStage to extension, promote members internal
Pilot for the SessionOrchestrator god-class decomposition. Moves executeStage
(426-line stage-execution driver) out to SessionOrchestratorExecution.kt as an
internal extension fun, hoists three nested holders (RunEffectives,
ArtifactLadderOutcome, RenderedToolResult) to top-level in
SessionOrchestratorTypes.kt, and promotes class/protected members to internal so
extensions can reach them. Behavior-preserving relocation; kernel+test+server
compile green.
2026-07-13 11:44:04 +04:00
kami aee2e67c66 refactor(detekt): extract magic-number constants, wrap long lines, drop legit suppressions
Mechanical detekt cleanup across apps/core/infrastructure (behavior-preserving):
- MagicNumber 62->10: named constants + removed 'legit' MagicNumber suppressions
  (Tier level from ordinal, ReplayInferenceProvider CHARS_PER_TOKEN, ConfigLoader
  DEFAULT_* consts, capability scores, HTTP ranges, column widths, etc.)
- MaxLineLength cut to ~0 in production modules (line wraps, no logic change)
- Dropped one dead parameter (ConfigLoader.parseArray lineNum)

Structural findings (ReturnCount/LongMethod/Complexity/LargeClass) left for a
deliberate decomposition pass. Full build + detekt gate green.
2026-07-12 23:12:28 +04:00
kami 2912799fe0 feat(tools): bound+frame tool results, spill full output to CAS, tool_output retrieval
Tool results injected into model context are now consistently framed and
globally bounded. Success output over the floor (TOOL_RESULT_MAX_CHARS=8000)
is head/tail-truncated with a marker naming a retrieval ref; the full raw
output spills to the artifact store (CAS) and its hash is recorded on the
ToolReceipt (fullOutputHash) in the event log. Agents recover the full text
via the new read-only tool_output(ref=...) tool.

- SessionOrchestrator: frameTruncatedToolResult + renderToolResult; char-cap
  head/tail so a single pathological long line can't defeat the bound.
- ToolReceipt.fullOutputHash (additive, nullable).
- ToolOutputTool (Tier T1, no fs capability) resolves ref -> full bytes.
- Wired via extraTools in Main.kt; tool_output added to ALWAYS_AVAILABLE_READ_TOOLS.
- Failure path keeps ERROR:/FATAL: prefixes (all-rejected breaker dependency).

Tests: FrameTruncatedToolResultTest, ToolOutputToolTest.
2026-07-12 19:46:02 +04:00
kami b2c7bbe401 feat(config): relocate orchestration loop/threshold/budget constants to [orchestration] config
The kernel's ReAct-loop tuning constants (max tool rounds, read/rejection-loop
nudge thresholds, feedback issue cap, repo-map top-k/files-per-dir, docs catalog
cap, clarification-round cap, review-block confidence/retry cap, default
refinement, recovery/intent route budgets) were hardcoded in SessionOrchestrator/
DefaultSessionOrchestrator. Relocated to a new OrchestrationTuning value threaded
through the orchestrator constructor, mapped from CorrexConfig.orchestration in
Main.kt, parsed in ConfigLoader, written by CorrexConfigWriter. Defaults equal the
former constants so an absent [orchestration] section reproduces prior behavior.

Startup-load (not hot-reload); pure output-truncation caps left as constants.

Vikunja #46 (task 76).
2026-07-12 17:45:54 +04:00
kami 3559ea67ef fix(kernel,server): evict per-session caches on workflow termination (#54)
Two unbounded per-session leaks that never released after a workflow ended:

- SessionOrchestrator.artifactContentCache (full file contents, keyed
  "<sessionId>:<path>") grew for the process lifetime. Added
  evictArtifactContentCache(sessionId) — session-prefix key removal,
  rehydrate-safe — called on both completeWorkflow and failWorkflow.
  cancellations was already evicted on both terminal paths.

- NarrationSubscriber leaked a Channel + worker coroutine + lanes map entry
  per session forever. closeLane() now closes the lane channel on
  WorkflowCompleted/WorkflowFailed (draining the terminal narration first);
  the worker self-removes from lanes when its for-loop exits on close.

Deferred: SqliteEventStore.subscriptions eviction — removing the SharedFlow
mid-life would strand LiveArtifactRepository's downstream collector (a
suspended-coroutine leak worse than the tiny empty-SharedFlow entry).

Green: :core:kernel compile+detekt, :apps:server *NarrationSubscriber* (8).
2026-07-12 17:05:34 +04:00
kami 97e56b9275 refactor(kernel): use real tokenizer for journal budgeting estimates (#58)
Real LLM Usage is already recorded (InferenceCompletedEvent.tokensUsed),
aggregated per-session (MetricsProjection), exposed via correx stats + server
MetricsInspectionService, and rendered in tui-go. The only remaining hardcoded
length/4 estimates were the journal context-entry budget and the compaction
threshold — both pre-injection (no LLM response exists for the journal text), so
estimation is legitimate, but they now use the real tokenizer estimateTokens()
for consistency with sibling context entries instead of length/4.
2026-07-12 13:24:50 +04:00
kami 530b118b67 perf(kernel): batch the artifact-lifecycle event triple via appendAll (#59)
The Created->Validating->Validated triple fires unconditionally back-to-back at
three artifact-emission sites — three single-event append transactions (each its
own commit + flow-publish pass) for one logical unit. Added emitAll(sessionId,
payloads) which routes 2+ events through EventStore.appendAll (one transaction,
one publish pass) and falls back to emit() for 0/1. Same events, same order.

Adjacent-but-conditional emits (clarification/approval pause+resume) are left on
emit(): they straddle suspension points, so batching would change observable
ordering.
2026-07-12 13:22:05 +04:00
kami bb73bc9371 perf(kernel): hoist read-only check out of per-tool filter (#51)
runInference filtered the offered tool list with isReadOnlyMode(sessionId)
evaluated inside the per-tool .filter{} — a full log read+fold per offered
tool, per inference round (the audit's dominant O(events^2) hot path). The
flag is tool-independent, so read it once before the filter.

The per-tool-CALL re-check in dispatchToolCalls is left as-is: it is
semantically load-bearing (a read completing earlier in the same batch lifts
read-only mode for a later write), not redundant.
2026-07-12 13:14:58 +04:00
kami 1fd878eb9b fix(approvals): a REJECTED decision must not satisfy the stage approval gate on retry 2026-07-12 13:08:42 +04:00
kami cf856742b1 fix(approval): resolve preview paths against workspace root, not server CWD (#57)
computeToolPreview/readFileIfExists resolved relative file_write/file_edit
paths against the daemon CWD, so the diff shown to the operator for approval
read the wrong file (or nothing) when server CWD != workspace_root. Thread the
bound workspaceRoot (effectives.policy.workspaceRoot) through and resolve
relative paths against it, same as the tools do.
2026-07-12 12:37:04 +04:00
kami 15248cae8a feat(recovery,context): tier-2 intent-holder arbiter + remaining-delta pinning + surfaced signal events
- recovery: two-tier repair ladder — owner then arbiter (recovery stage recast
  as intent-holder, holds initial intent, reconciles cross-file contract disputes)
  with independent INTENT_ROUTE_BUDGET; RecoveryRoutingTest coverage
- context: remaining-delta checklist pinning (RemainingDelta{Pinning,Entry}Test)
- surface write-only events (session name, quality signals) into stats/browse
- parseToolArguments lenient-Json fallback for bare-key drift
- tools: ChildProcess seam
2026-07-11 23:56:52 +04:00
kami 3d5e05c1fb feat(context): reasoning-thread continuity, L3 doc filtering, and gate hardening
Threads reasoning_content across inference calls and journal renders, filters
markdown noise out of L3 repo-knowledge retrieval, adds PlanLinter H3 checks,
and tightens filesystem tool output/dir-listing behavior surfaced by prior
live QA (see project_readloop_campaign memory).
2026-07-10 11:13:43 +04:00
kami f51a8dada4 feat(recovery): route failed write-less stages to recovery instead of futile retry
Adds the failure-ticket + recovery-routing mechanism: a deterministic
gate->capability table gates whether a stage has agency to fix its own
failure, opens a FailureTicketOpenedEvent when it doesn't, and routes to
a metadata role=recovery stage (per-stage budget, cap 2, not reset by
TransitionExecuted) instead of retrying in place. Extends salvage
decisions with a RECOVER option so the review-gate judge can also route
to recovery, unifies deterministic-gate and review-gate routing through
routeToRecovery(), and has ExecutionPlanCompiler synthesize a
write-capable recovery stage + edge for freestyle plans. Read-only tools
(file_read, list_dir) are now always available on any tool-granting
stage so a recovery stage can inspect the write-less stage's failure
without flooding context via shell ls -R.
2026-07-08 10:28:53 +04:00
kami d6bada6f10 feat(clarification): rehydrate pending questions on reconnect + restart
A stage parked on operator clarification lost its modal whenever the
client disconnected or the server restarted — the questions were stranded
in the event log with no way to re-surface them (unlike approvals, which
already rehydrate).

Two composing mechanisms, mirroring the approval path:

- On reconnect: SessionEventBridge.replaySnapshot() now re-pushes the last
  unanswered ClarificationRequestedEvent for a PAUSED/CLARIFICATION_PENDING
  session as a ClarificationRequired message. Gated on the orchestrator
  still holding a live deferred (SessionOrchestrator.liveClarificationRequestIds)
  so a client never sees a modal whose answer could not be delivered.

- On restart: ServerModule.resumeAbandonedSessions() eagerly relaunches
  clarification-parked sessions (approval-parked stay lazy). rehydrate+resume
  re-runs the parked stage, which re-emits a fresh, live
  ClarificationRequestedEvent (round budget MAX=3, 1 prior round recorded, so
  it re-parks rather than silently proceeding), re-arming the deferred that
  the reconnect path gates on.

pauseReason == "CLARIFICATION_PENDING" is the discriminator because the
reducer sets pendingApproval=true for every pause.
2026-07-07 18:30:09 +04:00
kami 82c55ec9c7 feat(context): doc catalog — frontmatter/H1 descriptor as read-on-demand index
Docs were indexed as up to 40 h1-h3 headings per .md (a TOC flood) and
force-fed or excluded wholesale by a binary docs gate. Replace with a
compact, always-on catalog the agent reads on demand.

- RepoMapIndexer: .md now yields ONE descriptor (frontmatter
  description/summary/title -> first # H1 -> first prose line), not a
  heading dump. Replay-safe (still List<String> symbols).
- SessionOrchestrator: always-on '## Docs available (file_read to open)'
  catalog — top-N docs by recency as 'path — descriptor', hard-capped at
  DOCS_CATALOG_MAX. One line each, so it stays present without the
  poisoning the docs gate suppressed. Source layout still excludes doc
  content for non-doc prompts.
2026-07-07 14:52:49 +04:00
kami a8c496e909 fix(toolintent): exempt list_dir from reference-exists + rejection-loop breaker
Greenfield discovery/analyst stages thrashed on plane-2 REFERENCE_EXISTS
rejections: the model listed a not-yet-created dir (e.g. frontend/), got
hard-BLOCKED every round, and burned MAX_TOOL_ROUNDS without progress.

- New ToolCapability.DIRECTORY_LIST; ListDirTool declares it alongside
  FILE_READ. ReferenceExistsRule now exempts directory enumeration (a
  listing of an absent dir truthfully reports empty; only hallucinated
  file *reads* still fail loud). Dispatch stays on capability, not name.
- Stage-agnostic rejection-loop breaker in the tool loop: after 3 rounds
  where every tool call is rejected (BLOCKED/ERROR) with no success,
  force the model to produce its output. Covers JSON-emitting stages the
  read-loop breaker (file_written-only) never protected.
2026-07-07 14:09:03 +04:00
kami efb6eb0334 feat(validation): capability-gap reflection rung (#30 part 2)
Bounded LLM "are you sure?" pass over the capability gaps part 1 detects, run
once at plan-lock before the operator is asked to approve — honest mistakes
self-correct without escalation; only a genuine tool need reaches the human.

- CapabilityGapReflector: fun-interface seam in core:kernel (plain-data in/out,
  no infrastructure:workflow dep), mirroring SemanticReviewer/SalvageJudge.
- CapabilityGapReflectorImpl (apps/server): one InferenceRouter call for the
  whole gap batch, no per-gap loop, no retry; runCatching degrades to a safe
  RESOLVED-advisory default on any failure (unroutable/timeout/malformed JSON) —
  a broken pass can never manufacture a NEEDS_TOOL escalation or a grant.
- CapabilityGapReflectedEvent(verdict RESOLVED|NEEDS_TOOL) recorded per gap and
  registered in eventModule; replay reads the event, never re-invokes inference
  (invariants #7/#8/#9).
- FreestyleDriver.lockAndRun: RESOLVED is advisory (recorded, plan untouched —
  invariant #3); NEEDS_TOOL is appended to the existing requestPlanApproval
  preview so the operator decides — no auto-grant anywhere (#4/#5).
- reflector nullable -> degrades to part-1 behavior when unwired.

Verified: ./gradlew :core:events:test :core:kernel:test :infrastructure:workflow:test :apps:server:test green.
2026-07-07 13:45:00 +04:00
kami 41ed6414c6 feat(guardrails): steering channel + shell-in-file rule + capability-gap detector
Bundles three operator-reliability guardrails (Vikunja #28/#29/#30) plus the
in-flight branch WIP they were built on top of (reasoning_content capture,
operator/project profile editor, write-jail workspaceRoot fix) — the tree is
interdependent (SessionOrchestrator references reasoningArtifactId from the WIP)
and does not compile as separable subsets, so it lands as one commit.

Guardrails:
- #28 mid-stage steering: ClientMessage.SteerSession -> GlobalStreamHandler ->
  orchestrator.submitSteering, reusing SteeringNoteAddedEvent + existing context
  fold (advisory, non-authoritative; invariants #3/#7). Closes the gap where
  steering typed off an approval gate was silently dropped.
- #29 shell-in-file guardrail: ShellInFileContentRule (core:toolintent) blocks a
  file_write whose content is a bare shell command (e.g. "mkdir -p ..."); FileWriteTool
  description now advertises auto-mkdir of parent dirs. Basename-allowlist so the
  extensionless case is caught; scripts/Makefiles/multiline exempt.
- #30 pt1 capability-gap detector: deterministic CapabilityGapDetector maps stage
  intent -> implied ToolCapability, compares to granted tools, emits advisory
  CapabilityGapDetectedEvent in FreestyleDriver.lockAndRun. Recorded, never fails
  the gate and never auto-grants (invariants #3/#4/#5). Reflection rung is pt2.

Verified: ./gradlew check green (whole tree).
2026-07-07 13:27:59 +04:00
kami 879672a47d feat(recovery): failure-ticket routing + review-gate RECOVER + return-to-sender
Retry-agency invariant: a stage may only retry a gate it has the capability
to change. A write-less stage failing a build/contract/static gate (e.g.
freestyle final_verification with allowedTools=[shell]) can never fix it, so
retrying in place is futile until the budget drains (Vikunja #41).

Instead: open a FailureTicketOpenedEvent (category + requiredCapability derived
deterministically from the gate id, no LLM) and route to a recovery stage that
holds the capability, bounded by a small per-stage route budget.

- Slice 2: SalvageDecision.RECOVER (ternary CONTINUE/RECOVER/FAIL) lets the
  review-gate salvage judge hand off to recovery. Shared routeToRecovery()
  unifies the deterministic agency guard and the judge on one destination;
  decideGateExhaustion now returns StepResult?.
- Return-to-sender: recovery's exit is dynamic (recoveryReturnMove) — it goes
  back to the exact ticket-origin stage to re-run its gate, so a write-less
  gate anywhere in the graph is handled without skipping intervening stages.
  The synthesized recovery->terminal edge is now only a no-ticket fallback.
- Freestyle wiring: ExecutionPlanCompiler injectRecovery flag (Main passes
  true) synthesizes a write-capable recovery stage; ticket evidence reaches it
  via buildRecoveryTicketEntry.
- Read-only tools always present: StageConfig.effectiveAllowedTools adds
  file_read/list_dir to any tool-granting stage (fixes the verifier reaching
  for `shell ls -R` to inspect the tree).

Tests: RecoveryRoutingTest (deterministic gate, no static return edge — proves
dynamic return) + GateRetryBudgetExhaustionTest RECOVER case + two compiler
tests. All green; detekt clean.
2026-07-07 12:05:26 +04:00
kami 597a26d551 feat(validation): auto build-gate on terminal freestyle stage (Gate 4 floor)
Closes the COMPLETED-lie where a freestyle stage passes by producing a
schema-valid artifact regardless of whether its build actually succeeded:
a verify stage could run 'npm run build', watch it fail, then self-attest a
build_verified artifact and transition to done. Tool failures never failed
the stage.

The compiler now flags the terminal stage autoBuildGate=true when no stage
declares an explicit build_expectation. Code-ness is decided at runtime, not
compile time (the declared kind is always the generic file_written; only the
written paths reveal code): runExecutionGate folds the FileWritten manifest via
sessionProducedBuildTarget() and promotes to a PROJECT build when a build
manifest (package_json/gradle_module) or an imports_resolve-bearing path was
written, then runs the real command from .correx/project.toml [commands]. A
non-clean exit fails the stage retryably; a docs-only plan is left untouched.

Live-proven (run 3, session 28812b44): gate fired on the real
'npm --prefix frontend run build', exit 2 on a bad tsconfig, RetryAttempted
instead of a false COMPLETED.

Modules green: transitions+workflow 67, kernel 60.
2026-07-07 01:01:12 +04:00
kami 79e2c38a88 feat(retry): per-gate retry budgets + progress-aware charging + hybrid salvage
Replaces the single shared per-stage retryCount (reset on TransitionExecuted,
shared across all post-stage gates) with a per-gate budget. Motivated by run
2e468f9f, where a promising freestyle run was terminally FAILED because the
contract gate burned all 3 shared retries on one trivial miss before the
semantic-review gate ever ran.

- StageExecutionResult.Failure gains a `gate` id; each gate tags its failures.
- OrchestrationState gains gateRetryBudgets / gateFailureFingerprints /
  gateSalvageUsed (all rebuilt from events, reset per-stage on TransitionExecuted).
- FailureFingerprint normalizes+hashes the failure reason; RetryCoordinator.decide
  charges a gate's budget only when the fingerprint is unchanged (no progress),
  so a moving run is never penalised. Returns Retry | Exhausted.
- Hybrid exhaustion: deterministic gates fail; the review gate consults a
  SalvageJudge (LLM, or a deterministic allow-one-reset fallback), recorded as
  RetrySalvageDecidedEvent (invariant #9). CONTINUE resets the gate budget once;
  a second exhaustion is terminal.
- Review gate is now the sole authority on review-retry termination; the old
  REVIEW_BLOCK_RETRY_CAP is repurposed as a high absolute backstop only.
- ServerModule escaped-exception path now records a truthful terminal
  WorkflowFailed (real stage/reason/retryExhausted) via recordUnhandledFailure.

Tests: DefaultRetryCoordinatorTest (fingerprint charging) +
GateRetryBudgetExhaustionTest (deterministic exhaust / review CONTINUE-then-
terminal / review FAIL / null-judge fallback) + reducer coverage.
2026-07-06 19:20:46 +04:00
kami 9f12c87bb1 feat(validation): plan-compile gate + semantic-review gate (staged verification #19/#20)
Plan-compile gate (#19): PlanCompilationCheck seam (null-on-replay),
runPlanCompileGate guards a produced execution_plan slot and compiles its
cached content via ExecutionPlanCompiler, emitting PlanCompileCheckedEvent.
On failure the architect gets a retryable hand-back and self-corrects;
exhaustion terminates the dead-end workflow instead of shipping an
uncompilable plan.

Semantic-review gate (#20): SemanticReviewer seam + StageConfig.semanticReview
opt-in (TOML semantic_review + freestyle plan JSON). runReviewGate reads the
FileWritten manifest (never blind context), asks a review-capable model for
PR-comment findings, and is advisory — blocks retryably only on FAIL plus a
correctness finding >=0.7 confidence, capped at 2 blocks so a stubborn stage
completes advisory-only rather than trapping. SemanticReviewerImpl talks to
the provider directly; any reviewer error degrades to PASS. REVIEW_MAX_TOKENS
is 8192: gemma is a reasoning model and 1024 truncated it mid-reasoning before
it reached the JSON verdict (empty content -> parse degraded to PASS).

Both gates live-QA'd on the real repo: plan-compile passed on a freestyle
web-UI scaffold; semantic review caught all 3 planted bugs in a maxOf() and
exercised the block->retry->cap safety valve end-to-end.
2026-07-06 13:11:16 +04:00
kami ff1a0ffdad feat(validation): staged-verification funnel — contract + execution gates
Gate 2 (contract) and Gate 4 (execution) of the 4-gate staged-verification
design, plus the two gate-quality fixes found in live QA.

- Gate 2: KindContractTable (kind→assertions registry, universal floor,
  additive project overrides) + KindInference (path→kind, pure/replay-safe)
  + ContractAssertion model; ContractGateEvaluatedEvent (registered);
  ContractAssertionEvaluator seam + FileSystemContractEvaluator (FS/TEXT,
  COMPILER skipped); runContractGate in runPostStageGates — a stage that
  produces file_written artifacts must have every declared+written file
  exist/non-empty/parse, else retryable hand-back with per-assertion feedback.
- Option A: ExecutionPlanCompiler puts concrete (non-glob) plan `writes` into
  StageConfig.expectedFiles, so a declared-but-unwritten file fails file_exists.
- Gate 4: BuildExpectation enum (none|module|project|tests) + plan field +
  closed-set compiler validation; runExecutionGate resolves the alias against
  the bound project profile commands and runs it as a deterministic gate.
- Live-QA fixes: react_entry kind so main.tsx/index.tsx aren't required to
  export a default component (was an unwinnable false-positive); JSONC
  tolerance (allowComments/allowTrailingComma) so tsconfig.json passes valid_json.

Seams are null on the replay harness (no-op) — recorded events are truth (#9).
2026-07-06 01:25:51 +04:00
kami 33ea44d8cc feat(context): required/optional bucket split, budget ceilings, grounded handoffs
Context rework from the 2026-07-05 context-poisoning diagnosis (session
6ca236c4 read-loop):

- ContextBucket on ContextEntry: REQUIRED (task, constraints, needed
  artifacts, failure feedback) is never pruned/compressed/dropped; the
  build fails fast (RequiredContextOverflowException -> non-retryable
  stage failure) if required alone exceeds the stage budget.
- OptionalCeilings: per-sourceType token caps for the optional block
  (repoMap 1200, relevantFiles 1600, journal 800, profiles 400/600,
  vocabulary 400), tuned for the 9-26B tier at 16k; enforced by
  truncation (journal keeps its tail, others their head).
- Repo map is now a structural 'what exists' listing (alphabetical,
  per-directory, no recency ranking, no symbols); .md/docs paths are
  gated behind an explicit docs mention in the stage prompt — retrieval
  hits remain the only other path for docs to enter context.
- file_written needs render as a manifest of ALL files the producing
  stage wrote (projected from ArtifactContentStored -> ToolInvocation
  -> FileWritten events), fixing the last-write-wins collapse; content
  stays lazy behind file_read.
- Decision journal folds a stage's RETRY records into one summary line
  once a later transition leaves that stage (render-time only).
- Dedupe retrieved-file lines in buildRelevantFilesEntry.
2026-07-05 18:05:02 +04:00
kami f90f2ab39d feat(validation): artifact-emission repair ladder + empty-schema fix
Deterministic repair pipeline (extraction, classification, policy, gated
LLM-repair rung) with ArtifactRepairAttempted/Failed events. Also fix
JsonSchemaValidator: a schema with no declared properties describes 'any
object' — unknown-property enforcement now only applies against a declared
shape (empty-properties schemas were rejecting every real artifact).
2026-07-05 18:04:36 +04:00
kami a0b3a1865a fix(kernel): deterministic repo-map floor + 0.0-hit filter + closest-path suggestion
RepoMapComputed recorded per session (in-memory scan cache keeps the walk-once
guarantee); zero-score embedder hits filtered before reaching the agent, falling
back to the deterministic repo map; REFERENCE_EXISTS surfaces a closest-path hint
via closestPathByFilename. Breaks the hallucinated-package-path read loop even
with a noop embedder.
2026-07-03 13:15:54 +04:00
kami 2698971082 fix(kernel): de-dup tool events, restore approval diff, break read-before-write deadlock
Three tool-path correctness fixes:

- Tool-execution events were emitted twice — SandboxedToolExecutor and
  SessionOrchestrator.recordToolExecution both emitted Completed/Failed/
  FileWritten per call (double TUI rows, double-counted metrics). Split
  ownership: orchestrator is the sole recorder of Completed/Failed (truncates
  output, drives the read-before-write gate); executor owns Started +
  FileWritten (pre/post-image hashes it alone can capture).

- computeToolPreview still gated on operation == "write", but file_write lost
  its operation param when delete was split out, so it bailed to raw-args for
  every write (approval card showed raw JSON). Drop the dead guard.

- A file_read blocked by REFERENCE_EXISTS can never complete, so it could never
  lift read-only mode: a stage writing a NEW file deadlocked (writes filtered,
  every read of the not-yet-created target blocked) until MAX_TOOL_ROUNDS.
  Lift read-only on a REFERENCE_EXISTS block, and reword the block message to
  tell the model to file_write directly. Regression test added.
2026-07-03 01:01:19 +04:00
kami 1a9e1019ec fix(sessions): terminalize session on workflow completion
DefaultSessionReducer skipped WorkflowCompletedEvent to avoid terminalizing on freestyle planning's mid-session completion, but that left execution/normal completions stuck ACTIVE forever — headless watchers and approval loops never saw a terminal state. Add workflowId to WorkflowCompletedEvent (defaulted for replay compat), pass graph.id at all completeWorkflow sites, and terminalize to COMPLETED unless it's the freestyle_planning handoff. Correct stale tests that pinned the ACTIVE-forever behavior.
2026-07-02 12:23:39 +04:00
kami 18cbd34739 fix(kernel,tools,workflow): session-robustness QA sweep
Uncommitted work from the session-robustness-and-dox branch sweep
(docs/qa/QA-session-robustness-and-dox.md), verified alongside the
compression/context fixes:

- SandboxedToolExecutor: validate tool args centrally before dispatch. A
  malformed/missing-arg call becomes a recoverable ERROR: (surfaced with the
  tool's arg schema so the model can correct + retry) instead of stranding
  the stage with no artifact. + validation test.
- PlanLinter: seed artifacts (analysis) produced by the planning phase count
  as available producers, so a plan stage that `needs` them isn't flagged as
  an unproduced-need; H1 unproduced-needs + trap-state checks. + tests.
- DefaultSessionOrchestrator: live-QA robustness fixes (event-tail /
  per-stage budget + retry handling).
- workflow prompts/configs: DOX AGENTS.md alignment + freestyle/task/role
  prompt tweaks.
- SessionOrchestratorIntegrationTest: coverage for the above.
- FreestylePlanningWorkflowTest: allow list_dir in analyst tools (follows the
  list_dir wiring in 968cbfa).
- QA plan doc for the branch sweep.
2026-07-02 00:56:45 +04:00
kami e0c222392c feat(context,kernel): compression pipeline stage 3 — replay-safe tier summarizer
ContextSummarizedEvent (registered) keyed on stable session sequence, not the
ephemeral pack ordinal, so replay reuses the recorded summary artifact and never
re-calls the LLM (invariants #6/#8). TierContextSummarizer mirrors
JournalCompactionService for freeform context (tool outputs/docs/retrieved text)
the journal path doesn't cover. ContextState + DefaultContextReducer fold the
event; reducer replay-safety unit-tested.
2026-07-01 14:03:47 +04:00
kami 4be8f292ae fix(kernel,server,workflow): per-stage retry budget, sane freestyle budget, tail /events
Three defects surfaced by a live headless freestyle_planning run (all made a
long execution workflow fail or stall in ways invisible to a REST operator):

1. retryCount was session-global (DefaultOrchestrationReducer): TransitionExecuted
   updated currentStageId but never reset retryCount, while shouldRetry gates on
   it per-stage. An early stage that burned its 3 retries starved every later
   stage of its own — the next stage's first *retryable* failure went terminal.
   Reset retryCount on stage entry; back-edge loops stay bounded by the separate
   refinementIterations counter. Regression test added.

2. Freestyle stages ran at StageConfig's 4096-token default (ExecutionPlanCompiler
   never set a budget) vs 16384-32768 for static role_pipeline stages. A tool-heavy
   stage truncated its context every round, evicting the file_read results it just
   gathered, so it re-read forever and never accumulated enough to write. Set 16384.

3. GET /events returned the OLDEST `limit` events (readFrom(0) then take): for a
   session past `limit` events the tail — including a pending ApprovalRequested —
   was invisible to a headless/REST poller, the exact caller POST /approve serves.
   Default now tails; explicit fromSeq still paginates forward.
2026-07-01 13:41:06 +04:00
kami c8c2521fa2 fix(kernel,inference): reliable artifact emission for local models
Two fixes that together let the freestyle analyst/architect stages actually
produce valid artifacts on a local model (verified end-to-end: analyst passed
first try, architect produced a validated plan):

1. Tools-less final emission (SessionOrchestrator): producing the artifact
   while tools are in the request is unreliable — llama.cpp's Gemma template
   switches to a channel format and leaks <|channel> markers into the text, and
   models keep tool-calling until the round cap without ever emitting. After the
   tool loop, fire ONE tools-less inference to get clean JSON (also re-enables
   the JSON grammar, since grammar+tools is rejected). Gated on the artifact not
   already being produced via emit_artifact.

2. GBNF grammar handles arrays/objects (GbnfGrammarConverter): array-typed
   properties fell through to a 'string' rule, so the grammar forced a string
   where the schema demanded an array — an unwinnable grammar-vs-validator
   disagreement. Add generic value/object/array rules and map types correctly.

Note: also needs schema 'items' on array props (runtime schemas updated; mirror
to repo schemas/*.json).
2026-07-01 03:08:24 +04:00
kami e72051a9ea feat(kernel): emit_artifact tool + actionable validation feedback
Two robustness fixes for LLM-emitted artifact stages (e.g. freestyle analyst/
architect) that were exhausting their retry budget on weak models:

1. emit_artifact meta-tool: stages with an LLM-emitted slot now expose an
   emit_artifact tool whose parameters ARE the artifact's JSON schema, giving
   the model a structured tool-call channel to produce the artifact instead of
   a free-form final message. Sidesteps llama.cpp's grammar+tools
   incompatibility (the artifact can't be grammar-constrained while tools are
   present). The call's arguments are captured as the artifact content.

2. Actionable retry feedback: validation rejection now surfaces the concrete
   error messages (e.g. 'required property summary missing') as the stage
   failure reason instead of a bare 'validation failed', so the retry-feedback
   entry tells the model what to fix.
2026-06-30 20:53:40 +04:00
kami fe6e530a75 Merge remote-tracking branch 'origin/feat/session-robustness-and-dox' into feat/session-robustness-and-dox 2026-06-29 20:58:07 +04:00
kami 82674e85c4 feat(inference): capability-aware routing for tool-heavy stages
Add CapabilityAwareRoutingStrategy — it hard-filters to providers that declare every
required capability (like FirstAvailable) but ranks the matches by summed required-capability
score; ties keep list order, so it is a strict superset of first-available. The server now
wires it as the routing policy.

The orchestrator augments a stage's required capabilities with ModelCapability.ToolCalling
when the stage grants tools, so tool-heavy stages route to the best tool-calling model rather
than whichever healthy provider comes first.

Scores come from each provider's declared capabilities(); observed per-model reliability
(GET /metrics/tool-reliability) can feed in later as an additional weight.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 11:06:20 +00:00