GenerationConfig only carried temperature/top_p/max_tokens/stop/seed. Added nullable
topK/minP/repeatPenalty, serialized to the llama.cpp and OpenAI-compat request bodies
via @EncodeDefault(NEVER) so an unset knob is omitted (the model keeps its own default)
and behavior is unchanged unless the operator opts in.
Surfaced as a new [sampling] config section feeding the default stage GenerationConfig
(the main agentic loop) through TomlWorkflowLoader + ExecutionPlanCompiler; the former
hardcoded temperature=0.7/topP=1.0 stage defaults now come from config. Talkie
chat/narration keep their own generation settings.
Vikunja #46 (task 76) — sampling half.
Threads reasoning_content across inference calls and journal renders, filters
markdown noise out of L3 repo-knowledge retrieval, adds PlanLinter H3 checks,
and tightens filesystem tool output/dir-listing behavior surfaced by prior
live QA (see project_readloop_campaign memory).
Bundles three operator-reliability guardrails (Vikunja #28/#29/#30) plus the
in-flight branch WIP they were built on top of (reasoning_content capture,
operator/project profile editor, write-jail workspaceRoot fix) — the tree is
interdependent (SessionOrchestrator references reasoningArtifactId from the WIP)
and does not compile as separable subsets, so it lands as one commit.
Guardrails:
- #28 mid-stage steering: ClientMessage.SteerSession -> GlobalStreamHandler ->
orchestrator.submitSteering, reusing SteeringNoteAddedEvent + existing context
fold (advisory, non-authoritative; invariants #3/#7). Closes the gap where
steering typed off an approval gate was silently dropped.
- #29 shell-in-file guardrail: ShellInFileContentRule (core:toolintent) blocks a
file_write whose content is a bare shell command (e.g. "mkdir -p ..."); FileWriteTool
description now advertises auto-mkdir of parent dirs. Basename-allowlist so the
extensionless case is caught; scripts/Makefiles/multiline exempt.
- #30 pt1 capability-gap detector: deterministic CapabilityGapDetector maps stage
intent -> implied ToolCapability, compares to granted tools, emits advisory
CapabilityGapDetectedEvent in FreestyleDriver.lockAndRun. Recorded, never fails
the gate and never auto-grants (invariants #3/#4/#5). Reflection rung is pt2.
Verified: ./gradlew check green (whole tree).
Adds a nullable `reasoning` field to InferenceResponse and maps the
llama.cpp / OpenAI-compat `reasoning_content` field into it, so local models
that emit their chain-of-thought have it captured. Provider/model side only —
the emit-site event field (reasoningArtifactId) + CAS storage in the
orchestrator are still pending (tracked in Vikunja Correx #4).
- LlamaCppEmbedder: parse llama.cpp --pooling mean nested response [[...]] (was
silently failing every embed with 'unexpected response shape')
- qa-stack: embedder -b/-ub 8192 so docs >512 tokens don't 500; fix default path
- TurboVecL3MemoryStore: send init handshake on startup (was 'Index not initialized'
on every add/query)
- turbovec_sidecar.py: rewrite add/search/save/load against the real turbovec API
(batched ndarray, L2-normalize so IP=cosine, prepare() before search, classmethod
load). The sidecar was scaffolding never run against the real lib.
Two fixes that together let the freestyle analyst/architect stages actually
produce valid artifacts on a local model (verified end-to-end: analyst passed
first try, architect produced a validated plan):
1. Tools-less final emission (SessionOrchestrator): producing the artifact
while tools are in the request is unreliable — llama.cpp's Gemma template
switches to a channel format and leaks <|channel> markers into the text, and
models keep tool-calling until the round cap without ever emitting. After the
tool loop, fire ONE tools-less inference to get clean JSON (also re-enables
the JSON grammar, since grammar+tools is rejected). Gated on the artifact not
already being produced via emit_artifact.
2. GBNF grammar handles arrays/objects (GbnfGrammarConverter): array-typed
properties fell through to a 'string' rule, so the grammar forced a string
where the schema demanded an array — an unwinnable grammar-vs-validator
disagreement. Add generic value/object/array rules and map types correctly.
Note: also needs schema 'items' on array props (runtime schemas updated; mirror
to repo schemas/*.json).
Align the provider's HTTP client timeout with the orchestrator per-stage
timeout (1_800_000ms). A slow local reasoning model with a large prompt and a
multi-thousand-token reasoning budget can exceed 10min on a single call; the
old 600s HTTP timeout failed the inference before the stage timeout applied,
exhausting retries on timeouts alone.
Small local models emit tool calls into message content but truncate them
(unterminated arguments string, missing closing braces), so strict decode
fails and the blob is treated as a failing artifact, exhausting the stage's
retry budget. Add a lenient fallback that recovers the function name and the
first brace-balanced arguments object, gated on a tool-call marker so genuine
artifact JSON is never misread as a call.
Add CapabilityAwareRoutingStrategy — it hard-filters to providers that declare every
required capability (like FirstAvailable) but ranks the matches by summed required-capability
score; ties keep list order, so it is a strict superset of first-available. The server now
wires it as the routing policy.
The orchestrator augments a stage's required capabilities with ModelCapability.ToolCalling
when the stage grants tools, so tool-heavy stages route to the best tool-calling model rather
than whichever healthy provider comes first.
Scores come from each provider's declared capabilities(); observed per-model reliability
(GET /metrics/tool-reliability) can feed in later as an additional weight.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Enable autonomous QA through a remote OpenAI-compatible provider (NVIDIA NIM)
and harden the tool/approval path so unattended multi-stage runs complete.
- inference: add openai_compat provider (Bearer chat-completions for NIM/OpenAI),
dispatched by provider type "nim"/"openai"; key via api_key/api_key_env.
- server: bind configured [server] host/port instead of a hardcoded 8080;
POST /sessions accepts an optional `intent` (WS parity) for intent-driven workflows.
- kernel: thread the bound operator profile's approval_mode into per-tool gating so
auto/yolo enable unattended approval (engine still consulted; policy/plane-2 BLOCK
stays terminal); on a recoverable tool failure feed the tool's arg-schema back into
context so the model self-corrects instead of repeating a malformed call.
- tools: split deletion out of file_write into a separate, explicitly-named file_delete
tool — a model can no longer delete a file by getting a write-mode parameter wrong.
- server: add GET /metrics/tool-reliability — per-model tool-call validity from the
event log (measurement groundwork for capability-aware routing).
- docs: update AGENTS.md across kernel, tools, server, inference.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root Child DOX Index assembled, plus a per-module AGENTS.md across the tree
(core/*, infrastructure/*, apps/*, testing/*, and docs/examples/frontend/etc),
each following the DOX section shape: Purpose, Ownership, Local Contracts,
Work Guidance, Verification, Child DOX Index.
Some local models write the tool call as a JSON blob in `content` instead of the
native tool_calls array (toolCalls arrives empty). The orchestrator then treated
the blob as a failing artifact and retry-looped to exhaustion. salvageToolCalls
parses content as a ToolCallRequest (object or array) when tool_calls is empty;
real artifact JSON lacks a `function` field so it's left untouched.
LlamaCppTokenizer posted {"tokens":[text]} but llama.cpp's /tokenize expects
{"content":text}; the server silently answered {"tokens":[]}, so countTokens
returned 0 for every string. Every context entry carried tokenEstimate=0 and
the whole budget/trim pipeline was blind — live QA saw a 34k-token prompt
sail through a 16384 stage budget (three raw file_read results of plan docs),
crater t/s, degrade output, and fail the stage on validation 3x.
Also guard estimateTokens: a zero count for non-blank content falls back to
the chars/4 heuristic so a lying tokenizer can never blind budgeting again.
- generic SystemResourceProbe: owner-agnostic /proc/meminfo system RAM
overlaid on the vendor GPU probe, wired on both managed and static paths
so the VRAM/GPU/RAM gauge renders with an external llama-server (was dormant)
- F-002: classify provider 4xx (e.g. llama.cpp 400) as terminal, not retryable
(except 408/429); stops retrying deterministic request failures to exhaustion
- F-003: FileSystemPromptLoader expands leading ~ / ~/ to user home
- F-004: map InferenceFailedEvent + RetryAttemptedEvent to new
ServerMessage.InferenceFailed / RetryAttempted, decoded+rendered in tui-go;
ArtifactContentStoredEvent explicitly mapped to null (internal, no warn)
- tests: tilde expansion, SystemResourceProbe (static/managed/fail-soft)
Two fixes surfaced by the 2026-06-08 QA audit running role_pipeline on a local
llama-server.
F-001: LlamaCppInferenceProvider sent both a GBNF grammar and tools on the same
request; llama.cpp rejects the combination ("Cannot use custom grammar
constraints with tools"), failing every stage that has tools and produces a
schema-constrained artifact. Drop the grammar when tools are present and rely on
post-hoc schema validation + retry (invariant #7 still holds). TODO left for a
first-class emit_artifact tool as the proper fix.
F-005: FileReadTool resolved relative paths via toAbsolutePath() against the JVM
process cwd instead of the bound workspace, so the file-read jail anchored to the
wrong root and disagreed with the Plane-2 PATH_CONTAINMENT check. Add workingDir
to FileReadConfig, wire workspace.workingDir in the per-workspace tool builder,
and give FileReadTool a resolvePath() mirroring FileWriteTool/FileEditTool.
The resource gauge was NVIDIA-only — on an AMD/ROCm box Main fell back to
UnavailableProbe, so the TUI showed no VRAM. Add AmdResourceProbe backed
by `rocm-smi --showmeminfo vram --showuse --csv`: it skips the warning
preamble, locates the header by column name (order/version tolerant), and
converts the byte VRAM figures to MiB. Process RSS reuses /proc/<pid>/status
(only populated for a managed-model pid). Main now selects NVIDIA → AMD →
Unavailable. Tests cover the real ROCm 4.0 CSV + reordered columns.
- LlamaCppInferenceProvider: check HTTP status and surface the real error body instead of masking it as a ChatCompletionResponse deserialization failure
- PromptRenderer: render the live conversation layer (L1) last so the user turn is the final message (fixes Qwen 'No user query found' 500 when L3 recall is present)
- TomlWorkflowLoader: resolve stage prompts relative to the workflow file's own dir (bundle), CWD-independent; config-dir fallback for globals
- SessionOrchestrator: hard-fail a stage whose declared prompt can't resolve (was a silent user-less request)
- move healthcheck prompts next to the example workflow (examples/workflows/prompts/)
Go TUI decodes model.changed / model.list / resource.status and renders a
status-bar VRAM/GPU%/RAM gauge plus a models overlay (m key / palette 'models'):
lists configured models with the resident one marked, enter swaps (SwapModel),
c clears the pin (ClearModelPin). All three new server messages are
non-event-bearing control/gauge frames, applied immediately like
provider.status_changed.
Server addition required for the picker: ServerMessage.ModelList (model.list,
NonEventMessage) sent once in the initial snapshot from
ManagedInferenceRouter.availableModelIds()/currentModelId() — the TUI otherwise
cannot enumerate swap targets.
Completes the 5-slice model-lifecycle feature. ./gradlew check green; go build/vet clean.
Plan: docs/plans/2026-05-31-model-lifecycle-management.md (slice 5 of 5).
Vendor-agnostic ResourceProbe (commons): NvidiaResourceProbe reads nvidia-smi
(injectable runner) for whole-device VRAM/util and /proc/<pid>/status for the
managed llama-server RSS, both fail-soft to null; UnavailableProbe for non-GPU
hosts / the static path. DefaultModelManager.currentPid() + LlamaProcess.pid
expose the managed pid. ServerMessage.ResourceStatus (resource.status,
NonEventMessage, all-nullable) pushed every 2.5s on the global stream plus one
in the initial snapshot. Live gauge only — never event-sourced (feeds no core
decision), so invariants #8/#9 hold. Main wires the NVIDIA probe on the managed
path when nvidia-smi is present, else UnavailableProbe.
Tests: csv parsing (single/multi/malformed), gpu+rss combine, fallbacks.
Plan: docs/plans/2026-05-31-model-lifecycle-management.md (slice 4 of 5).
ManagedInferenceRouter owns a live @Volatile pin (decision D1): swap(modelId)
makes a model resident and pins it; clearPin() releases it. Pin is highest
precedence in route() (pin > stage.modelId > capability > default). New
SwapModel/ClearModelPin client messages handled in GlobalStreamHandler via
ServerModule.modelSwapper (null on the static path). ServerMessage.ModelChanged
(model.changed) mapped from ModelLoadedEvent/ModelUnloadedEvent — the swap's load
event surfaces to clients through streamGlobal, so the handler doesn't push it
directly. Tests: pin precedence + swap/clear, Model* -> ModelChanged.
Plan: docs/plans/2026-05-31-model-lifecycle-management.md (slice 3 of 5).
StageConfig.modelId; InferenceRouter gains a backward-compatible model-aware
route() overload (default ignores modelId, so static routers and test fakes are
unaffected). New ManagedInferenceRouter (infra commons) resolves a target model
per stage (stage.modelId > capability match > default) and ensureLoaded()s it
before routing, returning the live managed provider — its id changes with the
resident model, so there is no stale health cache to invalidate on swap.
ModelManager.ensureLoaded default delegates to the idempotent load(). Main wires
the managed router on the [[models]] path; the static path keeps DefaultInferenceRouter.
Plan: docs/plans/2026-05-31-model-lifecycle-management.md (slice 2 of 5).
[[models]] config (ModelConfig/ModelsSettings) parsed by ConfigLoader;
InfrastructureModule.createModelManager + modelConfigToDescriptor; Main.kt
managed boot path spawns the default llama-server when [[models]] is present
(static [[providers]] path preserved when absent) and kills it on shutdown.
Plan: docs/plans/2026-05-31-model-lifecycle-management.md (slice 1 of 5).
Adds [router.embedder] and [router.l3] sections to CorrexConfig with
backend selectors. Ships LlamaCppEmbedder that hits llama.cpp's
/embedding endpoint (handles OpenAI-compatible, simple, and array
response shapes; validates dimension). InfrastructureModule gains
createEmbedderFromConfig and createL3MemoryStoreFromConfig that
dispatch on backend value.
Defaults preserve current behavior (noop embedder + in-memory L3).
Switching to "llamacpp" / "turbovec" is a config-only change — no code
edits required. For turbovec backend, the bundled python sidecar
script is extracted from classpath to ~/.cache/correx/ on first use.
Extract message-rendering logic from LlamaCppInferenceProvider into a
provider-agnostic PromptRenderer in core:inference. SessionOrchestrator
now serializes the rendered ChatMessage list as JSON for the prompt
artifact, so the audit trail reflects what the LLM actually saw.
Resolves the TODO at SessionOrchestrator.kt:691.
- GlobalStreamHandler.handleCreateGrant validates scope (SESSION/STAGE)
and sends ProtocolError to client on validation failure instead of
silent log.warn + drop.
- Derive event stageId directly from GrantScope type, removing
redundant stageIdForEvent tuple.
- SessionOrchestrator integrates ApprovalEngine to check active grants
before requesting user approval for T2+ tool calls.
- ContextEntry gains EntryRole (SYSTEM/USER/ASSISTANT/TOOL) field for
proper chat message role mapping.
- RouterContextBuilder.build is now suspend; uses Tokenizer for
accurate token estimation with fallback to character-based estimate.
- LlamaCppInferenceProvider maps EntryRole to ChatMessage role instead
of heuristic layer-based inference.
Implements the full conversational router facade: RouterState, RouterReducer,
RouterProjector, RouterRepository, RouterContextBuilder, RouterFacade, protocol
types, WebSocket wiring, infrastructure factory, and deterministic test suite.
Also fixes spec divergences found in post-implementation review:
- Add SteeringNote domain object to core:context (epic prerequisite)
- Rename RouterFacade.handleChat → onUserInput per spec interface contract
- Add in-memory ConcurrentHashMap conversation history to DefaultRouterFacade
- Make RouterRepository.getRouterState suspend
- Rename RouterConfig.keepLast → conversationKeepLast, fix defaults (6, 4096)
- Refactor InfrastructureModule.createRouterFacade to self-assemble internally
- Fix FileReadTool: allowedPaths was dead constructor param (@SuppressUnusedParameter);
now stored as private val and enforced in validateRequest
- Disable koverVerify on modules tested via testing/ submodules or with
hardware/integration dependencies (24 modules); build gate now passes clean
Deletes 21 Module.kt scaffolding objects that were never wired into any DI
registry. Removes unused imports across 8 production and 3 test files.
Restores StageExecutor, CyclePolicyResolver, PolicyValidation, and
ReplayContractTest — these are deferred features, not dead code.
- SessionOrchestrator: wire T2/T3/T4 approval gate before tool execution;
emit OrchestrationPausedEvent/ApprovalRequestedEvent, await decision,
return rejection as ERROR context entry so LLM sees the denial;
propagate tool ERROR entries as StageExecutionResult.Failure
- SandboxedToolExecutor: remove dead code and simplify
- InfrastructureModule: minor wiring cleanup
- LlamaCppInferenceProvider / build.gradle: related build fixes
New core:artifacts-store interface + infrastructure/artifacts-cas
implementation: segment files + SQLite index, Blake3 hashing,
group-commit fsync via flushBefore, recovery tail-scan, manual
compactor, and oldest-first disk-cap evictor.
Inference events now carry promptArtifactId / responseArtifactId;
orchestrators put bytes before emitting. SqliteEventStore wraps
its txn in artifactStore.flushBefore so segment data is fsynced
before the event commit, making the crash window non-corrupting
(TailScanner re-indexes orphan tail records on reopen).
Compactor and evictor are mutually exclusive via maintenanceMutex.
Step 9 (cloud sync) deferred to a later epic.
See docs/reviews/2026-05-18-cas-steps-1-8-review.md for the final
review.
kami