LlamaCppTokenizer posted {"tokens":[text]} but llama.cpp's /tokenize expects
{"content":text}; the server silently answered {"tokens":[]}, so countTokens
returned 0 for every string. Every context entry carried tokenEstimate=0 and
the whole budget/trim pipeline was blind — live QA saw a 34k-token prompt
sail through a 16384 stage budget (three raw file_read results of plan docs),
crater t/s, degrade output, and fail the stage on validation 3x.
Also guard estimateTokens: a zero count for non-blank content falls back to
the chars/4 heuristic so a lying tokenizer can never blind budgeting again.
Strict chat templates (Qwen3.5) reject any system message after index 0.
Recalled L3 memory, L2 stage summaries, and retrieval entries carry
EntryRole.SYSTEM and rendered as standalone mid-conversation system turns,
making llama-server 400 on every router turn once L3 recall returned hits.
Adds two test files to prove B4/B3 invariants end-to-end:
- RepoKnowledgeReplayTest (testing/replay): projection-based replay tests that
verify recorded RepoKnowledgeRetrievedEvent drives relevantFiles context assembly
without consulting a retriever (test A), and that a log without the retrieval event
falls back to the RepoMapComputedEvent legacy path (test B).
- RepoMapReuseIntegrationTest (testing/integration): end-to-end test using the real
ProjectMemoryService + L3RepoKnowledgeRetriever + InMemoryL3MemoryStore proving
exactly one index walk across two sessions sharing the same stateKey, and that the
second session receives non-empty repo hits via the shared L3 embeddings despite
having no per-session RepoMapComputedEvent.
New test deps (noted per constraint):
- testing/replay: adds core:context (needed to import ContextLayer/ContextEntry from
the buildRelevantFilesEntry return type).
- testing/integration: adds apps:server (ProjectMemoryService, L3RepoKnowledgeRetriever,
FakeWorkspaceStateProbe), core:router (InMemoryL3MemoryStore), core:config (ProjectConfig).
No new inter-core or upward production dependencies introduced.
- Add existsByTurnIdPrefix(prefix) to L3MemoryStore interface; implemented in
InMemoryL3MemoryStore (mutex-guarded) and TurboVecL3MemoryStore (metadata map).
All test fakes (TrackingL3MemoryStore, CapturingStore) updated.
- Introduce RepoMapIndexerPort interface; RepoMapIndexer implements it. Allows
injection of test doubles without framework overhead.
- ProjectMemoryService.indexAndRecord: reads latest WorkspaceStateObservedEvent
from event store (invariant #9 — recorded fact, not re-probe); skips scan+embed
when L3 already holds entries for "repomap:<root>:<stateKey>"; passes stateKey
to RepoMapComputedEvent; embeds each scanned entry after append; per-entry
failures warn-logged (CancellationException rethrown).
- Tests: InMemoryL3MemoryStoreTest +2 cases; new ProjectMemoryServiceReuseTest
covers same-key skip, changed-key re-index, no-observation always-reindex,
and embedding turnId tag verification.
emitLlmArtifacts previously emitted ArtifactCreated/Validating/Validated for every
llm-emitted slot unconditionally, fabricating "validated" artifacts when the model
produced no content. Gate now mirrors the F-015 file_written check: slots with null
or blank cached content are skipped with a warn log. Regression test added in
NeedsArtifactInjectionTest verifies no artifact events appear for a blank-response stage.
[router.narration] model_id selects which provider handles narration turns
instead of generic capability routing — lets a small/fast model own narration
while the main model keeps CHAT/STEERING.
- NarrationSettings.modelId parsed from TOML, threaded via RouterConfig
.narrationModelId into DefaultRouterFacade.narrate (3-arg route)
- DefaultInferenceRouter: exact-id match against healthy providers, with a
post-select health check; any miss logs WARN and falls back to capability
routing (never fails the narration turn)
- onUserInput unchanged — only narrate uses the pinned model (tested)
Implements the replay-safe half of preemptive redirect (graph re-routing). An
operator-confirmed PreemptRedirectEvent overrides the resolver's edge at the next
transition boundary; the pure resolver stays untouched.
- PreemptRedirectEvent / PreemptRedirectBlockedEvent (+ serialization registration)
- TransitionExecutedEvent + TransitionDecision.Move gain optional redirectId — the
durable 'consumed once' marker
- PreemptRedirect.decide: pure decision over the event log (override / block / none),
so replay reaches the identical outcome without re-classifying (invariant #8)
- override wired in DefaultSessionOrchestrator.step above resolveTransition; back-edge
jumps count against the existing maxRetries cap via executeMove
- blocks jumps to unknown stages or targets with unsatisfied needs
- DomainEventMapper: redirect events mapped to null (operator surface ships with the
LLM-proposal + approval-confirm front-half)
- tests: override+consume-once, block-on-needs, block-on-unknown, ignore-consumed
Front-half (LLM proposal -> approval-gate confirm -> emit PreemptRedirectEvent) is
deferred; needs live LLM verification. Until it ships no redirect event is emitted in
production, so the override is inert. Spec: docs/plans/2026-06-10-freestyle-graph-rerouting.md
- generic SystemResourceProbe: owner-agnostic /proc/meminfo system RAM
overlaid on the vendor GPU probe, wired on both managed and static paths
so the VRAM/GPU/RAM gauge renders with an external llama-server (was dormant)
- F-002: classify provider 4xx (e.g. llama.cpp 400) as terminal, not retryable
(except 408/429); stops retrying deterministic request failures to exhaustion
- F-003: FileSystemPromptLoader expands leading ~ / ~/ to user home
- F-004: map InferenceFailedEvent + RetryAttemptedEvent to new
ServerMessage.InferenceFailed / RetryAttempted, decoded+rendered in tui-go;
ArtifactContentStoredEvent explicitly mapped to null (internal, no warn)
- tests: tilde expansion, SystemResourceProbe (static/managed/fail-soft)
rehydrate() now scans ArtifactContentStoredEvent (the durable slot->CAS-hash
bridge) and fetches stored bytes by content hash, keying the cache by the
logical slot name. The slot name is never passed to the hash-keyed CAS, so the
odd-length-hex crash is gone and artifact content is restored on cold-start
resume. RehydrateTest reworked to use a content-addressed fake store (slot name
!= content hash) so it actually exercises the bug. Replay (#8) is unaffected —
rehydrate is live-resume-only.
Also bump router narration maxTokens 1024 -> 4096: reasoning models were
spending the whole budget thinking and emitting empty content (finishReason=
length), so narrations never reached the TUI.
Removes the stale TUI-refactor progress doc.
Extract a SubagentRunner fun-interface (SubagentRunRequest/SubagentRunResult)
and an InSessionSubagentRunner default that delegates to the existing
executeStage path, with per-run effectives captured in the lambda closure so
the seam stays free of the internal RunEffectives type.
DefaultSessionOrchestrator.enterStage now dispatches through subagentRunner.run
instead of calling executeStage inline; effectives threading is dropped from
run/step/executeMove/enterStage/resume. ReplayOrchestrator supplies its own
runner. Behaviour is identical — RefinementLoopTest and full check stay green.
Stages declaring needs=[...] now receive each needed artifact's validated
JSON (from artifactContentCache) as an L1/USER neededArtifact context entry,
so a one-shot subagent sees its inputs.
Also emit llm-emitted artifact lifecycle events (Created/Validating/Validated)
on successful validation via emitLlmArtifacts, closing a latent gap where
verifyProduces would fail for stages producing only an llm-emitted artifact.
Events are emitted only after validation passes, preserving invariant #7.
Replace the placeholder "You are a routing assistant" one-liner with two
purpose-written prompts: a conversational prompt that frames the router as
correx's operator-facing interface to the event-sourced engine (grounded in
workflow state, no fabrication, concise, steering-aware), and a dedicated
narrator prompt for buildNarrationContext that asks for one or two present-tense
sentences naming the stage and outcome.
The longer protected frame shifted token budgets, so the two L2 eviction tests
now size their budget from the measured protected-frame + per-entry cost instead
of hard-coded magic numbers, making them robust to prompt length.
Two issues surfaced during live QA once workflows began progressing past the
first stage:
1. Narration prompts went out with no user message, so the chat template
rejected them ("No user query found in messages"). buildNarrationContext
filed the trigger instruction (a user turn) under ContextLayer.L0, and
PromptRenderer folds every L0 entry into the system message regardless of
role. Move the trigger to L1 so it renders as the user turn.
2. TransitionExecutedEvent was emitted after the next stage had already run:
executeMove called enterStage (which executes the stage) before advanceStage
(which emits the transition), so the event log showed a stage running before
the event marking entry into it. Emit the transition before entering.
Adds a regression test asserting a rendered narration prompt carries a user
message.
Tool-calling loops were rendered as all-assistant-calls-then-all-tool-results
with the original task last, causing the model to re-issue the same tool call
indefinitely. Two stacked reorderings caused it:
- DefaultContextPackBuilder grouped entries by sourceType for per-type
compression, discarding a,t,a,t interleaving.
- PromptRenderer forced L1 (the live user turn) to render last, pushing the
task after the entire transcript.
- SessionOrchestrator's tool loop re-fed currentContext.layers.values.flatten()
(grouped by layer) each round, compounding the scramble.
Add a chronological `ordinal` to ContextEntry, stamped by the builder from
input order and restored after grouping/compression (the compressor preserves
entry identity, so ordinals survive). PromptRenderer now orders non-system
messages by ordinal, with the old L1-last layer priority kept only as a
tiebreak so router chat (ordinal 0) is unchanged. The orchestrator keeps a
running ordered accumulator instead of reading back the grouped pack.
Adds builder + renderer ordering regression tests.
- Add NarrationTrigger(kind, instruction) model to core:router
- Add RouterContextBuilder.buildNarrationContext: minimal ContextPack
(system + workflow status + L2 + trigger instruction); excludes
conversationHistory and L3 recalled memory
- Add RouterFacade.narrate: emits RouterNarrationEvent with content,
latencyMs, tokensUsed; skips event on blank inference; never writes
ChatTurnEvent or l3MemoryStore.store
- Add RouterNarrationTest covering all acceptance criteria
- Add narrate default to RouterContextBuilder interface so existing
anonymous test stubs compile without a stub override
ChatTurnEvent gains nullable latencyMs and tokensUsed fields (defaults preserve
backward-compat; legacy JSON without them deserializes cleanly). emitChatTurn
accepts optional metrics; the ROUTER emit site passes inferenceResponse values
through; the USER emit site leaves both null.
Wires the workspace handshake end to end so a session's workspace is bound
from the client's cwd at connect time and is event-sourced for replay.
- Go TUI sends Hello{workingDir=os.Getwd()} as the first WS frame on connect
(protocol.go encoder + client.go connect path; golden test pins the wire
format against the Kotlin discriminator).
- Server adds ClientMessage.Hello, stashes the per-connection workingDir, and
on session start resolves it through WorkspaceResolver's trust pipeline,
emits SessionWorkspaceBoundEvent (invariant #9), and threads the resolved
workspace into OrchestrationConfig for the live run. A Hello after the first
StartSession is ignored (warn); a rejected path binds the resolver fallback.
- Replay derives the workspace from the recorded event: SessionState gains
boundWorkspace, DefaultSessionReducer fills it from SessionWorkspaceBoundEvent,
and ReplayOrchestrator uses it (Path.of only, no filesystem re-query —
invariant #8) with graceful fallback to config for pre-Phase-B logs.
Absent Hello / null resolver degrades to the prior config-workspace behavior.
A stage's allowedTools only shaped which tool definitions were offered to
the model; the dispatch path resolved returned tool calls straight from the
full registry and executed them without checking stage membership. A
hallucinated, jailbroken, or edited-transcript tool call for any registered
tool would run unchecked (violating invariant #5 / policy-is-absolute).
dispatchToolCalls now rejects any tool call whose name is not in the stage's
allowedTools (STAGE_COMPLETE_TOOL exempt), emitting ToolExecutionRejectedEvent
and feeding an error ContextEntry back to the model instead of executing.
Empty allowedTools means deny-all domain tools, consistent with offering
only STAGE_COMPLETE_TOOL at inference time.
The resource gauge was NVIDIA-only — on an AMD/ROCm box Main fell back to
UnavailableProbe, so the TUI showed no VRAM. Add AmdResourceProbe backed
by `rocm-smi --showmeminfo vram --showuse --csv`: it skips the warning
preamble, locates the header by column name (order/version tolerant), and
converts the byte VRAM figures to MiB. Process RSS reuses /proc/<pid>/status
(only populated for a managed-model pid). Main now selects NVIDIA → AMD →
Unavailable. Tests cover the real ROCm 4.0 CSV + reordered columns.
A steering note attached to a tool approval was recorded but only
consumed by the NEXT stage's prompt build (buildSteeringNoteEntries runs
once, before the tool loop), so on a single-stage task it had no visible
effect. Two changes:
- Approve+note: capture userDecision.userSteering at the gate and inject
it as a context entry right after the tool result, so the same stage's
loop re-infers with the note and the model acts on it.
- Reject: buildSteeringNoteEntries now also emits anti-repeat feedback for
REJECTED decisions with no note, so the model does not re-propose the
identical call on the retryable re-run (a bare reject previously fed
back only "approval denied").
Integration test drives an approval with a steering note and asserts the
next inference's context carries it.
Compute the effective tool registry/executor and plane-2 WorkspacePolicy per run
from config.workspace (effectivesFor), threaded through the orchestrators; a null
workspace falls back to the boot instances byte-for-byte. Wire the concrete
WorkspaceToolRegistryProvider in Main (buildToolConfigForWorkspace). Session undo
unions the session's recorded workspace into its jail roots. (Axis 2 Phase A, tasks 3 + 7.)
A stage declaring no `produces` passed verifyProduces vacuously, letting an
agent stage_complete with zero work. Gate on allowedTools: a produce-less stage
with tools available must have >=1 ToolInvocationRequestedEvent scoped to its
stageId, else Failure(retryable). Produce-less + no tools is exempt (only
stage_complete is ever offered), so pure-reasoning stages aren't bricked.
Fix broken stage-summary interpolation ($payload.stageId rendered the event's toString()). RouterL2Entry gains reason + steeringNotes; the reducer folds SteeringNoteAddedEvent into pendingSteeringNotes and harvests them into the completing stage's L2 entry, then clears (adr-0003 §4, pure field extraction, zero inference). RouterContextBuilder renders summary + reason + steering.
DefaultContextPackBuilder already enforced the token budget, but SessionOrchestrator discarded the entriesDropped signal at both build sites — overflow was enforced yet unobserved. Emit ContextTruncatedEvent when entriesDropped>0, matching the router path (open-threads 6.2 contract).
Vendor-agnostic ResourceProbe (commons): NvidiaResourceProbe reads nvidia-smi
(injectable runner) for whole-device VRAM/util and /proc/<pid>/status for the
managed llama-server RSS, both fail-soft to null; UnavailableProbe for non-GPU
hosts / the static path. DefaultModelManager.currentPid() + LlamaProcess.pid
expose the managed pid. ServerMessage.ResourceStatus (resource.status,
NonEventMessage, all-nullable) pushed every 2.5s on the global stream plus one
in the initial snapshot. Live gauge only — never event-sourced (feeds no core
decision), so invariants #8/#9 hold. Main wires the NVIDIA probe on the managed
path when nvidia-smi is present, else UnavailableProbe.
Tests: csv parsing (single/multi/malformed), gpu+rss combine, fallbacks.
Plan: docs/plans/2026-05-31-model-lifecycle-management.md (slice 4 of 5).
ManagedInferenceRouter owns a live @Volatile pin (decision D1): swap(modelId)
makes a model resident and pins it; clearPin() releases it. Pin is highest
precedence in route() (pin > stage.modelId > capability > default). New
SwapModel/ClearModelPin client messages handled in GlobalStreamHandler via
ServerModule.modelSwapper (null on the static path). ServerMessage.ModelChanged
(model.changed) mapped from ModelLoadedEvent/ModelUnloadedEvent — the swap's load
event surfaces to clients through streamGlobal, so the handler doesn't push it
directly. Tests: pin precedence + swap/clear, Model* -> ModelChanged.
Plan: docs/plans/2026-05-31-model-lifecycle-management.md (slice 3 of 5).
StageConfig.modelId; InferenceRouter gains a backward-compatible model-aware
route() overload (default ignores modelId, so static routers and test fakes are
unaffected). New ManagedInferenceRouter (infra commons) resolves a target model
per stage (stage.modelId > capability match > default) and ensureLoaded()s it
before routing, returning the live managed provider — its id changes with the
resident model, so there is no stale health cache to invalidate on swap.
ModelManager.ensureLoaded default delegates to the idempotent load(). Main wires
the managed router on the [[models]] path; the static path keeps DefaultInferenceRouter.
Plan: docs/plans/2026-05-31-model-lifecycle-management.md (slice 2 of 5).
TurboVec persists vectors via the sidecar but kept entry metadata
(sessionId/turnId/text) only in an in-memory map, so restarting with
backend=turbovec silently dropped every query hit (the id→metadata
lookup missed). Rebuild that map from the ChatTurnEvent log at startup.
- RehydratableL3MemoryStore: capability interface for stores whose
vectors persist independently of the JVM. TurboVec implements it;
InMemory deliberately does not (its vectors are volatile too).
- L3MetadataRehydrator replays ChatTurnEvents into the metadata map.
No embedder: the query path never reads entry.vector, so vectors
stay in the sidecar and metadata comes from the event log (the
source of truth, invariant #1). Short-circuits for non-rehydratable
backends before scanning the log.
- Wired into Main before the server accepts queries; no sidecar spawn.
Backfill of never-embedded history (needs the embedder) is a separate
follow-up. The in_memory non-durability warning already exists.
Tools can declare how their output is compressed before it enters the
context pack, replacing uniform handling. Open-threads 6.3 / ADR-0003.
- core:tools/compression: ToolOutputCompressor interface,
IdentityCompressor default, declarative OutputCompressionSpec +
CompressionRule (StripBlankLines, StripLeadingWhitespace,
DropMatching, HeadTail), and the pure DeclarativeCompressor engine
(regexes compiled at construction; bypassOnError passes raw on
non-zero exit; never throws on input).
- Tool gains `outputCompressor` (default IdentityCompressor). FileReadTool
strips blanks + leading whitespace; ShellTool strips blanks + head/tail.
- SessionOrchestrator applies the resolved tool's compressor on the
ToolResult -> ContextEntry path only. Raw output stays authoritative
in the event log (invariant #6); compression is a pure function of
(raw, exitCode, spec), so no new event and replay stays deterministic.
- Spec is @Serializable/config-shaped (JSON round-trip tested). No
custom-tool-from-config path exists today, so first-party tools wire
in code; TOML-declared compression rides along when that path lands.
Tier B (format-aware parsers: failures-only test output, git-log oneline)
is a documented per-tool extension behind the interface, not built here.
Implements the full vertical slice for invariant #9 tool-call assessment:
- core:toolintent — new module with ToolCallRule seam, ToolCallAssessor,
WorldProbe/FileSystemWorldProbe, WorkspacePolicy, PathContainmentRule,
and RiskMapping (assessment → RiskSummary / AssessedIssue)
- core:tools — ToolCallAssessmentRecord + ToolInvocationRecord.assessment
field; DefaultToolReducer handles ToolCallAssessedEvent (replay proof)
- core:config — ToolsConfig gains workspaceRoot + privilegedLocations;
ConfigLoader parses both; DEFAULT_PRIVILEGED_LOCATIONS built-in
- core:kernel — OrchestratorEngines gains toolCallAssessor/workspacePolicy/
worldProbe fields; SessionOrchestrator.dispatchToolCalls runs
runPlane2Assessment before the tier gate: BLOCK → hard-reject without
executing; PROMPT_USER → elevates tier into approval path with plane2Risk
in the ApprovalRequestedEvent
- apps/server — constructs PathContainmentRule + ToolCallAssessor +
WorkspacePolicy from config and wires them into OrchestratorEngines
Assessment is recorded as ToolCallAssessedEvent (environment observed once,
facts stored, replay reads events — invariant #9). Assessor and WorldProbe
are only invoked on the live orchestrator path, never in replay.
kami