Files
correx/docs/audits/2026-07-26-context-role-audit.md
T
kami 514aeae75f docs(audit): context entry role/layer placement sweep (#312)
Audits every ContextEntry producer against PromptRenderer's three real
render destinations (system fold / inline / trailing user slot). Finds
four escalation-grade entries roled SYSTEM and therefore buried in the
leading system message: recoveryTicket, remainingDelta, groundingFeedback,
rejectionFeedback. remainingDelta additionally mutates the cached system
prefix every turn.

Report only, no code changes. Re-role work filed as #313.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-26 21:00:36 +04:00

6.6 KiB

Audit — ContextEntry role/layer placement (Vikunja #312)

Date: 2026-07-26 · Scope: every ContextEntry producer on the orchestrator stage path. Output: table + recommendations. No code changes.

How placement actually resolves

PromptRenderer.render (core/inference/.../PromptRenderer.kt):

  1. Any entry with layer == L0 or role == SYSTEM → folded into the single leading system message, ordered by (layer.ordinal, entry.ordinal). Role is irrelevant at L0.
  2. Everything else renders inline as its own message, ordered by entry.ordinal (layer priority is a tiebreak only when all ordinals are 0 — router chat).
  3. sourceType == "steeringNote" and SYSTEM-folded → additionally re-emitted as a trailing user "Reminder" turn.
  4. sourceType ∈ repairMandateSourceTypes (currently {"retryFeedback"}) → pulled out of the inline flow and emitted as the final message, role user.

So there are three destinations, not four: leading system fold, inline transcript, trailing user slot. EntryRole.SYSTEM is not "high salience" — it is "buried at the top".

Inventory

Entry (sourceType) Producer Current layer/role → renders as Recommended Rationale
retryFeedback ContextFeedback.kt:34 L1/USER → trailing keep Reference implementation (#293).
recoveryTicket ContextFeedback.kt:118 L1/SYSTEM → system fold → USER + trailing ⚠️ Highest-value defect. The recovery stage exists only because of this ticket, and its mandate ends up above the whole transcript. Same shape as retryFeedback; got the opposite treatment.
remainingDelta ContextFeedback.kt:186 L1/SYSTEM → system fold → USER + trailing ⚠️ Loop-state, recomputed every turn a write lands. It is the stage's completion signal and it is buried, and mutating it invalidates the cached system prefix each turn — the one cache anti-pattern #312 asked to flag. Trailing is both more salient and cache-safe.
groundingFeedback ContextFeedback.kt:89 L1/SYSTEM → system fold → USER + trailing Gate verdict on a returned plan; "emit a corrected plan" is a repair mandate by any reading.
rejectionFeedback SessionOrchestratorContext.kt:117 L2/SYSTEM → system fold → USER + trailing Literally operator voice ("the operator declined"). Escalation-grade; the user channel is where it belongs.
steeringNote (locked) Context.kt:80 L0/SYSTEM → system fold + trailing anchor keep Already double-anchored.
steeringNote (unlocked) Context.kt:94, ToolExec.kt:528 L2/USER → inline keep, note inconsistency The renderer's anchor only catches the SYSTEM variant, so the two steering paths get different salience. Cosmetic, not load-bearing.
artifactRepair Artifacts.kt:132 L2/USER → inline, but sole entry in its pack keep Isolated tools-less pack; already the last (only) message.
criticFeedback / neededArtifact Context.kt:444 L1/USER → inline keep Stage input, not a mid-loop correction. Trailing slot should not carry inputs.
unconfirmedFix Concepts.kt:107 L1/USER → inline keep Advisory, not a mandate.
toolResult (incl. failures) Execution.kt:351/564, ToolExec.kt:215… L2/TOOL → inline tool turns keep Routine self-correctable — "target not found", READ_BEFORE_WRITE, patch misses. Lifting these floods the user channel and destroys the effect. Escalation happens on repetition, and that path already exists (STAGE_LOOP_BREAK_GATErecoveryTicket); #309's loop-breaker is the right place, not the per-failure site.
assistantToolCall ToolExec.kt:205… L2/ASSISTANT → inline keep Correct.
initialIntent Context.kt:162 L1/USER, but L0 ⇒ system fold… no: L1/USER → inline keep Doc comment at Context.kt:148 claims "pinned L0 SYSTEM"; code says L1/USER. Comment is stale — fix the comment, not the code (L1/USER is right).
clarificationAnswer Context.kt:205 L0/SYSTEM → system fold keep, flag cache Mutable mid-run (grows per clarification) inside the cached prefix. Low frequency; acceptable.
schemaInstruction, systemPrompt, operatingGuidance, projectProfile, agentInstructions, operatorProfile, claimedTask, promotedConcept, verifiedBaseline, successfulPlanShape various L0/SYSTEM → system fold keep Stable per stage. Correct and cache-friendly.
agentPrompt Execution.kt:120/147 L1/USER → inline keep The stage task.
repoMap, docsCatalog, relevantFiles, decisionJournal Context.kt:289/376, ContextFeedback.kt:245, Execution.kt:189 L3/USER → inline keep Reference material; #290 already moved these out of the system fold.

Recommendations, in order

  1. Re-role the four escalation entries (recoveryTicket, remainingDelta, groundingFeedback, rejectionFeedback) to EntryRole.USER and add their sourceTypes to PromptRenderer.repairMandateSourceTypes. One-line change each plus the set.

  2. Guard the scarcity invariant first. repairMandateSourceTypes currently joins all matches with \n\n. With one member that is fine; with five, a recovery stage on a retry with an unmet delta emits three stacked "mandates" and the channel stops being authoritative. Before (1) lands, the trailing slot needs either a priority order emitting the single highest-precedence entry, or one consolidated block under a single header. Suggested precedence: recoveryTicket > retryFeedback > groundingFeedback > rejectionFeedback > remainingDelta. remainingDelta is the exception worth appending unconditionally — it is the completion signal, not a competing mandate.

  3. Cache note. All four moves are out of the system prefix and are therefore cache-positive. remainingDelta is the biggest win: per-turn mutation currently sits inside the cached prefix. No move into system is recommended anywhere.

  4. Do not touch tool-role failures. Escalation belongs at the repeat detector (#309), not at each failure site.

Everything above stays event-derived (invariant #9); the trailing block is synthetic and should keep its ## -headed, non-conversational framing so it never reads as a real operator turn.

Note on sequencing

The sprint deferred #312 behind #307 (manifest event) "so the audit has ground truth". Not needed — placement is statically determined by PromptRenderer + the producer's role/layer, both read directly. #307 remains useful for verifying the result of recommendation (1) on a live run.