e46777e29f
Two reviewer-reliability tracks, both deterministic and unit-verified (no model/network). §B-§5 — static_check stage seam. The StaticAnalysisRunner + StaticFindingsRecordedEvent + reviewer-context filter already existed; what was missing was a stage that runs the tool and emits the event. Added: - StaticCheckStageExecutor: reads stage metadata (static_tool/static_argv), runs the configured command via StaticAnalysisRunner, returns a StaticFindingsRecordedEvent. No-op (empty findings) when no runner is wired or no command is configured — safe to carry unconfigured. - A deterministic-stage seam in DefaultSessionOrchestrator.enterStage: any stage with metadata["stage_type"] == "static_check" is run by the executor instead of the LLM subagent, then advances on its (unconditional) exit edge. - TomlWorkflowLoader: stage_type/static_tool/static_argv fields → StageConfig.metadata. - role_pipeline.toml: a static_check stage between implementer and reviewer (no-op until static_argv + a CommandRunner are set; activation is live-QA-gated, see StaticAnalysisRunner doc). §B-§6 — critique-outcome producer. The CritiqueFinding type + CriticCalibrationProjection existed but nothing fed them. Added: - CritiqueFindingsRecordedEvent (+ CritiqueVerdict): the producing side — a critic's findings + verdict for one review iteration, carrying modelHash for per-model calibration. - CritiqueOutcomeCorrelator: pure loop-resolution logic deciding UPHELD (fixed between rounds) / DISMISSED (persisted into an approved final) / INCONCLUSIVE (open at a non-approved terminal), per critic (role + modelHash) and per finding id. - A hook in completeWorkflow/failWorkflow that correlates recorded findings into CritiqueOutcomeCorrelatedEvents at loop resolution — no-op when none recorded, idempotent. (LLM-side finding emission stays a separate model-gated activation.) Tests: StaticCheckStageExecutorTest (4), StaticCheckStageTest integration (1, fake runner → event + transition), CritiqueOutcomeCorrelatorTest (8), CritiqueCalibrationWiringTest integration (1, seeded findings → outcomes at completion), updated RolePipelineWorkflowTest. Full suites for core:events/kernel/critique, infrastructure:workflow, testing:integration green; detekt clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
162 lines
8.0 KiB
Kotlin
162 lines
8.0 KiB
Kotlin
import com.correx.core.approvals.ApprovalProjector
|
|
import com.correx.core.approvals.DefaultApprovalReducer
|
|
import com.correx.core.approvals.DefaultApprovalRepository
|
|
import com.correx.core.approvals.domain.DefaultApprovalEngine
|
|
import com.correx.core.artifacts.DefaultArtifactReducer
|
|
import com.correx.core.events.events.CritiqueFinding
|
|
import com.correx.core.events.events.CritiqueFindingsRecordedEvent
|
|
import com.correx.core.events.events.CritiqueOutcome
|
|
import com.correx.core.events.events.CritiqueOutcomeCorrelatedEvent
|
|
import com.correx.core.events.events.CritiqueRole
|
|
import com.correx.core.events.events.CritiqueSeverity
|
|
import com.correx.core.events.events.CritiqueVerdict
|
|
import com.correx.core.events.events.EventMetadata
|
|
import com.correx.core.events.events.NewEvent
|
|
import com.correx.core.events.types.EventId
|
|
import com.correx.core.events.execution.RetryPolicy
|
|
import com.correx.core.events.types.SessionId
|
|
import com.correx.core.events.types.StageId
|
|
import com.correx.core.events.types.TransitionId
|
|
import com.correx.core.inference.InferenceRepository
|
|
import com.correx.core.inference.InferenceState
|
|
import com.correx.core.journal.DecisionJournalProjector
|
|
import com.correx.core.journal.DefaultDecisionJournalReducer
|
|
import com.correx.core.journal.DefaultDecisionJournalRepository
|
|
import com.correx.core.kernel.orchestration.DefaultOrchestrationReducer
|
|
import com.correx.core.kernel.orchestration.DefaultSessionOrchestrator
|
|
import com.correx.core.kernel.orchestration.OrchestrationConfig
|
|
import com.correx.core.kernel.orchestration.OrchestrationProjector
|
|
import com.correx.core.kernel.orchestration.OrchestrationRepository
|
|
import com.correx.core.kernel.orchestration.OrchestratorEngines
|
|
import com.correx.core.kernel.orchestration.OrchestratorRepositories
|
|
import com.correx.core.kernel.retry.DefaultRetryCoordinator
|
|
import com.correx.core.risk.DefaultRiskAssessor
|
|
import com.correx.core.sessions.DefaultSessionRepository
|
|
import com.correx.core.sessions.projections.replay.DefaultEventReplayer
|
|
import com.correx.core.sessions.projections.replay.EventReplayer
|
|
import com.correx.core.transitions.graph.StageConfig
|
|
import com.correx.core.transitions.graph.TransitionEdge
|
|
import com.correx.core.transitions.graph.WorkflowGraph
|
|
import com.correx.core.validation.pipeline.ValidationPipeline
|
|
import com.correx.infrastructure.persistence.InMemoryEventStore
|
|
import com.correx.infrastructure.persistence.artifact.LiveArtifactRepository
|
|
import com.correx.testing.contracts.fixtures.artifactstore.NoopArtifactStore
|
|
import com.correx.testing.fixtures.InferenceFixtures
|
|
import com.correx.testing.fixtures.context.ContextFixtures
|
|
import com.correx.testing.fixtures.cyclePolicyMissingValidator
|
|
import com.correx.testing.fixtures.transitions.TransitionFixtures
|
|
import com.correx.testing.kernel.MockSessionEventReplayer
|
|
import kotlinx.coroutines.runBlocking
|
|
import kotlinx.datetime.Clock
|
|
import org.junit.jupiter.api.Assertions.assertEquals
|
|
import org.junit.jupiter.api.Assertions.assertTrue
|
|
import org.junit.jupiter.api.Test
|
|
|
|
/**
|
|
* End-to-end proof that the reviewer-loop runtime correlates recorded critique findings into
|
|
* outcome events at loop resolution (BACKLOG §B-§6), and that those events feed the
|
|
* `:core:critique` calibration projection. The LLM-side emission of findings is mocked here by
|
|
* seeding [CritiqueFindingsRecordedEvent]s directly (that half is a separate model-gated step).
|
|
*/
|
|
class CritiqueCalibrationWiringTest {
|
|
|
|
private val eventStore = InMemoryEventStore()
|
|
private val sessionReplayer = MockSessionEventReplayer()
|
|
private val sessionRepository = DefaultSessionRepository(sessionReplayer)
|
|
private val orchestrationRepository = OrchestrationRepository(
|
|
DefaultEventReplayer(eventStore, OrchestrationProjector(DefaultOrchestrationReducer())),
|
|
)
|
|
private val inferenceRepository = InferenceRepository(
|
|
object : EventReplayer<InferenceState> {
|
|
override fun rebuild(sessionId: SessionId) = InferenceState()
|
|
},
|
|
)
|
|
private val approvalRepository = DefaultApprovalRepository(
|
|
DefaultEventReplayer(eventStore, ApprovalProjector(DefaultApprovalReducer())),
|
|
)
|
|
private val decisionJournalRepository = DefaultDecisionJournalRepository(
|
|
DefaultEventReplayer(eventStore, DecisionJournalProjector(DefaultDecisionJournalReducer())),
|
|
)
|
|
private val retryCoordinator = DefaultRetryCoordinator(eventStore)
|
|
|
|
private val repositories = OrchestratorRepositories(
|
|
eventStore = eventStore,
|
|
inferenceRepository = inferenceRepository,
|
|
orchestrationRepository = orchestrationRepository,
|
|
sessionRepository = sessionRepository,
|
|
artifactRepository = LiveArtifactRepository(eventStore, DefaultArtifactReducer()),
|
|
approvalRepository = approvalRepository,
|
|
)
|
|
|
|
private val orchestrator = DefaultSessionOrchestrator(
|
|
repositories = repositories,
|
|
engines = OrchestratorEngines(
|
|
transitionResolver = TransitionFixtures.simpleResolver(),
|
|
contextPackBuilder = ContextFixtures.simpleBuilder(),
|
|
inferenceRouter = InferenceFixtures.fixedRouter(),
|
|
validationPipeline = ValidationPipeline(validators = listOf(cyclePolicyMissingValidator())),
|
|
approvalEngine = DefaultApprovalEngine(),
|
|
riskAssessor = DefaultRiskAssessor(),
|
|
),
|
|
retryCoordinator = retryCoordinator,
|
|
artifactStore = NoopArtifactStore(),
|
|
decisionJournalRepository = decisionJournalRepository,
|
|
)
|
|
|
|
private val defaultConfig = OrchestrationConfig(retryPolicy = RetryPolicy(maxAttempts = 1, backoffMs = 0))
|
|
|
|
private fun finding(id: String) = CritiqueFinding(
|
|
id = id,
|
|
role = CritiqueRole.CODE_REVIEWER,
|
|
severity = CritiqueSeverity.MAJOR,
|
|
category = "correctness",
|
|
message = "msg-$id",
|
|
)
|
|
|
|
private suspend fun seedReview(sessionId: SessionId, iteration: Int, verdict: CritiqueVerdict, findings: List<CritiqueFinding>) {
|
|
eventStore.append(
|
|
NewEvent(
|
|
EventMetadata(EventId("rev-$iteration"), sessionId, Clock.System.now(), 1, null, null),
|
|
CritiqueFindingsRecordedEvent(
|
|
sessionId = sessionId,
|
|
stageId = StageId("reviewer"),
|
|
role = CritiqueRole.CODE_REVIEWER,
|
|
modelHash = "model-A",
|
|
iteration = iteration,
|
|
verdict = verdict,
|
|
findings = findings,
|
|
),
|
|
),
|
|
)
|
|
}
|
|
|
|
@Test
|
|
fun `recorded findings are correlated at completion and feed the calibration projection`(): Unit = runBlocking {
|
|
val sessionId = SessionId("calib-1")
|
|
|
|
// Two review rounds: f1 fixed between them (→ UPHELD); f2 persists to an approved final (→ DISMISSED).
|
|
seedReview(sessionId, 1, CritiqueVerdict.CHANGES_REQUESTED, listOf(finding("f1"), finding("f2")))
|
|
seedReview(sessionId, 2, CritiqueVerdict.APPROVED, listOf(finding("f2")))
|
|
|
|
val stageA = StageId("A")
|
|
val done = StageId("done")
|
|
val graph = WorkflowGraph(
|
|
id = "calib-wiring",
|
|
stages = mapOf(stageA to StageConfig(), done to StageConfig()),
|
|
transitions = setOf(TransitionEdge(id = TransitionId("t1"), from = stageA, to = done, condition = { true })),
|
|
start = stageA,
|
|
)
|
|
|
|
orchestrator.run(sessionId, graph, defaultConfig)
|
|
|
|
// The orchestrator emits one outcome per recorded finding at completion — the events the
|
|
// :core:critique calibration projection consumes (its folding is covered by that module's
|
|
// DefaultCriticCalibrationReducerTest).
|
|
val outcomes = eventStore.read(sessionId).mapNotNull { it.payload as? CritiqueOutcomeCorrelatedEvent }
|
|
assertEquals(2, outcomes.size, "one outcome per recorded finding")
|
|
assertEquals(CritiqueOutcome.UPHELD, outcomes.single { it.findingId == "f1" }.outcome)
|
|
assertEquals(CritiqueOutcome.DISMISSED, outcomes.single { it.findingId == "f2" }.outcome)
|
|
assertTrue(outcomes.all { it.modelHash == "model-A" && it.role == CritiqueRole.CODE_REVIEWER })
|
|
}
|
|
}
|