feat(inference): operator-tunable sampling knobs (top_k/min_p/repeat_penalty) for stage requests

GenerationConfig only carried temperature/top_p/max_tokens/stop/seed. Added nullable
topK/minP/repeatPenalty, serialized to the llama.cpp and OpenAI-compat request bodies
via @EncodeDefault(NEVER) so an unset knob is omitted (the model keeps its own default)
and behavior is unchanged unless the operator opts in.

Surfaced as a new [sampling] config section feeding the default stage GenerationConfig
(the main agentic loop) through TomlWorkflowLoader + ExecutionPlanCompiler; the former
hardcoded temperature=0.7/topP=1.0 stage defaults now come from config. Talkie
chat/narration keep their own generation settings.

Vikunja #46 (task 76) — sampling half.
This commit is contained in:
2026-07-12 17:54:25 +04:00
parent b2c7bbe401
commit d89d4e32a9
13 changed files with 125 additions and 16 deletions
@@ -385,10 +385,23 @@ fun main() {
// Plan-compile gate: wrap the ExecutionPlanCompiler as a post-stage check so a compile-invalid
// architect plan is handed back through the retry-feedback loop instead of parking the session in
// ACTIVE (post-planning compile dead-end). Returns null on success, the compiler message on failure.
// Operator sampling defaults ([sampling] config) sent on every stage inference request. maxTokens=0
// is a placeholder; the loader/compiler pin it per-stage to the token budget via copy().
val stageSamplingDefaults = with(correxConfig.sampling) {
GenerationConfig(
temperature = temperature,
topP = topP,
maxTokens = 0,
topK = topK,
minP = minP,
repeatPenalty = repeatPenalty,
)
}
val planCompiler = ExecutionPlanCompiler(
artifactKindRegistry,
toolRegistry.all().map { it.name }.toSet(),
injectRecovery = true,
samplingDefaults = stageSamplingDefaults,
)
// Startup-load orchestration tuning from the [orchestration] config section into the kernel's
// OrchestrationTuning. ponytail: read once at boot — a config edit needs a restart to take effect
@@ -449,7 +462,7 @@ fun main() {
tuning = orchestrationTuning,
)
val workflowRegistry = FileSystemWorkflowRegistry(
InfrastructureModule.createWorkflowLoader(configArtifactKindsEarly),
InfrastructureModule.createWorkflowLoader(configArtifactKindsEarly, stageSamplingDefaults),
)
// Builds the router facade from a config snapshot, mapping the [router] block onto the domain
// TalkieConfig. Reused by ConfigService's rebuild hook so router knob edits apply live (the