feat(inference): operator-tunable sampling knobs (top_k/min_p/repeat_penalty) for stage requests
GenerationConfig only carried temperature/top_p/max_tokens/stop/seed. Added nullable topK/minP/repeatPenalty, serialized to the llama.cpp and OpenAI-compat request bodies via @EncodeDefault(NEVER) so an unset knob is omitted (the model keeps its own default) and behavior is unchanged unless the operator opts in. Surfaced as a new [sampling] config section feeding the default stage GenerationConfig (the main agentic loop) through TomlWorkflowLoader + ExecutionPlanCompiler; the former hardcoded temperature=0.7/topP=1.0 stage defaults now come from config. Talkie chat/narration keep their own generation settings. Vikunja #46 (task 76) — sampling half.
This commit is contained in:
@@ -17,6 +17,7 @@ import com.correx.core.events.EventDispatcher
|
||||
import com.correx.core.events.stores.EventStore
|
||||
import com.correx.core.inference.CapabilityScore
|
||||
import com.correx.core.inference.Embedder
|
||||
import com.correx.core.inference.GenerationConfig
|
||||
import com.correx.core.inference.InferenceProvider
|
||||
import com.correx.core.inference.InferenceRouter
|
||||
import com.correx.core.inference.ModelCapability
|
||||
@@ -210,10 +211,13 @@ object InfrastructureModule {
|
||||
),
|
||||
)
|
||||
|
||||
fun createWorkflowLoader(extraKinds: List<ArtifactKind> = emptyList()): WorkflowLoader {
|
||||
fun createWorkflowLoader(
|
||||
extraKinds: List<ArtifactKind> = emptyList(),
|
||||
samplingDefaults: GenerationConfig = GenerationConfig(temperature = 0.7, topP = 1.0, maxTokens = 0),
|
||||
): WorkflowLoader {
|
||||
val registry = DefaultArtifactKindRegistry()
|
||||
extraKinds.forEach { registry.register(it) }
|
||||
return TomlWorkflowLoader(registry)
|
||||
return TomlWorkflowLoader(registry, samplingDefaults)
|
||||
}
|
||||
|
||||
fun createPromptLoader(): PromptLoader = FileSystemPromptLoader()
|
||||
|
||||
Reference in New Issue
Block a user