fix(inference): fold non-L0 SYSTEM entries into the leading system message
Strict chat templates (Qwen3.5) reject any system message after index 0. Recalled L3 memory, L2 stage summaries, and retrieval entries carry EntryRole.SYSTEM and rendered as standalone mid-conversation system turns, making llama-server 400 on every router turn once L3 recall returned hits.
This commit is contained in:
@@ -23,15 +23,18 @@ object PromptRenderer {
|
||||
}
|
||||
|
||||
fun render(contextPack: ContextPack): List<ChatMessage> {
|
||||
val sorted = contextPack.layers.entries.sortedBy { it.key.ordinal }
|
||||
val systemContent = sorted
|
||||
.filter { it.key == ContextLayer.L0 }
|
||||
.flatMap { it.value }
|
||||
.joinToString("\n\n") { it.content }
|
||||
.takeIf { it.isNotBlank() }
|
||||
val conversationMessages = contextPack.layers.entries
|
||||
.filter { it.key != ContextLayer.L0 }
|
||||
// Every SYSTEM-role entry folds into the single leading system message, whatever its
|
||||
// layer (L0 additionally folds regardless of role). Strict chat templates (e.g. Qwen)
|
||||
// reject any system message that is not the first message, so recalled memory, stage
|
||||
// summaries, and retrieval entries must never render as standalone system turns.
|
||||
val (systemEntries, conversationEntries) = contextPack.layers.entries
|
||||
.flatMap { (layer, entries) -> entries.map { layer to it } }
|
||||
.partition { (layer, entry) -> layer == ContextLayer.L0 || entry.role == EntryRole.SYSTEM }
|
||||
val systemContent = systemEntries
|
||||
.sortedWith(compareBy({ it.first.ordinal }, { it.second.ordinal }))
|
||||
.joinToString("\n\n") { it.second.content }
|
||||
.takeIf { it.isNotBlank() }
|
||||
val conversationMessages = conversationEntries
|
||||
.sortedWith(compareBy({ it.second.ordinal }, { layerPriority(it.first) }))
|
||||
.map { (_, entry) -> entry.toChatMessage() }
|
||||
val messages = buildList {
|
||||
|
||||
Reference in New Issue
Block a user