feat(context,infra): compression pipeline stages 4-5 — token pruning + relevance + ToMe
- build() is now suspend: pipeline runs all stages in fixed order, each gated by level - TOKEN_PRUNE (level 3): TokenPruner interface + LLMLingua-2 sidecar (sidecars/llmlingua) + HttpTokenPruner adapter (fails open if sidecar down); prunes freeform, preserves protected spans, skips tier-0 turns when TIER_SPLIT on - TOME_MERGE (level 8): ToMeMerger collapses near-duplicate freeform turns (Jaccard) - Stage 5 selection: RelevanceScorer + EmbeddingRelevanceScorer (cosine over Embedder); query-conditioned reorder so least-relevant freeform drops first under budget - [orchestration] compression_level + token_pruner_url config, wired in Main - suspend ripple fixed across builder callers/stubs
This commit is contained in:
@@ -0,0 +1,24 @@
|
||||
plugins {
|
||||
id 'java-library'
|
||||
id 'org.jetbrains.kotlin.jvm'
|
||||
id 'org.jetbrains.kotlin.plugin.serialization'
|
||||
}
|
||||
|
||||
ext {
|
||||
ktor_version = '3.0.3'
|
||||
}
|
||||
|
||||
dependencies {
|
||||
implementation project(':core:context')
|
||||
implementation "org.slf4j:slf4j-api"
|
||||
|
||||
implementation "io.ktor:ktor-client-core:$ktor_version"
|
||||
implementation "io.ktor:ktor-client-cio:$ktor_version"
|
||||
implementation "io.ktor:ktor-client-content-negotiation:$ktor_version"
|
||||
implementation "io.ktor:ktor-serialization-kotlinx-json:$ktor_version"
|
||||
testImplementation "org.junit.jupiter:junit-jupiter"
|
||||
testImplementation "io.ktor:ktor-client-mock:$ktor_version"
|
||||
testImplementation "org.jetbrains.kotlin:kotlin-test"
|
||||
}
|
||||
|
||||
tasks.named("koverVerify").configure { enabled = false }
|
||||
Reference in New Issue
Block a user