feat(research): WebFetchTool — bounded network fetch → extract (research-workflow §3)
The first tool that punches through the sandbox to the network. T2, declares NETWORK_ACCESS with url marked NETWORK_TARGET, so egress is enforced by the harness (NetworkHostRule, plane 2) — not tool courtesy. Header-first guards (§3): content-type checked before the body is read (binary/media rejected at the header via the extractor's shared content-type policy); declared Content-Length capped; the read itself is bounded so a server lying about length can't blow the cap (default 10 MB). On success returns the extracted markdown plus metadata for citation lineage + replay: url, content_sha256 (over the raw bytes), fetched_bytes, extractor_version, quality. Consumes slice-1's HtmlMarkdownExtractor. Tested with Ktor MockEngine. NOT yet registered in the live tool registry: with an empty networkAllowedHosts the host rule allows everything, so exposing a fetch tool before the egress allowlist (SearXNG endpoint + approved-source hosts, per-session) is wired would be an open-egress hole. Registration + egress config land with the research workflow slice.
This commit is contained in:
@@ -4,9 +4,19 @@ plugins {
|
||||
id 'org.jetbrains.kotlin.plugin.serialization'
|
||||
}
|
||||
|
||||
ext {
|
||||
ktor_version = '3.0.3'
|
||||
}
|
||||
|
||||
dependencies {
|
||||
implementation project(":core:events")
|
||||
implementation project(":core:tools")
|
||||
implementation project(":core:approvals")
|
||||
implementation 'org.jsoup:jsoup:1.20.1'
|
||||
implementation "io.ktor:ktor-client-core:$ktor_version"
|
||||
implementation "io.ktor:ktor-client-cio:$ktor_version"
|
||||
|
||||
testImplementation "io.ktor:ktor-client-mock:$ktor_version"
|
||||
}
|
||||
|
||||
// Pure deterministic extraction utility; coverage gate is enforced via its own thorough
|
||||
|
||||
Reference in New Issue
Block a user