Files

T

kami 12ff2e97b0 docs(qa): live-QA plans + env bring-up for the burndown features

Six QA plans (docs/qa/QA-*.md) pinning observable evidence for the shipped tracks:
research egress + batch source-approval (D), architect contradiction (B§4), llama health
probe (A§4), idea promotion (E), reviewer static-first filter (B§5, partial), and the
brief-echo gate ARM-IT plan (C-A1) — the latter is the go/no-go for arming the gate in
production (it breaks the planner if the model can't emit a parseable brief_echo).

Plus the env: docs/qa/ENV.md runbook + docs/qa/README.md index, and scripts/qa/
(sync-config.sh, searxng-up/down.sh with the JSON-format gotcha, README). Scripts are
set -euo pipefail, syntax-checked, executable. Authored from the repo build/config
(server :apps:server:run, CLI :apps:cli:run, tui-go GOTOOLCHAIN=auto, sample-config.toml).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-06-21 09:14:36 +00:00

3.5 KiB

Raw Permalink Blame History

QA Plan: research egress allowlist + batch source-list approval — `ab7e4be` `4b0024f` `b098d87` `027ff1f`

Status: DRAFT Run date / operator: BACKLOG item: §D — "Dynamic per-session egress allowlist", "§3 batch fetch-approval at source-list level", "Dedicated SourceFetched / LowQualityExtraction events", plus the §D "Live-QA the full run" gate.

Preconditions

server build/branch: feat/backlog-burndown — rebuild only with the server stopped (lazy classloading breaks otherwise)
llama-server + model: a tool-calling-capable model (the research workflow drives web_search/web_fetch tool calls). Managed [[models]] or static [[providers]].
external deps: SearXNG at [tools.research] searxng_url with the JSON format enabled (scripts/qa/searxng-up.sh; see docs/qa/ENV.md §3). Outbound network for the fetched hosts.
config synced: ~/.config/correx matches examples/* + docs/schemas/* (scripts/qa/sync-config.sh); research.toml present; [tools.research] enabled = true.
fixtures/seed: a research prompt that yields ≥2 distinct source hosts (e.g. "summarize the latest on citing official docs").

Acceptance gate (one sentence)

Approving a source list once clears every subsequent web_fetch to those hosts without a per-URL T2 prompt, and the fetch quality lands as dedicated SourceFetched / LowQualityExtraction events — while a host NOT on the list still raises T2.

Checks

#	Action	Expected observable evidence
1	Start a research session; let it reach a state with candidate source URLs.	`web_search` T1 ran; the candidate URLs appear in the session (search result / dossier event).
2	`POST /sessions/{id}/approve-sources` with the candidate URL list (2+ hosts).	Exactly one `EgressHostsGrantedEvent` in `correx events {id}` — `hosts` = the distinct lower-cased hostnames (ports stripped), `reason = "batch source-list approval"`.
3	Let the workflow `web_fetch` each approved URL.	Each fetch to an approved host proceeds with no `ApprovalRequestedEvent` (T2) — `NetworkHostRule` PROCEEDs via the per-session union. A `SourceFetchedEvent` (with `content_sha256` + quality) is recorded per successful fetch.
4	Force a low-quality extraction (e.g. a near-empty / boilerplate page among the sources).	A `LowQualityExtractionEvent` is recorded (not just metadata on `ToolExecutionCompletedEvent`).
5	Have the agent attempt a `web_fetch` to a host not in the approved list.	A T2 `ApprovalRequestedEvent` IS raised for that host (the allowlist is a union, not a blanket open).
6	`correx replay {id}`.	Replay reproduces the same allowlist decisions deterministically (the grant + fetch events are the only inputs; no re-fetch).

Evidence sources: correx events {id} / GET /sessions/{id}/events, server logs (MDC sessionId), correx replay {id}.

Out of scope (explicitly NOT covered this pass)

The standalone §D web-approval client (third client) — separate track, not built.
Headless-browser / LLM-based extraction (non-responsibilities).

Disposition

PASS → move the §D egress/batch/source-event bullets to RETRO.md with this run date + the cited EgressHostsGrantedEvent / SourceFetchedEvent evidence. Status: PASSED.
FAIL → file each failure as a numbered finding in BACKLOG.md (action + repro + the wrong/missing signal). Status: FAILED until green.

3.5 KiB Raw Permalink Blame History

QA Plan: research egress allowlist + batch source-list approval — ab7e4be 4b0024f b098d87 027ff1f