correx/docs/qa/QA-brief-echo-gate.md

QA Plan: brief echo-back gate — bcc59d2

Plan-pipeline-addenda §A1. Drafted from the diff (commit bcc59d2) per the BACKLOG QA rule.

Status: DRAFT Run date / operator: BACKLOG item: §C A1 "brief echo-back gate".

---

Preconditions

• server build/branch: master @ bcc59d2 (or later) — rebuild only with the server stopped.
• real planner-capable model running (the echo stage is an inference; needs a live model that can emit the structured restatement — cannot be exercised with a stub).
• runtime install (like the research workflow): copy docs/schemas/brief_echo.json → ~/.config/correx/schemas/, examples/workflows/prompts/brief_echo.md → ~/.config/correx/prompts/, and add the brief_echo [[artifacts]] block from examples/workflows/artifacts.config.toml to ~/.config/correx/config.toml. Also copy the updated role_pipeline.toml.
• launch via the role_pipeline workflow.

Acceptance gate (one sentence)

The gate is correct iff a faithful restatement lets the pipeline proceed to the planner, while a restatement that drops a requirement or invents a file emits BriefEchoMismatchEvent, fails the brief_echo stage retryably (planner does NOT run), and the same mismatch is recomputed identically on replay.

Checks

#	Action	Expected observable evidence	Result
1	Run role_pipeline on a normal request; let analyst→architect→brief_echo run	correx events <id> shows a brief_echo stage running + producing a validated brief_echo artifact; flow reaches the planner stage; NO BriefEchoMismatchEvent
2	Inspect the brief_echo artifact (artifact viewer v / events)	It contains goal / referenced_files / referenced_symbols / acceptance_criteria reflecting the analyst brief
3	Force a misread: prompt/seed so the echo DROPS a requirement (or use a model run that does)	BriefEchoMismatchEvent with the dropped requirement in uncoveredRequirements; the brief_echo stage FAILS retryably; planner has NOT produced a plan
4	Force a hallucinated file in the echo (reference a path not in the brief)	BriefEchoMismatchEvent with that path in hallucinatedFiles; stage fails retryably; feedback names the invented file
5	Let the retry happen with the feedback	The re-run echo (corrected) passes → flow proceeds to planner; retry budget respected (≤ max_retries, then surfaces to operator)
6	Confirm symbols are non-blocking	An echo referencing a symbol absent from the brief is recorded in hallucinatedSymbols but does NOT by itself fail the stage (isMismatch ignores symbols in v1)
7	correx replay <id> on a session that had a mismatch	The mismatch is recomputed identically from the recorded analysis + brief_echo artifacts; double-read digest stable; no environment re-query (pure diff)
8	Non-opted pipeline (any workflow without brief_echo=true)	Behaves exactly as before — gate is a no-op; grounding still runs on the analyst stage

Evidence sources: correx events <id> (stage chain + BriefEchoMismatchEvent), the artifact viewer for the brief_echo content, correx replay <id> for determinism.

Out of scope (NOT covered this pass)

• freestyle_planning.toml wiring (deliberate follow-up — gate is on role_pipeline only).
• Tuning the COVERAGE_THRESHOLD (0.34) — adjust only if live runs show false mismatches/passes.
• A1's sibling addenda items A2 (stage checkpointing) / A3 (calibration) — separate BACKLOG items.

Disposition

• PASS → MOVE the §C A1 entry into RETRO.md (cite bcc59d2, run date, evidence).
• FAIL → file each failure as a numbered finding back into BACKLOG.md (action + repro + wrong/missing signal), fix, re-run only the failed checks. Set Status: FAILED until green.
  • If live runs show the deterministic diff is too strict/loose (false mismatches or missed misreads), that's a tuning finding on COVERAGE_THRESHOLD / the hallucination heuristic — record the observed case.