Files
correx/docs/qa/QA-architect-contradiction.md
T
kami 12ff2e97b0 docs(qa): live-QA plans + env bring-up for the burndown features
Six QA plans (docs/qa/QA-*.md) pinning observable evidence for the shipped tracks:
research egress + batch source-approval (D), architect contradiction (B§4), llama health
probe (A§4), idea promotion (E), reviewer static-first filter (B§5, partial), and the
brief-echo gate ARM-IT plan (C-A1) — the latter is the go/no-go for arming the gate in
production (it breaks the planner if the model can't emit a parseable brief_echo).

Plus the env: docs/qa/ENV.md runbook + docs/qa/README.md index, and scripts/qa/
(sync-config.sh, searxng-up/down.sh with the JSON-format gotcha, README). Scripts are
set -euo pipefail, syntax-checked, executable. Authored from the repo build/config
(server :apps:server:run, CLI :apps:cli:run, tui-go GOTOOLCHAIN=auto, sample-config.toml).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 09:14:36 +00:00

3.7 KiB

QA Plan: architect contradiction-check (display-only) — 4e08510 eae0a0c

Status: DRAFT Run date / operator: BACKLOG item: §B-§4 "architect contradiction-check" — checker eae0a0c, live server hook 4e08510.


Preconditions

  • server build/branch: feat/backlog-burndown.
  • llama-server + model: a model that runs the role_pipeline (analyst→architect→…) and emits a design artifact with a populated approach field (see docs/schemas/design.json).
  • external deps: none beyond the model.
  • config synced: [project] enabled = true (the checker queries the project:<repoRoot> L3 namespace that ProjectMemoryService.persist writes); [router.embedder] backend = "llamacpp" with a real embedder URL — noop embeddings make every distance meaningless and the gate never fires. [router.l3] backend = "in_memory" is fine for a single run, but note: in-memory L3 is process-lifetime, so the two sessions below must run against the same server process.
  • fixtures/seed: the same workspace/repo root for both sessions (the namespace is project:<repoRoot>).

Acceptance gate (one sentence)

When a later session's architect design decision is semantically near a prior session's recorded decision, a PossibleContradictionFlaggedEvent is appended — and it never halts the stage.

Checks

# Action Expected observable evidence Result
1 Run session A end-to-end on a repo; architect commits a clear decision (e.g. "use Postgres for persistence"). Let the session complete (distillation runs on completion). Session A completes; correx events {A} shows the architect design ArtifactCreatedEvent. The decision is now distilled into L3 under project:<repoRoot> (ProjectMemoryService.persist on completion).
2 Run session B (same server process, same repo) whose architect decision reverses A (e.g. "use SQLite / drop Postgres"). When B's architect design artifact is created, correx events {B} contains a PossibleContradictionFlaggedEventdecisionSummary = B's approach, related[] lists A's decision with a score ≥ 0.75 and source = the project:<repoRoot> turnId.
3 Confirm non-interference. Session B's architect stage still passes and proceeds to the planner — the flag is display-only (no WorkflowFailedEvent, no stage halt around the flag).
4 Run session C with an architect decision unrelated to A (different domain). No PossibleContradictionFlaggedEvent for C (nothing clears the 0.75 threshold). Absence is the evidence.
5 Confirm self-exclusion. The flag in check 2 never lists B's own in-flight decision as related (checker filters entry.sessionId != current).

Evidence sources: correx events {id}, server logs (the [Orchestrator]/checker warn line on a near hit is informative but the event is the gate).

Out of scope (explicitly NOT covered this pass)

  • In-session contradiction (two decisions inside one session) — by design the checker filters to PRIOR sessions; not built and not required.
  • LLM-judged contradiction — v1 is similarity-only, display-only.
  • ADR alternativesConsidered shape (BACKLOG marks this deliberately not taken).

Disposition

  • PASS → move the §B-§4 entry to RETRO.md with this run date + the PossibleContradictionFlaggedEvent evidence from check 2. Status: PASSED.
  • FAIL → file numbered findings in BACKLOG.md. Most likely failure = embedder is noop or the two sessions ran against different processes (in-memory L3 lost) — fix the env, not the code, then re-run. Status: FAILED until green.