docs(qa): live-QA plan for cross-session grants + revoke

QA-grants.md (10 checks): the headline is grant-in-session-A →
auto-clear-in-session-B, keyed on ApprovalDecisionResolved(AUTO_APPROVED,
reason="grant:<id>") with no APPROVAL_PENDING; plus tool-binding, no-cap
T3/T4, PROJECT same-repo vs different-repo isolation, restart persistence,
revoke-re-prompts, session-grant-no-leak regression, and replay determinism.
Indexed in README; BACKLOG drafted-plans pointer updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-21 22:41:02 +00:00
parent 67855d8593
commit 3f4c45a7b0
3 changed files with 64 additions and 6 deletions
+55
View File
@@ -0,0 +1,55 @@
# QA Plan: cross-session grants (PROJECT/GLOBAL) + revoke — `c36d41b` `8df0ec7`
**Status:** DRAFT
**Run date / operator:**
**BACKLOG item:** operator-requested wider grants — see `RETRO.md` 2026-06-21 "wider grants + revoke" (the cross-session live-QA gate noted there). On PASS the gate clears and the entry stays in RETRO with the run date; on FAIL the misses refile into `BACKLOG.md`.
---
## Preconditions
- [ ] **server build/branch:** `feat/backlog-burndown`_rebuild only with the server stopped (lazy classloading breaks otherwise)_
- [ ] **llama-server + model:** a tool-calling-capable model that drives a **T2+** tool call reproducibly across runs. The `healthcheck` workflow's `write_script``file_write` (T3) is the reference fixture (used by the other QA demos); any workflow with a deterministic write/shell call works.
- [ ] **external deps:** none (no SearXNG / network needed — this exercises the approval gate, not research).
- [ ] **config synced:** `~/.config/correx` matches `examples/*` + `docs/schemas/*` (`scripts/qa/sync-config.sh`).
- [ ] **fixtures/seed:** two distinct workspace dirs — **W1** (e.g. `/home/kami/qa/repoA`) and **W2** (`/home/kami/qa/repoB`) — each launchable as a session workspace, for the PROJECT-isolation check. The TUI binds its cwd as the workspace (the `Hello` frame), so run the TUI from W1 vs W2 to switch projects.
## Acceptance gate (one sentence)
> A GLOBAL/PROJECT grant created in one session auto-clears its **bound tool** in a *different* session — recorded as `ApprovalDecisionResolved(AUTO_APPROVED, reason="grant:<id>")` with **no** `APPROVAL_PENDING` pause — scoped correctly (PROJECT only on the same workspace; tool-bound; T3/T4 honoured), and **revoking** it makes that tool prompt again — all visible in the grant ledger.
## Checks
The decisive signal throughout: a grant match emits `ApprovalDecisionResolvedEvent` with
`outcome = AUTO_APPROVED` and `reason = "grant:<grantId>"` and **falls through to execute**
(no `OrchestrationPausedEvent("APPROVAL_PENDING")`, no `ApprovalRequestedEvent` for that
invocation). The human path emits both of those. "It didn't prompt" must be backed by the
*absence* of `APPROVAL_PENDING` in the log, not just a TUI impression.
| # | Action | Expected observable evidence | Result |
|---|--------|------------------------------|--------|
| 1 | **Baseline.** Run a session (W1) to a T2+ tool call with no grant in force. | The call pauses for approval: `OrchestrationPausedEvent(reason="APPROVAL_PENDING")` + `ApprovalRequestedEvent(toolName="file_write", tier=T3)` in `correx events {id}`; the TUI shows the approval band. | |
| 2 | **Create GLOBAL grant.** At that prompt press `A`, then `g`. | Exactly one `ApprovalGrantCreatedEvent` in the **grant ledger** (`GET /sessions/__grant_ledger__/events`) with `scope` = `GLOBAL`, `toolName="file_write"`, `permittedTiers` including T3. The current call then resolves `ApprovalDecisionResolved(AUTO_APPROVED, reason="grant:<id>")` and executes. The TUI grants viewer (`G`) lists the grant (GLOBAL · file_write · ≤T3). | |
| 3 | **Cross-session auto-clear (headline).** Start a **second, separate** session that makes the same `file_write` call. | That session records `ApprovalDecisionResolved(AUTO_APPROVED, reason="grant:<id>")` and **no** `OrchestrationPausedEvent("APPROVAL_PENDING")` / **no** `ApprovalRequestedEvent` for the invocation; the tool executes. Proves the ledger is unioned across sessions. | |
| 4 | **Tool-binding.** In a session have the agent call a *different* T2+ tool (e.g. `shell`). | `shell` still raises `OrchestrationPausedEvent("APPROVAL_PENDING")` + `ApprovalRequestedEvent(toolName="shell")` — the GLOBAL grant is bound to `file_write` only, never a blanket open. | |
| 5 | **No tier cap (T3/T4).** Confirm the auto-clear in #3 was a **T3** call. (Optional: at a prompt, grant a tool whose permitted tiers reach T4 and drive a T4 call.) | The ≥T3 call resolves `AUTO_APPROVED` — the removed T2 ceiling no longer blocks high tiers. (The old build rejected `CreateGrant` tiers > T2.) | |
| 6a | **PROJECT scope — same repo.** Revoke the GLOBAL grant (#8). From the TUI in **W1**, drive `file_write`, press `A` then `p`. Then start another session whose cwd is **W1**. | Ledger `ApprovalGrantCreatedEvent` `scope = PROJECT`, `projectId` = canonical absolute path of W1 (e.g. `/home/kami/qa/repoA`). The new W1 session auto-clears `file_write` (`AUTO_APPROVED, reason="grant:<id>"`). | |
| 6b | **PROJECT scope — different repo.** Start a session whose cwd is **W2**, same `file_write` call. | The W2 session **prompts** (`APPROVAL_PENDING` + `ApprovalRequestedEvent`) — the PROJECT grant does not match a different workspace's derived `projectId`. | |
| 7 | **Persistence across restart.** Stop and restart the server; reopen the TUI grants viewer (`G`) / re-read `GET /sessions/__grant_ledger__/events`. | The standing grant is still listed (the ledger stream replays); a fresh session still auto-clears its tool. | |
| 8 | **Revoke.** In the grants viewer press `x` on the grant (or send `RevokeGrant`). | An `ApprovalGrantExpiredEvent(grantId=<id>)` is appended to the **ledger**; the viewer's next `grant.list` no longer lists it. A new session calling that tool now **prompts again** (`APPROVAL_PENDING` + `ApprovalRequestedEvent` reappear). | |
| 9 | **Session-grant does not leak (regression).** At a prompt press `A` then `s` (or Enter); then start a *different* session with the same call. | The SESSION grant is written to the **session's own** stream (not `__grant_ledger__`); the other session still **prompts** (`APPROVAL_PENDING`). Confirms SESSION scope stayed per-session. | |
| 10 | **Audit + replay.** `GET /sessions/__grant_ledger__/events`; `correx replay {consuming-session-id}`. | The ledger shows the create + expire pair (nothing deleted — invariant #9). Replay reproduces the same AUTO_APPROVED / PENDING decisions deterministically from the recorded grants (no re-prompt). | |
Evidence sources: the TUI grants viewer (`G`, backed by `ListGrants``grant.list`), `GET /sessions/__grant_ledger__/events` (the ledger is a real event stream — `correx events __grant_ledger__` if the CLI accepts the raw stream id), `correx events {id}` / `GET /sessions/{id}/events` for the consuming sessions, server logs (MDC sessionId), and `correx replay {id}` for determinism.
## Out of scope (explicitly NOT covered this pass)
- The §D standalone web-approval client (third client) — separate track, not built.
- **STAGE**-scope grants — internal, unchanged by this work.
- Time-based expiry (`expiresAt`): the TUI does not set one today, so a grant is standing-until-revoked; the engine *does* honour `expiresAt` if a future client sets it, but that path is not exercised here.
- The LLM-annotation command-card layer and other unrelated TUI overlays.
## Disposition
- **PASS** → the cross-session gate in `RETRO.md` (2026-06-21) clears; set this plan's Status: PASSED with the run date and the cited `ApprovalGrantCreatedEvent` / `AUTO_APPROVED reason="grant:<id>"` / `ApprovalGrantExpiredEvent` evidence.
- **FAIL** → file each failure as a numbered finding in `BACKLOG.md` (action + exact repro + the wrong/missing signal — e.g. "step 3: session B still emitted APPROVAL_PENDING → ledger union not consulted"), fix, then re-run only the failed checks. Status: FAILED until green.
+6 -4
View File
@@ -17,14 +17,16 @@ to RETRO with the run date + cited evidence; on FAIL → refile each miss as a n
| `QA-idea-promotion.md` | idea-board → `.correx/project.toml` promotion | `f107ff5` | bound workspace (router model optional — can seed the event) |
| `QA-reviewer-static-first.md` | reviewer static-findings exclusion (PARTIAL — producing stage not built) | `447fc7a` | seed `StaticFindingsRecordedEvent` |
| `QA-brief-echo-gate.md` | **ARM-IT** plan: brief echo-back gate (off by default; risky to arm blind) | `1df7af5` | prod-candidate model |
| `QA-grants.md` | cross-session grants (PROJECT/GLOBAL) + revoke | `c36d41b` `8df0ec7` | model + a T2+ tool call, 2 sessions, 2 workspaces |
## Order of attack (suggested)
1. **`QA-llama-health-probe`** + **`QA-idea-promotion`** — cheapest, least model-dependent (health is liveness; promotion can be seeded). Quick wins.
2. **`QA-research-egress`** — once SearXNG is up.
3. **`QA-architect-contradiction`** — needs a real embedder + two sessions.
4. **`QA-brief-echo-gate`** — run this **before** arming the gate anywhere real; check 2 (100% parseable-echo across runs) is the go/no-go for production.
5. **`QA-reviewer-static-first`** — partial until the static-check stage seam lands.
2. **`QA-grants`** — no network/embedder; just a model that drives one repeatable T2+ tool call. The headline check (grant in session A → auto-clear in session B) is a strong, cheap signal.
3. **`QA-research-egress`** — once SearXNG is up.
4. **`QA-architect-contradiction`** — needs a real embedder + two sessions.
5. **`QA-brief-echo-gate`** — run this **before** arming the gate anywhere real; check 2 (100% parseable-echo across runs) is the go/no-go for production.
6. **`QA-reviewer-static-first`** — partial until the static-check stage seam lands.
## Still-open §F live-QA gates (no dedicated plan yet — same env applies)