From 3f4c45a7b08c3ddd51b64d1c28a6a9179d09f5e5 Mon Sep 17 00:00:00 2001 From: kami Date: Sun, 21 Jun 2026 22:41:02 +0000 Subject: [PATCH] docs(qa): live-QA plan for cross-session grants + revoke MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit QA-grants.md (10 checks): the headline is grant-in-session-A โ†’ auto-clear-in-session-B, keyed on ApprovalDecisionResolved(AUTO_APPROVED, reason="grant:") with no APPROVAL_PENDING; plus tool-binding, no-cap T3/T4, PROJECT same-repo vs different-repo isolation, restart persistence, revoke-re-prompts, session-grant-no-leak regression, and replay determinism. Indexed in README; BACKLOG drafted-plans pointer updated. Co-Authored-By: Claude Opus 4.8 --- BACKLOG.md | 5 ++-- docs/qa/QA-grants.md | 55 ++++++++++++++++++++++++++++++++++++++++++++ docs/qa/README.md | 10 ++++---- 3 files changed, 64 insertions(+), 6 deletions(-) create mode 100644 docs/qa/QA-grants.md diff --git a/BACKLOG.md b/BACKLOG.md index 439879e8..61059635 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -46,8 +46,9 @@ matching bullets below are superseded; only the noted follow-ups remain live: **๐Ÿ“‹ Live-QA plans drafted** (`docs/qa/QA-*.md`, index `docs/qa/README.md`, env `docs/qa/ENV.md` + `scripts/qa/`): research-egress (D), architect-contradiction (Bยง4), llama-health-probe (Aยง4), idea-promotion (E), -reviewer-static-first (Bยง5, partial), and the **brief-echo-gate ARM-IT** plan (C-A1 โ€” run it before -arming the gate; it's the go/no-go). These features stay listed until their plan **passes live**, then move to RETRO. +reviewer-static-first (Bยง5, partial), the **brief-echo-gate ARM-IT** plan (C-A1 โ€” run it before +arming the gate; it's the go/no-go), and **grants** (cross-session PROJECT/GLOBAL + revoke โ€” `c36d41b` `8df0ec7`, +see RETRO 2026-06-21). These features stay listed until their plan **passes live**, then move to RETRO. --- diff --git a/docs/qa/QA-grants.md b/docs/qa/QA-grants.md new file mode 100644 index 00000000..807fc676 --- /dev/null +++ b/docs/qa/QA-grants.md @@ -0,0 +1,55 @@ +# QA Plan: cross-session grants (PROJECT/GLOBAL) + revoke โ€” `c36d41b` `8df0ec7` + +**Status:** DRAFT +**Run date / operator:** +**BACKLOG item:** operator-requested wider grants โ€” see `RETRO.md` 2026-06-21 "wider grants + revoke" (the cross-session live-QA gate noted there). On PASS the gate clears and the entry stays in RETRO with the run date; on FAIL the misses refile into `BACKLOG.md`. + +--- + +## Preconditions + +- [ ] **server build/branch:** `feat/backlog-burndown` โ€” _rebuild only with the server stopped (lazy classloading breaks otherwise)_ +- [ ] **llama-server + model:** a tool-calling-capable model that drives a **T2+** tool call reproducibly across runs. The `healthcheck` workflow's `write_script` โ†’ `file_write` (T3) is the reference fixture (used by the other QA demos); any workflow with a deterministic write/shell call works. +- [ ] **external deps:** none (no SearXNG / network needed โ€” this exercises the approval gate, not research). +- [ ] **config synced:** `~/.config/correx` matches `examples/*` + `docs/schemas/*` (`scripts/qa/sync-config.sh`). +- [ ] **fixtures/seed:** two distinct workspace dirs โ€” **W1** (e.g. `/home/kami/qa/repoA`) and **W2** (`/home/kami/qa/repoB`) โ€” each launchable as a session workspace, for the PROJECT-isolation check. The TUI binds its cwd as the workspace (the `Hello` frame), so run the TUI from W1 vs W2 to switch projects. + +## Acceptance gate (one sentence) + +> A GLOBAL/PROJECT grant created in one session auto-clears its **bound tool** in a *different* session โ€” recorded as `ApprovalDecisionResolved(AUTO_APPROVED, reason="grant:")` with **no** `APPROVAL_PENDING` pause โ€” scoped correctly (PROJECT only on the same workspace; tool-bound; T3/T4 honoured), and **revoking** it makes that tool prompt again โ€” all visible in the grant ledger. + +## Checks + +The decisive signal throughout: a grant match emits `ApprovalDecisionResolvedEvent` with +`outcome = AUTO_APPROVED` and `reason = "grant:"` and **falls through to execute** +(no `OrchestrationPausedEvent("APPROVAL_PENDING")`, no `ApprovalRequestedEvent` for that +invocation). The human path emits both of those. "It didn't prompt" must be backed by the +*absence* of `APPROVAL_PENDING` in the log, not just a TUI impression. + +| # | Action | Expected observable evidence | Result | +|---|--------|------------------------------|--------| +| 1 | **Baseline.** Run a session (W1) to a T2+ tool call with no grant in force. | The call pauses for approval: `OrchestrationPausedEvent(reason="APPROVAL_PENDING")` + `ApprovalRequestedEvent(toolName="file_write", tier=T3)` in `correx events {id}`; the TUI shows the approval band. | | +| 2 | **Create GLOBAL grant.** At that prompt press `A`, then `g`. | Exactly one `ApprovalGrantCreatedEvent` in the **grant ledger** (`GET /sessions/__grant_ledger__/events`) with `scope` = `GLOBAL`, `toolName="file_write"`, `permittedTiers` including T3. The current call then resolves `ApprovalDecisionResolved(AUTO_APPROVED, reason="grant:")` and executes. The TUI grants viewer (`G`) lists the grant (GLOBAL ยท file_write ยท โ‰คT3). | | +| 3 | **Cross-session auto-clear (headline).** Start a **second, separate** session that makes the same `file_write` call. | That session records `ApprovalDecisionResolved(AUTO_APPROVED, reason="grant:")` and **no** `OrchestrationPausedEvent("APPROVAL_PENDING")` / **no** `ApprovalRequestedEvent` for the invocation; the tool executes. Proves the ledger is unioned across sessions. | | +| 4 | **Tool-binding.** In a session have the agent call a *different* T2+ tool (e.g. `shell`). | `shell` still raises `OrchestrationPausedEvent("APPROVAL_PENDING")` + `ApprovalRequestedEvent(toolName="shell")` โ€” the GLOBAL grant is bound to `file_write` only, never a blanket open. | | +| 5 | **No tier cap (T3/T4).** Confirm the auto-clear in #3 was a **T3** call. (Optional: at a prompt, grant a tool whose permitted tiers reach T4 and drive a T4 call.) | The โ‰ฅT3 call resolves `AUTO_APPROVED` โ€” the removed T2 ceiling no longer blocks high tiers. (The old build rejected `CreateGrant` tiers > T2.) | | +| 6a | **PROJECT scope โ€” same repo.** Revoke the GLOBAL grant (#8). From the TUI in **W1**, drive `file_write`, press `A` then `p`. Then start another session whose cwd is **W1**. | Ledger `ApprovalGrantCreatedEvent` `scope = PROJECT`, `projectId` = canonical absolute path of W1 (e.g. `/home/kami/qa/repoA`). The new W1 session auto-clears `file_write` (`AUTO_APPROVED, reason="grant:"`). | | +| 6b | **PROJECT scope โ€” different repo.** Start a session whose cwd is **W2**, same `file_write` call. | The W2 session **prompts** (`APPROVAL_PENDING` + `ApprovalRequestedEvent`) โ€” the PROJECT grant does not match a different workspace's derived `projectId`. | | +| 7 | **Persistence across restart.** Stop and restart the server; reopen the TUI grants viewer (`G`) / re-read `GET /sessions/__grant_ledger__/events`. | The standing grant is still listed (the ledger stream replays); a fresh session still auto-clears its tool. | | +| 8 | **Revoke.** In the grants viewer press `x` on the grant (or send `RevokeGrant`). | An `ApprovalGrantExpiredEvent(grantId=)` is appended to the **ledger**; the viewer's next `grant.list` no longer lists it. A new session calling that tool now **prompts again** (`APPROVAL_PENDING` + `ApprovalRequestedEvent` reappear). | | +| 9 | **Session-grant does not leak (regression).** At a prompt press `A` then `s` (or Enter); then start a *different* session with the same call. | The SESSION grant is written to the **session's own** stream (not `__grant_ledger__`); the other session still **prompts** (`APPROVAL_PENDING`). Confirms SESSION scope stayed per-session. | | +| 10 | **Audit + replay.** `GET /sessions/__grant_ledger__/events`; `correx replay {consuming-session-id}`. | The ledger shows the create + expire pair (nothing deleted โ€” invariant #9). Replay reproduces the same AUTO_APPROVED / PENDING decisions deterministically from the recorded grants (no re-prompt). | | + +Evidence sources: the TUI grants viewer (`G`, backed by `ListGrants`โ†’`grant.list`), `GET /sessions/__grant_ledger__/events` (the ledger is a real event stream โ€” `correx events __grant_ledger__` if the CLI accepts the raw stream id), `correx events {id}` / `GET /sessions/{id}/events` for the consuming sessions, server logs (MDC sessionId), and `correx replay {id}` for determinism. + +## Out of scope (explicitly NOT covered this pass) + +- The ยงD standalone web-approval client (third client) โ€” separate track, not built. +- **STAGE**-scope grants โ€” internal, unchanged by this work. +- Time-based expiry (`expiresAt`): the TUI does not set one today, so a grant is standing-until-revoked; the engine *does* honour `expiresAt` if a future client sets it, but that path is not exercised here. +- The LLM-annotation command-card layer and other unrelated TUI overlays. + +## Disposition + +- **PASS** โ†’ the cross-session gate in `RETRO.md` (2026-06-21) clears; set this plan's Status: PASSED with the run date and the cited `ApprovalGrantCreatedEvent` / `AUTO_APPROVED reason="grant:"` / `ApprovalGrantExpiredEvent` evidence. +- **FAIL** โ†’ file each failure as a numbered finding in `BACKLOG.md` (action + exact repro + the wrong/missing signal โ€” e.g. "step 3: session B still emitted APPROVAL_PENDING โ†’ ledger union not consulted"), fix, then re-run only the failed checks. Status: FAILED until green. diff --git a/docs/qa/README.md b/docs/qa/README.md index 00829ff8..31c44bdf 100644 --- a/docs/qa/README.md +++ b/docs/qa/README.md @@ -17,14 +17,16 @@ to RETRO with the run date + cited evidence; on FAIL โ†’ refile each miss as a n | `QA-idea-promotion.md` | idea-board โ†’ `.correx/project.toml` promotion | `f107ff5` | bound workspace (router model optional โ€” can seed the event) | | `QA-reviewer-static-first.md` | reviewer static-findings exclusion (PARTIAL โ€” producing stage not built) | `447fc7a` | seed `StaticFindingsRecordedEvent` | | `QA-brief-echo-gate.md` | **ARM-IT** plan: brief echo-back gate (off by default; risky to arm blind) | `1df7af5` | prod-candidate model | +| `QA-grants.md` | cross-session grants (PROJECT/GLOBAL) + revoke | `c36d41b` `8df0ec7` | model + a T2+ tool call, 2 sessions, 2 workspaces | ## Order of attack (suggested) 1. **`QA-llama-health-probe`** + **`QA-idea-promotion`** โ€” cheapest, least model-dependent (health is liveness; promotion can be seeded). Quick wins. -2. **`QA-research-egress`** โ€” once SearXNG is up. -3. **`QA-architect-contradiction`** โ€” needs a real embedder + two sessions. -4. **`QA-brief-echo-gate`** โ€” run this **before** arming the gate anywhere real; check 2 (100% parseable-echo across runs) is the go/no-go for production. -5. **`QA-reviewer-static-first`** โ€” partial until the static-check stage seam lands. +2. **`QA-grants`** โ€” no network/embedder; just a model that drives one repeatable T2+ tool call. The headline check (grant in session A โ†’ auto-clear in session B) is a strong, cheap signal. +3. **`QA-research-egress`** โ€” once SearXNG is up. +4. **`QA-architect-contradiction`** โ€” needs a real embedder + two sessions. +5. **`QA-brief-echo-gate`** โ€” run this **before** arming the gate anywhere real; check 2 (100% parseable-echo across runs) is the go/no-go for production. +6. **`QA-reviewer-static-first`** โ€” partial until the static-check stage seam lands. ## Still-open ยงF live-QA gates (no dedicated plan yet โ€” same env applies)