kami
3f4c45a7b0
QA-grants.md (10 checks): the headline is grant-in-session-A → auto-clear-in-session-B, keyed on ApprovalDecisionResolved(AUTO_APPROVED, reason="grant:<id>") with no APPROVAL_PENDING; plus tool-binding, no-cap T3/T4, PROJECT same-repo vs different-repo isolation, restart persistence, revoke-re-prompts, session-grant-no-leak regression, and replay determinism. Indexed in README; BACKLOG drafted-plans pointer updated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
56 lines
7.3 KiB
Markdown
56 lines
7.3 KiB
Markdown
# QA Plan: cross-session grants (PROJECT/GLOBAL) + revoke — `c36d41b` `8df0ec7`
|
|
|
|
**Status:** DRAFT
|
|
**Run date / operator:**
|
|
**BACKLOG item:** operator-requested wider grants — see `RETRO.md` 2026-06-21 "wider grants + revoke" (the cross-session live-QA gate noted there). On PASS the gate clears and the entry stays in RETRO with the run date; on FAIL the misses refile into `BACKLOG.md`.
|
|
|
|
---
|
|
|
|
## Preconditions
|
|
|
|
- [ ] **server build/branch:** `feat/backlog-burndown` — _rebuild only with the server stopped (lazy classloading breaks otherwise)_
|
|
- [ ] **llama-server + model:** a tool-calling-capable model that drives a **T2+** tool call reproducibly across runs. The `healthcheck` workflow's `write_script` → `file_write` (T3) is the reference fixture (used by the other QA demos); any workflow with a deterministic write/shell call works.
|
|
- [ ] **external deps:** none (no SearXNG / network needed — this exercises the approval gate, not research).
|
|
- [ ] **config synced:** `~/.config/correx` matches `examples/*` + `docs/schemas/*` (`scripts/qa/sync-config.sh`).
|
|
- [ ] **fixtures/seed:** two distinct workspace dirs — **W1** (e.g. `/home/kami/qa/repoA`) and **W2** (`/home/kami/qa/repoB`) — each launchable as a session workspace, for the PROJECT-isolation check. The TUI binds its cwd as the workspace (the `Hello` frame), so run the TUI from W1 vs W2 to switch projects.
|
|
|
|
## Acceptance gate (one sentence)
|
|
|
|
> A GLOBAL/PROJECT grant created in one session auto-clears its **bound tool** in a *different* session — recorded as `ApprovalDecisionResolved(AUTO_APPROVED, reason="grant:<id>")` with **no** `APPROVAL_PENDING` pause — scoped correctly (PROJECT only on the same workspace; tool-bound; T3/T4 honoured), and **revoking** it makes that tool prompt again — all visible in the grant ledger.
|
|
|
|
## Checks
|
|
|
|
The decisive signal throughout: a grant match emits `ApprovalDecisionResolvedEvent` with
|
|
`outcome = AUTO_APPROVED` and `reason = "grant:<grantId>"` and **falls through to execute**
|
|
(no `OrchestrationPausedEvent("APPROVAL_PENDING")`, no `ApprovalRequestedEvent` for that
|
|
invocation). The human path emits both of those. "It didn't prompt" must be backed by the
|
|
*absence* of `APPROVAL_PENDING` in the log, not just a TUI impression.
|
|
|
|
| # | Action | Expected observable evidence | Result |
|
|
|---|--------|------------------------------|--------|
|
|
| 1 | **Baseline.** Run a session (W1) to a T2+ tool call with no grant in force. | The call pauses for approval: `OrchestrationPausedEvent(reason="APPROVAL_PENDING")` + `ApprovalRequestedEvent(toolName="file_write", tier=T3)` in `correx events {id}`; the TUI shows the approval band. | |
|
|
| 2 | **Create GLOBAL grant.** At that prompt press `A`, then `g`. | Exactly one `ApprovalGrantCreatedEvent` in the **grant ledger** (`GET /sessions/__grant_ledger__/events`) with `scope` = `GLOBAL`, `toolName="file_write"`, `permittedTiers` including T3. The current call then resolves `ApprovalDecisionResolved(AUTO_APPROVED, reason="grant:<id>")` and executes. The TUI grants viewer (`G`) lists the grant (GLOBAL · file_write · ≤T3). | |
|
|
| 3 | **Cross-session auto-clear (headline).** Start a **second, separate** session that makes the same `file_write` call. | That session records `ApprovalDecisionResolved(AUTO_APPROVED, reason="grant:<id>")` and **no** `OrchestrationPausedEvent("APPROVAL_PENDING")` / **no** `ApprovalRequestedEvent` for the invocation; the tool executes. Proves the ledger is unioned across sessions. | |
|
|
| 4 | **Tool-binding.** In a session have the agent call a *different* T2+ tool (e.g. `shell`). | `shell` still raises `OrchestrationPausedEvent("APPROVAL_PENDING")` + `ApprovalRequestedEvent(toolName="shell")` — the GLOBAL grant is bound to `file_write` only, never a blanket open. | |
|
|
| 5 | **No tier cap (T3/T4).** Confirm the auto-clear in #3 was a **T3** call. (Optional: at a prompt, grant a tool whose permitted tiers reach T4 and drive a T4 call.) | The ≥T3 call resolves `AUTO_APPROVED` — the removed T2 ceiling no longer blocks high tiers. (The old build rejected `CreateGrant` tiers > T2.) | |
|
|
| 6a | **PROJECT scope — same repo.** Revoke the GLOBAL grant (#8). From the TUI in **W1**, drive `file_write`, press `A` then `p`. Then start another session whose cwd is **W1**. | Ledger `ApprovalGrantCreatedEvent` `scope = PROJECT`, `projectId` = canonical absolute path of W1 (e.g. `/home/kami/qa/repoA`). The new W1 session auto-clears `file_write` (`AUTO_APPROVED, reason="grant:<id>"`). | |
|
|
| 6b | **PROJECT scope — different repo.** Start a session whose cwd is **W2**, same `file_write` call. | The W2 session **prompts** (`APPROVAL_PENDING` + `ApprovalRequestedEvent`) — the PROJECT grant does not match a different workspace's derived `projectId`. | |
|
|
| 7 | **Persistence across restart.** Stop and restart the server; reopen the TUI grants viewer (`G`) / re-read `GET /sessions/__grant_ledger__/events`. | The standing grant is still listed (the ledger stream replays); a fresh session still auto-clears its tool. | |
|
|
| 8 | **Revoke.** In the grants viewer press `x` on the grant (or send `RevokeGrant`). | An `ApprovalGrantExpiredEvent(grantId=<id>)` is appended to the **ledger**; the viewer's next `grant.list` no longer lists it. A new session calling that tool now **prompts again** (`APPROVAL_PENDING` + `ApprovalRequestedEvent` reappear). | |
|
|
| 9 | **Session-grant does not leak (regression).** At a prompt press `A` then `s` (or Enter); then start a *different* session with the same call. | The SESSION grant is written to the **session's own** stream (not `__grant_ledger__`); the other session still **prompts** (`APPROVAL_PENDING`). Confirms SESSION scope stayed per-session. | |
|
|
| 10 | **Audit + replay.** `GET /sessions/__grant_ledger__/events`; `correx replay {consuming-session-id}`. | The ledger shows the create + expire pair (nothing deleted — invariant #9). Replay reproduces the same AUTO_APPROVED / PENDING decisions deterministically from the recorded grants (no re-prompt). | |
|
|
|
|
Evidence sources: the TUI grants viewer (`G`, backed by `ListGrants`→`grant.list`), `GET /sessions/__grant_ledger__/events` (the ledger is a real event stream — `correx events __grant_ledger__` if the CLI accepts the raw stream id), `correx events {id}` / `GET /sessions/{id}/events` for the consuming sessions, server logs (MDC sessionId), and `correx replay {id}` for determinism.
|
|
|
|
## Out of scope (explicitly NOT covered this pass)
|
|
|
|
- The §D standalone web-approval client (third client) — separate track, not built.
|
|
- **STAGE**-scope grants — internal, unchanged by this work.
|
|
- Time-based expiry (`expiresAt`): the TUI does not set one today, so a grant is standing-until-revoked; the engine *does* honour `expiresAt` if a future client sets it, but that path is not exercised here.
|
|
- The LLM-annotation command-card layer and other unrelated TUI overlays.
|
|
|
|
## Disposition
|
|
|
|
- **PASS** → the cross-session gate in `RETRO.md` (2026-06-21) clears; set this plan's Status: PASSED with the run date and the cited `ApprovalGrantCreatedEvent` / `AUTO_APPROVED reason="grant:<id>"` / `ApprovalGrantExpiredEvent` evidence.
|
|
- **FAIL** → file each failure as a numbered finding in `BACKLOG.md` (action + exact repro + the wrong/missing signal — e.g. "step 3: session B still emitted APPROVAL_PENDING → ledger union not consulted"), fix, then re-run only the failed checks. Status: FAILED until green.
|