kami
963f845733
The dislike lifecycle promised WARNED -> HIDDEN -> deleted, but nothing ever
removed a file: cleanup.service logged its intent behind
MUZICK_ALLOW_HARD_DELETE and returned, and the two backend delete paths
(hardDeleteTrack, permanentlyDeleteTrack) disagreed about what deletion
meant. The review recommended dropping hard deletion and making HIDDEN
terminal; the owner chose to make deletion real instead.
- cleanup.service performs a true unlink() — no trash directory — for
tracks that have been HIDDEN for a 7-day grace period, then settles the
row. This is the single unlink() call site in the system.
- permanentlyDeleteTrack is the one delete path; hardDeleteTrack is gone.
- a deleted_permanent audit row records what was removed, and
migration 20260730_hard_delete_audit_trail backs it.
MUZICK_ALLOW_HARD_DELETE remains OFF: the docker-compose entry is commented
out, there is no enabling default in code, and the worker's /music bind is
the only writable one. Deletion stays dry-run until the owner opts in
deliberately.
REVIEW-2026-07-30.md open decision: dislike lifecycle.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
62 lines
5.3 KiB
Markdown
62 lines
5.3 KiB
Markdown
# Invariants and Risks
|
|
|
|
This document outlines the critical rules that MUST be respected to maintain system integrity and the identified technical risks.
|
|
|
|
## 1. System Invariants (The "Never Break" Rules)
|
|
|
|
### **A. Data Consistency (The "No Ghost Tracks" Rule)**
|
|
* **Invariant:** Every `track` record in the database must correspond to a physical file on the disk.
|
|
* **Mechanism:** The **Consistency Worker** must run periodically to reconcile the database with the `/mnt/hdd1/media/Music` directory. Any discrepancy must result in the track being marked as `MISSING` in the DB, rather than deleted immediately.
|
|
|
|
### **B. Session Integrity (The "No Deadlocks" Rule)**
|
|
* **Invariant:** An `ACTIVE` recommendation batch must eventually reach a terminal state (`RESOLVED` or `FAILED`).
|
|
* **Mechanism:** Every batch must have a `last_interaction_at` timestamp. A background sweep must transition stale `ACTIVE` sessions to `RESOLVED` to allow new sessions to start.
|
|
|
|
### **C. Filesystem Safety (The "Irreversible Action" Rule)**
|
|
* **Invariant:** Hard deletion of a file from the filesystem is the final, irreversible step in the `PENDING_REMOVAL` lifecycle.
|
|
* **Invariant:** Exactly **one** process may unlink a library file, and exactly **one** code path may do it: the worker's cleanup sweep (`workers/src/cleanup.service.ts`). Nothing reachable from an HTTP request may write to the library.
|
|
|
|
#### Implemented behaviour
|
|
|
|
**Mount.** Only the `worker` service binds `/mnt/hdd1/media/Music` read-write. `backend` keeps `:ro`, so an API bug or a compromised request handler physically cannot remove a file. The backend's two deletion entry points (`permanentlyDeleteTrack`, the Quarantine immediate-delete endpoint; and `mergeDuplicates`, the admin dedup merge) therefore do **not** unlink — they commit the DB side and queue the path into `pending_file_deletions` for the worker to carry out under the gates below.
|
|
|
|
**Ordering.** The DB transaction commits *first*, the unlink happens *after*:
|
|
|
|
1. Select eligible candidates (all gates applied in SQL).
|
|
2. In one transaction: write the `deleted_permanent` audit row, insert a `pending_file_deletions` marker, `DELETE FROM tracks`. Commit.
|
|
3. `unlink()`. On success, clear the marker. On failure, log at **error** and increment `attempts` / record `last_error` on the marker.
|
|
|
|
A marker that outlives its sweep is the durable record of a file whose DB row is gone but whose bytes are not — from a failed unlink or a worker killed mid-sweep. The next sweep retries it. Failures are never swallowed.
|
|
|
|
**Audit row.** `feedback.track_id` is `ON DELETE SET NULL`, so the audit row survives its track but loses its identity. Deletion rows therefore carry denormalised `track_path`, `track_title`, `track_artist`. This row is the only forensic record that the file ever existed.
|
|
|
|
**Gates.** All three default to the safe value; none is set to an unsafe value anywhere in the repo.
|
|
|
|
| Env var | Default | Effect |
|
|
|---|---|---|
|
|
| `MUZICK_ALLOW_HARD_DELETE` | **off** | Master switch. While off the sweep runs its full selection logic and logs exactly which files it *would* delete, performs no unlink, and makes **no** DB write of any kind — not the audit row, not the terminal-state transition. The owner enables it by hand after reviewing a dry-run log. |
|
|
| `MUZICK_HARD_DELETE_MAX_PER_SWEEP` | `5` | Blast-radius cap. On hitting it the sweep stops, logs at error, and leaves the remainder for the next run, so a selection bug cannot empty the library in one pass. |
|
|
| `MUZICK_HARD_DELETE_GRACE_DAYS` | `7` | Minimum age since the track entered `HIDDEN`, measured from `dislikes.hidden_at` (an explicit column — an irreversible clock must not be inferred). Separate from, and much longer than, `dislikes.grace_hours` (default 48), which only governs `HIDDEN` → `WARNED`. |
|
|
|
|
* **Mechanism:** A track's file is deleted only when *all* of: 24h have passed since `warned_at`; `MUZICK_HARD_DELETE_GRACE_DAYS` have passed since `hidden_at`; the user has not triggered a `RESTORE` (which deletes the dislike row outright); the per-sweep cap has not been reached; and `MUZICK_ALLOW_HARD_DELETE` is on.
|
|
|
|
## 2. Known Technical Risks
|
|
|
|
### **A. The "Similarity Explosion" (Scalability)**
|
|
* **Risk:** Precomputing a $O(n^2)$ similarity matrix for large libraries will exhaust database resources.
|
|
* **Mitigation:**
|
|
* Use **Tiered Similarity**: Metadata-based matches (Instant) $\rightarrow$ Audio-feature matches (Asynchronous/On-demand).
|
|
* Limit similarity computation to tracks within the same genre or recent listening window.
|
|
|
|
### **B. Computational Exhaustion (Resource Management)**
|
|
* **Risk:** Heavy audio analysis (Essentia) can starve the API of CPU/RAM.
|
|
* **Mitigation:** Audio analysis and metadata enrichment must run in dedicated worker processes (containers) with strict resource limits (cgroups/Docker).
|
|
|
|
### **C. Metadata Drift**
|
|
* **Risk:** External providers (MusicBrainz/Discogs) may provide conflicting or low-quality data.
|
|
* **Mitigation:** Implement a priority-based enrichment pipeline and allow manual user overrides via the UI.
|
|
|
|
### **D. Race Conditions (The "Cleanup Race")**
|
|
* **Risk:** A user interacts with a track at the exact moment the Sweep Worker attempts to delete the file.
|
|
* **Mitigation:** Use transactional state transitions (e.g., `UPDATE tracks SET state = 'HIDDEN' WHERE id = X AND state = 'PENDING_REMOVAL'`) to ensure an action only happens if the state hasn't changed.
|