muzick/docs/architecture/02-invariants-and-risks.md

Invariants and Risks

This document outlines the critical rules that MUST be respected to maintain system integrity and the identified technical risks.

1. System Invariants (The "Never Break" Rules)

A. Data Consistency (The "No Ghost Tracks" Rule)

• Invariant: Every track record in the database must correspond to a physical file on the disk.
• Mechanism: The Consistency Worker must run periodically to reconcile the database with the /mnt/hdd1/media/Music directory. Any discrepancy must result in the track being marked as MISSING in the DB, rather than deleted immediately.

B. Session Integrity (The "No Deadlocks" Rule)

• Invariant: An ACTIVE recommendation batch must eventually reach a terminal state (RESOLVED or FAILED).
• Mechanism: Every batch must have a last_interaction_at timestamp. A background sweep must transition stale ACTIVE sessions to RESOLVED to allow new sessions to start.

C. Filesystem Safety (The "Irreversible Action" Rule)

• Invariant: Hard deletion of a file from the filesystem is the final, irreversible step in the PENDING_REMOVAL lifecycle.
• Invariant: Exactly one process may unlink a library file, and exactly one code path may do it: the worker's cleanup sweep (workers/src/cleanup.service.ts). Nothing reachable from an HTTP request may write to the library.

Implemented behaviour

Mount. Only the worker service binds /mnt/hdd1/media/Music read-write. backend keeps :ro, so an API bug or a compromised request handler physically cannot remove a file. The backend's two deletion entry points (permanentlyDeleteTrack, the Quarantine immediate-delete endpoint; and mergeDuplicates, the admin dedup merge) therefore do not unlink — they commit the DB side and queue the path into pending_file_deletions for the worker to carry out under the gates below.

Ordering. The DB transaction commits first, the unlink happens after:

1. Select eligible candidates (all gates applied in SQL).
2. In one transaction: write the deleted_permanent audit row, insert a pending_file_deletions marker, DELETE FROM tracks. Commit.
3. unlink(). On success, clear the marker. On failure, log at error and increment attempts / record last_error on the marker.

A marker that outlives its sweep is the durable record of a file whose DB row is gone but whose bytes are not — from a failed unlink or a worker killed mid-sweep. The next sweep retries it. Failures are never swallowed.

Audit row. feedback.track_id is ON DELETE SET NULL, so the audit row survives its track but loses its identity. Deletion rows therefore carry denormalised track_path, track_title, track_artist. This row is the only forensic record that the file ever existed.

Gates. All three default to the safe value; none is set to an unsafe value anywhere in the repo.

Env var	Default	Effect
MUZICK_ALLOW_HARD_DELETE	off	Master switch. While off the sweep runs its full selection logic and logs exactly which files it would delete, performs no unlink, and makes no DB write of any kind — not the audit row, not the terminal-state transition. The owner enables it by hand after reviewing a dry-run log.
MUZICK_HARD_DELETE_MAX_PER_SWEEP	5	Blast-radius cap. On hitting it the sweep stops, logs at error, and leaves the remainder for the next run, so a selection bug cannot empty the library in one pass.
MUZICK_HARD_DELETE_GRACE_DAYS	7	Minimum age since the track entered HIDDEN, measured from dislikes.hidden_at (an explicit column — an irreversible clock must not be inferred). Separate from, and much longer than, dislikes.grace_hours (default 48), which only governs HIDDEN → WARNED.

• Mechanism: A track's file is deleted only when all of: 24h have passed since warned_at; MUZICK_HARD_DELETE_GRACE_DAYS have passed since hidden_at; the user has not triggered a RESTORE (which deletes the dislike row outright); the per-sweep cap has not been reached; and MUZICK_ALLOW_HARD_DELETE is on.

2. Known Technical Risks

A. The "Similarity Explosion" (Scalability)

• Risk: Precomputing a O(n^2) similarity matrix for large libraries will exhaust database resources.
• Mitigation:
  • Use Tiered Similarity: Metadata-based matches (Instant) \rightarrow Audio-feature matches (Asynchronous/On-demand).
  • Limit similarity computation to tracks within the same genre or recent listening window.

B. Computational Exhaustion (Resource Management)

• Risk: Heavy audio analysis (Essentia) can starve the API of CPU/RAM.
• Mitigation: Audio analysis and metadata enrichment must run in dedicated worker processes (containers) with strict resource limits (cgroups/Docker).

C. Metadata Drift

• Risk: External providers (MusicBrainz/Discogs) may provide conflicting or low-quality data.
• Mitigation: Implement a priority-based enrichment pipeline and allow manual user overrides via the UI.

D. Race Conditions (The "Cleanup Race")

• Risk: A user interacts with a track at the exact moment the Sweep Worker attempts to delete the file.
• Mitigation: Use transactional state transitions (e.g., UPDATE tracks SET state = 'HIDDEN' WHERE id = X AND state = 'PENDING_REMOVAL') to ensure an action only happens if the state hasn't changed.