# Worker & Job Specification This document defines the background processing architecture using **BullMQ**. Workers are responsible for CPU-intensive tasks and time-sensitive cleanup. ## 1. Job Architecture All jobs are dispatched via the Backend API and processed by specialized Worker containers. | Job Name | Priority | Responsibility | Trigger | | :--- | :--- | :--- | :--- | | `metadata_refresh` | Medium | Re-scanning files for tag/metadata changes. | Manual (`POST /api/library/reindex`) | | `artwork_download` | Low | Fetching covers from Cover Art Archive/Discogs. | On metadata enrichment or new file. | | `lyrics_download` | Low | Fetching synced lyrics (LRCLib). | On metadata enrichment. | | `recommendation_gen` | Low | Calculating new "Vibe" seeds and batching. | On session start or after playback completion. | | `audio_analysis` | High | Running `essentia` for BPM, Key, Energy. | New file/re-index. | | `filesystem_rescan` | High | Reconciling DB with actual disk state. | Scheduled (Daily/Weekly). | | `cleanup_sweep` | Medium | Managing the Dislike Lifecycle/Deletion. | Scheduled (Hourly). | ## 2. Detailed Job Workflows ### **A. Audio Analysis Pipeline (`audio_analysis`)** This is the most resource-intensive job. 1. **Input:** `track_id`. 2. **Process:** * Spin up `essentia` subprocess. * Extract BPM, Key, Energy, and Melodic features. 3. **Output:** Update `track_audio_features` table and mark `audio_features_ready = true`. ### **B. Metadata Enrichment Pipeline (`metadata_refresh` / `artwork_download`)** Triggered when a new file is detected or a re-index occurs. 1. **Input:** `track_id`. 2. **Process:** * Lookup `mbid` via MusicBrainz. * Fetch lyrics via LRCLib. * Fetch artwork via Cover Art Archive. 3. **Output:** Update `tracks`, `artists`, and `albums` tables. ### **C. The "Consistency" Worker (`filesystem_rescan`)** Ensures the database is an accurate reflection of the disk. 1. **Process:** * Walk through `/mnt/hdd1/media/Music`. * Compare `mtime` and `size` against DB. * **Action:** If a file is missing, set `track.state = 'MISSING'`. If a new file is found, trigger `metadata_refresh`. ### **D. The "Cleanup" Worker (`cleanup_sweep`)** Handles the temporal logic of the dislike lifecycle. 1. **Process:** * Check `dislikes` where `state = 'warned'` and `warned_at < now - 24h`. * Trigger physical file deletion and DB row removal. * Check `dislikes` where `state = 'hidden'` and `disliked_at < now - 48h`. * Trigger `ntfy` notification. ## 3. Error Handling & Retries - **Exponential Backoff:** All external API jobs (MusicBrainz, etc.) must use exponential backoff to respect rate limits. - **Dead Letter Queue (DLQ):** Jobs that fail after 5 retries are moved to a DLQ for manual inspection via the Admin Dashboard. - **Idempotency:** All jobs must be idempotent. Running `audio_analysis` twice on the same `track_id` must not create duplicate data or errors.