muzick/docs/architecture/07-worker-spec.md

Worker & Job Specification

This document defines the background processing architecture using BullMQ. Workers are responsible for CPU-intensive tasks and time-sensitive cleanup.

1. Job Architecture

All jobs are dispatched via the Backend API and processed by specialized Worker containers.

Job Name	Priority	Responsibility	Trigger
metadata_refresh	Medium	Re-scanning files for tag/metadata changes.	Manual (POST /api/library/reindex)
artwork_download	Low	Fetching covers from Cover Art Archive/Discogs.	On metadata enrichment or new file.
lyrics_download	Low	Fetching synced lyrics (LRCLib).	On metadata enrichment.
recommendation_gen	Low	Calculating new "Vibe" seeds and batching.	On session start or after playback completion.
audio_analysis	High	Running essentia for BPM, Key, Energy.	New file/re-index.
filesystem_rescan	High	Reconciling DB with actual disk state.	Scheduled (Daily/Weekly).
cleanup_sweep	Medium	Managing the Dislike Lifecycle/Deletion.	Scheduled (Hourly).

2. Detailed Job Workflows

A. Audio Analysis Pipeline (audio_analysis)

This is the most resource-intensive job.

1. Input: track_id.
2. Process:
   • Spin up essentia subprocess.
   • Extract BPM, Key, Energy, and Melodic features.
3. Output: Update track_audio_features table and mark audio_features_ready = true.

B. Metadata Enrichment Pipeline (metadata_refresh / artwork_download)

Triggered when a new file is detected or a re-index occurs.

1. Input: track_id.
2. Process:
   • Lookup mbid via MusicBrainz.
   • Fetch lyrics via LRCLib.
   • Fetch artwork via Cover Art Archive.
3. Output: Update tracks, artists, and albums tables.

C. The "Consistency" Worker (filesystem_rescan)

Ensures the database is an accurate reflection of the disk.

1. Process:
   • Walk through /mnt/hdd1/media/Music.
   • Compare mtime and size against DB.
   • Action: If a file is missing, set track.state = 'MISSING'. If a new file is found, trigger metadata_refresh.

D. The "Cleanup" Worker (cleanup_sweep)

Handles the temporal logic of the dislike lifecycle.

1. Process:
   • Check dislikes where state = 'warned' and warned_at < now - 24h.
   • Trigger physical file deletion and DB row removal.
   • Check dislikes where state = 'hidden' and disliked_at < now - 48h.
   • Trigger ntfy notification.

3. Error Handling & Retries

• Exponential Backoff: All external API jobs (MusicBrainz, etc.) must use exponential backoff to respect rate limits.
• Dead Letter Queue (DLQ): Jobs that fail after 5 retries are moved to a DLQ for manual inspection via the Admin Dashboard.
• Idempotency: All jobs must be idempotent. Running audio_analysis twice on the same track_id must not create duplicate data or errors.