initial state: muzick music player + recommendation engine
This commit is contained in:
@@ -0,0 +1,52 @@
|
||||
# System Overview
|
||||
|
||||
## High-Level Architecture
|
||||
|
||||
`muzick` follows a distributed architecture centered around a shared PostgreSQL database and a distributed task queue (BullMQ).
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
User((User)) --> Web[Frontend - React/Vite]
|
||||
Web --> API[Backend - Fastify]
|
||||
API --> DB[(PostgreSQL)]
|
||||
API --> Search[Typesense]
|
||||
API --> Cache[(Redis)]
|
||||
API --> Queue[BullMQ]
|
||||
|
||||
subgraph Workers
|
||||
W1[Metadata Worker]
|
||||
W2[Essentia Audio Worker]
|
||||
W3[Cleanup/Sweep Worker]
|
||||
end
|
||||
|
||||
Queue --> W1
|
||||
Queue --> W2
|
||||
Queue --> W3
|
||||
|
||||
W1 --> DB
|
||||
W2 --> DB
|
||||
W3 --> DB
|
||||
W1 --> External[External APIs: MusicBrainz/Discogs]
|
||||
W2 --> Audio[/Filesystem/Music]
|
||||
```
|
||||
|
||||
## Component Roles
|
||||
|
||||
### **1. Frontend (The Interface)**
|
||||
- **React/Vite:** High-performance UI.
|
||||
- **TanStack Router/Query:** Handles complex navigation and provides the **Look-ahead Buffer** for the continuous playback stream.
|
||||
- **Zustand:** Manages the active "Vibe" session state and local playback queue.
|
||||
|
||||
### **2. Backend (The Brain)**
|
||||
- **Fastify:** High-throughput API server.
|
||||
- **Business Logic:** Manages the **Rolling Window** recommendation algorithm, the **Dislike State Machine**, and the **Session Management**.
|
||||
- **Typesense:** Provides ultra-fast fuzzy search across the entire library.
|
||||
|
||||
### **3. Workers (The Muscle)**
|
||||
- **Metadata Worker:** Orchestrates enrichment via MusicBrainz, Discogs, and LRCLib.
|
||||
- **Essentia Worker:** Performs heavy CPU-bound audio feature extraction (BPM, Key, etc.).
|
||||
- **Sweep Worker:** Handles periodic cleanup (Dislike $\rightarrow$ Deletion) and filesystem-to-DB reconciliation.
|
||||
|
||||
### **4. Data Layer**
|
||||
- **PostgreSQL:** The ultimate source of truth for metadata, user preferences, and session history.
|
||||
- **Redis:** Powers the task queue (BullMQ) and provides ephemeral session data.
|
||||
@@ -0,0 +1,37 @@
|
||||
# Invariants and Risks
|
||||
|
||||
This document outlines the critical rules that MUST be respected to maintain system integrity and the identified technical risks.
|
||||
|
||||
## 1. System Invariants (The "Never Break" Rules)
|
||||
|
||||
### **A. Data Consistency (The "No Ghost Tracks" Rule)**
|
||||
* **Invariant:** Every `track` record in the database must correspond to a physical file on the disk.
|
||||
* **Mechanism:** The **Consistency Worker** must run periodically to reconcile the database with the `/mnt/hdd1/media/Music` directory. Any discrepancy must result in the track being marked as `MISSING` in the DB, rather than deleted immediately.
|
||||
|
||||
### **B. Session Integrity (The "No Deadlocks" Rule)**
|
||||
* **Invariant:** An `ACTIVE` recommendation batch must eventually reach a terminal state (`RESOLVED` or `FAILED`).
|
||||
* **Mechanism:** Every batch must have a `last_interaction_at` timestamp. A background sweep must transition stale `ACTIVE` sessions to `RESOLVED` to allow new sessions to start.
|
||||
|
||||
### **C. Filesystem Safety (The "Irreversible Action" Rule)**
|
||||
* **Invariant:** Hard deletion of a file from the filesystem is the final, irreversible step in the `PENDING_REMOVAL` lifecycle.
|
||||
* **Mechanism:** A track only enters the `DELETE_FILE` state after it has been `warned` for at least 24 hours and the user has not explicitly triggered a `RESTORE`.
|
||||
|
||||
## 2. Known Technical Risks
|
||||
|
||||
### **A. The "Similarity Explosion" (Scalability)**
|
||||
* **Risk:** Precomputing a $O(n^2)$ similarity matrix for large libraries will exhaust database resources.
|
||||
* **Mitigation:**
|
||||
* Use **Tiered Similarity**: Metadata-based matches (Instant) $\rightarrow$ Audio-feature matches (Asynchronous/On-demand).
|
||||
* Limit similarity computation to tracks within the same genre or recent listening window.
|
||||
|
||||
### **B. Computational Exhaustion (Resource Management)**
|
||||
* **Risk:** Heavy audio analysis (Essentia) can starve the API of CPU/RAM.
|
||||
* **Mitigation:** Audio analysis and metadata enrichment must run in dedicated worker processes (containers) with strict resource limits (cgroups/Docker).
|
||||
|
||||
### **C. Metadata Drift**
|
||||
* **Risk:** External providers (MusicBrainz/Discogs) may provide conflicting or low-quality data.
|
||||
* **Mitigation:** Implement a priority-based enrichment pipeline and allow manual user overrides via the UI.
|
||||
|
||||
### **D. Race Conditions (The "Cleanup Race")**
|
||||
* **Risk:** A user interacts with a track at the exact moment the Sweep Worker attempts to delete the file.
|
||||
* **Mitigation:** Use transactional state transitions (e.g., `UPDATE tracks SET state = 'HIDDEN' WHERE id = X AND state = 'PENDING_REMOVAL'`) to ensure an action only happens if the state hasn't changed.
|
||||
@@ -0,0 +1,53 @@
|
||||
# Backend Specification
|
||||
|
||||
## Core Technology Stack
|
||||
- **Runtime:** Node.js (TypeScript)
|
||||
- **Framework:** Fastify
|
||||
- **Communication:** REST API (primary)
|
||||
- **Queueing:** BullMQ with Redis
|
||||
|
||||
## 1. API Architecture
|
||||
|
||||
### **Library Management**
|
||||
- `GET /api/tracks`: Paginated list of tracks (supports sort/filter).
|
||||
- `GET /api/tracks/{id}`: Full track metadata.
|
||||
- `GET /api/search?q={query}`: Fuzzy search via Typesense.
|
||||
- `POST /api/library/reindex`: Manual trigger for the indexing worker.
|
||||
|
||||
### **The Vibe Engine (Recommendation)**
|
||||
- `GET /api/vibe`: Returns a **Sequence Chunk** of track IDs.
|
||||
- *Client Implementation:* Uses TanStack Query to implement a "Look-ahead Buffer."
|
||||
- `GET /api/vibe/from-genre?genre={id}`: Generates a session based on a specific genre.
|
||||
|
||||
### **The Lifecycle (Dislike/Removal)**
|
||||
- `POST /api/dislike/{track_id}`: Moves track to `PENDING_REMOVAL` (State: `HIDDEN`).
|
||||
- `DELETE /api/dislike/{id}`: Restores track to `LIBRARY`.
|
||||
- `GET /api/dislikes`: Lists all tracks in the quarantine.
|
||||
- `POST /api/dislikes/sweep`: Manual trigger for the background cleanup worker.
|
||||
|
||||
### **Session Management**
|
||||
- `GET /api/sessions/current`: Returns the metadata for the current `ACTIVE` recommendation batch.
|
||||
- `POST /api/sessions/heartbeat`: Updates `last_interaction_at` for the active batch.
|
||||
|
||||
## 2. The "Rolling Window" Algorithm
|
||||
|
||||
To provide an infinite, evolving stream, the backend implements a **Stateful Sequence Generator**.
|
||||
|
||||
### **Algorithm Steps:**
|
||||
1. **Seed Selection:** Identify the `center_track_id` (the last successfully played track).
|
||||
2. **Candidate Generation:**
|
||||
- **Primary (80%):** Fetch tracks similar to the `center_track_id` (Metadata + Audio Feature match).
|
||||
- **Discovery (20%):** Fetch tracks from the **Probation Pool** (newly acquired recommendations).
|
||||
3. **Sequence Construction:**
|
||||
- Group results into a "Chunk" (e.g., 20 tracks).
|
||||
- Apply **Batch Rules**:
|
||||
- Max 1 song per artist in a single chunk.
|
||||
- Max 2 songs per genre in a single chunk.
|
||||
- Mix in "probation" tracks at a controlled rate.
|
||||
4. **Response:** Return a JSON array of track objects with pre-calculated sequence order.
|
||||
|
||||
## 3. Business Logic Invariants
|
||||
|
||||
- **The "No Ghost" Rule:** The backend MUST verify file existence before returning a track in a `Vibe` sequence.
|
||||
- **The "Success-Driven Center" Rule:** The `center_track_id` is updated ONLY when a track's `completed` flag is set to `true` via `/api/history`.
|
||||
- **The "Atomicity" Rule:** All state transitions (e.g., `PENDING_REMOVAL` $\rightarrow$ `DELETED`) must be handled within a database transaction.
|
||||
@@ -0,0 +1,45 @@
|
||||
# Frontend Specification
|
||||
|
||||
## Core Technology Stack
|
||||
- **Framework:** React (Vite-based)
|
||||
- **Routing:** TanStack Router (Type-safe routing)
|
||||
- **Data Fetching:** TanStack Query (Managing server state & caching)
|
||||
- **State Management:** Zustand (Managing local client-side playback and vibe state)
|
||||
- **Styling:** CSS Variables (Enabling easy theme switching)
|
||||
|
||||
## 1. Key UI Patterns
|
||||
|
||||
### **The "Look-ahead Buffer" (Seamless Playback)**
|
||||
To prevent playback gaps, the frontend implements a **Prefetching Queue**.
|
||||
- **Mechanism:** The client maintains a `playback_buffer` in Zustand containing the next 20 tracks.
|
||||
- **Implementation:** When the user reaches track $N$, TanStack Query triggers a fetch for the next chunk ($N+20$) in the background.
|
||||
- **User Experience:** Tapping "next" is near-instantaneous as the track is already in memory.
|
||||
|
||||
### **The "Vibe" Interface**
|
||||
The core feature is the **Active Vibe Page**.
|
||||
- **Visuals:** Shows the current "Rolling Window" as a progress bar/timeline.
|
||||
- **Real-time Updates:** Displays "Incoming Recommendations" (Probation tracks) as they are discovered.
|
||||
- **Controls:** Quick-access "Keep" / "Dislike" buttons that trigger the lifecycle state machine.
|
||||
|
||||
### **The "Quarantine" Page**
|
||||
A management view for the `PENDING_REMOVAL` state.
|
||||
- **Features:** List of hidden tracks, "time remaining" timers, and "Restore" / "Delete" actions.
|
||||
|
||||
## 2. State Management Strategy
|
||||
|
||||
### **Zustand Stores**
|
||||
- **`usePlaybackStore`:** Tracks `current_track`, `playback_position`, `is_playing`, and the `local_queue`.
|
||||
- **`useVibeStore`:** Manages the `active_session_id` and the current "Center Track" context.
|
||||
|
||||
### **TanStack Query**
|
||||
- Used for all standard CRUD operations (Library, Artists, Albums).
|
||||
- Configured with aggressive `staleTime` for static data (Artists/Albums) and low `staleTime` for dynamic data (History/Stats).
|
||||
|
||||
## 3. Navigation Structure
|
||||
|
||||
- **Home:** Continue Listening, Recently Added, Recently Played.
|
||||
- **Library:** Artists, Albums, Tracks, Genres (hierarchical).
|
||||
- **Vibe:** The infinite player/discovery interface.
|
||||
- **Discover:** Manual exploration of similar artists/tracks.
|
||||
- **Search:** Global fuzzy search.
|
||||
- **Settings:** Theme, playback modes, and automation rules.
|
||||
@@ -0,0 +1,60 @@
|
||||
# Recommendation & Vibe Specification
|
||||
|
||||
This document defines the logic for the "Vibe" engine, moving from simple similarity to a continuous, evolving listening experience.
|
||||
|
||||
## 1. The "Vibe" Concept
|
||||
The "Vibe" is an infinite, stateful listening session. Unlike a static playlist, it is a **Rolling Window** that evolves based on user interaction.
|
||||
|
||||
## 2. Scoring Logic
|
||||
|
||||
When generating a sequence, every candidate track is assigned a score.
|
||||
|
||||
$$Score = (W_{genre} \cdot S_{genre}) + (W_{artist} \cdot S_{artist}) + (W_{random} \cdot R)$$
|
||||
|
||||
### **Scoring Components**
|
||||
* **$S_{genre}$ (Genre Match):**
|
||||
* Uses hierarchical weights (e.g., `Deep House` (1.0) $\rightarrow$ `House` (0.8) $\rightarrow$ `Electronic` (0.5)).
|
||||
* Calculated as the highest weight match between candidate and "Center Track."
|
||||
* **$S_{artist}$ (Artist Similarity):**
|
||||
* A binary or weighted score based on whether the artist is in the user's "Liked" list or frequent listening history.
|
||||
* **$R$ (Randomness/Exploration):**
|
||||
* A jitter factor to ensure the queue doesn't feel repetitive.
|
||||
|
||||
## 3. The Rolling Window Algorithm
|
||||
|
||||
The backend does not return a fixed list. It returns a **Sequence Chunk**.
|
||||
|
||||
### **Step-by-Step Generation**
|
||||
1. **Identify the Center:** Find the `last_successfully_played_track_id`.
|
||||
2. **Candidate Selection:**
|
||||
* **Local Pool (80%):** Tracks from the user's library similar to the center track.
|
||||
* **Probation Pool (20%):** Tracks from the `recommendation_batch` that have `probation=1`.
|
||||
3. **Constraint Application (Batch Rules):**
|
||||
* **Diversity Check:** No more than 1 track from the same artist per chunk.
|
||||
* **Genre Cap:** No more than 2 tracks from the same genre per chunk.
|
||||
4. **Chunking:** Return a sequence of $N$ tracks (e.g., 20).
|
||||
|
||||
## 4. The "Vibe" Session Lifecycle
|
||||
|
||||
A "Vibe" is represented by a `recommendation_batch` record.
|
||||
|
||||
| State | Description |
|
||||
| :--- | :--- |
|
||||
| **ACTIVE** | The user is currently listening. The stream is being generated. |
|
||||
| **RESOLVED** | The session has ended naturally or timed out. |
|
||||
| **FAILED** | The session was interrupted by a critical error or manual reset. |
|
||||
|
||||
### **Transition Logic**
|
||||
* **Start:** User clicks "Start Vibe" $\rightarrow$ Create `ACTIVE` batch.
|
||||
* **Progress:** User listens $\rightarrow$ Update `last_interaction_at` in the batch.
|
||||
* **Termination:**
|
||||
* **Natural:** User exits the app $\rightarrow$ Session marked `RESOLVED` after 24h.
|
||||
* **Manual:** User ends session $\rightarrow$ Mark `RESOLVED` immediately.
|
||||
* **Timeout:** No interaction for 24h $\rightarrow$ Mark `RESOLVED`.
|
||||
|
||||
## 5. Success/Failure Feedback Loop
|
||||
|
||||
The engine learns from the `feedback` table:
|
||||
* **`action = 'promoted'`:** (High Score) Increase weight of this genre/artist in future seeds.
|
||||
* **`action = 'disliked'`:** (Negative Signal) Decrease weight of this genre/artist.
|
||||
* **`action = 'skipped'`:** (Transient Negative) Do not adjust long-term weights, but avoid this specific track in the current session window.
|
||||
@@ -0,0 +1,53 @@
|
||||
# Lifecycle & Removal Specification
|
||||
|
||||
This document defines the "Dislike $\rightarrow$ Delayed Deletion" lifecycle. This state machine ensures user intent is respected while preventing accidental permanent loss of music.
|
||||
|
||||
## 1. The Dislike State Machine
|
||||
|
||||
A track follows this state transition to ensure a "Grace Period" before physical deletion.
|
||||
|
||||
| Current State | Action | Next State | Side Effects |
|
||||
| :--- | :--- | :--- | :--- |
|
||||
| **LIBRARY** | User Dislikes | **PENDING_REMOVAL** | `dislikes` row created; Track becomes `HIDDEN`. |
|
||||
| **PENDING_REMOVAL** | User Restores | **LIBRARY** | `dislikes` row deleted; Track becomes `VISIBLE`. |
|
||||
| **PENDING_REMOVAL** | Grace Period Ends | **WARNING_SENT** | `ntfy` notification sent; `warned_at` timestamp set. |
|
||||
| **WARNING_SENT** | 24h Passes | **DELETED** | File deleted from FS; Track record removed from DB. |
|
||||
|
||||
## 2. Detailed Transitions
|
||||
|
||||
### **Phase 1: The Dislike (Immediate)**
|
||||
When a user triggers a dislike:
|
||||
1. **DB Transaction:**
|
||||
* Update `tracks.state = 'HIDDEN'`.
|
||||
* Insert into `dislikes` table `{track_id, disliked_at: now}`.
|
||||
* Log `feedback(action='disliked')`.
|
||||
2. **UI Update:** The track immediately disappears from all "active" views (Library, Playlists, Vibe Queue, Search).
|
||||
|
||||
### **Phase 2: The Grace Period (The "Safety Net")**
|
||||
* **Duration ($X$):** Configurable (default: 48 hours).
|
||||
* **Behavior:** The track remains on disk and in the database, but is filtered out of all user-facing discovery and playback.
|
||||
|
||||
### **Phase 3: The Warning (The "Nudge")**
|
||||
When `now > disliked_at + X`:
|
||||
1. **Worker Action:** The `Cleanup/Sweep Worker` identifies the track.
|
||||
2. **Notification:** Sends a message via `ntfy` (e.g., *"Are you sure? '<Track Name>' is scheduled for deletion in 24h"*).
|
||||
3. **DB Update:** Set `dislikes.warned_at = now`.
|
||||
|
||||
### **Phase 4: The Finality (The "Cleanup")**
|
||||
When `now > warned_at + 24h`:
|
||||
1. **FS Action:** Delete the physical file at `tracks.path`.
|
||||
2. **DB Action:**
|
||||
* Cascade delete all related records (history, play_counts, etc.).
|
||||
* Remove the `tracks` record.
|
||||
3. **Logging:** Log `feedback(action='deleted_permanent')`.
|
||||
|
||||
## 2. Safety Invariants
|
||||
|
||||
- **No Immediate Deletion:** No user action (other than a "Hard Delete" admin command) can trigger immediate file deletion.
|
||||
- **State Consistency:** A track cannot be in `PENDING_REMOVAL` and `LIBRARY` simultaneously.
|
||||
- **Atomic Deletion:** The file deletion and the database removal must be treated as a single logical unit of work to prevent "Orphaned Files" (files on disk with no DB record) or "Ghost Records" (DB records with no file).
|
||||
|
||||
## 3. User Recovery
|
||||
The "Restore" action is a simple reversal:
|
||||
- `DELETE FROM dislikes WHERE track_id = X;`
|
||||
- `UPDATE tracks SET state = 'LIBRARY' WHERE id = X;`
|
||||
@@ -0,0 +1,57 @@
|
||||
# Worker & Job Specification
|
||||
|
||||
This document defines the background processing architecture using **BullMQ**. Workers are responsible for CPU-intensive tasks and time-sensitive cleanup.
|
||||
|
||||
## 1. Job Architecture
|
||||
|
||||
All jobs are dispatched via the Backend API and processed by specialized Worker containers.
|
||||
|
||||
| Job Name | Priority | Responsibility | Trigger |
|
||||
| :--- | :--- | :--- | :--- |
|
||||
| `metadata_refresh` | Medium | Re-scanning files for tag/metadata changes. | Manual (`POST /api/library/reindex`) |
|
||||
| `artwork_download` | Low | Fetching covers from Cover Art Archive/Discogs. | On metadata enrichment or new file. |
|
||||
| `lyrics_download` | Low | Fetching synced lyrics (LRCLib). | On metadata enrichment. |
|
||||
| `recommendation_gen` | Low | Calculating new "Vibe" seeds and batching. | On session start or after playback completion. |
|
||||
| `audio_analysis` | High | Running `essentia` for BPM, Key, Energy. | New file/re-index. |
|
||||
| `filesystem_rescan` | High | Reconciling DB with actual disk state. | Scheduled (Daily/Weekly). |
|
||||
| `cleanup_sweep` | Medium | Managing the Dislike Lifecycle/Deletion. | Scheduled (Hourly). |
|
||||
|
||||
## 2. Detailed Job Workflows
|
||||
|
||||
### **A. Audio Analysis Pipeline (`audio_analysis`)**
|
||||
This is the most resource-intensive job.
|
||||
1. **Input:** `track_id`.
|
||||
2. **Process:**
|
||||
* Spin up `essentia` subprocess.
|
||||
* Extract BPM, Key, Energy, and Melodic features.
|
||||
3. **Output:** Update `track_audio_features` table and mark `audio_features_ready = true`.
|
||||
|
||||
### **B. Metadata Enrichment Pipeline (`metadata_refresh` / `artwork_download`)**
|
||||
Triggered when a new file is detected or a re-index occurs.
|
||||
1. **Input:** `track_id`.
|
||||
2. **Process:**
|
||||
* Lookup `mbid` via MusicBrainz.
|
||||
* Fetch lyrics via LRCLib.
|
||||
* Fetch artwork via Cover Art Archive.
|
||||
3. **Output:** Update `tracks`, `artists`, and `albums` tables.
|
||||
|
||||
### **C. The "Consistency" Worker (`filesystem_rescan`)**
|
||||
Ensures the database is an accurate reflection of the disk.
|
||||
1. **Process:**
|
||||
* Walk through `/mnt/hdd1/media/Music`.
|
||||
* Compare `mtime` and `size` against DB.
|
||||
* **Action:** If a file is missing, set `track.state = 'MISSING'`. If a new file is found, trigger `metadata_refresh`.
|
||||
|
||||
### **D. The "Cleanup" Worker (`cleanup_sweep`)**
|
||||
Handles the temporal logic of the dislike lifecycle.
|
||||
1. **Process:**
|
||||
* Check `dislikes` where `state = 'warned'` and `warned_at < now - 24h`.
|
||||
* Trigger physical file deletion and DB row removal.
|
||||
* Check `dislikes` where `state = 'hidden'` and `disliked_at < now - 48h`.
|
||||
* Trigger `ntfy` notification.
|
||||
|
||||
## 3. Error Handling & Retries
|
||||
|
||||
- **Exponential Backoff:** All external API jobs (MusicBrainz, etc.) must use exponential backoff to respect rate limits.
|
||||
- **Dead Letter Queue (DLQ):** Jobs that fail after 5 retries are moved to a DLQ for manual inspection via the Admin Dashboard.
|
||||
- **Idempotency:** All jobs must be idempotent. Running `audio_analysis` twice on the same `track_id` must not create duplicate data or errors.
|
||||
@@ -0,0 +1,63 @@
|
||||
# Data Model Specification
|
||||
|
||||
This document defines the data schema across PostgreSQL (Primary), Typesense (Search), and Redis (Caching/Queueing).
|
||||
|
||||
## 1. Relational Schema (PostgreSQL)
|
||||
|
||||
### **Core Tables**
|
||||
|
||||
#### `tracks`
|
||||
* `id` (UUID, PK)
|
||||
* `path` (TEXT, UNIQUE) - Physical disk path.
|
||||
* `hash` (TEXT, INDEX) - BLOB/MD5 hash for deduplication.
|
||||
* `title`, `artist`, `album` (TEXT)
|
||||
* `duration` (REAL) - In seconds.
|
||||
* `state` (ENUM) - `[LIBRARY, RECOMMENDED, HIDDEN, MISSING, DELETED]`
|
||||
* `play_count`, `skip_count`, `dislike_count` (INTEGER)
|
||||
* `last_played_at` (TIMESTAMP)
|
||||
* `mtime` (REAL) - File mtime at last index.
|
||||
* `source_type` (ENUM) - `[MANUAL, RECOMMENDATION]`
|
||||
|
||||
#### `artists` & `albums`
|
||||
* `artists`: `id`, `name`, `mbid`, `discogs_id`, `image_path`.
|
||||
* `albums`: `id`, `artist_id`, `title`, `year`, `artwork_id`.
|
||||
|
||||
#### `genre` & `track_genre`
|
||||
* `genre`: `id`, `name`, `parent_id` (Self-join for hierarchy).
|
||||
* `track_genre`: `track_id`, `genre_id`, `weight` (Decimal).
|
||||
|
||||
### **Recommendation & Lifecycle Tables**
|
||||
|
||||
#### `recommendation_batch`
|
||||
* `id` (UUID, PK)
|
||||
* `user_id` (UUID)
|
||||
* `status` (ENUM) - `[ACTIVE, RESOLVED, FAILED]`
|
||||
* `last_interaction_at` (TIMESTAMP)
|
||||
* `seed_track_id` (UUID, FK)
|
||||
|
||||
#### `dislikes`
|
||||
* `track_id` (UUID, FK)
|
||||
* `disliked_at` (TIMESTAMP)
|
||||
* `warned_at` (TIMESTAMP, NULLABLE)
|
||||
* `state` (ENUM) - `[HIDDEN, WARNED, DELETED]`
|
||||
|
||||
### **Metadata & Enrichment**
|
||||
* `track_audio_features`: `track_id`, `bpm`, `key`, `energy`, `danceability`, etc.
|
||||
* `track_lyrics`: `track_id`, `lyrics_text`, `provider`.
|
||||
* `mb_cache` / `lastfm_cache`: Key-Value stores for external API responses.
|
||||
|
||||
## 2. Search Schema (Typesense)
|
||||
|
||||
Typesense is used for ultra-fast, fuzzy search. Indices are rebuilt from PostgreSQL.
|
||||
|
||||
**Index: `tracks`**
|
||||
* `title` (string, facet)
|
||||
* `artist` (string, facet)
|
||||
* `album` (string, facet)
|
||||
* `genres` (string, facet)
|
||||
* `state` (string, filterable)
|
||||
|
||||
## 3. Cache & Queue (Redis)
|
||||
|
||||
* **BullMQ:** Stores job payloads and processing states.
|
||||
* **Session Cache:** Stores ephemeral playback metadata and current "rolling window" track IDs.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,797 @@
|
||||
# v2 Fix Plan — foolproof execution
|
||||
|
||||
This plan fixes the gaps between the overnight v2 work and
|
||||
`docs/architecture/09-recommendation-and-identity-v2.md`. Every step
|
||||
has exact file paths, exact old/new code, and a verification command.
|
||||
Execute steps 1–8 in order, then step 9 (build + deploy + verify).
|
||||
|
||||
**Rules for the executing agent:**
|
||||
|
||||
- Do NOT edit or remove existing entries in the `MIGRATIONS` array in
|
||||
`db.service.ts`. The three v2 migrations (`20260707_claim_fusion`,
|
||||
`20260707_backfill_claims`, `20260708_materialize_claim_fusion`) have
|
||||
not applied to the live DB yet, but leave them as-is — they run
|
||||
cleanly on first boot.
|
||||
- Do NOT delete the v1 `getNextVibeChunk` CTE or `vibe.routes.ts` in
|
||||
this plan. The doc says v1 is deleted *when D ships and is
|
||||
verified*. That's a follow-up, not this plan.
|
||||
- After all code edits (steps 1–7), run `npx tsc --noEmit` and
|
||||
`npx vitest run` from `backend/` before deploying.
|
||||
- All file paths are relative to `/home/kami/apps/muzick/`.
|
||||
|
||||
---
|
||||
|
||||
## Step 1 — claim_fusion MV refresh consumer (blocker)
|
||||
|
||||
**Problem:** `claim_fusion` is a MATERIALIZED VIEW. It is populated at
|
||||
creation time (migration) and never refreshed again. A trigger fires
|
||||
`NOTIFY claim_fusion_changed` on claims changes, but nothing LISTENs.
|
||||
Every generator and compat view reads the frozen MV — new claims are
|
||||
invisible.
|
||||
|
||||
**Fix:** Add a `refreshClaimFusion()` method to `DbService` and a
|
||||
background interval in `app.ts` that calls it every 10 seconds.
|
||||
`CONCURRENTLY` won't block reads.
|
||||
|
||||
### 1a. Add method to `backend/src/services/db.service.ts`
|
||||
|
||||
Insert this method immediately before the closing `}` of the class
|
||||
(after `seedDefaultDiversityBudgets`, which ends at line 2062):
|
||||
|
||||
```ts
|
||||
|
||||
/**
|
||||
* Refresh the claim_fusion materialised view. Called on a periodic
|
||||
* timer so the graph's read path stays current with new claims.
|
||||
* CONCURRENTLY requires the unique index (idx_claim_fusion_pk),
|
||||
* which the 20260708_materialize_claim_fusion migration creates.
|
||||
*/
|
||||
async refreshClaimFusion(): Promise<void> {
|
||||
try {
|
||||
await this.pgClient.query('SELECT refresh_claim_fusion()');
|
||||
} catch (err) {
|
||||
// Non-fatal: the MV may not exist yet on first boot before
|
||||
// migrations run. Log and move on; the next tick will retry.
|
||||
console.error('[DB] refresh_claim_fusion failed:', err);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The `oldString` to match for the edit (the end of
|
||||
`seedDefaultDiversityBudgets` + the class closing brace):
|
||||
|
||||
```
|
||||
for (const d of defaults) {
|
||||
await this.upsertDiversityBudget({
|
||||
user_id: userId,
|
||||
dimension: d.dimension,
|
||||
budget_share: d.share,
|
||||
horizon_min: d.horizon,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Replace with the same block + the new method inserted before the
|
||||
final `}`.
|
||||
|
||||
### 1b. Start the refresh interval in `backend/src/app.ts`
|
||||
|
||||
After the line `await dbService.runMigrations();` (line 53), add:
|
||||
|
||||
```ts
|
||||
|
||||
// Keep the claim_fusion materialised view fresh. The trigger on
|
||||
// `claims` fires NOTIFY on every change; rather than maintain a
|
||||
// LISTEN consumer (separate long-lived connection), we refresh on a
|
||||
// short interval. 10s staleness is well below any user-facing
|
||||
// latency for a homelab music player.
|
||||
const FUSION_REFRESH_MS = 10_000;
|
||||
const fusionTimer = setInterval(() => {
|
||||
dbService.refreshClaimFusion().catch(() => {});
|
||||
}, FUSION_REFRESH_MS);
|
||||
```
|
||||
|
||||
Then in the `onClose` hook (around line 128), add `clearInterval` for
|
||||
the new timer. Find:
|
||||
|
||||
```ts
|
||||
fastify.addHook('onClose', async () => {
|
||||
try {
|
||||
await pgClient.end();
|
||||
```
|
||||
|
||||
Insert before `await pgClient.end();`:
|
||||
|
||||
```ts
|
||||
clearInterval(fusionTimer);
|
||||
```
|
||||
|
||||
**Verify:** `npx tsc --noEmit` in `backend/` — 0 errors.
|
||||
|
||||
---
|
||||
|
||||
## Step 2 — daily belief decay + nightly forgotten derivation (blocker)
|
||||
|
||||
**Problem:** `listener_beliefs.last_decayed_at` is set at insert and
|
||||
never advanced. No decay job exists. This violates the core axiom
|
||||
"everything decays unless reinforced" and reproduces the v1 failure
|
||||
mode (heavily-played artists win forever). Also, the `forgotten`
|
||||
profile is "derived nightly" per the doc but nothing populates it, so
|
||||
the revival generator always returns empty.
|
||||
|
||||
**Fix:** Add `decayBeliefs()` and `deriveForgottenProfile()` methods
|
||||
to `DbService` and periodic intervals in `app.ts`.
|
||||
|
||||
### 2a. Add decay method to `backend/src/services/db.service.ts`
|
||||
|
||||
Insert after `refreshClaimFusion()` (the method added in step 1a):
|
||||
|
||||
```ts
|
||||
|
||||
/**
|
||||
* Decay all listener beliefs whose last_decayed_at is older than 1
|
||||
* hour. Implements the decay formula from spec §B.4:
|
||||
* value *= 0.5 ^ (elapsed / halflife)
|
||||
* confidence *= 0.5 ^ (elapsed / halflife)
|
||||
* Halflife is per-profile (longterm=365d, obsession=14d, discovery=30d,
|
||||
* negative=180d, contextual=7d). The 'forgotten' profile is excluded
|
||||
* — it is fully derived nightly by deriveForgottenProfile(), not
|
||||
* decayed.
|
||||
*/
|
||||
async decayBeliefs(): Promise<number> {
|
||||
const res = await this.pgClient.query(`
|
||||
WITH halflives AS (
|
||||
SELECT profile,
|
||||
CASE profile
|
||||
WHEN 'longterm' THEN 365 * 86400
|
||||
WHEN 'obsession' THEN 14 * 86400
|
||||
WHEN 'discovery' THEN 30 * 86400
|
||||
WHEN 'negative' THEN 180 * 86400
|
||||
WHEN 'contextual' THEN 7 * 86400
|
||||
ELSE 30 * 86400
|
||||
END AS halflife_sec
|
||||
)
|
||||
UPDATE listener_beliefs lb
|
||||
SET value = GREATEST(-1.0, LEAST(1.0, lb.value * POWER(0.5,
|
||||
EXTRACT(EPOCH FROM (NOW() - lb.last_decayed_at)) / h.halflife_sec))),
|
||||
confidence = GREATEST(0, LEAST(1.0, lb.confidence * POWER(0.5,
|
||||
EXTRACT(EPOCH FROM (NOW() - lb.last_decayed_at)) / h.halflife_sec))),
|
||||
last_decayed_at = NOW()
|
||||
FROM halflives h
|
||||
WHERE lb.profile = h.profile
|
||||
AND lb.profile <> 'forgotten'
|
||||
AND lb.last_decayed_at < NOW() - INTERVAL '1 hour'
|
||||
`);
|
||||
return res.rowCount ?? 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* Derive the 'forgotten' profile nightly (spec §B.2):
|
||||
* longterm affinity > 0.3 AND not reinforced in 90+ days.
|
||||
* Wipes and repopulates — 'forgotten' is fully derived, not evidence-fed.
|
||||
*/
|
||||
async deriveForgottenProfile(): Promise<number> {
|
||||
await this.pgClient.query(
|
||||
`DELETE FROM listener_beliefs WHERE profile = 'forgotten'`
|
||||
);
|
||||
const res = await this.pgClient.query(`
|
||||
INSERT INTO listener_beliefs
|
||||
(user_id, profile, entity_type, entity_id, dimension, value,
|
||||
confidence, evidence_count, last_reinforced_at, last_decayed_at)
|
||||
SELECT user_id, 'forgotten', entity_type, entity_id, dimension,
|
||||
value, confidence, evidence_count, last_reinforced_at, NOW()
|
||||
FROM listener_beliefs
|
||||
WHERE profile = 'longterm'
|
||||
AND dimension = 'affinity'
|
||||
AND value > 0.3
|
||||
AND last_reinforced_at < NOW() - INTERVAL '90 days'
|
||||
ON CONFLICT (user_id, profile, entity_type, entity_id, dimension)
|
||||
DO UPDATE SET
|
||||
value = EXCLUDED.value,
|
||||
confidence = EXCLUDED.confidence,
|
||||
evidence_count = EXCLUDED.evidence_count,
|
||||
last_reinforced_at = EXCLUDED.last_reinforced_at
|
||||
`);
|
||||
return res.rowCount ?? 0;
|
||||
}
|
||||
```
|
||||
|
||||
### 2b. Start decay + forgotten intervals in `backend/src/app.ts`
|
||||
|
||||
After the `fusionTimer` block added in step 1b, add:
|
||||
|
||||
```ts
|
||||
|
||||
// Daily belief decay (spec §B.4). Runs hourly; the SQL only touches
|
||||
// beliefs whose last_decayed_at is >1h old, so frequent runs are safe.
|
||||
const DECAY_INTERVAL_MS = 60 * 60 * 1000;
|
||||
const decayTimer = setInterval(() => {
|
||||
dbService.decayBeliefs().catch((e) => console.error('[DB] belief decay failed:', e));
|
||||
}, DECAY_INTERVAL_MS);
|
||||
|
||||
// Nightly 'forgotten' profile derivation (spec §B.2).
|
||||
const FORGOTTEN_INTERVAL_MS = 24 * 60 * 60 * 1000;
|
||||
const forgottenTimer = setInterval(() => {
|
||||
dbService.deriveForgottenProfile().catch((e) =>
|
||||
console.error('[DB] forgotten derivation failed:', e)
|
||||
);
|
||||
}, FORGOTTEN_INTERVAL_MS);
|
||||
|
||||
// Run both once at boot so the first session benefits.
|
||||
dbService.decayBeliefs().catch(() => {});
|
||||
dbService.deriveForgottenProfile().catch(() => {});
|
||||
```
|
||||
|
||||
In the `onClose` hook, add (after `clearInterval(fusionTimer);`):
|
||||
|
||||
```ts
|
||||
clearInterval(decayTimer);
|
||||
clearInterval(forgottenTimer);
|
||||
```
|
||||
|
||||
**Verify:** `npx tsc --noEmit` in `backend/` — 0 errors.
|
||||
|
||||
---
|
||||
|
||||
## Step 3 — fix MB spine writer generated-column bug
|
||||
|
||||
**File:** `workers/src/mb-spine-writer.ts`
|
||||
|
||||
**Problem:** `resolveArtist` (line 128) tries to INSERT into
|
||||
`normalized_name`, which is `GENERATED ALWAYS AS normalize_artist(name)
|
||||
STORED`. PostgreSQL rejects this:
|
||||
`ERROR: cannot insert a non-DEFAULT value into column "normalized_name"`.
|
||||
The stub-creation path is broken — the spine writer can only attach
|
||||
claims to *existing* artists; any newly-credited artist is dropped.
|
||||
|
||||
**Fix:** Remove `normalized_name` from the INSERT column list and the
|
||||
`normalized` value from the params. The generated column auto-computes
|
||||
from `name`.
|
||||
|
||||
Find (lines 127–134):
|
||||
|
||||
```ts
|
||||
const result = await this.pgClient.query<{ id: string }>(
|
||||
`INSERT INTO artists (name, canonical_name, sort_name, mbid, normalized_name)
|
||||
VALUES ($1, $2, $3, $4, $5)
|
||||
ON CONFLICT (mbid) DO UPDATE SET name = EXCLUDED.name, updated_at = CURRENT_TIMESTAMP
|
||||
RETURNING id`,
|
||||
[creditName, artistName, sortName, mbid, normalized]
|
||||
);
|
||||
```
|
||||
|
||||
Replace with:
|
||||
|
||||
```ts
|
||||
const result = await this.pgClient.query<{ id: string }>(
|
||||
`INSERT INTO artists (name, canonical_name, sort_name, mbid)
|
||||
VALUES ($1, $2, $3, $4)
|
||||
ON CONFLICT (mbid) DO UPDATE SET name = EXCLUDED.name, updated_at = CURRENT_TIMESTAMP
|
||||
RETURNING id`,
|
||||
[creditName, artistName, sortName, mbid]
|
||||
);
|
||||
```
|
||||
|
||||
**Verify:** `npx tsc --noEmit` in `workers/` — 0 errors.
|
||||
|
||||
---
|
||||
|
||||
## Step 4 — fix dislikeTrack evidence-in-catch bug
|
||||
|
||||
**File:** `backend/src/services/db.service.ts`
|
||||
|
||||
**Problem:** `dislikeTrack` (line 712) writes the `hidden` evidence
|
||||
row in the `catch` block — i.e. only when the transaction **fails**.
|
||||
A successful dislike writes no negative evidence via this path.
|
||||
|
||||
**Fix:** Move the evidence write out of the catch block to after the
|
||||
try/catch, so it runs only on success.
|
||||
|
||||
Find (lines 712–748):
|
||||
|
||||
```ts
|
||||
async dislikeTrack(userId: string, trackId: string): Promise<void> {
|
||||
try {
|
||||
await this.pgClient.query('BEGIN');
|
||||
|
||||
// Phase 1: hide the track in all active views
|
||||
await this.pgClient.query(
|
||||
"UPDATE tracks SET state = 'HIDDEN' WHERE id = $1 AND state = 'LIBRARY'",
|
||||
[trackId]
|
||||
);
|
||||
|
||||
// Phase 1: insert dislike row (idempotent — won't create duplicate)
|
||||
await this.pgClient.query(
|
||||
'INSERT INTO dislikes (track_id) VALUES ($1) ON CONFLICT (track_id) DO NOTHING',
|
||||
[trackId]
|
||||
);
|
||||
|
||||
// Phase 1: log feedback signal for the Vibe learning loop
|
||||
await this.pgClient.query(
|
||||
"INSERT INTO feedback (user_id, track_id, action) VALUES ($1, $2, 'disliked')",
|
||||
[userId, trackId]
|
||||
);
|
||||
|
||||
await this.pgClient.query('COMMIT');
|
||||
} catch (err) {
|
||||
await this.pgClient.query('ROLLBACK');
|
||||
// Write evidence: hidden → negative profile
|
||||
await this.recordEvidence({
|
||||
user_id: userId,
|
||||
entity_type: 'track',
|
||||
entity_id: trackId,
|
||||
signal: 'hidden',
|
||||
profile: 'negative',
|
||||
weight: -0.60,
|
||||
});
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Replace with:
|
||||
|
||||
```ts
|
||||
async dislikeTrack(userId: string, trackId: string): Promise<void> {
|
||||
try {
|
||||
await this.pgClient.query('BEGIN');
|
||||
|
||||
// Phase 1: hide the track in all active views
|
||||
await this.pgClient.query(
|
||||
"UPDATE tracks SET state = 'HIDDEN' WHERE id = $1 AND state = 'LIBRARY'",
|
||||
[trackId]
|
||||
);
|
||||
|
||||
// Phase 1: insert dislike row (idempotent — won't create duplicate)
|
||||
await this.pgClient.query(
|
||||
'INSERT INTO dislikes (track_id) VALUES ($1) ON CONFLICT (track_id) DO NOTHING',
|
||||
[trackId]
|
||||
);
|
||||
|
||||
// Phase 1: log feedback signal for the Vibe learning loop
|
||||
await this.pgClient.query(
|
||||
"INSERT INTO feedback (user_id, track_id, action) VALUES ($1, $2, 'disliked')",
|
||||
[userId, trackId]
|
||||
);
|
||||
|
||||
await this.pgClient.query('COMMIT');
|
||||
} catch (err) {
|
||||
await this.pgClient.query('ROLLBACK');
|
||||
throw err;
|
||||
}
|
||||
|
||||
// Write evidence: hidden → negative profile (only on success)
|
||||
await this.recordEvidence({
|
||||
user_id: userId,
|
||||
entity_type: 'track',
|
||||
entity_id: trackId,
|
||||
signal: 'hidden',
|
||||
profile: 'negative',
|
||||
weight: -0.60,
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
**Verify:** `npx tsc --noEmit` in `backend/` — 0 errors.
|
||||
|
||||
---
|
||||
|
||||
## Step 5 — fix hardDeleteTrack missing manual_deleted evidence
|
||||
|
||||
**File:** `backend/src/services/db.service.ts`
|
||||
|
||||
**Problem:** `hardDeleteTrack` (line 810) writes a legacy `feedback`
|
||||
row but no evidence. The doc's strongest negative signal
|
||||
(`manual_deleted → negative -0.90`) is missing. Permanent deletion has
|
||||
no effect on listener beliefs.
|
||||
|
||||
**Fix:** Add a `recordEvidence` call between the feedback insert and
|
||||
the track DELETE. `evidence.entity_id` has no FK to `tracks`, so the
|
||||
evidence row survives the deletion.
|
||||
|
||||
Find (lines 813–821):
|
||||
|
||||
```ts
|
||||
// Log the permanent deletion feedback event first (before the track is gone)
|
||||
await this.pgClient.query(
|
||||
"INSERT INTO feedback (user_id, track_id, action) VALUES ($1, $2, 'deleted_permanent')",
|
||||
[userId, trackId]
|
||||
);
|
||||
|
||||
// Delete DB record (ON DELETE CASCADE handles track_genre, play_history,
|
||||
// feedback, track_audio_features, track_lyrics, recommendation_batch_track)
|
||||
await this.pgClient.query('DELETE FROM tracks WHERE id = $1', [trackId]);
|
||||
```
|
||||
|
||||
Replace with:
|
||||
|
||||
```ts
|
||||
// Log the permanent deletion feedback event first (before the track is gone)
|
||||
await this.pgClient.query(
|
||||
"INSERT INTO feedback (user_id, track_id, action) VALUES ($1, $2, 'deleted_permanent')",
|
||||
[userId, trackId]
|
||||
);
|
||||
|
||||
// Write evidence: manual_deleted → negative profile (strongest negative
|
||||
// signal, spec §B.3). entity_id has no FK to tracks, so the row survives.
|
||||
await this.recordEvidence({
|
||||
user_id: userId,
|
||||
entity_type: 'track',
|
||||
entity_id: trackId,
|
||||
signal: 'manual_deleted',
|
||||
profile: 'negative',
|
||||
weight: -0.90,
|
||||
});
|
||||
|
||||
// Delete DB record (ON DELETE CASCADE handles track_genre, play_history,
|
||||
// feedback, track_audio_features, track_lyrics, recommendation_batch_track)
|
||||
await this.pgClient.query('DELETE FROM tracks WHERE id = $1', [trackId]);
|
||||
```
|
||||
|
||||
**Verify:** `npx tsc --noEmit` in `backend/` — 0 errors.
|
||||
|
||||
---
|
||||
|
||||
## Step 6 — switch session director artist reads to track_artists_v2
|
||||
|
||||
**File:** `backend/src/services/session-director.service.ts`
|
||||
|
||||
**Problem:** The session director reads the legacy `track_artists`
|
||||
table for artist fatigue, budget spend, repetition checks, seed
|
||||
resolution, and recent-play artist lookup. Doc D.2 requires artist
|
||||
fatigue to roll up via `alias_of` fusion (so DOOM / Madvillain / Viktor
|
||||
Vaughn collapse into one artist). By bypassing `claim_fusion` /
|
||||
`track_artists_v2`, the alias collapse cannot happen.
|
||||
|
||||
**Fix:** Replace all `track_artists ta` → `track_artists_v2 ta` and
|
||||
`track_artists ta3` → `track_artists_v2 ta3` in this file. The v2 view
|
||||
has the same `track_id`, `artist_id`, `role` columns, so it's a
|
||||
drop-in replacement. This depends on step 1 (MV refresh) being
|
||||
deployed so the view has data.
|
||||
|
||||
There are 11 occurrences across the file. Do two `replaceAll` edits:
|
||||
|
||||
### 6a. Replace all `track_artists ta` with `track_artists_v2 ta`
|
||||
|
||||
Use `replaceAll: true` on the string `track_artists ta` →
|
||||
`track_artists_v2 ta`. This covers 10 occurrences (buildState,
|
||||
computeFatigue, calcBudgetSpent artist, calcBudgetSpent new_artist,
|
||||
checkRepetition, rankCandidates, buildPlan, replan, resolveSeedArtistId).
|
||||
|
||||
### 6b. Replace `track_artists ta3` with `track_artists_v2 ta3`
|
||||
|
||||
Use `replaceAll: true` on the string `track_artists ta3` →
|
||||
`track_artists_v2 ta3`. This covers 1 occurrence in
|
||||
`calcBudgetSpent`'s `new_artist` case.
|
||||
|
||||
**Note:** Do step 6a first, then 6b. After 6a, the `ta3` occurrence
|
||||
will still be `track_artists ta3` (it wasn't matched by `track_artists ta`
|
||||
because that's a different string — `ta3` ≠ `ta`). So both
|
||||
replacements are needed.
|
||||
|
||||
**Verify:**
|
||||
- `npx tsc --noEmit` in `backend/` — 0 errors.
|
||||
- `grep -n "track_artists " backend/src/services/session-director.service.ts`
|
||||
should return **zero** matches (all replaced). If any `track_artists `
|
||||
without `_v2` remain, fix them.
|
||||
|
||||
---
|
||||
|
||||
## Step 7 — correct the session log
|
||||
|
||||
**File:** `SESSION-07-07-2026.md`
|
||||
|
||||
**Problem:** The session log overclaims: says "Full Stack" and
|
||||
"replaces getNextVibeChunk" but nothing is deployed, v1 is intact, and
|
||||
frontend is not wired. Also miscounts files (7 not 8) and says
|
||||
noveltyGenerator was "skipped" when it's implemented.
|
||||
|
||||
**Fix:** Make these edits:
|
||||
|
||||
### 7a. Fix the headline (line 3)
|
||||
|
||||
Find:
|
||||
```
|
||||
## Implemented: v2 Recommendation Engine — Full Stack (Systems A–E + Phase 4)
|
||||
```
|
||||
Replace with:
|
||||
```
|
||||
## Scaffolded: v2 Recommendation Engine — code complete, not yet deployed (Systems A–E + Phase 4)
|
||||
```
|
||||
|
||||
### 7b. Fix the file count (line 8)
|
||||
|
||||
Find:
|
||||
```
|
||||
### Files created (7 new)
|
||||
```
|
||||
Replace with:
|
||||
```
|
||||
### Files created (8 new)
|
||||
```
|
||||
|
||||
And add a row to the table after the `image-enrichment.service.ts` row
|
||||
(line 15). After:
|
||||
```
|
||||
| `backend/src/services/image-enrichment.service.ts` | 105 | **Phase 4** — Image candidate pipeline |
|
||||
```
|
||||
Add:
|
||||
```
|
||||
| `workers/src/mb-spine-writer.ts` | 136 | **A** — MB artist-credit → claims writer (wired into enrichment.service.ts) |
|
||||
```
|
||||
|
||||
### 7c. Fix the novelty generator claim (line 53)
|
||||
|
||||
Find:
|
||||
```
|
||||
- `noveltyGenerator`: skipped (no release_date column)
|
||||
```
|
||||
Replace with:
|
||||
```
|
||||
- `noveltyGenerator`: recent releases (≤60d) via same_scene_as/same_label_as/produced edges from trusted artists
|
||||
```
|
||||
|
||||
### 7d. Fix the "replaces getNextVibeChunk" claim (line 56)
|
||||
|
||||
Find:
|
||||
```
|
||||
**System D — Session Director (replaces getNextVibeChunk)**
|
||||
```
|
||||
Replace with:
|
||||
```
|
||||
**System D — Session Director (runs alongside v1; getNextVibeChunk not yet deleted)**
|
||||
```
|
||||
|
||||
### 7e. Fix the Verification section (lines 86–88)
|
||||
|
||||
Find:
|
||||
```
|
||||
### Verification
|
||||
- `npx tsc --noEmit` — 0 errors
|
||||
- No git repo — changes uncommitted
|
||||
```
|
||||
Replace with:
|
||||
```
|
||||
### Verification
|
||||
- `npx tsc --noEmit` — 0 errors
|
||||
- `npx vitest run` — 30/30 pass (mocked shape checks, not DB-state)
|
||||
- No git repo — changes uncommitted
|
||||
- NOT deployed: live backend container is pre-v2; `/api/v2/*` and `/api/graph/*` return 404; DB has zero v2 tables. See `docs/architecture/v2-fix-plan.md` for the fix + deploy plan.
|
||||
```
|
||||
|
||||
### 7f. Fix the Next section (lines 89–92)
|
||||
|
||||
Find:
|
||||
```
|
||||
### Next
|
||||
- Wire the v2 endpoint into the frontend Vibe page (replace v1 vibeService calls)
|
||||
- Build yt-dlp worker for System E acquisition (download candidates)
|
||||
- Runtime smoke-test after redeploy
|
||||
```
|
||||
Replace with:
|
||||
```
|
||||
### Next
|
||||
- Execute `docs/architecture/v2-fix-plan.md` (MV refresh, decay job, bug fixes, deploy)
|
||||
- After deploy + verify: wire the v2 endpoint into the frontend Vibe page (replace v1 vibeService calls)
|
||||
- Build yt-dlp worker for System E acquisition (download candidates)
|
||||
- After v2 is verified in production: delete v1 CTE (`getNextVibeChunk`), `vibe.routes.ts`, `feedback` table, `artist_similar` table per the doc's "Retiring v1" list
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 8 — run typecheck + tests before deploying
|
||||
|
||||
```bash
|
||||
cd /home/kami/apps/muzick/backend
|
||||
npx tsc --noEmit
|
||||
npx vitest run
|
||||
```
|
||||
|
||||
Both must pass (0 ts errors, 30/30 tests). If any test fails, do not
|
||||
deploy — re-read the relevant step and fix.
|
||||
|
||||
Also typecheck the worker:
|
||||
|
||||
```bash
|
||||
cd /home/kami/apps/muzick/workers
|
||||
npx tsc --noEmit
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 9 — build, deploy, and verify against the live DB
|
||||
|
||||
### 9a. Rebuild the backend + worker images
|
||||
|
||||
```bash
|
||||
cd /home/kami/apps/muzick
|
||||
docker compose build backend worker
|
||||
docker compose up -d backend worker
|
||||
```
|
||||
|
||||
Wait ~15 seconds for boot, then check the backend logs for migration
|
||||
output:
|
||||
|
||||
```bash
|
||||
docker logs muzick-backend-1 --tail 80 2>&1 | grep -E "Migration|migration|ERROR|error"
|
||||
```
|
||||
|
||||
You should see:
|
||||
```
|
||||
[DB] Running migration: 20260707_claim_fusion
|
||||
[DB] Migration applied: 20260707_claim_fusion
|
||||
[DB] Running migration: 20260707_backfill_claims
|
||||
[DB] Migration applied: 20260707_backfill_claims
|
||||
[DB] Running migration: 20260708_materialize_claim_fusion
|
||||
[DB] Migration applied: 20260708_materialize_claim_fusion
|
||||
```
|
||||
|
||||
If any migration fails, read the error, fix the SQL in a NEW migration
|
||||
(do not edit the failed one), rebuild, and redeploy.
|
||||
|
||||
### 9b. Verify v2 tables + views exist and are populated
|
||||
|
||||
```bash
|
||||
docker exec muzick-db-1 psql -U user -d muzick -c "
|
||||
SELECT 'migrations' AS check, COUNT(*) FROM schema_migrations
|
||||
UNION ALL SELECT 'claims', COUNT(*) FROM claims
|
||||
UNION ALL SELECT 'source_trust', COUNT(*) FROM source_trust
|
||||
UNION ALL SELECT 'evidence', COUNT(*) FROM evidence
|
||||
UNION ALL SELECT 'claim_fusion rows', COUNT(*) FROM claim_fusion
|
||||
UNION ALL SELECT 'track_artists_v2 rows', COUNT(*) FROM track_artists_v2
|
||||
UNION ALL SELECT 'recording_mbid set', COUNT(*) FROM tracks WHERE recording_mbid IS NOT NULL;
|
||||
"
|
||||
```
|
||||
|
||||
Expected after first boot:
|
||||
- `migrations` = 7 (4 old + 3 new)
|
||||
- `source_trust` = 7 (seed rows)
|
||||
- `claims` > 0 (backfilled from `track_artists` + `artist_similar`)
|
||||
- `claim_fusion rows` > 0 (populated by the materialize migration)
|
||||
- `track_artists_v2 rows` > 0 (view over claim_fusion)
|
||||
|
||||
If `claims` = 0, the backfill migration found nothing — check that
|
||||
`track_artists` and `artist_similar` have data in the live DB.
|
||||
|
||||
### 9c. Verify the v2 endpoints are live
|
||||
|
||||
```bash
|
||||
curl -s http://localhost:3000/api/graph/sources | head -c 200
|
||||
echo
|
||||
curl -s -o /dev/null -w "v2/state: HTTP %{http_code}\n" http://localhost:3000/api/v2/state
|
||||
curl -s -o /dev/null -w "graph/sources: HTTP %{http_code}\n" http://localhost:3000/api/graph/sources
|
||||
curl -s -o /dev/null -w "graph/summary: HTTP %{http_code}\n" http://localhost:3000/api/graph/summary
|
||||
```
|
||||
|
||||
Expected: `v2/state: HTTP 200`, `graph/sources: HTTP 200`,
|
||||
`graph/summary: HTTP 200`.
|
||||
|
||||
### 9d. Smoke-test the v2 session flow
|
||||
|
||||
```bash
|
||||
# Start a v2 session (use any library track ID as seed)
|
||||
SEED=$(docker exec muzick-db-1 psql -U user -d muzick -t -c "SELECT id FROM tracks WHERE state='LIBRARY' LIMIT 1" | tr -d ' \n')
|
||||
echo "Seed track: $SEED"
|
||||
curl -s -X POST http://localhost:3000/api/v2/vibe/start \
|
||||
-H 'Content-Type: application/json' \
|
||||
-H "x-user-id: 00000000-0000-0000-0000-000000000000" \
|
||||
-d "{\"seedTrackId\":\"$SEED\"}" | head -c 500
|
||||
echo
|
||||
# Get the next track from the plan
|
||||
curl -s http://localhost:3000/api/v2/vibe/next \
|
||||
-H "x-user-id: 00000000-0000-0000-0000-000000000000" | head -c 300
|
||||
echo
|
||||
# Check the plan
|
||||
curl -s http://localhost:3000/api/v2/vibe/plan \
|
||||
-H "x-user-id: 00000000-0000-0000-0000-000000000000" | head -c 300
|
||||
```
|
||||
|
||||
If `/v2/vibe/start` returns an empty plan `[]`, check the backend logs
|
||||
for generator errors. The most likely cause is `claim_fusion` being
|
||||
empty — verify step 9b showed `claim_fusion rows > 0`.
|
||||
|
||||
### 9e. Verify evidence is written on a completed play
|
||||
|
||||
```bash
|
||||
USER="00000000-0000-0000-0000-000000000000"
|
||||
TRACK=$(docker exec muzick-db-1 psql -U user -d muzick -t -c "SELECT id FROM tracks WHERE state='LIBRARY' LIMIT 1" | tr -d ' \n')
|
||||
|
||||
# Before
|
||||
docker exec muzick-db-1 psql -U user -d muzick -c "SELECT COUNT(*) AS evidence_before FROM evidence WHERE user_id='$USER'"
|
||||
|
||||
# Record a completed play via the v2 feedback endpoint
|
||||
curl -s -X POST http://localhost:3000/api/v2/vibe/feedback \
|
||||
-H 'Content-Type: application/json' \
|
||||
-H "x-user-id: $USER" \
|
||||
-d "{\"trackId\":\"$TRACK\",\"action\":\"completed\"}"
|
||||
|
||||
# After
|
||||
docker exec muzick-db-1 psql -U user -d muzick -c "SELECT COUNT(*) AS evidence_after, signal, profile, weight FROM evidence WHERE user_id='$USER' GROUP BY signal, profile, weight ORDER BY created_at DESC LIMIT 5"
|
||||
```
|
||||
|
||||
Expected: `evidence_after > evidence_before`, and you should see a
|
||||
`playback_completed` / `longterm` / `0.10` row.
|
||||
|
||||
### 9f. Verify the MV refresh is running
|
||||
|
||||
Wait 15 seconds after boot, then:
|
||||
|
||||
```bash
|
||||
docker logs muzick-backend-1 2>&1 | grep -i "refresh_claim_fusion" | tail -3
|
||||
```
|
||||
|
||||
You should see no errors (the method logs only on failure). If you see
|
||||
repeated `refresh_claim_fusion failed` errors, the MV or refresh
|
||||
function doesn't exist — re-check that the
|
||||
`20260708_materialize_claim_fusion` migration applied.
|
||||
|
||||
### 9g. Verify belief decay runs
|
||||
|
||||
```bash
|
||||
# Manually trigger decay and check it doesn't error
|
||||
docker exec muzick-backend-1 node -e "
|
||||
const { Client } = require('pg');
|
||||
const c = new Client({ connectionString: process.env.DATABASE_URL });
|
||||
(async () => {
|
||||
await c.connect();
|
||||
const r = await c.query('SELECT decay_beliefs()');
|
||||
console.log('decay result:', r.rows);
|
||||
await c.end();
|
||||
})().catch(e => { console.error('FAIL:', e.message); process.exit(1); });
|
||||
" 2>&1 || echo "decay_beliefs() not a SQL function — that's OK, the method runs the UPDATE directly"
|
||||
```
|
||||
|
||||
This is a soft check — the `decayBeliefs()` method runs raw SQL, not a
|
||||
stored function. The real verification is that `last_decayed_at`
|
||||
advances after 1 hour. Check:
|
||||
|
||||
```bash
|
||||
docker exec muzick-db-1 psql -U user -d muzick -c "
|
||||
SELECT user_id, profile, entity_type, last_decayed_at,
|
||||
EXTRACT(EPOCH FROM (NOW() - last_decayed_at))/3600 AS hours_since_decay
|
||||
FROM listener_beliefs
|
||||
ORDER BY last_decayed_at DESC LIMIT 5;
|
||||
"
|
||||
```
|
||||
|
||||
After 1+ hours of uptime, `hours_since_decay` should be < 1 for
|
||||
recently-decayed rows (the hourly job touched them).
|
||||
|
||||
---
|
||||
|
||||
## Summary of what each step fixes
|
||||
|
||||
| Step | Doc section | Problem | Fix |
|
||||
|---|---|---|---|
|
||||
| 1 | A.4 | `claim_fusion` MV never refreshed | 10s interval calls `refresh_claim_fusion()` |
|
||||
| 2 | B.4, B.2 | No belief decay; `forgotten` never derived | Hourly decay job + 24h forgotten derivation |
|
||||
| 3 | A.5 #1 | MB spine writer can't create new artists (generated column) | Drop `normalized_name` from INSERT |
|
||||
| 4 | B.3 | `dislikeTrack` writes evidence only on failure | Move evidence write to success path |
|
||||
| 5 | B.3 | `hardDeleteTrack` writes no `manual_deleted` evidence | Add `-0.90` evidence before track DELETE |
|
||||
| 6 | D.2 | Director reads legacy `track_artists`, bypassing alias fusion | Switch to `track_artists_v2` |
|
||||
| 7 | — | Session log overclaims | Correct the wording |
|
||||
| 8 | — | Pre-deploy gate | tsc + vitest pass |
|
||||
| 9 | A.7, B.6, D.11 | Not deployed; acceptance unverified | Build, deploy, verify against live DB |
|
||||
|
||||
## What is NOT in this plan (follow-ups, not blockers)
|
||||
|
||||
- **MB spine writer album + artist-relation claims** —
|
||||
`writeAlbumClaims` and `writeArtistRelationClaims` in
|
||||
`mb-spine-writer.ts` are stubs (`console.log` + `return 0`). They
|
||||
require new `MusicBrainzClient` methods (release-group artist-credit,
|
||||
artist-relations ARs). Not a blocker for first deploy; the recording-
|
||||
claim writer is the critical path. Implement as a follow-up.
|
||||
- **Frontend wiring** — the frontend still calls `/api/vibe/*` (v1).
|
||||
After v2 is verified in production, wire `/api/v2/vibe/*` into
|
||||
`frontend/src/services/vibeService.ts` and the Vibe page.
|
||||
- **v1 deletion** — `getNextVibeChunk`, `vibe.routes.ts`, the
|
||||
`feedback` table, `artist_similar` table, `genre.parent_id` are all
|
||||
still present. The doc says delete them when D ships and is
|
||||
verified. That's a separate, careful release after v2 is confirmed
|
||||
good in production.
|
||||
- **LISTEN-based MV refresh** — the 10s interval in step 1 is the
|
||||
simple, foolproof approach. Upgrading to a `LISTEN`/`NOTIFY` consumer
|
||||
(immediate refresh on claim change) is a follow-up if 10s staleness
|
||||
ever becomes a problem.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,139 @@
|
||||
# UI Rework Plan
|
||||
|
||||
> Status: **planned, not started.** This document captures the target direction for a
|
||||
> richer player UI (reference: the "LocalTunes" three-pane mockup) and what it implies for
|
||||
> both the frontend and the backend. It is a plan to execute later, not a description of the
|
||||
> current app.
|
||||
|
||||
## 1. Vision
|
||||
|
||||
Move from the current functional-but-plain single-content-column layout to a polished,
|
||||
artwork-forward **three-pane music player** in the spirit of modern desktop players
|
||||
(Spotify / Apple Music / the LocalTunes reference):
|
||||
|
||||
- **Left:** persistent navigation rail (library sections + a personal/"your music" group).
|
||||
- **Center:** scrollable content (Home, Library, Vibe, etc.) — artwork-rich cards, horizontal
|
||||
carousels, hover-to-play.
|
||||
- **Right:** persistent **Now Playing** panel — large artwork, track info, transport, and an
|
||||
**Up Next / queue** list.
|
||||
- **Bottom:** full-width global **playback bar** (shuffle / prev / play / next / repeat,
|
||||
scrubber, volume, queue toggle) that's always visible regardless of route.
|
||||
- **Top:** global search field + (future) user/account menu.
|
||||
|
||||
Accent-driven, rounded, soft-gradient cards; dark by default but fully themeable via tokens.
|
||||
|
||||
## 2. Layout structure
|
||||
|
||||
```
|
||||
┌────────────────────────────────────────────────────────────────────────────┐
|
||||
│ Top bar: [logo] [ global search ⌘K ] [bell] [avatar ▾] │
|
||||
├───────────────┬────────────────────────────────────────────┬───────────────┤
|
||||
│ Nav rail │ Content (router Outlet) │ Now Playing │
|
||||
│ - Home │ Good evening 👋 │ [ artwork ] │
|
||||
│ - Songs │ Quick Access cards │ Title/Artist │
|
||||
│ - Albums │ Recently Played (carousel, View all) │ scrubber │
|
||||
│ - Artists │ Made for you (mixes carousel) │ transport │
|
||||
│ - Genres │ ... │ Up Next list │
|
||||
│ - Playlists │ │ │
|
||||
│ - Folder │ │ (collapsible)│
|
||||
│ ────────── │ │ │
|
||||
│ Now Playing │ │ │
|
||||
│ Recently … │ │ │
|
||||
│ Most Played │ │ │
|
||||
│ Favorites │ │ │
|
||||
│ ────────── │ │ │
|
||||
│ Settings │ │ │
|
||||
│ Theme │ │ │
|
||||
│ About │ │ │
|
||||
├───────────────┴────────────────────────────────────────────┴───────────────┤
|
||||
│ Bottom bar: [art] Title/Artist ♥ ⇄ ◀ ▶▶ ⏯ ▶▶ ↻ 🔊────── queue ▤ │
|
||||
└──────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
The right Now-Playing panel and the bottom bar are partly redundant by design (desktop
|
||||
players do this): the bottom bar is the always-on minimal transport; the right panel is the
|
||||
expanded view with queue and large art, and is collapsible.
|
||||
|
||||
## 3. Design tokens / theming
|
||||
|
||||
This rework is the right moment to finish theming. Today only the shell consumes tokens
|
||||
(`--bg`, `--surface`, `--text`, `--accent` from `src/lib/theme.ts`). Target:
|
||||
|
||||
- **Expand the token set:** `--bg`, `--bg-elevated`, `--surface`, `--surface-hover`,
|
||||
`--border`, `--text`, `--text-muted`, `--accent`, `--accent-hover`, `--on-accent`,
|
||||
plus gradient stops for cards (`--card-grad-a/b`).
|
||||
- **Drive Tailwind from the tokens:** extend `tailwind.config.js` `theme.colors` to reference
|
||||
the CSS variables (e.g. `bg: 'var(--bg)'`, `surface: 'var(--surface)'`, `accent:
|
||||
'var(--accent)'`) so components use semantic classes (`bg-surface`, `text-muted`,
|
||||
`bg-accent`) instead of hard-coded `bg-zinc-900` etc. This makes every component themeable
|
||||
without per-component edits.
|
||||
- Keep the existing presets (Dark / Midnight / Forest / Plum), add a light option, and keep
|
||||
`initTheme()` applying the persisted choice before first paint.
|
||||
- The reference's purple accent → add a "Default (Purple)" preset.
|
||||
|
||||
## 4. Component inventory (new / reworked)
|
||||
|
||||
| Component | Purpose |
|
||||
| :--- | :--- |
|
||||
| `AppShell` | 3-pane grid (rail / content / now-playing) + top bar + bottom bar. Replaces `Layout`. |
|
||||
| `NavRail` | Sections + personal group + settings group; active state via `--accent`. |
|
||||
| `TopBar` | Global search (debounced, ⌘K focus), account menu (stub until auth). |
|
||||
| `NowPlayingPanel` | Right rail: large art, info, scrubber, transport, Up Next queue (reorder/remove). Collapsible. |
|
||||
| `PlaybackBar` | Bottom global transport (always visible). Reworks `NowPlayingBar`. |
|
||||
| `MediaCard` | Square artwork card with hover play overlay (used by carousels + grids). |
|
||||
| `Carousel` / `ShelfRow` | Horizontal scroll row with title + "View all". |
|
||||
| `QuickAccessCard` | Wide gradient card (Favorites / Recently Added / Most Played / Folder). |
|
||||
| `TrackRow` | Reusable list row (replaces the per-page `LibraryTrackRow`) with art, actions, now-playing highlight. |
|
||||
| `Artwork` | Resolves album/track artwork URL with a graceful gradient placeholder fallback. |
|
||||
|
||||
State: keep Zustand `usePlaybackStore` (current/queue/isPlaying/position/volume) and
|
||||
`useVibeStore`; add a small `useUiStore` for panel collapse + theme if useful. The Up Next
|
||||
list is just the playback `queue`.
|
||||
|
||||
## 5. Backend work this UI implies (gaps)
|
||||
|
||||
The mockup assumes data we don't serve yet. Each is a discrete backend task:
|
||||
|
||||
1. **Artwork serving** — `albums.artwork_id` / Cover-Art URLs are stored but never served.
|
||||
Need `GET /api/albums/:id/artwork` (and/or per-track) that streams/redirects to the cached
|
||||
cover, plus a placeholder when absent. Without this every card is a gradient placeholder.
|
||||
2. **Playlists** — the rail shows "Playlists"; there are no playlist tables/endpoints. Needs
|
||||
`playlists` + `playlist_track` schema and CRUD + reorder endpoints. (Net-new feature.)
|
||||
3. **"Most Played"** — derivable now via `GET /api/tracks?sort_by=play_count&order=DESC`.
|
||||
Wire a dedicated view/shelf.
|
||||
4. **"Recently Added"** — needs reliable `mtime`/`created_at` sorting (currently sorted
|
||||
client-side). Consider a `created_at` column + a sorted endpoint.
|
||||
5. **"Made for you" mixes** — map to the Vibe engine: per-genre/seed mixes via
|
||||
`/api/vibe/from-genre` and saved seeds. No new engine work, just presentation + maybe a
|
||||
"mixes" endpoint that returns a handful of seed suggestions.
|
||||
6. **Folder browse** — the rail shows "Folder"; there's no filesystem-browse endpoint. Needs
|
||||
a sandboxed `GET /api/library/browse?path=` under `MUSIC_DIR` (reuse the stream route's
|
||||
traversal guard). Optional / later.
|
||||
7. **Typesense search** — the redesigned top-bar search wants fast fuzzy results; finish the
|
||||
Typesense indexing pipeline (collection + reindex job + index-on-enrich) so search graduates
|
||||
from the Postgres ILIKE fallback. (Already tracked in `progress.md`.)
|
||||
|
||||
## 6. Suggested phasing
|
||||
|
||||
1. **Tokenise theming** — extend tokens + wire Tailwind to CSS vars; migrate existing
|
||||
components to semantic colour classes. (Unblocks real theming; low risk, high leverage.)
|
||||
2. **AppShell + PlaybackBar + NowPlayingPanel** — the structural 3-pane shell with the
|
||||
always-on transport and queue, reusing the current playback store/audio engine.
|
||||
3. **MediaCard / Carousel / Artwork** + **artwork backend endpoint** — make the content
|
||||
artwork-forward; redesign Home around Quick Access + shelves.
|
||||
4. **Library/Discover/Vibe pages** restyled onto the new components.
|
||||
5. **New features as desired:** Playlists, Folder browse, Most Played/Recently Added shelves,
|
||||
Typesense search.
|
||||
|
||||
## 7. Non-goals (for the first rework pass)
|
||||
|
||||
- Auth / multi-user (still single-user).
|
||||
- Mobile/responsive layout (target desktop first; the 3-pane collapses later).
|
||||
- Real-time collaborative features.
|
||||
|
||||
## 8. Open questions
|
||||
|
||||
- Keep both the right Now-Playing panel **and** the bottom bar, or collapse to one? (Plan
|
||||
assumes both, panel collapsible.)
|
||||
- Artwork storage: serve via a backend proxy/cache, or store files locally and serve static?
|
||||
- Playlists: is this in scope for the rework, or a separate feature track?
|
||||
Reference in New Issue
Block a user