feat(discovery): acquire recommendations that keep their names

Acquisition ran yt-dlp without --embed-metadata, so every download
arrived untagged. The scanner then stored the video id as the title and
"Unknown Artist" as the artist, the vetted-candidate tag check rejected
the mismatch, and all 18 acquired tracks were hidden and retired.

- Pass --embed-metadata so downloads carry real tags.
- Let a scan take fallback title/artist from the candidate, for sources
  that still ship untagged files.
- Install Deno alongside yt-dlp: YouTube guards some formats with a JS
  challenge yt-dlp must execute, and no other runtime is enabled.
- Dedupe candidates by artist and title. The (source, external_id) key
  misses the same song reaching us under two Deezer release ids.

Also carries the in-flight discovery work this builds on: the
Recommendations page replacing Discover, the discovery source service,
and the acquisition spec tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
kami
2026-08-08 18:43:58 +04:00
parent d371bd97f3
commit bfe22745bc
28 changed files with 1241 additions and 221 deletions
+60
View File
@@ -759,4 +759,64 @@ export const MIGRATIONS: Migration[] = [
ON CONFLICT (session_id) DO NOTHING;
`,
},
{
// Two external candidate strategies, kept as separate source_trust keys so
// meta-learning can compare their retention independently: "artists you
// already play just released something" is a much stronger prior than
// "Last.fm thinks this sounds similar", and the trust values say so.
id: '20260806_external_discovery_sources',
sql: `
INSERT INTO source_trust (key, trust, description) VALUES
('new_release', 0.60,
'New release by an artist already played from the local library.'),
('similar_recommendation', 0.45,
'External similarity (Last.fm) seeded from local play history.')
ON CONFLICT (key) DO NOTHING;
`,
},
{
// play_history is the only durable record of what was listened to and when,
// and it had two holes that only show up when you try to read a year back:
//
// 1. ON DELETE CASCADE meant the gated cleanup sweep silently erased the
// plays of every file it removed. A play happened; deleting the file
// later does not un-happen it. The FK becomes SET NULL and the track's
// identity is denormalised onto the row so it stays readable.
// 2. No duration, so listening time was only ever inferable from the
// track's current duration — itself gone once the file is.
id: '20260806_play_history_durable_facts',
sql: `
ALTER TABLE play_history ADD COLUMN IF NOT EXISTS listened_ms INTEGER;
ALTER TABLE play_history ADD COLUMN IF NOT EXISTS track_title TEXT;
ALTER TABLE play_history ADD COLUMN IF NOT EXISTS track_artist TEXT;
DO $$
DECLARE fk_name TEXT;
BEGIN
SELECT con.conname INTO fk_name
FROM pg_constraint con
JOIN pg_class rel ON rel.oid = con.conrelid
JOIN pg_attribute att ON att.attrelid = rel.oid AND att.attnum = con.conkey[1]
WHERE rel.relname = 'play_history'
AND con.contype = 'f'
AND att.attname = 'track_id'
AND con.confdeltype = 'c'
LIMIT 1;
IF fk_name IS NOT NULL THEN
EXECUTE format('ALTER TABLE play_history DROP CONSTRAINT %I', fk_name);
ALTER TABLE play_history
ADD CONSTRAINT play_history_track_id_fkey
FOREIGN KEY (track_id) REFERENCES tracks(id) ON DELETE SET NULL;
END IF;
END $$;
-- Backfill identity for rows written before the columns existed. Rows whose
-- track was already cascade-deleted are unrecoverable; this at least stops
-- the bleeding from here on.
UPDATE play_history ph
SET track_title = t.title, track_artist = t.artist
FROM tracks t
WHERE t.id = ph.track_id AND ph.track_title IS NULL;
`,
},
];
+9 -1
View File
@@ -302,12 +302,20 @@ CREATE TABLE IF NOT EXISTS recommendation_batch_track (
-- "Success-Driven Center" rule (a completed play moves the active batch's center)
-- and the feedback learning loop. Created with IF NOT EXISTS so it can be
-- self-provisioned on databases that predate this schema change.
-- A play is a historical fact: it stays true after the file is gone. Hence
-- ON DELETE SET NULL rather than CASCADE, plus the denormalised title/artist so
-- a row whose track was hard-deleted is still readable. listened_ms is what was
-- actually heard, recorded at play time; tracks.duration is not a substitute
-- because it disappears with the track.
CREATE TABLE IF NOT EXISTS play_history (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL,
track_id UUID REFERENCES tracks(id) ON DELETE CASCADE,
track_id UUID REFERENCES tracks(id) ON DELETE SET NULL,
batch_id UUID REFERENCES recommendation_batch(id) ON DELETE SET NULL,
completed BOOLEAN NOT NULL DEFAULT false,
listened_ms INTEGER,
track_title TEXT,
track_artist TEXT,
played_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);