Files
muzick/docs/architecture/09-recommendation-and-identity-v2.md
T

1182 lines
54 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Recommendation & Identity v2
This document retires the v1 recommendation engine; it does not tune it.
Where the old `05-recommendation-spec.md` shipped a single CTE inside
`db.service.ts:getNextVibeChunk` (line ~715) that simultaneously did
candidate generation, scoring, evaluation, exploration policy, and
session composition, v2 splits that loading into five cooperating
systems. The CTE stays untouched (and buggy) until System D lands,
then is deleted.
The two philosophy docs — *Music Intelligence System* and *Discovery
Pipeline / Session Director* — describe the shape. This doc is the
engineering plan: schema, write paths, read paths, acceptance, and an
explicit *replaces* list per system.
### What survives from the old v2 plan
- **Phase 4 — Image candidates** (`image_candidates` table). Preserved
verbatim in §F below. Orthogonal to recommendation; the bad-image
problem is a provenance problem, unrelated to the engine.
- **§1.1's intent** (the engine must learn from plays, not from a Keep
button the user does not press) — re-implemented under System B as
evidence rows feeding per-profile beliefs, not as a
`feedback(action='promoted')` row.
### What is retired by this plan
The old v2 **Phases 1, 2, 3, 5** are subsumed and displaced:
| Old phase | Retired by | Why |
|---|---|---|
| Phase 1 (engine tuning: recency, overplay, same-art, cap) | System D | All four are symptoms of doing the session director's job inside a scorer. In D they stop being tuning constants and become structural consequences of fatigue + budgets. |
| Phase 1 §1.1 (implicit promote) | System B | `feedback(action='promoted')` is the wrong shape; evidence rows under per-profile beliefs are the right shape. |
| Phase 2 (`album_artists` junction) | System A | A single probabilistic claims graph subsumes album ownership, MB credit, and identity groups as predicate triples. No separate `album_artists` table. |
| Phase 3 (MB authoritative credit, re-credit pass) | System A | MB is the structural *spine* (it seeds high-trust claims), not the *truth*. Re-credit pass becomes "fetch MB claims into the graph"; no destructive overwrite of `track_artists`. |
| Phase 5 (`artist_groups`) | System A | `alias_of` is a continuous belief (`P(DOOM ≡ Madvillain)`), not a curated flag table. |
### Architectural principles
1. **Music is a graph of entities + probabilistic claims**, not a
folder of files or a table of flat similarity rows.
2. **Truth is probabilistic fusion.** Every claim is evidence, not
fact. Conflicts coexist; resolution happens at read time, weighted by
source trust and recency.
3. **MusicBrainz is the structural spine** (MBIDs + credit bands as
high-trust claim seeds), never the truth by decree. When MB and tags
disagree, both claims live in the graph with different trust weights.
4. **Aliasing is continuous** and evolves with listening behavior. A
`alias_of` claim is a belief with a confidence value, reinforced
when the listener plays both aliases back-to-back in a session.
5. **Listener identity is multidimensional**, keyed on `user_id` from
the start (single user today, multi-user tomorrow — no retrofit).
Multiple profiles coexist: long-term, current obsession, discovery,
negative, forgotten, contextual.
6. **Sessions are directed, not scored.** The objective is the best
next *hour*, not the best next *track*. Fatigue, diversity budgets,
arcs, surprise, callbacks, and an entropy target all live in a
planner that re-plans continuously.
7. **Discovery is autonomous and separate from playback.** Acquisition
writes candidates into the graph; probation is a state on those
candidates; the session director consumes probation-tracked tracks
without knowing they are probation.
8. **Everything decays unless reinforced.** Beliefs, claims' confidence,
and fatigue all weaken over time. This prevents permanent historical
bias and keeps the system learning the *current* listener, not the
listener of two years ago.
---
## The five systems
```
A Knowledge graph (probabilistic fusion)
┌────┴────┐
B E
Listener Acquisition
model pipeline
C Candidate generators
D Session director
```
- **A** is the foundation; no other system can run without it.
- **B** and **E** depend only on A and may be built in parallel.
- **C** depends on A (graph traversal) and reads B (profiles inform
generator selection, e.g. revival generator reads the forgotten
profile, discovery generator reads the discovery profile).
- **D** depends on C (needs generators to populate the plan) and B
(needs listener state from beliefs); it is the final piece.
- **Phase 4 (image candidates)** ships any time, independent of AE.
Critical path: **A → {B, E} → C → D**.
---
## System A — Knowledge graph (probabilistic fusion)
Maintains the connected entity graph with probabilistic relationships.
Never contains user-specific knowledge (objective claims carry
`user_id = NULL`); listener-behavior-derived claims carry `user_id`
and fuse into the per-user graph view at read time.
### A.1 Schema
```sql
-- Source trust weights. One row per source of claims. Tunable.
CREATE TABLE source_trust (
key TEXT PRIMARY KEY, -- 'mb' | 'discogs' | 'lastfm' | 'tag' | 'listener_behavior' | 'curated'
trust REAL NOT NULL CHECK (trust >= 0 AND trust <= 1.0),
description TEXT NOT NULL
);
-- Claims: the spine of the graph. One row per (subject, predicate, object, source).
-- user_id is NULL for objective claims (MB, Discogs, tags), non-NULL for
-- listener-behavior-derived claims (e.g. weak alias_of from adjacent plays).
CREATE TABLE claims (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID, -- NULL = objective
subject_type TEXT NOT NULL, -- 'artist'|'album'|'track'|'label'|'genre'|'scene'
subject_id UUID NOT NULL,
predicate TEXT NOT NULL, -- see A.2
object_type TEXT NOT NULL,
object_id UUID NOT NULL,
source TEXT NOT NULL REFERENCES source_trust(key),
confidence REAL NOT NULL DEFAULT 1.0 CHECK (confidence >= 0 AND confidence <= 1.0),
evidence_at TIMESTAMPTZ NOT NULL, -- when the source asserted this
last_reinforced_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
raw JSONB, -- original payload for audit
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE (subject_type, subject_id, predicate, object_type, object_id, source, user_id)
);
CREATE INDEX claims_subject_idx ON claims (subject_type, subject_id, predicate);
CREATE INDEX claims_object_idx ON claims (object_type, object_id, predicate);
CREATE INDEX claims_user_idx ON claims (user_id) WHERE user_id IS NOT NULL;
-- Every entity that participates in the graph carries an optional MBID as
-- the structural spine anchor. These columns already exist on artists/albums
-- today; we add to tracks. MBID presence raises the entity's identity
-- resolution priority (see A.5).
ALTER TABLE tracks ADD COLUMN IF NOT EXISTS recording_mbid UUID;
CREATE INDEX IF NOT EXISTS tracks_recording_mbid_idx ON tracks (recording_mbid) WHERE recording_mbid IS NOT NULL;
```
### A.2 Predicates
The enumerated set. Adding a predicate is a code change (a generator
or fusion view that reads it), not a schema migration — predicates
live in the `claims.predicate` free-text column, validated in code.
| Predicate | subject → object | Meaning |
|---|---|---|
| `credited_main_on` | artist → track | artist is the main credit on the recording |
| `featured_on` | artist → track | artist is a featured performer on the track |
| `credited_main_on_album` | artist → album | artist is the main credit on the album as a whole |
| `featured_on_album` | artist → album | artist is a co-owner / featured credit on the album |
| `alias_of` | artist → artist | subject is an alias of object (directional; confidence = belief) |
| `member_of` | artist → artist | subject is a member of the group object |
| `produced` | artist → track | subject produced the track |
| `composed` | artist → track | subject composed the track |
| `same_label_as` | artist → artist | both artists release on the same label |
| `same_scene_as` | artist → artist | both artists belong to the same scene |
| `influences` | artist → artist | subject influenced object |
| `remix_of` | track → track | subject is a remix of object |
| `cover_of` | track → track | subject is a cover of object |
| `soundtrack_contrib` | artist → franchise | subject contributed to a soundtrack (anime/game/film) |
| `belongs_to_genre` | track/artist → genre | subject belongs to genre (weighted; replaces exact-id match) |
### A.3 Source trust seed
```sql
INSERT INTO source_trust (key, trust, description) VALUES
('curated', 1.00, 'Manual / human-curated claim. Never decayed.'),
('mb', 0.90, 'MusicBrainz structural spine. High-trust seed; not infallible (re-credits disagree with tags).'),
('cover_art_archive',0.85,'Cover Art Archive, MB-backed.'),
('discogs', 0.75, 'Discogs release/artist credits. Strong for releases, weaker for person aliases.'),
('lastfm', 0.50, 'Last.fm tags + similar. Noisy; used as weak signal.'),
('listener_behavior',0.40, 'Derived from observed play patterns (e.g. back-to-back play → weak alias_of). User-keyed.'),
('tag', 0.30, 'File-tag-derived via scanner heuristic. Lowest trust; the &-split fallback.')
ON CONFLICT (key) DO NOTHING;
```
### A.4 Fusion read path
Truth is resolved at read time as a weighted vote across all claims
for a given `(subject, predicate, object)`. The fusion formula:
```
fused(subject, pred, object, user_id) =
Σ over claims c with matching (subject, pred, object)
where c.user_id IS NULL OR c.user_id = $user_id
of source_trust(c.source) * c.confidence * recency(c)
```
where `recency(c) = clamp(0.1, 1.0, days_since(c.last_reinforced_at) / 180)`,
so a claim never reinforced for 180+ days contributes at 10% floor.
A materialised view `claim_fusion` exposes the per-(subject, pred,
object, user) fused value. **View shims** over `claim_fusion` provide
the v1 shapes so existing reads survive the transition without rewrite:
```sql
CREATE OR REPLACE VIEW track_artists_v2 AS
SELECT t.id AS track_id,
a.id AS artist_id,
CASE cf.predicate WHEN 'credited_main_on' THEN 'main' ELSE 'featured' END AS role,
cf.fused_value AS confidence
FROM tracks t
JOIN claim_fusion cf
ON cf.subject_type = 'track' AND cf.subject_id = t.id
AND cf.predicate IN ('credited_main_on','featured_on')
AND cf.object_type = 'artist'
JOIN artists a ON a.id = cf.object_id;
-- Same shape for albums. Drops albums.artist_id reads over time.
CREATE OR REPLACE VIEW album_artists_v2 AS
SELECT al.id AS album_id,
a.id AS artist_id,
CASE cf.predicate WHEN 'credited_main_on_album' THEN 'main' ELSE 'featured' END AS role,
cf.fused_value AS confidence
FROM albums al
JOIN claim_fusion cf
ON cf.subject_type = 'album' AND cf.subject_id = al.id
AND cf.predicate IN ('credited_main_on_album','featured_on_album')
AND cf.object_type = 'artist'
JOIN artists a ON a.id = cf.object_id;
```
`artists.artist_similar` is retired in favour of the graph itself; a
compatibility view maps `same_scene_as` + `alias_of` fused edges to
the old `(artist_id, similar_artist_id, match)` shape for the lifetime
of any read that still wants it.
Genre hierarchy (`genre.parent_id`) is retired as a separate column.
`belongs_to_genre` claims carry a `confidence` weight; the hierarchy
becomes a `parent_of` claim series on genre entities, and the fusion
view's hierarchical rollup walks these claims instead of a column.
### A.5 Write paths
Five independent writers; all UPSERT into `claims`:
1. **MB spine writer** (in the worker enrichment pipeline): when
`lookupRecording` resolves a recording MBID, fetch the full
`artist-credit` (not just the first entry — extend the MB client).
Write one `credited_main_on` claim (artist = first credit) and one
`featured_on` claim per additional credit, with `source='mb'`,
`confidence=1.0`, `evidence_at=NOW()`. For release-group MBIDs on
albums, write `credited_main_on_album` / `featured_on_album`
analogously. For artist-relation ARs (`member of`, ` collaborations`,
`vocal`/`instrument`), write `member_of` / `featured_on` claims.
MBIDs anchor identity: if the credited artist resolves to a known
MBID, claims attach to that artist entity; otherwise a new artist
is created with the MBID set.
2. **Discogs writer**: `discogs_id` already set on artists today;
extend to write `credited_main_on_album` / `same_label_as` claims
from the release's label and artist credits, `source='discogs'`,
`confidence=0.7`.
3. **Last.fm writer**: existing `artist_similar` fetcher writes
`same_scene_as` claims (`source='lastfm'`, `confidence=match/100`)
instead of the `artist_similar` table. Last.fm tags write
`belongs_to_genre` claims with `confidence=tag.count/100`.
4. **Tag-derived writer** (scanner fallback): when no MBID, the
existing `parseArtistsFromMetadata` heuristic writes
`credited_main_on` / `featured_on` claims with `source='tag'`,
`confidence=1.0` (the trust weight, low at 0.30, is what dims it).
`resolveOrCreateArtist` writes the entity row; the claim carries
the entity, not the name string. Re-keying when MB later resolves
is an UPSERT, not an overwrite.
5. **Listener-behavior writer** (writes into the *user-keyed* region
of `claims`): on play sessions, derive weak edges — adjacent plays
within 30 min of two artists write `same_scene_as` (confidence
0.3); back-to-back play of two artists within a session writes
`alias_of` (confidence 0.2). These are *evidence*, not conclusions;
they fuse with the objective `alias_of` claim from MB (if any) at
read time. Behavioral claims decay by `last_reinforced_at`; the
belief-strengthening path in System B reinforces them on repeat.
All writes UPSERT on `(subject, predicate, object, source, user_id)`:
re-fetching a source refreshes `last_reinforced_at` and `evidence_at`
without duplicating rows.
### A.6 Replaces
| Old surface | Status |
|---|---|
| `artist_similar` table | Retired; replaced by `same_scene_as` / `alias_of` claims. Compat view for the transition. |
| `track_artists` (as truth) | Retired as truth; survives as a *view* `track_artists_v2` over `claim_fusion`. No insert path; reads only. |
| `albums.artist_id` (single-FK ownership) | Survives as a denormalised pointer (written by a trigger off `claim_fusion`'s main credit) for back-compat. Truth lives in `album_artists_v2` view. |
| `genre.parent_id` (column) | Retired; replaced by `parent_of` claims on genre entities. |
| Scanner `resolveAlbumArtist` `parts[0]` behaviour | The heuristic now writes *claims*, not truth. `parts[1:]` become `featured_on_album` claims instead of being dropped. |
| MB client `best['artist-credit']?.[0]` first-credit-only read | Extended to read the full `artist-credit` array. |
| Old Phase 2 (`album_artists` junction table) | Not built — subsumed by A. |
| Old Phase 3 (re-credit pass over whole library) | Not built as a destructive overwrite. MB claims UPSERT into the graph; no `track_artists.source` column, no destructive re-credit. |
| Old Phase 5 (`artist_groups`, `artist_group_members`) | Not built — `alias_of` / `member_of` claims are the grouping. |
### A.7 Acceptance
- A track tagged `artist="MF DOOM & Madlib"` with a resolved recording
MBID shows a `credited_main_on` claim from MB for "Madvillain" (if MB
credits Madvillain) AND a `credited_main_on` claim from the tag for
"MF DOOM". Both coexist. The fusion view, weighted by trust (MB 0.90
> tag 0.30), shows "Madvillain" as the higher-confidence main credit.
- An album tagged `albumartist="A & B"` has two `credited_main_on_album`
claims — A from tags, A and B from MB if MB credits both. The album
appears on both A's and B's artist pages via `album_artists_v2`.
- `SELECT * FROM claims WHERE subject_type='artist' AND subject_id=$doom
AND predicate='alias_of'` returns rows from MB (if it asserts an
alias) and from listener_behavior (if the user has played DOOM and
Madvillain back-to-back). Both decay; both reinforce.
- A never-played genre fished via the future `/vibe/from-genre` path
reaches tracks through the `belongs_to_genre` claims + hierarchical
rollup, not exact-id match.
### A.8 Risks
- **Fusion reasoning cost.** Every read now resolves a vote across
multiple claims. Mitigation: `claim_fusion` materialised view,
refreshed on `claims` insert/update; reads hit the view, not the raw
table. Estimate: a single `(subject, predicate, object)` fused value
is a point lookup on the materialised view.
- **MB rate limits during spine backfill.** Initial population walks the
whole library (~3.9k tracks) fetching full `artist-credit`. Expected
hours, not minutes, gated by MB's rate policy.
- **Display flips during transition.** When the fusion view's
high-confidence main credit is "Madvillain" but the file tag says
"MF DOOM & Madlib", the library view shows "Madvillain". This is
intended (MB is the structural spine) but may surprise the user at
first. Both claims remain auditable via `SELECT * FROM claims`.
- **Tag-only tracks** (no MBID) inherit the lowest-trust claims. This
is correct: the graph is honest about how much it knows.
---
## System B — Listener model
Maintains probabilistic beliefs about the listener. Keyed on
`user_id` from the start; behaviours and beliefs cannot be retrofitted
later without painful re-keying once belief data accumulates.
### B.1 Schema
```sql
-- Evidence: every observed interaction that should influence a belief.
-- Append-only. Never edited or deleted (purge policy separate).
CREATE TABLE evidence (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL,
entity_type TEXT NOT NULL, -- 'track'|'artist'|'genre'|'album'
entity_id UUID NOT NULL,
signal TEXT NOT NULL, -- see B.3
profile TEXT NOT NULL, -- which profile this evidence feeds; see B.2
weight REAL NOT NULL, -- signal strength, set by the signal rule
context JSONB, -- optional: {session_id, hour, weekday, activity, ...}
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX evidence_user_entity_idx ON evidence (user_id, entity_type, entity_id, created_at DESC);
CREATE INDEX evidence_user_profile_idx ON evidence (user_id, profile, created_at DESC);
-- Listener beliefs: the derived state. Continuously decayed; reinforced by evidence.
CREATE TABLE listener_beliefs (
user_id UUID NOT NULL,
profile TEXT NOT NULL, -- see B.2
entity_type TEXT NOT NULL,
entity_id UUID NOT NULL,
dimension TEXT NOT NULL, -- 'affinity'|'fatigue'|'familiarity'|'novelty_tolerance'
value REAL NOT NULL CHECK (value >= -1.0 AND value <= 1.0),
confidence REAL NOT NULL CHECK (confidence >= 0 AND confidence <= 1.0),
evidence_count INTEGER NOT NULL DEFAULT 0,
last_reinforced_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
last_decayed_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
PRIMARY KEY (user_id, profile, entity_type, entity_id, dimension)
);
CREATE INDEX listener_beliefs_user_profile_idx ON listener_beliefs (user_id, profile, entity_type, entity_id);
```
### B.2 Profiles
Enumerated. Adding a profile is a code change (an evaluator or
generator that reads it), not a schema migration.
| Profile | Decay half-life | Fed by signals | Read by |
|---|---|---|---|
| `longterm` | 365 days (slow) | replays, manual search, add-to-favorites, multiple sessions | comfort, deep-dive generators |
| `obsession` | 14 days (fast) | disproportionate play concentration | adjacent, novelty generators (to bound) |
| `discovery` | 30 days | play-of-never-before-seen-entity, accept-after-skip | discovery, experimental generators |
| `negative` | 180 days | skips, queue-removal, hide, manual-delete | all generators (exclusion) |
| `forgotten` | n/a (derived) | `longterm` high-affinity + 90d no plays | revival generator |
| `contextual` | 7 days | context-tagged play sessions | contextual generator |
`forgotten` is derived nightly from `longterm` beliefs that have not
been reinforced in 90 days; it is not written by signals directly.
### B.3 Signal → weight rules
A signal writes one `evidence` row with a `weight`. The weight feeds
into the belief update (B.4). Signal weights are constants, tunable:
| Signal | Profile | Weight | Direction |
|---|---|---|---|
| `playback_completed` | longterm | +0.10 | affinity up |
| `replay_within_24h` | longterm | +0.25 | affinity up |
| `replay_within_24h` | obsession | +0.40 | affinity up |
| `manual_search` | longterm | +0.50 | affinity up |
| `add_to_favorites` | longterm | +0.60 | affinity up |
| `shared` | longterm | +0.70 | affinity up |
| `play_of_never_seen` | discovery | +0.05 | discovery tolerance up |
| `accept_after_probe` | discovery | +0.30 | affinity up (mild) |
| `skip_quick` (≤ 30s) | negative | -0.20 | affinity down |
| `skip_repeated` | negative | -0.40 | affinity down |
| `queue_removed` | negative | -0.30 | affinity down |
| `hidden` | negative | -0.60 | affinity down |
| `manual_deleted` | negative | -0.90 | affinity down (strong) |
Neutral actions (seek, pause, volume) write no evidence. Lack of
interaction writes no evidence (a core principle: *absence of
interaction is not dislike*).
`weight` is per-signal; an event may write multiple `evidence` rows
across multiple profiles (a `replay_within_24h` writes both a
`longterm` +0.25 row and an `obsession` +0.40 row).
### B.4 Belief update + decay
On each new evidence row matching `(user, profile, entity, dimension)`:
```
belief.value = clamp(-1, 1, belief.value + Σ new_evidence.weight * (1 - belief.confidence))
belief.confidence = clamp(0, 1, belief.confidence + 0.05)
belief.evidence_count += count(new rows)
belief.last_reinforced_at = NOW()
```
Daily decay job (or on-read lazy decay):
```
h = (NOW() - belief.last_decayed_at) / profile.halflife_days
belief.value *= 0.5 ^ h
belief.confidence *= 0.5 ^ h -- confidence also decays
belief.last_decayed_at = NOW()
```
A belief not reinforced for 3 halflives approaches zero. Old evidence
becomes irrelevant; new evidence dominates. This prevents the v1
failure mode where heavily-played artists keep winning forever.
### B.5 Replaces
| Old surface | Status |
|---|---|
| `feedback(action='promoted')` | Not written. The implicit-promote intent from old §1.1 lives as a `playback_completed` evidence row → longterm affinity up. |
| `feedback(action='disliked'|'skipped'|'deleted_permanent')` | Not written. Same signals now write `negative`-profile evidence rows. The `feedback` table is retired; existing data is backfilled into `evidence` once and then the table is dropped. |
| `favorites` table | Survives as a UI collection (the Keep button is a UI concept, separate from engine affinity). Old §1.1 explicitly kept this split; we keep it. Keep writes a `favorites` row AND a `add_to_favorites` evidence row. |
| The clamped genre-affinity term in `local_pool` | Retired. Affinity is now per-(user, profile, entity) in `listener_beliefs`; generators read beliefs directly. |
| `play_history` reads by nothing in v1 | `play_history` survives (it is the audit log of plays) but the scorer doesn't read it; the evidence writer does, once per play, converting a play row into evidence signals. |
### B.6 Acceptance
- After one completed play of a never-played track, a `longterm`
affinity belief for that track exists at `value=+0.05`, `confidence=0.05`.
- After 5 completed replays within a week, that track's `longterm`
affinity is above +0.30; the artist's affinity (rolled up from track
beliefs) is above +0.20.
- A track skipped 3× in 30 days has a `negative`-profile affinity below
-0.50. The session director's exclusion filter consults this.
- A track not played for 365 days has decayed its `longterm` affinity
to ~50% of peak; it appears in the `forgotten` derived profile.
- `SELECT value FROM listener_beliefs WHERE user_id=$1 AND profile='obsession'
AND entity_type='artist' ORDER BY value DESC LIMIT 5` returns the current
obsessions, which the session director uses to bound overplay.
---
## System C — Candidate generators
Recommendations originate from independent generators. Each proposes
candidates without knowledge of final ranking — the session director
(D) does the ranking, mixing, and session composition. Each generator
returns candidates with a **graph-path explanation** (the chain of
claims that led to this candidate), so every recommendation is
auditable.
### C.1 Generator interface
```ts
interface Generator {
id: string; // 'comfort' | 'adjacent' | 'discovery' | ...
run(ctx: GeneratorContext): Promise<Candidate[]>;
}
interface GeneratorContext {
userId: string;
listenerState: ListenerState; // from D's state builder
beliefs: BeliefReader; // reads listener_beliefs
graph: GraphReader; // reads claim_fusion (A)
profile: ProfileName; // which profile this generator prefers
recentExclusions: Set<string>; // (entityType, entityId) already churned this session
}
interface Candidate {
trackId: string;
generatorId: string;
explanation: ClaimEdge[]; // the graph path that produced this candidate
// No score. Generators don't rank; the director does.
}
interface ClaimEdge {
subjectType: string; subjectId: string;
predicate: string;
objectType: string; objectId: string;
fusedValue: number;
}
```
### C.2 The generators
Each generator wraps a graph query. All read `claim_fusion` and
`listener_beliefs`; all return `Candidate[]` with explanations.
1. **Comfort** — reads `longterm` affinity beliefs, picks tracks by
artist with affinity > +0.5, fuses with `credited_main_on` /
`featured_on` claims to find tracks by those artists. Goal:
maintain satisfaction.
2. **Adjacent** — for each seed artist in current session state, walks
12 graph hops: `seed → credited_main_on → track → featured_on →
artist → member_of → group → member_of → artist`. Returns tracks
by reached artists, excluding those in the comfort pool.
3. **Discovery** — picks tracks whose artists have no `longterm` /
`obsession` belief (truly unfamiliar), filtered to those with at
least one graph edge to a trusted artist (`same_scene_as`,
`same_label_as`, `produced` by a producer who produced a favourite).
Reads the `discovery` profile's `novelty_tolerance` to set how many
to return.
4. **Deep-dive** — prioritises complete albums. Picks an album owned
(via `album_artists_v2`) by an artist with `obsession` affinity and
returns overlooked tracks (those with low `familiarity` belief) in
album order. Prefers tracks with no play history.
5. **Revival** — reads the `forgotten` derived profile, returns tracks
whose `longterm` affinity is high but `last_reinforced_at` is old
(> 90 days). Time window is adaptive: nostalgia horizon scales with
how established the longterm profile is.
6. **Novelty** — queries for tracks with `release_date` in the last 60
days whose artists share a `same_label_as` / `same_scene_as` edge
with a favourite, OR a `produced` edge from a known producer. Most
recent first, gated by `discovery` profile tolerance.
7. **Experimental** — deliberately challenges current assumptions.
Finds genres with very few `longterm` beliefs of any sign (i.e.
the system is uncertain), picks tracks from those genres with the
highest network-distance from favourites. Goal: learning, not
satisfaction. Run rate is low (one track per N, configurable).
8. **Contextual** — reads the `contextual` profile. If the listener
state has a context tag (coding / driving / sleeping), returns
tracks whose `listener_beliefs` context entries match that
context.
### C.3 Replaces
| Old surface | Status |
|---|---|
| `local_pool` CTE | Retired. Comfort and adjacent generators together cover what local_pool tried to be (genre-overlap + artist_sim + same-artist + audio + jitter). |
| `probation_pool` CTE (gated on `artist_sim > 0`) | Retired. Discovery + deep-dive generators cover the intended-but-unshipped behavior. |
| `getVibeChunkFromGenre` (stateless genre seed) | Survives briefly as a thin wrapper over the discovery generator seeded with a genre; cleaned up when D lands. |
### C.4 Acceptance
- Each candidate returned by any generator carries a non-empty
`explanation` array (graph path). A recommendation with no graph
path is invalid; generators refuse to return it.
- Seeding a DOOM track: adjacent generator returns tracks by artists
reached via `featured_on` from DOOM tracks (i.e. Madlib's other
projects) and via `member_of` from DOOM (i.e. Madvillain tracks) —
*all as candidates*, with explanations; the session director decides
whether to use them given the fatigue model.
- A genre with no library coverage (seeded via `/vibe/from-genre`)
returns zero comfort candidates and a non-empty discovery candidate
list — surfacing fresh material, not the empty result of v1's
`artist_sim > 0` gate.
---
## System D — Session director
The planner. Replaces `getNextVibeChunk` entirely. Where the old CTE
selected the highest-scoring 20 tracks in one query, D maintains a
rolling 2050-track plan that is rewritten on every feedback event,
pursuing invisible long-term goals (finish an album over days,
introduce an artist gradually, balance decades) while optimising
multiple objectives simultaneously.
### D.1 Listener state
Built at the start of each session and updated on each play/skip:
```sql
-- Per-session state; persisted across heartbeats so resumes stay coherent.
CREATE TABLE session_state (
session_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL,
started_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
last_interaction TIMESTAMPTZ NOT NULL DEFAULT NOW(),
context TEXT, -- 'coding'|'driving'|'sleeping'|NULL (auto-detected or manual)
state_vector JSONB NOT NULL -- the computed state; see below
);
```
`state_vector` fields (computed on state build, refreshed on each event):
```jsonc
{
"energy": 0.62, // avg energy of last 5 plays
"focus": 0.40, // manual focus intent, 0..1
"novelty_hunger": 0.30, // from discovery profile's novelty_tolerance
"artist_fatigue": { "<artistId>": 0.71, ... }, // see D.2
"genre_fatigue": { "<genreId>": 0.55, ... },
"language_fatigue": { "ja": 0.83, "en": 0.10 },
"vocal_fatigue": 0.40, // 0 = vocals ok, 1 = want instrumental
"session_age_min": 73,
"current_mood": "energetic",
"target_entropy": 0.55 // see D.5
}
```
### D.2 Fatigue model
Everything gets fatigued. Everything recovers over time. Fatigues are
per-dimension cumulative decays over recent play history, NOT session
counters (v1's `artist_play_count` batch counter is retired).
For each dimension X (track / artist / genre / language / vocalist):
```
fatigue_X(entity, t) = Σ over plays p of X in last T_window
of exp( -(t - p.played_at) / decay_X )
T_window = 24h (artist, genre) | 7d (track) | 2h (language, vocalist)
decay_X = 8h (artist, genre) | 30d (track) | 1h (language, vocalist)
```
- `track` fatigue: a track played in the last hour `exp(-(0)/30d)=1`;
played yesterday `exp(-(1d)/30d)≈0.97` — *strong* recent-play penalty
on tracks; this is the v1 missing cross-session overplay fix, made
structural. Played a month ago: `exp(-(30d)/30d)≈0.37`.
- `artist` fatigue: rolled up from track fatigue over the artist's
tracks; collapses MF DOOM / Madvillain / Viktor Vaughn **iff** their
`alias_of` claims fuse them at read time (which depends on whether
MB or listener-behavior has asserted the alias). This is the honest
v1 Phase 5 fix — aliasing is a graph belief, not a flag.
- `genre` / `language` / `vocalist` fatigue: same formula, same
recency-aware decay.
A candidate's final rank incorporates `1 - fatigue_X(candidate, now)`
as multipliers per dimension; the v1 `GREATEST(0.15, ...)` floor
becomes a tunable per-dimension floor in `session_floor` config.
### D.3 Diversity budgets
Instead of hard caps ("max 1 per artist per chunk", "max 2 per genre"),
a budget the planner spends. Per-session, refreshed at session start:
```sql
INSERT INTO source_trust VALUES ('budget_default', 0.0, 'non-graph config sentinel') ON CONFLICT DO NOTHING;
CREATE TABLE diversity_budgets (
user_id UUID NOT NULL,
dimension TEXT NOT NULL, -- 'artist'|'genre'|'language'|'instrumental'|'new_artist'|'favorite'
budget_share REAL NOT NULL, -- fraction of session, e.g. 0.20
horizon_min INTEGER NOT NULL, -- budget window, e.g. 30 (min)
PRIMARY KEY (user_id, dimension, horizon_min)
);
```
Default budgets (seeded on first session per user):
```jsonc
{
"artist": { "share": 0.20, "horizon": 30 },
"genre": { "share": 0.40, "horizon": 30 },
"language": { "share": 0.60, "horizon": 30 },
"instrumental": { "share": 0.10, "horizon": 30 },
"new_artist": { "share": 0.15, "horizon": 60 },
"favorite": { "share": 0.25, "horizon": 60 }
}
```
"Not more than 2 songs of the same artist in the last 30 min" becomes
a 20%-of-30min budget. The planner spends, replenishes at window edge.
If recent listening has blown a budget, the planner refuses further
spends in that dimension — the structural replacement for v1's
diversity cap.
### D.4 Arcs
The planner doesn't pick songs; it picks *arcs* and slots songs into
them. Templates:
- **Comfort arc**: known → known → adjacent → favorite.
- **Discovery arc**: favorite → similar → new → favorite.
- **Energetic arc**: medium → high → peak → cooldown.
- **Late-night arc**: soft → ambient → acoustic → slow electronic.
At session start, given the state vector and the long-term schedule
(D.7), the planner picks an arc template and fills it. The plan is a
list of "slots" (`{arc_position, role}`, e.g. `{1, "peak"}`); the
planner queries the matching generator for each slot. On replan,
remaining slots can shift arc.
### D.5 Surprise, callbacks, entropy target, anti-loop
- **Surprise budget**: ~1 per hour (configurable). Reserved slot in
the arc for a forgotten favorite, live version, cover, acoustic
version, producer side project, or old obsession. Surfaced via the
revival generator with a `surprise=true` flag.
- **Callbacks**: every N tracks the planner intentionally re-introduces
an artist / theme / energy level from earlier in the session (or
earlier session that day). Makes the session feel intentional.
- **Entropy target**: the state vector carries `target_entropy`. If
the autocorrelation of last-20 chosen-track features is too high
(predictable), entropy goes up (planner prefers experimental /
discovery candidates). If too low (chaotic), planner injects
comfort. Target is **controlled unpredictability**, not randomness.
- **Anti-loop detector**: continuously monitor the last-50 plan
choices for collapsing into a narrow graph region (same artist /
label / producer / genre / decade / BPM / mood / language). If
collapse detected, the planner forcibly expands: zero-out the
dominant dimension's budget for the next window and surge a
non-dominant generator.
### D.6 Repetition rules
Adaptive minimum-distance, not absolute "don't repeat":
```sql
CREATE TABLE repetition_rules (
user_id UUID NOT NULL,
dimension TEXT NOT NULL, -- 'track'|'artist'|'album'|'genre'|'energy'
min_distance INTEGER NOT NULL, -- adaptive; minutes
PRIMARY KEY (user_id, dimension)
);
```
Defaults: `track=2h, artist=20min, album=no immediate; spread across
hours, genre=don't-dominate, energy=smooth transitions`. All adapt:
if a listener shows high `focus` (deep work signal), distances relax
(loop tolerance up); if skipping-after-replay pattern appears,
distances tighten.
### D.7 Long-term scheduling + invisible goals
The planner also has week-scale objectives, tracked in
`session_state.state_vector.goals`:
- Finish an album over several days (track which album is "in
progress"; the deep-dive generator keeps returning its overlooked
tracks until the album is fully played).
- Introduce a new artist gradually (e.g. one track per session for a
week, escalating if survival rate is high).
- Revisit old favorites monthly.
- Balance decades, languages, producers (the budgets cover most of
this; the planner periodically nudges an under-represented decade to
surge).
- Complete discovery probation (see System E).
- Guarantee at least one surprise per hour.
The listener should never notice these goals directly.
### D.8 Don't maximise enjoyment
The planner optimises multiple objectives simultaneously:
```
maximise:
enjoyment (predicted from beliefs × relevance)
discovery (fraction of unfamiliar entities in the plan)
diversity (1 - Herfindahl index across artists in horizon)
learning (information gain on uncertain beliefs)
session coherence (arc-template adherence)
long-term freshness (entropy target met)
minimise:
fatigue (cumulative per-dimension fatigue)
repetition (autocorrelation of recent choices)
predictability (1 - entropy)
wasted discoveries (candidates surfaced then immediately skipped)
```
This is why single-objective score maximisation (the v1 approach)
collapses to "ADO, Yoasobi, ADO, Zutomayo, ADO" — those are the
"optimal" tracks by predicted enjoyment alone.
### D.9 Plan + replan loop
```
session start
build state_vector (D.1)
pick arc template (D.4) + target entropy (D.5)
for each slot in arc:
query matching generator (C) → candidates
rank candidates across the D.8 objectives
pick winner, respecting budgets (D.3) + repetition rules (D.6)
2050 track plan
playback
on play / skip / manual action:
write evidence (B.3)
refresh state_vector fatigue (D.2)
if plan slot < 10 remaining OR anti-loop fires OR entropy drift > 0.2:
replan from current state
loop
```
### D.10 Replaces
| Old surface | Status |
|---|---|
| `getNextVibeChunk` CTE (~220 lines in `db.service.ts`) | Retired entirely on D ship. |
| `recommendation_batch_track` exclusion set | Survives as the recent-exclusions `Set` passed to generators; no longer the source of artist-play-count. |
| `recommendation_batch` row + `seed_track_id` center-walk in `recordPlay` | Retired. The center-walk was a hack for "engine can't escape the seed neighbourhood"; D's arc + fatigue together replace it. |
| Old §1.2 (recency term added to local_pool) | Not a scored term anymore; recency is a fatigue-dimension multiplier in D.2. |
| Old §1.3 (track-level overplay penalty) | D.2's `track` fatigue, structural. |
| Old §1.4 (artist-level overplay penalty, identity-best-effort) | D.2's `artist` fatigue rolled up via alias fusion in A. Identity collapse happens *iff the claims graph says so*, not as a separate code path. |
| Old §1.5 (lower W_SAMEART) | No `W_SAMEART` to tune; the comfort generator alone handles "more of this artist" and is naturally bounded by D.3's artist budget. |
| Old §1.7 (cap on `track_artists.artist_id` not name string) | Subsumed by D.3's budgets (artist dimension). |
### D.11 Acceptance
- After 6 hours of listening, the listener is still engaged, has
discovered at least one unfamiliar but tolerable track, has not
become fatigued by any single artist / genre / language, and the
next session would still feel fresh.
- Seeding a DOOM track does not collapse the next chunks into DOOM
pseudonyms even though alias fusion may treat them as one artist —
because D.2's artist fatigue rises fast in the session, D.3's
artist budget blocks further spends, and D.4's arc pulls toward
adjacent generators (Madlib's other projects reached via graph
hops, not DOOM).
- Within a session, tracks played in the last hour do not re-appear
(track fatigue multiplier ≈ 0 after recent plays).
- Across sessions, the same top-20 does not return: track fatigue
half-life of 30 days means yesterday's plays still dampen today's
rank.
- Forgotten favorites resurface naturally (revival generator + monthly
long-term goal).
- Anti-loop detector fires when the dominant dimension's share exceeds
budget × 1.5, forcibly diversifying the next window.
---
## System E — Acquisition pipeline
The library is not the universe. E continuously searches beyond the
current collection, identifies music worth evaluating, acquires it
(via the unbuilt yt-dlp worker, `progress.md:29`), validates it, and
either permanently integrates it into the graph (A) or discards it.
Discovery is independent of playback; it writes into A.
### E.1 Discovery sources
Six independent strategies run continuously as low-priority worker
jobs:
1. **Graph exploration** — walk the graph beyond the library. For each
favourite artist (per `longterm` beliefs), follow `featured_on` /
`member_of` / `produced` / `same_label_as` / `same_scene_as` edges
to artists not in the library. Each traversal is a discovery path
candidate.
2. **Release monitoring** — monitor favourite artists, related artists
(graph adjacents), labels, and producers for new releases. New
releases become discovery candidates at high priority.
3. **Scene exploration** — discover music through communities rather
than artists: city scenes, internet communities, niche genres,
underground movements, independent labels. Avoids recommendation
loops around the same popular artists.
4. **Temporal exploration** — search different musical eras for
forgotten classics, overlooked releases, albums that became
influential years later.
5. **Relationship expansion** — instead of "people also listen to",
prefer structural relationships: same producer, same composer,
live band members, guest vocalists, touring partners, soundtrack
contributors.
6. **Curiosity exploration** — dedicated exploration budget for
unfamiliar genres, different languages, experimental music,
geographically distant scenes. Success is measured by learning,
not immediate satisfaction.
### E.2 Candidate universe
Before downloading, discoveries live as `claims` rows with
`subject_type='track'` and a special marker — they are *candidate*
tracks, not library tracks. The candidate carries the discovery
source, the relationship path that led to it, an estimated relevance,
and an explanation.
This reuses `claims` rather than a dedicated table, with a dedicated
predicate:
```sql
-- A discovery candidate is a claim: subject=track (candidate), predicate='discovery_candidate', object=source artist / scene / label.
-- The 'confidence' field is the estimated relevance; 'raw' holds the full path + explanation.
INSERT INTO claims (subject_type, subject_id, predicate, object_type, object_id, source, confidence, raw)
VALUES ('track', $candidateId, 'discovery_candidate', 'artist', $relatedArtistId, 'graph_exploration', $relevance, $pathJson)
ON CONFLICT (subject_type, subject_id, predicate, object_type, object_id, source, user_id) DO NOTHING;
```
Candidate tracks themselves are stored as stub rows in a
`discovery_candidates` table — thin rows holding the external identity
only, no library path / audio / metadata yet:
```sql
CREATE TABLE discovery_candidates (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
source TEXT NOT NULL, -- 'mb'|'discogs'|'lastfm'|'spotify'|...
external_id TEXT NOT NULL, -- MBID / discogs_id / etc.
title TEXT,
artist_credit JSONB, -- the full artist-credit array from the source
notes JSONB, -- discovery path, source-only fields
first_seen_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
last_eval_at TIMESTAMPTZ,
status TEXT NOT NULL DEFAULT 'candidate', -- 'candidate'|'acquiring'|'probation'|'retained'|'retired'
UNIQUE (source, external_id)
);
-- Keeping the graph reference:
-- discovery_candidates.id is referenced by claims rows with subject_type='track' (UUID aligns).
```
### E.3 Acquisition policy + queue
Not every candidate downloads. Decision factors:
- expected usefulness (relevance confidence from the discovery claim)
- novelty (does the listener's `discovery` profile tolerate this
territory?)
- storage budget
- artist diversity (don't acquire 10 tracks from one new artist in a
day)
- existing backlog depth
- current listener fatigue (don't acquire more of a fatigued artist)
- current exploration budget share
The decision maximises **expected information gain**, not download
count.
Priority queue:
| Priority | Source |
|---|---|
| 1 (highest) | favourite artists' new releases |
| 1 | active obsession new releases |
| 2 | adjacent artists, collaborations, graph discoveries |
| 3 (lowest) | experimental discoveries, curiosity experiments |
Downloads happen invisibly (yt-dlp worker, throttled, low bandwidth).
On successful download, the audio is scanned (existing scanner), which
writes `credited_main_on` / `featured_on` claims and a `tracks` row.
The discovery_candidate row transitions to `probation`.
### E.4 Probation
Downloaded music is never trusted immediately. Every acquisition
enters probation. During probation, the session director (D)
occasionally injects probation tracks into normal sessions — the
listener should not feel they are being tested.
Probation is a `tracks.probation_status` column (new), one short
migration:
```sql
ALTER TABLE tracks ADD COLUMN IF NOT EXISTS probation_status TEXT
DEFAULT 'retained' CHECK (probation_status IN ('probation','retained','retired'));
ALTER TABLE tracks ADD COLUMN IF NOT EXISTS probation_entered_at TIMESTAMPTZ;
CREATE INDEX tracks_probation_idx ON tracks (probation_status) WHERE probation_status = 'probation';
```
Existing library tracks default to `retained`. Newly acquired tracks
are `probation` with `probation_entered_at = NOW()`.
Each probation track accumulates evidence (B.3) over multiple
sessions — single interactions rarely provide enough. Possible
outcomes:
- **retain** — survival threshold met; probation_status → `retained`.
Associated artist's `longterm` affinity gets a small bump, the
discovery path that produced this candidate gets reinforced (a meta
signal for E.5).
- **archive** — kept on disk but hidden from normal sessions; exempt
from the planner.
- **delete** — file removed, `tracks` row marked `retired`. Library
is not an ever-growing archive.
- **ignore temporarily** — back to candidate state for re-evaluation
later; rarer path.
Probation duration adapts to confidence: a candidate discovered via a
trusted path (favourite producer's new signing) gets a longer
probation than a curiosity-experiment candidate.
### E.5 Meta-learning
The discovery system continuously evaluates itself. A periodic job
writes claims back into the graph about which *strategies* and *graph
paths* have produced long-term retainers, vs which consistently retire:
`source_trust` already tunes per-source; meta-learning additionally
tunes per-predicate-path:
- which discovery sources (graph_exploration, release_monitoring,
scene_exploration, ...) produce long-term favourites?
- which graph paths consistently fail? (e.g. `same_label_as` may be a
weak edge; down-weight it.)
- which labels repeatedly introduce successful artists?
- which exploration depth performs best?
- which experiments produce the highest information gain?
This meta-learning itself writes back as `source_trust` adjustments
and as tunable per-path-weight constants. Discovery learns how to
discover better, not just what to recommend.
### E.6 Acceptance
- A discovery candidate surfaced via "producer A produced favourite B
AND new artist C" is auditable: `SELECT raw FROM claims WHERE
subject_id=$candidateId AND predicate='discovery_candidate'` shows
the full path.
- A downloaded probation track on which the listener completed 3 plays
in its first 2 sessions transitions to `retained` automatically.
- A downloaded probation track skipped on every injection retires to
`retired` after its probation window; file is removed; library does
not accumulate indefinitely.
- The meta-learning job, run weekly, down-weights a discovery strategy
(e.g. `scene_exploration`) whose recent candidates have a <20%
retention rate, observable in `source_trust` deltas or per-path
weight constants.
### E.7 Replaces
Nothing — E is net new. It consumes A (graph) and writes back into A.
Built on top of the yet-unbuilt yt-dlp worker (`progress.md:29`),
independent of B/C/D.
---
## §F — Phase 4: Image quality (preserved from old v2)
**This section is preserved verbatim from the previous v2 doc.** It is
orthogonal to recommendation; the bad-image problem is a provenance
problem, unrelated to the engine. Ships any time, independent of AE.
### F.1 Schema
```sql
CREATE TABLE image_candidates (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
entity_type TEXT NOT NULL CHECK (entity_type IN ('artist','album')),
entity_id UUID NOT NULL,
source TEXT NOT NULL,
url TEXT,
width INTEGER,
verified BOOLEAN DEFAULT FALSE,
fetched_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE (entity_type, entity_id, source)
);
CREATE INDEX image_candidates_entity_idx ON image_candidates (entity_type, entity_id);
```
- `artists.image_path` and `albums.artwork_id` remain as the
"currently preferred" denormalised pointer, written by the selection
step. Existing queries keep working.
### F.2 Enrichment write
Each image-fetch step (Wikidata, TheAudioDB, Fanart, iTunes, Deezer,
Discogs, Last.fm, Cover Art Archive) writes a `image_candidates` row
even on failure to find one — a "negative" row with `url=NULL` so we
don't re-fetch that source for that entity until the row is aged out.
### F.3 Selection
A selection step (worker job or enrichment sub-step) picks the
preferred URL by tier:
1. Wikidata via MBID (verified, broad coverage) — highest tier.
2. TheAudioDB via MBID.
3. Fanart via MBID.
4. Cover Art Archive (albums) / Deezer (albums) — high-res.
5. iTunes upscaled to 600 — broad coverage fallback.
6. Last.fm — last resort.
7. Wikimedia via name match — excluded (historically wrong; migrations
cleared these twice).
Tiers are a config table or constants, not magic strings in code.
Selection writes the winner into `artists.image_path` /
`albums.artwork_id`.
### F.4 Re-evaluation
- `image_candidates` rows older than `N` days (config, default 90) are
eligible for re-fetch. A periodic job re-runs enrichment for stale
candidates, replacing rows.
- The selector re-runs whenever candidates change. So if a low-tier
winner was selected and a higher-tier candidate lands later, the
preferred pointer is upgraded in place.
### F.5 Acceptance
- After re-enrichment, an artist that previously showed a 100×100
Last.fm thumbnail shows the Wikidata/TheAudioDB image instead.
- Re-running enrichment does not re-fetch sources that already returned
(negative cache).
- Selection is auditable: `SELECT * FROM image_candidates WHERE
entity_id = X` shows every candidate considered.
---
## Rollout
Recommended order, each system independently shippable; the old v1
CTE stays until D lands:
1. **A** (knowledge graph + claim_fusion view). Existing queries move
to the compat views (`track_artists_v2`, `album_artists_v2`); v1
engine keeps running against the views. MB spine backfill runs in
the background.
2. **B and E in parallel** (B listener model + evidence writer; E
acquisition pipeline + yt-dlp worker + probation). Both need only
A. The evidence writer starts converting play_history into
evidence; the existing CTE ignores evidence for now.
3. **C** (generators). Built and run in shadow mode alongside the v1
CTE — both produce chunks; the UI shows v1 chunks, but C's outputs
are logged for comparison. Generators don't replace v1 reads until
D ships.
4. **D** (session director). D switches over and the v1 CTE is
deleted in the same release. Acceptance is the session-feel test
(D.11).
5. **Phase 4 (image candidates)** at any point. Independent.
No phase is blocked except B/E on A, C on A+B, D on C+B. Phase 4 is
fully independent.
## Retiring v1 — explicit deletion list
On System D ship, this code goes:
- `db.service.ts` `getNextVibeChunk` (~220 lines).
- `recordPlay`'s center-walk (`UPDATE recommendation_batch SET
seed_track_id = $2 ...`, line ~580590).
- The `artist_play_count` decay term (line ~829).
- The `W_SAMEART`, `W_ARTSIM`, `W_FEEDBCK`, `W_AUDIO`, `W_RANDOM`
constants + `local_pool` / `probation_pool` CTEs (lines 717866).
- The "max 1 per artist per chunk" cap, replaced by D.3's budgets.
- `getVibeChunkFromGenre` (it becomes a thin wrapper over the discovery
generator; then collapses into D's session-by-genre entry).
- The `feedback` table write paths (`recordSkip`, `recordFeedback`,
hardDelete insert). The `feedback` table itself is dropped after
backfill into `evidence`.
- `recommendation_batch_track` exclusion set (replaced by D's
recent-exclusions set, in-memory per session).
- `albums.artist_id` single-FK pointer (kept as a denormalised
trigger-maintained column off the fusion view; removed as a *read*
source).
- `artist_similar` table (compat view retained briefly, then dropped).
- `genre.parent_id` column (replaced by `parent_of` claims; column
dropped after a backfill claims-migration).
## Out of scope
- **Filesystem reorganisation.** Confirmed out of scope. The bind is
read-only; the DB is the index; reorganising the FS inverts the
dependency in the wrong direction.
- **Auth itself.** Schemas are `user_id`-keyed from the start so no
retrofit is needed later, but building auth is a separate project
(progress.md #30).
- **A user-facing override UI for MB credits.** Competing claims
coexist in the graph; resolution at read time is weighted. A future
UI can show "per MB" vs "per tag" and let the user assert a
`curated` claim (trust 1.0) that overrides. Out of scope here.
- **Manual artist-group / alias curation UI.** `alias_of` is a graph
belief; `curated` claims (trust 1.0) override the learned belief
when human assertion is needed — but the UI to do that is separate.