fix: give the worker a pg Pool and real transactions
The worker ran every job through a single pg Client while BullMQ was configured with concurrency: 10. A Client is one connection with one protocol stream and no queueing: ten concurrent jobs interleave on it, and any BEGIN/COMMIT is shared by all of them, so an unrelated job's failure can roll back another's work and a rollback can discard a third's committed intent. Switched to a Pool, added a small withTransaction(pool, fn) helper that takes a dedicated connection per transaction, and threaded a Queryable interface through the services so they accept either a pool or a pooled client. Both reprocess_artists merge blocks — the artist merge and the duplicate-album merge — now run inside withTransaction; previously a failure partway through left artists merged and their tracks unmoved. integrity.service and cleanup.service get only the constructor type change here so this commit compiles; their own fixes follow in the next two commits. cleanup.service's BEGIN/COMMIT-on-a-Pool is therefore still wrong at this commit and is replaced wholesale by the hard-delete commit. REVIEW-2026-07-30.md finding 4 (and the concurrency note in finding 3). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,57 @@
|
||||
import type { Pool, PoolClient, QueryResult, QueryResultRow } from 'pg';
|
||||
|
||||
/**
|
||||
* The subset of pg's API the worker services actually need: a `query()` that
|
||||
* takes SQL plus positional params.
|
||||
*
|
||||
* Services are typed against this rather than against `Client` so they work
|
||||
* unchanged whether they are handed a `Pool` (the worker process — see
|
||||
* index.ts), a `PoolClient` checked out for a transaction, or a plain `Client`
|
||||
* (the one-off maintenance scripts in ./scripts, which are single-threaded and
|
||||
* own their connection).
|
||||
*
|
||||
* IMPORTANT: a `Queryable` gives NO transaction guarantees. When a `Pool` is
|
||||
* behind it, consecutive `query()` calls may land on different connections, so
|
||||
* bare `BEGIN`/`COMMIT` must never be issued through it — check out a dedicated
|
||||
* client with `withTransaction()` instead.
|
||||
*/
|
||||
export interface Queryable {
|
||||
query<R extends QueryResultRow = any>(
|
||||
sql: string,
|
||||
params?: any[]
|
||||
): Promise<QueryResult<R>>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Run `fn` inside a transaction on a dedicated pooled connection, committing on
|
||||
* success and rolling back on any throw. The client is always released.
|
||||
*
|
||||
* This is the only correct way to run a transaction in the worker: the process
|
||||
* runs BullMQ with `concurrency: 10`, so issuing `BEGIN` on a shared connection
|
||||
* would enrol another job's unrelated queries in this transaction — and discard
|
||||
* them on `ROLLBACK`. (The backend documents the same hazard as its reason for
|
||||
* using a `Pool`; see backend/src/app.ts.)
|
||||
*/
|
||||
export async function withTransaction<T>(
|
||||
pool: Pool,
|
||||
fn: (client: PoolClient) => Promise<T>
|
||||
): Promise<T> {
|
||||
const client = await pool.connect();
|
||||
try {
|
||||
await client.query('BEGIN');
|
||||
const result = await fn(client);
|
||||
await client.query('COMMIT');
|
||||
return result;
|
||||
} catch (err) {
|
||||
try {
|
||||
await client.query('ROLLBACK');
|
||||
} catch (rollbackErr) {
|
||||
// A failed ROLLBACK means the connection is unusable; log and move on —
|
||||
// release() below discards it rather than returning it to the pool.
|
||||
console.error('[DB] ROLLBACK failed:', rollbackErr);
|
||||
}
|
||||
throw err;
|
||||
} finally {
|
||||
client.release();
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user