kami
ee43995e96
The worker ran every job through a single pg Client while BullMQ was configured with concurrency: 10. A Client is one connection with one protocol stream and no queueing: ten concurrent jobs interleave on it, and any BEGIN/COMMIT is shared by all of them, so an unrelated job's failure can roll back another's work and a rollback can discard a third's committed intent. Switched to a Pool, added a small withTransaction(pool, fn) helper that takes a dedicated connection per transaction, and threaded a Queryable interface through the services so they accept either a pool or a pooled client. Both reprocess_artists merge blocks — the artist merge and the duplicate-album merge — now run inside withTransaction; previously a failure partway through left artists merged and their tracks unmoved. integrity.service and cleanup.service get only the constructor type change here so this commit compiles; their own fixes follow in the next two commits. cleanup.service's BEGIN/COMMIT-on-a-Pool is therefore still wrong at this commit and is replaced wholesale by the hard-delete commit. REVIEW-2026-07-30.md finding 4 (and the concurrency note in finding 3). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
58 lines
2.1 KiB
TypeScript
58 lines
2.1 KiB
TypeScript
import type { Pool, PoolClient, QueryResult, QueryResultRow } from 'pg';
|
|
|
|
/**
|
|
* The subset of pg's API the worker services actually need: a `query()` that
|
|
* takes SQL plus positional params.
|
|
*
|
|
* Services are typed against this rather than against `Client` so they work
|
|
* unchanged whether they are handed a `Pool` (the worker process — see
|
|
* index.ts), a `PoolClient` checked out for a transaction, or a plain `Client`
|
|
* (the one-off maintenance scripts in ./scripts, which are single-threaded and
|
|
* own their connection).
|
|
*
|
|
* IMPORTANT: a `Queryable` gives NO transaction guarantees. When a `Pool` is
|
|
* behind it, consecutive `query()` calls may land on different connections, so
|
|
* bare `BEGIN`/`COMMIT` must never be issued through it — check out a dedicated
|
|
* client with `withTransaction()` instead.
|
|
*/
|
|
export interface Queryable {
|
|
query<R extends QueryResultRow = any>(
|
|
sql: string,
|
|
params?: any[]
|
|
): Promise<QueryResult<R>>;
|
|
}
|
|
|
|
/**
|
|
* Run `fn` inside a transaction on a dedicated pooled connection, committing on
|
|
* success and rolling back on any throw. The client is always released.
|
|
*
|
|
* This is the only correct way to run a transaction in the worker: the process
|
|
* runs BullMQ with `concurrency: 10`, so issuing `BEGIN` on a shared connection
|
|
* would enrol another job's unrelated queries in this transaction — and discard
|
|
* them on `ROLLBACK`. (The backend documents the same hazard as its reason for
|
|
* using a `Pool`; see backend/src/app.ts.)
|
|
*/
|
|
export async function withTransaction<T>(
|
|
pool: Pool,
|
|
fn: (client: PoolClient) => Promise<T>
|
|
): Promise<T> {
|
|
const client = await pool.connect();
|
|
try {
|
|
await client.query('BEGIN');
|
|
const result = await fn(client);
|
|
await client.query('COMMIT');
|
|
return result;
|
|
} catch (err) {
|
|
try {
|
|
await client.query('ROLLBACK');
|
|
} catch (rollbackErr) {
|
|
// A failed ROLLBACK means the connection is unusable; log and move on —
|
|
// release() below discards it rather than returning it to the pool.
|
|
console.error('[DB] ROLLBACK failed:', rollbackErr);
|
|
}
|
|
throw err;
|
|
} finally {
|
|
client.release();
|
|
}
|
|
}
|