1 reply

High-Throughput Stream Processing: Resolving Node.js Event Loop Starvation with Custom Scheduler

Event loop starvation:
MixedScheduler
MixedScheduler
microtasks blocking Node.js I/O?

We have a high-throughput

Stream.mapEffect

Stream.mapEffect

pipeline (~8,000+ elements/sec from a WebSocket). We use Node.js

ws

ws

createWebSocketStream

createWebSocketStream

Stream.fromAsyncIterable

Stream.fromAsyncIterable

so that backpressure propagates back to TCP (the server respects it). The stream decodes events and writes some to Postgres (~100ms RTT).

We noticed

sql.execute

sql.execute

taking 2-4s despite <1ms server-side execution. A raw

pool.query("SELECT 1")

pool.query("SELECT 1")

probe in the same process showed the same latency — ruling out Effect SQL, Drizzle, connection pooling, etc.

We believe the issue is the default

MixedScheduler

MixedScheduler

scheduling fiber continuations via

Promise.resolve().then()

Promise.resolve().then()

. At high throughput the microtask queue stays full, preventing Node.js from reaching the I/O poll phase where TCP callbacks (DB responses) fire. The

setTimeout(0)

setTimeout(0)

fallback every 2048 scheduler cycles seems too infrequent at our volume.

Switching to a

setImmediate

setImmediate

scheduler resolved it:

const ioScheduler = Scheduler.makeBatched((runBatch) =>
  setImmediate(runBatch),
);

const main = program.pipe(Effect.withScheduler(ioScheduler));

const ioScheduler = Scheduler.makeBatched((runBatch) =>
  setImmediate(runBatch),
);

const main = program.pipe(Effect.withScheduler(ioScheduler));

DB latency dropped from 2-4s to 100ms (1 RTT). Throughput dropped ~12%.

withMaxOpsBeforeYield

withMaxOpsBeforeYield

can tune the tradeoff.

Is this the recommended approach for high-throughput stream processing alongside I/O, or is there something I am missing about effect?

High-Throughput Stream Processing: Resolving Node.js Event Loop Starvation with Custom Scheduler

Event loop starvation:
MixedScheduler
MixedScheduler
microtasks blocking Node.js I/O?

We have a high-throughput

Stream.mapEffect

Stream.mapEffect

pipeline (~8,000+ elements/sec from a WebSocket). We use Node.js

ws

ws

createWebSocketStream

createWebSocketStream

Stream.fromAsyncIterable

Stream.fromAsyncIterable

so that backpressure propagates back to TCP (the server respects it). The stream decodes events and writes some to Postgres (~100ms RTT).

We noticed

sql.execute

sql.execute

taking 2-4s despite <1ms server-side execution. A raw

pool.query("SELECT 1")

pool.query("SELECT 1")

probe in the same process showed the same latency — ruling out Effect SQL, Drizzle, connection pooling, etc.

We believe the issue is the default

MixedScheduler

MixedScheduler

scheduling fiber continuations via

Promise.resolve().then()

Promise.resolve().then()

. At high throughput the microtask queue stays full, preventing Node.js from reaching the I/O poll phase where TCP callbacks (DB responses) fire. The

setTimeout(0)

setTimeout(0)

fallback every 2048 scheduler cycles seems too infrequent at our volume.

Switching to a

setImmediate

setImmediate

scheduler resolved it:

const ioScheduler = Scheduler.makeBatched((runBatch) =>
  setImmediate(runBatch),
);

const main = program.pipe(Effect.withScheduler(ioScheduler));

const ioScheduler = Scheduler.makeBatched((runBatch) =>
  setImmediate(runBatch),
);

const main = program.pipe(Effect.withScheduler(ioScheduler));

DB latency dropped from 2-4s to 100ms (1 RTT). Throughput dropped ~12%.

withMaxOpsBeforeYield

withMaxOpsBeforeYield

can tune the tradeoff.

Is this the recommended approach for high-throughput stream processing alongside I/O, or is there something I am missing about effect?

High-Throughput Stream Processing: Resolving Node.js Event Loop Starvation with Custom Scheduler

Similar Threads

High-Throughput Stream Processing: Resolving Node.js Event Loop Starvation with Custom Scheduler

Similar Threads

Similar Threads

Similar Threads