Effect CommunityEC
Effect Community•5mo ago
Kam

Combining Array.map, Effect.all, and console logging the progress

Here is my implementation of a less verbose way to map on a list of items to create a list of Effects and then applying Effect.all, while also logging the current progress.
For that, I made a function call allMap (not sure if that's the best name). It supports data-first and data-last, and can be passed optional parameters for concurrency and for adding a delay between the batches.

import { dual } from 'effect/Function';
import { Effect as E, Array, pipe } from 'effect';
import { Concurrency } from 'effect/Types';

type Options = { concurrency?: Concurrency; delayMs?: number };

const _allMap = <A, R, Err, B>(
  items: A[],
  f: (a: A) => E.Effect<B, Err, R>,
  options: Options = {},
): E.Effect<B[], Err, R> => {
  const { concurrency, delayMs = 0 } = options;
  const total = items.length;

  return pipe(
    items,
    Array.map((a, i) =>
      pipe(
        f(a),
        E.tap(() => E.log(`Processed ${i + 1}/${total}`)),
        E.delay(delayMs),
      ),
    ),
    E.allWith({ concurrency }),
  );
};

// dual API
export const allMap: {
  <A, R, Err, B>(
    f: (a: A) => E.Effect<B, Err, R>,
    options?: Options,
  ): (items: A[]) => E.Effect<B[], Err, R>;
  <A, R, Err, B>(
    items: A[],
    f: (a: A) => E.Effect<B, Err, R>,
    options?: Options,
  ): E.Effect<B[], Err, R>;
} = dual(3, _allMap);


Example of usage:

export const runTest = async () => {
  const items = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
  const fn = (n: number) => E.succeed(String(n));

  await pipe(
    items,
    allMap(fn, { concurrency: 3, delayMs: 2000 }),
    E.match({ onSuccess: console.log, onFailure: console.error }),
    E.runPromise,
  );
};


Anybody has a suggestion to improve it? Are there some obvious issues with the implementation?
Was this page helpful?