Effect CommunityEC
Effect Community•4mo ago•
11 replies
BrokenSwing

Issues with @effect/cluster Runner Registration and Layer Setup

Hi, I'm trying to get @effect/cluster running consistently in the same process but I do not succeed.
The problem is that my runner does not register to the shard manager even though through the logs i can see that each layer properly initialise and starts listening on the provided port.

Here is my setup:
const shardManagerAddress = RunnerAddress.RunnerAddress.make({
  host: "localhost",
  port: 4000,
});

const shardManager = BunClusterShardManagerSocket.layer({
  storage: "sql",
  shardingConfig: {
    shardManagerAddress,
  },
});

const runner = BunClusterRunnerSocket.layer({
  storage: "sql",
  shardingConfig: {
    shardManagerAddress,
    runnerAddress: Option.some(RunnerAddress.make("localhost", 4001)),
  },
});


// My API which is using HttpApi from @effect/platform
const WebLive = HttpLive.pipe(Layer.provide(AllServices));

// My runner basically, which runs my workflow
const Workflows = Layer.mergeAll(EmailLayer).pipe(
  Layer.provide(ClusterWorkflowEngine.layer.pipe(Layer.provideMerge(runner)))
);

const finalLayer = Layer.mergeAll(shardManager, Workflows, WebLive).pipe(
  Layer.provide(PgLive)
);

BunRuntime.runMain(Layer.launch(finalLayer));


Sometimes the runner registers but sometimes it doesn't. I made a POC some time ago and I was running the cluster the following way which seemed to work perfectly:
const app = Effect.all(
  [
    Layer.launch(shardManager),
    Layer.launch(TestWorkflowLayer),
    Layer.launch(developersApi),
  ],
  {
    concurrency: "unbounded",
  }
).pipe(Effect.provide(pgClient));

BunRuntime.runMain(app);


Should I simply re-use the code from my POC or is there something I can do to gather everything together in a single layer and launch that layer?
Was this page helpful?