Seeking Advice on Migrating Background Event Processing from Rails to Effect
Hey everyone,
I’m currently migrating an application from Rails to Effect , and I’d love to get some suggestions regarding background event processing.
In the existing Rails setup, we have separate workers that consume messages from AWS SQS and process events.
I’m considering moving this logic to AWS Lambda, which would consume messages from SQS and make API requests to my server for further processing.
However, I’m unsure if this is the best approach.
An alternative is to consume messages directly from the server itself. But I see one issue with that , during new deployments, ongoing background jobs might get interrupted if the old server process is terminated.
One workaround would be to have some mechanism to inform the server about a new deployment so it can gracefully stop polling from SQS before shutting down , but I’m not sure if that’s worth the added complexity. is this a real concern, or am I looking at it the wrong way?
. Also is it recommended to use Effect/cluster for these kind of event processing jobs
I’m currently migrating an application from Rails to Effect , and I’d love to get some suggestions regarding background event processing.
In the existing Rails setup, we have separate workers that consume messages from AWS SQS and process events.
I’m considering moving this logic to AWS Lambda, which would consume messages from SQS and make API requests to my server for further processing.
However, I’m unsure if this is the best approach.
An alternative is to consume messages directly from the server itself. But I see one issue with that , during new deployments, ongoing background jobs might get interrupted if the old server process is terminated.
One workaround would be to have some mechanism to inform the server about a new deployment so it can gracefully stop polling from SQS before shutting down , but I’m not sure if that’s worth the added complexity. is this a real concern, or am I looking at it the wrong way?
. Also is it recommended to use Effect/cluster for these kind of event processing jobs
