Handling long-running tasks triggered by HTTP requests can be tricky, especially when dealing wit...
I'm trying to use the http module, I have a long running task which need to happen when the request comes in. The long running task doesn't need to respond to the request, but does need to be triggered by it. I tried to deploy this using the queue module and nodejs events, with an event listener to trigger a separate section of code to start the long running task.
This didn't seem to work for me, I kept getting interrupt signals stopping the process early. I assume this is because of the way effect handles runtimes and the event listener or something. I have since tried implementing kafka, without much success (actually worse results than the first implementation) - and wanted to revert to this method but with a few tweaks.
Wondering if anyone had experienced a simular workflow and had any advice. I'm thinking about simply enqueuing the long task using the queue, then have a loop instead of a listener which just grabs the first item out of the queue, or waits 1 second and retries.
(Got discord muted, so please tag in any replies - thank you in advance)
This didn't seem to work for me, I kept getting interrupt signals stopping the process early. I assume this is because of the way effect handles runtimes and the event listener or something. I have since tried implementing kafka, without much success (actually worse results than the first implementation) - and wanted to revert to this method but with a few tweaks.
Wondering if anyone had experienced a simular workflow and had any advice. I'm thinking about simply enqueuing the long task using the queue, then have a loop instead of a listener which just grabs the first item out of the queue, or waits 1 second and retries.
(Got discord muted, so please tag in any replies - thank you in advance)
