How do you approach long async types of operations on t3/next/vercel?

Say I need to run a process that takes 5 minutes, it gets data from DB, does some calculations, generates a CSV and drops it in s3.
4 Replies
oadster
oadster17mo ago
There are a couple of ways I know of, one is with Upstash https://docs.upstash.com/redis/tutorials/job_processing
Job Processing and Event Queue with Serverless Redis | Upstash: Doc...
This tutorial shows how to use Upstash Redis for job/task processing.
oadster
oadster17mo ago
The other is with trigger dev it is in beta / alpha now but you can apply and get access https://trigger.dev/
Trigger.dev - The Background Jobs framework for Next.js
Create long-running Jobs directly in your codebase with features like API integrations, webhooks, scheduling and delays.
oadster
oadster17mo ago
Otherwise you can make or use an external long running service and use a webhook between the service and next js when the job finishes
Grey
Grey17mo ago
A relatively inconvenient option: I run a separate node/go service and expose it as a private API. This just entirely decouples the delegation of long running jobs from the primary front-end stack.
Want results from more Discord servers?
Add your server