Is the queues product actually
Is the queues product actually production-ready? I added around 200k records for processing, and it lagged for almost 20 hours — jobs barely got picked up and it was completely unreliable. It’s been days and those records still never processed, so I ended up purging the queue.
Because of this, I’ve stopped using workers/queues altogether — my VPS handles the load way better. But if anyone from the queues/workers team wants to dig into this, let me know and I can share all the details.
7 Replies
Hm 200k messages should be no big deal, I regularly see Queues with significantly higher backlogs than this
What does your consumer do? Do you have concurrency set up? What about your batch sizes? I’m quite confident there’s something funky going on here (either on our side or yours), let’s investigate
Can you link your account to your Discord profile (use the /link command) and give me a queue ID?
My consumer just call aws kms to sign some data, do a db transactions(uses hyperdrive), and move an object from one r2 bucket to another.
Also the concurrency is set to defult and I have seen the queue stuck for more than 20 hours.
If you have a moment, please feel free to ping me and we can connect over a short call and I can show you how it failed for my use case.
Yeah I’d be interested. I’m in the US East time zone, let’s figure out a time that works for both of us? What time zone are you in?
I'm in IST timezone
How about 10AM EST, which would be 7:30PM for you on Friday (June 12)? If that doesn’t work, then same time Monday also works for me!
I can send you a Meet invite then
that works
we can connect on Friday
I’ll DM you