Need help running queue jobs in worker concurrently
Hey there
i’ve got a background job that i run for every user, and each job takes at least ~90 seconds wall time. what i want is to run them for all users concurrently, not sequentially.
so right now, for testing, i’m adding one queue message per user (say 10 users). i set max_batch_size = 1 and max_concurrency = 50 on the queue. i was expecting cloudflare to spin up multiple consumer workers so jobs don’t block each other.
but when i look at logs, it still runs sequentially like one finishes then next starts. i’m new to cloudflare workers so maybe i’m missing something about how queues concurrency actually works.
any tips on what i’m doing wrong or how to force it to run each job fully isolated/concurrent?
i’ve got a background job that i run for every user, and each job takes at least ~90 seconds wall time. what i want is to run them for all users concurrently, not sequentially.
so right now, for testing, i’m adding one queue message per user (say 10 users). i set max_batch_size = 1 and max_concurrency = 50 on the queue. i was expecting cloudflare to spin up multiple consumer workers so jobs don’t block each other.
but when i look at logs, it still runs sequentially like one finishes then next starts. i’m new to cloudflare workers so maybe i’m missing something about how queues concurrency actually works.
any tips on what i’m doing wrong or how to force it to run each job fully isolated/concurrent?
Welcome to the official Cloudflare Developers server. Here you can ask for help and stay updated with the latest news
85,042Members
Resources
Similar Threads
Was this page helpful?
