Performance: Sequential queue processing instead of parallel (horizontal scaling)
I recently converted a piece of logic into multiple workers, hoping to improve performance drastically by separating the logic in simple repeatable workers.
Im processing a list of data and passing this to the next worker using Queue [binding].
What i'm seeing as that the queue is being processed message per message instead of in parallel => concurrent => horizontal scaling.
I'm having a hard time figuring this out.
I've tried:
- seperate messages
- batches
- batch sizes
Can anyone help me find the problem?
Im processing a list of data and passing this to the next worker using Queue [binding].
What i'm seeing as that the queue is being processed message per message instead of in parallel => concurrent => horizontal scaling.
I'm having a hard time figuring this out.
I've tried:
- seperate messages
- batches
- batch sizes
Can anyone help me find the problem?
Welcome to the official Cloudflare Developers server. Here you can ask for help and stay updated with the latest news
85,042Members
Resources
Recent Announcements
Similar Threads
Was this page helpful?
