Hi everyone — I have a question, and I’m
Hi everyone — I have a question, and I’m not sure if this is the right place to ask. I’m trying to understand which Cloudflare product would be the best fit for my use case.
I’m building an application that, when a user clicks a button, needs to trigger an external process that runs independently from the app itself.
In my view, this process should follow three main steps:
1. Fetch data from the database
2. Call an LLM many times in parallel, maximizing throughput within a limited execution time
3. Write the results back to the database
This entire operation should happen outside the main application, which will later display the updated data once the process is completed. The goal is to run this background workflow efficiently and reliably on Cloudflare’s infrastructure.
I’m considering different ways to orchestrate these jobs:
(A) The application inserts items into a queue on Supabase and then triggers a process that reads and processes the queue.
(B) The queue itself automatically triggers the processing when new messages arrive, though I’m concerned this might rely on polling, which doesn’t seem the most elegant or efficient approach.
(C) The application sends the data directly for processing at the time of activation, without using any intermediate queue.
Each user can initiate this process multiple times in parallel, so concurrency, execution duration, and timeout management are important considerations.
Question: Based on these requirements, which Cloudflare products or combination of products would be best suited to implement this architecture?
If this isn’t the right place to ask — or if Workers / Workers AI aren’t the right tools for this kind of problem — could you please point me to the best channel or community to ask this question?
Thanks a lot!
I’m building an application that, when a user clicks a button, needs to trigger an external process that runs independently from the app itself.
In my view, this process should follow three main steps:
1. Fetch data from the database
2. Call an LLM many times in parallel, maximizing throughput within a limited execution time
3. Write the results back to the database
This entire operation should happen outside the main application, which will later display the updated data once the process is completed. The goal is to run this background workflow efficiently and reliably on Cloudflare’s infrastructure.
I’m considering different ways to orchestrate these jobs:
(A) The application inserts items into a queue on Supabase and then triggers a process that reads and processes the queue.
(B) The queue itself automatically triggers the processing when new messages arrive, though I’m concerned this might rely on polling, which doesn’t seem the most elegant or efficient approach.
(C) The application sends the data directly for processing at the time of activation, without using any intermediate queue.
Each user can initiate this process multiple times in parallel, so concurrency, execution duration, and timeout management are important considerations.
Question: Based on these requirements, which Cloudflare products or combination of products would be best suited to implement this architecture?
If this isn’t the right place to ask — or if Workers / Workers AI aren’t the right tools for this kind of problem — could you please point me to the best channel or community to ask this question?
Thanks a lot!
Welcome to the official Cloudflare Developers server. Here you can ask for help and stay updated with the latest news
85,500Members
Resources
Similar Threads
Was this page helpful?
