Hey. Mind sending your account ID (it's fine to make it public, it's not secret info) and I can see

Hey. Mind sending your account ID (it's fine to make it public, it's not secret info) and I can see if I can escalate this?

Isaac McFadyenOP•12/6/24, 4:51 AM

Given that a user was having the same issue the other day there's likely a bug in the Queues dashboard.

IIsaac McFadyen Hey. Mind sending your account ID (it's fine to make it public, it's not secret ...

James•12/6/24, 4:52 AM

sent thank you

JJames I cannot load any queues on my cloudflare dashboard, the dashboard site breaks w...

Isaac McFadyenOP•12/6/24, 3:42 PM

Hey - this is being investigated by the team, thanks for the report.

JJames I cannot load any queues on my cloudflare dashboard, the dashboard site breaks w...

Pranshu Maheshwari•12/6/24, 9:24 PM

We've found the issue here: the fix should roll out soon!

Walshy•12/6/24, 9:52 PM

Should be out and fixed

kimjongillest•12/12/24, 3:01 AM

is the best way to segregate HTTP access for queues to manage secrets within a worker?

eg: client calls a normal worker rest endpoint and passes the secret, then the worker does a switch statement to correctly forward the message to a queue while checking the secret against that queue name using a map?

Aries•12/17/24, 6:44 PM

Is there a way to de-dupe queue items, ie, have a unique-by key, which will ensure it is in the queue only once?

Aries•12/17/24, 6:45 PM

optimally, you'd be able to specific a key when writing a message.

sup filistine•1/5/25, 1:55 AM

maybe a short question on queues: x-posting from workers-help: workers-and-pages-helpQueues are not scaling up, consumer wise when using a regular (non pull consumer)

joezief•1/8/25, 4:29 PM

is there a good example anywhere of testing retry mechanism of queue consumer using vitest?

jacky•1/9/25, 9:33 AM

guys i got over an hour of consumer delay, is this normal?

slagathor•1/10/25, 4:39 PM

I currently have a queue setup where the consumer worker initiates a CSV download from an API, continues to poll the API until the CSV is available, downloads the CSV, and then stores it in R2.

Right now only 2 messages are getting sent to the queue at a time (via a cron job at the top of every hour), but only the first one ever gets processed (it's processed to completion).

Could this be a CPU limit issue? I don't see any error logs. These are my current .toml settings:

max_batch_size = 1
max_retries = 0

max_batch_size = 1
max_retries = 0

Suman!•1/11/25, 6:04 AM

Hi guys, is there a way i can export all the messages from my dead letter queue? Or may be to list all the messages
I am unable to get to read the latest messages from the dashboard since there are 4-500s of messages.

ViVi•1/11/25, 6:43 AM

In the tunnel, I tried to configure public host name for a localhost:port. Everything worked flawlessly.
However, when I tried to configure a https://username.subdomain.usbx.me/jellyfin:8096, it didn't work at all.
Where did I miss on the config process?

John Spurlock•1/11/25, 10:47 PM

hmm, I'm stumped, getting this 403 error message from the api when trying to pull messages from a particular queue: messages cannot be pulled unless http_pull mode is enabledmessages cannot be pulled unless http_pull mode is enabled

the queue has a single consumer of type http_pullhttp_pull, which I thought was the key to defining "http_pull mode":

{
  "queue_id": "8c720189e5c54eefa03f53eefb7162f1",
  "queue_name": "queue2-ci-dlq",
  "settings": {},
  "created_on": "2024-03-10T18:25:36.141403Z",
  "modified_on": "2024-03-10T18:25:36.141403Z",
  "producers_total_count": 0,
  "producers": [],
  "consumers_total_count": 1,
  "consumers": [
    {
      "settings": {
        "batch_size": 10,
        "max_retries": 100,
        "visibility_timeout_ms": 30000,
        "retry_delay": 0
      },
      "consumer_id": "d2a3644fd6cc40fab4e023ce65c451dc",
      "type": "http_pull"
    }
  ]
}

{
  "queue_id": "8c720189e5c54eefa03f53eefb7162f1",
  "queue_name": "queue2-ci-dlq",
  "settings": {},
  "created_on": "2024-03-10T18:25:36.141403Z",
  "modified_on": "2024-03-10T18:25:36.141403Z",
  "producers_total_count": 0,
  "producers": [],
  "consumers_total_count": 1,
  "consumers": [
    {
      "settings": {
        "batch_size": 10,
        "max_retries": 100,
        "visibility_timeout_ms": 30000,
        "retry_delay": 0
      },
      "consumer_id": "d2a3644fd6cc40fab4e023ce65c451dc",
      "type": "http_pull"
    }
  ]
}

tried removing/re-adding the pull consumer, double-checked the token has edit-queue permission, anyone else seen this before?

JJohn Spurlock hmm, I'm stumped, getting this 403 error message from the api when trying to pul...

ajgeiss0702•1/12/25, 4:17 PM

Have you tried enabling it via wrangler?

https://developers.cloudflare.com/queues/configuration/pull-consumers/#wrangler-cli

Cloudflare Docs

Cloudflare Queues - Pull consumers · Cloudflare Queues

A pull-based consumer allows you to pull from a queue over HTTP from any environment and/or programming language outside of Cloudflare Workers. A pull-based consumer can be useful when your message consumption rate is limited by upstream infrastructure or long-running tasks.

Aajgeiss0702 Have you tried enabling it via wrangler? https://developers.cloudflare.com/queu...

John Spurlock•1/12/25, 7:10 PM

yes, and through the dashboard - the config seems correct, it's similar to a prod dlq queue in that same account which works fine - it's just that queue 8c720189e5c54eefa03f53eefb7162f18c720189e5c54eefa03f53eefb7162f1 that gives messages cannot be pulled unless http_pull mode is enabledmessages cannot be pulled unless http_pull mode is enabled. I even tried previewing and deleting the 3 messages in there, didn't help

xintercept•1/15/25, 12:37 AM

anyone else run into problems with queues today? we hit a bunch of "Queue send failed: Service Temporarily Unavailable" around 2025-01-14T19:51:06.333Z

Xxintercept anyone else run into problems with queues today? we hit a bunch of "Queue send f...

xintercept•1/15/25, 12:38 AM

(doesn't look like anything was reported on https://www.cloudflarestatus.com/ related to Qs either)

Cloudflare Status

Welcome to Cloudflare's home for real-time and historical data on system performance.

chhatra_•1/15/25, 6:52 AM

Hello , I am building some application with cloudflare queues and worker . My consumer worker code checks if the system is free then it sends message from the queue to the system else the queue retries after cetain time 1200s .the queue retries the message but after delayed time has passed messages , the message is not processed from the queue but when I make a new request to the queue then the new message and previous message is processed ?

chhatra_•1/15/25, 6:53 AM

How to fix this ? what could be the causes for it . I want the consumer to automatically process the message after the retry time has passed

chhatra_•1/15/25, 7:04 AM

Does anyone know what settings should check ?

chhatra_•1/16/25, 3:42 AM

Actually , the queue was working fine previously . I noticed multiple repeatative requests from diffrent user agent so I applied code to filter that out. After that this happen, I commented out the code that filter it out on the basis of user agent , still it got the same problem. I had lost my original source code when my laptop broke down. I noticed to many requests on my queue so I made changes using the GUI interface provided by cloudflare. Directly modifying the compiled code . Can not using wrangler cli be a cause of this problem ?

Cchhatra_Hello , I am building some application with cloudflare queues and worker . My co...

Rakhim•1/16/25, 12:14 PM

Have you considered using Workflows instead? It feels better suited for retries/fallback scenarios, you can define the retry and backoff policy per workflow, and it can also sleep for long periods of time.

kcbrewron•1/17/25, 4:51 PM

Not sure this is the right channel but nothing for notifications that I saw. I would love to see the ability to configure notifications when my dlq reaches a certain threshold without having to query the metrics API constantly

allen•1/18/25, 11:49 AM

Hi
I'm using Cloudflare Workers and I'm passing messages between them using Queues. Within a batch of messages the code iterates over the messages in a for loop and after processing each message it explicitly acks the message, eg message.ack(). Despite this the message gets redelivered and the queue fills up with messages until they run out of retries.

ack'ing the batch doesn't work either (batch.ackAll()) and not acking at all and just returning also has no effect. So for now I've set the queus to have max_retries = 0 and no DLQ. Is there something obvious maybe that I've missed?

I'm going to head out now to try and enjoy the rest of my Saturday

sadel•1/18/25, 5:45 PM

would be amazing if you guys could add an extra configuration to queues for max dispatches / second so that we can fully wrap 3rd party apis and stay within rate limits

Cchhatra_Hello , I am building some application with cloudflare queues and worker . My co...

sup filistine•1/19/25, 11:56 AM

i saw this same behavior yesterday. curious what’s up. i didn’t pass an explicit retry delaySeconds for the message, just an empty .retry(), and was hoping it will respect the retry_delay configured under the wrangler consumer section. will do some more tests today

Skrrrrrrrrr•1/19/25, 7:55 PM

is there bug with the Turkey server?

TetraEtc•1/23/25, 12:55 AM

hey all, I've seen this before but can't find them right now.I'm looking for a good example repo for the following setup:
1. Publisher worker that recieves API request with data, and publishes the data to a queue. (Similar to this: https://developers.cloudflare.com/queues/examples/publish-to-a-queue-over-http/)
2. Consumer worker that takes individual (not batch) messages, and performs logic with them (specifically doing some API calls)

I'm also wanting it to retry the consumer twice, before sending to a dead letter queue that does stuff with the message data.

Brett Willis•1/23/25, 5:25 AM

I've had to disconnect the queue consumer in my application to effectively pause a certain processing pipeline while I sort out a bug.
As the messages accumulate in the queue without a consumer, will they safely stay around until I resume (until the 96hr retention period)?

rivage•1/23/25, 2:00 PM

Hey there, my Queue has had a concurrency of 3 for the last 12 hours. Failure rate is low. Maximum concurrency is 250. Any ideas on how to increase this?

pat•1/26/25, 12:12 PM

Hey all, wondering if theres a way to set a limit on the number of requests to a worker over a given period of time before it starts rejecting new requests? If I can avoid implementing it myself that'd be great

flof.fly•1/26/25, 8:22 PM

How do I increase the max concurrency workers to 500 (the maximum) when polling messages from Queue?

Rahul gangotri•1/27/25, 8:19 PM

I am running a browser rendering queue consumer which takes data and start the crawling of the mentioned link

If i start dev env, it says browser does not work in dev

And in remote queue is not working

So I decided to deploy but now there is an error and i am not able to get my console logs and any log on dashboard of worker also

Log page of worker just crashes constantly

Simon Colby•1/29/25, 4:36 PM

Hey guys - it's my first time building backend with cloudflare workers and I'm having a hard time with understanding queues behaviour. As you can see on my screenshot the deliverability is tragic, but the service in my mobile app works. What can be a reason?

SSimon Colby Hey guys - it's my first time building backend with cloudflare workers and I'm h...

Matt Silverlock•1/30/25, 12:54 PM

Messages are delivered to the DLQ when they reach the configured retry limit for the consumer. Without a DLQ configured, messages that reach the retry limit are deleted permanently.

https://developers.cloudflare.com/queues/configuration/dead-letter-queues/

your consumer is likely throwing exceptions over and over. Have you checked the logs?

Cloudflare Docs

Dead Letter Queues · Cloudflare Queues

A Dead Letter Queue (DLQ) is a common concept in a messaging system, and represents where messages are sent when a delivery failure occurs with a consumer after max_retries is reached. A Dead Letter Queue is like any other queue, and can be produced to and consumed from independently.

Mitya•1/30/25, 2:55 PM

Hi - I'm new to queues. Just made one, testing locally with

--remote

--remote

, and got:

Queues are currently in Beta and are not supported in wrangler dev remote mode.

Given this, is there no way for me to test in development? I need

--remote

--remote

for various other reasons; testing without that flag isn't an option.

MMitya Hi - I'm new to queues. Just made one, testing locally with `--remote`, and got:...

Stefan Stührmann•1/30/25, 4:05 PM

I'm halfway finished with the implementation of local queues using durable objects in miniflare. This will allow you to have local persistent queues that can be shared between multiple workers instead of the in memory array based queues and therefore you can test distributed / microservice architectures using queues fully locally. May I ask why you need --remote? For me the biggest issue is the lack of local queues between multiple workers (the main purpose of queues).

SStefan Stührmann I'm halfway finished with the implementation of local queues using durable objec...

Mitya•1/30/25, 5:32 PM

Thanks. I need remote for two reasons, mainly. Firstly, and for some unknown reason, my Worker refuses to connect to my remote PG DB (even though I can connect to it on my machine via terminal). The only workaround was to use the remote flag and access via a hyperdrive. This seems to be a problem unique to me and my DNS or network or whatever. Secondly, I use CF AI.

Mitya•1/30/25, 5:33 PM

Is there a preferred way to simulate queues locally with the remote flag? Sort of like the __schedule__schedule query string arg for testing CRON schedules locally.

Mitya•1/31/25, 10:49 PM

Question. The docs say consumers get 30 seconds of CPU time. I'm trying to understand what this means practically. If my consumer is sent a batch of 10 messages, and each of those messages needs sending to e.g. an AI analysis API, is that likely to max out the 30 seconds of CPU time? How can one reliably predict CPU time for a given task?

MMitya Question. The docs [say](https://developers.cloudflare.com/queues/platform/limit...

Isaac McFadyenOP•1/31/25, 10:52 PM

?cpu-time

SuperHelpflare•1/31/25, 10:52 PM

CPU Time: the amount of work the CPU needs to do, measured in msms. More CPU-heavy things will increase it (for example, hashing passwords). This is not the same as the time a

fetch

fetch

request takes.
Wall Time: the actual amount of time a Worker runs for, as though you measured it with a clock or your browser DevTools

Isaac McFadyenOP•1/31/25, 10:52 PM

CPU time is not wall time.

Isaac McFadyenOP•1/31/25, 10:52 PM

If you're making external requests then you're unlikely to get even near 30s.

Isaac McFadyenOP•1/31/25, 10:52 PM

More likely you'd be in the tens of ms total.

Hey. Mind sending your account ID (it's fine to make it public, it's not secret info) and I can see

Similar Threads

Similar Threads

Similar Threads