My team have plan move from Upstash to Queues, Upstash QStash support max delay is 7 days, also PAYG

My team have plan move from Upstash to Queues, Upstash QStash support max delay is 7 days, also PAYG is 1 year.

ajgeiss0702•6/11/24, 6:59 PM

does the delaySecondsdelaySeconds option in sendBatchsendBatch apply to each message in the batch? Or once for the entire batch.

E.g. if I set the delaySeconds to a minute (60) and have 5 messages in the batch, will it wait a minute and execute all 5 messages, or wait a minute and execute one, wait another minute and execute another, etc

ajgeiss0702•6/11/24, 7:03 PM

please ping me if you reply

ajgeiss0702•6/11/24, 7:14 PM

also, does local development not support the delaySecondsdelaySeconds option? If I try using it in local dev, I get these:

✘ [ERROR] Uncaught (in response) Error: Queue send failed: Internal Server Error

✘ [ERROR] Uncaught (in response) Error: Queue send failed: Internal Server Error

[wrangler:inf] GET /__scheduled 500 Internal Server Error (1904ms)
[wrangler-UserWorker:err] TypeError: Invalid statusText

[wrangler:inf] GET /__scheduled 500 Internal Server Error (1904ms)
[wrangler-UserWorker:err] TypeError: Invalid statusText

Aajgeiss0702 does the `delaySeconds` option in `sendBatch` apply to each message in the batch...

Sam•6/11/24, 9:24 PM

All will be executed after 60 seconds, depending on the settings of your consumer

Aancs21 My team have plan move from Upstash to Queues, Upstash QStash support max delay ...

Matt Silverlock•6/12/24, 5:27 AM

What are you delaying for 1+ year in the future? Curious as to how you also track those “pending” messages with QStash?

Aajgeiss0702 also, does local development not support the `delaySeconds` option? If I try usi...

Matt Silverlock•6/12/24, 5:28 AM

What version of wrangler are you using?

Isaac•6/12/24, 8:27 AM

Hey, I'm seeing a severe delay (up to 20 mins) in messages being consumed by my worker after (roughly) 5pm BST. Is there any way I can fix this on my side?

Isaac•6/12/24, 8:30 AM

(it's near instant at 9am BST)

MMatt Silverlock What version of wrangler are you using?

ajgeiss0702•6/12/24, 1:40 PM

I’m not sure off the top of my head (I can check in a bit), but I also tried running the dev command with npx wrangler@latest dev —test-schedulednpx wrangler@latest dev —test-scheduled and the same thing happened

Aajgeiss0702 I’m not sure off the top of my head (I can check in a bit), but I also tried run...

Matt Silverlock•6/12/24, 1:47 PM

—test-scheduled—test-scheduled is not related to Queues: https://developers.cloudflare.com/workers/runtime-apis/handlers/scheduled/

Aajgeiss0702 I’m not sure off the top of my head (I can check in a bit), but I also tried run...

Matt Silverlock•6/12/24, 1:47 PM

The last two times this came up it was because folks were not using a later version of wrangler - e.g. see the note here: https://developers.cloudflare.com/queues/configuration/batching-retries/#delay-messages

MMatt Silverlock `—test-scheduled` is not related to Queues: https://developers.cloudflare.com/wo...

ajgeiss0702•6/12/24, 1:48 PM

I know, but things are added to the queue via a cron trigger

MMatt Silverlock The last two times this came up it was because folks were not using a later vers...

ajgeiss0702•6/12/24, 1:49 PM

I would assume that using the command I sent (with wrangler@latest) that it would always use the latest available version of wrangler?

ajgeiss0702•6/12/24, 1:53 PM

looks like the wrangler in my repo was outdated, but not that outdated. It was on 3.58.03.58.0 (where the latest appears to e 3.60.23.60.2). npx wrangler@latest devnpx wrangler@latest dev did appear to run with 3.60.23.60.2. The weird thing is that when I updated it in that repo, it seems to be working now

RRohit Verma is there a way we can get the r2 file created event in the queues?

Todd@CF•6/12/24, 7:07 PM

Are you looking for something like Event Notifications? https://developers.cloudflare.com/r2/examples/upload-logs-event-notifications/

Cloudflare Docs

Log and store upload events in R2 with event notifications · Cloudf...

This example provides a step-by-step guide on using event notifications to capture and store R2 upload logs in a separate bucket.

deniska_10000_10000•6/14/24, 9:35 AM

Hey guys, playing with the CF workers and stuff and got a question on queues
Sending message to a queue takes 500ms. In rare cases it’s 200s, but usually 500ms+. I mean this code takes 500ms:

const start = Date.now();
await queue.send({ timestamp: new Date() });
console.log('--->', Date.now() - start);

const start = Date.now();
await queue.send({ timestamp: new Date() });
console.log('--->', Date.now() - start);

Is it always like this or it’s just temporary system issues?

Sam•6/14/24, 3:29 PM

The list messages in queue option don't seem to work correctly. It always returns 4 items, even when more than 100 are in the queue. So specifying 50, will return 4. Specifying 2, will return 4, etc...

Sam•6/14/24, 4:22 PM

Oh wait, the value of "max_batch_size" in wrangler is being used in the dashboard?!? That's pretty annoying.

Fabio•6/16/24, 11:05 PM

Is it possible for a route that calls instead of calling the worker to call the queue that will call the worker?
Because if it is necessary to call a worker on the route that adds it to the queue and this queue calls another worker to process it, I will have 2x the number of executions

FFabio Is it possible for a route that calls instead of calling the worker to call the ...

Matt Silverlock•6/17/24, 12:40 PM

Can you rephrase? Can you show what you are trying to do?

MMatt Silverlock Can you rephrase? Can you show what you are trying to do?

Fabio•6/17/24, 12:55 PM

I have a route that calls a worker that makes the same request to a lambda. What I wanted to do was use the Queue, but I thought I could use it directly in the route, but in reality from what I understand I will have to create an 'add to queue' worker, with a route trigger that receives the request data and then adds it to the Queue, which itself will call another Worker that will make the request to the lambda.
Would that be it? Because then each request will have 2x more requests to workers

FFabio I have a route that calls a worker that makes the same request to a lambda. What...

Matt Silverlock•6/17/24, 1:18 PM

You need a Worker to write to a Queue, yes. But you can consume directly from a Lambda - https://developers.cloudflare.com/queues/configuration/pull-consumers/

Cloudflare Docs

Cloudflare Queues - Pull consumers · Cloudflare Queues

A pull-based consumer allows you to pull from a queue over HTTP from any environment and/or programming language outside of Cloudflare Workers. A …

Sohaib Muhammad•6/20/24, 7:40 AM

hi, need to know that is there a way to clear or flush queue so that any pending events not gets fired

SSohaib Muhammad hi, need to know that is there a way to clear or flush queue so that any pending...

Matt Silverlock•6/20/24, 2:38 PM

Not currently, no. You can delete the queue / recreate it.

Sohaib Muhammad•6/20/24, 3:00 PM

Thanks @elithrar , can we create/recreate programmatically, if yes than after recreation is it auto connected with the consumers and producers ?

SSohaib Muhammad Thanks @elithrar , can we create/recreate programmatically, if yes than after re...

Matt Silverlock•6/20/24, 3:04 PM

No, you'd need to rebind it (in both cases)

Sohaib Muhammad•6/20/24, 3:12 PM

thanks @elithrar , highly appreciated

John Spurlock•6/20/24, 5:29 PM

Ah a new error msg variant noticed today: Error: Queue send failed: Service Temporarily UnavailableError: Queue send failed: Service Temporarily Unavailable

gonna assume this is client-side retryable

JJohn Spurlock Ah a new error msg variant noticed today: `Error: Queue send failed: Service Tem...

Matt Silverlock•6/20/24, 11:08 PM

Looks like that was during an incident (not Queues specific) today: https://www.cloudflarestatus.com/incidents/p7l6rrbhysck

Intermittent Connectivity Issue

ajgeiss0702•6/22/24, 5:40 PM

is there a reason that in the dasboard, there are 1-3 datapoints between the real data that is 0?

ajgeiss0702•6/22/24, 5:42 PM

basically i have a cron job that runs every 3 hours and adds a bunch of items to the queue, delayed to be spread over the next 3 hours. For some reason, the dashboard displays that there are 0 delayed messages between these "real" datapoints

Sohaib Muhammad•6/24/24, 7:44 AM

@ajgeiss0702 are you sending messages using send() method with delayed options OR you are using sendBatch() for sending all at once and use a global configuration for delaying

a5000•6/24/24, 2:02 PM

Hi there! Do you all have any recommendations on local dev for an api worker that takes requests, puts them in a queue, and the consumes the queue items with workers.ai. I know queues are beta and not fully supported by wrangler. Also I believe there's a limitation with local queues where they have to be in the same

wrangler

wrangler

project, so I can't run 2 node projects pointing at the same local queue.

I'm unable to spin up the project if I have a queue producer and workers ai enabled.

ajgeiss0702•6/24/24, 5:51 PM

I use send() to apply a different delay to each one (so that they’re spread out across those 3 hours)

Alec | Kriebel LLC.•6/25/24, 3:55 AM

I recently got an error when calling a sendBatch: Queue sendBatch failed: Internal Server ErrorQueue sendBatch failed: Internal Server Error

Does anyone have any details on what this error is or when it happens?

AAlec | Kriebel LLC.I recently got an error when calling a sendBatch: `Queue sendBatch failed: Inter...

Matt Silverlock•6/25/24, 1:21 PM

This is a Cloudflare internal error (e.g. no specifics to share). You can typically retry - are you seeing that repeat?

MMatt Silverlock This is a Cloudflare internal error (e.g. no specifics to share). You can typica...

Alec | Kriebel LLC.•6/25/24, 3:02 PM

No, it was a one time occurrence from what I can see in my logs. Wondering if there's anything us developers can do to minimize the chances of this happening

MMatt Silverlock This is a Cloudflare internal error (e.g. no specifics to share). You can typica...

SteffTek•6/25/24, 4:03 PM

tbh this can be a neck breaker for my application... well i think i have to teach the users to respam their request lol

SSteffTek tbh this can be a neck breaker for my application... well i think i have to teac...

Matt Silverlock•6/25/24, 4:06 PM

You can (should) retry in your producer Worker. There’s no case where 100% of requests succeed. Assume 99.9% success - if you have 1M queue sends, and .1% fail, that’s still up to 1k failures to potentially retry.

Matt Silverlock•6/25/24, 4:06 PM

You can retry in your code when you use a try-catch - no need to expose those failures directly to users.

MMatt Silverlock You can (should) retry in your producer Worker. There’s no case where 100% of re...

SteffTek•6/25/24, 4:08 PM

my api does allow rescheduling, i might thry the try catch, thx! i never had this problem until now. I had another one where my DLQ wouln't resend, because i forgor await

SSteffTek my api does allow rescheduling, i might thry the try catch, thx! i never had thi...

Matt Silverlock•6/25/24, 4:12 PM

Note that this isn’t specific to us - any API can fail intermittently at scale. Catching and retrying (before eventually failing) is a common strategy.

SteffTek•6/25/24, 4:12 PM

yeah, i know that ^^

Arthur🍃•6/25/24, 7:31 PM

Hi, I've had a worker for at least 8 months and it was working fine, but today I'm getting this error, does anyone know if there was any change in the queue?

AArthur🍃Hi, I've had a worker for at least 8 months and it was working fine, but today I...

Matt Silverlock•6/25/24, 8:05 PM

No changes to Queues APIs - are you seeing more than once occurrence of this?

MMatt Silverlock No changes to Queues APIs - are you seeing more than once occurrence of this?

Arthur🍃•6/25/24, 8:17 PM

Hello, yes, my queue simply stopped working, all requests were failing and throwing this error, my alternative was to create a new queue and connect it to my worker, after doing this the worker started working correctly again without throwing any more errors.

Note, I didn't make any changes to the code, I simply moved to a new queue and things started working again

AArthur🍃Hello, yes, my queue simply stopped working, all requests were failing and throw...

Matt Silverlock•6/25/24, 8:17 PM

Can you share your accountId and the (previous) queueId? Both are safe to share publicly.

Arthur🍃•6/25/24, 8:19 PM

This queue is very strange, I can't even look at the messages inside it

Arthur🍃•6/25/24, 8:19 PM

Arthur🍃•6/25/24, 8:20 PM

My account: c120f6c2efb6750841790c9355c6abe9

My team have plan move from Upstash to Queues, Upstash QStash support max delay is 7 days, also PAYG

Similar Threads

Similar Threads

Similar Threads