Any plans to support queue consumer workers running locally alongside a producer worker? The curren

Any plans to support queue consumer workers running locally alongside a producer worker? The current solution of putting the consumer code into the producer code to test it locally is... not ideal.

MMurder Chicken Any plans to support queue consumer workers running locally alongside a producer...

Hello, I’m Allie!•6/3/24, 1:59 PM

It should already? You mean Local Worker -> Local Queue -> Local Consumer?

Murder ChickenOP•6/3/24, 1:59 PM

Correct. I can't get it to work and the documentation explicitly calls out that it's not working.

MMurder Chicken Correct. I can't get it to work and the documentation explicitly calls out that...

Hello, I’m Allie!•6/3/24, 2:00 PM

Huh... I have it working for one app... Where does it say that?

Murder ChickenOP•6/3/24, 2:00 PM

MMurder Chicken Click to see attachment

Hello, I’m Allie!•6/3/24, 2:00 PM

Oh, separate....

Hello, I’m Allie!•6/3/24, 2:00 PM

Uh...

Hello, I’m Allie!•6/3/24, 2:00 PM

Right

Hello, I’m Allie!•6/3/24, 2:01 PM

Yeah, I guess that is just a wait until it is supported thing?

Murder ChickenOP•6/3/24, 2:01 PM

That's what I'm doing now but I was curious if it was a priority.

MMurder Chicken That's what I'm doing now but I was curious if it was a priority.

Hello, I’m Allie!•6/3/24, 2:02 PM

Based on the wording there, I would guess it is on the backlog, but not sure on the priority...

Murder ChickenOP•6/3/24, 2:07 PM

Local wrangler development with all of the features is literally one of the best features and simultaneously, one of the most confusing.

IIan I have a monorepo (managed by LernaJS) that has a cloudflare pages package and a...

johtso•6/4/24, 12:13 AM

did you figure out if using --persist-to in a monorepo is the right way to go?

Jjohtso did you figure out if using --persist-to in a monorepo is the right way to go?

Ian•6/4/24, 6:50 PM

I couldn't get it working. I wound up using a 3rd party queue service called QStash. However, this came with its own issues for doing local testing since now I need to use ngrok to create a local tunnel. You might be able to create a local tunnel using cloudflare tunnels as well.

Sohaib Muhammad•6/5/24, 11:17 AM

getting this when consumer worker get message from queue and called a service worker function "The destination execution context for this RPC was canceled while the call was still running."

kcbrewron•6/6/24, 1:26 AM

Are we still saying "soon" for general availability dates?

HHello, I’m Allie!Based on the wording there, I would guess it is on the backlog, but not sure on ...

Todd@CF•6/6/24, 3:23 PM

Yeah this is exactly correct. Running queues locally with Wrangler dev is not fully supported yet, but it is on the roadmap.

lukas•6/6/24, 10:33 PM

Huh, my consumers trigger locally?

Llukas Huh, my consumers trigger locally?

Ian•6/7/24, 5:51 AM

I think he means for Pages. From what I can tell, queues work with a workers project.

SSohaib Muhammad getting this when consumer worker get message from queue and called a service wo...

Marwan•6/8/24, 1:08 AM

غربي ؟

Rohit Verma•6/9/24, 6:52 AM

is there a way we can get the r2 file created event in the queues?

jordanfinners•6/9/24, 7:55 AM

Would it be possible to get a redrive button in the dashboard for pushing deadletter messages back onto the original queue? Please

ancs21•6/10/24, 2:08 AM

I see limit of delay message is 12 hours, can request increase this? Thanks.

ancs21•6/10/24, 2:10 AM

My team have plan move from Upstash to Queues, Upstash QStash support max delay is 7 days, also PAYG is 1 year.

ajgeiss0702•6/11/24, 6:59 PM

does the

delaySeconds

delaySeconds

option in

sendBatch

sendBatch

apply to each message in the batch? Or once for the entire batch.

E.g. if I set the delaySeconds to a minute (60) and have 5 messages in the batch, will it wait a minute and execute all 5 messages, or wait a minute and execute one, wait another minute and execute another, etc

ajgeiss0702•6/11/24, 7:03 PM

please ping me if you reply

ajgeiss0702•6/11/24, 7:14 PM

also, does local development not support the

delaySeconds

delaySeconds

option? If I try using it in local dev, I get these:

✘ [ERROR] Uncaught (in response) Error: Queue send failed: Internal Server Error

✘ [ERROR] Uncaught (in response) Error: Queue send failed: Internal Server Error

[wrangler:inf] GET /__scheduled 500 Internal Server Error (1904ms)
[wrangler-UserWorker:err] TypeError: Invalid statusText

[wrangler:inf] GET /__scheduled 500 Internal Server Error (1904ms)
[wrangler-UserWorker:err] TypeError: Invalid statusText

Aajgeiss0702 does the `delaySeconds` option in `sendBatch` apply to each message in the batch...

Sam•6/11/24, 9:24 PM

All will be executed after 60 seconds, depending on the settings of your consumer

Aancs21 My team have plan move from Upstash to Queues, Upstash QStash support max delay ...

Matt Silverlock•6/12/24, 5:27 AM

What are you delaying for 1+ year in the future? Curious as to how you also track those “pending” messages with QStash?

Aajgeiss0702 also, does local development not support the `delaySeconds` option? If I try usi...

Matt Silverlock•6/12/24, 5:28 AM

What version of wrangler are you using?

Isaac•6/12/24, 8:27 AM

Hey, I'm seeing a severe delay (up to 20 mins) in messages being consumed by my worker after (roughly) 5pm BST. Is there any way I can fix this on my side?

Isaac•6/12/24, 8:30 AM

(it's near instant at 9am BST)

MMatt Silverlock What version of wrangler are you using?

ajgeiss0702•6/12/24, 1:40 PM

I’m not sure off the top of my head (I can check in a bit), but I also tried running the dev command with

npx wrangler@latest dev —test-scheduled

npx wrangler@latest dev —test-scheduled

and the same thing happened

Aajgeiss0702 I’m not sure off the top of my head (I can check in a bit), but I also tried run...

Matt Silverlock•6/12/24, 1:47 PM

—test-scheduled

—test-scheduled

Aajgeiss0702 I’m not sure off the top of my head (I can check in a bit), but I also tried run...

Matt Silverlock•6/12/24, 1:47 PM

The last two times this came up it was because folks were not using a later version of wrangler - e.g. see the note here: https://developers.cloudflare.com/queues/configuration/batching-retries/#delay-messages

MMatt Silverlock `—test-scheduled` is not related to Queues: https://developers.cloudflare.com/wo...

ajgeiss0702•6/12/24, 1:48 PM

I know, but things are added to the queue via a cron trigger

MMatt Silverlock The last two times this came up it was because folks were not using a later vers...

ajgeiss0702•6/12/24, 1:49 PM

I would assume that using the command I sent (with wrangler@latest) that it would always use the latest available version of wrangler?

ajgeiss0702•6/12/24, 1:53 PM

looks like the wrangler in my repo was outdated, but not that outdated. It was on

3.58.0

3.58.0

(where the latest appears to e

3.60.2

3.60.2

npx wrangler@latest dev

npx wrangler@latest dev

did appear to run with

3.60.2

3.60.2

. The weird thing is that when I updated it in that repo, it seems to be working now

RRohit Verma is there a way we can get the r2 file created event in the queues?

Todd@CF•6/12/24, 7:07 PM

Are you looking for something like Event Notifications? https://developers.cloudflare.com/r2/examples/upload-logs-event-notifications/

Cloudflare Docs

Log and store upload events in R2 with event notifications · Cloudf...

This example provides a step-by-step guide on using event notifications to capture and store R2 upload logs in a separate bucket.

deniska_10000_10000•6/14/24, 9:35 AM

Hey guys, playing with the CF workers and stuff and got a question on queues
Sending message to a queue takes 500ms. In rare cases it’s 200s, but usually 500ms+. I mean this code takes 500ms:

const start = Date.now();
await queue.send({ timestamp: new Date() });
console.log('--->', Date.now() - start);

const start = Date.now();
await queue.send({ timestamp: new Date() });
console.log('--->', Date.now() - start);

Is it always like this or it’s just temporary system issues?

Sam•6/14/24, 3:29 PM

The list messages in queue option don't seem to work correctly. It always returns 4 items, even when more than 100 are in the queue. So specifying 50, will return 4. Specifying 2, will return 4, etc...

Sam•6/14/24, 4:22 PM

Oh wait, the value of "max_batch_size" in wrangler is being used in the dashboard?!? That's pretty annoying.

Fabio•6/16/24, 11:05 PM

Is it possible for a route that calls instead of calling the worker to call the queue that will call the worker?
Because if it is necessary to call a worker on the route that adds it to the queue and this queue calls another worker to process it, I will have 2x the number of executions

FFabio Is it possible for a route that calls instead of calling the worker to call the ...

Matt Silverlock•6/17/24, 12:40 PM

Can you rephrase? Can you show what you are trying to do?

MMatt Silverlock Can you rephrase? Can you show what you are trying to do?

Fabio•6/17/24, 12:55 PM

I have a route that calls a worker that makes the same request to a lambda. What I wanted to do was use the Queue, but I thought I could use it directly in the route, but in reality from what I understand I will have to create an 'add to queue' worker, with a route trigger that receives the request data and then adds it to the Queue, which itself will call another Worker that will make the request to the lambda.
Would that be it? Because then each request will have 2x more requests to workers

FFabio I have a route that calls a worker that makes the same request to a lambda. What...

Matt Silverlock•6/17/24, 1:18 PM

You need a Worker to write to a Queue, yes. But you can consume directly from a Lambda - https://developers.cloudflare.com/queues/configuration/pull-consumers/

Cloudflare Docs

Cloudflare Queues - Pull consumers · Cloudflare Queues

A pull-based consumer allows you to pull from a queue over HTTP from any environment and/or programming language outside of Cloudflare Workers. A …

Sohaib Muhammad•6/20/24, 7:40 AM

hi, need to know that is there a way to clear or flush queue so that any pending events not gets fired

SSohaib Muhammad hi, need to know that is there a way to clear or flush queue so that any pending...

Matt Silverlock•6/20/24, 2:38 PM

Not currently, no. You can delete the queue / recreate it.

Sohaib Muhammad•6/20/24, 3:00 PM

Thanks @elithrar , can we create/recreate programmatically, if yes than after recreation is it auto connected with the consumers and producers ?

Any plans to support queue consumer workers running locally alongside a producer worker? The curren

Similar Threads

Similar Threads

Similar Threads