Is there any plans on making it so CloudFlare queues works when running wrangler dev --remote, if so

TTobias Is there any plans on making it so CloudFlare queues works when running wrangler...

P

Pranshu Maheshwari•8/2/24, 2:32 PM

it's on our roadmap. we'll likely ship it in the next few months, but it's too soon to say exactly when

P

Ping for toast•8/4/24, 7:21 PM

How many fetch requests may I make from queues? I've found references to unbundled and bundled usage models, but these models are deprecated

PPing for toast How many fetch requests may I make from queues? I've found references to unbundl...

P

Pranshu Maheshwari•8/4/24, 9:29 PM

Queues consumers have the same limits as normal Workers (listed here: https://developers.cloudflare.com/workers/platform/limits/#subrequests)

Cloudflare Docs

Limits · Cloudflare Workers docs

Cloudflare Workers plan and platform limits.

PPranshu Maheshwari Queues consumers have the same limits as normal Workers (listed here: https://de...

P

Ping for toast•8/4/24, 9:30 PM

The limit for subrequests a Worker can make is 50 per request on the Bundled usage model or 1,000 per request on the Unbound usage model

Right, but aren't these models deprecated

P

Ping for toast•8/4/24, 9:30 PM

All users on the Workers Paid plan have been automatically migrated from the Bundled and Unbound usage models to the Standard usage model

P

Ping for toast•8/4/24, 9:31 PM

So I don't know what applies because I'm on the standard model

P

Ping for toast•8/4/24, 9:31 PM

ah, found it

PPing for toast So I don't know what applies because I'm on the standard model

P

Pranshu Maheshwari•8/4/24, 9:31 PM

standard / legacy paid plans have the same limits btw

P

Ping for toast•8/4/24, 9:31 PM

docs should be updated :)

PPranshu Maheshwari standard / legacy paid plans have the same limits btw 🙂

P

Ping for toast•8/4/24, 9:31 PM

cool, thanks

P

Ping for toast•8/4/24, 9:32 PM

thats a lot of requests, very exciting!!

M

mau•8/6/24, 8:51 AM

Getting weird issue with delays

const wait = getDelayInSeconds('14:12', 'Asia/Kolkata');
console.log(wait, 'wait');
await env.YOUR_QUEUE.send({ message: 'okay', date: new Date(), name: 'queue' }, { delaySeconds: wait });

const wait = getDelayInSeconds('14:12', 'Asia/Kolkata');
console.log(wait, 'wait');
await env.YOUR_QUEUE.send({ message: 'okay', date: new Date(), name: 'queue' }, { delaySeconds: wait });

const wait = getDelayInSeconds('14:12', 'Asia/Kolkata');
console.log(wait, 'wait');
await env.YOUR_QUEUE.send({ message: 'okay', date: new Date(), name: 'queue' }, { delaySeconds: wait });

const wait = getDelayInSeconds('14:12', 'Asia/Kolkata');
console.log(wait, 'wait');
await env.YOUR_QUEUE.send({ message: 'okay', date: new Date(), name: 'queue' }, { delaySeconds: wait });

wait has value of 85912

M

mau•8/6/24, 8:54 AM

error Error: Queue send failed: Internal Server Error
    at async Object.handler (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/src/index.ts:34:6)
    at async jsonError (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/node_modules/wrangler/templates/middleware/middleware-miniflare3-json-error.ts:22:10)
    at async drainBody (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/node_modules/wrangler/templates/middleware/middleware-ensure-req-body-drained.ts:5:10)
[wrangler-UserWorker:err] TypeError: Invalid statusText
    at HttpError.toResponse (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/node_modules/miniflare/src/workers/shared/router.worker.ts:16:10)
    at QueueBrokerObject.fetch (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/node_modules/miniflare/src/workers/shared/router.worker.ts:50:14)
    at async QueueBrokerObject.fetch (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/node_modules/miniflare/src/workers/shared/object.worker.ts:148:11)

error Error: Queue send failed: Internal Server Error
    at async Object.handler (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/src/index.ts:34:6)
    at async jsonError (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/node_modules/wrangler/templates/middleware/middleware-miniflare3-json-error.ts:22:10)
    at async drainBody (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/node_modules/wrangler/templates/middleware/middleware-ensure-req-body-drained.ts:5:10)
[wrangler-UserWorker:err] TypeError: Invalid statusText
    at HttpError.toResponse (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/node_modules/miniflare/src/workers/shared/router.worker.ts:16:10)
    at QueueBrokerObject.fetch (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/node_modules/miniflare/src/workers/shared/router.worker.ts:50:14)
    at async QueueBrokerObject.fetch (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/node_modules/miniflare/src/workers/shared/object.worker.ts:148:11)

error Error: Queue send failed: Internal Server Error
    at async Object.handler (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/src/index.ts:34:6)
    at async jsonError (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/node_modules/wrangler/templates/middleware/middleware-miniflare3-json-error.ts:22:10)
    at async drainBody (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/node_modules/wrangler/templates/middleware/middleware-ensure-req-body-drained.ts:5:10)
[wrangler-UserWorker:err] TypeError: Invalid statusText
    at HttpError.toResponse (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/node_modules/miniflare/src/workers/shared/router.worker.ts:16:10)
    at QueueBrokerObject.fetch (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/node_modules/miniflare/src/workers/shared/router.worker.ts:50:14)
    at async QueueBrokerObject.fetch (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/node_modules/miniflare/src/workers/shared/object.worker.ts:148:11)

error Error: Queue send failed: Internal Server Error
    at async Object.handler (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/src/index.ts:34:6)
    at async jsonError (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/node_modules/wrangler/templates/middleware/middleware-miniflare3-json-error.ts:22:10)
    at async drainBody (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/node_modules/wrangler/templates/middleware/middleware-ensure-req-body-drained.ts:5:10)
[wrangler-UserWorker:err] TypeError: Invalid statusText
    at HttpError.toResponse (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/node_modules/miniflare/src/workers/shared/router.worker.ts:16:10)
    at QueueBrokerObject.fetch (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/node_modules/miniflare/src/workers/shared/router.worker.ts:50:14)
    at async QueueBrokerObject.fetch (file:///Users/nicolethomas/Desktop/NTML/HomeworkHelp/throbbing-salad-88dd/node_modules/miniflare/src/workers/shared/object.worker.ts:148:11)

M

mau•8/6/24, 8:57 AM

I am trying to build a notifications system that allows users to configure the time for notifications, whether it's set for the end of the day, weekly, or at another interval.

any suggestions on how to do that?

Mmau I am trying to build a notifications system that allows users to configure the t...

B

boop•8/6/24, 10:44 AM

Durable object alarms?
https://developers.cloudflare.com/durable-objects/api/alarms/

Cloudflare Docs

Alarms · Cloudflare Durable Objects docs

Durable Objects alarms allow you to schedule the Durable Object to be woken up at a time in the future. When the alarm’s scheduled time comes, the …

M

mau•8/6/24, 10:46 AM

I’m using D1, so not sure I can use alarms. Until unless I implement some code which stores in durable objects and wake up at certain time and then pushes the data to d1.

Mmau I’m using D1, so not sure I can use alarms. Until unless I implement some code w...

H

Hard@Work•8/6/24, 10:53 AM

Create a DO for each notification the user sets, then set an alarm

M

mau•8/6/24, 10:58 AM

Just to clarify… you mean one do object per notification and then set the alarm?

M

mau•8/6/24, 10:58 AM

Is there is any limit on alarm time limits?

Mmau Just to clarify… you mean one do object per notification and then set the alarm?

H

Hard@Work•8/6/24, 11:19 AM

That is correct. As for limits for the time, as long as it is representable, it should be able to fire

M

mau•8/6/24, 11:20 AM

I will give it a try. Thanks for the solution

D

Dlanda Griffin•8/8/24, 4:17 PM

D

dsds10•8/8/24, 6:11 PM

Can somebody explain to me why there is a generous limit of 10k queues per account? Maybe I‘m missing something obvious, but only pull is supported over HTTP, push would still need a worker binding (there is a max of 500 workers per account, although you can bind multiple queues to a single worker iirc). The only explanation would be workers for platforms, but then kv namespace and r2 bucket limits won‘t match that idea

Ddsds10 Can somebody explain to me why there is a generous limit of 10k queues per accou...

M

Matt Silverlock•8/8/24, 8:52 PM

Many folks have more than one script and Workers for Platforms use-cases. One Worker script can effectively bind ~5000 or so resources. Other products have different metadata constraints.

MMatt Silverlock Many folks have more than one script _and_ Workers for Platforms use-cases. One ...

D

dsds10•8/9/24, 12:07 PM

got it thx

F

flyingshoes•8/12/24, 8:42 PM

Is there a limit to how many tasks I can push into a queue? I'm thinking of having a cron job push tasks into a queue, and the limit could be pretty high (potentially millions). would the limit be the 1200 requests/ 5 minutes?

Fflyingshoes Is there a limit to how many tasks I can push into a queue? I'm thinking of havi...

J

John Spurlock•8/12/24, 8:42 PM

the limits are documented here: https://developers.cloudflare.com/queues/platform/limits/

Cloudflare Docs

Limits · Cloudflare Queues

Many of these limits will increase during Queues’ public beta period. Follow our changelog or join the #queues-beta channel in our Developer Discord …

KKam I'm seeing queue `send` and `sendBatch` calls taking anywhere from ~250ms to ove...

B

Brett Willis•8/12/24, 8:47 PM

Can't comment on those latencies myself, but you could consider using

ctx.waitUntil()

ctx.waitUntil()

to more cheaply and easily defer those operations asynchronously without holding back the response.

JJohn Spurlock the limits are documented here: https://developers.cloudflare.com/queues/platfor...

F

flyingshoes•8/12/24, 8:58 PM

got it -- so 400 messages per second. thank you.

KKam I'm seeing queue `send` and `sendBatch` calls taking anywhere from ~250ms to ove...

P

Pranshu Maheshwari•8/12/24, 10:13 PM

it's a bit higher than expected, but not out of the ordinary. We're working on improvements that should improve latency for sending messages to a queue. Those improvements should ship by Oct.

KKam Thanks for the info Pranshu! Any idea what latency you are targeting with the im...

P

Pranshu Maheshwari•8/13/24, 3:11 PM

It's too soon to share specific targets, but stay tuned!

N

NoGameNoFun•8/14/24, 3:26 AM

Hi everyone, I like the worker and queue features in cloudflare, and been trying out about few weeks already.

I got a question, I noticed the consumer for the queue, after some time (like 2 days), it will stop processing the queue message. (the backlog size increase and doesnt get consume)

I attached the screenshot on the consumer setting, is this a bug or is just because of my consumer setting?

J

jordanfinners•8/14/24, 9:13 PM

Please can I provide some feedback/feature requests for cloudflare queues

Firstly the dev experience inside workers is pretty great! It would be awesome if it could work with Pages functions so I don't have to deploy separate consumers but thats a nit really.

The metrics in the dashboard are useful but its missing the main one for me, how many messages are in the queue at that point in time, especially important for deadletters
Redriving of deadletter messages, either a button in the UI or a way of doing it via HTTP or JS.
Currently you can pull the messages from the queue via HTTP but can't acknowledge them or push back into the queue locally, meaning I would need to deploy a worker just to redrive each queue which seems excessive and I need to control what I'm redriving or not.
I had to do 200 messages manually this morning
Alerts on the queue length, ie. If messages drop into a deadletter
Would be helpful if the error message was attached to the queue message in some way like SQS does
Being able to send multiple messages via the UI either file upload or pasting in
Documentation is missing for the Preview endpoint in the Queue REST API docs
Its only happened once but I've had one message not get sent, even though it said it has, which worries me slightly.
If possible increase the batch size, up to 200 would be helpful
Alongside this if possible to increase the timeout period as well, I'm running async reports and these can take several minutes to process and so they are timing out/hitting worker limits.

Happy to talk this through or provide more info if

Wwindygod9986 Does anyone know if cloudflare has any plans to support queue consumers on page...

M

Matt Silverlock•8/17/24, 12:38 AM

Not in the near term. Pages doesn’t have the concept of any non-request handler (cron, queues, tail)

L

lulupineapple•8/18/24, 2:06 PM

Hi, how do I deal with queue consumer timeout issue? i.e. make sure queue consumer completed the task with payload? Is there anyway to have queue retry calling consumer unless consumer explicitly acknowledge the payload was processed susessfully?

O

oof2win2•8/21/24, 12:21 PM

how can i set proper types for a queue?

O

oof2win2•8/21/24, 12:23 PM

like it seems that i need to change it on the generated env types but then i can't have them be generated

O

oof2win2•8/21/24, 12:45 PM

more importantly it doesn't work for receiving the messages via the same queue

Ooof2win2 more importantly it doesn't work for receiving the messages via the same queue

M

Matt Silverlock•8/21/24, 10:41 PM

The TypeScript types for the

queue

queue

queue

queue handler (your consumer) accept a type parameter - so

async queue<T>(batch: MessageBatch, env: Env)

async queue<T>(batch: MessageBatch, env: Env)

async queue<T>(batch: MessageBatch, env: Env)

async queue<T>(batch: MessageBatch, env: Env) - which will mean that each message in

MessageBatch

MessageBatch

MessageBatch

MessageBatch has a typed payload based on your type parameter

O

oof2win2•8/22/24, 5:55 AM

thanks!

S

snapo•8/22/24, 7:30 AM

Hey people!

When developing locally, I have configured my queue with following retry timeouts and counts:

max_retries = 10,  
retry_delay = 30

max_retries = 10,  
retry_delay = 30

max_retries = 10,  
retry_delay = 30

max_retries = 10,  
retry_delay = 30

I noticed that for some unknown reason, the retry logic only attempts up to 3 times. If I lower the retry_delay to 10 seconds, I can reach 6th attempt. If I finally lower the delay to 2 seconds, I can reach 10 retry attempts which is expected.

Any ideas, why the 30s delay is not working and 2 seconds is?

K

Kaique Nunes•8/22/24, 10:31 AM

it's possible to list messages inside dead queue?

KKaique Nunes it's possible to list messages inside dead queue?

S

snapo•8/22/24, 11:34 AM

You can do it via the dashboard. If you want to use http then you could configure a pull consumer for the dlq

Ssnapo You can do it via the dashboard. If you want to use http then you could configur...

K

Kaique Nunes•8/22/24, 11:59 AM

So I would need to implement my own api to retrieve the dlq messages?

S

snapo•8/22/24, 12:00 PM

No no. See here: https://developers.cloudflare.com/queues/configuration/pull-consumers/

Cloudflare Docs

Pull consumers | Cloudflare Queues

A pull-based consumer allows you to pull from a queue over HTTP from any environment and/or programming language outside of Cloudflare Workers. A pull-based consumer can be useful when your message consumption rate is limited by upstream infrastructure or long-running tasks.

Is there any plans on making it so CloudFlare queues works when running wrangler dev --remote, if so

Similar Threads

Similar Threads

Similar Threads