the issue with queues is theres the subrequest limit of 50 reqs

Ran across a new

.send

.send

error today: Error: Queue send failed: ConflictError: Queue send failed: Conflict

Is this something we should consider transient and immediately retry?

Ooof2win2 the issue with queues is theres the subrequest limit of 50 reqs

oof2win2OP•4/20/24, 5:31 PM

i mean worst case i send data to a queue to be distributed to individual queues…. it sounds horrendous as i would be making my own pubsub system essentially but it would work

avsaase•4/20/24, 8:26 PM

We are also waiting for pubsub. It would make a lot of things much simpler.

JJohn Spurlock Ran across a new `.send` error today: `Error: Queue send failed: Conflict` Is t...

Matt Silverlock•4/20/24, 8:39 PM

Investigating. You shouldn't get that at all.

Elliot Hesp•4/22/24, 9:42 AM

Hi All,

We have a queue setup (one message per batch), where say 50 or so items are created every 60 seconds.

The handler for the queue simply sends a fetch request to an endpoint, and if the response is not a 200 we trigger an alert.

Every so often one of these fetch requests seems to hang (until the worker times out). We added in some abort controller logic to cancel the fetch after 5 seconds.

Interestingly, we actually see the request coming in to handle the healthcheck which resolves in way under the 5 seconds timeout.

Does anyone know why this might happen? I know there's a limit on concurrent fetch requests from a worker at once, but I'd have assumed each queue invocation is it's own and doesn't have this limit?

JJohn Spurlock Ran across a new `.send` error today: `Error: Queue send failed: Conflict` Is t...

Matt Silverlock•4/24/24, 1:20 PM

Did you ever see a repeat of this?

MMatt Silverlock Did you ever see a repeat of this?

John Spurlock•4/24/24, 3:56 PM

Conflict

Conflict

~~I've only seen once~~. Still getting Error: Queue send failed: Internal Server ErrorError: Queue send failed: Internal Server Error occasionally on

.send

.send

, which calling-worker-side retrying does not seem to fix

LordSilver•4/25/24, 2:33 PM

Will you always required Paid sub to use Queues or this thing is subject to change?

LLordSilver Will you always required Paid sub to use Queues or this thing is subject to chan...

Matt Silverlock•4/25/24, 3:41 PM

Subject to change, but need to think through failure modes, limits, and more work to enforce limits

LordSilver•4/25/24, 4:04 PM

i cannot even find the page where to upgrade to Paid plan...

LordSilver•4/25/24, 4:04 PM

clicking this brings me to the overview page

LLordSilver clicking this brings me to the overview page

Matt Silverlock•4/25/24, 4:08 PM

LordSilver•4/25/24, 4:08 PM

the link was bringing me to overview somehow

JJohn Spurlock any guidance on what to do when receiving `Error: Queue send failed: Internal Se...

carlos-alberto•4/25/24, 6:23 PM

Hey @John Spurlock , did you ever get an answer to this? We're seeing these errors more and more frequently and can't find any guidance on how to handle.

Ccarlos-alberto Hey @John Spurlock , did you ever get an answer to this? We're seeing these erro...

John Spurlock•4/25/24, 7:06 PM

not yet - currently i retry a few times on the caller (worker) side, and then give up and send the message to a DO for sending later

sometimes, but not always, sending to the DO also fails (with something like

Network connection lost

Network connection lost

) - which is troubling since that is now lost payload.

I'm pretty sure this is just a function of Queues still being beta, and cf will try to make sure that

.send

.send

does whatever it needs to do in tough situations to retry under the hood if necessary (like saving to disk when the network is down and retrying later)

JJohn Spurlock not yet - currently i retry a few times on the caller (worker) side, and then gi...

carlos-alberto•4/25/24, 7:15 PM

Thank you, quite furstrating but yeah, hopefully will improve when it moves to GA.

Dalist•4/26/24, 1:20 PM

Cloudflare UI is really buggy, there's so many bugs going on

luna•4/26/24, 11:40 PM

could i have some help understanding what all of the stats mean? im a little confused.

luna•4/26/24, 11:43 PM

docs have a broken link https://developers.cloudflare.com/queues/reference/how-queues-works/#producers

Lluna docs have a broken link https://developers.cloudflare.com/queues/reference/how-q...

John Spurlock•4/27/24, 1:00 AM

good catch, I believe those docs are open - I just submitted a quick fix PR: https://github.com/cloudflare/cloudflare-docs/pull/14284/files

luna•4/27/24, 3:12 AM

i can't find any docs on this. what is the limit of messages

.sendBatch()

.sendBatch()

can send?

Lluna i can't find any docs on this. what is the limit of messages `.sendBatch()` can ...

John Spurlock•4/27/24, 3:40 AM

https://developers.cloudflare.com/queues/configuration/javascript-apis/#queue

JJohn Spurlock https://developers.cloudflare.com/queues/configuration/javascript-apis/#queue

luna•4/27/24, 3:44 AM

any chance we can get these docs and/or a link to docs pages like this in the jsdocs for the types? would make it a LOT easier than having to go digging into the docs for this.

Praefe•4/27/24, 5:15 AM

Why do queues bind via their names rather than their ids :NotLikeThis:

PPraefe Why do queues bind via their names rather than their ids :NotLikeThis:

luna•4/27/24, 5:26 AM

https://tenor.com/view/damn-these-good-ass-questions-good-questions-k-camp-good-ass-questions-questions-gif-12501849

Tenor

Lluna could i have some help understanding what all of the stats mean? im a little con...

luna•4/28/24, 10:31 AM

any chance i could get some help with this?

Baya•4/29/24, 12:23 AM

Is there a way to clear a backlog from queues? I have huge consumer delay (6000 seconds).

oof2win2OP•4/29/24, 6:11 PM

can i send messages to a queue without having it bound to a worker? like a http api for the queue? and additionally an option to send the same message to multiple queues

Ooof2win2 can i send messages to a queue without having it bound to a worker? like a http ...

John Spurlock•4/29/24, 11:55 PM

there is no http api for sending messages other than using a worker: https://developers.cloudflare.com/queues/examples/publish-to-a-queue-over-http/

you could write a worker with multiple named queue bindings that sent the same input message to n queues using

Promise.allSettled

Promise.allSettled

inside the worker, for example

oof2win2OP•4/30/24, 6:43 AM

i wanted to have a queue assigned on a per-user basis

oof2win2OP•4/30/24, 6:43 AM

but fair enough i guess, thanks

timmm•5/3/24, 2:21 AM

Is local dev possible with the producer in worker one and the consumer in worker two?

timmm•5/3/24, 2:21 AM

I'm using next-on-pages for worker one to produce the message and I'm not receiving it on worker two

timmm•5/3/24, 2:22 AM

Admittidely, this is the first time I've tried starting up two workers so maybe I'm not doing it right. But my understanding is you can start them both up and bindings should work

timmm•5/3/24, 2:29 AM

On nevermind! I see you can't have queues across workers atm in local dev

Praefe•5/3/24, 5:37 AM

When

wrangler deploy

wrangler deploy

ing multiple times quickly with queue config changes, the queue consumer config (and producer) page would crash with internal issues, and if I reaccessed it, the data hadn't been updated.

So the first time, it worked fine, but then the next times, while the command ran successfully, the web UI didn't reflect the config changes.

Praefe•5/3/24, 5:38 AM

Lead to my coworker thinking that the retry_delay option didn't work in wrangler.toml, but in reality, it wasn't being reflected in UI

Ashley•5/3/24, 11:09 AM

Q: the DLQ, if it has an active consumer, how long do the messages stay in the DLQ? Indefinitely? Until they fail a certain amount of time? A set time frame? (Docs doesn't seem to specify)

Tthecj247 Are there any issues with ACK via Python Request? Can pull messages just fine, i...

Matt Silverlock•5/5/24, 4:56 AM

@Rob could you have Todd & team look into this? The error message on a HTTP 500 should at least be clearer. The payload looks fine though? https://developers.cloudflare.com/queues/configuration/pull-consumers/#4-acknowledge-messages

Cloudflare Docs

Cloudflare Queues - Pull consumers · Cloudflare Queues

A pull-based consumer allows you to pull from a queue over HTTP from any environment and/or programming language outside of Cloudflare Workers. A …

John Spurlock•5/5/24, 4:53 PM

Shout out to the team: it's been over a week now since I've seen a single

Conflict

Conflict

Internal Server Error

Internal Server Error

prevent a

.send

.send

call (over 9 million sends in that time)

hob8yte•5/6/24, 2:59 PM

I'm seeing messages just sit and hang in a queue despite having concurrency enabled and a consumer worker attached. What steps can I take to debug this?

oof2win2OP•5/7/24, 8:30 PM

can i get local queues (only with wrangler) working without a paid cloudflare account? i want to use them for some local testing and dont want everyone on the team to require a paid cloudflare account

Ooof2win2 can i get local queues (only with wrangler) working without a paid cloudflare ac...

Hello, I’m Allie!•5/7/24, 8:34 PM

It should work fine out of the box

oof2win2OP•5/7/24, 8:34 PM

how do i enable r2 notifications on it though

oof2win2OP•5/7/24, 8:34 PM

is that supported on local dev?

the issue with queues is theres the subrequest limit of 50 reqs

Similar Threads

Similar Threads

Similar Threads