I am just using Queues for a few days. I suspect this is me hitting some rate limits? I also notice

I am just using Queues for a few days. I suspect this is me hitting some rate limits? I also notice a similar issue for Queue + D1.

Shihmin LeeOP•11/29/23, 1:59 PM

Although I doubt that I was hitting the rate limits since I am not seeing 429, also I only have 2 workers on 2 queue with 1 consumer each inserted like 1000 records over 5 mins on avg.

SShihmin Lee I am just using Queues for a few days. I suspect this is me hitting some rate li...

Matt Silverlock•11/29/23, 2:01 PM

Your issue appears identical to queues

Shihmin LeeOP•11/29/23, 2:01 PM

Gotcha. I've never seen Subrequest depth limit error message though.

SShihmin Lee Gotcha. I've never seen Subrequest depth limit error message though.

Matt Silverlock•11/29/23, 4:23 PM

Per that chat message, that error does not always manifest.

TTyler Benfield Is there any known degradation to queues in the last ~12 hours? We are seeing so...

Matt Silverlock•11/30/23, 1:14 PM

Not specific to the last 12 hours, but we have had a couple of reports about consumers not being invoked. We’re looking into the root cause.

TTyler Benfield Thanks @Matt! Let us know if there's any information we can provide. We're going...

Matt Silverlock•11/30/23, 1:36 PM

Note that it’s not related to concurrency - that just makes it more obvious.

If you have an account ID and queue ID we can add to our list.

MMatt Silverlock Per that chat message, that error does not always manifest.

Shihmin LeeOP•11/30/23, 3:33 PM

Hey Matt. So do you know if there's a timeline that this is going to be addressed? I'm interested in moving my infra from S3 to using R2, D1, Queues and KVs and such. But so far it's been harsh to see that I am getting so many 503s for combinations of kv, queue, d1, ... etc in the queue msg as well as all the immediate messages that got consumed.

SShihmin Lee Hey Matt. So do you know if there's a timeline that this is going to be addresse...

Matt Silverlock•11/30/23, 3:38 PM

We're working on it, but can't give an accurate timeline as it's non-trivial. Days to weeks, not months. Before Queues hits GA.

Shihmin LeeOP•11/30/23, 3:46 PM

Cool. That would be so nice.

Dj•11/30/23, 5:06 PM

Queues are awesome btw, I just found out about this product sometime last week. Using it to track some very dumb metrics from a Worker and it was very easy to figure out, both consumers and producers.

Dj•11/30/23, 5:07 PM

kian•11/30/23, 8:21 PM

Serializable object

kian•11/30/23, 8:22 PM

https://developers.cloudflare.com/queues/platform/javascript-apis/#queuescontenttype

TTyler Benfield Is there any known degradation to queues in the last ~12 hours? We are seeing so...

Arthur🍃•12/1/23, 1:49 AM

I have the same problem, my queue just stopped being consumed for no apparent reason

Alexander Blöndal•12/1/23, 2:16 PM

Hi! Is there any way of delaying a message, e.g. I want to this message to be consumed in 2 days?

TTyler Benfield @Matt do you see any potential issues with <:this:1010240499464024116> workaroun...

Matt Silverlock•12/1/23, 3:43 PM

I don't love the workaround (you shouldn't have to do this!) but there's no issues with it

AAlexander Blöndal Hi! Is there any way of delaying a message, e.g. I want to this message to be co...

Matt Silverlock•12/1/23, 3:44 PM

queues

Fun Miles•12/2/23, 11:21 PM

The docs (https://developers.cloudflare.com/workers/platform/pricing/) say queues worker get 15 minutes CPU times but my queue workers have gotten killed way before that (less than 5 minutes of wall time). There's a similar issue with Cron ones. Is this a known problem?

Pricing · Cloudflare Workers docs

Workers plans and pricing information.

RuthsPlace•12/4/23, 9:22 AM

having the same issue @Fun Miles . Any feedback?

FFun Miles The docs (https://developers.cloudflare.com/workers/platform/pricing/) say queue...

Matt Silverlock•12/4/23, 2:26 PM

What usage model is your Worker operating as? Also note, from the docs I linked before:

Matt Silverlock•12/4/23, 2:26 PM

I am being killed after 32,000ms. Is this a bug in the new billing?

32000ms = 30s (plus some leeway we provide)

Fun Miles•12/4/23, 2:30 PM

30s indeed but a query should go to 15 minutes, ie 900,000 ms.
My query job does D1 batched read requests with await, so most of that shouldn't count as CPU time and then some processing before some batched D1 writes.

FFun Miles 30s indeed but a query should go to 15 minutes, ie 900,000 ms. My query job does...

Matt Silverlock•12/4/23, 3:09 PM

Can you answer my other question? What usage model is your Worker running as? From the error you posted, your Queue consumer (I think?) is hitting the CPU limit posted above:

Queue schedule-retrieval (7 messages) - Exceeded CPU Limit @ 12/2/2023, 12:03:45 PM

Queue schedule-retrieval (7 messages) - Exceeded CPU Limit @ 12/2/2023, 12:03:45 PM

MMatt Silverlock Can you answer my other question? What usage model is your Worker running as? Fr...

Fun Miles•12/4/23, 3:10 PM

I don't understand what you mean by "model" then.

Fun Miles•12/4/23, 3:12 PM

It's the standard billing which says:

Fun Miles•12/4/23, 3:13 PM

As I've hit the limit both in Cron Trigger, then, after refactoring the code to divide in smaller queue works, the Queue Consumer, I am presuming there is a bug and it probably has the same root in both.

FFun Miles I don't understand what you mean by "model" then.

Matt Silverlock•12/4/23, 3:15 PM

https://developers.cloudflare.com/workers/platform/pricing/#how-to-switch-usage-model

your Worker will run under a usage model. You can check the dash. It will depend on when you created it.

Fun Miles•12/4/23, 3:17 PM

Standard model.

Fun Miles•12/4/23, 3:40 PM

RuthsPlace•12/4/23, 6:36 PM

For reference, the only way I could get more than 30 seconds was by
1) Moving from "standard" to "unbound"
2) Moving to "cron" trigger (I realize this is likely not helpful if you need queue but could work with my use case)
3) Adding this line to my wrangler.toml - workers_dev = false

RuthsPlace•12/4/23, 6:38 PM

You can see the success logs. The green ones are where the only change is to run under "unbound" usage model. The failed ones are where I run "standard" usage model.

RuthsPlace•12/4/23, 6:40 PM

Anyhow, just wanted to share in case it helps Cloudflare team diagnose issue. I don't think it's behaving as the documentation expresses it currently, unless I'm missing something

Ryan K•12/5/23, 12:05 AM

Hi can anyone help provide more context for this error I'm seeing when deploying workers with a queue:

Total Upload: 1810.83 KiB / gzip: 347.18 KiB
Uploaded dev-api-internal (2.44 sec)

✘ [ERROR] A request to the Cloudflare API (/accounts/2c/workers/queues/dev-queue/consumers/api-backend) failed.

  workers.api.error.queue_handler_missing [code: 11001]

Total Upload: 1810.83 KiB / gzip: 347.18 KiB
Uploaded dev-api-internal (2.44 sec)

✘ [ERROR] A request to the Cloudflare API (/accounts/2c/workers/queues/dev-queue/consumers/api-backend) failed.

  workers.api.error.queue_handler_missing [code: 11001]

kian•12/5/23, 12:10 AM

It's just saying that it doesn't detect a

queue

queue

function in your default export

Kkian It's just saying that it doesn't detect a `queue` function in your default expor...

Ryan K•12/5/23, 12:13 AM

I assumed that could be the case but I do indeed have a default export:

export default {
  fetch(request: Request, env: Env, ctx: ExecutionContext) {
    // Set env variables
    Object.keys(env).forEach((variable) => {
      process.env[variable] ??= `${env[variable]}`;
    })

    return fetch(request, env, ctx)
  },
  async queue(batch: MessageBatch<any>, env: Env): Promise<void> {
    let messages = JSON.stringify(batch.messages);
    console.log(`consumed from our queue: ${messages}`);
  },
}

export default {
  fetch(request: Request, env: Env, ctx: ExecutionContext) {
    // Set env variables
    Object.keys(env).forEach((variable) => {
      process.env[variable] ??= `${env[variable]}`;
    })

    return fetch(request, env, ctx)
  },
  async queue(batch: MessageBatch<any>, env: Env): Promise<void> {
    let messages = JSON.stringify(batch.messages);
    console.log(`consumed from our queue: ${messages}`);
  },
}

kian•12/5/23, 8:12 AM

They all should be available - have you tried them?

kian•12/5/23, 8:13 AM

I think that documentation aside is old, and should say within any handler context (fetch, scheduled, queue, email, etc)

Total Upload: 1810.83 KiB / gzip: 347.18 KiB Uploaded dev-api-internal (2.44 sec) ✘ [ERROR] A request to the Cloudflare API (/accounts/2c/workers/queues/dev-queue/consumers/api-backend) failed. workers.api.error.queue_handler_missing [code: 11001]

export default { fetch(request: Request, env: Env, ctx: ExecutionContext) { // Set env variables Object.keys(env).forEach((variable) => { process.env[variable] ??= `${env[variable]}`; }) return fetch(request, env, ctx) }, async queue(batch: MessageBatch<any>, env: Env): Promise<void> { let messages = JSON.stringify(batch.messages); console.log(`consumed from our queue: ${messages}`); }, }

I am just using Queues for a few days. I suspect this is me hitting some rate limits? I also notice

Similar Threads