47 replies

Hello, is there an issue with queues at

Hello, is there an issue with queues at the moment? I've got messages being reported as being in my deadletter, however I can't see them when listing messages and nothing else is reading from the queue

Pranshu Maheshwari•10/30/24, 7:34 PM

no general issues right now, but there might be a bug here! how are you seeing messages as reported in your DLQ? are you looking at the dashboard?

Pranshu Maheshwari•10/30/24, 7:37 PM

you might know this already but just in case, the dashboard shows the average backlog over the specified time period, not the current live backlog

jordanfinnersOP•10/31/24, 6:14 AM

Looking at the metrics from the queue which is showing messages should be in the deadletter

jordanfinnersOP•10/31/24, 6:14 AM

But then listing on the deadletter they do not show

jordanfinnersOP•10/31/24, 6:14 AM

I thought maybe it was some internal delay but it's been overnight and still nothing has appeared

jordanfinnersOP•10/31/24, 6:17 AM

jordanfinnersOP•10/31/24, 6:17 AM

jordanfinnersOP•11/1/24, 9:35 PM

@Pranshu Maheshwari I've checked again to see if there was any further delay and nothing has appeared so looks like messages haven't been delivered, is there anything you could check on your side please

jordanfinnersOP•11/3/24, 8:25 PM

@Pranshu Maheshwari this is definitely an issue across multiple of my queues. Please would you be able to help me with this as its impacting my business

jordanfinnersOP•11/3/24, 8:25 PM

jordanfinnersOP•11/3/24, 8:26 PM

100 have been reported as deadlettered here but I can't see those in my DLQ

jordanfinnersOP•11/3/24, 8:39 PM

Also I've got 2 messages in the DLQ that I can't acknowledge. Really not sure what's going on

jordanfinnersOP•11/4/24, 11:19 AM

Also raised this case 01257963 as its really impacting my business/service to my customers

Pranshu Maheshwari•11/4/24, 3:50 PM

Hey @jordanfinners, could you send me over your account ID & queue IDs?

Pranshu Maheshwari•11/4/24, 3:50 PM

both for your primary queue and your DLQ

jordanfinnersOP•11/4/24, 3:52 PM

Yeah it's affected multiple queues but I'll send the most pressing one

jordanfinnersOP•11/4/24, 3:52 PM

Queue ID b1368eea661b4d74bc8894e5dab8934b

jordanfinnersOP•11/4/24, 3:52 PM

DLQ a12fb186ec914f899d64e7044a284780

jordanfinnersOP•11/4/24, 3:53 PM

Account ID 3bfe7ad335003f84c4a9959ad57216de

jordanfinnersOP•11/4/24, 3:54 PM

Thank you! Sorry for chasing so much but had to manually recreate over 400 messages over the weekend while my customer was on a short timeline

Pranshu Maheshwari•11/4/24, 5:05 PM

Ofc! just to confirm, have I understood this right?
- You're sending messages to your primary queue, b1368eea661b4d74bc8894e5dab8934b
- There are 100 messages that were reported as delivered to your DLQ a12fb186ec914f899d64e7044a284780
- But a12fb186ec914f899d64e7044a284780 doesn't have those messages when you list them in the dashboard, and you're not able to acknowledge a couple of the messages that are listed there

Pranshu Maheshwari•11/4/24, 9:48 PM

By the way, as far as I can tell, a12fb186ec914f899d64e7044a284780 has several messages in the backlog. Several messages were written to that Queue on Nov 3 in particular.

But: it's possible there's a frontend bug on your account. We saw a few failed requests for the endpoints that our dashboard uses to preview & acknowledge messages in your acct

Pranshu Maheshwari•11/4/24, 9:55 PM

We found the bug: we're working on a fix! the bug is causing your dashboard view for Queue a12fb186ec914f899d64e7044a284780 to be "stuck" on two messages.

the rest of the messages are still in the DLQ, they're just not appearing on the dashboard. should be resolved once we have our fix deployed!

Pranshu Maheshwari•11/4/24, 9:55 PM

@jordanfinners

jordanfinnersOP•11/4/24, 10:02 PM

Yeah that's correct

jordanfinnersOP•11/4/24, 10:02 PM

It's happened in my reports queue as well

jordanfinnersOP•11/4/24, 10:02 PM

I assume this bug will fix it in all of them?

jordanfinnersOP•11/4/24, 10:03 PM

Also please could I ask for a send batch or deadletter redrive button in the UI as a future feature it would be a life saver

please

jordanfinnersOP•11/4/24, 10:03 PM

Thank you so much for investigating this for me! I'm glad I wasn't going mad

Jjordanfinners Also please could I ask for a send batch or deadletter redrive button in the UI ...

Pranshu Maheshwari•11/5/24, 3:33 PM

redriving messages is on the roadmap! it's planned for this quarter, although realistically it might get shipped early Q1 2025

Jjordanfinners I assume this bug will fix it in all of them?

Pranshu Maheshwari•11/5/24, 3:34 PM

yes, but just in case, could you send me the Queue IDs for the affected queues?

jordanfinnersOP•11/5/24, 4:07 PM

Queue ID 4965d3ae9ace413096703251979f14a5
DLQ 5c71988d2c8942c0b7eeccb143e0c696

Pranshu Maheshwari•11/18/24, 8:51 PM

Hey @jordanfinners, we pushed out a fix for this a few days ago! Could you let me know if it's working as expected now?

jordanfinnersOP•11/20/24, 9:38 PM

Hey, I can still only view the 2 failed items in the queue stuck on 2 messages

Pranshu Maheshwari•11/22/24, 4:03 PM

argh - let me track this down. sorry about this, must be very frustrating for you!

Todd@CF•11/25/24, 5:33 PM

@jordanfinners it looks like these are messages that were in a v1 queue and are stuck there after we migrated to v2 of queues (the current version). Have you connected a consumer to drain messages from the queue?

jordanfinnersOP•11/25/24, 5:45 PM

I haven't, I'll delete those 2 messages as I don't need them now. Then should that resolve it?

Todd@CF•11/25/24, 5:47 PM

It should, yeah, as messages should now be sent to your v2 queues. But if you still see an issue, please let us know.

jordanfinnersOP•12/4/24, 10:10 AM

Hey, I've deleted those messages now, but I can't see the new deadletter messages, will they take some time to come through?

jordanfinnersOP•12/5/24, 12:23 PM

Also not sure if its related but each time I try and view my queues I get this error

jordanfinnersOP•12/5/24, 12:23 PM

Looking through the console, its because service name is ""

Pranshu Maheshwari•12/6/24, 3:34 PM

I think we have a dashboard bug here - we'll sort this out. thank you!

jordanfinnersOP•12/31/24, 11:03 AM

Hey, I've checked my queues, as I thought I would give it a while incase of any caching etc.
But I still can't see any deadletter messages or even metrics for that queue now

jordanfinnersOP•12/31/24, 11:27 AM

Would it be possible to organise a call in the new year? So we can go through/debug this together, as it might be quicker/easier than back and forth, if possible? please

Pranshu Maheshwari•12/31/24, 6:24 PM

For sure - you seem to be hitting a bunch of edge cases somehow

thank you for working with us on this, really appreciate it

Pranshu Maheshwari•12/31/24, 6:25 PM

Would you like to get some time on my calendar here? https://calendar.app.google/YmGCcHW3GQXt8k2y5

Appointments

jordanfinnersOP•1/2/25, 9:57 AM

Thank you

jordanfinnersOP•1/2/25, 9:57 AM

Booked it!

Hello, is there an issue with queues at

Similar Threads

Similar Threads

Similar Threads