But yea, just reverted back to our old setup now, and immediately we return to no errors and latenci

But yea, just reverted back to our old setup now, and immediately we return to no errors and latencies return back to normal ranges. Not a single request >2 seconds

ZackOP•4/11/25, 10:40 AM

ZZack But yea, just reverted back to our old setup now, and immediately we return to n...

lambrospetrou•4/11/25, 10:49 AM

The latency spikes you see is probably because those fail. Anyway, we will check the errors and see what's going on with your database.

gruntlord6•4/11/25, 12:29 PM

With the read replicas being a thing, is it possible for things like the export to not lock up the db?

Or is that not possible due to how the infrastructure is setup?

gruntlord6•4/11/25, 12:29 PM

I know we can work around it by copying to a staging database first but I figured I'd ask

gruntlord6•4/11/25, 1:20 PM

Can't you just revert?

Brayden•4/11/25, 1:23 PM

@Roopendra I've mentioned this to the team just an FYI – sorry you're experiencing this!

Brayden•4/11/25, 1:24 PM

Oh good! Is your database back up now?

Ggruntlord6 With the read replicas being a thing, is it possible for things like the export ...

lambrospetrou•4/11/25, 1:24 PM

That's the plan, but not yet. Locking is still happening at the moment.

lambrospetrou•4/11/25, 1:28 PM

FYI, disabling replicas is not instant. If replicas are created and process some of your queries, it can take a few hours for them to stop processing queries and completely disappear.

Also, if the UI is slow sometimes, you can use the PATCH API directly to manage read replication: https://developers.cloudflare.com/api/resources/d1/subresources/database/methods/edit/

The read replication mode for the database. Use 'auto' to create replicas and allow D1 automatically place them around the world, or 'disabled' to not use any database replicas (it can take a few hours for all replicas to be deleted).

Also, as I mentioned yesterday our recommendation is to first try it in your staging databases or test environments and once verified that it works for your case enable it in production.

Cloudflare API | D1 › Database › Update D1 Database Partially

Interact with Cloudflare's products and services via the Cloudflare API

gruntlord6•4/11/25, 1:46 PM

In general I would not use a beta in prodution unless I have verified its working with test projects

gruntlord6•4/11/25, 1:46 PM

I know its not helpful after the fact, but figured I would mention that

RRoopendra Yeah, my fault.

Brayden•4/11/25, 1:50 PM

I'm adding a beta label to the UI to make it more clear. We should have mentioned it's still in beta in our UI as well.

gruntlord6•4/11/25, 1:59 PM

yea, it was clear in the announcement from what I recall but UI would be nice for those who don't read that diretcly or missed that part

RRoopendra Looks like it came back up again. Thank you for your help!

lambrospetrou•4/11/25, 2:15 PM

@Roopendra FYI, we disabled replication for your database altogether for now, so even if you enable it replicas won't be created. We will remove the switch on Monday, so that you can manage it again yourself.

Llambrospetrou @Roopendra FYI, we disabled replication for your database altogether for now, s...

Milan•4/11/25, 2:27 PM

In the meantime though it would be a good idea to turn replication off for that DB if you haven't already @Roopendra

Nikolas•4/11/25, 5:06 PM

Hello, we have the exact same issue. Queries fail, dashboard fails, can't turn off replication even with the API, it returns this

{"errors":[{"code":7500,"message":"internal error; reference = e_PVSMDp_a7bf1e15234b4277b9f32a19516fdc30"}],"success":false,"messages":[],"result":null}

{"errors":[{"code":7500,"message":"internal error; reference = e_PVSMDp_a7bf1e15234b4277b9f32a19516fdc30"}],"success":false,"messages":[],"result":null}

the database id is 943219ad-705a-4a47-a87d-9b56e1b8e7ea
Thank you

Nikolas•4/11/25, 5:09 PM

Also I was wondering why nothing is mentioned in cloudflarestatus.com

Nikolas•4/11/25, 5:09 PM

is it not affecting everyone?

NNikolas Hello, we have the exact same issue. Queries fail, dashboard fails, can't turn o...

Milan•4/11/25, 6:41 PM

Are you sending a bunch of requests at once? Just curious

Milan•4/11/25, 6:42 PM

What happens if you send a single request, then wait a minute or so

MMilan Are you sending a bunch of requests at once? Just curious

Nikolas•4/11/25, 6:43 PM

what kind of requests? For the api to disable replication or query requests?

Milan•4/11/25, 6:43 PM

A request to your DB

Nikolas•4/11/25, 6:45 PM

its hooked up to Workers as an api for our app. And all the db related requests fail

NNikolas its hooked up to Workers as an api for our app. And all the db related requests ...

Milan•4/11/25, 6:46 PM

Your production DB is down?

Nikolas•4/11/25, 6:46 PM

well, yes

Nikolas•4/11/25, 6:46 PM

since we enabled replication

Milan•4/11/25, 6:47 PM

okay, we'll disable replication on our end for your account, one sec

Nikolas•4/11/25, 6:48 PM

Thanks. I think you should just disable it for everyone until you fix this. Because it locks up the whole database and its options

Milan•4/11/25, 6:50 PM

Is your DB working?

Nikolas•4/11/25, 6:51 PM

now it works, yes

Nikolas•4/11/25, 6:51 PM

thanks again

Milan•4/11/25, 6:51 PM

I want to stress this again. Do not deploy beta products to your production service. It's in beta for a reason. @lambrospetrou can we pin this message somewhere?

Nikolas•4/11/25, 6:51 PM

but it didnt say beta when we enabled it...

Nikolas•4/11/25, 6:52 PM

we had no idea you launched this yesterday

Nikolas•4/11/25, 6:52 PM

and it said it would just improve the db latency so why not enable it?

Original message was deleted

Isaac McFadyen•4/11/25, 6:55 PM

IIRC that's newly added, it wasn't there at launch.

Nikolas•4/11/25, 6:55 PM

yes

But yea, just reverted back to our old setup now, and immediately we return to no errors and latenci

Similar Threads