Without specifying your workload (volume + kind of queries), location of source and target, and late

Without specifying your workload (volume + kind of queries), location of source and target, and latency needs, no benchmark would really make sense for you.

Llambrospetrou Without specifying your workload (volume + kind of queries), location of source ...

Matthew•4/14/25, 9:47 PM

I see, you're right. But it would generally help if we knew which one would be better for what type of use cases..

Llambrospetrou Without specifying your workload (volume + kind of queries), location of source ...

lambrospetrouOP•4/14/25, 9:48 PM

For example, if all your users are in EEUR/WEUR, and you use Hyperdrive with a Postgres database on a 4GB host in Frankfurt, that's probably going to perform amazingly well, and probably better than a single D1 database.
If you use tens of D1 databases spread around close to each of your user, and you have traffic from everywhere, then the comparion is more nuanced.

jed•4/15/25, 3:00 AM

is there a performance penalty for using D1 via the REST API from a worker, as opposed to using the Workers Binding API? or is the binding just a wrapper around the REST API?

Jjed is there a performance penalty for using D1 via the REST API from a worker, as o...

Cyb3r-Jak3•4/15/25, 3:04 AM

Yes. All requests have to go back to Cloudflare's core, so request latency is far from optimal from some locations. You would also be subject to the API Rate limit of 1200 requests per 5 minutes.

CCyb3r-Jak3 Yes. All requests have to go back to Cloudflare's core, so request latency is fa...

jed•4/15/25, 3:05 AM

awesome, thank you!

Jjed is there a performance penalty for using D1 via the REST API from a worker, as o...

lambrospetrouOP•4/15/25, 7:33 AM

Yes, at the moment we don't really recommend using the REST API as part of the hot path of your application.
We are working on making that path faster this quarter, but using the D1 Binding through a worker will always be the fastest since you are going directly to your database.

IIsaac McFadyen Correct on both accounts, yeah. Hopefully in the future there will be a workarou...

_Ultima_•4/15/25, 10:16 AM

So for now, bind e.g. 100 D1 instances manually to a worker and use KV to make sure the tenant is connecting to the correct D1 instance. (making sure the tenants are spread equally across the D1 instances)

FFlikTeoh I'm actually doing select 1 on every page load that uses d1 though 😬 helps to r...

Zack•4/15/25, 11:32 AM

I have considered doing this even for cached requests, just to keep things warm. Around 60% of our requests that would otherwise invoke the database are cache hits, including the other requests too, it's more like 85%

Zack•4/15/25, 11:32 AM

So could quite easily just do SELECT 1 everywhere and have a much warmer database

Zack•4/15/25, 11:33 AM

I don't like hacks like this though, so hopefully replication improves after this round of bug fixes and the ongoing improvements through the open beta period

FFlikTeoh Yeah thats what I was hoping to see from the replication. But still even with th...

lambrospetrouOP•4/15/25, 12:11 PM

same d1 could sometimes be in the 180ms range, or the 500ms range

Is this variance for the same query, from the same worker in the same location, for the same database?
Is this with or without replication?

FFlikTeoh Without replication, everything else is the same. Could be 180ms this hour, then...

lambrospetrouOP•4/15/25, 12:16 PM

Is it consistent for those periods though? Will it be mostly 180-200ms this hour, and 480-500ms 2 hours later, or do you see variance regardless?
If you are using the Workers binding, and your workers are roughly from the same location, then latency should be quite similar.

FFlikTeoh I haven’t keep track of the hours but I do notice sometimes it’s in 180-200ms ra...

lambrospetrouOP•4/15/25, 12:23 PM

If the locations of the caller are different the latency will be different too. Anyway, if you have more concrete data, we can revisit.

FFlikTeoh Oh and sorry. I did turn on replication in the dashboard but have not yet implem...

lambrospetrouOP•4/15/25, 12:24 PM

If you have replication enabled, even if you don't use the sessions API your queries will flow through replicas and forwarded to the primary. So, the performance could be different than when you were going straight to the single primary. Again, depending on your caller's location.

FFlikTeoh This is still a project under development. So basically I was the only person ac...

1984 Ford Laser•4/15/25, 9:40 PM

What colo are your requests being served from, and is that your closest location?

FFlikTeoh It says APAC, and yes that should be the closest location

lambrospetrouOP•4/16/25, 9:23 AM

APAC unfortunately is our worst region in terms of consistent latency. It spans quite large distances: https://where.durableobjects.live/region/apac
So, if your database moved from one location to another for any reason (e.g. datacenter downtime, network issues) then your latency could be impacted in large ways. That would be my guess what's going on, but without concrete data and timeranges I cannot say much else.

_Ultima_•4/16/25, 11:59 AM

Question, if i bind 100 D1 servers to a worker and use KV to target a specific one. (lets say i run 1 query on 1 specific D1), what would the billing look like? Do the other 99 D1's cost anything? (for now empty)

__Ultima_Question, if i bind 100 D1 servers to a worker and use KV to target a specific o...

lambrospetrouOP•4/16/25, 1:05 PM

No, you only pay for rows read and written, and your storage used. https://developers.cloudflare.com/d1/platform/pricing/

Note: empty databases still consume tiny bit of storage for SQLite metadata (few KBs), although for your 100 DBs it's negligible.

parisholley•4/17/25, 1:30 AM

can someone clarify what "region" means with replicas? is a single datacenter being chosen within a region (balanced across customers presumably with some internal algorithm) to host the replica, or is the replica always going to be on the nearest datacenter (pop) to the user?

Pparisholley can someone clarify what "region" means with replicas? is a single datacenter be...

lambrospetrouOP•4/17/25, 9:16 AM

The heuristic and algorithm for replica placement is ongoing and will change over time.
The current placement is that it picks a random location within each region and all requests originating from that region will go to that replica. The usual availability applies for replicas too, if the datacenter does down, it will move somewhere else in the same region until the original datacenter comes back online.

This will change down the line, though, and become smarter. We explore several heuristics at the moment how it would look like.

parisholley•4/17/25, 2:14 PM

good to know. i have a multi-tenant application for local businesses, so outside of developers, 99% of traffic is always going to originate from the same metro per tenant/d1 instance (eg: texas company, california company, etc). having a replica in another region isn't particularly useful in my case but i do have a lot of async processing (using inngest) and being able to fire off jobs and append a bookmark id could unlock some performance (given that those tend to be read heavy). being able to pin D1 (and its replicas) to a DC (as the primary) would be

shaw•4/19/25, 12:37 AM

Having a very strange issue.

D1_ERROR: internal error; reference = ...

Returning empty array of items

But before, it was working for me and not for the people I know in Europe until they got on a VPN to america

shaw•4/19/25, 12:38 AM

I disabled caching, enabled dev mode, now I have this error

shaw•4/19/25, 12:46 AM

shaw•4/19/25, 12:57 AM

any help much appreciated, had to pause our launch since the database unexpectedly doesnt work

Sshaw Having a very strange issue. D1_ERROR: internal error; reference = ... Return...

Chaika•4/19/25, 1:02 AM

do you have read replication enabled? d1-database

CChaika do you have read replication enabled? https://discord.com/channels/5953179901913...

shaw•4/19/25, 1:45 AM

i had it enabled but disabled since we were getting these issues

Sshaw i had it enabled but disabled since we were getting these issues

Chaika•4/19/25, 2:18 AM

They've said before

FYI, disabling replicas is not instant. If replicas are created and process some of your queries, it can take a few hours for them to stop processing queries and completely disappear.

you said you're getting issues on a prod db? It still impacting, does creating a new db work around it, etc?

lbguilherme•4/19/25, 10:24 AM

Hello there! D1 currently isn't available on south america and on middle east, it will currently place the database on other region and route requests. Is there any roadmap or at least an intent of adding those regions in the future? Thanks!

Kiwi•4/19/25, 9:43 PM

I've been wavering between PG and D1 because of:

Transaction support
Customer support
Questions about performance
After some experimentation switching in hyperdrive I think I'm resolved that D1 is definitely best for speed, but still have concerns about the other points.
How have others found D1 in practice?

KKiwi I've been wavering between PG and D1 because of: - Transaction support - Custome...

James•4/20/25, 1:03 PM

I’d go with PG, personally. I shared my thoughts a while back at d1-database

D1 is always improving, but in my experience you will always have the most flexibility both in terms of technology and support with something like PG.

Successful-Arm-3762•4/21/25, 6:36 AM

Workers + D1 API responding very slow

I deployed a CF worker binded with a D1 database.
I know about the cold start delay, but all my APIs are taking approx. 2 second each.

I have an API to fetch 150 product data. That takes 20+ seconds. I optimized it to use batch queries and everything. I did EXPLAIN QUERY and the query itself takes 20ms.

The worker colo is "SIN" and D1 is in "APAC".
Could this be a D1 issue?

Would be grateful if somebody could help me out.

Screenshot_2025-04-21_at_11.48.08_AM.png

sayonara•4/21/25, 10:05 AM

Our d1 database is giving error
@Tanmay Kapoor @zegevlier

{
    "errors": [
        {
            "code": 7500,
            "message": "Currently processing a long-running export."
        }
    ],
    "success": false,
    "messages": [],
    "result": []
}

{
    "errors": [
        {
            "code": 7500,
            "message": "Currently processing a long-running export."
        }
    ],
    "success": false,
    "messages": [],
    "result": []
}

Original message was deleted

itsmatteomanf•4/21/25, 10:08 AM

?pings

SuperHelpflare•4/21/25, 10:08 AM

Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.

sayonara•4/21/25, 10:18 AM

 "message": "internal error; reference = e_PVSMDp_85064edabe1b49f79d1f0a92065fcb4d"

 "message": "internal error; reference = e_PVSMDp_85064edabe1b49f79d1f0a92065fcb4d"

sayonara•4/21/25, 10:18 AM

for reference plz fix our db lol

Santi•4/21/25, 12:13 PM

D1_ERROR: Currently processing a long-running export.

Santi•4/21/25, 12:14 PM

Can someone help us out, our entire aplpication is down

sayonara•4/21/25, 12:58 PM

our case numnber-

01479222

01479222

{ "errors": [ { "code": 7500, "message": "Currently processing a long-running export." } ], "success": false, "messages": [], "result": [] }

Without specifying your workload (volume + kind of queries), location of source and target, and late

Similar Threads

Similar Threads

Similar Threads