Cloudflare Developers•9mo ago

So we'll be mainlining replicas for now 🙂

So we'll be mainlining replicas for now

ZZack So we'll be mainlining replicas for now 🙂

lambrospetrou•4/28/25, 2:59 PM

Keep us posted if you encounter anything else unexpected.

YYS•4/28/25, 8:23 PM

Are there plans to enable querying D1 from a database client like SQL developer?

YYYS Are there plans to enable querying D1 from a database client like SQL developer?

Cyb3r-Jak3•4/28/25, 8:32 PM

?d1-remote-connections

SuperHelpflare•4/28/25, 8:32 PM

Accessing D1 from outside workers:

D1 is based on SQLite, which does not natively support remote connections and isn't MySQL/PostgreSQL wire compatible, nor does D1 itself offer that functionality. As such, your options are:

1 -> The Cloudflare API Query Endpoint (https://developers.cloudflare.com/api/operations/cloudflare-d1-query-database), with the upside of needing less setup, all requests have to go back to Cloudflare's core, so request latency is far from optimal from some locations. You would also be subject to the API Rate limit of 1200 requests per 5 minutes.

2 -> Create a worker to proxy requests to your D1 Database. Either tightly bound to your application/schema or loose and just accepting any query. This would allow your worker to execute potentially closer to your D1 Database and avoid the api rate limit.

YYYS Are there plans to enable querying D1 from a database client like SQL developer?

Brayden•4/28/25, 11:10 PM

You can access your D1 from the open source Outerbase project which is a database client.

Madi•4/29/25, 6:03 AM

Hello,
Is there any workaround for D1's lack of geospatial indexing?

MMadi Hello, Is there any workaround for D1's lack of geospatial indexing?

Max (@rozenmd)•4/29/25, 8:30 AM

Hyperdrive + Postgres

Llambrospetrou Keep us posted if you encounter anything else unexpected.

ZackOP•4/29/25, 10:37 AM

Nothing more currently outside of the delays, which surprisingly are occuring more in WNAM atm. I was wondering if there is there is a better or worse region to be the primary region for your replica instance? Right now ours is in WEUR, but would ENAM or WNAM be better for instance, or maybe EEUR?

ZZack Nothing more currently outside of the delays, which surprisingly are occuring mo...

lambrospetrou•4/29/25, 11:16 AM

That really depends on your traffic patterns. There isn't a golden rule. All regions in the US and Europe are usually reliable and pretty good.

ZackOP•4/29/25, 11:17 AM

We get traffic from all over, mostly US and EU, but I was mostly wondering about the replicas spinning up in OC and APAC, is there a better primary location for that

sh4•4/29/25, 5:37 PM

Hi folks, apologies if this is already explained somewhere.. I noticed that via API I can create new D1 databases, but I cannot find information on how to direct a query on a per-worker-request basis towards a specific D1 Database. Imagine I have a multi-tenant SaaS with a new customer come in. Using the API I can create a new D1 database with just their data; But then imagine that user for that org wants to manipulate data in their database. I already know which requests should map to which database ID, but I have no idea how to construct/use a client for that database ID because it's not an environment binding.

Ssh4 Hi folks, apologies if this is already explained somewhere.. I noticed that via ...

lambrospetrou•4/29/25, 6:05 PM

Right now you need to have a d1 worker binding for each database you want to use from your worker.

sh4•4/29/25, 6:06 PM

I see. And that requires modifying the toml/jsonc, correct?

sh4•4/29/25, 6:06 PM

So if I want to onboard a new customer and them be able to query their own separate DB then that would require some manual work.

Ssh4 I see. And that requires modifying the toml/jsonc, correct?

lambrospetrou•4/29/25, 6:06 PM

It requires modifying and redeploying your worker, yes. Either through wrangler toml/jsonc or directly through the REST API for Patching the Worker settings. There are several threads in this channel with details on that.

sh4•4/29/25, 6:07 PM

Does patching worker settings update already-running workers? Thanks for the details.

Ssh4 So if I want to onboard a new customer and them be able to query their own separ...

Silvan•4/29/25, 6:55 PM

some people also use durable objects for this use case

Matthew•4/30/25, 5:12 PM

My Cloudflare Pages site is having problems connecting to D1. I've got 1000+ reports in Sentry of D1_ERROR: D1 DB storage operation exceeded timeout which caused object to be reset.D1_ERROR: D1 DB storage operation exceeded timeout which caused object to be reset. Looking at the D1 dashboard, query latency seems normal (p95 = ~1ms). But API routes that hit D1 are taking several seconds at best, or failing with the error above. The DB is in WEUR. This has been going on for about 6 hours now. Is there anything I can do to resolve?

Original message was deleted

Tait•4/30/25, 7:36 PM

I wonder about this too. However, I assume (possibly incorrectly) that there is a large silent majority who is not having issues.

gruntlord6•4/30/25, 8:37 PM

in general I haven't noticed any issues at all with d1, but I don't have hundreds of users or anything like that

Original message was deleted

lambrospetrou•5/1/25, 11:03 AM

I will reply assuming a friendly intent on your comment.

First of all there has been a lot of improvements over the past few quarters. We track the errors internally, and they are way less than say 8 months ago.
Second, there was big performance improvements for the Worker Binding requests, and soon similar improvements will apply to the REST API.
Third, read replication is finally out in beta, one of the most requested features. There were some rough edges initially, but they were fixed in a couple of days, hence why it was released as beta. This is going to be the basis for major simplifications like improving the exports of big databases for example. It will help scale reads significantly. And it's still early days in terms of smart placement of the replicas following your traffic transparently and all that.

Finally, if you scan ANY community forum, any channel in this Discord server, the vast majority of comments are negative. It's about issues people face exactly because this is a community support forum and they need help. That's the goal of these Discord channels. We have thousands of D1 users, some with thousands of databases literally, and you don't see them posting every morning that their services are running without issues.

People post about D1 having more errors than expected posting screenshots and experiences from last year or many months ago. It's not indicative to today. Yes, we have work to do around docs and improvements about these errors, but it gets better every day.

In closing, having said all the above, I by no means say D1 is perfect or that there is no justification to complaints. Yes, there are more errors than we want still, yes we are working on further improvements, and yes we are aware of most issues people raise. At the end of the day, if you use D1 and doesn't meet your needs, try Hyperdrive, DOs, or something else.

Matthew•5/1/25, 1:27 PM

After D1 was essentially unusable for ~6 hours for me yesterday, I created a new database in ENAM and migrated my data over it. Since then, I've had a handful of D1_ERROR: internal errorD1_ERROR: internal error on the new database. Hoping I don't have to do this migration more than once!

Brandon•5/1/25, 3:19 PM

hey all — brandon here (previously from Outerbase) we just shipped a new experience for D1 including our data studio. it's read only for now, but an editable experience is coming soon!

would love to get your thoughts!

BBrandon hey all — brandon here (previously from Outerbase) we just shipped a new experie...

James•5/1/25, 3:55 PM

love the new design and UX! One minor a11y comment about these: https://i.james.pub/file/2025/05/387f3575-9417-4610-a50c-c54484166e0f.png
github/deploy are buttonsbuttons that open new tabs so ideally should just be anchors so folks can middle click them and accessibility tooling understand what they do a little better :MeowHeartCloudflare:

BBrandon hey all — brandon here (previously from Outerbase) we just shipped a new experie...

Chaika•5/1/25, 5:58 PM

The sorting of queries/size is only for the DBs visible on the current page you are on, which is pretty confusing

Chaika•5/1/25, 5:58 PM

oh actually, not sure if that overview is strictly part of what you shipped? hmm

Brandon•5/1/25, 6:36 PM

Yeah we have a ticket open for this — right now the API is paginated, so sorting only applies to the current page. Hoping it's a fairly straightforward fix haha.

Successful-Arm-3762•5/2/25, 2:47 PM

Hi, been facing D1 network connection failure since some time now, on my production API.

Error: Database error during login: Error: D1_ERROR: Network connection lost.

Happens randomly.

Tim•5/2/25, 5:02 PM

My worker app is randomly getting errors like D1_ERROR: D1 DB storage operation exceeded timeout which caused object to be reset.D1_ERROR: D1 DB storage operation exceeded timeout which caused object to be reset..

The Cloudflare dashboard page for the worker shows 0 errors. However, I know this is happening, and have evidence in Sentry reporting.

James•5/2/25, 5:07 PM

It's generally recommended to build retries into your app for those kind of errors. Hopefully we'll see some official guidance soon for https://github.com/cloudflare/cloudflare-docs/issues/18485

가

가브리엘 | Verzach3•5/3/25, 2:10 AM

Hi all, im creating an app with workers and D1, whats the recommended way to create a multitenant app with D1, how i can create and assign database dynamically to workers?

가가브리엘 | Verzach3 Hi all, im creating an app with workers and D1, whats the recommended way to cre...

Silvan•5/3/25, 6:57 AM

the current easiest solution is to use durable objects instead

가가브리엘 | Verzach3 Hi all, im creating an app with workers and D1, whats the recommended way to cre...

lambrospetrou•5/3/25, 8:40 AM

As said, if you want to have great flexibility in create databases dynamically through code SQLite Durable Objects are more suitable.

가가브리엘 | Verzach3 Hi all, im creating an app with workers and D1, whats the recommended way to cre...

Successful-Arm-3762•5/3/25, 7:13 PM

D1 doesn't support transactions, but SQLite through Durable objects do.
I'm still using D1, not sure how difficult would the transition be, but I will soon be making that change or shift to a proper postgresql service

D1 batches do support transactions!

gruntlord6•5/4/25, 6:44 AM

batch is transactional
https://developers.cloudflare.com/d1/worker-api/d1-database/#batch

Cloudflare Docs

D1 Database

To interact with your D1 database from your Worker, you need to access it through the environment bindings provided to the Worker (env).

Mirith•5/4/25, 4:31 PM

I have what I think is going to be an obvious question, but I can't figure out how to properly look up the answer. I have a database (Surprise!) connected to a worker that I'm accessing on the worker. Locally everything works fine. However, when I deploy it to remote, I can insert new rows into my database, but I don't seem to be able to update them. I don't seem to be getting any errors, but I'm also wondering since I'm using a web socket to transmit some of that data involved if that is getting hidden in my logs.

Mirith•5/4/25, 4:32 PM

Is there a way to just look at the console.log output remotely?

Mirith•5/4/25, 4:40 PM

Actually, I think the error might be something else

MMirith Actually, I think the error might be something else

gruntlord6•5/4/25, 5:40 PM

the worker is transmitting the d1 operation by web socket?

Mirith•5/4/25, 6:01 PM

No. The problem involves the websocket but the operation is just not getting called

MMirith Is there a way to just look at the console.log output remotely?

Chaika•5/4/25, 6:34 PM

wrangler tail https://developers.cloudflare.com/workers/observability/logs/real-time-logs/ but you'll only see logs when the websocket ends

Ggruntlord6 batch is transactional https://developers.cloudflare.com/d1/worker-api/d1-databa...

Successful-Arm-3762•5/4/25, 7:03 PM

ooh, thanks for letting me know! this is helpful! I thought batch just reduces round trips, good to know!

gruntlord6•5/4/25, 7:04 PM

I think we all get caught by that one, luckily the docs are there

chook•5/6/25, 1:36 AM

Has anyone encountered a 504 response when running large migrations via wrangler?

I know there is this issue

If so how did you get around the error? I've tried running d1 execute --filed1 execute --file with the same result

GitHub

🐛 BUG: `504 Gateway Time-out` when applying migrations in D1 · ...

Which Cloudflare product(s) does this pertain to? D1 What versions are you using? 3.102.0 [Wrangler], 20.18.0 [Node.js] What operating system and version are you using? Mac Sequoia 15.2 Please prov...

Cchook Has anyone encountered a 504 response when running large migrations via wrangler...

lambrospetrou•5/6/25, 8:47 AM

How many rows and megabytes are you importing? Have you tried chunking the file into smaller imports? We still didn't get around to optimizing the import flows, and it has several limitations for long running jobs.

daycompacted•5/6/25, 5:25 PM

In order to avoid the 1m lag with KV propagating to all regions, I'm thinking to migrate everything into D1. It seems as though D1 is also cheaper for read/writes/storage. Aside from the difference in performance, is there any reason to stick with KV?

Ddaycompacted In order to avoid the 1m lag with KV propagating to all regions, I'm thinking to...

lambrospetrou•5/6/25, 5:29 PM

Scaling and throughput properties of KV and D1 are wildly different. D1 is a single Durable Object in a single location. KV is cached globally after first access.
Very highly dependent on your workload, requests per second, reads vs writes, locations you access it from, cachability, how much data, etc.

mon•5/7/25, 4:40 AM

Sanity check: wrangler no longer has the --experimental-backend--experimental-backend flag, and

wrangler d1 info

wrangler d1 info

no longer returns the version. Have all beta DBs been silently upgraded to the new backend?
Just trying to remove it as an explanation for some poor performance.

So we'll be mainlining replicas for now 🙂

Similar Threads