Cloudflare Developers•5mo ago

reads are blocked

BBilly In this doc: https://developers.cloudflare.com/d1/best-practices/import-export-...

gruntlord6OP•8/7/25, 8:32 PM

the whole thing is not accessible while its dumping the database

Ggruntlord6 reads are blocked

Billy•8/7/25, 9:10 PM

yikes, ok... but that blocking limitation is about to be eliminated, correct?

gruntlord6OP•8/7/25, 9:15 PM

I mean they want to eliminate it, they haven't given a timeline of any kind. You can always just copy the data directly instead of using that tool

ldo•8/8/25, 8:32 AM

Is there a page or dashboard where I can check the server health/status of D1? One of my D1 services has been returning timeouts or 500 errors for about two hours, and I would like to know when this issue is expected to be resolved.

Lldo Is there a page or dashboard where I can check the server health/status of D1? O...

lambrospetrou•8/8/25, 10:00 AM

There isn't any widespread D1 service issue or individual DBs with startup issues. If you continue having 5xx please send us more details. DB/Account IDs, and any timeranges you can specify to help us investigate.

Llambrospetrou There isn't any widespread D1 service issue or individual DBs with startup issue...

ldo•8/8/25, 10:21 AM

It was not a 100% failure, but the connection was very unstable.
The UUID of the problematic D1 is e6cdb51a-42e0-4a40-a342-bf1852204a87, and the following issues occurred:

D1_ERROR: Network connection lost.
D1_ERROR: D1 DB is overloaded. Requests queued for too long.
Timeout (30s)
When attempting to access the DB via the console, it would load indefinitely.

Currently, the situation has somewhat stabilized and console access is possible.
However, timeouts or response times exceeding 10 seconds still remain.

Lldo It was not a 100% failure, but the connection was very unstable. The UUID of the...

lambrospetrou•8/8/25, 11:10 AM

OK, these errors are not from the D1 service itself, so that's good at least

Network connection lost errors are indeed internal errors from the runtime, usually due to the underlying Durable Object connectivity issues. Some amount of these is to be expected but should be very transient and not ongoing, feel free to retry these if your query allows. If you see these continuously please let me/us know, since sometimes a database might be in a faulty location for a few minutes before it gets taken out of the network or might need something manual on our side.

The overloaded error indicates that either you send too many requests and they are slow being processed, so the queue is growing and at some point extra reqs will be rejected with that error.

I will take a glance on your DB, if you DM me your account ID too.

lambrospetrou•8/8/25, 11:12 AM

As a side note to everyone, it makes it faster for us to investigate issues in your accounts if you connect the account info to your Discord profile, otherwise we would need to ask you all the time about it, meaning more back and forth comms before we can do anything.

Run

/link

/link

and fill the info, so that we can figure out your account by your profile.

JJun The text has been updated FYI 😄

cortopy•8/8/25, 5:48 PM

This is so much clearer!! Would it be possible to maybe clarify the same for durable objects sql api? I’ve been trying to figure out if sql methods protect against sql injection

https://developers.cloudflare.com/durable-objects/api/storage-api/#returns

Cloudflare Docs

Durable Object Storage

The Durable Object Storage API allows Durable Objects to access transactional and strongly consistent storage. A Durable Object's attached storage is private to its unique instance and cannot be accessed by other objects.

Llambrospetrou As long as you use the D1 Binding from your worker, or Pages function, it should...

Dean•8/11/25, 6:19 AM

i enabled it and haven't been able to have a single instance where it's used the replicated db. served_by_region: "WNAM",served_by_primary: true. does it take a few days for it to start serving from different regions?

DDean i enabled it and haven't been able to have a single instance where it's used the...

lambrospetrou•8/11/25, 6:39 AM

No, creating the replicas is immediate, although the couple first requests per location will go to the primary until the replica boots. The most common case of not using replicas is when the request rate is not high enough, for example if you do a request every few minutes in each region then most likely all of them will go to the primary.

We know a small bug on our side for the "enabling" itself which we plan on fixing soon, so if you send constant traffic to your DB and still get no replicas, try disabling it and re-enabling it, and if you still get no replicas DM me your details to take a look.

Llambrospetrou No, creating the replicas is immediate, although the couple first requests per l...

Dean•8/11/25, 6:41 AM

thx - let me try that and if not ill send you db ids. edit disable/reenable fixed it.

can•8/11/25, 10:05 AM

Hello y'all, i ran

d1 export

d1 export

for one of my databases and now it says

Error in worker: Error: D1_ERROR: Currently processing a long-running export.

Error in worker: Error: D1_ERROR: Currently processing a long-running export.

and i'm not sure it's currently processing because it errored out on my terminal. Is there a way to cancel this process?

can•8/11/25, 10:11 AM

how am i suppose to stop polling for that process?

ToonyGen•8/11/25, 7:09 PM

I just found a fancy repo to manage d1. https://github.com/earth-app/CollegeDB.git

GitHub

GitHub - earth-app/CollegeDB: Cloudflare D1 Sharding Router for Hor...

Cloudflare D1 Sharding Router for Horizontal Scaling - earth-app/CollegeDB

red2green•8/12/25, 3:05 AM

are there any case stuides/examples of cloudflare customers that have thousands of dbs? Curious what its like to actually have to manage that many databases in production.

TToonyGen I just found a fancy repo to manage d1. https://github.com/earth-app/CollegeDB.g...

lambrospetrou•8/12/25, 9:41 AM

Really cool. I like the simplicity of using an explicitly provided "shard/hash key". It makes the routing much simpler. This is similar approach to how I shard across SQLite DOs.

I wonder how this was developed, was it a work project, college project (hint in the name), or just a side project? Majority of the logic seems to have been added in a single commit.

Either way, very nice to read through. Haven't checked the code yet. This is not an endorsement to use this in production, BTW.

Rred2green are there any case stuides/examples of cloudflare customers that have thousands ...

lambrospetrou•8/12/25, 9:44 AM

Unfortunately, I am not aware of publicly documented cases. We do have a few users that are like agencies developing apps for other companies and use thousands of D1 DBs. They mostly manage them through the REST API. Most of these cases are not attaching thousands of DBs to a single worker, as far as I know, although some did and use the Workers Patch API to update the bindings on their worker dynamically.

My suggestion, if you have a single worker project and want thousands of DBs using SQLite DOs will give you a much nicer experience. Now, if you are a platform creating thousands of projects and each one needs a small D1 DB that's a great use-case for D1 and managing the DBs through the API.

Llambrospetrou Unfortunately, I am not aware of publicly documented cases. We do have a few use...

Levi•8/12/25, 12:25 PM

I essentially came here to ask about dynamically creating and binding of D1 databases, but it seems like using Durable Objects would work better.

My goal is to have a database per user, org, or tenant for my one app.

Llambrospetrou Really cool. I like the simplicity of using an explicitly provided "shard/hash k...

ToonyGen•8/12/25, 1:25 PM

I think the author is using as the backend for its own social media project.

LLevi I essentially came here to ask about dynamically creating and binding of D1 data...

cloudkite•8/12/25, 8:27 PM

We are building a db per customer architecture and were holding out for dynamic bindings. However decided to bite the bullet and currently migrating from d1 to durable objects. Was initially reluctant but have to say it gives you a lot more flexibility and as a bonus has opened up some nice opportunities for performance optimisations.

Ccloudkite We are building a db per customer architecture and were holding out for dynamic ...

red2green•8/12/25, 8:43 PM

with a durable object, backed by sqlite, does it act just like a sqlite db? you can create many tables etc? DO is a little confusing.

red2green•8/12/25, 8:44 PM

with DO, you will have to migrate the schema using code ?

Rred2green with DO, you will have to migrate the schema using code ?

James•8/12/25, 8:50 PM

yeah, just plain jane sqlite, but only accessible via the DO runtime, no backdoor, file based access. Treat the DO as a web server running in the same process as the sqlite process and you're only way to run bindings/sql console. Which it literally just is that.

JJames yeah, just plain jane sqlite, but only accessible via the DO runtime, no backdoo...

Larry•8/12/25, 9:10 PM

I went straight to DOs so I don't know much about D1, but is there a backdoor to D1's SQLite file?

LLarry I went straight to DOs so I don't know much about D1, but is there a backdoor to...

James•8/12/25, 9:24 PM

no, but some people get the impression DO sqlite is something they get more 'native' sqlite access to the file system or extensions if they don't know otherwise

red2green•8/12/25, 9:27 PM

You can connect to D1 using wrangler

red2green•8/12/25, 9:27 PM

so that's a "backdoor"

Rred2green You can connect to D1 using wrangler

Larry•8/12/25, 9:33 PM

I was aware of that for D1 and DO, but it sounded like James was hinting at it being available for D1 but not DO for files in the Cloudflare cloud.

red2green•8/13/25, 4:12 PM

generally speaking, other than the limits of db size, query size, etc. Are there any requests per second limits?

Rred2green generally speaking, other than the limits of db size, query size, etc. Are th...

lambrospetrou•8/13/25, 4:22 PM

Each database is backed by a Durable Object, which is single threaded SQLite under the hood, so the limit depends on how fast/slow your queries are. If your queries execute at 1ms, you can do 1K of them at most, etc.
Rough guideline is that for most cases we see SELECTs with proper indexes usually finish in less than 1ms, and writes are a few milliseconds, so depends on your case.
My ballpark generic guideline is about 300-900 queries per second.

We have users doing more, and we have users doing less.

Llambrospetrou Each database is backed by a Durable Object, which is single threaded SQLite und...

red2green•8/13/25, 4:46 PM

great thanks. ok I didn't realize D1 uses Durable objects... I really need to understand DO b/c I'm a little confused now

I thought DO were backed by sqlite, but didn't realize d1 has DO "in front" of it.

red2green•8/13/25, 4:46 PM

so I should create multiple d1's based o the rough threshold of 300-900 qps.

Rred2green great thanks. ok I didn't realize D1 uses Durable objects... I really need to...

lambrospetrou•8/13/25, 4:48 PM

D1 is essentially a layer ontop of SQLite DOs. The underlying sqlite engine is the same, since D1 uses a DO for each database.

red2green•8/13/25, 4:48 PM

got it thanks.

red2green•8/13/25, 4:49 PM

1. llms.txt 
2. prompt.txt 
3. d1 llms-full.txt

1. llms.txt 
2. prompt.txt 
3. d1 llms-full.txt

when using claude cli, which file should I ask it to read?

red2green•8/13/25, 4:49 PM

(this is the d1 doc page)

Rred2green got it thanks.

Hard@Work•8/13/25, 4:51 PM

You can probably connect it to the CF Docs MCP Server: https://docs.mcp.cloudflare.com/sse

HHard@Work You can probably connect it to the CF Docs MCP Server: https://docs.mcp.cloudfla...

red2green•8/13/25, 4:57 PM

ah ok, i'll look into that. that's probably more streamlined...

[

[A]nToNin•8/14/25, 9:37 AM

im confused on the d1 free tier does it give 5gb or 1gb storage?

[[A]nToNin im confused on the d1 free tier does it give 5gb or 1gb storage?

lambrospetrou•8/14/25, 10:19 AM

Where did you see the 1GB mentioned? https://developers.cloudflare.com/d1/platform/pricing/

darkpool•8/14/25, 6:19 PM

Architecturally would it be possible to implement Cap'n Proto as a data type for D1? Spanner recently released protocol buffer support (apparently was available for years in their internal build) and it unlocks some really cool features like type safety and 10x faster performance vs storing nested data in JSON/JSONB. Compared to Spanner's approach, Cap'n Proto seems like it would be significantly better too since it's zero-copy. Is this something that's been considered or technically feasible with D1's architecture?

Lengo•8/15/25, 4:32 AM

Is Cloudflare working on removing the 10 GB limit from D1? This feels quite limiting for a production database…

Lengo•8/15/25, 4:34 AM

Also, will D1 auto scale to any number of requests?

LLengo Is Cloudflare working on removing the 10 GB limit from D1? This feels quite limi...

lambrospetrou•8/15/25, 6:37 AM

No, 10GB is a hard limit at the moment. No immediate plans to change that.

LLengo Also, will D1 auto scale to any number of requests?

lambrospetrou•8/15/25, 6:38 AM

Also no, see my answer a few messages above for the same question: d1-database

If you need a traditional "big machine" database, you might be better off with Hyperdrive + any database provider of your choice.

Lengo•8/15/25, 6:40 AM

okay thanks

Lengo•8/15/25, 6:40 AM

I'm probably just going to use Neon... if I could provide a suggestion, a service like theirs in the Cloudflare dev platform would be very cool

Lengo•8/15/25, 6:44 AM

I'm curious... who is D1 intended for?

LLengo I'm curious... who *is* D1 intended for?

lambrospetrou•8/15/25, 6:47 AM

D1 is a more managed database over using SQLite Durable Objects directly, so it's great when you can shard your application data model across several of them. Sweet spot is a few tens so not exactly per user but not limited by the 10GB limit either, if you need more than that in the thousands you are better to use DOs directly.

Imagine apps like the trendy vibe coding platforms nowadays where you need a database for millions of independent projects and each one is self-contained and small, just as an example where it fits great. Or internal projects that fit within 10GB (or N multiples of them where sharding comes naturally) and the traffic is not crazy.

reads are blocked

Similar Threads

Similar Threads

Similar Threads