It just seems like hacky "redeploys" where I eternally bind more D1 databases to my wrangler.json ca

It just seems like hacky "redeploys" where I eternally bind more D1 databases to my wrangler.json can't be the best way to go about this... but also I don't want to run unsafe SQL statements and I don't want to run an api call for each query... If there's no official way I'll figure something out but it seems like I must be missing something here.

Alternatively, is this something I should be solving with Queues?

JJael It just seems like hacky "redeploys" where I eternally bind more D1 databases to...

lambrospetrou•11/10/25, 9:55 PM

If you want thousands of DBs (tenants) you should use SQLite Durable Objects. Same underlying SQLite engine as D1, but you can dynamically create thousands of instances programmatically.

lambrospetrou•11/10/25, 9:59 PM

Hey folks, we updated the section "How much work can a D1 database do?" in our docs: https://developers.cloudflare.com/d1/platform/limits/#how-much-work-can-a-d1-database-do

It tries to give a guidance on what to expect, and set expectations a bit more realisitically. Let me know how you find it.

Cloudflare Docs

Limits

Cloudflare also offers other storage solutions such as Workers KV, Durable Objects, and R2. Each product has different advantages and limits. Refer to Choose a data or storage product to review which storage option is right for your use case.

Llambrospetrou If you want thousands of DBs (tenants) you should use SQLite Durable Objects. Sa...

JaelOP•11/10/25, 10:28 PM

looks like I have some reading to do thanks

JJael looks like I have some reading to do thanks 🙂

lambrospetrou•11/10/25, 10:38 PM

Start here: https://developers.cloudflare.com/durable-objects/concepts/what-are-durable-objects/

Cloudflare Docs

What are Durable Objects?

A Durable Object is a special kind of Cloudflare Worker which uniquely combines compute with storage. Like a Worker, a Durable Object is automatically provisioned geographically close to where it is first requested, starts up quickly when needed, and shuts down when idle. You can have millions of them around the world. However, unlike regular Wo...

thipperz•11/11/25, 7:15 PM

Are there any plans to add more read replicas locations? i.e. South America

Chaika•11/11/25, 7:32 PM

(which aren't in South America yet)

Chaika•11/11/25, 7:32 PM

not been any new DO regions for a bit, middle of last year they said they were trying to get India and then South America by end of 2024...
durable-objects

akicookz•11/12/25, 7:54 PM

anyway to connect d1 with a db GUI. Looked through docs and I think it is not possible. just wanted to make sure I have not missed a work around or something.
Thanks

Aakicookz anyway to connect d1 with a db GUI. Looked through docs and I think it is not po...

1984 Ford Laser•11/12/25, 8:00 PM

https://d1-studio.com/

d1-studio

The first native studio for D1 databases

11984 Ford Laser https://d1-studio.com/

akicookz•11/12/25, 8:02 PM

Does this let me pin queries? Cloudflare D1 also has a GUI but it's limited. I have some long admin queries I wanna keep around the db instead of my notepad

AAlper Dincer hi, I need help on the following problem at D1: https://community.cloudflare.co...

gruntlord6•11/13/25, 12:20 AM

You need to optimize your query, whatever you are doing is more then 128mb which is how much in memory state the d1 durable object has.

Have you tried splitting the reads into paginated requests rather then just selecting all 26 million rows?

AAlper Dincer hi, I need help on the following problem at D1: https://community.cloudflare.co...

1984 Ford Laser•11/13/25, 4:32 AM

You definitely wanna add the index before having 26mil rows. I would also consider if there are ways to split the data up into separate tables, or only select a subset at a time

Ggruntlord6 You need to optimize your query, whatever you are doing is more then 128mb which...

lambrospetrou•11/13/25, 7:34 AM

As others suggested, this is one of the most common ways folks hit the SQLite memory issues with D1 (and Durable Objects). Doing aggregations across the whole table of millions of rows is too easy to hit the memory/CPU limits.

In your specific query:

SELECT calc_date, indicator_id FROM GridData GROUP BY calc_date, indicator_id

SELECT calc_date, indicator_id FROM GridData GROUP BY calc_date, indicator_id

can you do multiple queries with "data filtering" in ranges so that the rows are filtered out?

Although, that would need an index to work properly if you don't already have one, but it's worth trying.

AAlper Dincer Thanks for the suggestions, but I will try to go with Parquet or other alternati...

Kevin•11/13/25, 10:26 AM

Isn't hyperdrive and option for you?

lambrospetrou•11/13/25, 4:59 PM

Hey folks, a quick update for those that enabled read replication in the past few weeks but didn't see replicas appearing. There was a control plane bug that in some edge cases even though the API persisted the decision, the database did not actually enable replicas, therefore no replica was serving queries.

We released a mitigation last Friday for this to stop happening in new cases, and we rolled out another fix today to retroactively enable replicas for those that have the read replication setting enabled.

If you try out read replication and even after sending some traffic for a few minutes you still don't see replicas serving requests please reach out. You can see if replicas are serving queries in the dashboard UI, and also in each query response from the property

served_by_primary: false

served_by_primary: false

Ronald•11/14/25, 1:31 PM

Hello

I'm new to this product, and as per the Docs it says data IS encrypted at rest/transit, so does it mean that I could store passwords/sensitive info without any further encryption? Without having for example a Worker to encrypt/decrypst a user password?

RRonald Hello 👋 I'm new to this product, and as per the Docs it says data IS encrypted ...

PhoenixSpace•11/14/25, 1:33 PM

You should still encrypt it

PhoenixSpace•11/14/25, 1:33 PM

It’s encrypted at rest, but there is still an attack vector of if someone gains access to your cloudflare account through a mistake. You should never be able to access someone’s plaintext password period

PhoenixSpace•11/14/25, 1:35 PM

To be more specific, cloudflare encrypts D1 at rest, but passwords should be one way hashed for security which is different

PPhoenixSpace It’s encrypted at rest, but there is still an attack vector of if someone gains ...

Ronald•11/14/25, 1:35 PM

Yup, makes sense

Ronald•11/14/25, 1:36 PM

Thanks @PhoenixSpace

PhoenixSpace•11/14/25, 1:36 PM

No problem :)

Matthew•11/15/25, 8:28 AM

SQLite 3.51.0 introduces

jsonb_each

jsonb_each

and

jsonb_tree

jsonb_tree

functions. Does D1 plan to support these? What version of SQLite is D1?

sqlite_version

sqlite_version

is not supported.

MMatthew SQLite 3.51.0 introduces `jsonb_each` and `jsonb_tree` functions. Does D1 plan t...

lambrospetrou•11/16/25, 9:40 AM

It's SQLite 3.47 right now: https://github.com/cloudflare/workerd/blob/964823ff45df4b5033bae9bba1b43baf6a8c556c/build/deps/deps.MODULE.bazel#L81

We do upgrade SQLite versions but not in a specific cadence. I will ask the team to see if we can upgrade soon again.

CCole Xemi Does d1 retry read queries automatically if they write to a temporary table? Lik...

lambrospetrou•11/16/25, 9:43 AM

As long as it doesn't cause writes to the underlying storage of the database, it should retry them, if the error is marked as retryable and the DB is not overloaded. If I remember correctly, temp tables are in memory, so it should be retried.

CubityFirst•11/16/25, 1:03 PM

https://i.cubityfir.st/firefox_f2MemxSOV7.png

CubityFirst•11/16/25, 1:03 PM

Am I doing something wildly wrong?

CubityFirst•11/16/25, 1:03 PM

How is cloudflare reporting that there's two entries with the same primary key? Shouldn't that be unique?

CubityFirst•11/16/25, 1:04 PM

Additionally, when I try to delete those rows via the studio, it just doesn't, same with editing the values

CubityFirst•11/16/25, 1:06 PM

https://i.cubityfir.st/firefox_ESgnqDdGh4.mp4

CubityFirst•11/16/25, 1:08 PM

https://i.cubityfir.st/firefox_bvoCV4kEWU.png

CubityFirst•11/16/25, 1:21 PM

i've gone so far as removing the binding from the worker to ensure that it's not some leftover worker that hasn't terminated or whatever, it's just not saving.

CubityFirst•11/16/25, 1:21 PM

I've recreated the table, and it's still not letting me save the value via studio

CubityFirst•11/16/25, 1:28 PM

Ah, well, Is ee

CubityFirst•11/16/25, 1:28 PM

Thank you so much haha

Original message was deleted

CubityFirst•11/16/25, 1:35 PM

Just swapped it over, working great now, thanks again.

Matthew•11/17/25, 9:36 AM

How are people handling database backups?

wrangler export

wrangler export

was fine when my db was 30Mb (3-4 seconds downtime is acceptable) but now the DB is 500Mb an export takes it offline for ~2 minutes.

scruffy•11/17/25, 9:48 AM

Once a D1 read replica has been created in a region, does it remain there "forever", or is it evicted after a certain period of inactivity. If evicted, what's the policy?

Sscruffy Once a D1 read replica has been created in a region, does it remain there "forev...

Matthew•11/17/25, 9:56 AM

I believe replicas are based on Durable Objects https://developers.cloudflare.com/durable-objects/concepts/durable-object-lifecycle/#:~:text=While%20in%20the%20idle%2C%20in,transition%20to%20the%20inactive%20state

Cloudflare Docs

Lifecycle of a Durable Object

This section describes the lifecycle of a Durable Object.

MMatthew I believe replicas are based on Durable Objects https://developers.cloudflare.co...

scruffy•11/17/25, 10:07 AM

That's a good point (I believe all of D1 is based on Durable Objects, not just read replicas). In that case, I guess my question was a bit off... if D1 read replicas follow the DO lifecycle, then they'd stay in memory for a while (as per linked docs). I was hoping to learn whether or not the underlying read replica instance would be cleaned up after a period of inactivity.

Sscruffy That's a good point (I believe all of D1 is based on Durable Objects, not just r...

lambrospetrou•11/17/25, 10:26 AM

I was hoping to learn whether or not the underlying read replica instance would be cleaned up after a period of inactivity.

As the lifecycle doc explains, if there are no requests handled within a region then after some period the replicas will hibernate or get evicted, and the next requests in that region will have to boot the replica again. Those first requests will be forwarded to the primary until the replica is ready again to serve traffic.

In the future we want to allow dynamic replicas that follow your traffic, but the lifecycle will still exist.

Llambrospetrou > I was hoping to learn whether or not the underlying read replica instance woul...

scruffy•11/17/25, 10:33 AM

Sorry, I think I'm still being unclear. Would the Read Replica's Durable Object be removed from the data centre? My reason for asking was that I'd imagine larger databases would take more time to move and boot and I wondered whether Cloudflare optimised for quicker boots (keep the replica in the data centre) or lower storage (remove the replica from the data centre).

Sscruffy Sorry, I think I'm still being unclear. Would the Read Replica's Durable Object ...

lambrospetrou•11/17/25, 10:37 AM

Still not clear what "removed from the data centre" means.

If you are asking if the replica moves data centers after creation, then at the moment it doesn't unless that specific DC has network issues. But, it will definitely move servers within that data centre. The normal DO routing applies here that attempts to keep a replica on the same DC and server, but it can move anytime due to many reasons under the hood.
Also, replicas need to re-fetch data from cold storage when they are cold starting since they need the latest state in case there were any writes.

Llambrospetrou Still not clear what "removed from the data centre" means. If you are asking if...

lambrospetrou•11/17/25, 10:38 AM

Overall, your questions seem to suggest that your worry is with low traffic how fast will cold starts be. Then, at the moment, you need to assume actual cold starts all the time. In the future there will be optimizations, but at the moment cold starts are not optimized yet.

If your worry is not about low traffic use-cases, then please provide more details.

Llambrospetrou Overall, your questions seem to suggest that your worry is with low traffic how ...

scruffy•11/17/25, 12:44 PM

You're right, my initial worry was about cold starts. I guess there are two scenarios I'm wondering about: when a read replica is first created in a new data centre, and when an existing read replica has to be cold started. Which leaves me with three questions: 1. what's the heuristic for creating a read replica in a new data centre? 2. how quickly can a new read replica be created and booted (not looking for precise numbers, just a way to think about the impact of sqlite storage size)? 3. what affects the cold start time for a read replica (sqlite size, amount of data to be updated, something else)? Apologies if I'm asking silly questions, just trying to get my head around how to think about this.