Cloudflare Developers•10mo ago

1. This really depends on each database. How many millions of rows, how big each row is, etc. 2. Pro

This really depends on each database. How many millions of rows, how big each row is, etc.
Probably reads could still flow through with the replicas, but the primary will still be blocked. We have some work streams to improve the import/export APIs over the next few months.

Llambrospetrou 1. This really depends on each database. How many millions of rows, how big each...

1984 Ford Laser•2/27/25, 10:53 PM

Replicas that don't exist publicly yet right?

Llambrospetrou 1. This really depends on each database. How many millions of rows, how big each...

dan—1106•2/27/25, 11:00 PM

On 1, makes sense. Can you give any guidance on the possible ranges of times the db might be blocked? Seconds, 10 minutes, 30 minutes?

Ddan—1106 On 1, makes sense. Can you give any guidance on the possible ranges of times the...

dan—1106•2/27/25, 11:02 PM

Is it possible to export a backup from Time Travel? Snapshots must be on disk somewhere?

RRoger Hello, is there any plan to create an official driver to manage D1 databases via...

Roger•2/28/25, 8:04 AM

Hello, I think this message has gone unnoticed among others. Is there any plan to collaborate with an existing database manager to support d1?

RRoger Hello, I think this message has gone unnoticed among others. Is there any plan t...

Silvan•2/28/25, 10:56 AM

pretty sure drizzle studio works?

RRoger Hello, I think this message has gone unnoticed among others. Is there any plan t...

Brayden•2/28/25, 12:26 PM

You can use Outerbase to view your D1 databases

Manuel Ogomigo•2/28/25, 3:16 PM

Hi my d1 all of a sudden went from less than 10m rows read to over 38B in 5 days.
I didn't make any change and still uses the same data, not sure what's causing it.

It keeps on increasing, is there any way I can fix this. Please

MManuel Ogomigo Hi my d1 all of a sudden went from less than 10m rows read to over 38B in 5 days...

Isaac McFadyen•2/28/25, 3:19 PM

Double-check that whatever Worker is querying the D1 DB isn't getting flooded with requests, perhaps?

Isaac McFadyen•2/28/25, 3:19 PM

And/or that you haven't accidentally leaked an access token that someone's using to hit the API

Manuel Ogomigo•2/28/25, 3:25 PM

This is the query that is being shown in the metrics with the higest count, multiples of it. And that query is only found in my cloudflare pages.

is there any way I can check to see where the D1 queries are coming from aside that, or a way to fix it, it's been that query for months with no issues

Manuel Ogomigo•2/28/25, 4:17 PM

I found the query already, using the d1 insights wrangler, thank you!

Xulo•2/28/25, 7:09 PM

Hi, I'm using Cloudflare D1 with Drizzle ORM and want to batch multiple queries in a single transaction. However, I need newUser.idnewUser.id and account.idaccount.id from the first two queries to use in the third one. Since the queries are in an array, I can't assign these values to a constant. Is there a way to achieve this within a single batch, or do I need to split it into two?

Thanks in advance! :)

atmajo•3/1/25, 3:29 AM

Hello, I'm using cloudflare D1 with drizzle orm from 28th February 2025, I am getting an error while applying migration changes to the databse can anyone please help me

$ bun wrangler d1 migrations apply megha-backend

wrangler 3.111.0
--------------------

*** Received structured exception #0xc0000005: access violation; stack: 7ffbf9c52c0f 7ff66a13147a 7ff66a131543 7ff66a0d84ac 7ff66a0d8457 7ff669627a8e 7ff669627d9f 7ff6680a3316 7ff66975200f 7ff6697591b6 7ff669752513 7ff6697591b6 7ff66974feac 7ff668091ee5 7ff66acaeb5f 7ffc142fe8d6 7ffc15bbbf2b
X [ERROR] The Workers runtime failed to start. There is likely additional logging output above.

Matthew•3/1/25, 2:50 PM

I'm seeing lots of D1 errors for brief periods yesterday and today, but there's nothing on the D1 status page. I get

D1 DB storage operation exceeded timeout which caused object to be reset

D1 DB storage operation exceeded timeout which caused object to be reset

and

D1 DB is overloaded. Too many requests queued.

D1 DB is overloaded. Too many requests queued.

Matthew•3/1/25, 2:52 PM

My database is about 30Mb, it gets ~50k write queries per 24 hours and I believe is well indexed

Matthew•3/1/25, 2:52 PM

Is anyone else seeing this?

James•3/1/25, 3:45 PM

Error rates on D1 are higher than most people would like. I've requested the team to document this so people understand how they should use D1 a bit better at https://github.com/cloudflare/cloudflare-docs/issues/18485, but the general wisdom unless you're seeing 100% failures is just that it's expected and you should retry your queries. Not great, I know, but sadly that's all we got for now.

Silvan•3/1/25, 4:29 PM

i have not ran into this but im assuming queries arent retried automatically either right? good to know

James•3/1/25, 4:55 PM

They are not, no, you would need to retry them yourself or implement them via another Cloudflare product that can do retries in failure conditions like Workflows, Queues, etc.

o7•3/2/25, 8:15 AM

does running a batch block the db and prevent other queries from running?

Update: Yes, very much so

1984 Ford Laser•3/2/25, 10:00 AM

Any plans to add sub-query support to generated columns, specifically when doing JSON querying? Being able to match an object from a list of objects and generate a column from that would be cool

⚙

⚙•3/2/25, 9:24 PM

#Complexity of scaling D1 in production with current limits? (10GB/db)

⚙

⚙•3/2/25, 9:25 PM

Posted this in the forum. If anyone knows the solution to this problem I'd love to read up more about it

toeyArit•3/3/25, 8:12 AM

I created a virtual FTS5 table. If I want to reduce rows_read when using ORDER BY, what approach should I take?

nukely•3/3/25, 5:13 PM

How can I reduce the 300-400ms latency of Cloudflare Workers when using D1 in free plan?

nukely•3/3/25, 5:13 PM

confused between normal and smart placement

Nnukely confused between normal and smart placement

ajgeiss0702•3/3/25, 5:51 PM

theres some explanation on smart placement here: https://developers.cloudflare.com/workers/configuration/smart-placement/

ajgeiss0702•3/3/25, 5:51 PM

basically if you enable smart placement, cloudflare will try to place the worker closer to the database rather than closer to the user, which can help reduce database latency

ajgeiss0702•3/3/25, 5:54 PM

also,

meta.duration

meta.duration

in the response object might be useful to look at, as it shows the time that ONLY the query took. If it is signifigant, consider re-evaluating the query you are running, and/or make sure you are using indexes

nukely•3/3/25, 6:58 PM

Thanks for the help

MMatthew I'm seeing lots of D1 errors for brief periods yesterday and today, but there's ...

lambrospetrouOP•3/3/25, 7:55 PM

The

storage operation exceeded timeout which caused object to be reset

storage operation exceeded timeout which caused object to be reset

is an error specific to slow user queries. What query were you trying to run? How often do you run this, and how often do you get this error?
Also, when reporting issues, putting your database ID is always helpful, otherwise we can't really help with abstract reports. In most cases this error is reported, it's a user error with a slow query or a query without indexes, so help us to help you

MMatthew My database is about 30Mb, it gets ~50k write queries per 24 hours and I believe...

lambrospetrouOP•3/3/25, 7:56 PM

Same as my other reply. Database ID and period this was happening?
Queries number across a whole day is not very relevant for this overload error. Overload error is usually because of sudden spikes, not because of total load across a day.

Llambrospetrou The `storage operation exceeded timeout which caused object to be reset` is an e...

Matthew•3/4/25, 9:02 AM

Thanks for getting back to me! Database ID is 31174a64-6897-47f9-8364-644637f90c40. Sentry shows

DB is overloaded

DB is overloaded

errors on 2025-02-28 around 14:26-14:33GMT, and

storage operation exceeded timeout

storage operation exceeded timeout

errors around 2025-03-01 14:33 GMT. I don't know exactly which queries were associated with the errors, but the Query Latency metrics show the P95 latency can't have been higher than 8-10ms. My most frequently run queries have a P95 latency of 0-3ms.

く

くもりにくもったクラウドサービスlocaler•3/4/25, 4:26 PM

Is it possible to use sql transactions in D1?

くくもりにくもったクラウドサービスlocaler Is it possible to use sql transactions in D1?

lambrospetrouOP•3/4/25, 4:30 PM

Not with interleaving JS code if that's what you mean. Each batch query or single query you send to D1 is running within a transaction.

く

くもりにくもったクラウドサービスlocaler•3/4/25, 4:38 PM

Thanks for letting me know.

MMatthew Thanks for getting back to me! Database ID is 31174a64-6897-47f9-8364-644637f90c...

lambrospetrouOP•3/4/25, 11:33 PM

I only see the

storage operation exceeded timeout

storage operation exceeded timeout

errors at that exact timestamp (

2025-03-01 14:33 GMT

2025-03-01 14:33 GMT

) on your database, nothing else before or after, so it might have been something transient or maybe a query slowdown momentarily if the machine had any issue.
Same for the overload errors. I don't see other errors except that one time.

If your queries are idempotent (e.g.

insert if not exists

insert if not exists

) or selects, then retrying them is always suggested.

We have some plan to document some of these errors and denote which are retryable, but if you design your queries to be "safe" to retry you can already do it yourself. This will pretty much cover these one-time errors.

Llambrospetrou I only see the `storage operation exceeded timeout` errors at that exact timesta...

Matthew•3/5/25, 12:46 PM

Thanks for looking into it! Have to say, there are more transient/one-off errors than I had expected when I based my app on D1, and I wasn't aware I would have to write query-retry code

andifined.dev•3/5/25, 4:40 PM

Maybe asked some time ago. But docs state some kind of dB per user. I‘m wondering how that should be possible with d1? And how to handle dB Migrations?

marko.333•3/5/25, 6:57 PM

Does anyone know if there are plans to add support for embeddings/vectors to D1 (maybe via Vectorize integration)? I would like to merge/rank D1 queries (BM25) with dense embedding ranking? For now are there any good workarounds?
Thx.

andifined.dev•3/5/25, 8:04 PM

Im looking for a example architecture on how the scale out of D1 should work for per user databases

jasonj•3/6/25, 1:20 AM

I hope this message finds you well. I’m reaching out to request your assistance with an issue I’m experiencing with my D1 database.

Currently, I’m unable to access my database with the ID 1bad90ef-990b-4ad1-908b-4bddd1f3ffd9. When I try to view its status through the Cloudflare Dashboard, it remains stuck in a "loading" state indefinitely. This issue is preventing me from managing or using the database effectively, and I’m unsure of the underlying cause.

Could you please help me investigate what might be going wrong? I’d greatly appreciate any insights or guidance you could provide to resolve this.

Thank you in advance for your support! Please let me know if you need any additional details from my side to assist with troubleshooting.

Best regards,

and I want check my timetravel by wrangler d1 time-travel info dbname

it's show me ✘ [ERROR] A request to the Cloudflare API (/accounts/{myid}/d1/database/1bad90ef-990b-4ad1-908b-4bddd1f3ffd9) failed.

internal error; reference = e_PVSMDp_1bdbed40ae1f4484a8b15b649291cf15 [code: 7500]

all other d1 is work well ,just this db is error

Jjasonj I hope this message finds you well. I’m reaching out to request your assistance ...

Walshy•3/6/25, 2:02 AM

what's your account id?
(it's safe to share)

toadfans•3/6/25, 2:12 AM

Same error:

D1_ERROR: internal error; reference = 3v37q0sukf03r4emb40aq73q

D1_ERROR: internal error; reference = 3v37q0sukf03r4emb40aq73q

, account_id:

75d8d68ecf8f9e8155f1ccc7dad82e00

75d8d68ecf8f9e8155f1ccc7dad82e00

WWalshy what's your account id? (it's safe to share)

jasonj•3/6/25, 2:24 AM

8863216c46bc0b22436dac7ad793a08f

WWalshy what's your account id? (it's safe to share)

jasonj•3/6/25, 4:41 AM

Any progress? Thanks a lot

beelzabub•3/6/25, 4:58 PM

I store base32 and base64 strings within sqlite. It would be nice if D1 supported extension functions to encode blobs as base32/base64 / decode base32/base64 strings (akin to hex/unhex that do the same)

beelzabub•3/6/25, 4:59 PM

It just makes it more convenient when running queries directly to debug stuff vs having to write JS to interpret the DB.

1. This really depends on each database. How many millions of rows, how big each row is, etc. 2. Pro

Similar Threads

Similar Threads

Similar Threads