anyone also experienced very slow queries on D1? My worker itseld responds quite fast ~50ms but when

anyone also experienced very slow queries on D1? My worker itseld responds quite fast ~50ms but when I add any D1 query it responds withing ~1000ms, which 90% of the time is spend on the query. Also from CF dashboard I see that response time is huge, while query is very fast (I basically have 0 data, just testing).
I'm on a free plan so maybe it's related but just wanted some confirmation, thx

Sszakszuk anyone also experienced very slow queries on D1? My worker itseld responds quite...

Ari•1/5/25, 9:39 PM

Same issue over here

AAri Same issue over here

Chaika•1/5/25, 9:53 PM

I would debug it using the following steps

Where is worker running (or rather, where are you connecting to)? You can check the response headers: Cf-Ray Header, at the end is the Airport code. Or you can go to the virtual path /cdn-cgi/trace/cdn-cgi/trace on your worker's hostname, look for colo=<airportCode>colo=<airportCode>. If you have Smart Placement enabled, then check the cf-placement response header, end of that is airport code.
Where is the database running? You can use npx wrangler d1 info <dbName>npx wrangler d1 info <dbName>, If you're not sure. you'll get a region returned like ENAM (East North America), or APAC (Asia Pacific)
If you're making multiple queries for a single incoming request, Smart Placement can help, or just placing your DB closer to your users
If you're connecting to a location reasonably near and your DB is near/latency is way higher then it should be, then debug the query itself.

If the query is simple, run the same query via the D1 Console Tab, below your query, you'll see
Response time 1354ms, query time 0.18ms
Query time is what you're interested in, it should be reasonable, like a few ms at max for simple queries.
Ignore Response Time via D1 Console or Wrangler, it's not representative of what you'd see in reality. It's going through API Layer -> Core (which only runs in a few locations), then to your D1 Instance. Worker/D1 Binding run at edge and connect directly to your database, not going through core/api layer/etc.

From the workers api, if using .run on a prepared statement check meta.duration for time in milliseconds spent executing. .exec has durationduration You can also do performance.now before/after query to measure end to end latency of each query and shove it into a response header or console.log it

Checking your database's graphs for query latency metrics and/or npx wrangler d1 insights <db>npx wrangler d1 insights <db> can be helpful for debugging slow queries as well

Chaika•1/5/25, 9:53 PM

same for @szakszuk

Chaika•1/5/25, 9:56 PM

usually some combination of multiple queries to far away DB/bad routing to CF/looking at d1 console/wrangler performance and not deployed worker, etc

CChaika I would debug it using the following steps 1. Where is worker running (or rathe...

Ari•1/5/25, 10:05 PM

I tested deploying a worker, the query time seems ok but the response time is too slow to be acceptable

AAri I tested deploying a worker, the query time seems ok but the response time is to...

Chaika•1/5/25, 10:10 PM

which region is your db?

CChaika which region is your db?

Ari•1/5/25, 10:19 PM

Its in WNAM, I am in Brazil, this maybe be the reason...

AAri Its in WNAM, I am in Brazil, this maybe be the reason...

Chaika•1/5/25, 10:51 PM

yea, D1 DBs/Durable Objects don't run in South America, WNAM is Western North America, like Dallas/Los Angeles/Seattle, going to be a decent latency to there from Brazil.
ENAM should be better, espec if your D1 DB gets placed in Miami, but Chicago/New Jersey are part of ENAM as well, so can vary a fair bit depending on that

Ferdinand•1/6/25, 1:34 AM

Hello, can I use D1 with server actions?

CChaika I would debug it using the following steps 1. Where is worker running (or rathe...

szakszukOP•1/6/25, 9:44 AM

Thanks that must be the reason, my worker runs in WAW (Poland) and d1 db in WNAM so ofc latency is huge. I used the automatic option for smart location assuming it will use some European region.
After moving db to eu response time is ~100ms which I guess is acceptable for my little side project

szakszukOP•1/6/25, 9:48 AM

@Chaika Just want to make sure, as I can't find any info. What it means --location=eeur vs --location=weur . I mean which exact location is that eg. city/country.

Sszakszuk @Chaika Just want to make sure, as I can't find any info. What it means --locati...

lambrospetrou•1/6/25, 11:48 AM

You cannot influence at the city level. At the moment location hints are very broad, and you can see similar hints for Durable Objects: https://developers.cloudflare.com/durable-objects/reference/data-location/#supported-locations-1
In https://where.durableobjects.live/ you can see which cities are covered in each of those hints.

Sszakszuk @Chaika Just want to make sure, as I can't find any info. What it means --locati...

Chaika•1/6/25, 2:07 PM

yea like they said there's the website for that, lots of cities in each one and it'll move within region if there's any issues/maintenance
https://where.durableobjects.live/region/weur
https://where.durableobjects.live/region/eeur

pat me•1/6/25, 3:44 PM

has anyone used auth js with cloudflare d1? since the billing is a bit more technical and I don't know what auth js is doing for sign ins and session, I am not sure what the costing would be like for how many users. any rough ideas?

lucasp•1/6/25, 5:00 PM

hi, how to backup a d1? i've tried to follow docs but dosent works

wrangler d1 backup create cyguro-db-dev

✘ [ERROR] A request to the Cloudflare API (/accounts/2d2da569ea5c41d87eba7d610279d290/d1/database/c25c6f21-3669-4672-9eaf-58242589258c/backup) failed.

  The request is malformed: Only alpha D1 databases support backups. Production databases support
  time-travel: c25c6f21-3669-4672-9eaf-58242589258c (v3) [code: 7400]

wrangler d1 backup create cyguro-db-dev

✘ [ERROR] A request to the Cloudflare API (/accounts/2d2da569ea5c41d87eba7d610279d290/d1/database/c25c6f21-3669-4672-9eaf-58242589258c/backup) failed.

  The request is malformed: Only alpha D1 databases support backups. Production databases support
  time-travel: c25c6f21-3669-4672-9eaf-58242589258c (v3) [code: 7400]

Llucasp hi, how to backup a d1? i've tried to follow docs but dosent works ```bash wrang...

Chaika•1/6/25, 5:03 PM

Manual backups like that were only an alpha thing. Production Databases support time travel like it says: https://developers.cloudflare.com/d1/reference/time-travel/#restore-a-database
You can jump back to any point in the last 30 days, nothing to do to enable

Chaika•1/6/25, 5:03 PM

You can export your prod db using the export command, if you just want the sql: https://developers.cloudflare.com/d1/best-practices/import-export-data/#export-an-existing-d1-database

CChaika You can export your prod db using the export command, if you just want the sql: ...

lucasp•1/6/25, 5:04 PM

yes i'd like to download the db tomake local tests

cosbgn•1/7/25, 7:16 AM

I'm getting quite a few of these.. Anything I could do on my end to avoid them?

Ccosbgn I'm getting quite a few of these.. Anything I could do on my end to avoid them?

lambrospetrou•1/7/25, 9:20 AM

What kind of queries are you doing? (rows accessed, row size, database size, etc)

cosbgn•1/7/25, 1:15 PM

@lambrospetrou this:

const [ widget ] = await db.select().from(schema.widgets).where( eq(schema.widgets.id, widget_id) ).limit(1)

const [ widget ] = await db.select().from(schema.widgets).where( eq(schema.widgets.id, widget_id) ).limit(1)

But I'm not sure if it always happens on this query

cosbgn•1/7/25, 1:16 PM

actually now that I opened every logs it also happened on a different route

jose luis•1/7/25, 3:23 PM

The type D1PreparedStatementD1PreparedStatement has a statementstatement property, but TSTS doesn't know about it

// This code works
const ps = env.DB.prepare(pragmaStmts[key]);
const data = await ps.all();
jsonData.push({ statement: ps.statement, [key]: data.results });
//ts complains:
`Property 'statement' does not exist on type 'D1PreparedStatement'.ts(2339)`

// This code works
const ps = env.DB.prepare(pragmaStmts[key]);
const data = await ps.all();
jsonData.push({ statement: ps.statement, [key]: data.results });
//ts complains:
`Property 'statement' does not exist on type 'D1PreparedStatement'.ts(2339)`

All the Cloudflare D1 types work as expected as far i can tell, except this one.

# npm ls
@cloudflare/workers-types@4.20241230.0

# npm ls
@cloudflare/workers-types@4.20241230.0

w3bcode•1/7/25, 3:47 PM

Is there any limit on number of D1 i can attach to a worker ? Also what would be preffered way to apply migrations when created a db through cloudflare api ?

Ww3bcode Is there any limit on number of D1 i can attach to a worker ? Also what would be...

Chaika•1/7/25, 3:50 PM

~5,000 or so https://developers.cloudflare.com/d1/platform/limits/
Using Durable Object's SQlite makes sense if you're going to have a good amount of them/create dynamically: https://blog.cloudflare.com/sqlite-in-durable-objects/
API has rate limits (1.2k / 5 mins per user) shared across dashboard and all endpoints. Durable Objects can be created/sent requests to dynamically

Jjose luis The type `D1PreparedStatement` has a `statement` property, but `TS` doesn't kno...

lambrospetrou•1/7/25, 3:57 PM

preparedStatement.statementpreparedStatement.statement is not meant to be used by client Worker code (i.e. you). You have access to the statement since you are the one providing it to the prepare()prepare() method, so use that instead.

Accessing private (as in non-documented) fields is a great recipe for future breakages when we change those fields. We do need to hide them properly (or whatever that will be in the JS world by obfuscating...) on our side though.

After you prepare()prepare() and optionally bind()bind(), only call these methods: https://developers.cloudflare.com/d1/worker-api/prepared-statements/

CChaika ~5,000 or so https://developers.cloudflare.com/d1/platform/limits/ Using Durable...

w3bcode•1/7/25, 4:11 PM

I assume durable objects would also have limits regarding storage size. Does this suit well, if we want to have multi tentant data ?

When using Durable Objects, it's important to remember that they are intended to scale out, not up. A single object is inherently limited in throughput since it runs on a single thread of a single machine. To handle more traffic, you create more objects. This is easiest when different objects can handle different logical units of state (like different documents, different users, or different "shards" of a database), where each unit of state has low enough traffic to be handled by a single object.

Ww3bcode I assume durable objects would also have limits regarding storage size. Does thi...

Chaika•1/7/25, 4:18 PM

D1's not really any different, it itself is just built on top of Durable Objects. Although currently DO SQlite while in beta has a storage limit of 1 GB per DB and D1 has 10 per GB, I would expect them to match eventually

Chaika•1/7/25, 4:19 PM

errr maybe some of this is confusion on how Durable Objects work? You code your Durable Object class and create a "Durable Object"/"Durable Object Namespace", and then you can create many instances of the Durable Object via the binding, all with their own storage limit, based on the same code

w3bcode•1/7/25, 4:20 PM

this looks interesting. I will try it out. Thanks

Chaika•1/7/25, 4:21 PM

D1's supposed to eventually have some sort of a dynamic binding api so you don't have to use the API/update worker/all that jazz, but for now Durable Object SQlite fill the gap

w3bcode•1/7/25, 4:22 PM

Ohh, I think D1 would be better choice, where we want to have backup of data, I dont know yet, how it will be supported with DO though.

Ww3bcode Ohh, I think D1 would be better choice, where we want to have backup of data, I ...

Chaika•1/7/25, 4:23 PM

DO Sqlite doesn't have exporting but it does have the same time travel stuff as D1 does

Chaika•1/7/25, 4:24 PM

https://developers.cloudflare.com/durable-objects/api/sql-storage/#point-in-time-recovery

w3bcode•1/7/25, 4:24 PM

durable-objects

Just found out about this in other channel.

Chaika•1/7/25, 4:25 PM

it's just manually looping overall the tables/fetching all the data https://github.com/outerbase/starbasedb/blob/main/src/export/dump.ts
should work, might incur a decent amount of reads but cheap

Llambrospetrou `preparedStatement.statement` is not meant to be used by client Worker code (i.e...

jose luis•1/7/25, 5:17 PM

Thanks for the answer. It's a shame though, because to me it's a useful piece of read-only information in the context of a prepared statement.

cmrpyro•1/8/25, 4:30 AM

hey folks. using the wrangler d1wrangler d1 command, is there any way to change the output mode to be insertinsert?

i'm trying to grab a few rows from prod and bring them to my local, its giving me JSON regardless of the --json--json flag value

morning_eagle•1/8/25, 6:33 AM

Can I use cloudflare D1 in an express.js app hosted outside of Cloudflare? In a VPS?

Mmorning_eagle Can I use cloudflare D1 in an express.js app hosted outside of Cloudflare? In a ...

Rúben Fernandes•1/8/25, 6:36 AM

https://developers.cloudflare.com/api/resources/d1/

Cloudflare API | D1

Interact with Cloudflare's products and services via the Cloudflare API

junaid•1/8/25, 7:43 AM

hi anyone can help me to migrate mysql to d1 We are currently attempting to migrate a 20GB MySQL database to Cloudflare D1. However, we’ve encountered challenges with file size limitations and memory constraints when using the wrangler CLI tool. Specifically, we received the following error while trying to execute the SQL file locally:

✘ [ERROR] Could not read file: /path/to/large-file.sql
Cannot create a string longer than 0x1fffffe8 characters

RRúben Fernandes https://developers.cloudflare.com/api/resources/d1/

Max (@rozenmd)•1/8/25, 8:55 AM

This approach would be subject to Cloudflare API's rate limit (1200 requests over 5 minutes)

Mmorning_eagle Can I use cloudflare D1 in an express.js app hosted outside of Cloudflare? In a ...

Max (@rozenmd)•1/8/25, 8:56 AM

You'd want a pass through Worker like https://github.com/elithrar/http-api-d1-example to talk to from Express for best performance

GitHub

GitHub - elithrar/http-api-d1-example: An example HTTP API for Clou...

An example HTTP API for Cloudflare's D1 database: https://developers.cloudflare.com/d1/

elithrar/http-api-d1-example

oscar•1/8/25, 10:26 AM

Hello! we're seeing some "put: We encountered an internal error. Please try again. (10001)" in our logs in the last few minutes. Notifying this because it's something we see more or less on a weekly basis.

Ooscar Hello! we're seeing some "put: We encountered an internal error. Please try agai...

Max (@rozenmd)•1/8/25, 10:41 AM

You're seeing that error thrown when running a D1 Client API function in your Worker?

oscar•1/8/25, 11:33 AM

Yes, this is production remote though, the error was caught and I received it at 11:12AM Barcelona time

Ooscar Yes, this is production remote though, the error was caught and I received it at...

Max (@rozenmd)•1/8/25, 11:35 AM

and you're not calling any other service in that Worker?

kaede•1/8/25, 2:17 PM

Can someone explain this please

Walshy•1/8/25, 2:17 PM

#r2

MMax (@rozenmd)and you're not calling any other service in that Worker?

oscar•1/8/25, 2:18 PM

honestly haven't had the time to dig deeper on that specific call but it's most probably an alarm trigger on a DO in a Worker that also interfaces with R2, KV and other Workers. Each specific DO alarm trigger may or may not interface with each of these services depending on the characteristics of the long running job it's running.

anyone also experienced very slow queries on D1? My worker itseld responds quite fast ~50ms but when

Similar Threads

Similar Threads

Similar Threads