After hitting the API about 10 times, it suddenly became faster. Does this mean that the location of

After hitting the API about 10 times, it suddenly became faster.
Does this mean that the location of the Durable Object hosting D1 has moved?

Ssor4chi/monica After hitting the API about 10 times, it suddenly became faster. Does this mean ...

lambrospetrou•12/10/24, 10:42 AM

After creation the database doesn't move no. The first query is always going to be slow since you are cold starting your database, its location being cached in Cloudflare's CDN etc.

Llambrospetrou After creation the database doesn't move no. The first query is always going to ...

sor4chi/monicaOP•12/10/24, 10:44 AM

Thank you, I understand now.
I apologize if my understanding is incorrect, but I had heard that D1 delivers “read replicas to the edge,” so I assumed that READ operations would be reasonably fast regardless of the location.

Ssor4chi/monica Thank you, I understand now. I apologize if my understanding is incorrect, but I...

lambrospetrou•12/10/24, 10:54 AM

Read replication is coming soon yes, but it's not rolled out yet.
Even with read replication though, you are still going to go over the network on a different machine, or location even, so it's not going to be a single-digit millisecond solution either (it cannot be due to physics!).

o7•12/10/24, 11:10 AM

if i run a count + 1count + 1 update query in a batch statement 1000 times, will it end up being 1000 at the end? ie batches are transactional?

Original message was deleted

o7•12/10/24, 12:11 PM

thanks !

Llambrospetrou Unfortunately, region-wise location hints are the only way to influence its loca...

o7•12/10/24, 12:24 PM

I know promises cant be made, but would this essentially just consist of a finer control for location hints?
e.g. when dynamic bindings arrive, and I create a DB for each client, I can ensure the DB spawns in as close as possible to them. Instead of randomly getting a DB handed to you in a random city.

This would be amazing especially if R2 matched this behavior when doing D1 + R2 writes together, which would improve performance by a lot...

Original message was deleted

Max (@rozenmd)•12/10/24, 1:56 PM

for pure query duration? seems pretty slow - does your WHERE clause hit your index?

Original message was deleted

lambrospetrou•12/10/24, 3:10 PM

How are you measuring this duration? How much data are those rows?

Original message was deleted

lambrospetrou•12/10/24, 3:14 PM

Couple hundreds rows, but how much data? 200KB, 2MB, 200MB?

Original message was deleted

lambrospetrou•12/10/24, 3:14 PM

So this is from your worker code, not the

meta.duration

meta.duration

field of the D1 response, right? If yes, tens of milliseconds is expected.
If you want to see the actual SQL query duration check the

meta.duration

meta.duration

field.

Original message was deleted

Max (@rozenmd)•12/10/24, 3:18 PM

Yeah

meta.duration

meta.duration

tells you how long it actually took - anything else gives you network noise

Original message was deleted

Max (@rozenmd)•12/10/24, 3:22 PM

meta.duration

meta.duration

measures how long sqlite took to actually get your data

measuring from your Worker includes time it took to reach D1's Worker, time for D1's Worker to reach the DB, and back

In short,

meta.duration

meta.duration

is all you can actually influence (apart from enabling smart placement, I guess)

Max (@rozenmd)•12/10/24, 3:23 PM

The Workers Platform is built on Workers

Max (@rozenmd)•12/10/24, 3:25 PM

I haven't used it apart from a demo or two, but I'd expect some kind of metadata returned from the .all().all() function?

Original message was deleted

Max (@rozenmd)•12/10/24, 3:30 PM

ah apparently not: https://github.com/drizzle-team/drizzle-orm/issues/1811

GitHub

[FEATURE]: Enhanced Metadata Support for Cloudflare D1 · Issue #181...

The Cloudflare D1 API can provide valuable metadata, including operation duration and rows read. As D1 pricing is based on rows read, incorporating this metadata into Drizzle queries would be immen...

James•12/10/24, 4:04 PM

I found drizzle has just enough edge cases to not trust it, and just stick to raw sql.

I do use drizzle to generate migrations and then do custom type casting w the drizzle schema definitions I make.

CedricHadjian•12/10/24, 4:12 PM

Hi peeps, we got a table that has 12,000,000 rows and we're facing this error:

D1_ERROR: D1 DB is overloaded. Requests queued for too long.

D1_ERROR: D1 DB is overloaded. Requests queued for too long.

This is the query that we're running:

SELECT query, COUNT(*) as occurrences FROM queries GROUP BY query ORDER BY occurrences DESC LIMIT ?;

SELECT query, COUNT(*) as occurrences FROM queries GROUP BY query ORDER BY occurrences DESC LIMIT ?;

We're passing a limit of 10 and still getting this error, what can we do to bypass the error?

James•12/10/24, 4:31 PM

Drizzle has some bugs i got irritated with, batch queries breaks when you have multiple schemas in the return set, some date js typing bugs

Original message was deleted

CedricHadjian•12/10/24, 4:38 PM

Yeah :/ these are our searches, any other way I could retrieve most searched queries?

James•12/10/24, 4:39 PM

example, ran into this bug, still open, have to patch my build until i migrate off drizzle for one set of my app
https://github.com/drizzle-team/drizzle-orm/issues/2323

GitHub

[BUG]: value.getTime is not a function · Issue #2323 · drizzle-tea...

What version of drizzle-orm are you using? 0.30.10 What version of drizzle-kit are you using? 0.21.2 Describe the Bug Bug Report: Error when using time-related queries Description When attempting t...

CCedricHadjian Yeah :/ these are our searches, any other way I could retrieve most searched que...

James•12/10/24, 4:40 PM

run a query plan and add the right index

Original message was deleted

CedricHadjian•12/10/24, 4:42 PM

I've never tried it before, lemme check out what it is, thanks!

JJames run a query plan and add the right index

CedricHadjian•12/10/24, 4:44 PM

Would you elaborate?

Original message was deleted

Max (@rozenmd)•12/10/24, 4:47 PM

I've been using raw SQL with my paid side projects for about 8 years now, it's honestly fine (and if you're strict about typing your query output, you won't notice the difference)

Max (@rozenmd)•12/10/24, 4:47 PM

(we also use raw SQL internally for D1)

MMax (@rozenmd)I've been using raw SQL with my paid side projects for about 8 years now, it's h...

Isaac McFadyen•12/10/24, 10:53 PM

^ I'd advocate the use of

zod

zod

or similar because it enforces exact 1:1 match between types and actual output share versus types drifting away from the actual type

Isaac McFadyen•12/10/24, 10:53 PM

IMO it's better to have an error thrown from type mismatch than have types that aren't actually correct

Kisaragi•12/11/24, 12:21 PM

Is there a way in D1 table schema to define a uuid column?
The id INTEGER PRIMARY KEY AUTOINCREMENTid INTEGER PRIMARY KEY AUTOINCREMENT is only one-digit, I want to replace this with 10-digit uuid. Anyone got a solution for this?
Thanks!!

Kisaragi•12/11/24, 12:29 PM

Dug around a bit, I guess manually generating id = str(uuid.uuid4())id = str(uuid.uuid4()) is the easiest. I thought there might be some native methods from cloudflare side.

Llimzykenneth Is something going on for D1 at the moment as I'm seeing gateway timeout when ru...

TeeLusk•12/11/24, 10:31 PM

We see it every so often on one of our apps as well. Seems to happen the first time the db is requested after sitting idle for a while (like in a dev environment) but then works after the first failure. Haven’t tracked down a cause

AAlexander Thoukydides I am currently seeing a lot of `500 Internal Server Error` errors from D1.

Max (@rozenmd)•12/12/24, 11:49 AM

Whats your database id?

AAlexander Thoukydides I am currently seeing a lot of `500 Internal Server Error` errors from D1.

Max (@rozenmd)•12/12/24, 11:57 AM

Also where are you seeing the errors? wrangler/dash, HTTP API, your own Worker?

Mm0c Random question, a D1 read on an index, that has 0 rows as result, would that al...

lambrospetrou•12/13/24, 12:14 AM

It's not the rows in the result that count, but how many rows are read as part of the query execution.

MMatt Silverlock With the initial support for Python + Workers (https://blog.cloudflare.com/pytho...

Someone•12/13/24, 1:04 AM

How would I get just the resultsresults from the return Response.json(results) area? I'm getting this:

{"success":true,"meta":{"served_by":"v3-prod","duration":0.1977,"changes":0,"last_row_id":0,"changed_db":false,"size_after":16384,"rows_read":1,"rows_written":0},"results":[{"test":"a"}]}

{"success":true,"meta":{"served_by":"v3-prod","duration":0.1977,"changes":0,"last_row_id":0,"changed_db":false,"size_after":16384,"rows_read":1,"rows_written":0},"results":[{"test":"a"}]}

Someone•12/13/24, 1:05 AM

results.get('results')results.get('results') throws an error. Also, type(results)type(results) is

undefined

undefined

dev•12/13/24, 4:09 AM

is pages/d1 available in Canada when using the enterprise plan?

Ddev is pages/d1 available in Canada when using the enterprise plan?

Isaac McFadyen•12/13/24, 4:12 AM

Is this for data residency? Both services are available in Canada on all plans but the data doesn't live in Canada.

Isaac McFadyen•12/13/24, 4:13 AM

See https://where.durableobjects.live for D1 locations. They're not 1:1 but you can get a sense of where D1 doesn't live based on the colos without Durable Objects on that map (which is all of Canada).

Isaac McFadyen•12/13/24, 4:13 AM

Pages uses KV which has 2 central stores, 1 in the US and one in the EU (so again, not in Canada but a Pages site can be accessed from Canada).

IIsaac McFadyen Is this for data residency? Both services are *available* in Canada on all plans...

dev•12/13/24, 4:42 AM

it is, yes. data needs to live in Canada for compliance reasons. I know CF doesn't have that region available for regular users, but I thought it was worth asking if Enterprise customers had that option.

dev•12/13/24, 4:42 AM

that's basically an entire segment of apps that can't be hosted in CF due to that

dev•12/13/24, 4:42 AM

in my case, edtech

Ddev it is, yes. data needs to live in Canada for compliance reasons. I know CF doesn...

Hello, I’m Allie!•12/13/24, 4:50 AM

It may help to ask your CSM anyway, as expressing interest in supporting a certain region may make them more likely to prioritize that region in the future

Hello, I’m Allie!•12/13/24, 4:50 AM

No guarantees either way, but I don’t think it could hurt

SSomeone How would I get just the `results` from the return Response.json(results) area? ...

lambrospetrou•12/13/24, 9:44 AM

The response is an object, not a MapMap so the

.get()

.get()

doesn't exist on objects. You can just do response.resultsresponse.results.

edit: Just realized you are talking about Python, not JavaScript. Not sure if that's different.

After hitting the API about 10 times, it suddenly became faster. Does this mean that the location of

Similar Threads

Similar Threads

Similar Threads