Hey Team, I am based in Sydney Australia, and trying D1 + Cloudflare pages using Drizzle ORM. I am e

Hey Team, I am based in Sydney Australia, and trying D1 + Cloudflare pages using Drizzle ORM.
I am experiencing, each DB Call takes at least > 230 miliseconds.
I found multiple forum, people discussing D1 queries taking time. (I have created indexes, but that should not be a problem, since my DB is very small , with few test data in KBs).
Is this latency due to D1 not closser to my Pages Worker?

Aanand248 Hey Team, I am based in Sydney Australia, and trying D1 + Cloudflare pages using...

1984 Ford Laser•3/6/25, 9:18 PM

Smart Placement might help, are your requests going to the Sydney colo or elsewhere?

BiffBaffBoff•3/6/25, 9:40 PM

Hi, if you have multiple workers needing to talk to the same d1 instance when doing local development? How do you achieve that?

/root
  /app1
     wrangler.jsonc -> "d1_databases": [{ "binding": "DB", "database_name": "a-shared-name", "database_id": "1234..." }]
  /app2
     wrangler.jsonc -> "d1_databases": [{ "binding": "DB", "database_name": "a-shared-name", "database_id": "1234..." }]

/root
  /app1
     wrangler.jsonc -> "d1_databases": [{ "binding": "DB", "database_name": "a-shared-name", "database_id": "1234..." }]
  /app2
     wrangler.jsonc -> "d1_databases": [{ "binding": "DB", "database_name": "a-shared-name", "database_id": "1234..." }]

If I run both workers, they each create a .wrangler.wrangler directory with each state independant from each other. Thanks

James•3/6/25, 10:27 PM

'persist' option on wrangler when you run it, pick one dir for both

11984 Ford Laser Smart Placement *might* help, are your requests going to the Sydney colo or else...

anand248OP•3/7/25, 5:53 AM

how to figure out, where request goes when we query D1 database?

1984 Ford Laser•3/7/25, 5:59 AM

In the response from the Worker, look at the cf: colocf: colo header

anand248OP•3/7/25, 6:06 AM

Thanks, I get-

cf-placement: local-SYD
cf-ray: 91c7d72ecb3a572d-SYD

cf-placement: local-SYD
cf-ray: 91c7d72ecb3a572d-SYD

Isn;t it about the location of pages or worker? I am curious about my D1 DB location where read happens, I guess for each query to take >240s, it seem it is network latency.

LLazyGeniusMan hello, I have monorepo setup with this structure ``` ├── apps/ │ ├── foo-worke...

Silvan•3/7/25, 11:08 AM

if you scroll up 5 messages theres your answer x)

BBiffBaffBoff Hi, if you have multiple workers needing to talk to the same d1 instance when do...

! Nakoshi 恨•3/7/25, 2:10 PM

Hello

Aanand248 Thanks, I get- ``` cf-placement: local-SYD cf-ray: 91c7d72ecb3a572d-SYD ``` Isn...

lambrospetrou•3/7/25, 6:51 PM

Every D1 response now has a

meta.served_by_region

meta.served_by_region

information. That's the location (region) of your database instance that processed the query.
To get the location of your worker, as others mentioned above you can use Cloudflare headers of the request telling you the

colo

colo

: https://developers.cloudflare.com/workers/runtime-apis/request/#incomingrequestcfproperties
Or fetch the URL https://cloudflare.com/cdn-cgi/trace from within your worker and check the

colo=

colo=

line.

Iiandees Is it possible to build a Sqlite database separately, publish it into D1, and us...

lambrospetrou•3/7/25, 6:53 PM

What does "build an SQLite database separately" mean? You mean uploading an existing SQLite database file?
If yes, you can use the import API: https://developers.cloudflare.com/d1/best-practices/import-export-data/
If not, I don't understand the question, so please elaborate.

Llambrospetrou Every D1 response now has a `meta.served_by_region` information. That's the loca...

anand248OP•3/7/25, 10:04 PM

I mange to find DB location from command

wrangler d1 info <db_name>

wrangler d1 info <db_name>

, which tells me my db is in Wesern north america, while the only time i ussed it ws from AU. Now I am moving my DB Back to AU by creating new db/exporting data to it.

anand248OP•3/7/25, 10:11 PM

command

wrangler d1 export --remote <db_name> --output=x_backup.sql

wrangler d1 export --remote <db_name> --output=x_backup.sql

does not create DDL & DML in correct order. (Table being referred get created later.).

Iiandees is the database size limit something that can be increased? a couple of the data...

Tait•3/8/25, 12:39 AM

Pretty sure the answer to this is no. Cloudflare announced some D1 changes last April. Maybe they'll have more updates around the same time this year.

JOEY•3/8/25, 10:22 AM

▲ [WARNING] No migrations folder found. Set

migrations_dir

migrations_dir

in your wrangler.json file to choose a different path.

i;m getting this error why?

PS : Solved it, was a stupid fix

JOEY•3/8/25, 10:22 AM

i already set up the path

く

くもりにくもったクラウドサービスlocaler•3/9/25, 8:25 AM

Is it possible to do something like enum type with d1?

James•3/9/25, 3:17 PM

sqlite doesn't support enums directly, but you could use a

CHECK

CHECK

constraint on your column to accomplish a similar result.

disaster.js•3/9/25, 4:17 PM

Make a not null foreign key constraint to an enumerated table

TTait Pretty sure the answer to this is no. Cloudflare announced some D1 changes last...

ajgeiss0702•3/9/25, 6:56 PM

yeah they bank a bunch of updates for developer week each year

ajgeiss0702•3/9/25, 7:01 PM

im personally hoping for some replica news, but we'll see

Aajgeiss0702 im personally hoping for some replica news, but we'll see

Tait•3/9/25, 11:32 PM

Hoping for larger max size of databases or some easy way in workers to dynamically access many databases I shard accounts to...

Disciple of Caoximeihao•3/10/25, 6:33 AM

hiep@MBP-Hiep payalert-backend-v2 % wrangler d1 execute pay-alert --local --file=./database.sql

✘ [ERROR] Could not read file: ./database.sql

Cannot create a string longer than 0x1fffffe8 characters

HELP: database sql file 2GB

dimsid•3/10/25, 9:12 AM

Hello! I am using a D1 database and have golang script for upserting some rows in D1.

The cloudflare-go sdk that i am using is v0.89.0.

Everything worked good until last Monday that i started getting this error:

failed to upsert customers to D1 {"error": "error from makeRequest: received internal server error response (HTTP 500), please try again later"}

Any known issue or api change?
Wasn't able to find a relevant changelog

Ddimsid Hello! I am using a D1 database and have golang script for upserting some rows i...

lambrospetrou•3/10/25, 10:23 AM

If you query your database directly or through the dashboard UI console does it work? That error message is not a D1 error so not sure which layers fails for you there.

Llambrospetrou If you query your database directly or through the dashboard UI console does it ...

dimsid•3/10/25, 10:37 AM

yes quering the d1 directly from the Ui console works.

I have noticed some other api compatibility issues that i have fixed.

My code was originally like this:

res, err := c.api.QueryD1Database(c.ctx, cloudflare.AccountIdentifier(c.cfg.Cloudflare.AccountID), cloudflare.QueryD1DatabaseParams{
DatabaseID: c.cfg.Cloudflare.DBID,
SQL: sqlQuery.String(),
// error was originally here
})

The first error that i started getting was this:

Invalid property: params => Expected array, received null (7400)"}

When i changed the code to include a string array with params i started getting:

failed to upsert customers to D1 {"error": "error from makeRequest: received internal server error response (HTTP 500), please try again later"}

This started happening one week ago.

Ddimsid yes quering the d1 directly from the Ui console works. I have noticed some oth...

lambrospetrou•3/10/25, 10:50 AM

We recently added some validation on that endpoint. The

Invalid property: params => Expected array, received null (7400)

Invalid property: params => Expected array, received null (7400)

seems to be coming from that validation. The

params

params

is an optional list argument that should only be provided if you have bind parameters in the SQL query.

I will see if we can make the

params === null

params === null

be treated the same is not provided at all, since

null

null

is not a valid value per se for that list argument.

When i changed the code to include a string array with params i started getting:

Not sure what you mean here. Did you provide an empty slice?

Llambrospetrou We recently added some validation on that endpoint. The `Invalid property: param...

dimsid•3/10/25, 10:59 AM

I mean i fixed the 7400 null error by passing an empty string array. But after fixing this error i started having the ""error": "error from makeRequest: received internal server error response (HTTP 500), please try again later"}"

ambergristle•3/10/25, 11:10 AM

hi! can someone confirm that "bound parameters" refers to the total number of args passed into the query?

ambergristle•3/10/25, 11:12 AM

*number of placeholders

Llambrospetrou We recently added some validation on that endpoint. The `Invalid property: param...

lambrospetrou•3/10/25, 11:13 AM

The SDK docs explicitly mention that non-provided fields are not sent: https://github.com/cloudflare/cloudflare-go/tree/main?tab=readme-ov-file#request-fields

All request parameters are wrapped in a generic Field type, which we use to distinguish zero values from null or omitted fields.
This prevents accidentally sending a zero value if you forget a required parameter, and enables explicitly sending null, false, '', or 0 on optional parameters. Any field not specified is not sent.

Weird that in your case @dimsid

null

null

is being sent for

params

params

. Did you not provide it at all (as your snippet above implies), or did you provide it explicitly as null?

Can you upgrade your SDK version used to https://pkg.go.dev/github.com/cloudflare/cloudflare-go/v4#section-readme and try that?

Llambrospetrou The SDK docs explicitly mention that non-provided fields are not sent: https://g...

dimsid•3/10/25, 1:19 PM

Let me rephrase my problem so that i can be clear!

I have a golang script using cloudflare-go sdk version 0.89.0.

In my scipt there was this code:

res, err := c.api.QueryD1Database(c.ctx, cloudflare.AccountIdentifier(c.cfg.Cloudflare.AccountID), cloudflare.QueryD1DatabaseParams{
DatabaseID: c.cfg.Cloudflare.DBID,
SQL: sqlQuery,
})

and it was working.

Last Monday 3 March 2025, this script stopped working, with this error:

Invalid property: params => Expected array, received null (7400).

Following the error message, I updated my code to this:

res, err := c.api.QueryD1Database(c.ctx, cloudflare.AccountIdentifier(c.cfg.Cloudflare.AccountID), cloudflare.QueryD1DatabaseParams{
DatabaseID: c.cfg.Cloudflare.DBID,
SQL: sqlQuery,
Parameters: []string{},
})

and the 7400 above was gone but a new error ocuured which is this:

error from makeRequest: received internal server error response (HTTP 500), please try again later

And I am wondering if except from the validation check you mentioned, something else also changed and my query does not work.

As mentioned i am using sdk version 0.89.0 and I would not like to make a version upgrade to v4 if possible

Ddimsid Let me rephrase my problem so that i can be clear! I have a golang script using...

lambrospetrou•3/10/25, 1:26 PM

As I said, there is a new validation now on our API for that endpoint. The error you get

Invalid property: params => Expected array, received null (7400)

Invalid property: params => Expected array, received null (7400)

is from that validation. That old version of the Go SDK I believe sends

null

null

when it should not send anything.

I already made an improvement to allow

null

null

s for now for these cases, but it needs some time to be rolled out.

I do suggest upgrading at some point though to make sure you get the improvements of the SDK.

Llambrospetrou As I said, there is a new validation now on our API for that endpoint. The error...

dimsid•3/10/25, 1:39 PM

Hello thanks for all the replies! As i described above the 7400 error was fixed. What about the "error from makeRequest: received internal server error response (HTTP 500), please try again later"

Ddimsid Hello thanks for all the replies! As i described above the 7400 error was fixed....

lambrospetrou•3/10/25, 1:44 PM

It's not fixed really. By passing empty array you trigger some other flow to fail since the params should not be empty. That's another thing to be fixed somewhere across the path, either in the SDK or the API itself to return a proper error instead of 500.

I am rolling out the validation fix to not fail with

null

null

for now so your original code should continue working once that's out everywhere.

Llambrospetrou It's not fixed really. By passing empty array you trigger some other flow to fai...

dimsid•3/10/25, 1:56 PM

Thank you! I would appreciate if you could let me know with a reply here, when this is rolled out

Ddimsid Thank you! I would appreciate if you could let me know with a reply here, when t...

lambrospetrou•3/10/25, 3:44 PM

Should be complete now. Please let me know if you still get that validation error.

Llambrospetrou Should be complete now. Please let me know if you still get that validation erro...

dimsid•3/10/25, 5:20 PM

Thanks a lot for all your support! it is working now

LLeesy Is anyone seeing random errors on D1? I have a DB bound to a worker and I keep g...

lambrospetrou•3/11/25, 11:13 AM

Make sure you don't have any typos when querying the table. In the UI console run

select * from sqlite_master;

select * from sqlite_master;

and see if the output contains the table you expect to be there.

lambrospetrou•3/11/25, 11:31 AM

This is not a bug that can happen randomly. The query must be wrong in the worker that fails. Either the table always fails to be found, or never.

/root /app1 wrangler.jsonc -> "d1_databases": [{ "binding": "DB", "database_name": "a-shared-name", "database_id": "1234..." }] /app2 wrangler.jsonc -> "d1_databases": [{ "binding": "DB", "database_name": "a-shared-name", "database_id": "1234..." }]

Hey Team, I am based in Sydney Australia, and trying D1 + Cloudflare pages using Drizzle ORM. I am e

Similar Threads

Similar Threads

Similar Threads