Hello! I am using a D1 database and have golang script for upserting some rows in D1. The cloudfla

Hello! I am using a D1 database and have golang script for upserting some rows in D1.

The cloudflare-go sdk that i am using is v0.89.0.

Everything worked good until last Monday that i started getting this error:

failed to upsert customers to D1 {"error": "error from makeRequest: received internal server error response (HTTP 500), please try again later"}

Any known issue or api change?
Wasn't able to find a relevant changelog

Ddimsid Hello! I am using a D1 database and have golang script for upserting some rows i...

lambrospetrou•3/10/25, 10:23 AM

If you query your database directly or through the dashboard UI console does it work? That error message is not a D1 error so not sure which layers fails for you there.

Llambrospetrou If you query your database directly or through the dashboard UI console does it ...

dimsidOP•3/10/25, 10:37 AM

yes quering the d1 directly from the Ui console works.

I have noticed some other api compatibility issues that i have fixed.

My code was originally like this:

res, err := c.api.QueryD1Database(c.ctx, cloudflare.AccountIdentifier(c.cfg.Cloudflare.AccountID), cloudflare.QueryD1DatabaseParams{
DatabaseID: c.cfg.Cloudflare.DBID,
SQL: sqlQuery.String(),
// error was originally here
})

The first error that i started getting was this:

Invalid property: params => Expected array, received null (7400)"}

When i changed the code to include a string array with params i started getting:

failed to upsert customers to D1 {"error": "error from makeRequest: received internal server error response (HTTP 500), please try again later"}

This started happening one week ago.

Ddimsid yes quering the d1 directly from the Ui console works. I have noticed some oth...

lambrospetrou•3/10/25, 10:50 AM

We recently added some validation on that endpoint. The

Invalid property: params => Expected array, received null (7400)

Invalid property: params => Expected array, received null (7400)

seems to be coming from that validation. The

params

params

is an optional list argument that should only be provided if you have bind parameters in the SQL query.

I will see if we can make the

params === null

params === null

be treated the same is not provided at all, since

null

null

is not a valid value per se for that list argument.

When i changed the code to include a string array with params i started getting:

Not sure what you mean here. Did you provide an empty slice?

Llambrospetrou We recently added some validation on that endpoint. The `Invalid property: param...

dimsidOP•3/10/25, 10:59 AM

I mean i fixed the 7400 null error by passing an empty string array. But after fixing this error i started having the ""error": "error from makeRequest: received internal server error response (HTTP 500), please try again later"}"

ambergristle•3/10/25, 11:10 AM

hi! can someone confirm that "bound parameters" refers to the total number of args passed into the query?

ambergristle•3/10/25, 11:12 AM

*number of placeholders

Llambrospetrou We recently added some validation on that endpoint. The `Invalid property: param...

lambrospetrou•3/10/25, 11:13 AM

The SDK docs explicitly mention that non-provided fields are not sent: https://github.com/cloudflare/cloudflare-go/tree/main?tab=readme-ov-file#request-fields

All request parameters are wrapped in a generic Field type, which we use to distinguish zero values from null or omitted fields.
This prevents accidentally sending a zero value if you forget a required parameter, and enables explicitly sending null, false, '', or 0 on optional parameters. Any field not specified is not sent.

Weird that in your case @dimsid

null

null

is being sent for

params

params

. Did you not provide it at all (as your snippet above implies), or did you provide it explicitly as null?

Can you upgrade your SDK version used to https://pkg.go.dev/github.com/cloudflare/cloudflare-go/v4#section-readme and try that?

Llambrospetrou The SDK docs explicitly mention that non-provided fields are not sent: https://g...

dimsidOP•3/10/25, 1:19 PM

Let me rephrase my problem so that i can be clear!

I have a golang script using cloudflare-go sdk version 0.89.0.

In my scipt there was this code:

res, err := c.api.QueryD1Database(c.ctx, cloudflare.AccountIdentifier(c.cfg.Cloudflare.AccountID), cloudflare.QueryD1DatabaseParams{
DatabaseID: c.cfg.Cloudflare.DBID,
SQL: sqlQuery,
})

and it was working.

Last Monday 3 March 2025, this script stopped working, with this error:

Invalid property: params => Expected array, received null (7400).

Following the error message, I updated my code to this:

res, err := c.api.QueryD1Database(c.ctx, cloudflare.AccountIdentifier(c.cfg.Cloudflare.AccountID), cloudflare.QueryD1DatabaseParams{
DatabaseID: c.cfg.Cloudflare.DBID,
SQL: sqlQuery,
Parameters: []string{},
})

and the 7400 above was gone but a new error ocuured which is this:

error from makeRequest: received internal server error response (HTTP 500), please try again later

And I am wondering if except from the validation check you mentioned, something else also changed and my query does not work.

As mentioned i am using sdk version 0.89.0 and I would not like to make a version upgrade to v4 if possible

Ddimsid Let me rephrase my problem so that i can be clear! I have a golang script using...

lambrospetrou•3/10/25, 1:26 PM

As I said, there is a new validation now on our API for that endpoint. The error you get

Invalid property: params => Expected array, received null (7400)

Invalid property: params => Expected array, received null (7400)

is from that validation. That old version of the Go SDK I believe sends

null

null

when it should not send anything.

I already made an improvement to allow

null

null

s for now for these cases, but it needs some time to be rolled out.

I do suggest upgrading at some point though to make sure you get the improvements of the SDK.

Llambrospetrou As I said, there is a new validation now on our API for that endpoint. The error...

dimsidOP•3/10/25, 1:39 PM

Hello thanks for all the replies! As i described above the 7400 error was fixed. What about the "error from makeRequest: received internal server error response (HTTP 500), please try again later"

Ddimsid Hello thanks for all the replies! As i described above the 7400 error was fixed....

lambrospetrou•3/10/25, 1:44 PM

It's not fixed really. By passing empty array you trigger some other flow to fail since the params should not be empty. That's another thing to be fixed somewhere across the path, either in the SDK or the API itself to return a proper error instead of 500.

I am rolling out the validation fix to not fail with

null

null

for now so your original code should continue working once that's out everywhere.

Llambrospetrou It's not fixed really. By passing empty array you trigger some other flow to fai...

dimsidOP•3/10/25, 1:56 PM

Thank you! I would appreciate if you could let me know with a reply here, when this is rolled out

Ddimsid Thank you! I would appreciate if you could let me know with a reply here, when t...

lambrospetrou•3/10/25, 3:44 PM

Should be complete now. Please let me know if you still get that validation error.

Llambrospetrou Should be complete now. Please let me know if you still get that validation erro...

dimsidOP•3/10/25, 5:20 PM

Thanks a lot for all your support! it is working now

LLeesy Is anyone seeing random errors on D1? I have a DB bound to a worker and I keep g...

lambrospetrou•3/11/25, 11:13 AM

Make sure you don't have any typos when querying the table. In the UI console run

select * from sqlite_master;

select * from sqlite_master;

and see if the output contains the table you expect to be there.

lambrospetrou•3/11/25, 11:31 AM

This is not a bug that can happen randomly. The query must be wrong in the worker that fails. Either the table always fails to be found, or never.

Sveinn•3/11/25, 3:54 PM

Hey,

A few weeks ago, my Cloudflare account started racking up a massive bill. I reached out to support to figure out what was going on, but it took around 20 days to get a response. The reply just said the billing was correct but didn’t explain what was actually causing the charges. I asked for clarification about a week ago but still haven’t gotten an answer.

After digging into it myself, I found that a simple uptime check my load balancer does every minute was somehow pulling tens of thousands of rows each time, leading to crazy charges. I’ve stopped the uptime check, but now I’m stuck with a credit card bill of thousands of dollars for my small hobby project.

Is there any way to get in touch with support to discuss these charges? @lambrospetrou

SSveinn Hey, A few weeks ago, my Cloudflare account started racking up a massive bill. ...

Vy•3/11/25, 4:30 PM

@Sveinn can you DM me your support case?

cosbgn•3/12/25, 1:38 PM

I think this graph is off or weird

cosbgn•3/12/25, 1:38 PM

by db size is about 7mb, but it adds 7mb each day

cosbgn•3/12/25, 1:39 PM

like it shows like it would now be 80mb, I think it should only add the delta daily

SSveinn Hey, A few weeks ago, my Cloudflare account started racking up a massive bill. ...

cosbgn•3/12/25, 1:40 PM

I've been begging cloudflare to add a max-spend limit since forever

Ccosbgn I've been begging cloudflare to add a max-spend limit since forever

Sveinn•3/12/25, 1:58 PM

Spend-limit would be awesome

Spuckwaffel•3/13/25, 1:46 PM

any way to export the DB, make modifications and then reload the DB? Have to alter a table and adding a foreign key constraint and cant delete the table

Kenny•3/13/25, 3:34 PM

I have two workers that bind into the same D1 database:

...
[[d1_databases]]
binding = "FIRESTORM_DB"
database_name = "firestorm-db"
database_id = "c020574a-5623-407b-be0c-cd192bab9545"
migrations_dir = "../d1/migrations"

...
[[d1_databases]]
binding = "FIRESTORM_DB"
database_name = "firestorm-db"
database_id = "c020574a-5623-407b-be0c-cd192bab9545"
migrations_dir = "../d1/migrations"

I have set the database id to be the same for both. However, when I run this locally for testing and start up one worker and have it add data to the db the other worker has no recollection of it. Is there a fix anyone knows about?

Kenny•3/13/25, 3:36 PM

I should also mention that I went into each .wrangler/state/d1 to check the sqlite data directly and it seems that each one spins up it's own sqlite db for local testing. I looked through the docs and haven't found anything that could solve it yet.

KKenny I have two workers that bind into the same D1 database: ``` ... [[d1_databases]]...

lambrospetrou•3/13/25, 4:01 PM

Have you tried the --persist-to--persist-to option? https://developers.cloudflare.com/d1/wrangler-commands/ and https://developers.cloudflare.com/workers/wrangler/commands/#d1:~:text=Minify%20the%20Worker.-,%2D%2Dpersist%2Dto,-string

Kenny•3/13/25, 4:02 PM

can that be specified in the wrangler.toml?

Kenny•3/13/25, 4:03 PM

bc i'm not running wrangler d1 executewrangler d1 execute in my workers. Or maybe is it a config when I bind the like ENV.DB.execute?

KKenny bc i'm not running `wrangler d1 execute` in my workers. Or maybe is it a config ...

lambrospetrou•3/13/25, 4:06 PM

You use it with wrangler devwrangler dev

Kenny•3/13/25, 4:06 PM

that's great, i'll test it rn

Kenny•3/13/25, 4:10 PM

worked perfectly, thanks!

cj•3/15/25, 12:09 PM

I am working on an app and I plan to use a durable object per team (group of users). For this I'll use the team-id as the unique name for instance

const id = this.env.MY_DURABLE_OBJECT.idFromName(teamId);
const stub = this.env.MY_DURABLE_OBJECT.get(id);

const id = this.env.MY_DURABLE_OBJECT.idFromName(teamId);
const stub = this.env.MY_DURABLE_OBJECT.get(id);

But how should I manage auth? Shouldn't auth hit a single DB?

Should I be using D1 for tables like - users, teams, etc (other global tables) and DO for tables specific to the team?

Ccj I am working on an app and I plan to use a durable object per team (group of use...

James•3/15/25, 12:58 PM

I would personally stick to and recommend something more traditional and external for auth and global info. Postgres + Hyperdive for example works great.

Cloudflare’s recommendation to build horizontally with D1 and DO SQL just doesn’t really work in my experience in the real world, coupled with the reliability issues and support concerns with D1: https://www.reddit.com/r/CloudFlare/s/4D9AeoAHm0

JJames I would personally stick to and recommend something more traditional and externa...

cj•3/15/25, 1:06 PM

Thank you. I didn't see anyone building auth on top of D1. And so the question.

James•3/15/25, 3:52 PM

I use a central db for auth, but they only need to hit that when the session expires, as well I only store data there for central lookups. Sessions are kept in DO per user, bypassing central db for a faster request avg.

will smith•3/16/25, 12:51 AM

status dashboard says no, but just for my sanity there's no known issue with d1 is there?
getting sudden huge spikes in latency (47,000ms) for very simple requests which were fine previously

    const findActivityStartTime = performance.now();
    const activity = await prisma.activity.findFirst({
      where: {
        id: activityId,
        goalId: goalId,
        goal: {
          userId: user.id,
        },
      },
    });
    const findActivityEndTime = performance.now();
    console.log(
      `Time to find activity: ${findActivityEndTime - findActivityStartTime}ms`
    );

    const findActivityStartTime = performance.now();
    const activity = await prisma.activity.findFirst({
      where: {
        id: activityId,
        goalId: goalId,
        goal: {
          userId: user.id,
        },
      },
    });
    const findActivityEndTime = performance.now();
    console.log(
      `Time to find activity: ${findActivityEndTime - findActivityStartTime}ms`
    );

I don't actually see the requests in the metrics page

Kotkoroid•3/17/25, 12:23 PM

Hi, how is it looking with stored procedures? https://blog.cloudflare.com/whats-new-with-d1/#sneak-peek-how-were-bringing-javascript-transactions-to-d1

I would like to write some JS code inside SQL database thus I can't use batch()batch() but I cannot find any other resources than in this blog post.

Kotkoroid•3/17/25, 12:24 PM

Ok, I found this: https://community.cloudflare.com/t/d1-stored-procedures-can-not-use/649207/2

Kotkoroid•3/17/25, 12:25 PM

What is the current state in D1 to solve this?

KKotkoroid Hi, how is it looking with stored procedures? https://blog.cloudflare.com/whats-...

lambrospetrou•3/17/25, 1:59 PM

Not released, and no way to do that now. Ideally, try to design your queries not to need transactions interleaved with JS code.

flowwise•3/17/25, 3:31 PM

was anyone able to use Prisma + D1? My app works locally but when I deploy it to Cloudflare i'm getting the error: fs.readdir is not implemented yet. Why would Prisma need access to the filesystem if it is configured to connect to D1? Its a real showstopper for me. I guess i will have to self-host the app if there is no solution.

Boetro•3/17/25, 4:52 PM

Hi! I'm looking for some docs on how d1 handles a colo going doing? If the location housing my d1 database goes down am I just hosed until it comes back up or is their some kind of built in failover?

Zack•3/17/25, 4:53 PM

Hey, I recently moved some of our infra from KV to D1, with a database situated in every location to be nearer to the user while awaiting global replication. This was working great, however after updating that data through our warming process (import all new data into a tmp database, rename old database, rename new database to old database name, drop old database), the databases all appear to be experiencing significant random latency issues. In our monitoring I can see average duration to call D1 jumping from ~20-50ms depending on worker location to some queries hanging for multiple seconds up to a minute. I'm wondering if someone can help me investigate what is going on here. Beyond completely recreating all these databases as part of our warming process I am not sure what the option is here. Our database is ~2GB in size and has 3.24 million rows. I am wondering if there's something going on internally like DO resets that is causing this behaviour. I've attached an example graph where you can see the moment the warming happened and the duration metrics afterwards. This is from the HKG PoP and with a D1 instance in APAC

Zack•3/17/25, 4:55 PM

Hello! I am using a D1 database and have golang script for upserting some rows in D1. The cloudfla

Similar Threads