Cloudflare Developers•8mo ago

It’s a shame that really wasn’t communicated at the start. I’ll have to rewrite everything which is

It’s a shame that really wasn’t communicated at the start. I’ll have to rewrite everything which is a shame.

DDean It’s a shame that really wasn’t communicated at the start. I’ll have to rewrite ...

lambrospetrou•5/20/25, 9:00 AM

At the start, SQLite DOs didn't exist

Ashish Rawat•5/20/25, 9:42 AM

Durable object is too much boilerplate.

planet•5/20/25, 2:18 PM

meh it only took less than a week to learn how to use SQLite in DO and refactor code

planet•5/20/25, 2:18 PM

you get input gates & (in my testing) much better latencies & less reads/writes because you dont need to index a user id

Ashish Rawat•5/20/25, 2:51 PM

problem is you can't query across multiple database for getting list of users, do joins and so on. So you've to use D1 for that with different concept. Cloudflare database (D1, DO) is broken. D1 has different limitation while DO has different. So basically to make everything work, you need to use both D1, DO and Queues. So instead of this, I better off go with managed postgres.

AAshish Rawat problem is you can't query across multiple database for getting list of users, d...

lambrospetrou•5/20/25, 3:47 PM

So you've to use D1 for that with different concept. Cloudflare database (D1, DO) is broken. D1 has different limitation while DO has different

I wonder why you say this. Actually, D1 has exactly the same limitations as a single Durable Object, and even more since your worker code is remotely from the actual storage and you don't have things like transactions across queries interleaved with JS code, whereas in the DO case your code runs next to the storage and is fast. Resource limits wise, a D1 database IS a single Durable Object, so zero differences.

I assume your different viewpoint is if you design your DO architecture using per-resource/per-tenant DO vs a single D1 database, but then you are not comparing the same thing since managing multiple D1 databases in the same way would be much more complicated.

planet•5/20/25, 8:35 PM

Much less of a problem and more of a design consideration.

Personally I use D1 as a lightweight directory DBs that handles background cleanups/etc (via CRON) and login lookups. emails/sso metadata matches get logged into their SQLite in DO DB.

You also have Alarms which is part of Durable Objects (so you don't necessarily need Queues/Workflows)
You can also set up basic rate limiting in memory using

this

this

in each new constructor instance

planet•5/20/25, 8:37 PM

Now I do wish my R2 bucket had spawned in the same DC as my DO... praying for finer grained location hints one day

workerBee•5/20/25, 9:13 PM

Is it possible to debug these platform errors?

D1_ERROR: D1 DB storage operation exceeded timeout which caused object to be reset.

D1_ERROR: D1 DB storage operation exceeded timeout which caused object to be reset.

.... Or just accept and move on?

Llambrospetrou D1 dynamic bindings are not coming in the short-term, and still undecided for lo...

ac•5/21/25, 12:43 AM

This is definitely different from the messaging that I've heard from other Cloudflare folks recently, I was definitely hoping for dynamic bindings for D1 as well. It's a bit disappointing how unclear this all is.

SQLite in DO's is a nice idea, but it's fairly unusable right now when you can't actually view DO instances and cannot view what is in the databases....

Aac This is definitely different from the messaging that I've heard from other Cloud...

1984 Ford Laser•5/21/25, 5:50 AM

There are absolutely ways to view the instances and view what's in the databases. Unsure where that idea comes from exactly

Aac This is definitely different from the messaging that I've heard from other Cloud...

lambrospetrou•5/21/25, 6:15 AM

This is definitely different from the messaging that I've heard from other Cloudflare folks recently

I have been in this channel for 9 months now and we never gave an ETA for dynamic bindings. It was (and still is) one of the future possible features. But, I don't want folks to start designing their new applications now depending on a feature that does not yet exist and is not going to land within say the next quarter. That's my viewpoint.

SQLite in DO's is a nice idea, but it's fairly unusable right now when you can't actually view DO instances and cannot view what is in the databases....

You cannot (yet) view them natively in the Cloudflare dashboard but there are several open source solutions to inspect your Durable Objects. For example, you could use something like https://github.com/outerbase/browsable-durable-object
I agree that DOs don't (yet) have some of the niceties of D1 like import/export but you get the flexibility of creating as many as you want much easier than with D1. So, it's about tradeoffs and what you need right this moment.

Aac This is definitely different from the messaging that I've heard from other Cloud...

DeanOP•5/21/25, 7:01 AM

Yeah it truly sucks. I have workarounds for it and looking at using durable objects but think I’ll end up just using another database provider that’ll allow me to do everything d1 does but have a database for each user,

Ashish Rawat•5/21/25, 8:02 AM

Well that proves the point, neither D1 nor DO solves the basic problems completely. Why would we should use 10 different things when all can be done with postgres and postgres doesn't have 10 GB limitation.
Also this is not just the problem with DO or D1, even workers doesn't have many limitations, basically all the development products of cloudflare are just WIP and not ready for production usage completely.

AAshish Rawat Well that proves the point, neither D1 nor DO solves the basic problems complete...

lambrospetrou•5/21/25, 8:50 AM

If Postgres solves your needs, then using Workers + Hyperdrive is a perfectly fine combination. There is no need to shoehorn your architecture into the D1/DO model.

Himanshu•5/21/25, 11:30 AM

I'm unable to use D1 outside of the Workers environment. I tried in Node.js, but I'm having trouble accessing the bindings. Any solution for this?

SuperHelpflare•5/21/25, 11:31 AM

Please do not post your question in multiple channels/post it multiple times per the rules at #

welcome-and-rules. It creates confusion for people trying to help you and doesn't get your issue or question solved any faster.

11984 Ford Laser There are absolutely ways to view the instances and view what's in the databases...

ac•5/21/25, 11:41 AM

The only solution I'm aware of is using a REST API and sending raw queries through that, would love to know if there is a better/more native way

Aac The only solution I'm aware of is using a REST API and sending raw queries throu...

gruntlord6•5/21/25, 12:19 PM

you can also use a worker as a proxy

DDean Yeah it truly sucks. I have workarounds for it and looking at using durable obje...

gruntlord6•5/21/25, 12:20 PM

honestly its not that big of a limit when you consider all the benifits of a durable object for each user

gruntlord6•5/21/25, 12:21 PM

things like alarms are huge

Aac The only solution I'm aware of is using a REST API and sending raw queries throu...

lambrospetrou•5/21/25, 12:30 PM

See my link to outerbase's projects that exposes the Studio UI they have for each of your DOs. At some point we should be adding that natively into the dashboard, but until then you can use the library version.

Llambrospetrou > This is definitely different from the messaging that I've heard from other Clo...

João Castro•5/21/25, 5:23 PM

Dynamic bindings and transactions were two things that we saw being mentioned as coming :soontm: multiple times in the past, even in official blog posts

João Castro•5/21/25, 5:23 PM

Honestly today I don't really care anymore because DOs with SQLite solve those problems so that is great

João Castro•5/21/25, 5:24 PM

Just that D1 feels like something that currently sits in a very weird spot

João Castro•5/21/25, 5:42 PM

And I have the impression that we might see something like the Functions/Pages and Workers convergence happening with D1 and DOs in the future

Original message was deleted

lambrospetrou•5/22/25, 8:00 AM

That error doesn't seem to be a D1 error. Are you using any ORM library? If yes, how are you using it?

blackfish•5/22/25, 8:55 AM

Prisma + SQLite/D1: Error when creating record with null values

Hey guys I'm encountering a

PrismaClientUnknownRequestError

PrismaClientUnknownRequestError

when trying to create a record with null values using Prisma with D1 database SQLite. The error suggests a JSON parsing failure, but the input data appears valid.

Error Message:

PrismaClientUnknownRequestError: Invalid `prisma.table_name.create()` invocation:
Error occurred during query execution:
ConnectorError(ConnectorError { user_facing_error: None, kind: QueryError(SqliteError { extended_code: 1, message: Some("D1_ERROR: Failed to parse body as JSON, got: ") }), transient: false })

PrismaClientUnknownRequestError: Invalid `prisma.table_name.create()` invocation:
Error occurred during query execution:
ConnectorError(ConnectorError { user_facing_error: None, kind: QueryError(SqliteError { extended_code: 1, message: Some("D1_ERROR: Failed to parse body as JSON, got: ") }), transient: false })

Reproduction Steps:

Schema (Prisma): ```prismamodel TableName {id Int @id @default(autoincrement())owner Stringamount IntamountUsd Float? @map("amount_usd")// ... other fields with nullable ones marked as ?@@map("table_name")

}

2. Schema (D1 DB):

2. Schema (D1 DB):

sql
CREATE TABLE table_name (
id INTEGER PRIMARY KEY AUTOINCREMENT,
owner TEXT NOT NULL,
amount INTEGER NOT NULL,
amount_usd REAL,
... other fields
)

3. Attempted Query:

3. Attempted Query:

typescript
await prisma.tableName.create({
data: {
amount: 100000,
amountUsd: null, // This nullable field seems to trigger the issue
owner: "xyz_onwer",
// ... other fields
}
});

``

Record should be created with

``

Record should be created with

NULL

 for nullable fields (like

 for nullable fields (like

amountUsd

).

**Issue:**  
Fails with JSON parsing error, even though:  
- Schema correctly marks field as optional (

).

**Issue:**  
Fails with JSON parsing error, even though:  
- Schema correctly marks field as optional (

Float?

)  
- SQLite table allows NULL (

)  
- SQLite table allows NULL (

amount_usd REAL

)  

Here one thing to note (which is kind of weird) is that, once I try to insert

)  

Here one thing to note (which is kind of weird) is that, once I try to insert

null` all the subseuent values into the field will result in same error, like 1, 1.343 etc. And after a while, it wil work again without fail.

Also sometimes the null value also gets accepted.
I am not clear what I am doing wrong here.
If anyone could help me on this, it would be very helpful.

Original message was deleted

Isaac McFadyen•5/22/25, 12:01 PM

How are you defining a "connection is not possible"? Can you show the actual code that you are throwing the error with?

jonast•5/22/25, 2:40 PM

When using cloudflare D1 and drizzle, how can I do data migrations (in addition to schema migrations)

Specifically, I have a table like this

export const users = sqliteTable("users", {
id: text("id")
.primaryKey()
.$defaultFn(() => prefixID("user")),
googleId: text("google_id").notNull().unique().default(""),
email: text("email").notNull().unique(),
name: text("name").notNull(),
avatarUrl: text("avatar_url"),
createdAt: integer("created_at", { mode: "timestamp" }).$defaultFn(() => new Date())
});

I want to now break out the googleId into a separate table that maps external Id (google ID) to internal user ID. But I want to migrate all values

What are my options?

Just stick that data migration into a SQL file?

Jjonast When using cloudflare D1 and drizzle, how can I do data migrations (in addition ...

CodingCoop | Co-Founder Nullshot•5/22/25, 2:57 PM

I usually stay in the migration file and what wrangler has versus trusting anything outside of that.

It works well.

verdverm.com•5/25/25, 11:18 PM

Did the concept of D1 Procedures ever get implemented? (https://blog.cloudflare.com/whats-new-with-d1/)

The Cloudflare Blog

D1: our quest to simplify databases

Get an inside look on the D1 experience today, what the team is currently working on and what’s coming up!

Vverdverm.com Did the concept of D1 Procedures ever get implemented? (https://blog.cloudflare....

Larry•5/26/25, 12:08 AM

I'm not sure if they ever intended differently but a version of that was implemented when they brought the SQLite storage engine to Durable Objects. My impression is that the success of DOs is the reason why features like D1 Procedures and Dynamic Binding have not been implemented.

gruntlord6•5/26/25, 1:59 AM

I think they more or less said DOs will support the same things as D1 at some point, but they each have some features that the other does not

gruntlord6•5/26/25, 1:59 AM

I could be remembering what they said wrong though

cosbgn•5/26/25, 12:42 PM

Hey id d1 down or something?

cosbgn•5/26/25, 12:42 PM

I keep getting :D1 DB exceeded its CPU time limit and was reset. [code: 7500] when trying to run a migration

cosbgn•5/26/25, 12:46 PM

I'm stuck - migration passed locally but d1 just runs out of cpu

aliendev•5/26/25, 8:21 PM

Hi, I've just discovered Outerbase Studio from the recent Acquisition announcement, it's great news to see it become part of CloudFlare. I was thinking about implementing it in a project I'm working on, but I found that the license is AGPL which prevents me from doing that without making my codebase public. Since the original cloud offering is now closed for new users and sunsetting soon, and seeing that the original repo is no longer active, would it be possible to make the license less prohibitive ?

Aaliendev Hi, I've just discovered Outerbase Studio from the recent Acquisition announceme...

lambrospetrou•5/27/25, 9:19 AM

I was thinking about implementing it in a project I'm working on

Do you mean using the Outerbase Studio with your D1 databases, or do you just want to use it in a completely different project unrelated to D1? Just clarifying since this the D1 channel.

aliendev•5/27/25, 11:36 AM

I want to make it part of my hosted dashboard, so that my users can access and edit their own DBs, which are all part of a single multi-tenant D1 database (and the project itself will be deployed using CF workers).
so to answer your question, my use case is for a different project, but related to D1 and fully part of the CF ecosystem.

Aaliendev I want to make it part of my hosted dashboard, so that my users can access and e...

lambrospetrou•5/27/25, 11:55 AM

I will ask the outerbase folks and let you know (or they will reply directly)

aliendev•5/27/25, 12:15 PM

thank you

Brayden•5/27/25, 2:56 PM

@aliendev At the moment there are no plans to change the license on the repositories, sorry...

BBrayden @aliendev At the moment there are no plans to change the license on the reposito...

aliendev•5/27/25, 3:11 PM

got it, no worries. thanks !

Paul Mihai•5/28/25, 2:23 PM

Hi everyone, I have a 3gb database with multiple keys (one line can be quite large - a few kb - but the blobs are not part of any keys), the maximum size is 10gb, but for some reason querying even this small DB is very slow (even simple queries take up to a minute). I use composite keys. Anybody else seeing this type of behavior? Noteworthy is the fact that this db experiences a lot of writes as it is a live stats marting table.

Spencer•5/28/25, 2:39 PM

hey, i need some help please, got an error D1 ERROR: Currently processing a long-running export, how to cancel this operation ? cause because of it my app stops working

SSpencer hey, i need some help please, got an error D1 ERROR: Currently processing a long...

lambrospetrou•5/28/25, 3:40 PM

If you are doing an export the whole DB is blocked until it uploads everything to R2. If you want to abort the export, then unfortunately you need to stop polling and wait a few seconds for it to stop.

PPaul Mihai Hi everyone, I have a 3gb database with multiple keys (one line can be quite lar...

lambrospetrou•5/28/25, 3:42 PM

What is the .meta.meta information telling you about the duration of the query itself? 1 minute is very long either way and probably hints to other issues, commonly overloaded DB due to traffic or slow-ish consecutive queries.
Also, are all your queries slow, or just that specific one?

It’s a shame that really wasn’t communicated at the start. I’ll have to rewrite everything which is

Similar Threads