Cloudflare Developers•16mo ago

out of closed-source db?

�𝓈𝒽𝓎 out of closed-source db?

Cyb3r-Jak3•9/28/24, 8:07 PM

D1 is just SQLite and you can export it

dan—1106•9/28/24, 9:01 PM

Hi, I'm running into a problem using

npx prisma migrate diff

npx prisma migrate diff

with my D1 database locally (because I have a qa and prod DB instance and the command can't deal with multiple DBs). Looks like someone else ran into the same problem a few months ago: https://www.answeroverflow.com/m/1266887370653962422

I was wondering if there were any suggested approaches here?

prisma migrations fail if there are multiple databases - Cloudflare...

running migrations throws an error if there are multiple databases.

```
❯ npx prisma migrate diff --from-local-d1 --to-schema-datamodel ./prisma/schema.prisma --script --output migrations/0002_rename_referrer.sql

Error: Multiple Cloudflare D1 databases found in .wrangler/state/v3/d1/miniflare-D1DatabaseObject. Please manually specify the loca...

Original message was deleted

�

𝓈𝒽𝓎OP•9/29/24, 6:11 AM

The product is just isn’t complete, tired of people bringing workarounds instead of just addressing the real issues while they’re very obvious issues

�

𝓈𝒽𝓎OP•9/29/24, 6:13 AM

The workaround to your issue is to migrate

akashra•9/29/24, 6:13 AM

No IT product is ever complete

�

𝓈𝒽𝓎OP•9/29/24, 6:13 AM

Is this a workaround anymore?

�

𝓈𝒽𝓎OP•9/29/24, 6:14 AM

It’s just a nice way to say the product was not complete and you shouldn’t have used it to begin with

Aakashra No IT product is ever complete 😅

�

𝓈𝒽𝓎OP•9/29/24, 6:15 AM

I honestly never used a tool that clearly has a dead end

�

𝓈𝒽𝓎OP•9/29/24, 6:15 AM

Just a giant wall waiting for you when you reach 10GB

akashra•9/29/24, 6:15 AM

I'm trying to see what the issue is from reading to briefly, but what I can see are features WAY outside the scope is expect of a lightweight database.

�

𝓈𝒽𝓎OP•9/29/24, 6:15 AM

And it call the tool GA is crazy

akashra•9/29/24, 6:18 AM

If you're using this for databases larger than 10gb, you very clearly have picked the wrong tool, and done so knowingly up-front.

�

𝓈𝒽𝓎OP•9/29/24, 6:18 AM

Okay I agree with you how should I use the tool

�

𝓈𝒽𝓎OP•9/29/24, 6:22 AM

The tooling to use the tool the way it’s intended it’s not there

Ashish Rawat•9/29/24, 11:13 AM

Now we have to wait for developer day which wil be in 4-5 months for any new updates.

Ashish Rawat•9/29/24, 11:14 AM

Cloudflare ship fast, no doubt, but they have to ship one product completely stable and feature complete instead of shipping lots of products.

hardik043•9/29/24, 11:59 AM

can i use rpc client with D1 database?

Larry•9/29/24, 12:04 PM

@celestial, have you looked into the possibility of moving to Durable Objects? The limit there is actually only 1GB for now but it's going up to 10GB when it goes GA. However, you can create millions of them and not have to specify each instance in your

workers.toml

workers.toml

file. That's the architecture I'm switching to.

Original message was deleted

�

𝓈𝒽𝓎OP•9/29/24, 4:34 PM

may I ask what other providers you were using and what was the deal breaker for you

Carson•9/29/24, 9:34 PM

Does d1 support transitions at all and if so how would I use them? I see some documentation for the procedures but it doesnt look like thats supported anymore and nothing I find seems to work. And just using normal sqlite transactions dont work.

Original message was deleted

akashra•9/29/24, 11:35 PM

Everything about Grafana is terrible. Used at my previous employer. It's like budget splunk, for those who think they can save a few bucks.

CCarson Does d1 support transitions at all and if so how would I use them? I see some do...

akashra•9/29/24, 11:40 PM

Kinda. Not I. The db. It does JavaScript transactions, so because the code has to run in multiple places it's really difficult to have a tradition transaction. You can call things in a batch though,if you just want SQL.

CCarson Does d1 support transitions at all and if so how would I use them? I see some do...

akashra•9/29/24, 11:40 PM

https://blog.cloudflare.com/whats-new-with-d1/

The Cloudflare Blog

D1: our quest to simplify databases

Get an inside look on the D1 experience today, what the team is currently working on and what’s coming up!

Aakashra Kinda. Not I. The db. It does JavaScript transactions, so because the code has t...

o7•9/30/24, 2:23 AM

it depends on the persons use case also. I sent 10k requests from 3 different countries doing a count + 1 update query, and all 30k requests were added up

pablo•9/30/24, 7:05 AM

is there any guide to setup d1 with drizzle properly

pablo•9/30/24, 7:05 AM

:NotLikeThis:

Ppablo is there any guide to setup d1 with drizzle properly

boop•9/30/24, 8:08 AM

https://orm.drizzle.team/docs/get-started-sqlite#cloudflare-d1

Drizzle ORM - SQLite

Drizzle ORM is a lightweight and performant TypeScript ORM with developer experience in mind.

pablo•9/30/24, 8:22 AM

Migration guide is incomplete

Ppablo Migration guide is incomplete

boop•9/30/24, 8:33 AM

https://sat0shi.dev/posts/drizzle-intro/

Type-Safely Access Cloudflare D1 with Drizzle ORM

Looking for a ORM to access Cloudflare D1 in a type-safe manner? Look no further than Drizzle ORM!

Original message was deleted

trace•9/30/24, 10:35 AM

I can understand this. 10gb needs to be fix.

Original message was deleted

akashra•9/30/24, 12:08 PM

It's only free if your time is worth nothing... or if you're happy with a tool that doesn't really do what you want.

Aakashra Kinda. Not I. The db. It does JavaScript transactions, so because the code has t...

Carson•9/30/24, 2:55 PM

I insert a user, grab the auto incremnet pk, and then update the same table with a snowflake I created using the auto incremenet pk.

Right now if the snowflake fails or something else fails then the user will be left with no snowflake and I want to be able to revert changes.

Im guessing this isnt possible with d1 yet?

CCarson I insert a user, grab the auto incremnet pk, and then update the same table with...

o7•9/30/24, 3:06 PM

cant u do that in the same query? since its the same table

Carson•9/30/24, 3:14 PM

no because the snowflake is created using a function,

user is inseted without snowflake, just the email, hash, firstname, lastname, returning pk
funciton to turn pk to snowflake
then I update the user with the snowflake where pk matches

Carson•9/30/24, 3:22 PM

this is the workflow

        const hash = utils.hashPassword(request.json.password);
        const account = await env.DB.prepare("INSERT INTO users (email, password, firstname, lastname) VALUES (?, ?, ?, ?) RETURNING pk;")
            .bind(request.json.email, hash, request.json.firstname, request.json.lastname)
            .run();

        if (!account.success) throw new Error();

        const snowflake = utils.createSnowflake(account.results[0].pk);
        const set_id = await env.DB.prepare("UPDATE users SET id = ? WHERE pk = ?;")
            .bind(snowflake, account.results[0].pk)
            .run();
            
        if (!set_id.success) throw new Error();

        const hash = utils.hashPassword(request.json.password);
        const account = await env.DB.prepare("INSERT INTO users (email, password, firstname, lastname) VALUES (?, ?, ?, ?) RETURNING pk;")
            .bind(request.json.email, hash, request.json.firstname, request.json.lastname)
            .run();

        if (!account.success) throw new Error();

        const snowflake = utils.createSnowflake(account.results[0].pk);
        const set_id = await env.DB.prepare("UPDATE users SET id = ? WHERE pk = ?;")
            .bind(snowflake, account.results[0].pk)
            .run();
            
        if (!set_id.success) throw new Error();

CCarson this is the workflow ```js const hash = utils.hashPassword(request.json....

lambrospetrou•9/30/24, 7:35 PM

Since this is 2 remote calls, they are not connected, and cannot rollback. Is Snowflake just random IDs? If yes, can you create them ahead of time without the exact autoincrement and send them together in one query? Or use some other mechanism to generate your ID like UUID, ULID, Nanoid, etc.

Llambrospetrou Since this is 2 remote calls, they are not connected, and cannot rollback. Is Sn...

Carson•9/30/24, 7:47 PM

Ah alright thats what I was afraid of. Snowflake is just random ids but they are calculated using the pk just so that there arent 2 of the same ids ever. But ill prob just switch over to uuid or something for simplicity sake

Carson•9/30/24, 7:48 PM

Thank you for your help

CCarson Ah alright thats what I was afraid of. Snowflake is just random ids but they are...

lambrospetrou•9/30/24, 7:49 PM

No problem. I personally use Nanoid with 32+ characters, or any UUIDv4/v7 should be OK too.

CCarson Ah alright thats what I was afraid of. Snowflake is just random ids but they are...

o7•9/30/24, 7:49 PM

u can make the snowflake a primary key, or unique, which means sqlite will deny creating another row of the same snowflake

Carson•9/30/24, 7:52 PM

I have it that way curently for my mysql database. My curent codebase uses transactions and the snowflake is unique. The reason I did this was because I didnt want it to fail in the first place. if I just use the auto increment id in my creation of the snowflake then I know that it will be random.

Carson•9/30/24, 7:53 PM

I know its a bit overkill and will prob never happen but I wanted to account for it

CCarson I have it that way curently for my mysql database. My curent codebase uses trans...

lambrospetrou•9/30/24, 7:55 PM

If you are going to use an autoincrement key to generate another primary key you already did the job of guaranteeing uniqueness in the first query, so why use a snowflake afterwards anyway. It's extra overhead really. Either generate it and send with the first query and retry in case a collision happens in a few years, or keep the autoincrement key.

Carson•9/30/24, 7:57 PM

because I want that seccond primary key to be a set length, if I just give each user the id of the auto increment, then users 1-9 will have a id length of 1, 10-99 would be 2, and so on

CCarson because I want that seccond primary key to be a set length, if I just give each ...

lambrospetrou•9/30/24, 7:59 PM

My suggestion is to then to add a PRIMARY KEY on the column of your ID, generate a random ID with UUIDv4 or Nanoid (or UUIDV7 if you need time-ordered IDs), and add a simple retry wrapper around your calls. The collisions will be extremely rare to worry about the 2nd roundtrip that you anyway do right now for all calls, and you get what you want.

naourass•9/30/24, 10:13 PM

Is there currently some workaround to have a db.batchdb.batch operation where a step requires the result of the previous one (the

id

id

of an inserted element)?

naourass•9/30/24, 10:15 PM

Or some similar workaround that would allow such a transaction?

Juan•9/30/24, 10:22 PM

Hey! I wanted to ask if D1 would be suitable for this scenario. I’m migrating an e-commerce webapp from a VPS to Cloudflare Workers, so that it can scale easier. We were already using R2 for images. Now the problem comes on choosing where to host the database. At the moment we were using a self hosted PostgreSQL server. We first thought on moving to Neondb. However we also thought on D1 as it would be easier to have all our services in Cloudflare.

Would D1 be suitable for an e-commerce website with around 1000 orders per month and around 5000 visits per day considering it should be able to scale ~x10 times that?

const hash = utils.hashPassword(request.json.password); const account = await env.DB.prepare("INSERT INTO users (email, password, firstname, lastname) VALUES (?, ?, ?, ?) RETURNING pk;") .bind(request.json.email, hash, request.json.firstname, request.json.lastname) .run(); if (!account.success) throw new Error(); const snowflake = utils.createSnowflake(account.results[0].pk); const set_id = await env.DB.prepare("UPDATE users SET id = ? WHERE pk = ?;") .bind(snowflake, account.results[0].pk) .run(); if (!set_id.success) throw new Error();

out of closed-source db?

Similar Threads

Similar Threads

Similar Threads