Hello, Is there any way to migrate aws rds mariadb database to cloudflare database?

Ggaminski I need help understanding something. On the D1 marketing page it states we can h...

You can have multiple D1 bindings in your

wrangler.toml

wrangler.toml

. Repeat the

[[d1_databases]]

[[d1_databases]]

section for each of your databases. That's defining an array, and each time you use it, it's an item in that array. The newer

wrangler.json

wrangler.json

format makes that a bit easier to really understand though if you are not familiar with TOML syntax.

Example:

[[d1_databases]]
binding = "DB_01"
database_name = "db-01"
database_id = "<unique-ID-for-your-database>"

[[d1_databases]]
binding = "DB_02"
database_name = "db-02"
database_id = "<unique-ID-for-your-database>"

[[d1_databases]]
binding = "DB_01"
database_name = "db-01"
database_id = "<unique-ID-for-your-database>"

[[d1_databases]]
binding = "DB_02"
database_name = "db-02"
database_id = "<unique-ID-for-your-database>"

DDevid Hello, Is there any way to migrate aws rds mariadb database to cloudflare databa...

lambrospetrou•1/29/25, 5:24 PM

Export/SQL dump your existing database and import it in D1: https://developers.cloudflare.com/d1/best-practices/import-export-data/

Llambrospetrou You can have multiple D1 bindings in your `wrangler.toml`. Repeat the `[[d1_data...

gaminski•1/29/25, 5:25 PM

Thank you - that sounds simple! Is there a way to limit access to a db to specific subdomains in the config file or elsewhere on the worker, or does it need to be done at the application level?

Ggaminski Thank you - that sounds simple! Is there a way to limit access to a db to specif...

lambrospetrou•1/29/25, 5:25 PM

The worker that has this binding has full access to the database. If you have code that you don't want to have access in some of your bindings then it should be in a different worker.

Llambrospetrou The worker that has this binding has full access to the database. If you have co...

gaminski•1/29/25, 5:28 PM

Ok thanks. I think im struggling to grasp some fundamental concepts. Can I have multiple workers for one project? Maybe what I’m looking for is one worker per database? I’m trying to allocate one database per user (I’m not expecting thousands of users so I’m happy with a manual solution for now)

Ggaminski Ok thanks. I think im struggling to grasp some fundamental concepts. Can I have ...

lambrospetrou•1/29/25, 5:30 PM

Number of workers and how you allocate each database to users is very different. Your Worker code is the one that has access to the databases, so your code controls which database to access. So you can have for example 10 D1 databases attached to your worker, and then depending on the request context (e.g. user ID, account ID, or whatever your user identifier is) you decide which of those databases to use.

You definitely don't need 1000 worker projects!

gaminski•1/29/25, 5:38 PM

Thank you. Let’s see if I understand correctly: I have one worker with multiple databases. One TOML file with the bindings defined as you explained earlier. Somewhere in my application I keep a record of which database each user should be able to access and route the queries accordingly

Ashkan•1/29/25, 5:40 PM

AAshkan Click to see attachment

Ashkan•1/29/25, 5:40 PM

rufi•1/29/25, 6:49 PM

ok but I assume D1 is geo-distributed. So If my worker uses smart placement, it would reduce latency between the worker and the D1 databases?
Also, if I need a better latency (something like 20 ms tops), should I rather use KV, or D1 backend on durable objects, ...?

Rrufi ok but I assume D1 is geo-distributed. So If my worker uses smart placement, it ...

lambrospetrou•1/29/25, 7:05 PM

D1 is not yet replicated. It's in a single location.
With Worker smart placement, your worker will be running close to the database assuming you have enough traffic (100+ requests per day) for smart placement to do its magic.

If you need 20ms tops from anywhere in the world there isn't much you can do with ANY database. You need to have replicas all over the place. Either you do them yourself, or wait for D1 replication. Workers KV has very different semantics from D1 or DOs.

Llambrospetrou D1 is not yet replicated. It's in a single location. With Worker smart placement...

rufi•1/29/25, 7:14 PM

ah ok interesting thanks a lot. So when I create the D1 database I can also select a location. Most of our users are in NA anyway.
The use case I have is basically I want to "in memory sync" the configuration to be used when some endpoint is called. That configuration has to be updated say every hour but I want to avoid having to redeploy the workers to embed the config in place.

Rrufi ah ok interesting thanks a lot. So when I create the D1 database I can also sele...

lambrospetrou•1/29/25, 7:29 PM

If you rarely update that config and you have many requests needing it, then you can have D1 as the source of truth where you write your config updates, and then also put them in Workers KV. Your worker can read from Workers KV when a stale value (up to 60s) is OK for that case, and go to D1 if you need the latest/freshest configuration.

Using the local CDN Cache API from your worker sometimes give you faster lookups than Workers KV as well, but it also depends on your scenario. It's not clear from what you said above.

This design can give you the speed when you can use cached values and have D1 as a fallback for your endpoints that need the latest.

Ggaminski Ok thanks. I think im struggling to grasp some fundamental concepts. Can I have ...

Larry•1/30/25, 12:24 AM

"one worker per database" sounds exactly like Cloudflare Durable Objects

gaminski•1/30/25, 12:55 AM

Thanks - I didn’t realise Durable Objects had an SQLite component

Ggaminski Thanks - I didn’t realise Durable Objects had an SQLite component

森

森@VerseLink (RW)•1/30/25, 4:26 AM

fairly new additional, very cool concept, i've been cooking up some really cool prototypes that magically takes the burden away from traditional large db (and also writing JS logic + SQL together is magical, very different compare to Postgres or MongoDB)

AAshkan Click to see attachment

Max (@rozenmd)•1/30/25, 9:56 AM

Assuming you're attempting to provide feedback (hard to tell with your use of images), 1.2 ms means 1.2 milliseconds, 5.05k means 5050 (no unit provided)

MMax (@rozenmd)Assuming you're attempting to provide feedback (hard to tell with your use of im...

Ashkan•1/30/25, 3:50 PM

I wanted to say thanks and great job

majestic•1/30/25, 6:33 PM

How do do you guys manage that working in d1 using drizzle to create and alter table which is using commands like generate & migrate.
Im facing an error after generating new schema and migrate it into db it shows table already existed error

Mmajestic How do do you guys manage that working in d1 using drizzle to create and alter t...

森

森@VerseLink (RW)•1/30/25, 6:53 PM

you need to use wrangler migrate

森森@VerseLink (RW)you need to use wrangler migrate

majestic•1/30/25, 7:00 PM

Pls specific, can u show me the documentations or code ?

Mmajestic Pls specific, can u show me the documentations or code ?

森

森@VerseLink (RW)•1/30/25, 7:08 PM

https://developers.cloudflare.com/d1/reference/migrations/

Cloudflare Docs

Migrations · Cloudflare D1 docs

Database migrations are a way of versioning your database. Each migration is stored as an .sql file in your migrations folder. The migrations folder is created in your project directory when you create your first migration. This enables you to store and track changes throughout database development.

森

森@VerseLink (RW)•1/30/25, 7:09 PM

you can then list or apply (no need for create since drizzle handles that for you)

森

森@VerseLink (RW)•1/30/25, 7:09 PM

https://developers.cloudflare.com/workers/wrangler/commands/#d1-migrations-apply

Cloudflare Docs

Commands - Wrangler · Cloudflare Workers docs

Create, develop, and deploy your Cloudflare Workers with Wrangler commands.

DDivinitii Just got first-round funding on a new product. Intend to build using Durable Obj...

森

森@VerseLink (RW)•1/31/25, 12:13 AM

in your case is it important that your db always read up to date data?

森

森@VerseLink (RW)•1/31/25, 12:13 AM

i imagine that using KV would actually be potentially a good option

森

森@VerseLink (RW)•1/31/25, 12:16 AM

alternatively you can create a DO with sqlite that basically is d1, but with built in replication ability

森

森@VerseLink (RW)•1/31/25, 12:16 AM

(which would be a lot more complicated, but i believe it could be done)

森

森@VerseLink (RW)•1/31/25, 1:06 AM

yeh well that's kind of your best bet at the moment, I believe DO /w sqlite is free on storage prior to GA (which i imagine D1 read replica would be available after GA), and with DO you would have better control over how the underlying sql mechnism works

森

森@VerseLink (RW)•1/31/25, 1:07 AM

I would implement all write operations using Workflow (to write all DOs) and read directly on DOs, depends on if how important up to date data is

Mmajestic How do do you guys manage that working in d1 using drizzle to create and alter t...

戴

戴きますReina•1/31/25, 3:40 PM

I already tried using prisma and drizzle for d1 and difficult to migrate, we can't use url since binding in wrangler

Mmajestic How do do you guys manage that working in d1 using drizzle to create and alter t...

戴

戴きますReina•1/31/25, 3:42 PM

My frustration in here
d1-database

戴戴きますReina My frustration in here https://discord.com/channels/595317990191398933/992060581...

森

森@VerseLink (RW)•1/31/25, 5:17 PM

I use drizzle for migration, so far no problems in my test environment

戴戴きますReina I already tried using prisma and drizzle for d1 and difficult to migrate, we can...

James•1/31/25, 5:47 PM

drizzle migrations works great for me with D1 and DO sqlite, i dont even use drizzle for the ORM, just the migrations and schema definition

JJames drizzle migrations works great for me with D1 and DO sqlite, i dont even use dri...

森

森@VerseLink (RW)•1/31/25, 5:52 PM

Same, I use Kysely for building complex query and raw SQL if it’s very simple operations

森

森@VerseLink (RW)•1/31/25, 5:52 PM

But drizzle migration helps alleviate a lot of headaches when maintaining SQL migrations

森森@VerseLink (RW)But drizzle migration helps alleviate a lot of headaches when maintaining SQL mi...

James•1/31/25, 5:54 PM

its also helpful, I re-use the schema definitions for type casting raw sql response data as well

verdverm.com•1/31/25, 10:59 PM

Is there an example that shows how one would talk to and execute SQL from a worker against a D1 when you use the recommendation to have something like a D1 per customer when using workers-for-platforms?

i40west•2/1/25, 2:12 AM

Is there a way to tell if an error thrown by D1 is due to a constraint violation? More generally, is there a way to detect a unique constraint violation? Everything seems to just be a D1_ERROR

Ii40west Is there a way to tell if an error thrown by D1 is due to a constraint violation...

森

森@VerseLink (RW)•2/1/25, 2:53 AM

I believe the error message would include SQLITE_CONSTRAINT?

森

森@VerseLink (RW)•2/1/25, 2:54 AM

https://www.sqlite.org/rescode.html#constraint

i40west•2/1/25, 3:00 AM

parsing the message isn't great, but it'll do

Khafra•2/2/25, 2:33 AM

Hi, I'm using triggers and encounter an "internal error" that times out after 30 seconds when the abort occurs. It's not a massive problem, but it would be nice if this was fixed.

The trigger:

CREATE TRIGGER deduct_coins_on_consumable
BEFORE INSERT ON ActiveConsumables
  BEGIN
    UPDATE accounts
    SET pseudoCoins = pseudoCoins - (SELECT cost FROM Consumables WHERE id = NEW.consumableId)
    WHERE
      id = NEW.accountId 
        AND
      pseudoCoins >= (SELECT cost FROM Consumables WHERE id = NEW.consumableId);

  SELECT RAISE(ROLLBACK, 'Insufficient coins') WHERE changes() = 0;
END

CREATE TRIGGER deduct_coins_on_consumable
BEFORE INSERT ON ActiveConsumables
  BEGIN
    UPDATE accounts
    SET pseudoCoins = pseudoCoins - (SELECT cost FROM Consumables WHERE id = NEW.consumableId)
    WHERE
      id = NEW.accountId 
        AND
      pseudoCoins >= (SELECT cost FROM Consumables WHERE id = NEW.consumableId);

  SELECT RAISE(ROLLBACK, 'Insufficient coins') WHERE changes() = 0;
END

the internal error occurs when coins < cost.

"database_name": "discord-oauth",
"database_id": "2bd61136-7eb6-4b5d-82ce-f3405294398d"

Khafra•2/2/25, 2:34 AM

afaict the trigger works fine, other than it timing out/taking 30 seconds to throw

KKhafra Hi, I'm using triggers and encounter an "internal error" that times out after 30...

Isaac McFadyen•2/2/25, 3:41 AM

D1 has no manual transactions so I expect the error is a result of you trying to manually roll back (unsure though)

Khafra•2/2/25, 4:11 AM

Right.. I'll see if RAISE(ABORT, ...) has the same issues.

[[d1_databases]] binding = "DB_01" database_name = "db-01" database_id = "<unique-ID-for-your-database>" [[d1_databases]] binding = "DB_02" database_name = "db-02" database_id = "<unique-ID-for-your-database>"

CREATE TRIGGER deduct_coins_on_consumable BEFORE INSERT ON ActiveConsumables BEGIN UPDATE accounts SET pseudoCoins = pseudoCoins - (SELECT cost FROM Consumables WHERE id = NEW.consumableId) WHERE id = NEW.accountId AND pseudoCoins >= (SELECT cost FROM Consumables WHERE id = NEW.consumableId); SELECT RAISE(ROLLBACK, 'Insufficient coins') WHERE changes() = 0; END

Hello, Is there any way to migrate aws rds mariadb database to cloudflare database?

Similar Threads

Similar Threads

Similar Threads