You should be able to use miniflare for this. See https://github.com/cloudflare/miniflare/releases/t

You should be able to use miniflare for this. See https://github.com/cloudflare/miniflare/releases/tag/v3.20230918.0

GitHub

Release v3.20230918.0 · cloudflare/miniflare

What's Changed

Implement simulators as Durable Objects by @mrbbot in #656.
Allow GET requests with bodies in entry worker by @mrbbot in #677
Remove unused handleQueue() function from entry wor...

CCyb3r-Jak3 You should be able to use miniflare for this. See https://github.com/cloudflare/...

GuillaumeLakano•10/16/23, 3:27 PM

Thank you, indeed there is example to access any kind of databases, ( « const d1Database = await mf.getD1Database("DATABASE");» ), I'm not sure this also permit to create one, but I'll check, thank you for the link!

GGuillaumeLakano This project isn't what I needed, it's a proxy to uses a remote D1 database, and...

mrwind•10/17/23, 1:20 AM

my project offer a self-hosted version so i create a local sqlite it doesn't depend on miniflare or cloudflare and use better-sqlite3 to run query

WWill Hackett Okay, that's all good. I would ADORE a best guess timeline if you have one. ❤️

forslunds•10/17/23, 10:14 AM

The latest dev blog about D1 says the limit will be upped coming weeks and months. (note, Several times) Once they have fast read on big sizes they would love to be on double-digit GB size. And they are aiming on GA Q1 2024. So i would guess.. "soon"

Elliot Hesp•10/17/23, 10:35 AM

Would it be expected that KV is quicker than D1 (indexed) reads?

I've got a performance sensitive worker, and I'm not sure whether adding a KV check before D1 query makes much sense or not

zegevlier•10/17/23, 10:52 AM

KV is extremely fast (faster than D1 without query caching) if the read is hot. If it's a cold read, it would probably depend on the location of you D1 database relative to the person requesting.

niconiahi•10/17/23, 1:59 PM

hey! is it possible to change the

migrations

migrations

directory?

I don't want my migrations directory to be at the top of my repository, but instead in a folder

db

db

, the complete location being

./db/migrations

./db/migrations

in the config toml I have this configuration but it doesn't read it

migrations_dir = "db/migrations"

migrations_dir = "db/migrations"

thanks in advance

James•10/17/23, 6:20 PM

The pricing did change recently, yes, during birthday week

https://blog.cloudflare.com/d1-open-beta-is-here/#row-based-pricing

Unsmart•10/17/23, 6:40 PM

2 different pricing models is super confusing hence why they are removing the bundled/unbound models and making 1 standard pricing.

And SQL in general almost never follows denormalization as its pretty much built to normalize and support joins. But im not sure how the new billing model encourages normalization when it costs more to get 2 records via a join than it does to get 1?

Unsmart•10/17/23, 6:42 PM

Oh ok well I dont see why thats a problem? Denormalization makes it faster computationally and storage is cheaper than compute

Unsmart•10/17/23, 6:43 PM

SQL may be designed with joins in mind but denormalization is faster because of the fact it doesnt need to join.

Original message was deleted

Michał•10/17/23, 7:25 PM

Can you share any sources to support that?

Michał•10/17/23, 7:30 PM

I see, so you can't. That's what I thought, thanks for confirming!

Michał•10/17/23, 7:33 PM

This book is 20 years old and how could it even be used to prove anything when we're discussing benchmarks?

Michał•10/17/23, 7:34 PM

How would they be still valid after 20 years?

Michał•10/17/23, 7:35 PM

So can you or can you not share any benchmarks supporting the claim that joins are faster than lack of joins?

James•10/17/23, 7:37 PM

Let's keep it civil please folks

Michał•10/17/23, 7:38 PM

@bitte chill, it's okay if you can't. No worries.

kian•10/18/23, 6:25 PM

I suspect it's unlikely

kian•10/18/23, 6:25 PM

#hyperdrive-beta exists for connecting to PostgreSQL (and MySQL on the roadmap) databases

MMusfiq It would be great if we can import a PostgreSQL database in Cloudflare D1. For n...

Michał•10/18/23, 6:28 PM

The ideas behind sqlite and postgresql are incompatible. Sqlite is great for replicating on the edge, and it's cheap. Postgres not so much.

Michał•10/18/23, 7:03 PM

As far as I know D1 databases are not currently replicated to different regions so there's only primary location.
But I'm too hoping this will be added when replication is introduced.

Original message was deleted

Cyb3r-Jak3OP•10/18/23, 7:03 PM

Planned for GA d1-database

MAHMULA•10/19/23, 12:11 AM

hey

Old Man Maker•10/19/23, 4:44 AM

Apologies if this has been answered, but I am not able to find the answer.

If I create a D1 database using the REST api, is there anyway to then use it in an existing worker without needing to define the Binding at design time? I.e, is it possible to create a dynamic binding to a D1 inside a worker at runtime?

Michał•10/19/23, 6:32 AM

You can update bindings via rest api after you create the database

MMichał You can update bindings via rest api after you create the database

Old Man Maker•10/19/23, 7:47 AM

Thanks for the response, I assume this is under a bindings API?

Old Man Maker•10/19/23, 7:50 AM

Actually I see worker binding api is depreciated

Old Man Maker•10/19/23, 8:04 AM

I think you're referring to the patch functions for worker script API? That's about all I could find. The problem with that is we are limited to the number of bindings as far as I know to about 5000 per worker. So this would not be a suitable solution if we have say 20k databases.
If the only option is to query a DB with REST calls, I assume there is a latency penalty as opposed to a binding call if executed from the same worker?

Michał•10/19/23, 8:16 AM

Yes, that's what I was referring to. Why do you have 5000 databases binded to a single worker though? It sounds curious, what's the usecase?

MMichał Yes, that's what I was referring to. Why do you have 5000 databases binded to a ...

Old Man Maker•10/19/23, 8:29 AM

It fits the overall use case of D1 which is intended to open up database-per-user use cases. In our use case, when a customer creates a new project, we generate several databases for them. We need to to this, because of the 2Gb limit / database. If we have several thousand projects created, single DBs sizes are exceeded quickly, as well our customers want their data isolated. With even just 1000 customers, we could end up with around ~35K databases.

Managing and querying these with REST is fine for the most part, except for a very specific scenario involving AI where the latency of a REST call would be an issue - but that worker making the query, needs to be able to access any project DB, so a scenario where we could call by name or ID would be great.

Michał•10/19/23, 8:30 AM

We deploy a copy of the worker along with a database for each client

Not sure if that helps your case though

MMichał We deploy a copy of the worker along with a database for each client 🙂 Not sure...

Old Man Maker•10/19/23, 8:30 AM

hmmm - thats very interesting and I had not considered that - thank you so much for that insight

You should be able to use miniflare for this. See https://github.com/cloudflare/miniflare/releases/t

Similar Threads

Similar Threads

Similar Threads