Cloudflare Developers•3y ago

I mean, if you are going to self-host anyway, then you might as well use a DB built for larger sizes

Hello, I’m Allie!OP•8/28/23, 9:56 PM

You might be able to push 100 GB through local D1, but why risk it when you can just spin up Postgres or MySQL

Rabid•8/28/23, 10:21 PM

Yeah if I was going to self host I would just use postgres, or just stick with RDS

Unsmart•8/28/23, 11:00 PM

self hosting sucks just use planetscale if u need more then use d1 when its ga

HHello, I’m Allie!I mean, if you are going to self-host anyway, then you might as well use a DB bu...

chientrm.com•8/29/23, 1:52 AM

Then you don't need to rewrite queries

jared•8/29/23, 4:13 PM

Hi. Is there a way to call external endpoints (or Cloudflare Worker APIs) on D1 table changes (insert, update, delete)? Something like Firebase Functions for Firestore or Webhooks for Supabase (for those that are familiar).

Jjared Hi. Is there a way to call external endpoints (or Cloudflare Worker APIs) on D1 ...

chientrm.com•8/29/23, 4:26 PM

https://github.com/chientrm/wrangler-proxy

GitHub

GitHub - chientrm/wrangler-proxy: Wrangler Proxy exposes Workers AP...

Wrangler Proxy exposes Workers API to outside and integrate to your favorite frameworks. Compatible with DrizzleORM. - GitHub - chientrm/wrangler-proxy: Wrangler Proxy exposes Workers API to outsid...

Cchientrm.com https://github.com/chientrm/wrangler-proxy

jared•8/29/23, 7:03 PM

How can I use this to watch for changes in the D1 tables? To be more clear, the goal is that every time there are inserts, updates, deletes in my tables, a function should be automatically triggered to send the corresponding data to an external endpoint.

Jjared How can I use this to watch for changes in the D1 tables? To be more clear, the ...

chientrm.com•8/29/23, 7:04 PM

AFAIK D1 doesn't have trigger

Jjared How can I use this to watch for changes in the D1 tables? To be more clear, the ...

Max (@rozenmd)•8/30/23, 6:19 AM

we don't support this yet, but it's cool and something we want to look into

Jjared How can I use this to watch for changes in the D1 tables? To be more clear, the ...

Mia•8/30/23, 9:24 AM

i suppose a workaround would be to have the code that's making the changes fire a webhook to another endpoint

itsmatteomanf•8/30/23, 9:24 AM

Any plans to expose rows read/written + time on console queries?

itsmatteomanf•8/30/23, 9:24 AM

And indexes on tables in the UI?

MMia i suppose a workaround would be to have the code that's making the changes fire ...

Mia•8/30/23, 9:26 AM

not a super useful response if you dont have control over that code

itsmatteomanf•8/30/23, 9:26 AM

I presume there is no way to see query history with stats, right?

MMia not a super useful response if you dont have control over that code

itsmatteomanf•8/30/23, 9:27 AM

D1 can be controlled only via Workers in your account, so something you control is always in the middle.

Mia•8/30/23, 9:27 AM

fair enough

Mia•8/30/23, 9:29 AM

i finally found a project to actually use d1 with so im rly excited

Mia•8/30/23, 9:29 AM

spent like 3 days reading documentation and messing around with postman to figure out some new apis

Mia•8/30/23, 9:33 AM

currently figuring out a diagram for the db & tables

Mia•8/30/23, 9:33 AM

figure out all the data i need to store and how to store it

Iitsmatteomanf Any plans to expose rows read/written + time on console queries?

Max (@rozenmd)•8/30/23, 11:53 AM

yes, i've been meaning to add a status bar-like thingy

Iitsmatteomanf And indexes on tables in the UI?

Max (@rozenmd)•8/30/23, 11:54 AM

can do that, yeah

MMax (@rozenmd)can do that, yeah

itsmatteomanf•8/30/23, 12:00 PM

Thank you! Might be cool to have, but not sure if SQLite has it, the full create table exposed.

Elliot Hesp•8/30/23, 12:55 PM

Reading this blog: https://blog.cloudflare.com/d1-turning-it-up-to-11/

It mentions:

Importantly, when we enable global read replication, you won’t have to pay extra for it, nor will replication multiply your storage consumption

Importantly, when we enable global read replication, you won’t have to pay extra for it, nor will replication multiply your storage consumption

Does D1 read globally? Or is it still region based?

The Cloudflare Blog

D1: We turned it up to 11

We've been heads down iterating on D1, and we've just shipped a major new version that's substantially faster, more reliable, and introduces Time Travel: the ability to restore a D1 database to any point in time.

EElliot Hesp Reading this blog: https://blog.cloudflare.com/d1-turning-it-up-to-11/ It menti...

Hello, I’m Allie!OP•8/30/23, 12:56 PM

Single region, atm

Elliot Hesp•8/30/23, 1:11 PM

Ah, is there any rough ETA on when that'll land?

EElliot Hesp Ah, is there any rough ETA on when that'll land?

DaniFoldi•8/30/23, 1:19 PM

, no ETA or timeline to my knowledge

itsmatteomanf•8/30/23, 5:44 PM

@rozenmd any chance the metrics UI gets the same Workers treatment with percentiles?

Iitsmatteomanf @rozenmd any chance the metrics UI gets the same Workers treatment with percenti...

Max (@rozenmd)•8/30/23, 5:47 PM

yeah we're just getting started lol

MMax (@rozenmd)yeah we're just getting started lol

itsmatteomanf•8/30/23, 5:49 PM

That’s wonderful ahah I wish I had that before so I could realize I had a 6s+ query every hour, which read 42 million rows.

Sameer Ali•8/30/23, 6:31 PM

Does D1 support (or will) streams?

SSameer Ali Does D1 support (or will) streams?

Unsmart•8/30/23, 6:42 PM

d1-database

Sameer Ali•8/30/23, 6:45 PM

I think that is different question than mine. I am trying to see if i can stream the response from SELECT query directly to client instead of loading the result in worker memory. let's say i have a query that returns large amount of data, which is over the worker memory limit of 128MB, streaming the response without loading into buffer would be only way to workaround worker memory limit.

SSameer Ali I think that is different question than mine. I am trying to see if i can stream...

Max (@rozenmd)•8/30/23, 7:08 PM

That's another thing we're looking at

Max (@rozenmd)•8/30/23, 7:09 PM

but main priority is d1-database

Sameer Ali•8/30/23, 7:12 PM

Thanks. It is must thing to have for many use cases. But it can wait.

itsmatteomanf•8/30/23, 8:17 PM

People, use indexes OMEGALUL

itsmatteomanf•8/30/23, 8:17 PM

itsmatteomanf•8/30/23, 8:21 PM

Yeah, they do, but given my volume of reads... it makes sense to improve them.

itsmatteomanf•8/30/23, 8:23 PM

Yeah, of course it's a tradeoff. One is required to be an index, the other might not make sense.

Unsmart•8/30/23, 8:25 PM

Using a NoSQL db it would be 1 write 1 delete and most auto delete a ttl

UUnsmart Using a NoSQL db it would be 1 write 1 delete and most auto delete a ttl <:EZ:74...

itsmatteomanf•8/30/23, 8:26 PM

But it wouldn't work well for my use case. I like NoSQL, but it's not always the best.

Unsmart•8/30/23, 8:26 PM

I wasnt even talking about your usecase cuz idk what it is but Leos would

Unsmart•8/30/23, 8:28 PM

If DOs had TTL on keys it would be better priced than kv but you would have to set an alarm so there it would still be multiple writes in weird ways lol

Original message was deleted

itsmatteomanf•8/30/23, 8:31 PM

Yeah, SQL is nice for those things.

Unsmart•8/30/23, 8:32 PM

Yeah KV has ttl so you wouldnt need to cron and actually afaik those deletes are free

itsmatteomanf•8/30/23, 8:33 PM

You could honestly avoid the expire index, tbh. If it takes a few milliseconds more in a cron it's totally fine.

itsmatteomanf•8/30/23, 8:33 PM

D1 is actually impressively fast.

Original message was deleted

Matt Silverlock•8/30/23, 9:06 PM

At those rates will you ever exceed the included read & write usage in the $5/mo plan? Feels like the wrong kind of optimization.

I mean, if you are going to self-host anyway, then you might as well use a DB built for larger sizes

Similar Threads