I'm developing an application for Cloudflare that will use D1 as database. It's semi-static data tha

I'm developing an application for Cloudflare that will use D1 as database.
It's semi-static data that will not change frequenly. Bi-weekly I must basically re-import the whole database (~250.000 records).

What's the best option, during that process just drop the tables and re-import.
Or create a new database, import the data, update the binding and redeploy/restart? the Cloudflare Pages/workers app?

MMarcel Overdijk I'm developing an application for Cloudflare that will use D1 as database. It's ...

Rubi•4/29/24, 9:37 AM

how much downtime your app could tolerate?

Rubi•4/29/24, 9:38 AM

this might be dirt in the code, but I might do: create new table name, import the data, put the table name to env, use retry deployment or not at all

Marcel OverdijkOP•4/29/24, 11:42 AM

A bit of downtime is no problem at the moment, but if I could avoid it that would be a plus.

MMarcel Overdijk A bit of downtime is no problem at the moment, but if I could avoid it that woul...

Rubi•4/29/24, 1:37 PM

yeah, I would use the new table and query using join with dynamic table name

SELECT *
FROM (
    SELECT latest_table_name
    FROM table_source
) AS dynamic_table
JOIN table_name ON dynamic_table.value = table_name.table_name;

SELECT *
FROM (
    SELECT latest_table_name
    FROM table_source
) AS dynamic_table
JOIN table_name ON dynamic_table.value = table_name.table_name;

Oom Is there a way i can use D1 for this? Like download the D1 db through workers, a...

Rubi•4/29/24, 1:42 PM

1. download the D1 db through worker, idk, but wrangler able to do that by

npx wrangler d1 export dbname --remote --output db.sql

npx wrangler d1 export dbname --remote --output db.sql

so there is a way
2. someone can upload and use it, the D1 REST API should do the job

Kotamigo•4/29/24, 3:29 PM

Hello everyone! I'm currently integrating SvelteKit with Prisma and D1 for a web application, utilizing pages and workers. I've set up authentication with Google and have a corresponding auth handle in hooks.server.ts. However, I'm encountering an issue post-authentication:

After the Google auth redirect to http://localhost:5173/auth/callback/google with the necessary query params for code and scope, I try to access my database configuration from the platform environment (event?.platform?.env?.DB) within the hook, but it returns as undefined. Does anyone have insights on why this might be happening or how to ensure the DB variable is properly set and accessible within this context?

For all other requests, the platform attribute is present on the event object. I have been able to keep local development moving using

getPlatformProxy

getPlatformProxy

imported from wrangler, but this doesn't work in production because wrangler depends on node libraries that aren't available in production. I need the database in this context to create user records (they are created by an auth.js prisma adapter)

Related code: https://pastebin.com/ynzs24Jh

Thanks in advance for any help or pointers!

Pastebin

export async function getPrismaClient(event?: RequestEvent): Promise

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

Ccopper just read your comparisons with ps, have the same experience (ps is ultra fast a...

Super Morris•4/29/24, 5:02 PM

hello cooper, our benchmarks are from last week. however my dev did some magic with planet scale to keep the connection open and stuff (i dont really know) to achieve this results

feras•4/30/24, 3:31 AM

Hello everyone, is there a way to expose sqlite functions like sqlite3_update_hook?

kian•4/30/24, 3:39 AM

Not currently - it looks like it was requested last year and at the time there was no plan to implement a feature like that (https://community.cloudflare.com/t/watching-for-changes/476125/3)

feras•4/30/24, 3:44 AM

what about triggers in sqlite, are they supported?

kian•4/30/24, 4:12 AM

Yes

feras•4/30/24, 4:17 AM

thank you kian, is there any documentation on this?

kian•4/30/24, 4:29 AM

Nothing D1 specific, just https://www.sqlite.org/lang_createtrigger.html

feras•4/30/24, 5:06 AM

thank you for your answers, is there access to sqlite3_create_function?

kian•4/30/24, 5:29 AM

Nope

kian•4/30/24, 5:29 AM

You only have access to what's on https://developers.cloudflare.com/d1/build-with-d1/d1-client-api/

feras•4/30/24, 6:05 AM

it would've been awesome to have functions or update hook to listen to changes

Fferas it would've been awesome to have functions or update hook to listen to changes

Max (@rozenmd)•4/30/24, 6:59 AM

we're just getting started with D1 fwiw

MMax (@rozenmd)we're just getting started with D1 fwiw

feras•4/30/24, 7:25 AM

great job so far, hope to see full features soon

Ccopper does d1 sleep?

Hello, I’m Allie!•4/30/24, 12:56 PM

They do, yes, though you aren't billed for whether the D1 instance is asleep or not

Hello, I’m Allie!•4/30/24, 12:56 PM

You only pay for storage/ops

Ccopper ok and do you know the time d1 needs to wake up before it can respond to a reque...

Isaac McFadyen•4/30/24, 4:34 PM

It's fairly low - on the order of 10s to 100s of ms. I believe the more you have stored the longer it will be, but still fairly low in any case.

James•4/30/24, 5:22 PM

Quick question, and I know alpha databases should be avoided, but doesn't "deprecation" not mean "completely disabling functionality"?

From the recent email:

On July 1, 2024, we will deprecate D1 alpha databases. Upon deprecation, alpha databases will reject live SQL queries

I don't personally have any alpha DBs that I use in production anyway, but describing this as a "deprecation" but then completely disabling all queries to it seems to counter eachother a little. This was the first real direct comms I've received about D1 alpha DBs being formally deprecated. cc @Vy

James•4/30/24, 5:43 PM

To be clear, I'm not against the deprecation of the old alpha DBs, but this seems to be the first formal comms I've received about it. I totally understand preventing new ones being created, and actively discouraging their usage even, but a comms window of ~2 months until all queries just start getting rejected feels a little hasty. What if someone was on vacation for the next month or two, and hasn't been actively following discord or places where this deprecation may have been discussed? cc @Matt

Ryley•4/30/24, 6:08 PM

Sorry to be that guy... but are there any plans to expand beyond 10 free dbs. I have like many side projects. So for those...

PS: I am very grateful for free dbs with really good performance.

JJames Quick question, and I know alpha databases should be avoided, but doesn't "depre...

Vy•4/30/24, 6:08 PM

Email was meant to be first direct comms to keep raising awareness (will send periodic reminders), docs were already updated. Alpha should be considered deprecated now, with July as target for EOL (no live SQL queries). Given the increased reliability and performance of the current GA architecture, we want to heavily encourage folks to recreate their databases. Your alpha data is still accessible via backup through the end of the year.

Vy•4/30/24, 6:30 PM

hasn't been actively following discord or places where this deprecation may have been discussed

Agree passive outreach not ideal, we would love to keep improving here (maybe something like changelog subscriptions)

VVy Email was meant to be first direct comms to keep raising awareness (will send pe...

James•4/30/24, 6:31 PM

From the email, July 1st is the date for deprecation, which has a very different meaning from essentially making them non-functional, imo:

On July 1, 2024, we will deprecate D1 alpha databases. Upon deprecation, alpha databases will reject live SQL queries

~60 days until a database starts outright rejecting queries is extremely poor communication. I understand it was Alpha, but this feels like a pretty hasty and unfortunate way to treat your early adopters.

Chaika•4/30/24, 6:34 PM

the D1 Dashboard doesn't do a good job of exposing alpha dbs either, afaik it's just the existence of the backups tab and no metrics/time travel, would be nice if they were way better highlighted, with a warning symbol or something in the overview and a banner when you go into one of them.
The email itself should really highlight which DBs you have that are effected too imo

James•4/30/24, 6:35 PM

Agreed. The communication around this deprecation really leaves a lot to be desired. The timeline and way it was communicated is truly not great.

CChaika the D1 Dashboard doesn't do a good job of exposing alpha dbs either, afaik it's ...

Vy•4/30/24, 6:35 PM

wrangler d1 infowrangler d1 info will also tell you. Can take feedback on the UI callout

JJames Agreed. The communication around this deprecation really leaves a lot to be desi...

Vy•4/30/24, 6:36 PM

Thanks for feedback, we can always keep improving. What timeline would you expect? We're trying to balance tradeoffs, with no impending target date our active outreach can go ignored

VVy Thanks for feedback, we can always keep improving. What timeline would you expec...

James•4/30/24, 6:38 PM

Announcing the deprecation for July 1st to me would be totally fine. At that point, discourage usage, clearly show DBs as alpha, and actively prompt me to upgrade with dash banners, etc, with an impending cut-off date.
But deprecation is not the time to completely cut off all queries from databases. That should happen maybe 3-6 months after the deprecation is announced, and should be announced at the same time, with ongoing comms throughout that period as it gets closer.

JJames Announcing the deprecation for July 1st to me would be totally fine. At that poi...

Vy•4/30/24, 6:46 PM

thanks! I tripped up on language here. Our intention was with D1 GA, alpha dbs should be considered deprecated (everything still works). And then July 1st (3 months after), alpha dbs stop accpecting SQL queries. Can work to clarify in the future

VVy thanks! I tripped up on language here. Our intention was with D1 GA, alpha dbs s...

Isaac McFadyen•4/30/24, 6:46 PM

Definitely understand the idea of deprecation at D1 GA but it wasn't really communicated to the user very well.

VVy thanks! I tripped up on language here. Our intention was with D1 GA, alpha dbs s...

James•4/30/24, 6:46 PM

That may have been the intention, but that was never communicated to people, and that's the core problem here.

Isaac McFadyen•4/30/24, 6:47 PM

They were "superceded" but no real indication that they were going to be turned off until now.

James•4/30/24, 6:48 PM

I would strongly encourage you to push this "cut-off" date back until after the communicated deprecation date (after July 1st), and re-communicate that. It's going to be really hard to want to adopt something early from Cloudflare in future if this becomes commonplace.

Chaika•4/30/24, 6:49 PM

there was even that D1 GA Email, but it didn't mention anything about deprecation or cutoff, other then that "you can" migrate

Unsmart•4/30/24, 6:53 PM

I'm surprised alpha dbs arent already yeeted lol

Vy•4/30/24, 6:58 PM

Team can re-evaluate "cut off" (stop SQL requests) date.

Unsmart•4/30/24, 7:00 PM

I dont think any of the convo here holds any value since we dont have any and I cant imagine that many people have alpha dbs still. Would probably be more useful to find out who has alpha still and just email with them to get the old ones deleted

Unsmart•4/30/24, 7:01 PM

Or at least of those who still do I doubt they are really used, maybe a lot of people have abandoned old ones

UUnsmart I dont think any of the convo here holds any value since we dont have any and I ...

Chaika•4/30/24, 7:04 PM

me, james, and isaac at the very least all have Alpha DBs, I think I have one or two old projects pointed at them, not that they're super important but more notice is appreciated regardless/a proper deprecation

Chaika•4/30/24, 7:04 PM

iirc only people with alpha dbs only got the email, I only got it on 2/3 out of my accounts, and both had alpha dbs

I'm developing an application for Cloudflare that will use D1 as database. It's semi-static data tha

Similar Threads

Similar Threads

Similar Threads