would be cool if it was possible to do it automatically for everyone and would avoid a lot of it, I

would be cool if it was possible to do it automatically for everyone and would avoid a lot of it, I assume a lot of DBs could be migrated without issue automagically

CChaika would be cool if it was possible to do it automatically for everyone and would a...

josh•4/30/24, 7:14 PM

Yeah. That would have generated less pushback. Doing a 0 downtime migration is a lot of work, and frankly not worth it for an alpha product. We could migrate DBs that are not in use with no problem, but it might be pointless to migrate them. We could take downtime to migrate DBs for active use cases, but that would still have to be coordinated with customers.

Jjosh Yeah. That would have generated less pushback. Doing a 0 downtime migration is a...

ChaikaOP•4/30/24, 7:16 PM

That makes sense, can understand why that would be difficult, then I would just hope there's a proper shut off, deprecation happening on x date, shutdown on y, emails telling you which DBs need to be upgraded with reminders, and stuff in the dash is all that would be needed

DaniFoldi•4/30/24, 7:16 PM

I'd argue it's still way less frustrating to take an alpha db offline for a minute or two to keep it going on the new backend

DaniFoldi•4/30/24, 7:16 PM

vs dumping it from the old backend, recreating it, ingesting data and changing bindings

CChaika there was even that D1 GA Email, but it didn't mention anything about deprecatio...

Vy•4/30/24, 7:24 PM

@Chaika did you recieve this email recently? The copy is from our alpha to beta email, not for GA

kingmesal•4/30/24, 7:24 PM

I've got a use case where I am running into the D1 LIMITS on sql queries - I'm on paid plan
Queries per Worker invocation (read subrequest limits) 50 (Bundled) / 1000 (Unbound)Queries per Worker invocation (read subrequest limits) 50 (Bundled) / 1000 (Unbound)

I allow my customer to upload a csv of users. I insert or update the user records into my d1 table. So, if the unique key (email address) already exists, I perform an update. (2 queries). Can the insert in conflict be batched? Is there a limit on how many statements can get put into a batch? Is this a potential way to get around the 1000 queries?

VVy @Chaika did you recieve this email recently? The copy is from our alpha to beta ...

ChaikaOP•4/30/24, 7:26 PM

oh you're right that is older opps, that's from August of last year. That doesn't really help though, still lack of communication and suddenly an email about them being gone in 2 months

CChaika oh you're right that is older opps, that's from August of last year. That doesn'...

Vy•4/30/24, 7:26 PM

thanks, we're workign to clarify here based on feedback. just wanted to check somethign else weird wasnt happening on email side

James•4/30/24, 7:27 PM

thanks Vy. If we can help in any way, please just let us know

VVy thanks, we're workign to clarify here based on feedback. just wanted to check so...

ChaikaOP•4/30/24, 7:27 PM

nah you're good, I was just looking for anything mentioning d1 alpha (besides the latest one today) and that's all I found, could have sworn it was more recent

somoni•4/30/24, 8:22 PM

is there a tutorial or guide on how to do schema upgrades on a live production site using d1 ? i was not able to find one in the d1 docs. By schema upgrade i mean adding and/or removing fields from a table that would make it backward incompatible with the previous deployed code

Vy•4/30/24, 8:50 PM

I'm working to update various channels but re-worked proposal is:
* With D1 generally available, we're deprecating alpha databases. Clarifying that deprecation does not impact D1 alpha operations, everything still works.
* Users should follow the alpha database migration guide to recreate a new D1 database from their alpha database backup. Following a 3-month deprecation period, D1 alpha databases will stop accepting live SQL queries on August 1, 2024.
* After August 1, 2024, users can still access alpha database backups until the end of 2024.

VVy I'm working to update various channels but re-worked proposal is: * With D1 gene...

James•4/30/24, 8:56 PM

3 months notice until complete inoperability is a little shorter than I would like/expect, but this is much nicer than 60 days, assuming it's actively and well communicated over the coming weeks and months. I appreciate you hearing our concerns and feedback here :MeowHeartCloudflare:

James•4/30/24, 8:59 PM

In future, I think it would really good to communicate deprecations like this significantly more actively. It sounds like this has been the plan since GA announcement at dev week, and if that's the case, the deprecation should have been announced as early as then.

VVy I'm working to update various channels but re-worked proposal is: * With D1 gene...

ChaikaOP•4/30/24, 9:04 PM

that is way better, just please have the emails include which D1 DBs are effected and the dash communicate that as well. People will react very differently to "in 3 months D1 Alpha is shutting down..migrate now" vs "This list of DBs <list of DBS> need to be migrated before then" when the list of dbs may include a super important one. Not everyone follows alpha vs beta/production differences, some people may have setup something a while ago and just not be paying attention, need to be directly told something of importance is going to break soon

F0rce•4/30/24, 9:21 PM

Just wanted to put this on the radar because with D1 reaching GA some time ago, this is still a major game-breaker during local development or migrating to d1

Vlady•4/30/24, 11:53 PM

Hey folks, how do I reset my remote database without deleting the binding? I tried running delete from sqlite_master where type in ('table', 'index', 'trigger')delete from sqlite_master where type in ('table', 'index', 'trigger'), but got this error

D1_ERROR: table sqlite_master may not be modified

D1_ERROR: table sqlite_master may not be modified

di_uthred•5/1/24, 3:54 AM

Am the only one who had a bunch a DBs dissappear from the console recently? I had like 12 - now I see 6. What's interesting, they still seem to be accessible by the workers.

Ddi_uthred Am the only one who had a bunch a DBs dissappear from the console recently? I h...

Hello, I’m Allie!•5/1/24, 7:06 AM

FF0rce Just wanted to put [this](https://github.com/cloudflare/workers-sdk/issues/3160)...

geg•5/1/24, 8:12 AM

HHello, I’m Allie!May be related to this? https://canary.discord.com/channels/595317990191398933/9...

Max (@rozenmd)•5/1/24, 8:34 AM

Definitely not - we're not touching any databases

Ddi_uthred Am the only one who had a bunch a DBs dissappear from the console recently? I h...

Max (@rozenmd)•5/1/24, 8:35 AM

What's your accountId?

Jordan•5/1/24, 11:12 AM

Is deploying D1 myself (e.g. in my own k8s cluster for example) supported?

Jordan•5/1/24, 11:13 AM

I assume this is done with the local version of D1? If that's true is the local D1 version via Wrangler prod-capable? (ignoring the fact that D1 is in beta currently)

JJordan I assume this is done with the local version of D1? If that's true is the local ...

Rubi•5/1/24, 11:15 AM

its basically an sqlite file, maybe you need to do some load test

RRubi its basically an sqlite file, maybe you need to do some load test

Jordan•5/1/24, 11:17 AM

I know it uses SQLite, but D1 is an actual API in-front of that

JJordan I assume this is done with the local version of D1? If that's true is the local ...

Hello, I’m Allie!•5/1/24, 11:24 AM

The D1 HTTP API is not available anywhere other than in “prod”

Hello, I’m Allie!•5/1/24, 11:25 AM

You could probably build a fake one with a Worker

Hello, I’m Allie!•5/1/24, 11:25 AM

But then you might as well use another SQLite provider

kian•5/1/24, 11:28 AM

Yeah, I'd just use normal SQLite if you're not using it for just local development

Jordan•5/1/24, 11:35 AM

Roger

Softbear•5/1/24, 3:55 PM

Hello in the docs for backups there is this line
"Time Travel does not yet allow you to clone or fork an existing database to a new copy. In the future, Time Travel will allow you to fork (clone) an existing database into a new database, or overwrite an existing database." this makes the restore way less useful than I imagined, I can't event restore a single table without loosing all my state.

https://developers.cloudflare.com/d1/reference/time-travel/

Do you plan to implement this in the near future ? And what about the export to sql or .sqlite, I read that you planned to reitroduce it, but still no news.

Cloudflare Docs

Time Travel and backups · Cloudflare D1 docs

Time Travel is D1’s approach to backups and point-in-time-recovery, and allows you to restore a database to any minute within the last 30 days.

kian•5/1/24, 4:03 PM

wrangler d1 exportwrangler d1 export

kian•5/1/24, 4:03 PM

https://developers.cloudflare.com/d1/build-with-d1/import-data/#export-an-existing-d1-database

ajgeiss0702•5/1/24, 4:22 PM

Was I imagining things or did the blog post talking about D1 replication and sessions used to say that the API is available today and will use the old model and will automatically upgrade when replication is released?

I thought i read that when it was released, and sat down to look into it now, and I dont see it. It's very possible I just misread it the first time

kian•5/1/24, 4:56 PM

https://blog.cloudflare.com/building-d1-a-global-database is the blog you're thinking of but the Sessions API isn't a thing yet & there are no plans to make the existing API behave differently.

Since the Sessions API changes D1's consistency model, developers must opt-in to the new API. Existing D1 API methods are unchanged and will still have the same snapshot isolation consistency model as before. However, only queries made using the new Sessions API will use replicas.

ajgeiss0702•5/1/24, 5:04 PM

Yeah, I was searching through that again. I could have sworn it said something about how you can write code for sessions today and theyll fall back to the old behaviour, then automatically use the new behaviour once its available (and the old api would remain the same)

Vlady•5/1/24, 7:27 PM

Here goes nothing

VVlady Here goes nothing 👀 🤞🏼

Frank•5/1/24, 9:09 PM

nice. i assume we expect the 10gb or 50k db limit to increase sooner than later

FFrank nice. i assume we expect the 10gb or 50k db limit to increase sooner than later

ChaikaOP•5/1/24, 9:11 PM

The DB limit is soft, the size per db limit is hard

ChaikaOP•5/1/24, 9:12 PM

d1-database

ChaikaOP•5/1/24, 9:12 PM

d1-database

ChaikaOP•5/1/24, 9:13 PM

they've already said they can support virtually limitless (millions) of separate dbs

CChaika they've already said they can support virtually limitless (millions) of separate...

Frank•5/1/24, 9:15 PM

awesome. thanks. just waiting on dynamic binding then.
and be careful about limit bindings per worker at a given time. I think ~5k right now

Frank•5/1/24, 9:16 PM

hopefully bindings can be both created and released dynanmically

FFrank awesome. thanks. just waiting on dynamic binding then. and be careful about limi...

ChaikaOP•5/1/24, 9:26 PM

when I think about dynamic bindings I think about Worker for Platform's Dynamic Dispatch binding, where it's just a singular binding you can use to get access to a worker/instance, ex:

    const worker = env.DISPATCHER.get("customer-worker-1");
    var responseFromUserScript = worker.fetch(req);

    const worker = env.DISPATCHER.get("customer-worker-1");
    var responseFromUserScript = worker.fetch(req);

would be cool if it was possible to do it automatically for everyone and would avoid a lot of it, I

Similar Threads

Similar Threads

Similar Threads