Cloudflare Developers•3y ago

Ugh: Discord made it look like it was today 🙃

Ugh: Discord made it look like it was today

MMatt Silverlock Ugh: Discord made it look like it was today 🙃

DanTheGoodman•9/29/23, 2:07 PM

Someone replied to it yesterday

Laurent Pellegrino•9/29/23, 4:24 PM

Hello!

Is there any official docs about how to make transactions with D1? I can only find this old blog post: https://blog.cloudflare.com/whats-new-with-d1/

Laurent Pellegrino•9/29/23, 4:30 PM

How to increment a counter value stored as column in a row by 1 atomically?

Laurent Pellegrino•9/29/23, 4:30 PM

Is that supported?

Laurent Pellegrino•9/29/23, 4:48 PM

Interesting. Thanks a lot.

Laurent Pellegrino•9/29/23, 4:55 PM

Loving how D1 looks so far. I have a unique use case and need some guidance. I'm working with a database that contains roughly 50M entries. Interestingly, this database is updated only once every day, and after that, we only perform read operations. I'm curious – is there a method to either import an SQL backup or establish the database locally and then migrate it to D1, without incurring charges for the 50M writes daily? Appreciate any insights on this!

Laurent Pellegrino•9/29/23, 8:50 PM

Do you plan to keep the manual backup commands with wrangler on GA?

Laurent Pellegrino•9/29/23, 8:51 PM

Does a backup restore (not using time traval) incurs a charge (i.e. does it rows as writes)?

LLaurent Pellegrino Does a backup restore (not using time traval) incurs a charge (i.e. does it rows...

Max (@rozenmd)•9/29/23, 9:13 PM

The backup commands are from a time when it was necessary to do backups with D1. Time travel lets you rollback commit by commit, so it's not as useful.

Laurent Pellegrino•9/29/23, 9:14 PM

@rozenmd I am asking because of the use case I described just above.

Laurent Pellegrino•9/29/23, 9:16 PM

I'd be willing to shell out $0.1 per million writes for my situation, but $1 per million writes is certainly off-putting.

Old Man Maker•9/30/23, 2:02 AM

Hey, can someone help me understand the D1 insert SQL a little better please.

I have an object def that looks like this:

interface DetailsDataType{
    description:string,
    ref:string //Primary Key in Table
}

interface DetailsDataType{
    description:string,
    ref:string //Primary Key in Table
}

I'm getting an array of these objects that I want to insert into my D1.

I get how to inject a single string, but the object array is not happening for me.
I assume there is some shorthand way of doing this, don't want to put an ugly for loop in there.

This is the INSERT code, where the bind statement is incorrect because descData is an array:

descData: DetailsDataType[] = <ARRAY OF OBJECTS>;

const { results } = await c.env.DATABASE.prepare("INSERT INTO details (ref, description) VALUES (?, ?) RETURNING *")
        .bind(descData.description, descData.ref)
        .run()

const { results } = await c.env.DATABASE.prepare("INSERT INTO details (ref, description) VALUES (?, ?) RETURNING *")
        .bind(descData.description, descData.ref)
        .run()

Can this even be done this way?

Old Man Maker•9/30/23, 2:24 AM

I'm assuming I will need to use the Batching statement?

Old Man Maker•9/30/23, 2:42 AM

Ok - just got this to work with a looping batcher. Pretty sweet.

Can I make a suggestion please - if the use of hyphens '-' in names is causing so many issues, why not just blanket ban the use in any named object.

It just took me 10 mins to spot the issue with the table name, when it should have been rejected as a valid table name at the point of creation. Perhaps the wrangler tool does this, but the dashboard does not.

Thoughts?

franz101•9/30/23, 11:34 AM

What's the best practices for an ETL to export data from D1?

Ffranz101 What's the best practices for an ETL to export data from D1?

Dahoom•9/30/23, 11:37 AM

For the beta I'm currenly using community member nora's repo https://github.com/nora-soderlund/cloudflare-d1-backups

GitHub

GitHub - nora-soderlund/cloudflare-d1-backups: A package to dump a ...

A package to dump a copy of your experimental D1 database through a worker. - GitHub - nora-soderlund/cloudflare-d1-backups: A package to dump a copy of your experimental D1 database through a worker.

Dahoom•9/30/23, 11:38 AM

it works pretty well but I dont think there is any official system the beta system yet

franz101•9/30/23, 11:44 AM

Ah interesting I saw the dump function. I was curious as I was wondering about a local python connection to query the db. I guess you would need a worker that evals the query. I'm testing d1 for my ai news aggregator:
https://ml-news-liard.vercel.app/

Ffranz101 Ah interesting I saw the dump function. I was curious as I was wondering about a...

Dahoom•9/30/23, 11:49 AM

You probably need a worker yeah, I know for a fact you can connect directly to turso currently with libsql

Jason Hostetter•9/30/23, 3:25 PM

Is it correct that there is no way to programmatically create or connect to different databases for users without an enterprise account?

JJason Hostetter Is it correct that there is no way to programmatically create or connect to diff...

Cyb3r-Jak3•9/30/23, 3:30 PM

I'm pretty sure even with enterprise you can't

Jason Hostetter•9/30/23, 3:32 PM

There was mention of creating one db per client in the beta announcement, but only for enterprise

Cyb3r-Jak3•9/30/23, 3:32 PM

Oh workers for platforms

Jason Hostetter•9/30/23, 3:32 PM

Although the description wasn’t super clear to me

Jason Hostetter•9/30/23, 3:32 PM

Yeah

Cyb3r-Jak3•9/30/23, 3:33 PM

Yeah, sorry haven't used WfP but that will allow you to

Jason Hostetter•9/30/23, 3:34 PM

Is workers for platforms enterprise only?

Cyb3r-Jak3•9/30/23, 3:35 PM

It is

Jason Hostetter•9/30/23, 3:35 PM

Bahhh

Jason Hostetter•9/30/23, 3:38 PM

Ok well, maybe I’ll have to try to hack something together with turso or something

JJason Hostetter Is it correct that there is no way to programmatically create or connect to diff...

Matt SilverlockOP•9/30/23, 4:01 PM

You can programmatically create databases - the D1 API isn’t limited to enterprise accounts at all. There is a limit on the # of DBs you create (at the moment) on regular accounts but we will increase this very soon

Jason Hostetter•9/30/23, 4:14 PM

Oh nice! Can I dynamically bind a database depending on some parameter or key passed in the request to the worker?

JJason Hostetter Oh nice! Can I dynamically bind a database depending on some parameter or key pa...

Matt SilverlockOP•9/30/23, 4:17 PM

No, that is something we don’t yet support. Working on it. If you can bind any resource in your account it would break the security model (how do you bind only the right things?) - so the work is focused on APIs to help group/tag resources vs “any”

Jason Hostetter•9/30/23, 4:37 PM

Great to hear, looking forward to it. Was worried that functionality would be locked behind enterprise, which would effectively kill my project (at least with D1)

Samantas5855•9/30/23, 10:00 PM

Can I add data to a d1 database from workers and a python script?

Samantas5855•9/30/23, 10:04 PM

also does delete count as a write or is it free?

SSamantas5855 Can I add data to a d1 database from workers and a python script?

Cyb3r-Jak3•9/30/23, 10:19 PM

You'd would have the use the API to add the content from a python script

Cyb3r-Jak3•9/30/23, 10:22 PM

And I'd imagine that deleting rows counts as a write but not about if you delete a D1 database entirely

ajgeiss0702•10/1/23, 12:58 AM

Once d1 has read replication, any idea on how consistent it would be? I’m currently planning a feature that would require semi-strong consistency
(Please ping me if you reply)