hey folks. I'm new to D1 (and databases in General). My current problem is data conversion. It seems

hey folks. I'm new to D1 (and databases in General).
My current problem is data conversion. It seems to me I need to manually convert JS data types to SQL data types and vice versa.
For example, for D1 BOOLEAN values I get

in my function code. I expect to have

true

true

false

false

in my function code instead.
I looked into some ORMs but didn't find that functionality for automatic types conversion.

What are the ways to handle this problem? How do you approach it?

AAlexey hey folks. I'm new to D1 (and databases in General). My current problem is data ...

Bram•11/21/23, 4:30 PM

SQLite does not have a boolean storage class, but is instead stored as 0 or 1. You can use an ORM for this, I recommend

drizzle

drizzle

, a relatively new ORM that has a lot of momentum and is build with edge in mind.

Bram•11/21/23, 4:34 PM

it let's you declare sql schema's directly in typescript, where you can hint the ORM that that the integer should be interpreted as a boolean on application level: https://orm.drizzle.team/docs/column-types/sqlite

integer('id', { mode: 'boolean' })

integer('id', { mode: 'boolean' })

, same for timestamps, json etc

SQLite column types - Drizzle ORM

Drizzle ORM is a lightweight and performant TypeScript ORM with developer experience in mind.

AlexeyOP•11/21/23, 4:35 PM

ok, interesting, will give drizzle a try. Thank for the suggestion @Bram

Unsmart•11/21/23, 5:32 PM

The main way would be to create a worker as a proxy

James•11/21/23, 5:53 PM

I think this is that same obscure issue we've seen a few times, cc @Matt: https://community.cloudflare.com/t/d1-database-is-crashed/583564

Cloudflare Community

D1 database is crashed

My D1 database is suddenly crashed. And it is not recovered. In D1 tab, there is a message “Failed to reach database. Please try again later”. What’s going on?

Cyb3r-Jak3•11/21/23, 6:49 PM

Old vs new backend. My guess is the first one hasn’t had the backend migrated.

RRed Spider Is it possible to bulk insert via the API? I have an array of items i want to in...

Hello, I’m Allie!•11/21/23, 7:53 PM

Depending on the size of the items, and whether they are rows in a single table, or multiple tables, you might be able to do a multi-insert

Hello, I’m Allie!•11/21/23, 7:54 PM

INSERT INTO table1 (column1,column2 ,..)
VALUES 
   (value1,value2 ,...),
   (value1,value2 ,...),
    ...
   (value1,value2 ,...);

INSERT INTO table1 (column1,column2 ,..)
VALUES 
   (value1,value2 ,...),
   (value1,value2 ,...),
    ...
   (value1,value2 ,...);

Hello, I’m Allie!•11/21/23, 7:54 PM

Something like this

EXtremeExploit•11/21/23, 8:17 PM

is D1 down?

EXtremeExploit•11/21/23, 8:18 PM

alrighty just wanted to make sure it wasnt only me, thankie

Unsmart•11/21/23, 8:18 PM

Seems fine for me... is this every query or just some?

EXtremeExploit•11/21/23, 8:18 PM

all of them

EXtremeExploit•11/21/23, 8:18 PM

even the site fails to fetch the data

EXtremeExploit•11/21/23, 8:22 PM

EXtremeExploit•11/21/23, 8:23 PM

Unsmart•11/21/23, 8:24 PM

Looks like 1 of my databases isnt loading in the dashboard for some reason not sure whats going on with it but I havent used it at all. My other 2 databases seem just fine

Matt Silverlock•11/21/23, 8:33 PM

Account IDs and database IDs please, folks

mceu•11/21/23, 8:35 PM

Is anyone using d1 with Superset?

1.) Understand Your Tools:
Apache Superset: It's an open-source data exploration and visualization platform.
Cloudflare D1: A distributed SQL database that is part of Cloudflare’s suite of products. It's designed to be easily scalable and managed in a cloud environment.

2.) Prerequisites:
Ensure you have Apache Superset installed and properly configured.
Have access to a Cloudflare D1 instance and ensure it's operational.

3.) Database Connectivity:
Superset connects to various data sources through SQL Alchemy, a Python SQL toolkit and Object-Relational Mapping (ORM) library.
You need to find out if Cloudflare D1 provides a SQL Alchemy-compatible connector. If it does, you can use it to establish a connection between Superset and D1.
If there's no direct connector available, you might need to use a generic SQL Alchemy connector or a connector for a database that's compatible with Cloudflare D1's SQL dialect.

4.) Configure the Database Connection:
In Superset, go to the ‘Data’ menu and select ‘Databases’.
Add a new database connection.
Enter the connection details for your Cloudflare D1 database. This typically includes the database type, connection URL (which comprises the username, password, server address, and database name), and other parameters.

5.) Test the Connection:
After configuring, test the connection to ensure that Superset can communicate with Cloudflare D1.
Address any connection errors by reviewing the connection parameters and consulting the documentation for both Superset and Cloudflare D1.

6.) Import and Visualize Data:
Once connected, you can start importing data from Cloudflare D1 into Superset.
Use Superset’s features to create dashboards, charts, and reports based on the data stored in your Cloudflare D1 database.

MMatt Silverlock Account IDs and database IDs please, folks 🙂

Unsmart•11/21/23, 8:35 PM

Account:

dc941e8156f4a1336ca08481cb6d4222

dc941e8156f4a1336ca08481cb6d4222

Database:

a42194f9-3137-41a1-8f64-df0cf045a6d6

a42194f9-3137-41a1-8f64-df0cf045a6d6

mceu•11/21/23, 8:36 PM

Per the Superset folks - "Hi there, I can’t say I’ve heard of such an integration… but if there’s a sqlalchemy dialect for it (which I don’t see at a cursory glance) then it could be added. It might be worth reaching out to the D1 and/or SQLAlchemy communities to see if they’ve been connected. If so, we can probably carry that into Superset :)"

MMatt Silverlock Account IDs and database IDs please, folks 🙂

EXtremeExploit•11/21/23, 8:37 PM

account id (from workers overview):

6099b559aa0e86a9471d30f3dccafec6

6099b559aa0e86a9471d30f3dccafec6

db id:

2452c369-1d24-4bc9-9994-029c71619ca3

2452c369-1d24-4bc9-9994-029c71619ca3

EEXtremeExploit account id (from workers overview): `6099b559aa0e86a9471d30f3dccafec6` db id: `2...

Matt Silverlock•11/21/23, 9:05 PM

@EXtremeExploit @Unsmart | Tech debt We see a small number of databases failing to start - coinciding with a release. We're about to roll that back.

EXtremeExploit•11/21/23, 9:06 PM

alrighty thankie

MMatt Silverlock Account IDs and database IDs please, folks 🙂

oscar•11/21/23, 9:10 PM

Hey thanks!
account id: 34c0d8412965b33f782ad043d255542c
database id: 5d736d0e-4b0f-4726-935e-5ec1d1352825
Shows up as reachable and can see the data, but my worker still fails with D1_ERROR

Ooscar Hey thanks! account id: 34c0d8412965b33f782ad043d255542c database id: 5d736d0e-4...

Matt Silverlock•11/21/23, 9:16 PM

If you can see the data (in wrangler? the dashboard?) then you may have another issue.

D1_ERROR

D1_ERROR

may be an issue with your query.

oscar•11/21/23, 9:22 PM

oh, sorry! I didn't realize D1_ERROR was for queries too, I thought I had seen a different message earlier

clearwater4227•11/21/23, 9:46 PM

hey all, I am trying to create a D1 database via terraform and am getting this error:

Error: failed to create D1 database
with module.lookup_codes.cloudflare_d1_database.lookup-codes_database
on .terraform/modules/lookup_codes/cloudflare.tf line 29, in resource "cloudflare_d1_database" "lookup-codes_database":
resource "cloudflare_d1_database" "lookup-codes_database" {
error from makeRequest: Authentication error (10000)

It seems pretty easy to do...not sure what would be wrong. Was able to create from UI. Here is what Im doing:

resource "cloudflare_d1_database" "lookup-codes_database" {
account_id = var.cloudflare_account_id
name = "lookup_codes_database"
}

This account id is right as it create other resources just fine...

clearwater4227•11/21/23, 9:46 PM

wait...I think I know...

MMatt Silverlock @EXtremeExploit @Unsmart | Tech debt We see a small number of databases failing ...

Matt Silverlock•11/21/23, 9:48 PM

@EXtremeExploit @Unsmart | Tech debt we’re seeing databases recover (aka successfully load again).

EXtremeExploit•11/21/23, 9:48 PM

YokoWasis•11/22/23, 12:52 AM

is D1 is down right now ?

YYokoWasis is D1 is down right now ?

Cyb3r-Jak3•11/22/23, 12:56 AM

Shouldn't be. What errors are you getting?

YokoWasis•11/22/23, 12:56 AM

YokoWasis•11/22/23, 12:58 AM

i can't query my database

YYokoWasis i can't query my database

Cyb3r-Jak3•11/22/23, 12:58 AM

Can you share your account ID and database ID?

YokoWasis•11/22/23, 12:58 AM

CCyb3r-Jak3 Can you share your account ID and database ID?

YokoWasis•11/22/23, 12:58 AM

id : df608b9e06cca37a7fc399c8ca86eab4
database : 0f3e37af-ecd2-4e73-9766-72e181305f10

YYokoWasis id : df608b9e06cca37a7fc399c8ca86eab4 database : 0f3e37af-ecd2-4e73-9766-72e1813...

Cyb3r-Jak3•11/22/23, 12:59 AM

CC @Matt

YokoWasis•11/22/23, 1:03 AM

anyone experiencing the same, or is it just me ?

Cyb3r-Jak3•11/22/23, 1:09 AM

There were some reports earlier (d1-database) but they rolled back

Unsmart•11/22/23, 1:17 AM

My table that was broken seems like it still is so

maybe its not fully rolled back yet or they redid it not sure. Not too worried as I wouldnt use anything production with a beta product and it happened on a table I dont even use and I dont know why I made KEKW

YokoWasis•11/22/23, 1:23 AM

anyone can give an eta about this ?

hey folks. I'm new to D1 (and databases in General). My current problem is data conversion. It seems

Similar Threads

Similar Threads

Similar Threads