Is there a way to tell wrangler where to store the db locally? I have a monorepo with CF pages and

Is there a way to tell wrangler where to store the db locally?

I have a monorepo with CF pages and a worker both using the same DB.
Using miniflare on the pages app to do this. I'm guessing I should just do the same for the worker?

Yyeehow Is there a way to tell wrangler where to store the db locally? I have a monorep...

Benedikt•1/26/24, 12:55 AM

You're looking for "--persist-to" I believe

BBenedikt You're looking for "--persist-to" I believe

yeehowOP•1/26/24, 12:59 AM

Aha! Totally missed that, thanks!

thiccvitalik•1/26/24, 1:09 AM

This just made me monkey tilt. It's not even listed on the d1 limits page.

sqlite3_limit(db, SQLITE_LIMIT_COMPOUND_SELECT, 5);

sqlite3_limit(db, SQLITE_LIMIT_COMPOUND_SELECT, 5);

The SQLite default is 500 btw.

EDIT: just found out SQLite has window functions

Tthiccvitalik This just made me monkey tilt. It's not even listed on the d1 limits page. ```cp...

kian•1/26/24, 2:01 AM

3 is the recommended high-security value by SQLite.

kian•1/26/24, 2:01 AM

D1 aims for https://www.sqlite.org/security.html over the defaults.

mian.m.arslan•1/26/24, 10:16 AM

Hi Everyone
Can anyone help me to connect D1 database to mySql workbench?

Mmian.m.arslan Hi Everyone Can anyone help me to connect D1 database to mySql workbench?

Max (@rozenmd)•1/26/24, 10:23 AM

D1 is SQLite over HTTP, not MySQL - it might work with tools that support custom JDBC drivers: https://github.com/isaac-mcfadyen/d1-jdbc-driver

Max (@rozenmd)•1/26/24, 1:49 PM

https://github.com/isaac-mcfadyen/d1-jdbc-driver/blob/main/src/main/java/org/isaacmcfadyen/D1Queryable.java#L80 surprised this endpoint hasn't changed in 2 years

spookily_•1/26/24, 3:07 PM

quick question: do timestamptimestamp / datetimedatetime fields work in UTC?

spookily_•1/26/24, 3:07 PM

eg if i do something like...

insert into table (expires_at) values (datetime('now', '+1 hour'));

insert into table (expires_at) values (datetime('now', '+1 hour'));

will that give the same result regardless of the TZ the worker node is on?

Sspookily_quick question: do `timestamp` / `datetime` fields work in UTC?

Matt Silverlock•1/26/24, 3:12 PM

They default to UTC

Matt Silverlock•1/26/24, 3:13 PM

https://www.sqlite.org/lang_datefunc.html

spookily_•1/26/24, 3:14 PM

awesome, thank you, figured as much just wanted to confirm

Original message was deleted

Isaac McFadyen•1/26/24, 3:20 PM

lol yes.

Isaac McFadyen•1/26/24, 3:20 PM

I kinda gave up on that project since I was busy and didn't have much time to maintain.

PPaul Shriner Basically what I am trying to do is take a comma delimited list of sorted number...

Chinoman10•1/26/24, 4:22 PM

Funny, that's how vector databases work

jeremy•1/26/24, 4:42 PM

Hey, I don't know if someone can help me understand how best to integrate D1 with NextJS. I am confused between whether I should be using a separate Worker, or if I should be using a binding directly in the Page (web app).

WcaleNieWolny•1/26/24, 6:13 PM

Does D1 live in the edge or is it run centrally?

Martin•1/26/24, 6:31 PM

@nora do we have a horizon for that?

Softbear•1/26/24, 7:48 PM

The ability to export the database as a .sqlite was very useful, is there any plan to bring it back ?

Original message was deleted

Max (@rozenmd)•1/26/24, 9:25 PM

correction @WcaleNieWolny: D1 lives on the edge, but only one DB gets created near you. When replication ships, there'll be dbs around the world.

the D1 API (as documented here: https://developers.cloudflare.com/api/operations/cloudflare-d1-query-database) lives centrally, and is slow to query as a result (and why we recommend querying D1 from a Worker), until we rewrite it to run from a Worker.

Will•1/27/24, 1:23 AM

Is there anyway to switch database dynamically? I am building a multi tenant app, and would like to just use one worker for each client

Pepper•1/27/24, 2:47 AM

suggestion: when adding data through the web ui, there should be a confirmation alert if you try to close the tab

angora•1/27/24, 5:49 AM

What is the best way to set up transfer of a set of tables from Google bigquery to D1 on daily basis

Bram•1/27/24, 11:32 AM

Hi all, have more of a general question, considering D1 is a new kind of database (basically replicated sqlite files at edge) and considering the very fast reads of SQLite, is there still a need to use traditional key value stores for perf / latency reasons?

kian•1/27/24, 11:59 AM

D1 isn’t replicated at edge currently

Samantas5855•1/27/24, 12:13 PM

Error inserting data for channel1: 500, {
"result": [],
"success": false,
"errors": [
{
"code": 7500,
"message": "logEvent(): Cumulative size of blobs exceeds 5120 bytes)."
}
],
"messages": []
}

Samantas5855•1/27/24, 12:13 PM

is that new?

Samantas5855•1/27/24, 12:14 PM

I never had this one before

Samantas5855•1/27/24, 12:30 PM

my script can't update my d1 db since 9 hours ago and my app is now broken because of this

Samantas5855•1/27/24, 12:31 PM

changelog mentions nothing and the only identical limit I could find is on analytics

SSamantas5855 Error inserting data for channel1: 500, { "result": [], "success": false, ...

Max (@rozenmd)•1/27/24, 2:22 PM

that looks like a bug, will investigate when I'm at a laptop

WcaleNieWolny•1/27/24, 2:58 PM

Can some please explain to me how on earth did i get 4 billion writes to the db? I included some screenshots of the usage reported in different places. I do not get this at all. I get the reads but not the writes

WcaleNieWolny•1/27/24, 3:00 PM

Yeah i get that

WcaleNieWolny•1/27/24, 3:00 PM

but it still makes no sense to me. I only do single insert queries

WcaleNieWolny•1/27/24, 3:00 PM

inserting 1 row into the db per query

WcaleNieWolny•1/27/24, 3:01 PM

i also have 5 indexes but that is still way lower then the billions written/read

WcaleNieWolny•1/27/24, 3:02 PM

this does not make sense. How does a read query cause a write?

Max (@rozenmd)•1/27/24, 3:09 PM

if it's something like insert ... returning *, with inefficient indexes you could end up with several writes and a read per insert

Aangora What is the best way to set up transfer of a set of tables from Google bigquery ...

Matt Silverlock•1/27/24, 3:15 PM

A Cron Trigger that reads from BQ -> does what kind of aggregation you need -> writes to D1: https://developers.cloudflare.com/workers/configuration/cron-triggers/

Cron Triggers · Cloudflare Workers docs

Enable your Worker to be executed on a schedule.

WWcaleNieWolny Can some please explain to me how on earth did i get 4 billion writes to the db?...

Matt Silverlock•1/27/24, 3:17 PM

Look at the returned row counts for your queries: https://developers.cloudflare.com/d1/observability/metrics-analytics/#row-counts

Metrics and analytics · Cloudflare D1 docs

D1 exposes database analytics that allow you to inspect query volume, response size, and query latency across all and/or each database in your …

WcaleNieWolny•1/27/24, 3:19 PM

My writes are near constant. But as you can see when I enable reads (select) I get a lot of write queries which is strange

WWcaleNieWolny this does not make sense. How does a read query cause a write?

Matt Silverlock•1/27/24, 3:19 PM

Without your actual queries it’s impossible to say. EXPLAIN QUERY PLAN <query>EXPLAIN QUERY PLAN <query> will show you what your queries are doing. Functions like random()random() and similar will cause writes.

WcaleNieWolny•1/27/24, 3:19 PM

I use drizzle ORM, is there any way to see query history?

WcaleNieWolny•1/27/24, 3:20 PM

I don’t write the sql myself. I use an ORM