but if a customers needs way more than 2gb of data, then do you see this viable? I am trying to set

but if a customers needs way more than 2gb of data, then do you see this viable?
I am trying to setup a flow where a js script on websites sends events data to a cloudflare worker, from which I am trying to understand how to best forward it to a database

Max (@rozenmd)•11/22/23, 11:49 AM

for events data d1 is going to have a bad time, you might want to look into https://developers.cloudflare.com/analytics/analytics-engine/

Ffedero but if a customers needs way more than 2gb of data, then do you see this viable?...

zegevlier•11/22/23, 11:49 AM

also see: d1-database

federoOP•11/22/23, 11:52 AM

analytics engine is not feasible either with a retention period of only 3 months

Zzegevlier also see: https://discord.com/channels/595317990191398933/992060581832032316/117...

federoOP•11/22/23, 12:04 PM

it seems like Hyperdrive is just for reads not writes

GGustavo One idea would be to aggregate and move to somewhere else

federoOP•11/22/23, 12:05 PM

how do you move it to somewhere else? I am struggling to understand how to take payloads coming into workers and load them into a solid database

Ffedero it seems like Hyperdrive is just for reads not writes

kian•11/22/23, 12:06 PM

Writes will just passthrough the caching and still take advantage of the Cloudflare backbone.

GGustavo I mean write to Analytics Engine, then have a cron job run each month to aggrega...

Rob M.•11/22/23, 12:29 PM

But what about WAE sampling? Looks like we can lose data in some cases.

RRob M.But what about WAE sampling? Looks like we can lose data in some cases.

Max (@rozenmd)•11/22/23, 12:46 PM

if you need every single row, i would just use postgres

Ffedero it seems like Hyperdrive is just for reads not writes

Matt Silverlock•11/22/23, 1:28 PM

Hyperdrive doesn’t restrict you. Curiously: why do you think it’s just reads?

Nnick_jordan When running a query from the UI Console I'm getting the following error... ```...

Max (@rozenmd)•11/22/23, 3:07 PM

That looks like an alpha database - the current default backend for D1 performs around 10-11x faster for writes, 20x faster for reads: [https://blog.cloudflare.com/d1-turning-it-up-to-11/]

ajgeiss0702•11/22/23, 5:25 PM

in the dashboard, is there a plan to add rows read/written so I can have an idea on what my bill would be? I know I can see it for the past week by going to the per-db metrics, but I would like to be able to see the total for the month since that what i'll be billed for once billing gets enabled

Aajgeiss0702 in the dashboard, is there a plan to add rows read/written so I can have an idea...

Max (@rozenmd)•11/22/23, 5:31 PM

That's a good point - I'll add it to our backlog

MMax (@rozenmd)That's a good point - I'll add it to our backlog

ajgeiss0702•11/22/23, 5:38 PM

ty!

YokoWasis•11/23/23, 12:05 AM

Let's hope today D1 won't be down like yesterday. I made 4 replica of the database. It would be absurd of all of them going down.

YokoWasis•11/23/23, 12:52 AM

My D1 database said too many queue operation. Any recommendation ? is D1 is not feasible for many users ?

Tesbarmix•11/23/23, 12:53 AM

Hello

YokoWasis•11/23/23, 12:59 AM

What about kv ? does kv can handle R/W requests better than d1 ?

YYokoWasis My D1 database said too many queue operation. Any recommendation ? is D1 is not ...

Max (@rozenmd)•11/23/23, 8:54 AM

what's your use case?

MMax (@rozenmd)what's your use case?

YokoWasis•11/23/23, 9:17 AM

student's national exam.

YYokoWasis student's national exam.

Max (@rozenmd)•11/23/23, 9:20 AM

as in, are you using D1 as a single database for an API that your frontend talks to? how many requests per second?

MMax (@rozenmd)as in, are you using D1 as a single database for an API that your frontend talks...

YokoWasis•11/23/23, 9:54 AM

Yes, worker as a backend api. About 1k req per second.

Hearse•11/23/23, 11:17 AM

wrangler d1 execute 'cloudflare-db' --local --file='./drizzle/0000_short_lockheed.sql'

✘ [ERROR] Could not read file: ./drizzle/0000_short_lockheed.sql

  ENOENT: no such file or directory, open
  '/Users/administrator/backend/project/drizzle/0000_short_lockheed.sql'
  
  If you think this is a bug, please open an issue at:
  https://github.com/cloudflare/workers-sdk/issues/new/choose

wrangler d1 execute 'cloudflare-db' --local --file='./drizzle/0000_short_lockheed.sql'

✘ [ERROR] Could not read file: ./drizzle/0000_short_lockheed.sql

  ENOENT: no such file or directory, open
  '/Users/administrator/backend/project/drizzle/0000_short_lockheed.sql'
  
  If you think this is a bug, please open an issue at:
  https://github.com/cloudflare/workers-sdk/issues/new/choose

Hearse•11/23/23, 11:18 AM

Why is it doing this when following https://github.com/drizzle-team/drizzle-orm/blob/main/examples/cloudflare-d1/README.md

GitHub

drizzle-orm/examples/cloudflare-d1/README.md at main · drizzle-team...

TypeScript ORM that feels like writing SQL. Contribute to drizzle-team/drizzle-orm development by creating an account on GitHub.

Rob M.•11/23/23, 11:19 AM

Is the file really there?

RRob M.Is the file really there?

Hearse•11/23/23, 11:20 AM

it doesnt need to be i thought .

RRob M.Is the file really there?

Hearse•11/23/23, 11:20 AM

https://orm.drizzle.team/docs/quick-sqlite/d1

Cloudflare D1 - Drizzle ORM

Drizzle ORM is a lightweight and performant TypeScript ORM with developer experience in mind.

Hearse•11/23/23, 11:21 AM

i thought the command creates it

Hearse•11/23/23, 11:22 AM

literally just run the command in the directory

Hearse•11/23/23, 11:22 AM

i have d1 setup in my wrangler config and its working fine, running this command in the same project folder doesnt work

Hearse•11/23/23, 11:23 AM

if i do create a dummy file, it literally say this.

mkdir drizzle
touch drizzle/0000_short_lockheed.sql
wrangler d1 execute 'cloudflare-db' --local --file='./drizzle/0000_short_lockheed.sql'

✘ [ERROR] Error: must provide --command or --file.


If you think this is a bug then please create an issue at https://github.com/cloudflare/workers-sdk/issues/new/choose
🪵  Logs were written to "/Users/administrator/Library/Preferences/.wrangler/logs/wrangler-2023-11-23_11-23-02_962.log"

mkdir drizzle
touch drizzle/0000_short_lockheed.sql
wrangler d1 execute 'cloudflare-db' --local --file='./drizzle/0000_short_lockheed.sql'

✘ [ERROR] Error: must provide --command or --file.


If you think this is a bug then please create an issue at https://github.com/cloudflare/workers-sdk/issues/new/choose
🪵  Logs were written to "/Users/administrator/Library/Preferences/.wrangler/logs/wrangler-2023-11-23_11-23-02_962.log"

Hearse•11/23/23, 11:24 AM

wrangler 3.17.1

wrangler 3.17.1

btw, it should be latest.

Rob M.•11/23/23, 11:44 AM

maybe, there is a newline symbol after "--"?

HHearse if i do create a dummy file, it literally say this. ``` mkdir drizzle touch driz...

Max (@rozenmd)•11/23/23, 12:06 PM

thats weird, do you mind filing a github issue? looks like a bug

Max (@rozenmd)•11/23/23, 12:07 PM

ohhhh

Hearse•11/23/23, 12:07 PM

Will do when i wake up

Max (@rozenmd)•11/23/23, 12:07 PM

no thats just a bad error message

Max (@rozenmd)•11/23/23, 12:07 PM

it throws "Error: must provide --command or --file" if there's an empty string too

Max (@rozenmd)•11/23/23, 12:08 PM

I think you need to run

npm run generate

npm run generate

for drizzle to create the SQL

MMax (@rozenmd)I think you need to run `npm run generate` for drizzle to create the SQL

Hearse•11/23/23, 12:14 PM

huh thats weird.

Hearse•11/23/23, 12:14 PM

because in the quickstart it doesnt address that

Hearse•11/23/23, 12:14 PM

https://orm.drizzle.team/docs/quick-sqlite/d1

Cloudflare D1 - Drizzle ORM

Drizzle ORM is a lightweight and performant TypeScript ORM with developer experience in mind.

MMax (@rozenmd)I think you need to run `npm run generate` for drizzle to create the SQL

Hearse•11/23/23, 12:15 PM

literally just says have your wrangler.toml which i already know is right

Hearse•11/23/23, 12:15 PM

and then run the command.

Hearse•11/23/23, 12:16 PM

ill do that though

Max (@rozenmd)•11/23/23, 12:16 PM

actually wait

Max (@rozenmd)•11/23/23, 12:16 PM

it seems like their docs are written for you to clone their example project

wrangler d1 execute 'cloudflare-db' --local --file='./drizzle/0000_short_lockheed.sql' ✘ [ERROR] Could not read file: ./drizzle/0000_short_lockheed.sql ENOENT: no such file or directory, open '/Users/administrator/backend/project/drizzle/0000_short_lockheed.sql' If you think this is a bug, please open an issue at: https://github.com/cloudflare/workers-sdk/issues/new/choose

mkdir drizzle touch drizzle/0000_short_lockheed.sql wrangler d1 execute 'cloudflare-db' --local --file='./drizzle/0000_short_lockheed.sql' ✘ [ERROR] Error: must provide --command or --file. If you think this is a bug then please create an issue at https://github.com/cloudflare/workers-sdk/issues/new/choose 🪵 Logs were written to "/Users/administrator/Library/Preferences/.wrangler/logs/wrangler-2023-11-23_11-23-02_962.log"

but if a customers needs way more than 2gb of data, then do you see this viable? I am trying to set

Similar Threads