Trying to load my alpha db dump into the new backend via wrangler, with 27k or so lines, first just

Trying to load my alpha db dump into the new backend via wrangler, with 27k or so lines, first just hung as it did for @mozzy , tried breaking into a file with 4999 lines, now getting

[ERROR] A request to the Cloudflare API (/accounts/c6ad2463148cc51ec38821637c5af28f/d1/database/f90e9b86-b9df-4257-b400-827dac122d25/query) failed.

  statement too long [code: 7500]

[ERROR] A request to the Cloudflare API (/accounts/c6ad2463148cc51ec38821637c5af28f/d1/database/f90e9b86-b9df-4257-b400-827dac122d25/query) failed.

  statement too long [code: 7500]

[ERROR] A request to the Cloudflare API (/accounts/c6ad2463148cc51ec38821637c5af28f/d1/database/f90e9b86-b9df-4257-b400-827dac122d25/query) failed.

  statement too long [code: 7500]

[ERROR] A request to the Cloudflare API (/accounts/c6ad2463148cc51ec38821637c5af28f/d1/database/f90e9b86-b9df-4257-b400-827dac122d25/query) failed.

  statement too long [code: 7500]

Jason HostetterOP•5/19/23, 6:00 PM

Tried pasting entire 27k sql lines into the web console, after a minute or two all the statements showed up in the console but the last line is again: statement too longstatement too long

Original message was deleted

nickchomey•5/19/23, 6:12 PM

I'm confused about the db size limits. The blog says in the pricing that 1GB is available to free users and 5GB included and $0.75/GB beyond that.

But then it says "Developerson the Workers Paid plan will get access to 1GB dbs in the very near future".

Does this mean that the limit is currently still 100mb, will soon be 1Gb and later will change to what the pricing shows?

Ffidozo Am I crazy to ask for one D1 database per user on my app? Similar to DOs, the D1...

Bulbasaur•5/19/23, 6:16 PM

maybe a better approach would be dbs per regional area

Bulbasaur•5/19/23, 6:17 PM

when a user creates their account, pick a db based on their geography to route to, and always use that db for that user

Bulbasaur•5/19/23, 6:18 PM

depending on how far you want to take that, you could do an NA region, EU region and Asia region for example, or more

Bulbasaur•5/19/23, 6:19 PM

Location Hints will allow you to peg where the write region is placed

Nnickchomey I'm confused about the db size limits. The blog says in the pricing that 1GB is ...

Isaac McFadyen•5/19/23, 6:28 PM

Yes. Limit is still 100mb, will be raised to 1GB, and then likely further eventually.

Ttouhidurrr 🇵🇸i want to access d1 not from workers but from api, is that possible? if yes then...

Isaac McFadyen•5/19/23, 6:28 PM

Yes it's possible but undocumented as far as I know (and the API is slower than D1 from a Worker)

Nnickchomey I'm confused about the db size limits. The blog says in the pricing that 1GB is ...

Unsmart•5/19/23, 6:28 PM

Currently in alpha it is free to use and limited to 100mb. In the future those payment plans will come into play and the limits for dbs will be higher

Isaac McFadyen•5/19/23, 6:28 PM

You can look at how I'm doing it here: https://github.com/isaac-mcfadyen/d1-console

Isaac McFadyen•5/19/23, 6:28 PM

(think those endpoints should still work)

UUnsmart Currently in alpha it is free to use and limited to 100mb. In the future those p...

nickchomey•5/19/23, 6:33 PM

Thanks! Very exciting stuff

DDong Another use case is that I have multiple applications needs to fetch data from t...

Matt Silverlock•5/19/23, 7:26 PM

Yep, you can bind to a D1 DB from multiple Workers, just like Queues/KV/R2/etc.

JJason Hostetter Trying to load my alpha db dump into the new backend via wrangler, with 27k or s...

Jason HostetterOP•5/19/23, 7:46 PM

Looks like it was one of the statements in my dumped/converted .sql file that was inserting a very long binary blob and this was throwing the error. If I skipped that insert statement, the rest of it imported alright

Murder Chicken•5/19/23, 8:04 PM

Any ideas what might be causing this or a way to debug this issue a bit more? https://github.com/cloudflare/workers-sdk/issues/3294

GitHub

🐛 BUG: wrangler pages dev does not work with D1 · Issue #3294 · clo...

Which Cloudflare product(s) does this pertain to? Wrangler What version of Wrangler are you using? 3.0.0 What operating system are you using? Windows Describe the Bug create a fresh D1 local DB wit...

JJason Hostetter Looks like it was one of the statements in my dumped/converted .sql file that wa...

Matt Silverlock•5/19/23, 8:05 PM

How long? https://developers.cloudflare.com/d1/platform/limits/

I bet you hit the max row length with a big blob. Would be helpful to know so we can think about adjusting over time.

MMatt Silverlock How long? https://developers.cloudflare.com/d1/platform/limits/ - I bet you hit ...

Jason HostetterOP•5/19/23, 8:14 PM

267k characters

Jason HostetterOP•5/19/23, 8:16 PM

oh yeah i see the max statement length is 100kb

Jason HostetterOP•5/19/23, 8:17 PM

It's interesting that it let me insert that row initially from within the worker, but not via wrangler or the console

Original message was deleted

Matt Silverlock•5/19/23, 8:52 PM

Right now, deletes are treated as writes. Subject to change. Unless you're deleting a LOT of data, all the time, I'd be very surprised if your write+delete costs on D1 exceeded DO.
No limit on tables, but any table is non-zero bytes (to store the schema and allocate storage for an empty table). If an empty table is ~4KB or so you could have thousands of empty tables with no data, if you really desired to do so

Mozzy•5/19/23, 9:17 PM

Damn, I'm not even able to return a single column now :(
SELECT date FROM outputsSELECT date FROM outputs

MMozzy Damn, I'm not even able to return a single column now :( `SELECT date FROM outpu...

Mozzy•5/19/23, 11:30 PM

Did the sensible thing and stopped relying on the bigger endpoints and instead created a few more specialized ones

Unsmart•5/20/23, 1:35 AM

There will be some limit which right now is on the lower end but they are working on increasing it... which I assume the new backend has increased that a good amount given what I read from the blog

Uuser6251 Thanks for the clarification! I am indeed surprised how competitive it is priced...

Isaac McFadyen•5/20/23, 2:18 AM

DO has somewhat different goals than D1 so I expect they will continue to exist.

Isaac McFadyen•5/20/23, 2:19 AM

DO = global "actor" model, there will always be one unique actor per ID/name in the world, that actor has access to key-value state
D1 = a "database", with tables/rows/columns etc.

Unsmart•5/20/23, 4:17 AM

Its friday night and this is an alpha product so I doubt anything will be done until monday when they are back at work.

Mozzy•5/20/23, 10:14 AM

Technically, Wrangler calls the new D1 a Beta

Trying to load my alpha db dump into the new backend via wrangler, with 27k or so lines, first just

Similar Threads

Similar Threads

Similar Threads