Cloudflare Developers•3y ago

https://developers.cloudflare.com/workers/wrangler/commands/#backup-download

Commands - Wrangler · Cloudflare Workers docs

Wrangler offers a number of commands to manage your Cloudflare Workers.

captainOP•10/5/23, 4:20 PM

this also seems to be out of date

captainOP•10/5/23, 4:22 PM

@Matt you around?

Dahoom•10/5/23, 4:22 PM

dont @ the devs, they have a lot going on as is

Ccaptain https://developers.cloudflare.com/workers/wrangler/commands/#backup-download

Dahoom•10/5/23, 4:23 PM

theres no current official solution i dont think but for now you can use this https://github.com/nora-soderlund/cloudflare-d1-backups

GitHub

GitHub - nora-soderlund/cloudflare-d1-backups: A package to dump a ...

A package to dump a copy of your experimental D1 database through a worker. - GitHub - nora-soderlund/cloudflare-d1-backups: A package to dump a copy of your experimental D1 database through a worker.

Dahoom•10/5/23, 4:25 PM

I've been using it for a while and honestly i prefer this method since I can basically mould it into exactly what I would want from a backup system

Dahoom•10/5/23, 4:25 PM

this is an ad sponsored by @nora

DDahoom theres no current official solution i dont think but for now you can use this ht...

captainOP•10/5/23, 4:28 PM

thanks, and yikes...at a minimum probably should be added all the documentation links I referenced that there is no replacement solution to download a back up for beta from either command line or dashboard...i can't be the first person that circled around and around this for an hour

Dahoom•10/5/23, 4:29 PM

yeah trust me they are making a lot of changes to the docs recently and sometimes the miss some legacy aspects

Original message was deleted

Dahoom•10/5/23, 4:29 PM

exactly what i was about to say

DDahoom exactly what i was about to say

captainOP•10/5/23, 4:40 PM

https://github.com/cloudflare/cloudflare-docs/pull/11108

GitHub

Update commands.md to reflect the ability to "Download existing dat...

…to your local machine" not functioning with a beta database
Currently there is not an option to "Download existing data to your local machine" for a beta database. The documentation...

Ccaptain https://github.com/cloudflare/cloudflare-docs/pull/11108

Dahoom•10/5/23, 4:43 PM

I dont work at cloudflare, there's dev's working at Cloudflare specifically on the docs that will approve your pull request :)

dave•10/5/23, 5:18 PM

Error: D1_ERROR: D1 storage operation exceeded timeout which caused object to be reset.

dave•10/5/23, 5:18 PM

oh no

dave•10/5/23, 6:24 PM

I forget, do D1 requests count towards the overall Worker subrequest limit?

Original message was deleted

dave•10/5/23, 6:25 PM

Did you find an answer for this?

Ddave I forget, do D1 requests count towards the overall Worker subrequest limit?

Matt Silverlock•10/5/23, 6:27 PM

Yes

MMatt Silverlock Yes

dave•10/5/23, 6:30 PM

thanks! And

.batch()

.batch()

only counts as one subrequest I assume, regardless of the number of statements? https://developers.cloudflare.com/d1/platform/client-api/#dbbatch

Unsmart•10/5/23, 6:59 PM

The "new" backend which has been the default and only really supported option for quite some time now limits depths of queries to 20. You basically need to create queries that do not go over that limit.

There was a github issue about it you can see here if you want to add your example and request for this limit to be upped in the future: https://github.com/cloudflare/workers-sdk/issues/4094

Unsmart•10/5/23, 7:15 PM

I have no clue I do not use SQL databases and have no knowledge of query depths or anything

SSPS | Shootmail `Expression tree is too large (maximum depth 20)` I deleted my old dB that was c...

Jason Hostetter•10/6/23, 1:46 AM

I hit this same error a while back, and after a lot of digging I could not for the life of me figure out how expression depth was calculated, whether my query had a depth of 21 or 121, or any useful information about how to decrease depth. I ended up splitting the query into two separate queries instead of using a common table expression (an ordinary CTE, not a recursive one even), and the error went away. Cloudflare devs at the time sounded open to increasing the max depth (default sqlite is set at like 1000 or something), but again I couldn't say what number I would need! So all that to say...

Jason Hostetter•10/6/23, 1:51 AM

Just saw the comment from @Unsmart | Tech debt and the github issue after writing all that lol. Really wish the sqlite error also reported the query expression depth.

Hữu Phong•10/6/23, 4:13 AM

Is there anyway to use D1 with TablePlus?

Cretezy•10/6/23, 4:53 AM

It should be noted in the documentation for the client API (https://developers.cloudflare.com/d1/platform/client-api) how

success

success

is handled.
It's returned from

run

run

batch

batch

, but

first

first

all

all

don't have it

D1 client API · Cloudflare D1 docs

As part of our Client API, both static and prepared statements are supported. Best practice is to use prepared statements which are precompiled …

Cretezy•10/6/23, 4:54 AM

Also seems like

success: true

success: true

is forced in the typings, so presumably it's not even useful?

Cretezy•10/6/23, 6:03 AM

Also, are

batch

batch

ran sequencially? Would it work if I read a row in the 1st batched statement, but delete the same row in the 2nd statement?

CCretezy Also seems like `success: true` is forced in the typings, so presumably it's not...

Max (@rozenmd)•10/6/23, 8:16 AM

  success: boolean, // true if the operation was successful, false otherwise

  success: boolean, // true if the operation was successful, false otherwise

MMatt Silverlock Yes, if you are using `—remote` mode in wrangler

Adi//•10/6/23, 9:24 AM

is this solved now?

AAdi//is this solved now?

Matt Silverlock•10/6/23, 9:40 AM

Yes, the incident you were asking about has been closed since yesterday

MMatt Silverlock Yes, the incident you were asking about has been closed since yesterday

Adi//•10/6/23, 9:44 AM

ok ok, lol i was trying to work on a cron worker at the same time and --remote kept failing

good to know there was no mistake on my end

SSPS | Shootmail https://github.com/cloudflare/workerd/pull/1279 Is this being merged today to fi...

Matt Silverlock•10/6/23, 10:02 AM

To be transparent: no. Also merging is not the same as releasing. Hold tight.

SSPS | Shootmail Any expected date, so that i can decide my next steps. I cannot move forward wit...

Matt Silverlock•10/6/23, 10:15 AM

Our goal is within the next week. I strongly suggest you refactor how you are performing your joins in the meantime if you need to unblock yourself. The current limit isn’t high but we’ve rarely had folks run into it so directly.

VVikash The database is for testing, feel free to takeover if you want to test? Or shoul...

Vikash•10/6/23, 10:20 AM

@rozenmd Any suggestion on 1 second slowness in all queries?

VVikash @rozenmd Any suggestion on 1 second slowness in all queries?

Matt Silverlock•10/6/23, 11:37 AM

You should not expect D1 to be outright faster than a hot (cached) KV read. How are you measuring query performance? What is the duration of the query itself, as reported in

result.meta.duration

result.meta.duration

and then the time for the D1 API call without your application code?

MMatt Silverlock You should not expect D1 to be outright faster than a hot (cached) KV read. How ...

Vikash•10/6/23, 11:46 AM

Yes, I am not considering that. Here is response on a worker where duration is : 0.689.
I guess that's more half second.

"success":true,"meta":{"served_by":"v3-prod","duration":0.689,"changes":0,"last_row_id":0,"changed_db":false,"size_after":81920,"rows_read":43,"rows_written":0},"results": [....]

"success":true,"meta":{"served_by":"v3-prod","duration":0.689,"changes":0,"last_row_id":0,"changed_db":false,"size_after":81920,"rows_read":43,"rows_written":0},"results": [....]

https://development.pricetree-angular.pages.dev/api/products

Vikash•10/6/23, 11:48 AM

And some time even more than 1 second -

Kuririn•10/6/23, 12:01 PM

I have query like this:

const response = await env.DB.prepare(`
                UPDATE Promotions 
                SET content = ?, last_update_time = ?
                WHERE promo_id = ?
            `).bind(JSON.stringify(page), currentTimeInSeconds, page.id).run();

const response = await env.DB.prepare(`
                UPDATE Promotions 
                SET content = ?, last_update_time = ?
                WHERE promo_id = ?
            `).bind(JSON.stringify(page), currentTimeInSeconds, page.id).run();

but it doesn't update the record and doesn’t return anything or throw an error.

My only idea is that page is too big, but something similar works for insert.
But it would be weird that I can't update big values (article).

VVikash And some time even more than 1 second -

Matt Silverlock•10/6/23, 12:19 PM

Duration is in milliseconds

KKuririn I have query like this: ```const response = await env.DB.prepare(` ...

Matt Silverlock•10/6/23, 12:19 PM

Try it without calling stringify, and just pass in a plain value first

MMatt Silverlock Try it without calling stringify, and just pass in a plain value first

Vikash•10/6/23, 12:21 PM

I am using Hono

.json()

.json()

, I will test without that -

products.get('/', async (c) => {
const { offset = 0, limit = 1000 } = c.req.query();
const results = await getProducts(c, Number(offset), Number(limit));
return c.json(results as Product[]);
});

VVikash I am using Hono `.json()`, I will test without that - products.get('/', async...

Matt Silverlock•10/6/23, 12:25 PM

I replied to the other user

MMatt Silverlock I replied to the other user 🙂

Vikash•10/6/23, 12:26 PM

MMatt Silverlock Try it without calling stringify, and just pass in a plain value first

Kuririn•10/6/23, 12:55 PM

It doesn’t work without stringify, but I noticed it worked once finally (after many trials). It was probably cloudflare issue (didn't execute code and didn't return the promise). I will test it again.

const response = await env.DB.prepare(` UPDATE Promotions SET content = ?, last_update_time = ? WHERE promo_id = ? `).bind(JSON.stringify(page), currentTimeInSeconds, page.id).run();

https://developers.cloudflare.com/workers/wrangler/commands/#backup-download

Similar Threads