Hi! Quick question: How can I create (and download) a backup with wrangler? Using `wrangler d1 backu

I

With wrangler v3,

--persist-to=./path-to-persist-db

--persist-to=./path-to-persist-db

--persist-to=./path-to-persist-db

--persist-to=./path-to-persist-db is not working as expected.

I don't see any directory being populated with the sqlite DB

CCretezy Hi! Quick question: How can I create (and download) a backup with wrangler? Usin...

M

Matt Silverlock•8/22/23, 6:49 AM

There is no support for downloading backups at this point. You don’t need to with Time Travel. https://developers.cloudflare.com/d1/learning/time-travel/

Time Travel and backups · Cloudflare D1 docs

Time Travel is D1’s approach to backups and point-in-time-recovery, and allows you to restore a database to any minute within the last 30 days.

M

Mikyari•8/22/23, 8:13 AM

When I try to backup my db manually, I got this error. Do I have to submit the request to enable this feature?
X [ERROR] A request to the Cloudflare API (/accounts/c3b90596442e73b936440cef3f7895f8/d1/database/ebe13d38-cbc0-465f-a1dd-0f533ea1aeed/backup) failed.

The request is malformed: Experimental DBs do not support backups yet:
ebe13d38-cbc0-465f-a1dd-0f533ea1aeed (v3) [code: 7400]

MMikyari When I try to backup my db manually, I got this error. Do I have to submit the r...

H

Hello, I’m Allie!•8/22/23, 8:17 AM

No, it means that this feature just doesn’t exist yet, so it can’t be enabled

M

Mikyari•8/22/23, 8:18 AM

thank you

M

Mikyari•8/22/23, 8:21 AM

I get stuck when I tried to input real number on the web UI.

M

Mikyari•8/22/23, 8:22 AM

Even though I set the column to

REAL

REAL

REAL

REAL value

MMikyari I get stuck when I tried to input real number on the web UI.

M

Max (@rozenmd)•8/22/23, 8:25 AM

looks like a bug, will investigate

F

fork•8/22/23, 10:37 AM

Thanks for great product, I've done some searches in here but can't get to bottom of what is likely pricing structure going forward, I've seen one based on pure rows read/written and another based on 4kb/1kb blocks read/written. Which is the model planned to take forward? Or are they the same model? Best

Ffork Thanks for great product, I've done some searches in here but can't get to botto...

M

Max (@rozenmd)•8/22/23, 10:38 AM

https://developers.cloudflare.com/d1/platform/pricing/#billing-metrics

Pricing · Cloudflare D1 docs

While in public Alpha, D1 is currently free to use on all Workers plans. Refer to the our recent announcement ) for more information.

M

Max (@rozenmd)•8/22/23, 10:39 AM

you'll be able to see how many rows your database reads/writes in the metrics tab in the dashboard very soon

MMax (@rozenmd)https://developers.cloudflare.com/d1/platform/pricing/#billing-metrics

F

fork•8/22/23, 10:40 AM

Thanks @rozenmd, does that mean a structure that makes use of the rich json querying may fare better than breaking more of the data into rows? For example, logging timestamps or user logins may be more affordable in a single text column in json format, then a table with rows marking timestamps?

M

Max (@rozenmd)•8/22/23, 10:42 AM

it may, but I wonder if performance would suffer vs using a regular table with an index

MMax (@rozenmd)it may, but I wonder if performance would suffer vs using a regular table with a...

F

fork•8/22/23, 10:43 AM

Many thanks, seems like a very clear pricing model. Do you have a sense of how likely it will be you stick with it through to production?

Ffork Many thanks, seems like a very clear pricing model. Do you have a sense of how l...

M

Max (@rozenmd)•8/22/23, 10:45 AM

can't say/don't know tbh, but we're aiming to be affordable w/ very clear pricing, so would iterate towards that goal if we did change

F

fork•8/22/23, 10:46 AM

(You might want to update either this blog post "D1: We turned it up to 11" or the pricing page header as it seems to suggest the blog post has the more up-to-date information)

Ffork (You might want to update either this blog post "D1: We turned it up to 11" or t...

M

Max (@rozenmd)•8/22/23, 10:49 AM

fair point, will do

N

Nat•8/22/23, 1:22 PM

I'm trying to use d1 in my worker but something about adding the wrangler config totally broke my

wrangler dev

wrangler dev

:

⎔ Starting local server...
workerd/server/workerd-api.c++:615: error: wrapped binding module can't be resolved (internal modules only); moduleName = cloudflare-internal:d1-api
workerd/util/symbolizer.c++:99: warning: Not symbolizing stack traces because $LLVM_SYMBOLIZER is not set. To symbolize stack traces, set $LLVM_SYMBOLIZER to the location of the llvm-symbolizer binary. When running tests under bazel, use `--test_env=LLVM_SYMBOLIZER=<path>`.
workerd/jsg/util.c++:275: error: e = workerd/server/workerd-api.c++:635: failed: expected !value.IsEmpty(); global did not produce v8::Value
stack: 104c8d3cf 104f000cf 104c55d6f 104c5ad4f 104c5d717 104c5c397 104c4fe87 104c4fc23 104c4fa8f 104c4fa77 106562367 1065625c3 106560d1b 106560aeb 104c441ab 1a8577f27; sentryErrorContext = jsgInternalError
service core:user:<MY_PROJECT_NAME>: Uncaught Error: internal error
✘ [ERROR] MiniflareCoreError [ERR_RUNTIME_FAILURE]: The Workers runtime failed to start. There is likely additional logging output above.

⎔ Starting local server...
workerd/server/workerd-api.c++:615: error: wrapped binding module can't be resolved (internal modules only); moduleName = cloudflare-internal:d1-api
workerd/util/symbolizer.c++:99: warning: Not symbolizing stack traces because $LLVM_SYMBOLIZER is not set. To symbolize stack traces, set $LLVM_SYMBOLIZER to the location of the llvm-symbolizer binary. When running tests under bazel, use `--test_env=LLVM_SYMBOLIZER=<path>`.
workerd/jsg/util.c++:275: error: e = workerd/server/workerd-api.c++:635: failed: expected !value.IsEmpty(); global did not produce v8::Value
stack: 104c8d3cf 104f000cf 104c55d6f 104c5ad4f 104c5d717 104c5c397 104c4fe87 104c4fc23 104c4fa8f 104c4fa77 106562367 1065625c3 106560d1b 106560aeb 104c441ab 1a8577f27; sentryErrorContext = jsgInternalError
service core:user:<MY_PROJECT_NAME>: Uncaught Error: internal error
✘ [ERROR] MiniflareCoreError [ERR_RUNTIME_FAILURE]: The Workers runtime failed to start. There is likely additional logging output above.

⎔ Starting local server...
workerd/server/workerd-api.c++:615: error: wrapped binding module can't be resolved (internal modules only); moduleName = cloudflare-internal:d1-api
workerd/util/symbolizer.c++:99: warning: Not symbolizing stack traces because $LLVM_SYMBOLIZER is not set. To symbolize stack traces, set $LLVM_SYMBOLIZER to the location of the llvm-symbolizer binary. When running tests under bazel, use `--test_env=LLVM_SYMBOLIZER=<path>`.
workerd/jsg/util.c++:275: error: e = workerd/server/workerd-api.c++:635: failed: expected !value.IsEmpty(); global did not produce v8::Value
stack: 104c8d3cf 104f000cf 104c55d6f 104c5ad4f 104c5d717 104c5c397 104c4fe87 104c4fc23 104c4fa8f 104c4fa77 106562367 1065625c3 106560d1b 106560aeb 104c441ab 1a8577f27; sentryErrorContext = jsgInternalError
service core:user:<MY_PROJECT_NAME>: Uncaught Error: internal error
✘ [ERROR] MiniflareCoreError [ERR_RUNTIME_FAILURE]: The Workers runtime failed to start. There is likely additional logging output above.

⎔ Starting local server...
workerd/server/workerd-api.c++:615: error: wrapped binding module can't be resolved (internal modules only); moduleName = cloudflare-internal:d1-api
workerd/util/symbolizer.c++:99: warning: Not symbolizing stack traces because $LLVM_SYMBOLIZER is not set. To symbolize stack traces, set $LLVM_SYMBOLIZER to the location of the llvm-symbolizer binary. When running tests under bazel, use `--test_env=LLVM_SYMBOLIZER=<path>`.
workerd/jsg/util.c++:275: error: e = workerd/server/workerd-api.c++:635: failed: expected !value.IsEmpty(); global did not produce v8::Value
stack: 104c8d3cf 104f000cf 104c55d6f 104c5ad4f 104c5d717 104c5c397 104c4fe87 104c4fc23 104c4fa8f 104c4fa77 106562367 1065625c3 106560d1b 106560aeb 104c441ab 1a8577f27; sentryErrorContext = jsgInternalError
service core:user:<MY_PROJECT_NAME>: Uncaught Error: internal error
✘ [ERROR] MiniflareCoreError [ERR_RUNTIME_FAILURE]: The Workers runtime failed to start. There is likely additional logging output above.

The wrangler.toml config seems pretty straightforward:

[[d1_databases]]
  binding = "DB" # i.e. available in your Worker on env.DB
  database_name = "REDACTED"
  database_id = "REDACTED"

[[d1_databases]]
  binding = "DB" # i.e. available in your Worker on env.DB
  database_name = "REDACTED"
  database_id = "REDACTED"

[[d1_databases]]
  binding = "DB" # i.e. available in your Worker on env.DB
  database_name = "REDACTED"
  database_id = "REDACTED"

[[d1_databases]]
  binding = "DB" # i.e. available in your Worker on env.DB
  database_name = "REDACTED"
  database_id = "REDACTED"

Could someone help me sort this out?

N

Nat•8/22/23, 1:36 PM

Looks like ^ happens when you try to add a d1 config to your

wrangler.toml

wrangler.toml

but you're still using the old ServiceWorker format.

wrangler deploy

wrangler deploy

gave a much better error message that helped me sort this out. But maybe y'all can improve that error message on

wrangler dev

wrangler dev

MMatt Silverlock There is no support for downloading backups at this point. You don’t need to wit...

C

CretezyOP•8/22/23, 2:05 PM

I think it's very important to be able to export data out of 3rd parties. I strongly believe backups/exports should be implemented

CCretezy I think it's _very_ important to be able to export data out of 3rd parties. I st...

H

Hello, I’m Allie!•8/22/23, 2:07 PM

Note too that afaik this is a planned feature at some point, it just isn’t built yet.

C

CretezyOP•8/22/23, 2:07 PM

Thanks, that makes sense.

C

CretezyOP•8/23/23, 4:17 AM

When will D1 be accessible through the Cloudflare API?
What would be the best way to manually implement exporting/backup?

C

CretezyOP•8/23/23, 4:29 AM

Is appears probably using dump and a custom endpoint? https://developers.cloudflare.com/d1/platform/client-api/#await-dbdump

D1 client API · Cloudflare D1 docs

As part of our Client API, both static and prepared statements are supported. Best practice is to use prepared statements which are precompiled …

CCretezy When will D1 be accessible through the Cloudflare API? What would be the best wa...

C

chientrm.com•8/23/23, 4:30 AM

you can use this package to access D1 from outside: https://github.com/chientrm/wrangler-proxy

TTom Hi there ! I've made a url shortening service using workers & a D1 database. I...

M

Max (@rozenmd)•8/23/23, 3:26 PM

you could create a user-land backup using a script like d1-database

M

Max (@rozenmd)•8/23/23, 3:29 PM

otherwise, you can time travel via the D1 console in Cloudflare's dashboard:

TTom Hi there ! I've made a url shortening service using workers & a D1 database. I...

D

Dahoom•8/23/23, 10:40 PM

I recommend using this repo by Nora soderlund, it's amazing and I'm using it myself https://github.com/nora-soderlund/cloudflare-d1-backups

GitHub

GitHub - nora-soderlund/cloudflare-d1-backups: A package to dump a ...

A package to dump a copy of your experimental D1 database through a worker. - GitHub - nora-soderlund/cloudflare-d1-backups: A package to dump a copy of your experimental D1 database through a worker.

D

Dahoom•8/23/23, 10:44 PM

It saves the db as an sql file which can be set to a cron job and you can easily input the created file in wrangler and reset it as shown in the md of the repo

B

Ben-xD•8/24/23, 8:13 AM

I'm trying to run multiple migration/sql files on a D1 database. Is there a way I can have access to the D1 binding from inside a script (so I can read sql files and execute sql into D1)? Without being a "module worker" or "service worker"

M

Mikyari•8/24/23, 8:23 AM

Is procedure is available to use in D1 for now?

M

Mikyari•8/24/23, 8:23 AM

I did not see it in docs, but only this article: https://blog.cloudflare.com/whats-new-with-d1/

The Cloudflare Blog

D1: our quest to simplify databases

Get an inside look on the D1 experience today, what the team is currently working on and what’s coming up!

BBen-xD I'm trying to run multiple migration/sql files on a D1 database. Is there a way ...

A

advanced•8/24/23, 8:59 AM

you can try cfw-bindings-wrangler-bridge or the latest version of miniflare with getBindings api

BBen-xD I'm trying to run multiple migration/sql files on a D1 database. Is there a way ...

C

chientrm.com•8/24/23, 9:00 AM

https://github.com/chientrm/wrangler-proxy

Ssourabh.sashank Database Id: 08c3458c-812d-48bf-801d-5a87b4649f49 Not opening in dashboard, con...

M

Max (@rozenmd)•8/24/23, 9:09 AM

looks like an overloaded db using the old backend, would recommend the new backend

Ssourabh.sashank Can't we upgrade the same DB to be using new backend ?

M

Max (@rozenmd)•8/24/23, 10:35 AM

You'll need to export your database: https://developers.cloudflare.com/d1/learning/backups/#downloading-a-backup-locally
and import it into a new database running on the new backend: https://developers.cloudflare.com/d1/learning/importing-data/#import-an-existing-database

J

James•8/25/23, 1:06 AM

What does your worker look like? It has to be in modules format to use D1 from that error: https://developers.cloudflare.com/workers/learning/migrate-to-module-workers/

Migrate to ES modules format · Cloudflare Workers docs

This guide will show you how to migrate your Workers from the Service Worker format to the ES modules format.

Hi! Quick question: How can I create (and download) a backup with wrangler? Using `wrangler d1 backu

Similar Threads

Similar Threads

Similar Threads