`wrangler d1 export`

wrangler d1 export

wrangler d1 export

kianOP•5/1/24, 4:03 PM

https://developers.cloudflare.com/d1/build-with-d1/import-data/#export-an-existing-d1-database

ajgeiss0702•5/1/24, 4:22 PM

Was I imagining things or did the blog post talking about D1 replication and sessions used to say that the API is available today and will use the old model and will automatically upgrade when replication is released?

I thought i read that when it was released, and sat down to look into it now, and I dont see it. It's very possible I just misread it the first time

kianOP•5/1/24, 4:56 PM

https://blog.cloudflare.com/building-d1-a-global-database is the blog you're thinking of but the Sessions API isn't a thing yet & there are no plans to make the existing API behave differently.

Since the Sessions API changes D1's consistency model, developers must opt-in to the new API. Existing D1 API methods are unchanged and will still have the same snapshot isolation consistency model as before. However, only queries made using the new Sessions API will use replicas.

ajgeiss0702•5/1/24, 5:04 PM

Yeah, I was searching through that again. I could have sworn it said something about how you can write code for sessions today and theyll fall back to the old behaviour, then automatically use the new behaviour once its available (and the old api would remain the same)

Vlady•5/1/24, 7:27 PM

Here goes nothing

VVlady Here goes nothing 👀 🤞🏼

Frank•5/1/24, 9:09 PM

nice. i assume we expect the 10gb or 50k db limit to increase sooner than later

FFrank nice. i assume we expect the 10gb or 50k db limit to increase sooner than later

Chaika•5/1/24, 9:11 PM

The DB limit is soft, the size per db limit is hard

Chaika•5/1/24, 9:12 PM

d1-database

Chaika•5/1/24, 9:12 PM

d1-database

Chaika•5/1/24, 9:13 PM

they've already said they can support virtually limitless (millions) of separate dbs

CChaika they've already said they can support virtually limitless (millions) of separate...

Frank•5/1/24, 9:15 PM

awesome. thanks. just waiting on dynamic binding then.
and be careful about limit bindings per worker at a given time. I think ~5k right now

Frank•5/1/24, 9:16 PM

hopefully bindings can be both created and released dynanmically

FFrank awesome. thanks. just waiting on dynamic binding then. and be careful about limi...

Chaika•5/1/24, 9:26 PM

when I think about dynamic bindings I think about Worker for Platform's Dynamic Dispatch binding, where it's just a singular binding you can use to get access to a worker/instance, ex:

    const worker = env.DISPATCHER.get("customer-worker-1");
    var responseFromUserScript = worker.fetch(req);

    const worker = env.DISPATCHER.get("customer-worker-1");
    var responseFromUserScript = worker.fetch(req);

Chaika•5/1/24, 9:27 PM

I don't think they've said much about dynamic bindings, but I imagine it'd be something like that where you've got maybe an entry binding (or maybe no entry binding at all and it's just part of the runtime/env object, all dynamic), but eitherway each binding wouldn't become part of your env or stick around, except for during that request

Frank•5/1/24, 9:32 PM

any idea how migrations work when u have so many dbs

Rubi•5/2/24, 12:38 AM

basically a Go script to execute .sql concurrently on d1 rest api

Kkian `wrangler d1 export`

Softbear•5/2/24, 5:02 AM

Missed that news

CChaika when I think about dynamic bindings I think about Worker for Platform's Dynamic ...

Beny•5/2/24, 7:12 AM

I created something like this with Worker for Platforms. It uploads a new script with the binding that is essentially a D1 proxy over RPC. The only downside is that it can't be used immediately because the dispatcher doesn't think the script is valid straight away. The only solution I could get to work was implementing a delay of a few seconds before the script is usable. Apart from that it works very well

Sam•5/2/24, 8:12 PM

Hey all I wanted to echo https://community.cloudflare.com/t/different-errors-on-d1-randomly-happening-once-in-a-while/599421 -- Every few days now I get a "D1_ERROR: D1 storage operation exceeded timeout which caused object to be reset."

Is this a known issue? Very limited results on google & none in Discord

(fdb7837214fdd80e018b0bf3ae6355c9fdb7837214fdd80e018b0bf3ae6355c9 - <redacted>-nx-production<redacted>-nx-production)

malone•5/2/24, 9:11 PM

I'm a bit confused with the upgrade process. When i ran wrangler d1 info, it said 'beta' instead of alpha and I didn't do anything yet.

malone•5/2/24, 9:11 PM

When i updated wrangler the version has been removed from the table entirely

malone•5/2/24, 9:11 PM

malone•5/2/24, 9:12 PM

with a brand new db i just created it also says beta

Unsmart•5/2/24, 9:13 PM

an alpha db would say alpha

Unsmart•5/2/24, 9:13 PM

beta is the most up to date version

malone•5/2/24, 9:13 PM

ok cool! so nothing to worry about. thanks @Unsmart | Tech debt

ALECKS•5/3/24, 12:26 AM

Is it possible to connect to a D1 database programmatically, from a remote host? For ex, I want to run a js script on my machine that connects to a D1 instance and does some stuff. I want to do this with JS, instead of .sql files, which I know I can run through wrangler d1 execute

Original message was deleted

forslunds•5/3/24, 3:05 AM

The local is a sqlite file inside wrangler folder. Open it in a sqlite program and export

stibbs•5/3/24, 3:35 AM

Apologies if this has been asked before, but is there a way to run down migrations? Or is it advised to:
1) create a new up migration to unwind a change [I am assuming this is the answer]; or
2) revert to prior state via time travel?

Frank•5/3/24, 4:59 AM

https://alexgarcia.xyz/blog/2024/building-new-vector-search-sqlite/index.html
pretty cool if this works well and eventually lands in D1 land

I'm writing a new vector search SQLite Extension

sqlite-vec is an new vector search SQLite extension, coming soon!

Kysan•5/3/24, 6:28 PM

https://developers.cloudflare.com/api/operations/cloudflare-d1-query-database
what is the rate limit for the http api endpoint ?

Cloudflare API Documentation

Interact with Cloudflare's products and services via the Cloudflare API

Kysan•5/3/24, 6:29 PM

1200 requests per 5 minutes ty

KKysan 1200 requests per 5 minutes ty

Chaika•5/3/24, 6:32 PM

yea and that's shared among all endpoints/dashboard/etc for your user, some endpoints like purge by url have special ratelimits as well, but d1 doesn't

Chaika•5/3/24, 6:33 PM

?d1-remote-connections

Flare•5/3/24, 6:33 PM

Accessing D1 from outside workers:

D1 is based on SQLite, which does not natively support remote connections and isn't MySQL/PostgreSQL wire compatible, nor does D1 itself offer that functionality. As such, your options are:

1 -> The Cloudflare API Query Endpoint (https://developers.cloudflare.com/api/operations/cloudflare-d1-query-database), with the upside of needing less setup, all requests have to go back to Cloudflare's core, so request latency is far from optimal from some locations. You would also be subject to the API Rate limit of 1200 requests per 5 minutes.

2 -> Create a worker to proxy requests to your D1 Database. Either tightly bound to your application/schema or loose and just accepting any query. This would allow your worker to execute potentially closer to your D1 Database and avoid the api rate limit.

Miyagi•5/3/24, 8:18 PM

Hello, I have had success with the D1 in development all the way until production deployment. I have bound it using the updated syntax from docs (Pages). When i attempt to query when deployed, It crashes with Application Error.

Works just fine in wrangler preview.

AccountId: 77e1b1d919d86d253e4967517aab4437

Steved•5/3/24, 10:11 PM

Does anyone have an example of storing binary data in D1 using Drizzle?
Drizzle seems to be expecting a 'Buffer' type, but when I pass one in I'm getting an assert in Miniflare where it isn't expecting a Buffer in ArrayBufferView (ALLOWED_ARRAY_BUFFER_VIEW_CONSTRUCTORS). It's expecting one of the typed array names. At this point I can't tell whether the issue is in Drizzle or Miniflare (or devalue package).

Steved•5/3/24, 10:19 PM

If I ignore types and just pass a Uint8Array through it works.

skip•5/3/24, 10:51 PM

I am having trouble running my migrations on Prod. Works file locally, and I can manually copy and paste them into the console fine. But when I try "wrangler d1 migrations apply xxx" I get "No migrations to apply!" even though it's a fresh database.

I can see there is a table "sqlite_schema". Does this track what migrations have been applied so far?

I thought I'd try to rename that table, and see if that resets my migration point to see if it starts fresh - but I get an error "table sqlite_master may not be modified"

Sskip 👋 I am having trouble running my migrations on Prod. Works file locally, and I...

Isaac McFadyen•5/3/24, 11:09 PM

sqlite_schema is unrelated to migrations, it's an internal table SQLite uses for various things.

IIsaac McFadyen sqlite_schema is unrelated to migrations, it's an internal table SQLite uses for...

skip•5/3/24, 11:16 PM

OK thanks... would you know how migrations decide what migration point has or has not run yet?

Because ATM my database it totally empty and it thinks none of the migrations need to run

Isaac McFadyen•5/3/24, 11:17 PM

I'm not sure, sorry - someone else would probably know the answer.

stibbs•5/4/24, 4:46 AM

How do I use sqlite extensions with D1? E.g., uuid.cuuid.c (https://sqlite.org/src/file/ext/misc/uuid.c) to generate UUIDs

EDIT: Or should I be creating the UUID's myself prior to interacting with D1?

kianOP•5/4/24, 5:16 AM

If the extension isn't already enabled in D1 then you can't use it - so you'll need to make the UUIDs yourself, i.e with crypto.randomUUID()

Bilal B•5/4/24, 6:15 AM

Hey I'm trying to run SQL commands from consoleconsole tab in the interface but it says I'm not authorized

Bilal B•5/4/24, 6:15 AM

I'm the owner of the account

`wrangler d1 export`

Similar Threads

Similar Threads

Similar Threads