Heya, I want to give a data-curious team mate access to our D1. Is there an option to generate a rea

Ggregor Heya, I want to give a data-curious team mate access to our D1. Is there an opti...

M

nothing like that yet, but you could run

wrangler d1 export

wrangler d1 export

wrangler d1 export

wrangler d1 export on a newer version of wrangler and send them the sql?

MMax (@rozenmd)nothing like that yet, but you could run `wrangler d1 export` on a newer version...

G

gregorOP•5/27/24, 3:38 PM

Thanks, I might do that. I wanted him to be able to do it on new data, even if I'm not around. So I guess I'll just teach him how to wrangle

B

benjamin•5/27/24, 6:43 PM

I've been getting a lot a errors using wrangler and D1 the past couple days. I've signed out and in multiple times with the CLI. Occasionally commands work but mostly I'm getting this:

wrangler d1 list

✘ [ERROR] A request to the Cloudflare API (/accounts/xxxxx/d1/database) failed.

  Authentication error [code: 10000]

wrangler d1 list

✘ [ERROR] A request to the Cloudflare API (/accounts/xxxxx/d1/database) failed.

  Authentication error [code: 10000]

wrangler d1 list

✘ [ERROR] A request to the Cloudflare API (/accounts/xxxxx/d1/database) failed.

  Authentication error [code: 10000]

wrangler d1 list

✘ [ERROR] A request to the Cloudflare API (/accounts/xxxxx/d1/database) failed.

  Authentication error [code: 10000]

B

benjamin•5/27/24, 6:44 PM

with wrangler 3.57.1. Any ideas or suggestions?

Bbenjamin I've been getting a lot a errors using wrangler and D1 the past couple days. I'v...

M

Max (@rozenmd)•5/27/24, 7:01 PM

I ran into this today,

wrangler logout

wrangler logout

wrangler logout

wrangler logout and

wrangler login

wrangler login

wrangler login

wrangler login worked

Bbenjamin I've been getting a lot a errors using wrangler and D1 the past couple days. I'v...

M

Max (@rozenmd)•5/27/24, 7:03 PM

do any commands work after you log back in? or always failing?

MMax (@rozenmd)do any commands work after you log back in? or always failing?

B

benjamin•5/27/24, 8:15 PM

Thanks, today always failing. Yesterday it would work and then immediately fail. Now failing after login.

Successfully logged in.

✘ [ERROR] A request to the Cloudflare API (/accounts/xxxxx/d1/database) failed.

  Authentication error [code: 10000]


Getting User settings...
👋 You are logged in with an OAuth Token, associated with the email xxxxxxx!
┌──────────────────────────────────────────┬──────────────────────────────────┐
│ Account Name                             │ Account ID                       │
├──────────────────────────────────────────┼──────────────────────────────────┤
│ xxxxxxx's Account │ xxxxxxx │
└──────────────────────────────────────────┴──────────────────────────────────┘
🔓 Token Permissions: If scopes are missing, you may need to logout and re-login.
Scope (Access)
- account (read)
- user (read)
- workers (write)
- workers_kv (write)
- workers_routes (write)
- workers_scripts (write)
- workers_tail (read)
- d1 (write)
- pages (write)
- zone (read)
- ssl_certs (write)
- constellation (write)
- ai (write)
- queues (write)
- offline_access

Successfully logged in.

✘ [ERROR] A request to the Cloudflare API (/accounts/xxxxx/d1/database) failed.

  Authentication error [code: 10000]


Getting User settings...
👋 You are logged in with an OAuth Token, associated with the email xxxxxxx!
┌──────────────────────────────────────────┬──────────────────────────────────┐
│ Account Name                             │ Account ID                       │
├──────────────────────────────────────────┼──────────────────────────────────┤
│ xxxxxxx's Account │ xxxxxxx │
└──────────────────────────────────────────┴──────────────────────────────────┘
🔓 Token Permissions: If scopes are missing, you may need to logout and re-login.
Scope (Access)
- account (read)
- user (read)
- workers (write)
- workers_kv (write)
- workers_routes (write)
- workers_scripts (write)
- workers_tail (read)
- d1 (write)
- pages (write)
- zone (read)
- ssl_certs (write)
- constellation (write)
- ai (write)
- queues (write)
- offline_access

Successfully logged in.

✘ [ERROR] A request to the Cloudflare API (/accounts/xxxxx/d1/database) failed.

  Authentication error [code: 10000]


Getting User settings...
👋 You are logged in with an OAuth Token, associated with the email xxxxxxx!
┌──────────────────────────────────────────┬──────────────────────────────────┐
│ Account Name                             │ Account ID                       │
├──────────────────────────────────────────┼──────────────────────────────────┤
│ xxxxxxx's Account │ xxxxxxx │
└──────────────────────────────────────────┴──────────────────────────────────┘
🔓 Token Permissions: If scopes are missing, you may need to logout and re-login.
Scope (Access)
- account (read)
- user (read)
- workers (write)
- workers_kv (write)
- workers_routes (write)
- workers_scripts (write)
- workers_tail (read)
- d1 (write)
- pages (write)
- zone (read)
- ssl_certs (write)
- constellation (write)
- ai (write)
- queues (write)
- offline_access

Successfully logged in.

✘ [ERROR] A request to the Cloudflare API (/accounts/xxxxx/d1/database) failed.

  Authentication error [code: 10000]


Getting User settings...
👋 You are logged in with an OAuth Token, associated with the email xxxxxxx!
┌──────────────────────────────────────────┬──────────────────────────────────┐
│ Account Name                             │ Account ID                       │
├──────────────────────────────────────────┼──────────────────────────────────┤
│ xxxxxxx's Account │ xxxxxxx │
└──────────────────────────────────────────┴──────────────────────────────────┘
🔓 Token Permissions: If scopes are missing, you may need to logout and re-login.
Scope (Access)
- account (read)
- user (read)
- workers (write)
- workers_kv (write)
- workers_routes (write)
- workers_scripts (write)
- workers_tail (read)
- d1 (write)
- pages (write)
- zone (read)
- ssl_certs (write)
- constellation (write)
- ai (write)
- queues (write)
- offline_access

B

benjamin•5/27/24, 9:08 PM

I created a new project and everything seems to be working fine. My earlier one had an older wranger.toml file and I've updated it. Seems to be working now.

K

kian•5/27/24, 9:32 PM

Did you have an

account_id

account_id

account_id

account_id in your Wrangler.toml?

B

benjamin•5/27/24, 10:23 PM

Yes, was working off an open-source starter that had it there and I updated it with mine.

F

FroggyKnight•5/28/24, 3:18 AM

What's the disadvantage of using one db in D1 over one db per user? Is it just the 10GB limit?

F

FroggyKnight•5/28/24, 3:18 AM

Also what is the cost of a db after the 10 free db's?

M

Mrinank•5/28/24, 5:23 AM

Why d1 reads/writes are cheaper than kv's?

MMrinank Why d1 reads/writes are cheaper than kv's?

M

Max (@rozenmd)•5/28/24, 6:01 AM

D1's read/writes only need to be replicated to a few dbs, and scale is limited (currently) to a single DO instance
while KV can handle millions of RPS and gets replicated globally

FFroggyKnight What's the disadvantage of using one db in D1 over one db per user? Is it just t...

M

Max (@rozenmd)•5/28/24, 6:01 AM

Managing the schema becomes annoying

FFroggyKnight Also what is the cost of a db after the 10 free db's?

M

Max (@rozenmd)•5/28/24, 6:02 AM

we don't charge per DB - only read/writes and storage https://developers.cloudflare.com/d1/platform/pricing/

MMax (@rozenmd)Managing the schema becomes annoying

F

FroggyKnight•5/28/24, 9:44 AM

Why is that?

FFroggyKnight Why is that?

M

Max (@rozenmd)•5/28/24, 10:35 AM

"migrations" currently are just us using the /query endpoint immediately and tracking the progress in your DB

doing that at scale doesn't really work, you don't want to wake up potentially millions of DBs, plus every migration might not succeed across the fleet

M

Max (@rozenmd)•5/28/24, 10:36 AM

so as part of dynamic bindings for D1 we're thinking about how to solve that too

I

imnotgoodatchoosingusernames•5/28/24, 10:56 AM

Is there any reason why the ability to export the entire database has been removed in more recent builds? I appreciate that the Time Travel feature allows 'rollbacks' but is there any intention of allowing full database downloads again in the future? There are a number of use cases this functionality is very useful for

Iimnotgoodatchoosingusernames Is there any reason why the ability to export the entire database has been remov...

M

Max (@rozenmd)•5/28/24, 10:56 AM

what do you mean?

M

Max (@rozenmd)•5/28/24, 10:57 AM

wrangler d1 export

wrangler d1 export

wrangler d1 export

wrangler d1 export hasn't been removed

M

Max (@rozenmd)•5/28/24, 11:00 AM

(there was a time where it didn't exist, but there were 3rd party options on github in the meanwhile)

I

imnotgoodatchoosingusernames•5/28/24, 11:00 AM

Clearly I had not read the docs recently enough. Thanks @Max (@rozenmd) !

MMax (@rozenmd)(there was a time where it didn't exist, but there were 3rd party options on git...

H

Hello, I’m Allie!•5/28/24, 11:01 AM

I forget, is there a documented API call for

export

export

export

export?

HHello, I’m Allie!I forget, is there a documented API call for `export`?

M

Max (@rozenmd)•5/28/24, 11:02 AM

https://developers.cloudflare.com/workers/wrangler/commands/#export

Cloudflare Docs

Commands - Wrangler · Cloudflare Workers docs

Create, develop, and deploy your Cloudflare Workers with Wrangler commands.

MMax (@rozenmd)https://developers.cloudflare.com/workers/wrangler/commands/#export

H

Hello, I’m Allie!•5/28/24, 11:02 AM

Sorry, I meant on the HTTP API

M

Max (@rozenmd)•5/28/24, 11:03 AM

oh, no there isn't

MMax (@rozenmd)oh, no there isn't

H

Hello, I’m Allie!•5/28/24, 11:05 AM

Would D1 support a

wrangler d1 pull

wrangler d1 pull

wrangler d1 pull

wrangler d1 pull command, that

exports

exports

exports

exports the DB into the correct place in the

.wrangler

.wrangler

.wrangler

.wrangler directory so you can use it locally? You can do that manually rn, but it isn't the most ergonomic thing to do

HHello, I’m Allie!Would D1 support a `wrangler d1 pull` command, that `exports` the DB into the co...

C

cosbgn•5/28/24, 11:14 AM

This would be fantastic, I'm currently doing a mix of export + execute but it's not ideal

HHello, I’m Allie!Would D1 support a `wrangler d1 pull` command, that `exports` the DB into the co...

M

Max (@rozenmd)•5/28/24, 11:23 AM

that'd be cool, but atm we don't have a means of pulling the sqlite file out

MMax (@rozenmd)that'd be cool, but atm we don't have a means of pulling the sqlite file out

H

Hello, I’m Allie!•5/28/24, 11:31 AM

It wouldn’t be ideal, but couldn’t you just run the commands against the DB manually?

MMax (@rozenmd)"migrations" currently are just us using the /query endpoint immediately and tra...

F

FroggyKnight•5/28/24, 11:37 AM

I see, so if I wanted to use a single D1 for my app (or maybe a DB per region). I wouldn't see any noticable performance issues in reads/writes?

M

Max (@rozenmd)•5/28/24, 11:39 AM

You'd be capped by DO's single instance limits until we release read replication

MMax (@rozenmd)You'd be capped by DO's single instance limits until we release read replication

F

FroggyKnight•5/28/24, 11:39 AM

What are those limits?

M

Max (@rozenmd)•5/28/24, 11:40 AM

Depends on how heavy your queries are, you're getting a single CPU thread and 128MB RAM - i've seen 1k requests per second go through

MMax (@rozenmd)Depends on how heavy your queries are, you're getting a single CPU thread and 12...

F

FroggyKnight•5/28/24, 11:42 AM

OK great ty, so my 10 user app will scale

F

FroggyKnight•5/28/24, 11:49 AM

So when read replicas release it will actually scale right?

FFroggyKnight So when read replicas release it will actually scale right?

M

Max (@rozenmd)•5/28/24, 12:22 PM

That's the idea - it'll also unlock forking/branching, and exports should get faster

G

gerbyzation•5/28/24, 12:57 PM

Heyhey! Are there any good ways to unit test code that relies on D1 w/o having to test it through the full worker request/response cycle? For example I have a service module that is passed a D1 database as an argument and I want to write unit tests for this?

GGyurmatag Is anybody having issue with time travel? Our instance is displaying this error:...

M

Max (@rozenmd)•5/28/24, 2:04 PM

This issue has been fixed

M

Max (@rozenmd)•5/28/24, 3:12 PM

https://developers.cloudflare.com/d1/build-with-d1/import-data/#known-limitations (i didn't know about this page)

M

Max (@rozenmd)•5/28/24, 3:14 PM

i wouldn't have expected it to output the error message into the dump, will raise an issue

M

Mrinank•5/28/24, 5:07 PM

const memory = new BufferMemory({
    returnMessages: true,
    chatHistory: new CloudflareD1MessageHistory({
    tableName: 'stored_message',
        sessionId: 'example2',
    database: env.DB,
    }),
});

await memory.saveContext({ input: 'My name is TOMER' }, { output: 'Ok got it' });

const memory = new BufferMemory({
    returnMessages: true,
    chatHistory: new CloudflareD1MessageHistory({
    tableName: 'stored_message',
        sessionId: 'example2',
    database: env.DB,
    }),
});

await memory.saveContext({ input: 'My name is TOMER' }, { output: 'Ok got it' });

const memory = new BufferMemory({
    returnMessages: true,
    chatHistory: new CloudflareD1MessageHistory({
    tableName: 'stored_message',
        sessionId: 'example2',
    database: env.DB,
    }),
});

await memory.saveContext({ input: 'My name is TOMER' }, { output: 'Ok got it' });

const memory = new BufferMemory({
    returnMessages: true,
    chatHistory: new CloudflareD1MessageHistory({
    tableName: 'stored_message',
        sessionId: 'example2',
    database: env.DB,
    }),
});

await memory.saveContext({ input: 'My name is TOMER' }, { output: 'Ok got it' });

For some reason everything is working perfectly fine in langchain js but I cant see the changes/records in my D1 db dashboard. I am running this in localhost

MMrinank ```javascript const memory = new BufferMemory({ returnMessages: true, ch...

M

Matt Silverlock•5/28/24, 5:29 PM

Localhost is local only: it's not updating the remote (deployed/production) database that the dashboard is connected to.

MMatt Silverlock Localhost is local only: it's not updating the remote (deployed/production) data...

M

Mrinank•5/28/24, 6:13 PM

Thanks for reminding, there is one -- remote flag in Wrangler :bibicat:

L

Laurent Pellegrino•5/28/24, 7:04 PM

Does D1 read replication is going to replicate in 1 datacenter per continent or all Cloudflare datacenters (i.e. 300+ cities)?

LLaurent Pellegrino Does D1 read replication is going to replicate in 1 datacenter per continent or ...

H

Hello, I’m Allie!•5/28/24, 7:05 PM

It's not going to be every datacenter, as not every datacenter supports Durable Objects at the moment. For the lower limit, I'm not sure

Heya, I want to give a data-curious team mate access to our D1. Is there an option to generate a rea

Similar Threads

Similar Threads

Similar Threads