I have an SQLite on my VPS. Can I create an script or something to push my whole database (50MB) to

Mmaguayo I have an SQLite on my VPS. Can I create an script or something to push my whole...

H

Hard@Work•7/4/24, 9:12 AM

Maybe do an initial sync, then mirror all commands to the local SQLite db to D1?

W

WillTolmie•7/4/24, 10:17 AM

Strange one this - my worker still seems to have access to a deleted database. I was wondering why my API was working fine (new rows being added and being queried fine) but new rows weren't showing in the CF dashboard

W

WillTolmie•7/4/24, 10:17 AM

I tried D1 export from the root of the worker via wrangler and it complained the database didn't exist

W

WillTolmie•7/4/24, 10:20 AM

Then I realised I hadn't updated the D1 binding with the new ID

W

WillTolmie•7/4/24, 10:21 AM

But it still works!

W

WillTolmie•7/4/24, 10:50 AM

anyway FWIW the deleted months ago db id is 8001ccc4-1a33-435e-bbb5-7d8723a3a58a

Original message was deleted

B

boop•7/4/24, 4:56 PM

No replication at the moment, your db sits where you put it and stays there

B

boop•7/4/24, 4:59 PM

Yup, it's one of the features that they're planning on rolling out :soontm:

B

boop•7/4/24, 5:01 PM

I don't think they have ever set a date in stone for any of the features, it's just a waiting game

HHello, I’m Allie!Afaik it isn’t available in any capacity atm

B

boop•7/4/24, 5:02 PM

Judging by this message though I wouldn't keep your hopes on it coming soon

B

boop•7/4/24, 5:03 PM

But yes, workers is a completely separate product and is in every available DC

Original message was deleted

B

boop•7/4/24, 5:03 PM

You could try your luck with smart placement, although CloudFlare often times still gets the location wrong

B

boop•7/4/24, 5:04 PM

Sorry for tagging you HardlyWorking, I meant to just quote your message

Bboop Judging by this message though I wouldn't keep your hopes on it coming soon

H

Hello, I’m Allie!•7/4/24, 5:04 PM

Note, this was said in my capacity as a community member

H

Hello, I’m Allie!•7/4/24, 5:04 PM

And thus I can’t speak to when exactly this feature may or may not be coming

B

boop•7/4/24, 5:04 PM

Gotcha, I misunderstood your orange CloudFlare role

Bboop Gotcha, I misunderstood your orange CloudFlare role

H

Hello, I’m Allie!•7/4/24, 5:05 PM

Yeah, it can be a tad confusing, sorry for adding to it. I’m on the #r2 team, but even questions around R2 I am not 100% knowledgeable

Original message was deleted

H

Hello, I’m Allie!•7/4/24, 5:06 PM

KV is replicated US/EU. R2 isn’t replicated yet, but you can do manually replication by copying between buckets in different regions

Original message was deleted

B

boop•7/4/24, 5:06 PM

KV is replicated, it's just a key-value store though

Original message was deleted

B

boop•7/4/24, 5:09 PM

KV is pretty dang expensive

B

boop•7/4/24, 5:10 PM

Besides, even without replication, on single-query database operations D1 is pretty speedy

Original message was deleted

B

boop•7/4/24, 5:18 PM

You're battling the speed of light here unfortunately

B

boop•7/4/24, 5:18 PM

In the known universe that is currently unachievable

B

boop•7/4/24, 5:21 PM

They do have Argo, not sure if they implement it in the worker to d1 communication though

Bboop They do have Argo, not sure if they implement it in the worker to d1 communicati...

C

Chaika•7/4/24, 5:24 PM

it'd help inbound (ent like inbound routing), but shouldn't matter for communication to D1 afaik. Smart Placement can help though, by making your worker run close

E

engi.dev•7/4/24, 11:19 PM

Hello everyone
I am using free account. I have deployed my next.js project on cloudflare D1.
I tried many times, but they said that project was deployed successfully then I can't see website.
I dont know that 's why.
if someone knows this error. pls messag me.

S

SteffTek•7/5/24, 5:19 AM

Can we check where our d1 database is created?

S

SteffTek•7/5/24, 5:19 AM

ive used terraform to instantiate it and we cannot give a location hint there

S

SteffTek•7/5/24, 5:23 AM

and the "message wait time" settings say 0-60, but you cannot actually set it to 0

Eengi.dev Hello everyone I am using free account. I have deployed my next.js project on cl...

H

harshil1712•7/5/24, 9:19 AM

Hey, answered on your thread

SSteffTek Can we check where our d1 database is created?

H

harshil1712•7/5/24, 9:22 AM

You can use the

wrangler d1 info

wrangler d1 info

wrangler d1 info

wrangler d1 info command. https://developers.cloudflare.com/workers/wrangler/commands/#info

Cloudflare Docs

Commands - Wrangler · Cloudflare Workers docs

Create, develop, and deploy your Cloudflare Workers with Wrangler commands.

Hharshil1712 You can use the `wrangler d1 info` command. https://developers.cloudflare.com/wo...

S

SteffTek•7/5/24, 9:26 AM

thanks, looks like my databases are running in WNAM... great for european users /s

C

cosbgn•7/5/24, 11:31 AM

hey I'm getting a lot of

too many SQL variables at offset

too many SQL variables at offset

too many SQL variables at offset

too many SQL variables at offset seems d1 supports only 10 parameters, so even simple inserts like this will fail:

const sqlite = new Database("db.sqlite");

const db = drizzle<typeof schema>(sqlite, { schema });

const books = new Array(35).fill({}).map((b, i) => ({
  id: i.toString(),
  title: "The Great Gatsby",
  description: "A book about a rich guy",
}));

await db.insert(schema.books).values(books);

const sqlite = new Database("db.sqlite");

const db = drizzle<typeof schema>(sqlite, { schema });

const books = new Array(35).fill({}).map((b, i) => ({
  id: i.toString(),
  title: "The Great Gatsby",
  description: "A book about a rich guy",
}));

await db.insert(schema.books).values(books);

const sqlite = new Database("db.sqlite");

const db = drizzle<typeof schema>(sqlite, { schema });

const books = new Array(35).fill({}).map((b, i) => ({
  id: i.toString(),
  title: "The Great Gatsby",
  description: "A book about a rich guy",
}));

await db.insert(schema.books).values(books);

const sqlite = new Database("db.sqlite");

const db = drizzle<typeof schema>(sqlite, { schema });

const books = new Array(35).fill({}).map((b, i) => ({
  id: i.toString(),
  title: "The Great Gatsby",
  description: "A book about a rich guy",
}));

await db.insert(schema.books).values(books);

Is there any workaround?

C

cosbgn•7/5/24, 11:32 AM

seems crazy I can't just insert 30 rows

C

cosbgn•7/5/24, 11:32 AM

do I need to do one by one?

O

oof2win2•7/5/24, 1:26 PM

so if you have over 10 columns you cant insert a row?

V

Vikash•7/6/24, 4:07 AM

What is

offset

offset

offset

offset in errors ?

I have just 100 lines in my insert file and it errors with message

syntax error at offset 245

syntax error at offset 245

syntax error at offset 245

syntax error at offset 245 On what line I should look for the error to correct it.

U

Unsmart•7/6/24, 4:14 AM

character 245

UUnsmart character 245

V

Vikash•7/6/24, 4:17 AM

Got it, thanks.

V

Vikash•7/6/24, 10:10 AM

New error:

{"D1_RESET_DO":true}

{"D1_RESET_DO":true}

{"D1_RESET_DO":true}

{"D1_RESET_DO":true} Any idea what is this?

O

Old Man Maker•7/6/24, 2:16 PM

it is seriously rediculous that we can name a table with a - dash in it within the CF dashboard, but doing a PRAGMA table_info fails. I think I mentioned this several months ago.

C

Chinoman10•7/6/24, 11:54 PM

Error: D1_TYPE_ERROR: Type 'object' not supported for value '123string,134,123string,0,0.17,comercial@

Error: D1_TYPE_ERROR: Type 'object' not supported for value '123string,134,123string,0,0.17,comercial@

Error: D1_TYPE_ERROR: Type 'object' not supported for value '123string,134,123string,0,0.17,comercial@

Error: D1_TYPE_ERROR: Type 'object' not supported for value '123string,134,123string,0,0.17,comercial@ (...)

Any ideas? I already tried searching around and reading all the docs back and forth.

C

Chinoman10•7/7/24, 12:07 AM

... found it :facepalm:

C

Chinoman10•7/7/24, 12:08 AM

turns out building wrappers for nice abstraction is only good if you test it our extensively...

C

Chinoman10•7/7/24, 12:08 AM

I was missing the spread operator on the params that I was passing.

C

Chinoman10•7/7/24, 12:41 AM

D

dog1283•7/7/24, 6:58 PM

Hi All, I am facing an issue using D1 locally. The issue is my worker cannot see any of the DB objects I have created.

First, I setup my DB with some DDL:

wrangler d1 execute appdb-dev --local --file=./1.sql

wrangler d1 execute appdb-dev --local --file=./1.sql

wrangler d1 execute appdb-dev --local --file=./1.sql

wrangler d1 execute appdb-dev --local --file=./1.sql

Then I can query the DB:

wrangler d1 execute appdb-dev --local --command="SELECT * FROM User"

wrangler d1 execute appdb-dev --local --command="SELECT * FROM User"

wrangler d1 execute appdb-dev --local --command="SELECT * FROM User"

wrangler d1 execute appdb-dev --local --command="SELECT * FROM User" // works

However, when I try to access my local d1 instance in the worker, it can't find any tables. It is obviously looking at some other local instance, but I can't find any way to list the local instances, or don't know the commands to understand whether I am looking at the right one.

Any ideas? Wrangler.toml below:

name = "app-services"
compatibility_date = "2023-12-01"
route = "api.myapp.app/*"
compatibility_flags = ["nodejs_compat"]

[vars]
LOG_LEVEL="debug"
TEST_MODE="TRUE"
kv_namespaces = [
    { binding = "MYAPP_TOKENS", preview_id = "###", id = "###" },
]

[[d1_databases]]
binding = "DB"
database_name = "appdb-dev"
database_id = "###"
preview_database_id = "appdb-dev"

[env.dev]
route = "dev-api.myapp.app/*"

name = "app-services"
compatibility_date = "2023-12-01"
route = "api.myapp.app/*"
compatibility_flags = ["nodejs_compat"]

[vars]
LOG_LEVEL="debug"
TEST_MODE="TRUE"
kv_namespaces = [
    { binding = "MYAPP_TOKENS", preview_id = "###", id = "###" },
]

[[d1_databases]]
binding = "DB"
database_name = "appdb-dev"
database_id = "###"
preview_database_id = "appdb-dev"

[env.dev]
route = "dev-api.myapp.app/*"

name = "app-services"
compatibility_date = "2023-12-01"
route = "api.myapp.app/*"
compatibility_flags = ["nodejs_compat"]

[vars]
LOG_LEVEL="debug"
TEST_MODE="TRUE"
kv_namespaces = [
    { binding = "MYAPP_TOKENS", preview_id = "###", id = "###" },
]

[[d1_databases]]
binding = "DB"
database_name = "appdb-dev"
database_id = "###"
preview_database_id = "appdb-dev"

[env.dev]
route = "dev-api.myapp.app/*"

name = "app-services"
compatibility_date = "2023-12-01"
route = "api.myapp.app/*"
compatibility_flags = ["nodejs_compat"]

[vars]
LOG_LEVEL="debug"
TEST_MODE="TRUE"
kv_namespaces = [
    { binding = "MYAPP_TOKENS", preview_id = "###", id = "###" },
]

[[d1_databases]]
binding = "DB"
database_name = "appdb-dev"
database_id = "###"
preview_database_id = "appdb-dev"

[env.dev]
route = "dev-api.myapp.app/*"

VVikash New error: `{"D1_RESET_DO":true}` Any idea what is this?

S

slagathor•7/8/24, 3:41 AM

I'm running into this as well. Have you been able to resolve it?

Sslagathor I'm running into this as well. Have you been able to resolve it?

V

Vikash•7/8/24, 3:57 AM

No, the import on D1 is super buggy. I was unable to import a small DB of 1gb as well.

VVikash No, the import on D1 is super buggy. I was unable to import a small DB of 1gb as...

S

slagathor•7/8/24, 5:08 AM

Hmm...interesting. This is the first time I've ran into this issue

I have an SQLite on my VPS. Can I create an script or something to push my whole database (50MB) to

Similar Threads

Similar Threads

Similar Threads