What will happen in case of pop outage? Like if my D1 is in Singapore and that pop faces down time o

ⱤⱤØ₥₳₦What will happen in case of pop outage? Like if my D1 is in Singapore and that p...

C

Chaika•2/15/24, 2:16 AM

D1 runs on Durable Objects. Durable Object Storage is replicated to mutiple nearby datacenters (also why DOs/D1 is only in a few locations). If a specific colo/pop goes down, it should just failover/recreate in another, and it'd have the storage to do so.
Some of that might be slightly different with how D1 stores things, but should be generally the same

G

gratitude3908•2/15/24, 6:14 AM

I'm working on a new startup and have used various cloudflare services for over a decade (huge fan) . I'm looking at database solutions and am considering d1 for a laravel application. My previous application had 12,000 batch requests per second and approximately 500GB of data. We used elastic caching etc. in front. Looking at the pricing I cannot figure out how to think about it. Storage price seems high by itself. Does the group have any suggestions or thoughts?

Ggratitude3908 I'm working on a new startup and have used various cloudflare services for over ...

M

Max (@rozenmd)•2/15/24, 6:29 AM

for that kind of load I would use Postgres and Hyperdrive

SShyim Can I see somewhere where my D1 is hosted? I want to debug why the performance i...

M

Max (@rozenmd)•2/15/24, 9:26 AM

wrangler d1 info <dbname>

wrangler d1 info <dbname>

wrangler d1 info <dbname>

wrangler d1 info <dbname>

A

amigoz•2/15/24, 9:31 AM

Hi
I am having a specific issue with D1.

I am able to read data from the D1 console/dashboard. But request from worker returns no data. It just says success

{"success":true,"meta":{"served_by":"v3-prod","duration":0.1461,"changes":0,"last_row_id":0,"changed_db":false,"size_after":3919872,"rows_read":10,"rows_written":0}}

Writing data works. But reading does not.

Can anyone help?

A

amigoz•2/15/24, 9:32 AM

Able to read from the console

A

amigoz•2/15/24, 9:44 AM

yes using .run()

A

amigoz•2/15/24, 9:45 AM

Now it works. Thanks a lot.

A

amigoz•2/15/24, 9:46 AM

What about insert? We are still using .run() and it works. Is that also going to change

Aamigoz What about insert? We are still using .run() and it works. Is that also going to...

A

amigoz•2/15/24, 9:49 AM

Got it. Just looked at the doc again. Thanks

B

bun•2/15/24, 11:31 AM

what does d1 database need to be production ready ?

Ɽ

ⱤØ₥₳₦OP•2/15/24, 11:37 AM

I'm using worker, how do i exactly know the time D1 took to solve the query

I

impairedFelon•2/15/24, 11:37 AM

Check meta that query returns.

I

impairedFelon•2/15/24, 11:37 AM

Object {
  served_by: v3-prod,
  duration: 0.2097,
  changes: 0,
  last_row_id: 0,
  changed_db: false
  ...
}

Object {
  served_by: v3-prod,
  duration: 0.2097,
  changes: 0,
  last_row_id: 0,
  changed_db: false
  ...
}

Object {
  served_by: v3-prod,
  duration: 0.2097,
  changes: 0,
  last_row_id: 0,
  changed_db: false
  ...
}

Object {
  served_by: v3-prod,
  duration: 0.2097,
  changes: 0,
  last_row_id: 0,
  changed_db: false
  ...
}

I

impairedFelon•2/15/24, 11:38 AM

const { results, meta } = await stmt.bind(email).all();

const { results, meta } = await stmt.bind(email).all();

const { results, meta } = await stmt.bind(email).all();

const { results, meta } = await stmt.bind(email).all();

I

impairedFelon•2/15/24, 12:09 PM

After running this.

npx wrangler@latest d1 insights <myDB> --sortBy reads --timePeriod 7d

npx wrangler@latest d1 insights <myDB> --sortBy reads --timePeriod 7d

npx wrangler@latest d1 insights <myDB> --sortBy reads --timePeriod 7d

npx wrangler@latest d1 insights <myDB> --sortBy reads --timePeriod 7d

I find out this.

{
    "query": "SELECT (SELECT count(*) FROM \"table_one\") as \"table_one\",(SELECT count(*) FROM \"table_two\") as \"table_two\",(SELECT count(*) FROM \"table_three\") as \"table_three\",(SELECT count(*) FROM \"table_four\") as \"table_four\",(SELECT count(*) FROM \"table_five\") as \"table_five\",(SELECT count(*) FROM \"table_six\") as \"table_six\",(SELECT count(*) FROM \"table_seven\") as \"table_seven\",(SELECT count(*) FROM \"table_eight\") as \"table_eight\";",
    "avgRowsRead": 396035,
    "totalRowsRead": 792070,
    "avgRowsWritten": 0,
    "totalRowsWritten": 0,
    "avgDurationMs": 18.983150000000002,
    "totalDurationMs": 37.966300000000004,
    "numberOfTimesRun": 2
  },

{
    "query": "SELECT (SELECT count(*) FROM \"table_one\") as \"table_one\",(SELECT count(*) FROM \"table_two\") as \"table_two\",(SELECT count(*) FROM \"table_three\") as \"table_three\",(SELECT count(*) FROM \"table_four\") as \"table_four\",(SELECT count(*) FROM \"table_five\") as \"table_five\",(SELECT count(*) FROM \"table_six\") as \"table_six\",(SELECT count(*) FROM \"table_seven\") as \"table_seven\",(SELECT count(*) FROM \"table_eight\") as \"table_eight\";",
    "avgRowsRead": 396035,
    "totalRowsRead": 792070,
    "avgRowsWritten": 0,
    "totalRowsWritten": 0,
    "avgDurationMs": 18.983150000000002,
    "totalDurationMs": 37.966300000000004,
    "numberOfTimesRun": 2
  },

{
    "query": "SELECT (SELECT count(*) FROM \"table_one\") as \"table_one\",(SELECT count(*) FROM \"table_two\") as \"table_two\",(SELECT count(*) FROM \"table_three\") as \"table_three\",(SELECT count(*) FROM \"table_four\") as \"table_four\",(SELECT count(*) FROM \"table_five\") as \"table_five\",(SELECT count(*) FROM \"table_six\") as \"table_six\",(SELECT count(*) FROM \"table_seven\") as \"table_seven\",(SELECT count(*) FROM \"table_eight\") as \"table_eight\";",
    "avgRowsRead": 396035,
    "totalRowsRead": 792070,
    "avgRowsWritten": 0,
    "totalRowsWritten": 0,
    "avgDurationMs": 18.983150000000002,
    "totalDurationMs": 37.966300000000004,
    "numberOfTimesRun": 2
  },

{
    "query": "SELECT (SELECT count(*) FROM \"table_one\") as \"table_one\",(SELECT count(*) FROM \"table_two\") as \"table_two\",(SELECT count(*) FROM \"table_three\") as \"table_three\",(SELECT count(*) FROM \"table_four\") as \"table_four\",(SELECT count(*) FROM \"table_five\") as \"table_five\",(SELECT count(*) FROM \"table_six\") as \"table_six\",(SELECT count(*) FROM \"table_seven\") as \"table_seven\",(SELECT count(*) FROM \"table_eight\") as \"table_eight\";",
    "avgRowsRead": 396035,
    "totalRowsRead": 792070,
    "avgRowsWritten": 0,
    "totalRowsWritten": 0,
    "avgDurationMs": 18.983150000000002,
    "totalDurationMs": 37.966300000000004,
    "numberOfTimesRun": 2
  },

What is this, i don't have this query in any part of my code.

IimpairedFelon After running this. ```npx wrangler@latest d1 insights <myDB> --sortBy reads --t...

M

Max (@rozenmd)•2/15/24, 12:18 PM

it's SQL from the cloudflare dashboard, it runs when you open the list of tables

M

Max (@rozenmd)•2/15/24, 12:19 PM

will likely take out the row count from the list of tables in the future to avoid this

I

impairedFelon•2/15/24, 12:29 PM

Got it.

Ɽ

ⱤØ₥₳₦OP•2/15/24, 12:52 PM

how to know particular query's run time and other details in wrangler cli

ⱤⱤØ₥₳₦how to know particular query's run time and other details in wrangler cli

M

Max (@rozenmd)•2/15/24, 1:33 PM

was this answer insufficient? d1-database

MMax (@rozenmd)was this answer insufficient? https://discord.com/channels/595317990191398933/99...

Ɽ

ⱤØ₥₳₦OP•2/15/24, 2:09 PM

It just shows data only for creating table and stuff not for queries

ⱤⱤØ₥₳₦It just shows data only for creating table and stuff not for queries

M

Max (@rozenmd)•2/15/24, 2:12 PM

WRANGLER_LOG=debug wrangler d1 execute <dbname> --command=<command>

WRANGLER_LOG=debug wrangler d1 execute <dbname> --command=<command>

WRANGLER_LOG=debug wrangler d1 execute <dbname> --command=<command>

WRANGLER_LOG=debug wrangler d1 execute <dbname> --command=<command>

S

Santosh•2/15/24, 2:22 PM

@Max (@rozenmd) Definitely stuck with local development. Any help? wrangler dev doesnt seem to be able to locate local tables at all. I tried both case (CamelCase, All lowercase) as well for the table names (edited)

S

Santosh•2/15/24, 2:24 PM

Whats required in wrangler.toml for "wrangler dev" to be able to recognize tables? I run all my commands (wrangler d1 execute --local) and they work fine and I can see the tables, but wrangler dev is not able to recognize those tables. What could I be doing wrong?

SSantosh @Max (@rozenmd) Definitely stuck with local development. Any help? wrangler dev...

M

Max (@rozenmd)•2/15/24, 2:24 PM

I strongly recommend you go through https://developers.cloudflare.com/d1/get-started/ step by step

SSantosh Whats required in wrangler.toml for "wrangler dev" to be able to recognize table...

M

Max (@rozenmd)•2/15/24, 2:25 PM

hard to say without seeing your entire project

MMax (@rozenmd)`WRANGLER_LOG=debug wrangler d1 execute <dbname> --command=<command>`

Ɽ

ⱤØ₥₳₦OP•2/15/24, 2:25 PM

Thanks sir for this, I'll try this.

I must admit that D1 is incredible because I tried same database (200k rows) both on mysql (with indexes) and D1 (with indexes)

Mysql takes 250-300ms on average while D1 takes 30-50ms for same queries.

I love what you all have built; this magnificent thing is truly amazing!

SSantosh Whats required in wrangler.toml for "wrangler dev" to be able to recognize table...

M

Max (@rozenmd)•2/15/24, 2:32 PM

I would start by console logging

env

env

and

env.DB

env.DB

env.DB

env.DB (or whatever binding you set it to in

wrangler.toml

wrangler.toml

wrangler.toml

wrangler.toml)

if you can see the client API,

console.log(await env.DB.prepare("SELECT name FROM sqlite_schema WHERE type='table' ORDER BY name ;").all());

console.log(await env.DB.prepare("SELECT name FROM sqlite_schema WHERE type='table' ORDER BY name ;").all());

console.log(await env.DB.prepare("SELECT name FROM sqlite_schema WHERE type='table' ORDER BY name ;").all());

console.log(await env.DB.prepare("SELECT name FROM sqlite_schema WHERE type='table' ORDER BY name ;").all()); to see what tables your Worker can see

if

env

env

or

env.DB

env.DB

env.DB

env.DB are undefined, you haven't configured your wrangler.toml correctly

S

Santosh•2/15/24, 2:38 PM

Followed the tutorial verbatim, still doesnt see local customers table: Error: D1_ERROR: SqliteError: no such table: customers. And console.log of the binding (env.DB ) prints

DB ===> D1Database {
  fetcher: Fetcher
}

DB ===> D1Database {
  fetcher: Fetcher
}

DB ===> D1Database {
  fetcher: Fetcher
}

DB ===> D1Database {
  fetcher: Fetcher
}

All I have changed is the database_id to match mine:

[[d1_databases]]
binding = "DB"
database_name = "prod-d1-tutorial"
database_id = "413dbdd7-5226-4d5c-a57e-0c898aa19002"

[[d1_databases]]
binding = "DB"
database_name = "prod-d1-tutorial"
database_id = "413dbdd7-5226-4d5c-a57e-0c898aa19002"

[[d1_databases]]
binding = "DB"
database_name = "prod-d1-tutorial"
database_id = "413dbdd7-5226-4d5c-a57e-0c898aa19002"

[[d1_databases]]
binding = "DB"
database_name = "prod-d1-tutorial"
database_id = "413dbdd7-5226-4d5c-a57e-0c898aa19002"

SSantosh Followed the tutorial verbatim, still doesnt see local customers table: Error: D...

M

Max (@rozenmd)•2/15/24, 2:47 PM

I just ran

npx wrangler d1 execute prod-d1-tutorial --local --file=./schema.sql

npx wrangler d1 execute prod-d1-tutorial --local --file=./schema.sql

npx wrangler d1 execute prod-d1-tutorial --local --file=./schema.sql

npx wrangler d1 execute prod-d1-tutorial --local --file=./schema.sql before running

wrangler dev

wrangler dev

, and my console log output is:

{
  success: true,
  meta: {
    served_by: 'miniflare.db',
    duration: 0,
    changes: 0,
    last_row_id: 0,
    changed_db: false,
    size_after: 12288,
    rows_read: 4,
    rows_written: 0
  },
  results: [ { name: 'Customers' }, { name: '_cf_KV' } ]
}

{
  success: true,
  meta: {
    served_by: 'miniflare.db',
    duration: 0,
    changes: 0,
    last_row_id: 0,
    changed_db: false,
    size_after: 12288,
    rows_read: 4,
    rows_written: 0
  },
  results: [ { name: 'Customers' }, { name: '_cf_KV' } ]
}

{
  success: true,
  meta: {
    served_by: 'miniflare.db',
    duration: 0,
    changes: 0,
    last_row_id: 0,
    changed_db: false,
    size_after: 12288,
    rows_read: 4,
    rows_written: 0
  },
  results: [ { name: 'Customers' }, { name: '_cf_KV' } ]
}

{
  success: true,
  meta: {
    served_by: 'miniflare.db',
    duration: 0,
    changes: 0,
    last_row_id: 0,
    changed_db: false,
    size_after: 12288,
    rows_read: 4,
    rows_written: 0
  },
  results: [ { name: 'Customers' }, { name: '_cf_KV' } ]
}

S

Santosh•2/15/24, 2:48 PM

Thats whole thing

M

Max (@rozenmd)•2/15/24, 2:48 PM

also, what version of wrangler are you using?

S

Santosh•2/15/24, 2:48 PM

When I run from commands line using wrangler d1 execute --local, I can see all of them.

S

Santosh•2/15/24, 2:48 PM

wrangler 3.7.0 (update available 3.28.2)

S

Santosh•2/15/24, 2:48 PM

its only when I do wrangler dev that it doesnt see the local tables

S

Santosh•2/15/24, 2:49 PM

I swear it worked when I did a poc last month

M

Max (@rozenmd)•2/15/24, 2:49 PM

how are you starting wrangler?

S

Santosh•2/15/24, 2:49 PM

wrangler dev

S

Santosh•2/15/24, 2:49 PM

just from the command line

SSantosh wrangler dev

M

Max (@rozenmd)•2/15/24, 2:50 PM

what does

console.log(await env.DB.prepare("SELECT name FROM sqlite_schema WHERE type='table' ORDER BY name ;").all());

console.log(await env.DB.prepare("SELECT name FROM sqlite_schema WHERE type='table' ORDER BY name ;").all());

console.log(await env.DB.prepare("SELECT name FROM sqlite_schema WHERE type='table' ORDER BY name ;").all());

console.log(await env.DB.prepare("SELECT name FROM sqlite_schema WHERE type='table' ORDER BY name ;").all()); output, in your Worker?

S

Santosh•2/15/24, 2:50 PM

z ===>  Object {
  success: true,
  meta: Object,
  results: Array(0)
}

z ===>  Object {
  success: true,
  meta: Object,
  results: Array(0)
}

z ===>  Object {
  success: true,
  meta: Object,
  results: Array(0)
}

z ===>  Object {
  success: true,
  meta: Object,
  results: Array(0)
}

SSantosh ```z ===> Object { success: true, meta: Object, results: Array(0) }```

M

Max (@rozenmd)•2/15/24, 2:52 PM

how about:

const result = await env.DB.prepare("SELECT name FROM sqlite_schema WHERE type='table' ORDER BY name ;").all();
console.log(result.meta);

const result = await env.DB.prepare("SELECT name FROM sqlite_schema WHERE type='table' ORDER BY name ;").all();
console.log(result.meta);

const result = await env.DB.prepare("SELECT name FROM sqlite_schema WHERE type='table' ORDER BY name ;").all();
console.log(result.meta);

const result = await env.DB.prepare("SELECT name FROM sqlite_schema WHERE type='table' ORDER BY name ;").all();
console.log(result.meta);

S

Santosh•2/15/24, 2:53 PM

z ===>  Object {
  duration: 0.28007400035858154,
  last_row_id: 0,
  changes: 0,
  served_by: miniflare.db,
  internal_stats: null
}

z ===>  Object {
  duration: 0.28007400035858154,
  last_row_id: 0,
  changes: 0,
  served_by: miniflare.db,
  internal_stats: null
}

z ===>  Object {
  duration: 0.28007400035858154,
  last_row_id: 0,
  changes: 0,
  served_by: miniflare.db,
  internal_stats: null
}

z ===>  Object {
  duration: 0.28007400035858154,
  last_row_id: 0,
  changes: 0,
  served_by: miniflare.db,
  internal_stats: null
}

M

Max (@rozenmd)•2/15/24, 2:55 PM

oh wtf that wrangler version is cursed

SSantosh ```z ===> Object { duration: 0.28007400035858154, last_row_id: 0, changes...

M

Max (@rozenmd)•2/15/24, 2:55 PM

can you try install wrangler 3.28.2 real quick?

S

Santosh•2/15/24, 2:55 PM

I did and that seems to have worked

What will happen in case of pop outage? Like if my D1 is in Singapore and that pop faces down time o

Similar Threads

Similar Threads

Similar Threads