Is there any overhead to just using the D1 REST API to query the databases? (https://developers.clo

Eericwilhite Is there any overhead to just using the D1 REST API to query the databases? (ht...

H

Quite a bit. Going through a Worker, a query can head directly to the database. Via the API, it has to hit the API Core datacenters first, then back out to the datacenter your database is in, along with auth layers, etc.

E

ericwilhiteOP•3/8/24, 1:47 PM

Original message was deleted

E

ericwilhiteOP•3/8/24, 1:50 PM

Nice! this looks helpful thanks for the link

J

jason•3/8/24, 2:32 PM

that worked @Max (@rozenmd) , or at least there were no errors, will continue playing around with it to get a better understanding, thanks for your help

Jjason that worked @Max (@rozenmd) , or at least there were no errors, will continue pl...

M

Max (@rozenmd)•3/8/24, 2:40 PM

unfortunately you need to specify

--persist-to

--persist-to

--persist-to

--persist-to in every command you use if you customise it

T

Tony Leung•3/8/24, 3:34 PM

Hi, do cf has any plan to extend KV to some kind of db like DynamoDB? I find that half of my case I don't really need complex JOIN or select by key but a simple query on a data partition with hash key defined and filtering on a key in the document. D1 is kinda overkill for me and with a large table I don't know that can D1 handle a query like that.

U

Unsmart•3/8/24, 3:35 PM

Durable objects is very similar to DynamoDB

I

Ian•3/8/24, 3:36 PM

Hi folks - somewhat stuck with bringing D1 into our next-on-pages project. We have pages up and running with our code base (previously hosted on Amplify) but having crawled all over the docs I don't see clearly my steps to integrate D1 into a NextJS code base specifically.

UUnsmart Durable objects is very similar to DynamoDB

T

Tony Leung•3/8/24, 3:40 PM

DO is transactional, so when I make multiple queries at the same time, can DO returns data for me fast enough :? BTW how can I implements smth like dynamodb with DO though?

U

Unsmart•3/8/24, 3:43 PM

Queries are extremely fast so you wont really notice if 2 storage requests are not happening at the exact same time. And why would you implement DynamoDB just use DOs as a DO or just use Dynamo instead

UUnsmart Queries are extremely fast so you wont really notice if 2 storage requests are n...

T

Tony Leung•3/8/24, 3:54 PM

AWS outbound data transfer is not cheap, I was living in fear that someday Jeff Bozos would come to my house to slap a big fat bill in my face...

IIan Hi folks - somewhat stuck with bringing D1 into our next-on-pages project. We ha...

A

alpha1119179•3/8/24, 3:56 PM

Exactly my problem. I'm doing the same thing. Pages built on nextjs 14. I've been trying to integrate d1 with limited success. DM me Ian if you wanna share my findings.

Original message was deleted

D

DaniFoldi•3/8/24, 4:19 PM

fun fact it also converts

number[]

number[]

number[]

number[]s in line 227-230

Original message was deleted

I

Ian•3/8/24, 4:19 PM

Relatively new to this business - I was having trouble running npx wrangler pages dev -- npm run dev but have fixed that and have been able to bind the DB by adding to wrangler.toml so making progress. Will see if I can get at it next. Thanks for the reply.

T

Tomas Pietravallo•3/8/24, 4:20 PM

Hey everyone! Sorry if this is not the correct channel but I am having trouble finding a solution to this...

I've got a table TABLE_NAME on my D1. pragma table_list does show it, and pragma table_info(TABLE_NAME) indeed shows the correct info:

// This code lives inside a route.tsx file running locally with:
// npm run pages:preview -- --d1 DB=xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx
// console.log(await env.DB.prepare(`pragma table_info(TABLE_NAME)`).all());
{
  success: true,
  meta: {
    served_by: 'miniflare.db',
    // ...
  },
  results: [
    {
      cid: 0,
      name: 'id',
      type: 'INT AUTO_INCREMENT',
      notnull: 0,
      dflt_value: null,
      pk: 1
    },
    {
      cid: 1,
      name: 'url',
      type: 'VARCHAR(2048)',
      notnull: 1,
      dflt_value: null,
      pk: 0
    },
// ...
// ...

// This code lives inside a route.tsx file running locally with:
// npm run pages:preview -- --d1 DB=xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx
// console.log(await env.DB.prepare(`pragma table_info(TABLE_NAME)`).all());
{
  success: true,
  meta: {
    served_by: 'miniflare.db',
    // ...
  },
  results: [
    {
      cid: 0,
      name: 'id',
      type: 'INT AUTO_INCREMENT',
      notnull: 0,
      dflt_value: null,
      pk: 1
    },
    {
      cid: 1,
      name: 'url',
      type: 'VARCHAR(2048)',
      notnull: 1,
      dflt_value: null,
      pk: 0
    },
// ...
// ...

// This code lives inside a route.tsx file running locally with:
// npm run pages:preview -- --d1 DB=xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx
// console.log(await env.DB.prepare(`pragma table_info(TABLE_NAME)`).all());
{
  success: true,
  meta: {
    served_by: 'miniflare.db',
    // ...
  },
  results: [
    {
      cid: 0,
      name: 'id',
      type: 'INT AUTO_INCREMENT',
      notnull: 0,
      dflt_value: null,
      pk: 1
    },
    {
      cid: 1,
      name: 'url',
      type: 'VARCHAR(2048)',
      notnull: 1,
      dflt_value: null,
      pk: 0
    },
// ...
// ...

// This code lives inside a route.tsx file running locally with:
// npm run pages:preview -- --d1 DB=xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx
// console.log(await env.DB.prepare(`pragma table_info(TABLE_NAME)`).all());
{
  success: true,
  meta: {
    served_by: 'miniflare.db',
    // ...
  },
  results: [
    {
      cid: 0,
      name: 'id',
      type: 'INT AUTO_INCREMENT',
      notnull: 0,
      dflt_value: null,
      pk: 1
    },
    {
      cid: 1,
      name: 'url',
      type: 'VARCHAR(2048)',
      notnull: 1,
      dflt_value: null,
      pk: 0
    },
// ...
// ...

But then trying to run

SELECT * FROM TABLE_NAME

SELECT * FROM TABLE_NAME

SELECT * FROM TABLE_NAME

SELECT * FROM TABLE_NAME returns an empty array, whereas I can see the actual data (not empty) on my CF dashboard, and I can query the records correctly using

npx wrangler d1 execute site-db --command="SELECT * FROM TABLE_NAME"

npx wrangler d1 execute site-db --command="SELECT * FROM TABLE_NAME"

npx wrangler d1 execute site-db --command="SELECT * FROM TABLE_NAME"

npx wrangler d1 execute site-db --command="SELECT * FROM TABLE_NAME"

My wrangler.toml looks like this:

[[d1_databases]]
binding = "DB" # i.e. available in your Worker on env.DB
database_name = "site-db"
database_id = "xxxxxxxxxx-xxxx-xxxx-xxxxxxxxxxx"
preview_database_id = "DB"

[[d1_databases]]
binding = "DB" # i.e. available in your Worker on env.DB
database_name = "site-db"
database_id = "xxxxxxxxxx-xxxx-xxxx-xxxxxxxxxxx"
preview_database_id = "DB"

[[d1_databases]]
binding = "DB" # i.e. available in your Worker on env.DB
database_name = "site-db"
database_id = "xxxxxxxxxx-xxxx-xxxx-xxxxxxxxxxx"
preview_database_id = "DB"

[[d1_databases]]
binding = "DB" # i.e. available in your Worker on env.DB
database_name = "site-db"
database_id = "xxxxxxxxxx-xxxx-xxxx-xxxxxxxxxxx"
preview_database_id = "DB"

Any help is appreciated, thanks!

T

Tomas Pietravallo•3/8/24, 4:23 PM

This is the entire pages:preview command my bad

npm run pages:build && wrangler pages dev .vercel/output/static --compatibility-date=2024-01-17 --compatibility-flag=nodejs_compat --d1 DB=xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx

npm run pages:build && wrangler pages dev .vercel/output/static --compatibility-date=2024-01-17 --compatibility-flag=nodejs_compat --d1 DB=xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx

npm run pages:build && wrangler pages dev .vercel/output/static --compatibility-date=2024-01-17 --compatibility-flag=nodejs_compat --d1 DB=xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx

npm run pages:build && wrangler pages dev .vercel/output/static --compatibility-date=2024-01-17 --compatibility-flag=nodejs_compat --d1 DB=xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx

Where pages:build is just

npx @cloudflare/next-on-pages

npx @cloudflare/next-on-pages

npx @cloudflare/next-on-pages

npx @cloudflare/next-on-pages

T

Tomas Pietravallo•3/8/24, 4:26 PM

Haha, thanks! Well that solves it

So https://developers.cloudflare.com/d1/configuration/local-development/ is outdates then?

"""
To access your production (remote) database, pass the --remote flag when calling wrangler dev. Any changes you make when running in --remote mode cannot be undone.
"""

This gave me the impression I would've been able to connect to the db locally

Original message was deleted

T

Tomas Pietravallo•3/8/24, 4:28 PM

Thank you very much!

They weren't joking when they said cf discord was incredibly active and helpful

Original message was deleted

R

RexNormous•3/8/24, 4:30 PM

Do you actually not need to run any

wrangler

wrangler

commands to use a D1 binding in a next-on-pages application if calling

setupDevPlatform

setupDevPlatform

setupDevPlatform

setupDevPlatform in

nextConfig

nextConfig

nextConfig

nextConfig?

Original message was deleted

R

RexNormous•3/8/24, 4:34 PM

Interesting, thanks … I’m getting undefined for my D1 binding always, though a similar setup has worked for KV bindings

B

boywonder350•3/8/24, 11:48 PM

Are there any updates to the maximum database size? I really want to use D1 but 2GB worries me

B

boywonder350•3/8/24, 11:49 PM

Seems like the idea is to a db per user

B

boywonder350•3/8/24, 11:50 PM

Which would be 50,000 dbs only. Is there a plan to increase this limit?

Bboywonder350 Are there any updates to the maximum database size? I really want to use D1 but ...

C

Cyb3r-Jak3•3/8/24, 11:56 PM

Planed for GA which is planned for April

Bboywonder350 Which would be 50,000 dbs only. Is there a plan to increase this limit?

M

Max (@rozenmd)•3/9/24, 6:44 AM

there's no limit, we just put one so people would believe us

M

matt•3/9/24, 7:20 AM

is this the right way of setting up the column? it's from the d1-drizzle-remix-example:

 createdAt: integer("created_at").notNull().default(sql`CURRENT_TIMESTAMP`),

 createdAt: integer("created_at").notNull().default(sql`CURRENT_TIMESTAMP`),

 createdAt: integer("created_at").notNull().default(sql`CURRENT_TIMESTAMP`),

 createdAt: integer("created_at").notNull().default(sql`CURRENT_TIMESTAMP`),

F

forslunds•3/9/24, 10:19 AM

Really hope we have some ”dynamic binding” when it goes GA, cant see going prod otherwise with it

Aalpha1119179 Exactly my problem. I'm doing the same thing. Pages built on nextjs 14. I've bee...

I

Ian•3/9/24, 11:36 AM

DM denied. Have sent friend request.

Mmatt is this the right way of setting up the column? it's from the d1-drizzle-remix-e...

M

mw10013•3/9/24, 3:34 PM

I also used the remix example and it didn't work so well for me. What works better for me so far is:

  createdAt: integer("created_at", { mode: "timestamp" })
    .notNull()
    .default(sql`(unixepoch())`),

  createdAt: integer("created_at", { mode: "timestamp" })
    .notNull()
    .default(sql`(unixepoch())`),

  createdAt: integer("created_at", { mode: "timestamp" })
    .notNull()
    .default(sql`(unixepoch())`),

  createdAt: integer("created_at", { mode: "timestamp" })
    .notNull()
    .default(sql`(unixepoch())`),

MMax (@rozenmd)there's no limit, we just put one so people would believe us

O

ozgurgul1203•3/9/24, 5:12 PM

I have a hobby project with potentially a few TB of storage need for database. can d1 handle that?

Oozgurgul1203 I have a hobby project with potentially a few TB of storage need for database. c...

M

Max (@rozenmd)•3/9/24, 5:19 PM

no, I'd recommend you look into Hyperdrive if you're using Workers

Hhannes quick question regarding programmatic D1 usage (idea of many databases, per user...

D

DaniFoldi•3/9/24, 5:29 PM

You can use the query endpoint to run a

CREATE TABLE

CREATE TABLE

query: https://developers.cloudflare.com/api/operations/cloudflare-d1-query-database

W

wasauce•3/9/24, 11:01 PM

Did https://developers.cloudflare.com/d1/changelog/ move? I am having trouble finding

U

Unsmart•3/9/24, 11:01 PM

https://developers.cloudflare.com/d1/platform/changelog/

Cloudflare Docs

Changelog · Cloudflare D1 docs

As of 2024-03-05, D1 usage will start to be counted and may incur charges for an account’s future billing cycle.

U

Unsmart•3/9/24, 11:02 PM

Also you can search it works well (how I found it)

W

wasauce•3/9/24, 11:44 PM

Thank you

MMax (@rozenmd)there's no limit, we just put one so people would believe us

W

wasauce•3/9/24, 11:46 PM

Is the thought that the 2GB limit will remain or that that also might increase?

Wwasauce Did https://developers.cloudflare.com/d1/changelog/ move? I am having trouble fi...

W

wasauce•3/9/24, 11:46 PM

Note - Links in documentation point to this old URL

Wwasauce Note - Links in documentation point to this old URL

M

Max (@rozenmd)•3/10/24, 8:48 AM

cheers for that, will add a redirect on Monday (if someone else doesn't PR it first)

Wwasauce Is the thought that the 2GB limit will remain or that that also might increase?

M

Max (@rozenmd)•3/10/24, 8:49 AM

I was specifically talking about the number of DBs, we're working towards a 10GB limit

MMax (@rozenmd)I was specifically talking about the number of DBs, we're working towards a 10GB...

W

wasauce•3/10/24, 8:54 AM

Thank you! Looking forward to needing 10GB.

J

Jonathan Williams•3/10/24, 8:54 AM

J

Jonathan Williams•3/10/24, 8:55 AM

#d1-beta

E

Enduriel•3/10/24, 11:28 AM

Have there been any comments with regards to timeline or expectations for JS (or maybe wasm/rust???) procedures? Using workers-rs and my 2 options seem to be leaving a bunch of race conditions around or developing a system to handle the JsValue errors d1 returns from batch transactions so I can figure out why transactions failed

M

mnik01•3/10/24, 2:36 PM

I was trying to create Table with ENUM
but there was error about syntax

with string value table was created successfully

so d1 not supporting ENUM?

Mmnik01 I was trying to create Table with ENUM but there was error about syntax with st...

M

Max (@rozenmd)•3/10/24, 4:18 PM

D1 is SQLite under the hood

M

Max (@rozenmd)•3/10/24, 4:19 PM

ENUM looks like a MySQL feature

N

Nicolas•3/11/24, 12:20 AM

Is there a reason i cannot see the local D1 data with VSCode? I am using the SQLite Viewer extension (which I confirmed works with Django), but for some reason its not showing the data from the

Customers

Customers

Customers

Customers table, as you can see with the wrangler query, there is data locally

NNicolas Is there a reason i cannot see the local D1 data with VSCode? I am using the SQL...

C

Cyb3r-Jak3•3/11/24, 12:47 AM

Local dev doesn't have all the data in a single database. Best way to query is to use miniflare

Is there any overhead to just using the D1 REST API to query the databases? (https://developers.clo

Similar Threads

Similar Threads

Similar Threads