for the dbs with millions of rows, what's the approximate size?

J

jayMakes11OP•4/5/24, 2:07 PM

i've seen postgres dbs at 300 to 400 GB... that's my only context for this... maybe sqlite is inherently smaller or something?

JjayMakes11 for the dbs with millions of rows, what's the approximate size?

M

Max (@rozenmd)•4/5/24, 2:11 PM

2ish GB

JjayMakes11 i've seen postgres dbs at 300 to 400 GB... that's my only context for this... ma...

M

Max (@rozenmd)•4/5/24, 2:11 PM

Workers also supports big classic postgres DBs - using Hyperdrive

J

jayMakes11OP•4/5/24, 2:13 PM

This is all very helpful. Thank you.

Yeah, no base64 images or anything, mostly just simple String and such.

JjayMakes11 i've seen postgres dbs at 300 to 400 GB... that's my only context for this... ma...

M

Max (@rozenmd)•4/5/24, 2:20 PM

we picked sqlite particularly because we could load millions of dbs across the world really quickly - that pattern breaks down for extremely large, monolith-style DBs

MMax (@rozenmd)we picked sqlite particularly because we could load millions of dbs across the w...

J

jayMakes11OP•4/5/24, 2:20 PM

makes sense... trying to work with 300-400 GB DBs is also just flat-out painful...

J

jayMakes11OP•4/5/24, 2:21 PM

(from my experience at least)

TTony Leung It's hard to maintain a large mysql cluster for the edge. Even if they do, the p...

A

adjlsncm•4/5/24, 3:12 PM

this makes sense, thanks

A

adjlsncm•4/5/24, 4:01 PM

is there a guide on setting up d1 with prisma edge in local development?

Aadjlsncm is there a guide on setting up d1 with prisma edge in local development?

M

Matt Silverlock•4/5/24, 4:14 PM

Did you see https://developers.cloudflare.com/d1/tutorials/d1-and-prisma-orm/

Cloudflare Docs

Query D1 using Prisma ORM · Cloudflare D1 docs

Prisma ORM is a next-generation JavaScript and TypeScript ORM that unlocks a new level of developer experience when working with databases thanks to …

R

rufi•4/5/24, 4:16 PM

I recently tried D1 and i had bad results (wall times exploded). Worked fine with KV however so looking at possible alternatives. I built a reverse proxy that needs to be sync with a database from time to time. Interested about your experiences

A

adjlsncm•4/5/24, 4:23 PM

i dont understand this part, in next you access the env through process.env, so how do you initialize the prisma client?

Screenshot_2024-04-05_at_12.23.04_PM.png

A

adjlsncm•4/5/24, 4:24 PM

for context

Screenshot_2024-04-05_at_12.24.18_PM.png

A

adjlsncm•4/5/24, 4:48 PM

oh

Screenshot_2024-04-05_at_12.48.31_PM.png

Aadjlsncm oh

J

jayMakes11OP•4/5/24, 5:13 PM

really looking forward to this hopefully one day not being a limitation

JjayMakes11 really looking forward to this hopefully one day not being a limitation

M

Max (@rozenmd)•4/5/24, 5:50 PM

my workaround is a separate worker - locally use the localhost URL, in prod, use that workers prod URL

A

adjlsncm•4/5/24, 5:55 PM

so if i understand correctly workers are their own thing and pages is its own thing, next-on-pages specifically is pages but it turns routes into workers ?

H

Hannes•4/5/24, 7:07 PM

Is the

Maximum bound parameters per query

Maximum bound parameters per query

limit of 100 going to be increased eventually? Just ran into a bug with Prisma, because that affects basically every query with relations as soon as you have more than 100 rows in a table

A

adjlsncm•4/5/24, 8:28 PM

has anybody set up a next-pages app with D1 & Prisma? would love to see how you set up a basic route to read the request.env because i cant at all

Aadjlsncm has anybody set up a next-pages app with D1 & Prisma? would love to see how you ...

N

Nob•4/5/24, 8:28 PM

https://developers.cloudflare.com/d1/tutorials/d1-and-prisma-orm/

Cloudflare Docs

Query D1 using Prisma ORM · Cloudflare D1 docs

Prisma ORM is a next-generation JavaScript and TypeScript ORM that unlocks a new level of developer experience when working with databases thanks to …

A

adjlsncm•4/5/24, 8:33 PM

A

adjlsncm•4/5/24, 8:45 PM

M

Marcel Overdijk•4/5/24, 9:22 PM

@Nob I was trying this evening to get Prisma.io with D1 working.
Locally (wrangler dev) it works for me, but when deploying to CF my build starts failing...:

23:08:22.672    Using v2 root directory strategy
23:08:22.696    Success: Finished cloning repository files
23:08:24.728    wrangler.toml config file detected
23:08:25.838    Failed: an internal error occurred. If this continues, contact support: https://cfl.re/3WgEyrH

23:08:22.672    Using v2 root directory strategy
23:08:22.696    Success: Finished cloning repository files
23:08:24.728    wrangler.toml config file detected
23:08:25.838    Failed: an internal error occurred. If this continues, contact support: https://cfl.re/3WgEyrH

23:08:22.672    Using v2 root directory strategy
23:08:22.696    Success: Finished cloning repository files
23:08:24.728    wrangler.toml config file detected
23:08:25.838    Failed: an internal error occurred. If this continues, contact support: https://cfl.re/3WgEyrH

23:08:22.672    Using v2 root directory strategy
23:08:22.696    Success: Finished cloning repository files
23:08:24.728    wrangler.toml config file detected
23:08:25.838    Failed: an internal error occurred. If this continues, contact support: https://cfl.re/3WgEyrH

Before adding Prisma I did several deployments without any issue.
Not sure what is going wrong, logs are not telling much unfortunately...

M

Marcel Overdijk•4/5/24, 9:27 PM

To give a bit more detail:
I'm integrating Prisma in my

./functions

./functions

API endpoints.
As documented here: https://developers.cloudflare.com/pages/functions/typescript/ I did this:

Then configure the runtime types by creating a functions/tsconfig.json file:

functions/tsconfig.json
{
  "compilerOptions": {
    "target": "esnext",
    "module": "esnext",
    "lib": ["esnext"],
    "types": ["@cloudflare/workers-types"]
  }
}

If you already have a tsconfig.json at the root of your project, you may wish to explicitly exclude the /functions directory to avoid conflicts. To exclude the /functions directory:

tsconfig.json
{
  "include": ["src/**/*"],
  "exclude": ["functions/**/*"],
  "compilerOptions": {

  }
}

Then configure the runtime types by creating a functions/tsconfig.json file:

functions/tsconfig.json
{
  "compilerOptions": {
    "target": "esnext",
    "module": "esnext",
    "lib": ["esnext"],
    "types": ["@cloudflare/workers-types"]
  }
}

If you already have a tsconfig.json at the root of your project, you may wish to explicitly exclude the /functions directory to avoid conflicts. To exclude the /functions directory:

tsconfig.json
{
  "include": ["src/**/*"],
  "exclude": ["functions/**/*"],
  "compilerOptions": {

  }
}

Then configure the runtime types by creating a functions/tsconfig.json file:

functions/tsconfig.json
{
  "compilerOptions": {
    "target": "esnext",
    "module": "esnext",
    "lib": ["esnext"],
    "types": ["@cloudflare/workers-types"]
  }
}

If you already have a tsconfig.json at the root of your project, you may wish to explicitly exclude the /functions directory to avoid conflicts. To exclude the /functions directory:

tsconfig.json
{
  "include": ["src/**/*"],
  "exclude": ["functions/**/*"],
  "compilerOptions": {

  }
}

Then configure the runtime types by creating a functions/tsconfig.json file:

functions/tsconfig.json
{
  "compilerOptions": {
    "target": "esnext",
    "module": "esnext",
    "lib": ["esnext"],
    "types": ["@cloudflare/workers-types"]
  }
}

If you already have a tsconfig.json at the root of your project, you may wish to explicitly exclude the /functions directory to avoid conflicts. To exclude the /functions directory:

tsconfig.json
{
  "include": ["src/**/*"],
  "exclude": ["functions/**/*"],
  "compilerOptions": {

  }
}

I noticed that in my functions folder where I import these:

import { PrismaClient } from '@prisma/client';
import { PrismaD1 } from '@prisma/adapter-d1';

import { PrismaClient } from '@prisma/client';
import { PrismaD1 } from '@prisma/adapter-d1';

VScode complains about

Cannot find module '@prisma/client'. Did you mean to set the 'moduleResolution' option to 'nodenext', or to add aliases to the 'paths' option?ts(2792)

Cannot find module '@prisma/client'. Did you mean to set the 'moduleResolution' option to 'nodenext', or to add aliases to the 'paths' option?ts(2792)

Cannot find module '@prisma/client'. Did you mean to set the 'moduleResolution' option to 'nodenext', or to add aliases to the 'paths' option?ts(2792)

Cannot find module '@prisma/client'. Did you mean to set the 'moduleResolution' option to 'nodenext', or to add aliases to the 'paths' option?ts(2792)

M

Marcel Overdijk•4/5/24, 9:27 PM

Not sure if that is relates, as locally it works

MMarcel Overdijk @Nob I was trying this evening to get Prisma.io with D1 working. Locally (wrangl...

A

adjlsncm•4/5/24, 9:27 PM

did you develop locally on next pages

M

Marcel Overdijk•4/5/24, 9:29 PM

I've run the Pages functions locally with

npx wrangler dev

npx wrangler dev

I've also create the database locally, including migrations (which I executed both locally and remote)

A

adjlsncm•4/5/24, 9:31 PM

how are you accessing the request.env.DB ?

A

adjlsncm•4/5/24, 9:31 PM

i cant seem to access the database binding in local development

M

Marcel Overdijk•4/5/24, 9:33 PM

import { PrismaClient } from '@prisma/client';
import { PrismaD1 } from '@prisma/adapter-d1';

export const onRequestGet: PagesFunction<Env> = async (context) => {
  const adapter = new PrismaD1(context.env.DB)
  const prisma = new PrismaClient({ adapter })

import { PrismaClient } from '@prisma/client';
import { PrismaD1 } from '@prisma/adapter-d1';

export const onRequestGet: PagesFunction<Env> = async (context) => {
  const adapter = new PrismaD1(context.env.DB)
  const prisma = new PrismaClient({ adapter })

import { PrismaClient } from '@prisma/client';
import { PrismaD1 } from '@prisma/adapter-d1';

export const onRequestGet: PagesFunction<Env> = async (context) => {
  const adapter = new PrismaD1(context.env.DB)
  const prisma = new PrismaClient({ adapter })

import { PrismaClient } from '@prisma/client';
import { PrismaD1 } from '@prisma/adapter-d1';

export const onRequestGet: PagesFunction<Env> = async (context) => {
  const adapter = new PrismaD1(context.env.DB)
  const prisma = new PrismaClient({ adapter })

M

Marcel Overdijk•4/5/24, 9:36 PM

@adjlsncm did you create the binding in wrangler.toml ? and did you start the local wrangler dev instance?

A

adjlsncm•4/5/24, 9:38 PM

im only running

npx wrangler pages dev .vercel/output/static --compatibility-flags="nodejs_compat" --port 8788

npx wrangler pages dev .vercel/output/static --compatibility-flags="nodejs_compat" --port 8788

A

adjlsncm•4/5/24, 9:39 PM

after building it with @Cloudflare/next-on-pages

M

Marcel Overdijk•4/5/24, 9:46 PM

I'm just running it like

npx wrangler pages dev

npx wrangler pages dev

but I think that makes no difference...
how does your

wrangler.toml

wrangler.toml

look?

K

kian•4/5/24, 9:48 PM

That doesn't define a D1 database for the

dev

dev

env, fwiw

K

kian•4/5/24, 9:48 PM

That shouldn't matter for Pages though

Kkian That doesn't define a D1 database for the `dev` env, fwiw

M

Marcel Overdijk•4/5/24, 9:57 PM

@adjlsncm and if you try something like:

npx wrangler d1 execute audition --command "SELECT * FROM _some_table_;" --local

npx wrangler d1 execute audition --command "SELECT * FROM _some_table_;" --local

does that work?

J

James•4/5/24, 10:08 PM

Browsers generally make 2 requests when they hit a page - one for the page itself, and then one for

/favicon.ico

/favicon.ico

J

James•4/5/24, 10:09 PM

checking the URL is generally a good solution

const url = new URL(request.url);
if (url.pathname === '/something') {
   // do your work
}

const url = new URL(request.url);
if (url.pathname === '/something') {
   // do your work
}

const url = new URL(request.url);
if (url.pathname === '/something') {
   // do your work
}

const url = new URL(request.url);
if (url.pathname === '/something') {
   // do your work
}

J

James•4/5/24, 10:09 PM

Or if you're doing something more complex, using a router like https://hono.dev/ or https://itty.dev/itty-router/ is recommended

for the dbs with millions of rows, what's the approximate size?

Similar Threads

Similar Threads

Similar Threads