What is the concern here? The 50K limit is not a hard limit. We communicate a number because often "

VVy Thanks for catching the async piece, I'll get that corrected. would also note th...

J

James•9/2/24, 6:37 PM

I hear you about "global replication" vs "global", but I'm not sure most people share that same conclusion Cloudflare does. Is my Postgres database I have hosted on some random server a "global database" just because I can access it globally? It's a little meaningless in terms of marketing, and I completely understand why it confuses people.

J

João Castro•9/2/24, 7:26 PM

Are there any news about supporting transactions in D1? Transactions are fundamental for building reliable applications, and even though D1 is in GA, we don't have this functionality.

JJoão Castro Are there any news about supporting transactions in D1? Transactions are fundame...

V

VyOP•9/2/24, 7:32 PM

explicit transactions are not in development right now. D1's batch() API does execute a SQL transaction - https://developers.cloudflare.com/d1/build-with-d1/d1-client-api/#batch-statements

Cloudflare Docs

Query D1 | Cloudflare D1 docs

D1 is compatible with most SQLite’s SQL convention since it leverages SQLite’s query engine. D1 client API allows you to interact with a D1 database from within a Worker.

O

one.coser•9/3/24, 9:28 AM

May I ask how to resolve this error ： D1_ERROR: D1's isolate exceeded its memory limit and was reset.

M

Martin•9/3/24, 11:41 AM

Pricing question: do upsert count as write and read each time or if nothing to update then only one read?

I have many update in one table and i wonder how to optimise, should i do 2 query myself, to prevent read or upsert do it already

R

razbomi•9/4/24, 12:05 AM

+1 on the increase of

Internal error in D1 storage caused object to be reset.

Internal error in D1 storage caused object to be reset.

Internal error in D1 storage caused object to be reset.

Internal error in D1 storage caused object to be reset. errors

I

IEatBeans•9/4/24, 3:04 AM

Im still seeing a ton of these D1 errors

D1_ERROR: Internal error in D1 storage caused object to be reset.

D1_ERROR: Internal error in D1 storage caused object to be reset.

D1_ERROR: Internal error in D1 storage caused object to be reset.

D1_ERROR: Internal error in D1 storage caused object to be reset.. They are happening on all of my queries and causing lots of 500 errors

J

Jacob•9/4/24, 4:43 AM

Hi, I am having troubble setting up drizzle with the local d1 database, when running

bun drizzle-kit migrate

bun drizzle-kit migrate

bun drizzle-kit migrate

bun drizzle-kit migrate i receive the following error:

$ bun db:migrate
$ drizzle-kit migrate
No config path provided, using default 'drizzle.config.ts'
Reading config file 'G:\GitHub\BuzzTrip\apps\web\drizzle.config.ts'
Using G:\GitHub\BuzzTrip\apps\web\.wrangler\state\v3\d1\miniflare-D1DatabaseObject\097dd93c38860e13c35b3676a4f13f4950f08ed9c239ec99db5716131b0e2718.sqlite
[⣷] applying migrations...Error: Can't find meta/_journal.json file
    at readMigrationFiles (G:\GitHub\BuzzTrip\node_modules\src\migrator.ts:41:9)
    at migrate (G:\GitHub\BuzzTrip\node_modules\src\libsql\migrator.ts:10:21)
    at migrateFn (G:\GitHub\BuzzTrip\node_modules\drizzle-kit\bin.cjs:72211:18)
    at Object.handler (G:\GitHub\BuzzTrip\node_modules\drizzle-kit\bin.cjs:83593:11)
    at async run (G:\GitHub\BuzzTrip\node_modules\drizzle-kit\bin.cjs:82064:7)
error: script "db:migrate" exited with code 1

$ bun db:migrate
$ drizzle-kit migrate
No config path provided, using default 'drizzle.config.ts'
Reading config file 'G:\GitHub\BuzzTrip\apps\web\drizzle.config.ts'
Using G:\GitHub\BuzzTrip\apps\web\.wrangler\state\v3\d1\miniflare-D1DatabaseObject\097dd93c38860e13c35b3676a4f13f4950f08ed9c239ec99db5716131b0e2718.sqlite
[⣷] applying migrations...Error: Can't find meta/_journal.json file
    at readMigrationFiles (G:\GitHub\BuzzTrip\node_modules\src\migrator.ts:41:9)
    at migrate (G:\GitHub\BuzzTrip\node_modules\src\libsql\migrator.ts:10:21)
    at migrateFn (G:\GitHub\BuzzTrip\node_modules\drizzle-kit\bin.cjs:72211:18)
    at Object.handler (G:\GitHub\BuzzTrip\node_modules\drizzle-kit\bin.cjs:83593:11)
    at async run (G:\GitHub\BuzzTrip\node_modules\drizzle-kit\bin.cjs:82064:7)
error: script "db:migrate" exited with code 1

$ bun db:migrate
$ drizzle-kit migrate
No config path provided, using default 'drizzle.config.ts'
Reading config file 'G:\GitHub\BuzzTrip\apps\web\drizzle.config.ts'
Using G:\GitHub\BuzzTrip\apps\web\.wrangler\state\v3\d1\miniflare-D1DatabaseObject\097dd93c38860e13c35b3676a4f13f4950f08ed9c239ec99db5716131b0e2718.sqlite
[⣷] applying migrations...Error: Can't find meta/_journal.json file
    at readMigrationFiles (G:\GitHub\BuzzTrip\node_modules\src\migrator.ts:41:9)
    at migrate (G:\GitHub\BuzzTrip\node_modules\src\libsql\migrator.ts:10:21)
    at migrateFn (G:\GitHub\BuzzTrip\node_modules\drizzle-kit\bin.cjs:72211:18)
    at Object.handler (G:\GitHub\BuzzTrip\node_modules\drizzle-kit\bin.cjs:83593:11)
    at async run (G:\GitHub\BuzzTrip\node_modules\drizzle-kit\bin.cjs:82064:7)
error: script "db:migrate" exited with code 1

$ bun db:migrate
$ drizzle-kit migrate
No config path provided, using default 'drizzle.config.ts'
Reading config file 'G:\GitHub\BuzzTrip\apps\web\drizzle.config.ts'
Using G:\GitHub\BuzzTrip\apps\web\.wrangler\state\v3\d1\miniflare-D1DatabaseObject\097dd93c38860e13c35b3676a4f13f4950f08ed9c239ec99db5716131b0e2718.sqlite
[⣷] applying migrations...Error: Can't find meta/_journal.json file
    at readMigrationFiles (G:\GitHub\BuzzTrip\node_modules\src\migrator.ts:41:9)
    at migrate (G:\GitHub\BuzzTrip\node_modules\src\libsql\migrator.ts:10:21)
    at migrateFn (G:\GitHub\BuzzTrip\node_modules\drizzle-kit\bin.cjs:72211:18)
    at Object.handler (G:\GitHub\BuzzTrip\node_modules\drizzle-kit\bin.cjs:83593:11)
    at async run (G:\GitHub\BuzzTrip\node_modules\drizzle-kit\bin.cjs:82064:7)
error: script "db:migrate" exited with code 1

I attach my drizzle.config.ts and wrangler.toml below.

Anyone have any ideas on how i could fix my error?

J

Jacob•9/4/24, 4:43 AM

drizzle.config.ts

import { defineConfig } from "drizzle-kit";
import fs from "node:fs";
import path from "node:path";

function getLocalD1DB() {
  try {
    const basePath = path.resolve(".wrangler");
    const dbFile = fs
      .readdirSync(basePath, { encoding: "utf-8", recursive: true })
      .find((f) => f.endsWith(".sqlite"));

    if (!dbFile) {
      throw new Error(`.sqlite file not found in ${basePath}`);
    }

    const url = path.resolve(basePath, dbFile);
    console.log(`Using ${url}`);
    return `file:${url}`;
  } catch (err) {
    console.log(`Error  ${err}`);
  }
}

export default defineConfig({
  dialect: "sqlite",
  schema: "./app/server/db/schema.ts",
  out: "./drizzle",
  ...(process.env.NODE_ENV === "production"
    ? {
        driver: "d1-http",
        dbCredentials: {
          accountId: process.env.CLOUDFLARE_D1_ACCOUNT_ID,
          databaseId: process.env.DATABASE,
          token: process.env.CLOUDFLARE_D1_API_TOKEN,
        },
      }
    : {
        dbCredentials: {
          url: getLocalD1DB(),
        },
      }),
});

import { defineConfig } from "drizzle-kit";
import fs from "node:fs";
import path from "node:path";

function getLocalD1DB() {
  try {
    const basePath = path.resolve(".wrangler");
    const dbFile = fs
      .readdirSync(basePath, { encoding: "utf-8", recursive: true })
      .find((f) => f.endsWith(".sqlite"));

    if (!dbFile) {
      throw new Error(`.sqlite file not found in ${basePath}`);
    }

    const url = path.resolve(basePath, dbFile);
    console.log(`Using ${url}`);
    return `file:${url}`;
  } catch (err) {
    console.log(`Error  ${err}`);
  }
}

export default defineConfig({
  dialect: "sqlite",
  schema: "./app/server/db/schema.ts",
  out: "./drizzle",
  ...(process.env.NODE_ENV === "production"
    ? {
        driver: "d1-http",
        dbCredentials: {
          accountId: process.env.CLOUDFLARE_D1_ACCOUNT_ID,
          databaseId: process.env.DATABASE,
          token: process.env.CLOUDFLARE_D1_API_TOKEN,
        },
      }
    : {
        dbCredentials: {
          url: getLocalD1DB(),
        },
      }),
});

import { defineConfig } from "drizzle-kit";
import fs from "node:fs";
import path from "node:path";

function getLocalD1DB() {
  try {
    const basePath = path.resolve(".wrangler");
    const dbFile = fs
      .readdirSync(basePath, { encoding: "utf-8", recursive: true })
      .find((f) => f.endsWith(".sqlite"));

    if (!dbFile) {
      throw new Error(`.sqlite file not found in ${basePath}`);
    }

    const url = path.resolve(basePath, dbFile);
    console.log(`Using ${url}`);
    return `file:${url}`;
  } catch (err) {
    console.log(`Error  ${err}`);
  }
}

export default defineConfig({
  dialect: "sqlite",
  schema: "./app/server/db/schema.ts",
  out: "./drizzle",
  ...(process.env.NODE_ENV === "production"
    ? {
        driver: "d1-http",
        dbCredentials: {
          accountId: process.env.CLOUDFLARE_D1_ACCOUNT_ID,
          databaseId: process.env.DATABASE,
          token: process.env.CLOUDFLARE_D1_API_TOKEN,
        },
      }
    : {
        dbCredentials: {
          url: getLocalD1DB(),
        },
      }),
});

import { defineConfig } from "drizzle-kit";
import fs from "node:fs";
import path from "node:path";

function getLocalD1DB() {
  try {
    const basePath = path.resolve(".wrangler");
    const dbFile = fs
      .readdirSync(basePath, { encoding: "utf-8", recursive: true })
      .find((f) => f.endsWith(".sqlite"));

    if (!dbFile) {
      throw new Error(`.sqlite file not found in ${basePath}`);
    }

    const url = path.resolve(basePath, dbFile);
    console.log(`Using ${url}`);
    return `file:${url}`;
  } catch (err) {
    console.log(`Error  ${err}`);
  }
}

export default defineConfig({
  dialect: "sqlite",
  schema: "./app/server/db/schema.ts",
  out: "./drizzle",
  ...(process.env.NODE_ENV === "production"
    ? {
        driver: "d1-http",
        dbCredentials: {
          accountId: process.env.CLOUDFLARE_D1_ACCOUNT_ID,
          databaseId: process.env.DATABASE,
          token: process.env.CLOUDFLARE_D1_API_TOKEN,
        },
      }
    : {
        dbCredentials: {
          url: getLocalD1DB(),
        },
      }),
});

toml

#:schema node_modules/wrangler/config-schema.json
name = "buzztrip"
compatibility_date = "2024-08-21"
compatibility_flags = ["nodejs_compat"]
pages_build_output_dir = ".vercel/output/static"

[[d1_databases]]
binding = "BUZZTRIP_DATABASE"
database_name = "BuzzTrip-db"
database_id = "<database-id>"
migrations_dir = "drizzle/migrations"

#:schema node_modules/wrangler/config-schema.json
name = "buzztrip"
compatibility_date = "2024-08-21"
compatibility_flags = ["nodejs_compat"]
pages_build_output_dir = ".vercel/output/static"

[[d1_databases]]
binding = "BUZZTRIP_DATABASE"
database_name = "BuzzTrip-db"
database_id = "<database-id>"
migrations_dir = "drizzle/migrations"

#:schema node_modules/wrangler/config-schema.json
name = "buzztrip"
compatibility_date = "2024-08-21"
compatibility_flags = ["nodejs_compat"]
pages_build_output_dir = ".vercel/output/static"

[[d1_databases]]
binding = "BUZZTRIP_DATABASE"
database_name = "BuzzTrip-db"
database_id = "<database-id>"
migrations_dir = "drizzle/migrations"

#:schema node_modules/wrangler/config-schema.json
name = "buzztrip"
compatibility_date = "2024-08-21"
compatibility_flags = ["nodejs_compat"]
pages_build_output_dir = ".vercel/output/static"

[[d1_databases]]
binding = "BUZZTRIP_DATABASE"
database_name = "BuzzTrip-db"
database_id = "<database-id>"
migrations_dir = "drizzle/migrations"

K

Keavon•9/4/24, 5:30 AM

I want to write a backend accounts API using Workers and D1, but using Rust instead of TypeScript. I see some ORMs listed at https://developers.cloudflare.com/d1/reference/community-projects/ for TS, but I don't see anything Rust related. Are there any I can look into? I'm basically very unsure how to dive into building this tech stack. It would also be nice to take advantage of Rust's ecosystem with libraries like Axum, is that feasible?

Rrazbomi +1 on the increase of `Internal error in D1 storage caused object to be reset.` ...

J

James•9/4/24, 11:34 AM

For those hitting this, and other similar errors, we’ve escalated and are told the team are actively investigating.
I am trying to get them to post something more official, but hopefully we’ll see a real resolution soon.

D

Dean•9/4/24, 12:50 PM

kind of wild to think this is a GA product but can’t get a status advisory posted

DDean kind of wild to think this is a GA product but can’t get a status advisory poste...

M

Max (@rozenmd)•9/4/24, 12:56 PM

https://www.cloudflarestatus.com/incidents/s8p372y81z28

Cloudflare D1 Issues

L

lucasp•9/4/24, 3:48 PM

Hi, I wanted to ask you a question: should I push the database_id to the repository? If not, how do I put it as an environment variable? I can't find it in the documentation.

[[env.dev.d1_databases]]
binding = "DB"
database_name = "d-dev"
database_id = "67d41931-140c-418b-xxxx-xxxxx"

[[env.dev.d1_databases]]
binding = "DB"
database_name = "d-dev"
database_id = "67d41931-140c-418b-xxxx-xxxxx"

[[env.dev.d1_databases]]
binding = "DB"
database_name = "d-dev"
database_id = "67d41931-140c-418b-xxxx-xxxxx"

[[env.dev.d1_databases]]
binding = "DB"
database_name = "d-dev"
database_id = "67d41931-140c-418b-xxxx-xxxxx"

Llucasp Hi, I wanted to ask you a question: should I push the database_id to the reposit...

C

Cyb3r-Jak3•9/4/24, 4:06 PM

The ID is safe to commit

CCyb3r-Jak3 The ID is safe to commit

L

lucasp•9/4/24, 4:07 PM

ty i thought it was sesitive data

Llucasp ty i thought it was sesitive data

C

Cyb3r-Jak3•9/4/24, 4:21 PM

Everything in the wrangler.toml is safe to commit. The only thing would be environment variables which are sensitive. Those should be stored as secrets

J

jasperagrante•9/5/24, 1:55 AM

Hi there! I'm doing a usage counter with max cap. What would be the best way to increment the counter that I can detect if it has reached the threshold without going over?

Jjasperagrante Hi there! I'm doing a usage counter with max cap. What would be the best way to ...

C

Cyb3r-Jak3•9/5/24, 2:39 AM

I would look at durable objects

J

jasperagrante•9/5/24, 3:41 AM

Thanks! I was try to avoid durable objects because it will double the work (durable object logic, durable object management). I was able to to it with sqlite trigger with

RAISE

RAISE

RAISE

RAISE

Ssurgery18 Ok, this is strange. I cannot do the triggers during migrations, but I can run a...

J

jasperagrante•9/5/24, 4:47 AM

Unforunately I've experience this same error as well. Had to execute trigger creation manually via wrangler.

MMax (@rozenmd)https://www.cloudflarestatus.com/incidents/s8p372y81z28

M

Max (@rozenmd)•9/5/24, 1:52 PM

We've closed the incident.

Here's what happened: we rolled out a fix for rare cases where D1 DBs weren't booting up successfully last week. This fix uncovered a bug from an update we released months ago, and caused a small percentage of requests to fail (roughly 1 error for every 22.5k requests) and return an error of

Internal error in D1 storage caused object to be reset

Internal error in D1 storage caused object to be reset

Internal error in D1 storage caused object to be reset

Internal error in D1 storage caused object to be reset.

We have rolled back the update from months ago, and the errors have stopped as of 2024-09-04 21:55 UTC

M

Mozzy•9/5/24, 2:17 PM

X

xiaolong01•9/5/24, 5:26 PM

Hey Cloudflare team,

I saw that the current limit is 10GB per database, but we need to store data for about 200 million users, which would require something like 200GB. With the 10GB limit, we can only fit data for around 10-20 million users.

Are there any plans to increase the limit, or is there any way to upgrade to larger storage? Just trying to figure out the best approach for our project.

Thanks in advance!

Xxiaolong01 Hey Cloudflare team, I saw that the current limit is 10GB per database, but we ...

M

Max (@rozenmd)•9/5/24, 5:29 PM

If you're set on storing that data in a single database, we'd recommend using Hyperdrive instead

M

Max (@rozenmd)•9/5/24, 7:42 PM

D1 is built around Cloudflare's network, which is really efficient at loading lots of small files really quickly - we'd offer bigger DBs but the cold starts would be brutal

GGustavo Do you think it's *technically possible* to raise the limit to hundreds of GBs i...

M

Max (@rozenmd)•9/5/24, 7:46 PM

Unlikely, Hyperdrive is a much better fit for that

M

Max (@rozenmd)•9/5/24, 7:47 PM

With the 50k DBs D1 starts with, we're also building a means of dynamic binding as many DBs as you want (and related tooling) to make it easier to manage, but that's a new way of building software that folks might not be ready for

G

Gagan Suie•9/6/24, 3:57 AM

is there anything wrong with code like this?

/**
 * Update milestone by milestoneId
 * @param request
 * @param env
 * @returns
 */
const updateMilestone = async (request: Request, env: any) => {
    try {
        const params = new URL(request.url).searchParams
        const milestoneId = parseInt(params.get('milestoneId') || '')
        const milestoneData: any = await request.json()
        const fields: string[] = []
        const values: any[] = []
        let index = 1
        for (const [key, value] of Object.entries(milestoneData)) {
            if (value !== undefined) {
                fields.push(`${key} = ?${index}`)
                values.push(value)
                index++
            }
        }
        if (fields.length === 0) {
            return new Response(JSON.stringify({ error: 'No fields to update' }), { status: 400 })
        }
        values.push(milestoneId)
        const query = `UPDATE milestone SET ${fields.join(', ')}, updatedAt = CURRENT_TIMESTAMP WHERE _id = ?${index}`
        const { success } = await env.DB.prepare(query)
            .bind(...values)
            .run()
        return new Response(JSON.stringify({ success }), { status: 200 })
    } catch (err: any) {
        return new Response(JSON.stringify(err), { status: 500 })
    }
}

/**
 * Update milestone by milestoneId
 * @param request
 * @param env
 * @returns
 */
const updateMilestone = async (request: Request, env: any) => {
    try {
        const params = new URL(request.url).searchParams
        const milestoneId = parseInt(params.get('milestoneId') || '')
        const milestoneData: any = await request.json()
        const fields: string[] = []
        const values: any[] = []
        let index = 1
        for (const [key, value] of Object.entries(milestoneData)) {
            if (value !== undefined) {
                fields.push(`${key} = ?${index}`)
                values.push(value)
                index++
            }
        }
        if (fields.length === 0) {
            return new Response(JSON.stringify({ error: 'No fields to update' }), { status: 400 })
        }
        values.push(milestoneId)
        const query = `UPDATE milestone SET ${fields.join(', ')}, updatedAt = CURRENT_TIMESTAMP WHERE _id = ?${index}`
        const { success } = await env.DB.prepare(query)
            .bind(...values)
            .run()
        return new Response(JSON.stringify({ success }), { status: 200 })
    } catch (err: any) {
        return new Response(JSON.stringify(err), { status: 500 })
    }
}

/**
 * Update milestone by milestoneId
 * @param request
 * @param env
 * @returns
 */
const updateMilestone = async (request: Request, env: any) => {
    try {
        const params = new URL(request.url).searchParams
        const milestoneId = parseInt(params.get('milestoneId') || '')
        const milestoneData: any = await request.json()
        const fields: string[] = []
        const values: any[] = []
        let index = 1
        for (const [key, value] of Object.entries(milestoneData)) {
            if (value !== undefined) {
                fields.push(`${key} = ?${index}`)
                values.push(value)
                index++
            }
        }
        if (fields.length === 0) {
            return new Response(JSON.stringify({ error: 'No fields to update' }), { status: 400 })
        }
        values.push(milestoneId)
        const query = `UPDATE milestone SET ${fields.join(', ')}, updatedAt = CURRENT_TIMESTAMP WHERE _id = ?${index}`
        const { success } = await env.DB.prepare(query)
            .bind(...values)
            .run()
        return new Response(JSON.stringify({ success }), { status: 200 })
    } catch (err: any) {
        return new Response(JSON.stringify(err), { status: 500 })
    }
}

/**
 * Update milestone by milestoneId
 * @param request
 * @param env
 * @returns
 */
const updateMilestone = async (request: Request, env: any) => {
    try {
        const params = new URL(request.url).searchParams
        const milestoneId = parseInt(params.get('milestoneId') || '')
        const milestoneData: any = await request.json()
        const fields: string[] = []
        const values: any[] = []
        let index = 1
        for (const [key, value] of Object.entries(milestoneData)) {
            if (value !== undefined) {
                fields.push(`${key} = ?${index}`)
                values.push(value)
                index++
            }
        }
        if (fields.length === 0) {
            return new Response(JSON.stringify({ error: 'No fields to update' }), { status: 400 })
        }
        values.push(milestoneId)
        const query = `UPDATE milestone SET ${fields.join(', ')}, updatedAt = CURRENT_TIMESTAMP WHERE _id = ?${index}`
        const { success } = await env.DB.prepare(query)
            .bind(...values)
            .run()
        return new Response(JSON.stringify({ success }), { status: 200 })
    } catch (err: any) {
        return new Response(JSON.stringify(err), { status: 500 })
    }
}

G

Gagan Suie•9/6/24, 3:58 AM

im specifically referring to the update query

GGagan Suie is there anything wrong with code like this? ```/** * Update milestone by mile...

M

Max (@rozenmd)•9/6/24, 4:53 AM

apart from the SQL injection?

G

Gagan Suie•9/6/24, 4:54 AM

Yes that's what I was worried about. Is there a better approach to writing something like this?

M

Max (@rozenmd)•9/6/24, 5:01 AM

nothing that comes to mind (its 7am here, brain still booting) other than strictly validating that the inbound fields are what you expect via zod (+ it integrates with hono: https://hono.dev/docs/guides/validation#zod-validator-middleware)

Validation - Hono

Ultrafast web framework for Cloudflare Workers, Fastly Compute, Deno, Bun, Vercel, Node.js, and others. Fast, but not only fast.

MMax (@rozenmd)nothing that comes to mind (its 7am here, brain still booting) other than strict...

G

Gagan Suie•9/6/24, 5:23 AM

We use @cloudflare/itty-router-openapi

神神龙出海 Click to see attachment

神

神龙出海•9/6/24, 6:02 AM

@Max (@rozenmd) Brother, I have this problem again in my database

神

神龙出海•9/6/24, 6:03 AM

Please help me check the problem, which directly causes my product to be unavailable

R

Rubi•9/6/24, 6:11 AM

hi umm, I got an error when exporting D1 database

✘ [ERROR] near "limit": syntax error at offset 40: SQLITE_ERROR

✘ [ERROR] near "limit": syntax error at offset 40: SQLITE_ERROR

✘ [ERROR] near "limit": syntax error at offset 40: SQLITE_ERROR

✘ [ERROR] near "limit": syntax error at offset 40: SQLITE_ERROR

R

Rubi•9/6/24, 6:11 AM

command:

npx wrangler d1 export db-name --remote --output ./database.sql

npx wrangler d1 export db-name --remote --output ./database.sql

npx wrangler d1 export db-name --remote --output ./database.sql

npx wrangler d1 export db-name --remote --output ./database.sql

神神龙出海 Please help me check the problem, which directly causes my product to be unavail...

M

Max (@rozenmd)•9/6/24, 7:33 AM

Are you seeing intermittent errors or is it completely unavailable?

神

神龙出海•9/6/24, 7:34 AM

There have been a lot of intermittent errors in the past few days, and they went away after a while just now.

神神龙出海 There have been a lot of intermittent errors in the past few days, and they went...

M

Max (@rozenmd)•9/6/24, 7:34 AM

I'll have the team investigate deeper when they get online (US hours)

神

神龙出海•9/6/24, 7:35 AM

Okay, please solve this problem completely as soon as possible, otherwise D1 will be very unreliable as a production environment database.

P

petar•9/6/24, 10:43 AM

Hi folks. My team and I are starting to use D1 where we need to have one simple table and do some simple checks from a CF worker. The way we want to populate the database (regularrly, every two weeks or so) is from a Rails app, where the app downloads a CSV file, process the data and insert/update the D1 table. Pretty straight forward, but the issue we are facing is with the CF API as it's not clear how to insert the new data and populate the table.
In the POST body there are 3 possible objects that we can pass: https://developers.cloudflare.com/api/operations/cloudflare-d1-import-database but it's not clear what we should do next, because the response always have an "upload_url" which is not clear if we need to use that URL for something or not. Is there an example, even better if it's in Ruby, so we can test it? Or a written SOP or something of how should the whole process go.

Thanks in advance!

Cloudflare API Documentation

Interact with Cloudflare's products and services via the Cloudflare API

M

matthew•9/6/24, 12:38 PM

Do you have to use the REST API? Might be easier to use the D1 API in Workers directly.

M

matthew•9/6/24, 1:07 PM

Question: can I get raw results when using D1 batch()? It seems there is no way of avoiding the cost of turning everything into JSON, if I want the perf benefits of batching queries.

Mmatthew Question: can I get raw results when using D1 batch()? It seems there is no way ...

H

Hard@Work•9/6/24, 1:08 PM

Not sure I understand what you mean by "raw results"?

What is the concern here? The 50K limit is not a hard limit. We communicate a number because often "

Similar Threads

Similar Threads

Similar Threads