Should be complete now. Please let me know if you still get that validation error.

Llambrospetrou Should be complete now. Please let me know if you still get that validation erro...

D

dimsid•3/10/25, 5:20 PM

Thanks a lot for all your support! it is working now

LLeesy Is anyone seeing random errors on D1? I have a DB bound to a worker and I keep g...

L

lambrospetrouOP•3/11/25, 11:13 AM

Make sure you don't have any typos when querying the table. In the UI console run

select * from sqlite_master;

select * from sqlite_master;

select * from sqlite_master;

select * from sqlite_master; and see if the output contains the table you expect to be there.

L

lambrospetrouOP•3/11/25, 11:31 AM

This is not a bug that can happen randomly. The query must be wrong in the worker that fails. Either the table always fails to be found, or never.

S

Sveinn•3/11/25, 3:54 PM

Hey,

A few weeks ago, my Cloudflare account started racking up a massive bill. I reached out to support to figure out what was going on, but it took around 20 days to get a response. The reply just said the billing was correct but didn’t explain what was actually causing the charges. I asked for clarification about a week ago but still haven’t gotten an answer.

After digging into it myself, I found that a simple uptime check my load balancer does every minute was somehow pulling tens of thousands of rows each time, leading to crazy charges. I’ve stopped the uptime check, but now I’m stuck with a credit card bill of thousands of dollars for my small hobby project.

Is there any way to get in touch with support to discuss these charges? @lambrospetrou

SSveinn Hey, A few weeks ago, my Cloudflare account started racking up a massive bill. ...

V

Vy•3/11/25, 4:30 PM

@Sveinn can you DM me your support case?

C

cosbgn•3/12/25, 1:38 PM

I think this graph is off or weird

C

cosbgn•3/12/25, 1:38 PM

by db size is about 7mb, but it adds 7mb each day

C

cosbgn•3/12/25, 1:39 PM

like it shows like it would now be 80mb, I think it should only add the delta daily

SSveinn Hey, A few weeks ago, my Cloudflare account started racking up a massive bill. ...

C

cosbgn•3/12/25, 1:40 PM

I've been begging cloudflare to add a max-spend limit since forever

Ccosbgn I've been begging cloudflare to add a max-spend limit since forever

S

Sveinn•3/12/25, 1:58 PM

Spend-limit would be awesome

S

Spuckwaffel•3/13/25, 1:46 PM

any way to export the DB, make modifications and then reload the DB? Have to alter a table and adding a foreign key constraint and cant delete the table

K

Kenny•3/13/25, 3:34 PM

I have two workers that bind into the same D1 database:

...
[[d1_databases]]
binding = "FIRESTORM_DB"
database_name = "firestorm-db"
database_id = "c020574a-5623-407b-be0c-cd192bab9545"
migrations_dir = "../d1/migrations"

...
[[d1_databases]]
binding = "FIRESTORM_DB"
database_name = "firestorm-db"
database_id = "c020574a-5623-407b-be0c-cd192bab9545"
migrations_dir = "../d1/migrations"

...
[[d1_databases]]
binding = "FIRESTORM_DB"
database_name = "firestorm-db"
database_id = "c020574a-5623-407b-be0c-cd192bab9545"
migrations_dir = "../d1/migrations"

...
[[d1_databases]]
binding = "FIRESTORM_DB"
database_name = "firestorm-db"
database_id = "c020574a-5623-407b-be0c-cd192bab9545"
migrations_dir = "../d1/migrations"

I have set the database id to be the same for both. However, when I run this locally for testing and start up one worker and have it add data to the db the other worker has no recollection of it. Is there a fix anyone knows about?

K

Kenny•3/13/25, 3:36 PM

I should also mention that I went into each .wrangler/state/d1 to check the sqlite data directly and it seems that each one spins up it's own sqlite db for local testing. I looked through the docs and haven't found anything that could solve it yet.

KKenny I have two workers that bind into the same D1 database: ``` ... [[d1_databases]]...

L

lambrospetrouOP•3/13/25, 4:01 PM

Have you tried the

--persist-to

--persist-to

--persist-to

--persist-to option? https://developers.cloudflare.com/d1/wrangler-commands/ and https://developers.cloudflare.com/workers/wrangler/commands/#d1:~:text=Minify%20the%20Worker.-,%2D%2Dpersist%2Dto,-string

K

Kenny•3/13/25, 4:02 PM

can that be specified in the wrangler.toml?

K

Kenny•3/13/25, 4:03 PM

bc i'm not running

wrangler d1 execute

wrangler d1 execute

wrangler d1 execute

wrangler d1 execute in my workers. Or maybe is it a config when I bind the like ENV.DB.execute?

KKenny bc i'm not running `wrangler d1 execute` in my workers. Or maybe is it a config ...

L

lambrospetrouOP•3/13/25, 4:06 PM

You use it with

wrangler dev

wrangler dev

K

Kenny•3/13/25, 4:06 PM

that's great, i'll test it rn

K

Kenny•3/13/25, 4:10 PM

worked perfectly, thanks!

C

cj•3/15/25, 12:09 PM

I am working on an app and I plan to use a durable object per team (group of users). For this I'll use the team-id as the unique name for instance

const id = this.env.MY_DURABLE_OBJECT.idFromName(teamId);
const stub = this.env.MY_DURABLE_OBJECT.get(id);

const id = this.env.MY_DURABLE_OBJECT.idFromName(teamId);
const stub = this.env.MY_DURABLE_OBJECT.get(id);

const id = this.env.MY_DURABLE_OBJECT.idFromName(teamId);
const stub = this.env.MY_DURABLE_OBJECT.get(id);

const id = this.env.MY_DURABLE_OBJECT.idFromName(teamId);
const stub = this.env.MY_DURABLE_OBJECT.get(id);

But how should I manage auth? Shouldn't auth hit a single DB?

Should I be using D1 for tables like - users, teams, etc (other global tables) and DO for tables specific to the team?

Ccj I am working on an app and I plan to use a durable object per team (group of use...

J

James•3/15/25, 12:58 PM

I would personally stick to and recommend something more traditional and external for auth and global info. Postgres + Hyperdive for example works great.

Cloudflare’s recommendation to build horizontally with D1 and DO SQL just doesn’t really work in my experience in the real world, coupled with the reliability issues and support concerns with D1: https://www.reddit.com/r/CloudFlare/s/4D9AeoAHm0

JJames I would personally stick to and recommend something more traditional and externa...

C

cj•3/15/25, 1:06 PM

Thank you. I didn't see anyone building auth on top of D1. And so the question.

J

James•3/15/25, 3:52 PM

I use a central db for auth, but they only need to hit that when the session expires, as well I only store data there for central lookups. Sessions are kept in DO per user, bypassing central db for a faster request avg.

W

will smith•3/16/25, 12:51 AM

status dashboard says no, but just for my sanity there's no known issue with d1 is there?
getting sudden huge spikes in latency (47,000ms) for very simple requests which were fine previously

    const findActivityStartTime = performance.now();
    const activity = await prisma.activity.findFirst({
      where: {
        id: activityId,
        goalId: goalId,
        goal: {
          userId: user.id,
        },
      },
    });
    const findActivityEndTime = performance.now();
    console.log(
      `Time to find activity: ${findActivityEndTime - findActivityStartTime}ms`
    );

    const findActivityStartTime = performance.now();
    const activity = await prisma.activity.findFirst({
      where: {
        id: activityId,
        goalId: goalId,
        goal: {
          userId: user.id,
        },
      },
    });
    const findActivityEndTime = performance.now();
    console.log(
      `Time to find activity: ${findActivityEndTime - findActivityStartTime}ms`
    );

    const findActivityStartTime = performance.now();
    const activity = await prisma.activity.findFirst({
      where: {
        id: activityId,
        goalId: goalId,
        goal: {
          userId: user.id,
        },
      },
    });
    const findActivityEndTime = performance.now();
    console.log(
      `Time to find activity: ${findActivityEndTime - findActivityStartTime}ms`
    );

    const findActivityStartTime = performance.now();
    const activity = await prisma.activity.findFirst({
      where: {
        id: activityId,
        goalId: goalId,
        goal: {
          userId: user.id,
        },
      },
    });
    const findActivityEndTime = performance.now();
    console.log(
      `Time to find activity: ${findActivityEndTime - findActivityStartTime}ms`
    );

I don't actually see the requests in the metrics page

K

Kotkoroid•3/17/25, 12:23 PM

Hi, how is it looking with stored procedures? https://blog.cloudflare.com/whats-new-with-d1/#sneak-peek-how-were-bringing-javascript-transactions-to-d1

I would like to write some JS code inside SQL database thus I can't use

batch()

batch()

batch()

batch() but I cannot find any other resources than in this blog post.

K

Kotkoroid•3/17/25, 12:24 PM

Ok, I found this: https://community.cloudflare.com/t/d1-stored-procedures-can-not-use/649207/2

K

Kotkoroid•3/17/25, 12:25 PM

What is the current state in D1 to solve this?

KKotkoroid Hi, how is it looking with stored procedures? https://blog.cloudflare.com/whats-...

L

lambrospetrouOP•3/17/25, 1:59 PM

Not released, and no way to do that now. Ideally, try to design your queries not to need transactions interleaved with JS code.

F

flowwise•3/17/25, 3:31 PM

was anyone able to use Prisma + D1? My app works locally but when I deploy it to Cloudflare i'm getting the error: fs.readdir is not implemented yet. Why would Prisma need access to the filesystem if it is configured to connect to D1? Its a real showstopper for me. I guess i will have to self-host the app if there is no solution.

B

Boetro•3/17/25, 4:52 PM

Hi! I'm looking for some docs on how d1 handles a colo going doing? If the location housing my d1 database goes down am I just hosed until it comes back up or is their some kind of built in failover?

Z

Zack•3/17/25, 4:53 PM

Hey, I recently moved some of our infra from KV to D1, with a database situated in every location to be nearer to the user while awaiting global replication. This was working great, however after updating that data through our warming process (import all new data into a tmp database, rename old database, rename new database to old database name, drop old database), the databases all appear to be experiencing significant random latency issues. In our monitoring I can see average duration to call D1 jumping from ~20-50ms depending on worker location to some queries hanging for multiple seconds up to a minute. I'm wondering if someone can help me investigate what is going on here. Beyond completely recreating all these databases as part of our warming process I am not sure what the option is here. Our database is ~2GB in size and has 3.24 million rows. I am wondering if there's something going on internally like DO resets that is causing this behaviour. I've attached an example graph where you can see the moment the warming happened and the duration metrics afterwards. This is from the HKG PoP and with a D1 instance in APAC

Z

Zack•3/17/25, 4:55 PM

BBoetro Hi! I'm looking for some docs on how d1 handles a colo going doing? If the locat...

L

lambrospetrouOP•3/17/25, 6:59 PM

D1 is built ontop of Durable Objects (DO). It's true that once the colo hosting the Durable Object for your database there will be some downtime. But, the Durable Objects platform will move your database DO on a healthy colo/server as soon as the faulty colo is detected by the automation and taken out of the network. Depending on how severe the issue is this move could take seconds or a few minutes. Usually there shouldn't be much of a downtime for this reason.

Unfortunately, most of the errors you will encounter are transient and sporadic, not a whole colo going down. In those cases, making your queries idempotent and having some retries will greatly help.

ZZack Hey, I recently moved some of our infra from KV to D1, with a database situated ...

L

lambrospetrouOP•3/17/25, 7:02 PM

You need to provide us database IDs to be able to look anything for you.
Also, some details on volume, queries, how much data are you returning each time, etc are helpful.

This is from the HKG PoP and with a D1 instance in APAC

APAC is a huge region and you could be 100-200ms away from your database unfortunately. Having said that anything more than a few hundreds of ms even in those cases is a bigger unexpected issue, either with the responses getting back to your worker or something else.

Also, I assume you are using the Workers binding right? Not the REST API.

Llambrospetrou D1 is built ontop of Durable Objects (DO). It's true that once the colo hosting ...

B

Boetro•3/17/25, 7:03 PM

Thanks for the response exactly what I was looking for!

Llambrospetrou You need to provide us database IDs to be able to look anything for you. Also, s...

Z

Zack•3/17/25, 7:17 PM

Just heading back to my office, will get those to you shortly. Will throw them in a thread

SSaku75 Hey everyone, just a curious question, that i have been searching for an answer ...

L

lambrospetrouOP•3/17/25, 8:44 PM

If you need more than that, it's better to use Durable Objects directly, really. At least, at the current state of D1.

L

Ludo | Txtx•3/17/25, 10:05 PM

howdy! is it possible to have a worker W1 passing its binding to a Service worker W2?
I'd like to have to have a pool of workers, all bound to different D1 databases, being able to delegate the execution of the same logic to one worker (ideally a durable object) that I can upgrade easily

LLudo | Txtx howdy! is it possible to have a worker W1 passing its binding to a Service worke...

I

Isaac McFadyen•3/17/25, 10:06 PM

?crossposting

S

SuperHelpflare•3/17/25, 10:06 PM

Please do not post your question in multiple channels/post it multiple times per the rules at #

welcome-and-rules. It creates confusion for people trying to help you and doesn't get your issue or question solved any faster.

IIsaac McFadyen ?crossposting

L

Ludo | Txtx•3/17/25, 10:09 PM

I initially posted in workers-for-platform, which I don't think was the right one\

W

will smith•3/18/25, 1:43 AM

Any update on this? the db id should be in the metrics link
d1-database

P

Panoptikym•3/18/25, 2:42 AM

#d1-database

Should be complete now. Please let me know if you still get that validation error.

Similar Threads