Hi, I'm using triggers and encounter an "internal error" that times out after 30 seconds when the ab

Hi, I'm using triggers and encounter an "internal error" that times out after 30 seconds when the abort occurs. It's not a massive problem, but it would be nice if this was fixed.

The trigger:

CREATE TRIGGER deduct_coins_on_consumable
BEFORE INSERT ON ActiveConsumables
  BEGIN
    UPDATE accounts
    SET pseudoCoins = pseudoCoins - (SELECT cost FROM Consumables WHERE id = NEW.consumableId)
    WHERE
      id = NEW.accountId 
        AND
      pseudoCoins >= (SELECT cost FROM Consumables WHERE id = NEW.consumableId);

  SELECT RAISE(ROLLBACK, 'Insufficient coins') WHERE changes() = 0;
END

CREATE TRIGGER deduct_coins_on_consumable
BEFORE INSERT ON ActiveConsumables
  BEGIN
    UPDATE accounts
    SET pseudoCoins = pseudoCoins - (SELECT cost FROM Consumables WHERE id = NEW.consumableId)
    WHERE
      id = NEW.accountId 
        AND
      pseudoCoins >= (SELECT cost FROM Consumables WHERE id = NEW.consumableId);

  SELECT RAISE(ROLLBACK, 'Insufficient coins') WHERE changes() = 0;
END

CREATE TRIGGER deduct_coins_on_consumable
BEFORE INSERT ON ActiveConsumables
  BEGIN
    UPDATE accounts
    SET pseudoCoins = pseudoCoins - (SELECT cost FROM Consumables WHERE id = NEW.consumableId)
    WHERE
      id = NEW.accountId 
        AND
      pseudoCoins >= (SELECT cost FROM Consumables WHERE id = NEW.consumableId);

  SELECT RAISE(ROLLBACK, 'Insufficient coins') WHERE changes() = 0;
END

CREATE TRIGGER deduct_coins_on_consumable
BEFORE INSERT ON ActiveConsumables
  BEGIN
    UPDATE accounts
    SET pseudoCoins = pseudoCoins - (SELECT cost FROM Consumables WHERE id = NEW.consumableId)
    WHERE
      id = NEW.accountId 
        AND
      pseudoCoins >= (SELECT cost FROM Consumables WHERE id = NEW.consumableId);

  SELECT RAISE(ROLLBACK, 'Insufficient coins') WHERE changes() = 0;
END

the internal error occurs when coins < cost.

"database_name": "discord-oauth",
"database_id": "2bd61136-7eb6-4b5d-82ce-f3405294398d"

KhafraOP•2/2/25, 2:34 AM

afaict the trigger works fine, other than it timing out/taking 30 seconds to throw

KKhafra Hi, I'm using triggers and encounter an "internal error" that times out after 30...

Isaac McFadyen•2/2/25, 3:41 AM

D1 has no manual transactions so I expect the error is a result of you trying to manually roll back (unsure though)

KhafraOP•2/2/25, 4:11 AM

Right.. I'll see if RAISE(ABORT, ...) has the same issues.

KhafraOP•2/2/25, 4:12 AM

I think maybe there is still something to fix with the internal error I was getting though

ikiryo•2/2/25, 3:19 PM

Hi, I have followed the tutorial here - https://www.prisma.io/docs/orm/overview/databases/cloudflare-d1#3-generate-sql-statements-using-prisma-migrate-diff

I ran this command:

npx prisma migrate diff \
--from-empty \
--to-schema-datamodel ./prisma/schema.prisma \
--script \
--output migrations/0001_create_user_table.sql
It works great. However I don't want to apply this migration yet. I want to make some more changes to the schema (add a "color" column to the "User" model), and then I want to run npx prisma migrate diff to a 2nd file called migrations/0002_add_user_color.sql.

However the docs only show show how to do this with --from-local-d1. Is there a way to make npx prisma migrate diff just look at the existing migrations and go from there?

Cloudflare D1 | Prisma Documentation

Guide to Cloudflare D1

kamama•2/2/25, 3:20 PM

Any roadmap to support replication like Turso for D1?

Kkamama Any roadmap to support replication like Turso for D1?

lambrospetrou•2/2/25, 5:55 PM

d1-database

Original message was deleted

lambrospetrou•2/3/25, 11:05 AM

Focusing just on the Worker code, since the first is mostly Drizzle stuff I guess which I am not familiar. What is the createDB()createDB() function?
Your log clearly shows that your D1 database is correctly bound to the worker. So, it's either the way Drizzle is used, or that createDBcreateDB function messing up something.

Llambrospetrou Focusing just on the Worker code, since the first is mostly Drizzle stuff I gues...

Althaf•2/3/25, 11:23 AM

Hey thankyou for the response. I have finally fixed it

Llambrospetrou Focusing just on the Worker code, since the first is mostly Drizzle stuff I gues...

Althaf•2/3/25, 11:24 AM

It was bind name mismatch

kamama•2/3/25, 1:15 PM

Btw, anyway to create and bind D1 on runtime ?

Kkamama Btw, anyway to create and bind D1 on runtime ?

Max (@rozenmd)•2/3/25, 3:42 PM

only via the HTTP API

jd•2/3/25, 8:02 PM

hi qq -- im trying to convert a largely kv-based lookup metadata of ~7GB in size of json and gets 10+ GB in size later with parallel reads, updates. Metadata needs persistence storage for consistency and availability too.

My metadata has 10M+ entries where each entry is a small dict of mostly nested strings of a loose schema, mixed with floats.

Is this size a good use case for cloudflare d1 / kv? thanks!

Jjd hi qq -- im trying to convert a largely kv-based lookup metadata of ~7GB in size...

Max (@rozenmd)•2/3/25, 8:05 PM

I'd probably pick KV for that, but there's a doc to help you choose: https://developers.cloudflare.com/workers/platform/storage-options/#choose-a-storage-product

Cloudflare Docs

Choosing a data or storage product. · Cloudflare Workers docs

Storage and database options available on Cloudflare's developer platform.

sahilmob•2/4/25, 7:53 PM

Hi, I'm building a backend with hono and better-auth, since better auth needs a DB I can only pass the DB instance using I middleware so I wonder if it is ok I initialize better auth only once in cloudflare worker, I mean to create a singleton that lives as long as the worker do. here is an example:

export class Auth {
  static initialized = false;
  private static instance: ReturnType<typeof betterAuth>;

  static async init(c: Context<Env>) {
    if (Auth.instance) {
      await Auth.getSession(c);
      Auth.initialized = true;
      return Object.freeze(Auth.instance);
    }

    Auth.instance = betterAuth({
      database: drizzleAdapter(
        {
          dialect: new D1Dialect({
            database: c.env.DB,
          }),
        },
      ),
    });

    Auth.initialized = true;

    await Auth.getSession(c);

    return Object.freeze(Auth.instance);
  }

  static async getSession(c: Context) {
    const session = await Auth.instance.api.getSession({
      headers: c.req.raw.headers,
    });

    c.set("user", session?.user ?? null);
    c.set("session", session?.session ?? null);

    return session;
  }

  static getInstance() {
    if (!Auth.instance) {
      throw new Error("Auth instance not initialized");
    }

    return Object.freeze(Auth.instance);
  }
}

export class Auth {
  static initialized = false;
  private static instance: ReturnType<typeof betterAuth>;

  static async init(c: Context<Env>) {
    if (Auth.instance) {
      await Auth.getSession(c);
      Auth.initialized = true;
      return Object.freeze(Auth.instance);
    }

    Auth.instance = betterAuth({
      database: drizzleAdapter(
        {
          dialect: new D1Dialect({
            database: c.env.DB,
          }),
        },
      ),
    });

    Auth.initialized = true;

    await Auth.getSession(c);

    return Object.freeze(Auth.instance);
  }

  static async getSession(c: Context) {
    const session = await Auth.instance.api.getSession({
      headers: c.req.raw.headers,
    });

    c.set("user", session?.user ?? null);
    c.set("session", session?.session ?? null);

    return session;
  }

  static getInstance() {
    if (!Auth.instance) {
      throw new Error("Auth instance not initialized");
    }

    return Object.freeze(Auth.instance);
  }
}

and in the worker handler:

const app = new Hono<Env>();

app.use("*", async (c, next) => {
  if (Auth.initialized) return next();

  await Auth.init(c);
  return next();
});

app.all("/api/**", (c) => Auth.getInstance().handler(c.req.raw));

export default app;
export type ApiRoutes = typeof apiRoutes;

const app = new Hono<Env>();

app.use("*", async (c, next) => {
  if (Auth.initialized) return next();

  await Auth.init(c);
  return next();
});

app.all("/api/**", (c) => Auth.getInstance().handler(c.req.raw));

export default app;
export type ApiRoutes = typeof apiRoutes;

The rational behind this approach is to not reinitialize the auth instance and make a db connection for each request

Ssahilmob Hi, I'm building a backend with hono and better-auth, since better auth needs a ...

lambrospetrou•2/5/25, 11:02 AM

It's not recommended in general to reuse envenv across Worker requests, which is why you receive it in every request.
Also, D1 doesn't have the traditional TCP connection establishing as other databases like Postgres. Every D1 query you make uses our internal networking to reach your remote database anyway, there isn't a D1 connection concept at all, it's similar as calling a service binding/WorkerEntrypoint.

So, my suggestion is to not store env globally and reuse it, otherwise you might be seeing weird behavior occassionally which will be hard to debug.

ct_roy•2/5/25, 12:34 PM

Hi folks, i'm seeing 25-30 sec response times on basic inserts into a simple D1 table today. Is there something up with D1 at the moment? i'm also seeing frequent "Error: D1_ERROR: Network connection lost." messages.

kake•2/5/25, 4:58 PM

Hey guys, this is my first time using d1 and im getting > 1000ms latencies for simple selects. is this to be expected? ive seen other people having the same issue on the forum aswell but there seems to be no responses. even if i was on the other side of the world to the db the latency should be max 300ms. 1500ms is unacceptable for users in most cases. I have tried to access it from workers aswell with more success (getting down to 800ms) but this is still way too high

KhafraOP•2/5/25, 5:14 PM

Started getting this error and the database is extremely slow:

D1_ERROR: D1 DB is overloaded. Requests queued for too long., stack: Error: D1_ERROR: D1 DB is overloaded. Requests queued for too long.

D1_ERROR: D1 DB is overloaded. Requests queued for too long., stack: Error: D1_ERROR: D1 DB is overloaded. Requests queued for too long.

the error is happening on a batch query. Every column has an index. This started happening out of the blue, but it's not the first time I've had similar issues with d1 (the same database, different tables): workers-and-pages-helpD1 in workers constant query errors & timeouts

Kkake Hey guys, this is my first time using d1 and im getting > 1000ms latencies for s...

lambrospetrou•2/5/25, 5:17 PM

ive seen other people having the same issue on the forum aswell but there seems to be no responses

It's mentioned many times in the group actually why that is.

The REST API is not how you are meant to be using D1 at the moment as part of your production applications. To get the best performance, you should use the Workers binding and access D1 through your worker. The REST API goes through a centralized location and has different layers adding latency, more aggressive rate limiting, and is therefore not suitable to be used in your actual application to access D1 in a hot path.

There has been recent improvements in the D1 latencies when using the Worker binding and you get pretty much a roundtrip's worth of latency: https://developers.cloudflare.com/d1/platform/changelog/#2025-01-07

Cloudflare Docs

Changelog · Cloudflare D1 docs

Subscribe to RSS

Llambrospetrou > ive seen other people having the same issue on the forum aswell but there seem...

kake•2/5/25, 5:19 PM

i see, thank you for your reply

KKhafra Started getting this error and the database is extremely slow: `D1_ERROR: D1 DB ...

KhafraOP•2/5/25, 5:22 PM

bumping so it doesn't get hidden

krofdrakula•2/5/25, 7:52 PM

we've just had a massive hit on performance when adding a few new indexes on one of the tables in our D1 database because the query planner used the incorrect index. we pushed out a quick fix by forcing queries to use the old index using INDEXED BYINDEXED BY while i read through the sqlite docs to determine what the correct course of action is.

reading through the query planner docs, i found the chapter on ANALYZEANALYZE and am wondering whether D1 periodically runs PRAGMA optimizePRAGMA optimize. if not, is this something that should be done by us (the users of D1) periodically or after running migrations that would update tables and add/remove indexes?

zwily•2/6/25, 4:59 AM

is the d1 api struggling right now?

zwily•2/6/25, 5:02 AM

my D1 database has gotten super slow (bound to a worker) too... ugh

Zzwily is the d1 api struggling right now?

Max (@rozenmd)•2/6/25, 6:23 AM

just to help us look into this, is it also slow from the worker?

yinxingmaiming6409•2/6/25, 8:28 AM

Is there a problem with R2? Failed to upload images and access.

Nlea•2/6/25, 10:28 AM

Hello everyone!

@boots and I were discussing taking D1 with Drizzle to production during our stream yesterday. We are working on a setup and scripts to make the process as easy and seamless as possible. We would love to hear your feedback on our ideas :
https://www.youtube.com/watch?v=4x-AvV5x1mE&t=6s

YouTubeFiberplane

Honc Out #4: Taking D1 Into Production

In this episode of Honc Out, we’re taking D1 into production!

HONC comes with a D1 template that makes it easy to spin up D1 locally for development. However, when deploying a Cloudflare Worker to production, we need to create a D1 instance in Cloudflare and connect it to our deployed worker.

We’ll walk through the configuration steps to ...

Tthomfre I'm getting lots of D1 errors today, in addition to it being very slow. Random e...

Max (@rozenmd)•2/6/25, 1:42 PM

We've seen a few reports of this over the past few days, what's your databaseId so we can investigate?

KKhafra Started getting this error and the database is extremely slow: `D1_ERROR: D1 DB ...

Max (@rozenmd)•2/6/25, 1:43 PM

We've seen a few reports of this over the past few days, what's your databaseId so we can investigate?

edit: nvm found it

Cct_roy Hi folks, i'm seeing 25-30 sec response times on basic inserts into a simple D1 ...

Max (@rozenmd)•2/6/25, 1:43 PM

We've seen a few reports of this over the past few days, what's your databaseId so we can investigate?

Max (@rozenmd)•2/6/25, 1:50 PM

For future reference:

Max (@rozenmd)•2/6/25, 1:50 PM

?databaseid

SuperHelpflare•2/6/25, 1:50 PM

Please post your D1 Database ID so we can help you debug your issue.

MMax (@rozenmd)just to help us look into this, is it also slow from the worker?

zwily•2/6/25, 2:05 PM

It’s not slow anymore, but I’m a bit freaked out by that happening. Was going to go to production in a couple months

Zzwily It’s not slow anymore, but I’m a bit freaked out by that happening. Was going to...

Max (@rozenmd)•2/6/25, 2:05 PM

what's your databaseId?

MMax (@rozenmd)what's your databaseId?

zwily•2/6/25, 2:06 PM

4a773374-363b-48c6-8044-df268ac46283

FFlikTeoh Would the DB in DO be considered more reliable in this case or it is going to en...

lambrospetrou•2/6/25, 3:21 PM

Same reliability exactly. Excluding human errors/bugs.

FFlikTeoh 😪 Is there a plan to make this better though?

Max (@rozenmd)•2/6/25, 3:38 PM

Yes.

FFlikTeoh Is there a time frame on this? Sorry for asking 🫣 been watching this for like 3...

lambrospetrou•2/6/25, 3:42 PM

Reliability doesn't have timelines. We constantly try to improve things as we find opportunities. There is an ongoing workstream for the forecoming months. There is nothing someone can promise you "by end of X it will be awesome".
If what we provide now suits your use-case, awesome

FFlikTeoh Totally understand, but D1 has been really unstable, at least for the recent 6-7...

lambrospetrou•2/6/25, 3:47 PM

Anything we promise won't fix it all of a sudden. Things will improve over time.

D1 has been really unstable

That's quite subjective though. If it's already that bad for your case, then maybe it's not the right product for that.

meow•2/6/25, 5:12 PM

Does anyone know the minimum set of permissions I need to give my API key for listing and creating new Cloudflare D1's? I've tried doing only read and edit for D1 and that doesn't seem to be enough.