Correct, location hints won't be enough if it's for compliance. And I don't believe D1 supports Juri

Correct, location hints won't be enough if it's for compliance. And I don't believe D1 supports Jurisdiction Hints like R2 (and even if it did, there's no Canada jurisdiction at the moment).

IIsaac McFadyen Correct, location hints won't be enough if it's for compliance. And I don't beli...

dev•11/27/24, 2:24 AM

That’s very unfortunate. I love the d1 experience, but yeah… compliance is compliance

DDeleted User How do I make sure 2 databases in the same region spawn in the same colo? For ex...

lambrospetrou•11/27/24, 11:36 AM

No, you cannot control more fine-grained locations than regional location hints.

juanferreras•11/27/24, 7:24 PM

I was using nanoid on a small pet project and wanted to change the id lengths to start pretty small (probably 4 characters), and start increasing as we get more rows populated (to avoid collissions). Mainly to make some of the shareable URLs nicer for first users too

generally speaking, I wouldn't be concerned on doing a

COUNT(id)

COUNT(id)

every X minutes and storing it somewhere like KV (from a database workload nor a performance standpoint). But I've just realized that'd be an EXTREMELY bad idea with SQLLite and D1 row-based pricing right?

juanferreras•11/27/24, 7:54 PM

aah that's a great solution! I think I can just setup triggers to keep those +1 automatic

juanferreras•11/27/24, 7:54 PM

thanks, Leo!

sz19•11/28/24, 7:16 AM

Is there anyway to find out which table is being accessed the most?

Ssz19 Is there anyway to find out which table is being accessed the most?

lambrospetrou•11/28/24, 11:43 AM

You can find some query metrics for your usage as described in https://developers.cloudflare.com/d1/observability/metrics-analytics/
I guess the most relevant to what you asked is seeing the top queries and checking the tables accessed.

lucasp•11/28/24, 1:53 PM

Hello, I'm using Prisma with Hono and Workers. I've noticed that when I use Prisma, it takes more time to execute queries compared to using db.prepare. Is that normal?

Dean•11/28/24, 9:42 PM

is cf aware of random issues still with D1? Getting internal error's every 5 - 6 hours. Error processing database Error: Failed to execute query: 400 Bad Request - {"errors":[{"code":7500,"message":"Internal error while starting up D1 DB storage caused object to be reset."}],"success":false,"messages":[],"result":[]}

sz19•11/29/24, 8:44 AM

PRO TIP: Index your tables

DDean is cf aware of random issues still with D1? Getting internal error's every 5 - 6...

sz19•11/29/24, 8:49 AM

I also still get D1_ERROR: Network connection lost. from time to time

DDean is cf aware of random issues still with D1? Getting internal error's every 5 - 6...

DaniFoldi•11/29/24, 2:38 PM

@Dean @sz19 @Shannor can you share database IDs of the dbs where you're experiencing the issues?

DDaniFoldi @Dean @sz19 @Shannor can you share database IDs of the dbs where you're experien...

Dean•11/29/24, 2:47 PM

2206f7ed-1476-40a0-9758-f1bc3d47e5ad for the last 12 hours it’s been okay, but it comes and goes. I have a cf worker that connects to it every hour to do auditing and it fails

D Trombett•11/29/24, 3:10 PM

Hi, sometimes I'm getting a

D1_ERROR: D1 DB storage operation exceeded timeout which caused object to be reset.

D1_ERROR: D1 DB storage operation exceeded timeout which caused object to be reset.

. This happens rarely, but has been happening for weeks and when it happens no request is able to query the database. It seems like it randomly freezes for 30s and then returns that error
Database ID:

8a298455-ac11-4e0c-b160-4db807cf3572

8a298455-ac11-4e0c-b160-4db807cf3572

DD Trombett Hi, sometimes I'm getting a `D1_ERROR: D1 DB storage operation exceeded timeout ...

Isaac McFadyenOP•11/29/24, 3:11 PM

Can you give the database ID please (it's not a secret/can't be used except for on your own account so you can share it here)?

D Trombett•11/29/24, 3:12 PM

sure

Isaac McFadyenOP•11/29/24, 3:12 PM

Thanks.

Original message was deleted

Isaac McFadyenOP•11/29/24, 3:13 PM

Not sure there's an immediate fix but we're surfacing these errors to the team so hopefully there's some communication at some point.

D Trombett•11/29/24, 3:14 PM

Don't worry, I'm using D1 just for a small hobby project so it's nothing critical

Shannor•11/29/24, 3:38 PM

Hey @DaniFoldi and @Isaac McFadyen the Database ID is 7356accc-d066-45c4-a577-9b9009c822eb. We use the DB pretty often at my company so it's definitely a weird error though sporadic. We're not able to reproduce it on purpse and it also seems like we get the error but the DB action still happens.

James•11/29/24, 3:47 PM

These kind of sporadic errors are unfortunately par for the course with D1. Here's some stats from an active DB for example: https://canary.discord.com/channels/595317990191398933/992060581832032316/1300948096964100097
The best you can do is retry queries until they work, sadly, but I wish I had better news for you.

SShannor Hey @DaniFoldi and @Isaac McFadyen the Database ID is 7356accc-d066-45c4-a577-...

Isaac McFadyenOP•11/29/24, 3:49 PM

it also seems like we get the error but the DB action still happens

This part seems incredibly concerning...?

Isaac McFadyenOP•11/29/24, 3:50 PM

So to confirm, you're making a query and it's succeeding in doing whatever the query is supposed to, but you're still getting an error?

JJames These kind of sporadic errors are unfortunately par for the course with D1. Here...

James•11/29/24, 4:29 PM

From the D1 team: "[the user should] retry their query and unless there is an underlying issue with the database, it should eventually work". That's the best we've got for now, sorry folks.

Kotkoroid•11/29/24, 6:41 PM

Hey, does D1 engine has public sources?

KKotkoroid Hey, does D1 engine has public sources?

Chaika•11/29/24, 6:46 PM

public sources? like source code, or what?

Kotkoroid•11/29/24, 6:46 PM

Yes, is D1 open source?

KKotkoroid Yes, is D1 open source?

Chaika•11/29/24, 6:47 PM

It's just SQlite with a few patches: https://github.com/cloudflare/workerd/tree/main/patches/sqlite

GitHub

workerd/patches/sqlite at main · cloudflare/workerd

The JavaScript / Wasm runtime that powers Cloudflare Workers - cloudflare/workerd

Kotkoroid•11/29/24, 6:48 PM

Thanks

JJames From the D1 team: "[the user should] retry their query and unless there is an un...

Dean•11/30/24, 12:50 AM

this seems kind of wild to me, right? hey the product doesn't work sometimes, just hope it comes back or that customers aren't interrupted while it waits X amount of secs to timeout.

DDean this seems kind of wild to me, right? hey the product doesn't work sometimes, ju...

James•11/30/24, 12:55 AM

I wholeheartedly agree, and am raising issues around D1 stability and error rates pretty regularly. I opened https://github.com/cloudflare/cloudflare-docs/issues/18485 today in hopes that there's some public guidance of the kinds of architecture people are expected to build into their apps to have a stable experience with D1.

lucasp•11/30/24, 4:22 PM

Hello, I'm using Prisma with Hono and Workers. I've noticed that when I use Prisma, it takes more time to execute queries compared to using db.prepare. Is that normal? or whay i could being doing wrong

Llucasp Hello, I'm using Prisma with Hono and Workers. I've noticed that when I use Pris...

James•11/30/24, 4:35 PM

With the exact same query? Using any ORM is almost always going to create slightly more unoptimised queries because they're designed for flexibility and general use-cases vs doing very specific queries yourself.

JJames With the exact same query? Using any ORM is almost always going to create slight...

lucasp•11/30/24, 4:38 PM

yes i did this query

 const [total_records, users] = await Promise.all([
    prisma.user.count({
      where: whereClause
    }),
    prisma.user.findMany({
      take: limit || undefined,
      skip: page && limit ? (page - 1) * limit : undefined,
      where: whereClause,
      orderBy: sort ? getOrderBy(sort) : undefined,
      select: {
        id: true,
        first_name: true,
        last_name: true,
        role: {
          select: { name: true }
        },
        email: true,
        transportist: { select: { phone: true, status: true } },
        cargo_giver: { select: { phone: true, status: true } },
        created_at: true
      }
    })
  ])

 const [total_records, users] = await Promise.all([
    prisma.user.count({
      where: whereClause
    }),
    prisma.user.findMany({
      take: limit || undefined,
      skip: page && limit ? (page - 1) * limit : undefined,
      where: whereClause,
      orderBy: sort ? getOrderBy(sort) : undefined,
      select: {
        id: true,
        first_name: true,
        last_name: true,
        role: {
          select: { name: true }
        },
        email: true,
        transportist: { select: { phone: true, status: true } },
        cargo_giver: { select: { phone: true, status: true } },
        created_at: true
      }
    })
  ])

with prism findmany and selects it takes me 5 seconds and I changed it to using $queryRaw and it takes me 2 seconds which seems like quite a difference to me

Llucasp yes i did this query ```js const [total_records, users] = await Promise.all([ ...

James•11/30/24, 5:51 PM

You gotta check the raw query output to verify that. Probably not. Also orms spend time converting raw data to js objects.

Frederik•11/30/24, 8:53 PM

Hi folks, is there an example somewhere on how to use transactions with D1? (rolling back a sequence of inserts/updates if one failed)

esselungo•12/1/24, 2:22 AM

d1 really bills by rows written and not operations? is there no plan covering a bit more ops? i’m thinking of moving out of d1 completely for an outside db

Eesselungo d1 really bills by rows written and not operations? is there no plan covering a ...

Isaac McFadyenOP•12/1/24, 4:02 AM

What is an "operation" in this context?

Isaac McFadyenOP•12/1/24, 4:02 AM

And yes, D1 bills for rows read/written.

Original message was deleted

Frederik•12/1/24, 9:48 AM

Sorry I was not specific enough with what i meant with "sequence": with being able to use the result of a query/insert/update for the query in the sequence. This is not really possible with batch AFAIK

Original message was deleted

Frederik•12/1/24, 10:44 AM

thanks, i will try work my way around

Mas Inis•12/2/24, 2:22 AM

is D1 support branching (like neon, supabase, turso)?

MMas Inis is D1 support branching (like neon, supabase, turso)?

Isaac McFadyenOP•12/2/24, 2:45 AM

No. D1 has Time Travel (aka PITR, point-in-time recovery) but no branching.

IIsaac McFadyen No. D1 has Time Travel (aka PITR, point-in-time recovery) but no branching.

Mas Inis•12/2/24, 6:14 AM

Thanks!

IIsaac McFadyen No. D1 has Time Travel (aka PITR, point-in-time recovery) but no branching.

Max (@rozenmd)•12/2/24, 1:33 PM

(yet)

MMax (@rozenmd)(yet)

Larry•12/2/24, 1:49 PM

Is this on the roadmap or do you just like to tease us?

LLarry Is this on the roadmap or do you just like to tease us?

Max (@rozenmd)•12/2/24, 1:52 PM

regardless, it becomes a lot easier to create branch/preview DBs once you have infrastructure in place to rapidly spin up replicas of D1 DBs

dmaretskyi•12/2/24, 4:46 PM

Hello, I'm seeing high latency on a D1 database (around 1 sec RTT per write) on my workers project. The service is a simple CRUD api hooked up to D1 with Prisma. The database is located in WNAM while I'm calling the worker from Europe.

Is there anything I can do to mitigate the latency on writes? I could shard the data so that it's located closer to the user. D1 docs say that it's design to scale accross multiple smaller per-user databases? How is that achieved in practice? In my worker's TOML config I need to define each DB binding manually.

const [total_records, users] = await Promise.all([ prisma.user.count({ where: whereClause }), prisma.user.findMany({ take: limit || undefined, skip: page && limit ? (page - 1) * limit : undefined, where: whereClause, orderBy: sort ? getOrderBy(sort) : undefined, select: { id: true, first_name: true, last_name: true, role: { select: { name: true } }, email: true, transportist: { select: { phone: true, status: true } }, cargo_giver: { select: { phone: true, status: true } }, created_at: true } }) ])

Correct, location hints won't be enough if it's for compliance. And I don't believe D1 supports Juri

Similar Threads

Similar Threads

Similar Threads