Hey all, we have a simple status endpoint which we use for uptime on a worker which uses D1, using D

Hey all, we have a simple status endpoint which we use for uptime on a worker which uses D1, using Drizzle ORM. We have a schedule which checks every 1 minute this endpoint.

The code is simple:

app.get("/status", async (c) => {
  await getDatabaseClient(c.env).select().from(schema.variables).limit(1);
  return c.text("OK");
});

app.get("/status", async (c) => {
  await getDatabaseClient(c.env).select().from(schema.variables).limit(1);
  return c.text("OK");
});

app.get("/status", async (c) => {
  await getDatabaseClient(c.env).select().from(schema.variables).limit(1);
  return c.text("OK");
});

app.get("/status", async (c) => {
  await getDatabaseClient(c.env).select().from(schema.variables).limit(1);
  return c.text("OK");
});

Every so often (maybe once every few weeks) we get alerts coming through reporting the same error:

Error: D1_ERROR: Error 9000: something went wrong

Error: D1_ERROR: Error 9000: something went wrong

Error: D1_ERROR: Error 9000: something went wrong

Error: D1_ERROR: Error 9000: something went wrong

No other errors / messages. Any idea what causes this? We use D1 for a critical part of our infra, so these sort of "unexplained" issues are a little concerning for us. Thanks!

EElliot Hesp Hey all, we have a simple status endpoint which we use for uptime on a worker wh...

Max (@rozenmd)•4/17/24, 11:12 AM

Hey Elliot, let me investigate - can you let me know the accountId and databaseId?

MMax (@rozenmd)Hey Elliot, let me investigate - can you let me know the accountId and databaseI...

Elliot HespOP•4/17/24, 11:13 AM

Thanks and sure - want me to DM or ok in here?

Elliot HespOP•4/17/24, 11:14 AM

I have DM'd you

Max (@rozenmd)•4/17/24, 11:32 AM

Just a reminder:

Max (@rozenmd)•4/17/24, 11:32 AM

?d1-alpha-migration

Flare•4/17/24, 11:32 AM

On the off-chance you're still using an alpha D1 database, we've made a guide to make it easier to move across to the newer, faster (40x more requests per second) D1 DBs:

https://developers.cloudflare.com/d1/platform/alpha-migration

Max (@rozenmd)•4/17/24, 11:32 AM

run npx wrangler d1 infonpx wrangler d1 info if you're not sure which version you're on (it'll tell you if it's alpha)

EElliot Hesp Hey all, we have a simple status endpoint which we use for uptime on a worker wh...

Matt Silverlock•4/17/24, 1:09 PM

For others here as well: you're using the original alpha DB infra, which was superseded in July last year and by default in Sept when D1 went into open beta.

https://developers.cloudflare.com/d1/platform/changelog/#d1-alpha-databases-will-stop-accepting-live-sql-queries-on-july-1-2024

Cloudflare Docs

Changelog · Cloudflare D1 docs

Previously, D1’s HTTP API API link label Open API docs link returned a HTTP 500 Internal Server error for an invalid query. An invalid SQL query now …

rye•4/17/24, 2:31 PM

Doing some load testing and getting weird results. Any recommendations to get the average response time down to something manageable so I can use D1 in the wild

Doing simple query SELECT * FROM table_name WHERE updated_at >= ? AND name > ? ORDER BY updated_at ASC, name LIMIT 1000SELECT * FROM table_name WHERE updated_at >= ? AND name > ? ORDER BY updated_at ASC, name LIMIT 1000

100k row table, 17mb
30s load test, 1k requests, average response time is 40s
normal response time is 230ms
Confirmed it's using an index with updated_atupdated_at and
```
name
```
```
name
```
using cloudflare worker to query database

F0rce•4/17/24, 3:04 PM

is there a way to some sort of "prioritise" this issue ? This has been around for quite some time (and fixed in prod / when deployed) but still happening in local development. With D1 reaching GA I think this should be fixed, before other people fall over it.

Unsmart•4/17/24, 5:56 PM

how are you determining where it is running?

Unsmart•4/17/24, 5:58 PM

Im 99% sure that the request.cf.colo will be the one you connect to and not where the worker actually runs in the end. I think theres supposed to be a way to know where its placing but dont know off the top of my head will try to find out

Isaac McFadyen•4/17/24, 5:59 PM

Yeah I think request.cf.colo will be the ingress colo. And 0.2ms is quite good to be honest - unless you mean 0.2s?

Isaac McFadyen•4/17/24, 6:00 PM

The three-letter IATA airport code of the data center that the request hit, for example, "DFW".

Nope, doesn't look so.

Isaac McFadyen•4/17/24, 6:01 PM

Try checking the cf-placement header on the response.

Unsmart•4/17/24, 6:01 PM

https://developers.cloudflare.com/workers/configuration/smart-placement/#cf-placement-header

Isaac McFadyen•4/17/24, 6:01 PM

I think that's the name?

Unsmart•4/17/24, 6:01 PM

yeah

Chaika•4/17/24, 6:05 PM

yea I just tested and request.cf.colo is indeed the entry colo

NNeves Cf-Placement: remote-ATL, got it, the https://developers.cloudflare.com/d1/build...

Chaika•4/17/24, 6:05 PM

meta.duration isn't network time btw, its purely the time for the d1 db to execute the query

Chaika•4/17/24, 6:06 PM

due to smart placement you set yea

Chaika•4/17/24, 6:06 PM

ATL is probably exactly where your D1 db lives: https://where.durableobjects.live/colo/ATL

Where Durable Objects Live

Atlanta, Georgia, United States(colo) | Where Durable Objects Live

Tracking where Durable Objects are created, wherever you are in the world.

Unsmart•4/17/24, 6:07 PM

It would be impossible for the db to know the network time

Chaika•4/17/24, 6:09 PM

you could time it in your worker to get the network time from your worker -> d1
https://developers.cloudflare.com/workers/runtime-apis/performance/

Chaika•4/17/24, 6:10 PM

Really you have this:
You
-> Local CF Colo
-> Remote CF Colo
-> Worker
-> D1 DB
-> SQLITE (meta.duration)
<- D1 DB
<- Worker (you can time this within the worker)
<- Remote Cf Colo
<- Local CF Colo
You (overall time you can see on your end)

NNeves The time to query from worker is around 20ms, that's the fetch request to the DO...

Chaika•4/17/24, 6:58 PM

you timed it like
perf.Now
env.D1.Query
console.log(now - oldnow)?
If so yea, that'd be the full time including network latency it took for the D1 DB to return a response

NNeves Do I need to enable smart placement when using D1 or it will always place next t...

Chaika•4/17/24, 8:25 PM

for your worker to run close to the db, you need to have smart placement enabled

森

森@VerseLink (RW)•4/17/24, 10:54 PM

is it possible to clone a d1 database (for example cloned for developement)

fforres•4/18/24, 1:36 AM

Hey folks! Random question, but do you know there an auto-generated uuid on D1D1 ?

fforres•4/18/24, 1:37 AM

We have a DB on neon, and I'm considering the migration. but the UUID part might be a bit of a hassle

:Club_Penguin_cute_sad:

fforres•4/18/24, 1:37 AM

(I know turso had a sqlite extension, unsure if we have an alternative on D1

)

roftt•4/18/24, 4:29 AM

Does 'rows_read' still count in the metrics when I open my database/tables in the dashboard, or have you already fixed it? I'm still seeing huge numbers

Rroftt Does 'rows_read' still count in the metrics when I open my database/tables in th...

Max (@rozenmd)•4/18/24, 6:16 AM

to be clear, if you read your data, you will be billed for it - it's not a bug.

i've removed the row counts from the list of tables though, so you don't accidentally get billed for reads you didn't know about

MMax (@rozenmd)to be clear, if you read your data, you will be billed for it - it's not a bug. ...

roftt•4/18/24, 6:29 AM

Something isn't right then. I go to the list of tables in the dashboard, and the metric shows +22,000 rows read (the total number of rows in all my tables combined)

Max (@rozenmd)•4/18/24, 6:42 AM

ah the fix isn't live yet

avsaase•4/18/24, 8:35 AM

Is the fix to not show the number of rows in the table?

avsaase•4/18/24, 8:38 AM

So there is no way to see your table size without incurring the cost of reading all rows?

Hey all, we have a simple status endpoint which we use for uptime on a worker which uses D1, using D

Similar Threads

Similar Threads

Similar Threads