Cloudflare Developers•15mo ago

always happens to me early morning on a simple select on a pk with a very low traffic db🤷‍♂️

always happens to me early morning on a simple select on a pk with a very low traffic db

Oo7 always happens to me early morning on a simple select on a pk with a very low tr...

toadfans•10/10/24, 12:46 PM

Same here

NickP•10/10/24, 2:28 PM

Me too

Sam•10/10/24, 5:07 PM

Is the D1 HTTP API down right now?

SSam Is the D1 HTTP API down right now?

Chaika•10/10/24, 5:17 PM

Thanks for the extra info, raised

SSam Is the D1 HTTP API down right now?

ajgeiss0702•10/10/24, 5:33 PM

this is also appears to be effecting the dasboard. You can see database and their tables, but anything other than that wont load

Aajgeiss0702 this is also appears to be effecting the dasboard. You can see database and thei...

Chaika•10/10/24, 5:42 PM

yea they use basically the same api. It looks like the query api is fine (if using direct) but not getting db/table info, which in the dash also prevents using console/metrics/etc tabs. They ack'd the issue

nullptr•10/10/24, 5:45 PM

Is the GUI broken for D1?

Nnullptr Is the GUI broken for D1?

Chaika•10/10/24, 5:46 PM

yes see above

Chaika•10/10/24, 5:49 PM

https://www.cloudflarestatus.com/incidents/p1jpzdbz4sbc

D1 API Performance issues - Increased Errors

Chaika•10/10/24, 5:59 PM

Seems fixed now

Norris Nam•10/11/24, 2:48 AM

Can I use Navicat to connect to Cloudflare D1 Database?

NNorris Nam Can I use Navicat to connect to Cloudflare D1 Database?

Sameer Ali•10/11/24, 6:13 AM

yes, there is a .sqlite file in .wrangler directory. you can open that in Navicat.

aj•10/11/24, 12:25 PM

Hi, could somebody help in short with D1? Using the standard API is really slow. Should I setup a new API using workers and connect it to D1? Or how does it work, so it's quick? A random VPS server is much quicker than the default D1. I don't want to believe that

Aaj Hi, could somebody help in short with D1? Using the standard API is really slow....

Isaac McFadyen•10/11/24, 2:58 PM

The standard API is indeed slow because of the technical design of D1. If you hit the standard API it goes:
you -> Cloudflare core (central location) which processes things -> D1 in a CF datacenter somewhere

Because it bounces through the core it can be slower. If you use the D1 API from a Worker directly, as is the main use-case of D1, it goes:
you -> Worker -> D1 (often near the Worker)

Meaning it skips the core.

So TLDR: Yes, use a Worker for best performance.

aj•10/11/24, 3:07 PM

@Isaac | AS213339 Is it possible to achieve speeds of a VPS server? with the worker we improved the loading times from 1.3 seconds to 700 ms, but VPS is still 50ms. Are there any paid features that make the Worker API almost instant?

Aaj @Isaac | AS213339 Is it possible to achieve speeds of a VPS server? with the wor...

boop•10/11/24, 4:58 PM

You could try smart placement to move the worker closer to the database

Aaj @Isaac | AS213339 Is it possible to achieve speeds of a VPS server? with the wor...

lambrospetrou•10/11/24, 5:47 PM

Is it possible to achieve speeds of a VPS server?

What is a VPS server here? A server accessing a remote database on a different server/same region, accessing database on the same server, your browser accessing a database on a VPS server (where?)? It can mean many things.

In general the D1 REST API is slow, and not meant for your interactive low-latency queries. Using D1 through a Worker is much better. Your D1 database will also be in a fixed location, so your query request location also matters here. Smart placement could help to avoid routing over slow networks and use Cloudflare's network handle routing.

The 50ms you mention is latency for what exactly?

aj•10/12/24, 6:02 PM

@lambrospetrou @Isaac | AS213339
50 ms is the latency of a VPS server where the database is.

Here are some live tests:
This is D1, R2 with an API via Worker: https://cloud.simyz.com/categories
This is a VPS server where the database is stored + R2 for asset storing: https://cloud-db.simyz.com/categories

What options are available to make D1 as fast as possible? If this is the best I can have, this is by far not a competitive product to use for a database.

Aaj @lambrospetrou @Isaac | AS213339 50 ms is the latency of a VPS server where the...

boop•10/12/24, 6:12 PM

Are you using the D1 HTTP API in the worker or accessing the database with a binding?

Bboop Are you using the D1 HTTP API in the worker or accessing the database with a bin...

aj•10/12/24, 6:18 PM

accessing it via binding

Aaj accessing it via binding

boop•10/12/24, 6:18 PM

Is smart placement enabled?

Bboop Is smart placement enabled?

aj•10/12/24, 6:19 PM

I think I just enabled it for Worker. A minute ago. For D1 it was enabled before.

Aaj I think I just enabled it for Worker. A minute ago. For D1 it was enabled before...

boop•10/12/24, 6:23 PM

There is no smart placement for D1 yet unfortunately, it will always live in the region that you created it in. Here's to hoping that smart placement makes it better

Bboop There is no smart placement for D1 yet unfortunately, it will always live in the...

aj•10/12/24, 6:25 PM

I'm wondering if I switch from Free to Paid plan could it help improve the results? by using some of the unlocked paid features and limits?

Aaj I'm wondering if I switch from Free to Paid plan could it help improve the resul...

boop•10/12/24, 6:27 PM

I don't think that would make much of a difference with D1 to Workers

Isaac McFadyen•10/12/24, 6:40 PM

Latency shouldn't be affected by paid plan, nope.

aj•10/12/24, 7:11 PM

@Isaac | AS213339 Is my understanding correct that in production, using D1 with Workers and Smart Placements optimizes and improves delivery performance specifically for the most frequently accessed locations?

aj•10/12/24, 7:12 PM

So at the start latency to Japan could be much worse than after some time if it's commonly accessed?

Aaj @Isaac | AS213339 Is my understanding correct that in production, using D1 with ...

Isaac McFadyen•10/12/24, 7:14 PM

Smart Placement with D1 just moves your Worker near the DB (always, and D1 doesn't move). So the Worker will always be placed near D1.

Martin•10/13/24, 4:34 AM

i’m trying to use session method withSession for read replication but i get method undefined.
I’m on the wrangler version 2024-09-23 what else should i do to get access to read replication?

MMartin i’m trying to use session method withSession for read replication but i get met...

Isaac McFadyen•10/13/24, 4:39 AM

Read Replication for isn't released yet so you won't be able to use it.

aj•10/13/24, 6:38 AM

@Isaac | AS213339 do you think that enabling Caching API would help speed up the performance of the API?

IIsaac McFadyen Read Replication for isn't released yet so you won't be able to use it.

Martin•10/13/24, 1:19 PM

Oh ok I tough so with the blog of april

Jacob Marshall•10/14/24, 12:07 AM

Speaking of... do we know roughly when read replication may see a release (in any form, eg. private beta)?

JJacob Marshall Speaking of... do we know roughly when read replication may see a release (in an...

Chaika•10/14/24, 12:42 AM

hmm I think last date I heard was Q3 d1-database

Chaika•10/14/24, 12:42 AM

well, "Targeting"

Chaika•10/14/24, 12:42 AM

then "private beta soon" last month d1-database

Jacob Marshall•10/14/24, 12:44 AM

Oh thanks, @Chaika. For a second there when you said Q3 I thought you meant next year haha (considering we're already in Q4)

Jacob Marshall•10/14/24, 12:44 AM

I was prepared to cry ;P

bennycondon•10/14/24, 1:11 AM

Does D1 support escaping single quotes by doubling up the single quotes? e.g. INSERT INTO test VALUES ('This''ll work just fine.')INSERT INTO test VALUES ('This''ll work just fine.').

It seems to work fine when I run npx wrangler d1 execute <database> --file=<file>npx wrangler d1 execute <database> --file=<file> for local development.

But gives me an internal error when I run npx wrangler d1 execute <database> --file=<file> --remotenpx wrangler d1 execute <database> --file=<file> --remote for production.

Georg Obermayr•10/14/24, 4:03 PM

I'm having some performances troubles with D1 databases. I have a Worker that is bound to a D1 database and using smart placement. We are on a enterprise plan. I have a very simple API in that Worker that just selects some items by an id in the route, parses some JSON fields and returns the data. With that I have response times starting at 500 ms, sometimes up to 2s! We also did some load testing with 500 users, and with that the response time exploded to 20 seconds and more. Obvously we can't use that in production. I'm requesting from Munic/Germany, and the response says remote-HAM for CF-placement. Is there anything that can be done?

o7OP•10/14/24, 9:11 PM

Will the D1 Dynamic Binding future integration include a batch API to delete more than 1000 (worker subrequest limit) databases in a single go? This would be a handy feature for deleting many expired user accounts efficiently in a cron worker, as my usecase.

AsyncBanana•10/15/24, 12:15 AM

I am getting the error {"D1_RESET_DO":true}{"D1_RESET_DO":true} whenever I try to execute a SQL file on a remote D1 database. No errors appear when I execute the same file on a local database, and the source file is from a working D1 local database. Any ideas what could be happening?

ilsubyeega•10/15/24, 11:06 AM

Is there any way to optimize the cost of writes when indexing? It seems like each index adds one write. Not that it is really important, but I'm curious.

Iilsubyeega Is there any way to optimize the cost of writes when indexing? It seems like eac...

lambrospetrou•10/15/24, 11:23 AM

That's expected and documented (your writes need to update your indices). Optimizing that out means removing your index, but then you make your reads much slower/costlier, so it depends on the frequency between reads and writes and what's more cost-effective for your case.
I wouldn't recommend removing your indices though since latencies could go high depending on what your queries are, so it's not just a cost optimization.

ilsubyeega•10/15/24, 11:24 AM

thanks :D

Hhannes More data needed, too fuzzy: parsing JSON fields how? do you read whole JSON obj...

Georg Obermayr•10/15/24, 4:15 PM

I mostly did JSON.parse, but in this test scenario it was a single small field in a single record. I also tried to just do the query (Select * from table where id=?Select * from table where id=?) and not parse/return anything, just testing the timing. Still a very high latency and response time.

nato•10/15/24, 8:10 PM

is d1 faster when used with workers compared to cloudflare pages http client? considering if I should use opennext or next-on-pages

Nnato is d1 faster when used with workers compared to cloudflare pages http client? co...

Sameer Ali•10/15/24, 8:49 PM

yes

always happens to me early morning on a simple select on a pk with a very low traffic db🤷‍♂️

Similar Threads