Hello! I'm building out an analytics tool that makes a lot of db calls (10+ calls to get the things

Hello! I'm building out an analytics tool that makes a lot of db calls (10+ calls to get the things like total users, unique sessions, events, etc).

I'm using cloudflare pages, so locally, it's using a sqlite db and the requests can take up to 2 seconds. When I go to staging it uses the D1 db and my requests can take up to 3 minutes.

Any ideas why it would be so different? I'm thinking maybe the db connections are what's taking so long, I am making a lot of connections but don't want to spend the time trying to combine the sql if it's not going to get me down to that ~2 second mark.

TTeamDave Hello! I'm building out an analytics tool that makes a lot of db calls (10+ call...

AJR•7/15/25, 2:40 PM

I think you might get better luck in #d1-database . I'm afraid I'm not as up to speed on the internals there.

I will say that it definitely shouldn't take 3 minutes to do 10 queries, regardless of anything else, so I'm optimistic there's a solution there.

AAJR I think you might get better luck in #d1-database . I'm afraid I'm not as up to ...

TeamDaveOP•7/15/25, 2:40 PM

Man... I thought that's where I was, thanks!

ALECKS•7/17/25, 2:25 PM

Why doesn't the hyperdrive binding contain the connection details without having to call .get().get() ? This is adds significant friction to the DX.

We're using a lot of RCP/Service workers, and it would be so natural to configure/initialize database stuff in a constructor and attach it to this.this.
Then every public method that could be invoked via RPC could simply use this.db...this.db.... But because .get().get() is async, we need to do this dance in every public method.

AALECKS Why doesn't the hyperdrive binding contain the connection details without having...

Isaac McFadyen•7/17/25, 2:31 PM

where/how are you using get()get()? Hyperdrive connection details should be on the binding itself, according to docs?

IIsaac McFadyen 🤔 where/how are you using `get()`? Hyperdrive connection details should be on t...

ALECKS•7/17/25, 2:34 PM

I'm sorry, my statement was incorrect. There's nothing wrong with the hyperdrive binding. The issue is with the secrets_store binding and I got them mixed up. Will repost in the right channel

karimulm•7/17/25, 7:10 PM

Is hyperdrive accessible from cloudflare container?

AJR•7/17/25, 9:04 PM

Not at this time, but it's something we intend to add as it's an obviously strong use case for both products.

AAJR Not at this time, but it's something we intend to add as it's an obviously stron...

ALECKS•7/17/25, 9:43 PM

Would it be okay to get the connectionString in a worker and pass it to a container or is that not advised?

AJR•7/17/25, 9:45 PM

I'm afraid it won't work. The connection string provided by the Hyperdrive binding is generated at runtime and is scoped to just that request. It won't work anywhere else or at any other time.

AJR•7/17/25, 9:45 PM

Part of the security model.

AAJR I'm afraid it won't work. The connection string provided by the Hyperdrive bindi...

Isaac McFadyen•7/18/25, 12:35 AM

Interesting, if a connection is kept open (i.e. a global connection against recommendations of closing it at the end of a request) will it continue to work until it's closed then?

IIsaac McFadyen Interesting, if a connection is kept open (i.e. a global connection against reco...

AJR•7/18/25, 12:54 AM

I'm not an expert in how the Worker Runtime manages state in edge cases like that, honestly. What I know for sure is that when the connection goes the connectionString goes with it

andrzej•7/18/25, 10:45 AM

Hi! I'm working on some Worker code which needs to fetch about a 1000 of small rows from a MySQL database (hosted by PlanetScale). When I connect using Hyperdrive, the CPU usage per request is way higher than when connecting directly. For example, 5ms with a direct connection, compared to 40ms with Hyperdrive.
The root cause - to my understanding - is the fact that when receiving data from Hyperdrive, each resulting row (mysql protocol packet) is returned in what appears to be a separate TCP packet. In my tests, each call to ReadableStreamBYOBReader.read returns only 22 bytes when receiving query results (this matches the size of the mysql packet), despite having lots of free space in the buffer.
I find this surprising, because conventional TCP mechanisms such as Nagle's algorithm should merge these small packets. If my understanding is correct, then this behaviour also introduces inefficiencies at other OS/network layers.
For comparison, when I connect directly to planetscale, the data is returned in much larger chunks (multiple kilobytes), allowing for more efficient processing.
A dirty workaround is to simply add a delay before receiving the query result. This allows one read() call to process a larger chunk of data.

AJR•7/18/25, 11:07 AM

That's so interesting.

The underlying cause here is that under the hood Hyperdrive isn't TCP except for the very last hop to/from the origin database. The overwhelming majority of the transit uses different transport.

I'll make a ticket to investigate our batching to see if we can't improve it to at least parity with a direct connection. We focused far more on latency and throughput for the beta, and our Postgres handler has a bunch of other buffering built in by nature of the protocol. However we definitely don't want to be running up CPU time on MySQL either.

Thanks for bringing this to our attention.

AJR•7/18/25, 11:07 AM

Fyi @thomasgauvin ^

Aandrzej Hi! I'm working on some Worker code which needs to fetch about a 1000 of small r...

AJR•7/18/25, 3:35 PM

Which drivers are you using for Hyperdrive and the direct connection to Planetscale?

andrzej•7/18/25, 4:12 PM

I've started with mysql2, but then I've moved to a handcrafted mysql protocol implementation, hoping to avoid any unnecessary CPU overheads.

Aandrzej I've started with mysql2, but then I've moved to a handcrafted mysql protocol im...

AJR•7/18/25, 4:14 PM

Sounds good. Wanted to make sure that the difference was visible even with the same driver, to rule out having to explore driver-level differences also. Thank you!

dave•7/21/25, 1:19 PM

Error: write CONNECTION_CLOSED REMOVED.hyperdrive.local:5432

Error: write CONNECTION_CLOSED REMOVED.hyperdrive.local:5432

Got a few of these last night.

dave•7/21/25, 1:32 PM

Is there a way I should be retrying connections with postgres.js to avoid the request from completely failing?

AJR•7/21/25, 2:41 PM

We do encourage application-level retries, yes. Hyperdrive will in the near future do some amount of internal retrying on non-volatile reads, but in general we will always have to be conservative with that given the limited scope of information available. Application-level will likely always be able to do better.

omar•7/21/25, 7:30 PM

Adding hyperdrive to our MySQL database on planetscale resulted in higher latency

surgery18•7/22/25, 5:05 PM

So I am getting this strange issue now. My workers were working fine till today. All of the sudden it is throwing the error "Hyperdrive does not currently support MySQL multi-statement queries" Did Cloudflare do an update to hyperdrive and it broke it? Nothing is saving now.

Ssurgery18 So I am getting this strange issue now. My workers were working fine till today....

AJR•7/22/25, 5:07 PM

Hyperdrive has not supported multi-statement queries in MySQL since the beta launched, that hasn't changed. Did they used to work for you?

surgery18•7/22/25, 5:09 PM

So the statement was working for me for a while await connection2.query("INSERT into revolv_events SET ? ON DUPLICATE KEY UPDATE ?", [ins, ins])await connection2.query("INSERT into revolv_events SET ? ON DUPLICATE KEY UPDATE ?", [ins, ins]). It was saving into the db and everything just perfectly. Just today it stopped saving and instead throw that error.

Ssurgery18 So the statement was working for me for a while `await connection2.query("INSERT...

AJR•7/22/25, 5:10 PM

We're taking a look. We have been collecting MySQL feedback from the beta (and investigating some of the feedback given here in particular) but haven't made any changes to it yet.

surgery18•7/22/25, 5:12 PM

Ok, in the meantime what do you suggest I do if that query doesn't work? Just do a regular insert and surround with try/catch without the dup key thing?

Ssurgery18 Ok, in the meantime what do you suggest I do if that query doesn't work? Just do...

AJR•7/22/25, 5:14 PM

For now, if that works for you, yes. We're still taking a look to see if anything changed unintentionally.

AAJR For now, if that works for you, yes. We're still taking a look to see if anythin...

surgery18•7/22/25, 5:14 PM

Ok.

Ssurgery18 Ok.

AJR•7/22/25, 5:15 PM

What driver are you using, and any ORMs with it?

AAJR What driver are you using, and any ORMs with it?

surgery18•7/22/25, 5:16 PM

Using the driver mysql2 and no ORM.

AJR•7/22/25, 5:17 PM

Okay. Hmmm. Mind sharing your Hyperdrive ID?

AAJR Okay. Hmmm. Mind sharing your Hyperdrive ID?

surgery18•7/22/25, 5:17 PM

Sure, I will DM you it so you can look into it.

surgery18•7/22/25, 5:19 PM

DM sent.

AsyncBanana•7/24/25, 9:56 PM

For whatever reason, hyperdrive registers almost exactly 2/3 of my queries as uncacheable. All of my queries (using postgres.js) are selects outside of a transaction that do not use any volatile functions. To make matters even more confusing, I separated each query type into its own hyperdrive connection, and they all had a 2/3 uncacheable rate. Any ideas why this is happening?

NNeves I'm using mysql2 to connect to planet scale dev database locally, but I'm gettin...

AJR•7/25/25, 11:37 AM

Local connections to remote databases are a challenge due to binding limitations. It's on our radar and there's some org-wide work going into addressing it, but it will be a while yet.

AAsyncBanana For whatever reason, hyperdrive registers almost exactly 2/3 of my queries as un...

AJR•7/25/25, 11:39 AM

If you wouldn't mind sharing your Hyperdrive ID I can take a look.

Also, sharing for general information, the full granularity of cacheability status for all of your queries are available via our graphQL endpoint. Thata's worth checking as it'll have more detail than the graphs on the dashboard.

NNeves For now I can switch to direct connection locally. Is there a best way to know I...

AJR•7/25/25, 11:44 AM

I believe env vars are the way folks usually handle that. something like:

    export default {
      async fetch(request, env, ctx) {
        if (env.DEV) {
          // Logic specific to local development
          console.log("Running in local development mode.");
        } else {
          // Logic specific to production
          console.log("Running in production mode.");
        }
        // ... rest of your Worker logic
      },
    };

    export default {
      async fetch(request, env, ctx) {
        if (env.DEV) {
          // Logic specific to local development
          console.log("Running in local development mode.");
        } else {
          // Logic specific to production
          console.log("Running in production mode.");
        }
        // ... rest of your Worker logic
      },
    };

AJR•7/25/25, 11:45 AM

Ah, found a very simple blog showing it off (apologies for no docs link, on my phone and don't have it handy)
https://www.fabiofranchino.com/log/how-to-detect-a-cloudflare-worker-in-dev-local-mode/

AAJR If you wouldn't mind sharing your Hyperdrive ID I can take a look. Also, shar...

AsyncBanana•7/25/25, 12:11 PM

thanks, I will dm you

AJR•7/25/25, 8:55 PM

Yeah, it does. Hyperdrive will place your connection pool near your database on creation, but smart placement for where your Worker itself runs takes quite a lot of traffic to converge, as I understand it

Hello! I'm building out an analytics tool that makes a lot of db calls (10+ calls to get the things

Similar Threads