Sending many prisma queries at once accumulates delay between them 200ms, 600ms, 1.1s, 1.5s...
I'm having big troubles finding the root of the bottleneck in our app. I have metrics, telemetry and postgres pg_stat_statements enabled and did some deep dive.
Behavior and facts:
- sending many requests in parallel (20) causes batches of time dalays, some arrive after 200ms, some after 400, 600, etc... its a pretty consistent delay accumulation
- The delay is purely caused by prisma/db., iI added console.time before and after the findUnique / findMany
- on the database level the queries never run longer than 150/200ms, both pg_stat_statements and telemetry via jaeger prove that
- connection-string to our db has even
?connection_limit=40 which does seem to be used up after a while of running I have around 30, sometimes more connection on the pool.
- the time at which the server sends the request is the same (all go to prisma/db simultanously)
It's not Http/1.1 dev behavior where the request are actually stalled client side. All of the prismaClient.findUnique requests happen simultanously
even the request controller isnt starting to process the other concurrent requests. they are stalled nby what feels like something synchronous that prisma does. It can only be prisma since the controller only takes the body and calls prisma.findMany etc... The first few times this happens the event loops seems to be completely blocked for like 100ms and only then the next request is being processed. As if prisma.FindMany does some heavy task before delegating the rest asynchronously to postgres. We're still using the old Rust query engine, not the preview feature. Changing it to the new method also doesn't change this behavior2 Replies
You selected to wait for the human sages. They'll share their wisdom soon.
Grab some tea while you wait, or check out
#ask-ai if you'd like a quick chat with the bot anyway!It's difficult to predict what might be going wrong without looking at the code. Is your repository open source? Anyways you can provide a reproduction so that we can investigate?