tRPC response time sometimes takes very long
I'm noticing on my deployed next app that my trpc requests in the chrome browser takes up to 2 seconds, but i've hit the endpoint manually with curl and thunderclient and the request never takes more than 300ms. Has anyone seen something similar to this?
52 Replies
Yan Cui
Lumigo
AWS Lambda Cold Starts: Solving the Problem - Lumigo
Understand the Lambda cold start problem, learn how to solve it with Provisioned Concurrency, and how to fine tune concurrency and improve Lambda performance.
Nah it’s not cold start from what I can tell
Like if I refresh my ui, the request takes 2 seconds in browser almost every time
If I copy as curl and run it, it’s usually 300ms
Let me debug more, I’m probably doing something dumb
following this issue. i had a more exterme version of this
tRPC requests were taking 10s to process (>7s Init duration in vercel)
i thought i had it pinned down but after testing i have no idea
what i do know is my main branch and my dev branch do pretty much the same thing but the dev branch take 2s to cold start and the main branch takes 10s
I'm rewriting the same endpoint using a rest endpoint to see if it is trpc related or not
hmm
I'm getting MUCH lower request times using rest compared to tRPC
is next using some type of scheduled web workers under the hood or something?
at this point I'm not sure if it is react-query, super json, or trpc causing the longer request times... the highest I've seen my manual fetch against and api endpoint was 600ms, and the tRPC endpoint was consistently giving me 2s often
I'm deploying to aws amplify, so it's not a host issue
or shouldn't be..
react query might do some batching behind the scenes
my app only uses tRPC for mutation requests (fetches all get pushed down to me thru cloud firestore)
but the outgoing request always includes ?batch=1 in the url
so maybe its waiting to see if more requests are coming to be batched
yeah I thought maybe batching could be related to the issue
I'm going to turn off batching and see if that fixes it
gl
@Tom3 removing batching did not help
Dude I really want to like TRPC, but I had multiple different issues with it. I'm leaving comment here to later see if you found the solution
@Web Dev Cody Btw I saw you on YT yesterday, thank you for the content
sure thing, glad you like it
ok so I guess it is cold starts...
I deployed the same next app to an ec2 instance using trpc with batching, every request is sub 80ms
but from what I know, aws is supposed to warm up the lambda and keep it warm for 30 minutes, so I'm not sure why I'm seeing a lot of these requests take 2s
will continue investigating
Try trpc with express
Unknown User•16mo ago
Message Not Public
Sign In & Join Server To View
Yes
Unknown User•16mo ago
Message Not Public
Sign In & Join Server To View
Serverless needs more time to figure out many things
Maybe over time with get to the rest standards
Unknown User•16mo ago
Message Not Public
Sign In & Join Server To View
Where did you see the 30m time? I’ve not seen any guarantees on how long the lambda will stay up. For my app it seems to be ~5m
I mean I'm seeing 2s requests times happen like once every couple of requests
from various articles I've read, not from aws docs
they say the larger the mem size the longer it'll stay warm
and only usually 1% of requests are cold starts when on production
Unknown User•16mo ago
Message Not Public
Sign In & Join Server To View
idk, but again, I have the exact same endpoint in my next app using a REST endpoint
takes 600 max and usually averages 100-200ms
so tRPC is adding a lot of overhead somehow
I’m not sure how that would happen. That would need to like all be in the cpu
Trpc doesn’t do any awaiting or timeouts you didn’t ask for in your code as far as I’m aware
So those 600ms would need to be like actually running js
Unless I’m wrong. Which is possible
Unknown User•16mo ago
Message Not Public
Sign In & Join Server To View
@Pix yeah, I added a next rest api route, runs the same logic, returns the same data
highest it ever gets to is 600ms
average is like 100-200ms
this is what I get with tRPC
almost every other request is 2s
then it drops down to like 150ms which is "normal"
throw in some console.logs with a timestamp?
idk. im very interested in how to debug this
because my app does the same thing sometimes and i have no clue whats wrong
only when deployed
console.time() and
console.timeEnd()
Right?
Unknown User•16mo ago
Message Not Public
Sign In & Join Server To View
yeah, that exists but i tend to just use console.log(Date.now()) cause i might want to throw a bunch of them in to find out what part is slow
console.time() seems better for benchmarking than for finding problems
I added that inside the function already, the query to planetscale takes like 30ms
I mean it has to be related to cold starts
but idk why tRPC would take that much longer on a coldstart
Try a different browser like Firefox
because on my ec2 instance everything is sub 100ms every single request
Chrome is strange at times
how big is the trpc router?
you know... it might be related to my page having a lot of images on it?
I'm loading 100 images that use the <Image> tag
that makes 100 separate requests to the next image endpoint
maybe the endpoint actually resolves in the browser, but the dev tools makes it seem like it is still pending because it's trying to wait for 100 image api requests
nvm, it happens on a fresh page reload and it waits 2 seconds, then after that api endpoint finish, it loads the images
🤷
I have like 3 methods
Unknown User•16mo ago
Message Not Public
Sign In & Join Server To View
can you comment out like the whole ui?
so its just making the requests?
so like don't try to display images and instead just do the api request
yeah good idea let me try
yeah. like get the website down to a real bare shell that makes an api requests
just to rule out your ui / the browser's effect
I know this is going to end up being something dumb on my end
have the same issue. Does your app use getStaticProps with trpc or something?
nope
k I removed all the images from the page and now I do not see the requests taking 2s
must be that my image requests are causing the network tab to not report a request as "finished" until it completes all the sub-sequential image requests?
can you put the rest of the ui back in and just leave the images out and verify that the behavior is the same?
just to be sure its the images and not something else
that's basically what I did, it's just a single next page with an h1 and .map of <Image components
I commented out the .map and verified, the requests went down to like 100ms every time
hmmm.... interesting
how about... does it happen if you replace the next/Image with img?
(sorry if some random guy on the internet is telling you what to do 😛 )
lol nah these are good ideas
let me try that
@Tom3 yes, using img instead of Image fixes it
Huh interesting. I converted my app from cra and I have had a todo note for months to convert all the images to next image. But I haven’t because it seems really hard to work with in responsive layouts
And most of my images come from third party storage and are user submitted anyway
So I haven’t done it
Unknown User•16mo ago
Message Not Public
Sign In & Join Server To View
yeah but if next/Image also makes the api requests take 3x as long.....
idk. depending on the app that might be a deal breaker moreso than image caching
Maybe tRPc is streaming the json down? Is that a thing?
So as my array of objects stream down it starts loading the images at the same time?