Time for my client to get request back is much longer than CFW wall time.
Hey guys, running into an issue that I'm not sure if it's CFW or me.
Context:
I'm just using CFW as middleware to authenticate session tokens using Jose. That's the only thing my instance is doing. The request is to the worker like
/llm/request to be able to get a response back.
Specifics:
The problem I'm running into is that for some reason, extremely randomly, the client-side time to get a response back is really really slow, like 5-6 seconds, even though wall time for that same request is 200ms or so.
Here's an example of what I've logged from one of these faulty requests:
Server-Side:
Wall time (233ms)
Request from upstream provider (LLM Inference) - (181ms)
Client-Side:
While connecting
š summary | reused:0 proto:h3 dns:1ms tcp:118ms tls:117ms server:2918ms total:3154ms
-> Reused indicates if this was a previously open connection.
I'm confused how server ends up being around 2918 ms, even though the wall time is 233 ms?3 Replies
@Flare
@HardlyWorkin'
?pings
Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.