lars
RRunPod
•Created by lars on 4/3/2025 in #⚡｜serverless
DeleayTime beeing really high
I am running an efficientnet on the pod.
The runpod api gives me 2 times, the response is looking like that:
for the first run: {'delayTime': 9169, 'executionTime': 173, .... "output": ....}
for subsequent runs: {'delayTime': 850, 'executionTime': 160, .... "output": ....}
i timed the execution time of my code, which is a little below the stated executionTime, about what i expect. I restricted the cpu usage on my machine and it was about the same as the executionTime of runpod. I thought that the delayTime is something in the background of runpod that i personally can't influence. But almost a second is way to high, especially if they say in the documentation that it should be very small.
15 replies
RRunPod
•Created by lars on 4/3/2025 in #⚡｜serverless
DeleayTime beeing really high
The initial loading does take longer, approximately 8 seconds when the worker is turned on. However, the delay time remains around 1 second for subsequent requests while the worker stays on. If i am sending 100 requests the delayTime remains high, so i guess that it is no boot/loading issue. As mentioned, it's CPU-only, so there is no GPU booting or loading involved. I am using a custom image, but the initial loading time is not my concern. Rather, I am concerned about the continuous high delay time after the first run.
15 replies