Costs
I ran two serverless jobs.
Each took about 30 seconds compute on a 16gb machine. There was a delay, and a cold start time for both.
The total billing each was about 2 cents, in total 4 cents.
I was expecting about a 1 cent total. Am I also charged for cold start time?
The cold start time for each request was about 70 seconds, I think mostly time unpacking disk images, (I can't see those logs right now)
8 Replies
Yes, the cold start is billed too. That's why you want to optimize your images as much as possible and use Flashboot.
I do use Flashboot. How do I "optimize my image?"
Bake everything you can into it—models, files, installations, processes, etc. You don't want to download or process anything at runtime you don't need to.
It doesn't
It's all time downloading images from the runpod server and expanding them.
Do you mean pulling and extracting the container image itself? That's worker initialization and it's not billed. You're billed only for the execution time of your code once you send a request.
I see.
There is significant time just having cuda start. Is there some way to speed this up?
I guess I'll put some more logging in, so I can figure out what is taking so long. It's not downloading anything, it's possible its compiling kernels.
I don't know what is your container doing, so unfortunately I can't help you much without knowing the goal, logs or even handler code.
No problem, thanks for your input