Serveless prices and technical questions
Hi I'm having a great time using the runpod at first it was a little bit hard to make my ai app compatible to serverless but it now working great.
I wanted to make sure what prices are.
What I assume is having idle serverless is free and the only cost is when running and handling requests which is visible in panel.
What I am confused about is the storage cost, for example when I set worker storage to 25 instead of 5 because the model doesn't fit in it should raise the price I assume
Or maybe I'm doing something wrong, should I normally try to download from hugging face?
Or there is better solution for example there is a hugging face link place in start of serverless creation that might be helpful but I don't know how to use yet.
If there is any documentation that I need to follow to understand model load in serverless setup please let me know.
I wanted to make sure what prices are.
What I assume is having idle serverless is free and the only cost is when running and handling requests which is visible in panel.
What I am confused about is the storage cost, for example when I set worker storage to 25 instead of 5 because the model doesn't fit in it should raise the price I assume
Or maybe I'm doing something wrong, should I normally try to download from hugging face?
Or there is better solution for example there is a hugging face link place in start of serverless creation that might be helpful but I don't know how to use yet.
If there is any documentation that I need to follow to understand model load in serverless setup please let me know.
Solution
read the docs about model cache / model store its compatible with hf models
or if you're not interested with that, put the models in your network storage / inside the docker image then load it on runtime (on your handler.py, before rp.serverless.start() call)
or if you're not interested with that, put the models in your network storage / inside the docker image then load it on runtime (on your handler.py, before rp.serverless.start() call)
