Serverless container storage
I made a script that downloads various models and packages and incorporated it in the docker image that is being run on serverless endpoint, i want it to run only once and download all the packages and store them for use, do i have to use network volume storage or is there another way to store the packages in container storage so they do not download every time the container is removed and started again?
11 Replies
Unknown User•16mo ago
Message Not Public
Sign In & Join Server To View
the total download data amounts to about 25GBs
Unknown User•16mo ago
Message Not Public
Sign In & Join Server To View
I was hoping for another solution 😩
Unknown User•16mo ago
Message Not Public
Sign In & Join Server To View
I would suggest creating a version with the models built into the image and another version that uses network volume and compare the response time. There is an open issue with network volumes that cause them to add delaytime to the process so you might be better of with your 25gb image.
well, the reason i went with script was to decrease the time it takes to push and deploy the image so i don't think that will work
Unknown User•16mo ago
Message Not Public
Sign In & Join Server To View
With either choice you can speed up your requests by having at least 1 active worker.
and since you are only charged when your requests are being processed I suggest you set max workers to 30.
Unknown User•16mo ago
Message Not Public
Sign In & Join Server To View
got it, thanks!