My serverless does not deploy the new releases
I was using 1.2.4 version of my image on an endpoint. I started a new release (1.2.5) but workers still works on 1.2.4. To be sure, I released 1.2.6 version and deployed. I have waited 3-4 hours between this version deployments. However, still it uses 1.2.4. What could be the problem?
Thanks.
33 Replies
Do you see Latest and Stale workers on your endpoint page when doing a new release?
Yes, I have 3 stale worker, and 1 extra worker.
When I start a new release, it says initializing.. however still uses the same image
Unknown User•17mo ago
Message Not Public
Sign In & Join Server To View
Nah its not a browser issue, he said its still using the old image version, must be some serverless bug, I would log a support issue for this on the website.
There should be "Latest Workers" with the new image and there aren't any, so something is broken.
Yes, I agree with you. I deployed 1.2.5 yesterday. However I still have 1.2.4 on workers 😄
Unknown User•17mo ago
Message Not Public
Sign In & Join Server To View
Maybe low availability of the required GPU type?
Unknown User•17mo ago
Message Not Public
Sign In & Join Server To View
I have found that with :latest it will not update. Rather if you specify a specific version i.e. :1.01 when you change it to 1.02 (or other) the update will work. You cannot use :latest and assume it will update. It will not. You have to edit the template and provide a tag runpod has never seen.
I dont use latest tag.
I use the cheapiest worker option 16 GB GPU but it says high availability. However, the fact that it was not deployed even after 1 day seemed to be a problem.
Unknown User•17mo ago
Message Not Public
Sign In & Join Server To View
You updated your template with a new tag? Like this?
I would create a new enpoint right now because my app is currently in development. However, if the same issue occurs on production, It would be problem 😄 Because of that I'm trying to see if there is missing
Unknown User•17mo ago
Message Not Public
Sign In & Join Server To View
I don't know..
Unknown User•17mo ago
Message Not Public
Sign In & Join Server To View
It says "Waiting for logs" for the extra worker
Have you tested your image locally?
Nope, I don't have GPU environment on my local 😦, but there is no dramatical change between 1.2.4 and 1.2.5
Unknown User•17mo ago
Message Not Public
Sign In & Join Server To View
I reported thank you. I will update here if there is a problem
Support suggested me to edit endpoint to set worker count to 0, then increase again. This way fixed the issue. Thank you 🙏
You should not have to do that though
Yes, I think an issue occurred, but I just got rid of the problem
Unknown User•17mo ago
Message Not Public
Sign In & Join Server To View
Thats a pain, easier just to set max workers to zero and back up again
Unknown User•17mo ago
Message Not Public
Sign In & Join Server To View
What is your point? Its stupid to waste unneccessary time recreating endpoint, getting a different endpoint id, having to change your code that calls the endpoint to use the new endpoint id etc etc, its a stupid point to be honest
Unknown User•17mo ago
Message Not Public
Sign In & Join Server To View
Both will create an unavailability anyway 😄 So I think runpod is aware of the problem since they suggested this to reset like this
I had this issue occasionally. Just change the maximum or minimum number of worker or any param in the endpoint. It will trigger a new deployment
Unknown User•17mo ago
Message Not Public
Sign In & Join Server To View
Yes