how can I upgrade vllm version in serverless worker image?
https://docs.runpod.io/serverless/endpoints/model-caching
Hi, I was trying the new Qwen3-next-80B-a3b-instruct in the example, and I selected vllm worker image to run it, but it seems that the vllm version in the image doesn't support the model yet. so I added the pip upgrade transformers and vllm in the docker start command, and the image still tries to load the model in the old version.
Runpod Documentation
Cached models - Runpod Documentation
Accelerate cold starts and reduce costs by using cached models.
3 Replies
Plus, I have a network storage with fine tuned model of my company. Can I change the model path in the serverless image to the /workspace in the network volume so that it can skip the model pulling processes? Cheers.
Inside serverless it will always be mounted to
/runpod-volume, sorry :(I understand. Then what's the point of enabling network in the 'Advanced' in the serverless config? I wonder if i can use it for some other workarounds.