Issues with building the new `worker-vllm` Docker Image
I've been using the previous version of
I've tried the newest version of the
Unfortunately, there are issues when building a Docker Image (even without any modifications).
There are already 3 issues related to that on GitHub:
https://github.com/runpod-workers/worker-vllm/issues/21#issuecomment-1862188983
https://github.com/runpod-workers/worker-vllm/issues/25
https://github.com/runpod-workers/worker-vllm/issues/26
Could you, please, take a look on it? Or provide with a solution for scaling the previous version of
worker-vllmawqI've tried the newest version of the
worker-vllmvllmUnfortunately, there are issues when building a Docker Image (even without any modifications).
There are already 3 issues related to that on GitHub:
https://github.com/runpod-workers/worker-vllm/issues/21#issuecomment-1862188983
https://github.com/runpod-workers/worker-vllm/issues/25
https://github.com/runpod-workers/worker-vllm/issues/26
Could you, please, take a look on it? Or provide with a solution for scaling the previous version of
worker-vllmGitHub
I'm building the image with WORKER_CUDA_VERSION=12.1 on an M1 Mac using command docker buildx build -t antonioglass/worker-vllm-new:1.0.0 . --platform linux/amd64 and getting errors. See below....
GitHub
I tried to build the docker from scratch but also get an error (using CUDA 11.8, runpod/base:0.4.4) RUN python3.11 -m pip install -e git+https://github.com/runpod/vllm-fork-for-sls-worker.git@cuda-...
GitHub
Around 1-3% the download of model while building docker image get stuck and don't move forward. This happens with different models too and wasn't happening earlier. Outside of this docker i...
