Error building worker-vllm docker image for mixtral 8x7b
I'm running the following command to build and tag a docker worker image based off of worker-vllm:
docker build -t lesterhnh/mixtral-8x7b-instruct-v0.1-runpod-serverless:1.0 --build-arg MODEL_NAME="mistralai/Mixtral-8x7B-Instruct-v0.1" --build-arg MODEL_BASE_PATH="/models" .
I'm getting the following error:
------
Dockerfile:23
--------------------
22 | # Install torch and vllm based on CUDA version
23 | >>> RUN if [[ "${WORKER_CUDA_VERSION}" == 11.8* ]]; then \
24 | >>> python3.11 -m pip install -U --force-reinstall torch==2.1.2 xformers==0.0.23.post1 --index-url https://download.pytorch.org/whl/cu118; \
25 | >>> python3.11 -m pip install -e git+https://github.com/runpod/[email protected]#egg=vllm; \
26 | >>> else \
27 | >>> python3.11 -m pip install -e git+https://github.com/runpod/vllm-fork-for-sls-worker.git#egg=vllm; \
28 | >>> fi && \
29 | >>> rm -rf /root/.cache/pip
30 |
--------------------
ERROR: failed to solve: process "/bin/bash -o pipefail -c if [[ "${WORKER_CUDA_VERSION}" == 11.8* ]]; then python3.11 -m pip install -U --force-reinstall torch==2.1.2 xformers==0.0.23.post1 --index-url https://download.pytorch.org/whl/cu118; python3.11 -m pip install -e git+https://github.com/runpod/[email protected]#egg=vllm; else python3.11 -m pip install -e git+https://github.com/runpod/vllm-fork-for-sls-worker.git#egg=vllm; fi && rm -rf /root/.cache/pip" did not complete successfully: exit code: 1
docker build -t lesterhnh/mixtral-8x7b-instruct-v0.1-runpod-serverless:1.0 --build-arg MODEL_NAME="mistralai/Mixtral-8x7B-Instruct-v0.1" --build-arg MODEL_BASE_PATH="/models" .
I'm getting the following error:
------
Dockerfile:23
--------------------
22 | # Install torch and vllm based on CUDA version
23 | >>> RUN if [[ "${WORKER_CUDA_VERSION}" == 11.8* ]]; then \
24 | >>> python3.11 -m pip install -U --force-reinstall torch==2.1.2 xformers==0.0.23.post1 --index-url https://download.pytorch.org/whl/cu118; \
25 | >>> python3.11 -m pip install -e git+https://github.com/runpod/[email protected]#egg=vllm; \
26 | >>> else \
27 | >>> python3.11 -m pip install -e git+https://github.com/runpod/vllm-fork-for-sls-worker.git#egg=vllm; \
28 | >>> fi && \
29 | >>> rm -rf /root/.cache/pip
30 |
--------------------
ERROR: failed to solve: process "/bin/bash -o pipefail -c if [[ "${WORKER_CUDA_VERSION}" == 11.8* ]]; then python3.11 -m pip install -U --force-reinstall torch==2.1.2 xformers==0.0.23.post1 --index-url https://download.pytorch.org/whl/cu118; python3.11 -m pip install -e git+https://github.com/runpod/[email protected]#egg=vllm; else python3.11 -m pip install -e git+https://github.com/runpod/vllm-fork-for-sls-worker.git#egg=vllm; fi && rm -rf /root/.cache/pip" did not complete successfully: exit code: 1
GitHub
A high-throughput and memory-efficient inference and serving engine for LLMs - GitHub - runpod/vllm-fork-for-sls-worker: A high-throughput and memory-efficient inference and serving engine for LLMs
