How to build worker-vllm Docker Image without a model inside?

antoniog · 2023-12-19T11:08:18.678Z

I would like to build a worker-vllm with slightly customized vLLM. However, I don't want to build it with a model inside. Basically, it should be the same as your Pre-Built Docker Image that would download the model to the Network Volume. Thanks! https://github.com/runpod-workers/worker-vllm/

K

kopyl•12/19/23, 11:34 AM

Just replace this:

export MODEL_BASE_PATH=$MODEL_BASE_PATH && \
export MODEL_NAME=$MODEL_NAME && \
python3.11 /download_model.py --model $MODEL_NAME; \

export MODEL_BASE_PATH=$MODEL_BASE_PATH && \
export MODEL_NAME=$MODEL_NAME && \
python3.11 /download_model.py --model $MODEL_NAME; \

export MODEL_BASE_PATH=$MODEL_BASE_PATH && \
export MODEL_NAME=$MODEL_NAME && \
python3.11 /download_model.py --model $MODEL_NAME; \

export MODEL_BASE_PATH=$MODEL_BASE_PATH && \
export MODEL_NAME=$MODEL_NAME && \
python3.11 /download_model.py --model $MODEL_NAME; \

With this

export MODEL_BASE_PATH=$MODEL_BASE_PATH && \
export MODEL_NAME=$MODEL_NAME; \

export MODEL_BASE_PATH=$MODEL_BASE_PATH && \
export MODEL_NAME=$MODEL_NAME; \

export MODEL_BASE_PATH=$MODEL_BASE_PATH && \
export MODEL_NAME=$MODEL_NAME; \

export MODEL_BASE_PATH=$MODEL_BASE_PATH && \
export MODEL_NAME=$MODEL_NAME; \

in your Dockerfile

A

antoniogOP•12/19/23, 11:54 AM

thanks!

K

kopyl•12/19/23, 12:23 PM

@antoniog7545 did it work?

Kkopyl @antoniog7545 did it work?

A

antoniogOP•12/19/23, 1:34 PM

there was another issue, so I couldn't build the image...

Aantoniog there was another issue, so I couldn't build the image...

K

kopyl•12/19/23, 3:41 PM

Which issue?

Kkopyl Which issue?

A

antoniogOP•12/19/23, 9:34 PM

I'm getting error when bulding a Docker Image. See below.

1.527   Running command git clone --filter=blob:none --quiet https://github.com/antonioglass/vllm-fork-for-sls-worker.git /src/vllm
4.184   Resolved https://github.com/antonioglass/vllm-fork-for-sls-worker.git to commit 9797380bd9a3eef6a70e899d03b81f6967ecd287
4.192   Installing build dependencies: started
68.88   Installing build dependencies: still running...
131.1   Installing build dependencies: still running...
199.6   Installing build dependencies: still running...
964.9   Installing build dependencies: finished with status 'error'
965.0   error: subprocess-exited-with-error
965.0   
965.0   × pip subprocess to install build dependencies did not run successfully.
965.0   │ exit code: 2

1.527   Running command git clone --filter=blob:none --quiet https://github.com/antonioglass/vllm-fork-for-sls-worker.git /src/vllm
4.184   Resolved https://github.com/antonioglass/vllm-fork-for-sls-worker.git to commit 9797380bd9a3eef6a70e899d03b81f6967ecd287
4.192   Installing build dependencies: started
68.88   Installing build dependencies: still running...
131.1   Installing build dependencies: still running...
199.6   Installing build dependencies: still running...
964.9   Installing build dependencies: finished with status 'error'
965.0   error: subprocess-exited-with-error
965.0   
965.0   × pip subprocess to install build dependencies did not run successfully.
965.0   │ exit code: 2

1.527   Running command git clone --filter=blob:none --quiet https://github.com/antonioglass/vllm-fork-for-sls-worker.git /src/vllm
4.184   Resolved https://github.com/antonioglass/vllm-fork-for-sls-worker.git to commit 9797380bd9a3eef6a70e899d03b81f6967ecd287
4.192   Installing build dependencies: started
68.88   Installing build dependencies: still running...
131.1   Installing build dependencies: still running...
199.6   Installing build dependencies: still running...
964.9   Installing build dependencies: finished with status 'error'
965.0   error: subprocess-exited-with-error
965.0   
965.0   × pip subprocess to install build dependencies did not run successfully.
965.0   │ exit code: 2

1.527   Running command git clone --filter=blob:none --quiet https://github.com/antonioglass/vllm-fork-for-sls-worker.git /src/vllm
4.184   Resolved https://github.com/antonioglass/vllm-fork-for-sls-worker.git to commit 9797380bd9a3eef6a70e899d03b81f6967ecd287
4.192   Installing build dependencies: started
68.88   Installing build dependencies: still running...
131.1   Installing build dependencies: still running...
199.6   Installing build dependencies: still running...
964.9   Installing build dependencies: finished with status 'error'
965.0   error: subprocess-exited-with-error
965.0   
965.0   × pip subprocess to install build dependencies did not run successfully.
965.0   │ exit code: 2

A

antoniogOP•12/19/23, 9:36 PM

I'm not sure but it may be related to the small modifications in the

vllm-fork-for-sls-worker

vllm-fork-for-sls-worker

vllm-fork-for-sls-worker

vllm-fork-for-sls-worker. I'll try to build the image with the original

vllm-fork-for-sls-worker

vllm-fork-for-sls-worker

vllm-fork-for-sls-worker

vllm-fork-for-sls-worker.

A

antoniogOP•12/19/23, 10:33 PM

Yes, it's the same issue with the original vllm-fork-for-sls-worker too. I opened an issue: https://github.com/runpod-workers/worker-vllm/issues/25

GitHub

Errors when building the image · Issue #25 · runpod-workers/worker-...

I'm building the image with WORKER_CUDA_VERSION=12.1 on an M1 Mac using command docker buildx build -t antonioglass/worker-vllm-new:1.0.0 . --platform linux/amd64 and getting errors. See below....

How to build worker-vllm Docker Image without a model inside?

Similar Threads

Similar Threads

Similar Threads