worker vllm 'build docker image with model inside' fails

from the page https://github.com/runpod-workers/worker-vllm?tab=readme-ov-file

Option 2: Build Docker Image with Model Inside
To build an image with the model baked in, you must specify the following docker arguments when building the image.

Prerequisites
RunPod Account
Docker
Arguments:
Required

MODEL_NAME
Optional
MODEL_BASE_PATH: Defaults to /runpod-volume for network storage. Use /models or for local container storage.
QUANTIZATION
WORKER_CUDA_VERSION: 11.8.0 or 12.1.0 (default: 11.8.0 due to a small amount of workers not having CUDA 12.1 support yet. 12.1.0 is recommended for optimal performance).
For the remaining settings, you may apply them as environment variables when running the container. Supported environment variables are listed in the Environment Variables section.

Example: Building an image with OpenChat-3.5
sudo docker build -t username/image:tag --build-arg MODEL_NAME="openchat/openchat_3.5" --build-arg MODEL_BASE_PATH="/models" .

so I cloned the github thing into a folder, then opened a command prompt inside the same folder as the dockerfile, then put in

docker build -t toxibunny/RPmixtralAPI:0.1 --build-arg MODEL_NAME="TheBloke/Mixtral-8x7B-MoE-RP-Story-AWQ" --build-arg MODEL_BASE_PATH="/models"

but it came back

ERROR: "docker buildx build" requires exactly 1 argument.
See 'docker buildx build --help'.

Usage: docker buildx build [OPTIONS] PATH | URL | -

what am I doing wrong?

GitHub

GitHub - runpod-workers/worker-vllm: The RunPod worker template for...

The RunPod worker template for serving our large language model endpoints. Powered by vLLM. - GitHub - runpod-workers/worker-vllm: The RunPod worker template for serving our large language model en...

BoxitunnyOP•1/28/24, 11:09 PM

here's what chatgpt says btw:
The error mentioning docker buildx could be related to the fact that you have Docker Buildx installed on your system. Docker Buildx is an extended version of the Docker CLI with support for multiple builder instances. It might be automatically intercepting your docker build command.

You can try running the docker build command without the Buildx features. You can do this by setting the DOCKER_BUILDKIT=0 environment variable.

I did that with $env:DOCKER_BUILDKIT=0 and after setting my repository name to all lowercase, it's doing something. fingers crossed

BoxitunnyOP•1/28/24, 11:29 PM

Step 5/13 : RUN --mount=type=cache,target=/root/.cache/pip python3 -m pip install --upgrade pip && python3 -m pip install --upgrade -r /requirements.txt
the --mount option requires BuildKit. Refer to https://docs.docker.com/go/buildkit/ to learn how to build images with BuildKit enabled

so nah. I need help

Docker Documentation

BuildKit

Introduction and overview of BuildKit

BoxitunnyOP•1/29/24, 12:03 AM

now I'm getting this:

BoxitunnyOP•1/29/24, 12:04 AM

Traceback (most recent call last):
File "/src/download_model.py", line 3, in <module>
from vllm.model_executor.weight_utils import prepare_hf_model_weights
ModuleNotFoundError: No module named 'vllm'

BoxitunnyOP•1/29/24, 12:07 AM

Dockerfile:29
--------------------
28 |
29 | >>> RUN --mount=type=secret,id=HF_TOKEN,required=false \
30 | >>> if [ -f /run/secrets/HF_TOKEN ]; then \
31 | >>> export HF_TOKEN=$(cat /run/secrets/HF_TOKEN); \
32 | >>> fi && \
33 | >>> if [ -n "$MODEL_NAME" ]; then \
34 | >>> python3 /src/download_model.py --model $MODEL_NAME; \
35 | >>> fi
36 |
--------------------
ERROR: failed to solve: process "/bin/sh -c if [ -f /run/secrets/HF_TOKEN ]; then export HF_TOKEN=$(cat /run/secrets/HF_TOKEN); fi && if [ -n "$MODEL_NAME" ]; then python3 /src/download_model.py --model $MODEL_NAME; fi" did not complete successfully: exit code: 1

BoxitunnyOP•1/29/24, 12:19 AM

I am really trying. I give up for tonight though