How can I cause models to download on initialization?

FROM runpod/pytorch:2.2.1-py3.10-cuda12.1.1-devel-ubuntu22.04
WORKDIR /content
ENV PATH="/home/camenduru/.local/bin:${PATH}"
RUN adduser --disabled-password --gecos '' camenduru && \
adduser camenduru sudo && \
echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers && \
chown -R camenduru:camenduru /content && \
chmod -R 777 /content && \
chown -R camenduru:camenduru /home && \
chmod -R 777 /home

RUN apt update -y && add-apt-repository -y ppa:git-core/ppa && apt update -y && apt install -y aria2 git git-lfs unzip ffmpeg

USER camenduru

RUN pip install -q opencv-python imageio imageio-ffmpeg ffmpeg-python av runpod \
xformers==0.0.25 torchsde==0.2.6 einops==0.8.0 diffusers==0.28.0 transformers==4.41.2 accelerate==0.30.1

RUN git clone -b flux https://github.com/camenduru/ComfyUI /content/ComfyUI

COPY --chown=camenduru:camenduru ./worker_runpod.py /content/ComfyUI/worker_runpod.py
COPY --chown=camenduru:camenduru ./download_models.sh /content/ComfyUI/download_models.sh

WORKDIR /content/ComfyUI

USER root
RUN chmod +x download_models.sh
USER camenduru

# Create a startup script
RUN echo '#!/bin/bash\n./download_models.sh\npython worker_runpod.py' > /content/ComfyUI/startup.sh
RUN chmod +x /content/ComfyUI/startup.sh

CMD ["/bin/bash", "/content/ComfyUI/startup.sh"]
FROM runpod/pytorch:2.2.1-py3.10-cuda12.1.1-devel-ubuntu22.04
WORKDIR /content
ENV PATH="/home/camenduru/.local/bin:${PATH}"
RUN adduser --disabled-password --gecos '' camenduru && \
adduser camenduru sudo && \
echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers && \
chown -R camenduru:camenduru /content && \
chmod -R 777 /content && \
chown -R camenduru:camenduru /home && \
chmod -R 777 /home

RUN apt update -y && add-apt-repository -y ppa:git-core/ppa && apt update -y && apt install -y aria2 git git-lfs unzip ffmpeg

USER camenduru

RUN pip install -q opencv-python imageio imageio-ffmpeg ffmpeg-python av runpod \
xformers==0.0.25 torchsde==0.2.6 einops==0.8.0 diffusers==0.28.0 transformers==4.41.2 accelerate==0.30.1

RUN git clone -b flux https://github.com/camenduru/ComfyUI /content/ComfyUI

COPY --chown=camenduru:camenduru ./worker_runpod.py /content/ComfyUI/worker_runpod.py
COPY --chown=camenduru:camenduru ./download_models.sh /content/ComfyUI/download_models.sh

WORKDIR /content/ComfyUI

USER root
RUN chmod +x download_models.sh
USER camenduru

# Create a startup script
RUN echo '#!/bin/bash\n./download_models.sh\npython worker_runpod.py' > /content/ComfyUI/startup.sh
RUN chmod +x /content/ComfyUI/startup.sh

CMD ["/bin/bash", "/content/ComfyUI/startup.sh"]
My internet connection doesn't support uploading massive dockerbuilds to the hub. How can i get the models to download on runpod using a strategy similar to this?
16 Replies
Unknown User
Unknown User16mo ago
Message Not Public
Sign In & Join Server To View
gnarley_farley.
gnarley_farley.OP16mo ago
Trying to launch this on serverless worker https://github.com/camenduru/flux.1-dev-tost
GitHub
GitHub - camenduru/flux.1-dev-tost
Contribute to camenduru/flux.1-dev-tost development by creating an account on GitHub.
Unknown User
Unknown User16mo ago
Message Not Public
Sign In & Join Server To View
gnarley_farley.
gnarley_farley.OP16mo ago
Yeah thanks I saw your message. I am a bit stupid. First time using runpod Nothing downloads
Unknown User
Unknown User16mo ago
Message Not Public
Sign In & Join Server To View
gnarley_farley.
gnarley_farley.OP16mo ago
I think I have something wrong in my setup Yeah me too lol. Been trying all night
Unknown User
Unknown User16mo ago
Message Not Public
Sign In & Join Server To View
gnarley_farley.
gnarley_farley.OP16mo ago
If you get a chance later, would be much appreciated
Unknown User
Unknown User16mo ago
Message Not Public
Sign In & Join Server To View
gnarley_farley.
gnarley_farley.OP16mo ago
❤️
tim
tim16mo ago
What is your actual use case for using ComfyUI with Flux? Do you want an API to use ComfyUI or do you want to play around with Flux? Because if you are doing this in serverless, then every time your endpoint is shut down as there is no activity, you will then have to download the models again when a request comes in. So I would first love to know what your goal is with Flux! Then we can take a look on how we can achieve this.
gnarley_farley.
gnarley_farley.OP16mo ago
I just want to use it as an API, sent a request, get some images. Same as one would do with the comfyui runpod worker/serverless I was trying to use a download script within my code rather than pushing all the files to docker at once. hence the setup. Not sure if it's required though. If anyone has a working image would happily use that(flux).
Unknown User
Unknown User16mo ago
Message Not Public
Sign In & Join Server To View
tim
tim16mo ago
I was working on the image for runpod-worker-comfy yesterday and it seems it is working. Will have to update the README + provide an example workflow. Then I will ping you!
tim
tim15mo ago
@gnarley_farley. both FLUX.1 dev and schnell are released (including example workflows): https://github.com/blib-la/runpod-worker-comfy?tab=readme-ov-file#quickstart
GitHub
GitHub - blib-la/runpod-worker-comfy: ComfyUI as a serverless API o...
ComfyUI as a serverless API on RunPod. Contribute to blib-la/runpod-worker-comfy development by creating an account on GitHub.
tim
tim15mo ago
For the other idea related to download models when the serverless endpoint does init: https://github.com/blib-la/runpod-worker-comfy/issues/60
GitHub
Download models on the fly · Issue #60 · blib-la/runpod-worker-comfy
Is your feature request related to a problem? Please describe. The Docker images are very huge and users would like to just use the base image (which doesn't contain any models). This would mak...

Did you find this page helpful?