ComfyUI Worker: FLUX.1 dev & Network Volume Setup Questions
Few questions:
https://github.com/runpod-workers/worker-comfyui
runpod/worker-comfyui:<version>-flux1-dev: Includes checkpoint, text encoders, and VAE for FLUX.1 dev <---- This model is using the fp8 version right now and not the full version, is that right?
Thats why I want to now use this because I want to use the full size flux version:
https://github.com/runpod-workers/worker-comfyui/blob/main/docs/customization.md
I want to use network storage but I am not sure how i need to do it.
What i wanted to do was create pod temporarily where the network storage is attached to and then just upload the full flux model etc. into the /workspace directory with this strucutre that is described in the customization.md
Example structure inside the Network Volume:
/models/checkpoints/your_model.safetensors
/models/loras/your_lora.pt
/models/vae/your_vae.safetensors
But I think I need to use a symlink so it will work on server less later as well? Is that correct?
Since here it mentions /runpod-volume <--- https://docs.runpod.io/serverless/storage/network-volumes
So i am confused on what I need to do now
so would something like this be correct then?
If not what would be the correct way to do it?
39 Replies
uploaded my stuff to network drive manually with temporay pod and now using:
this is my dockerfile:
another thing I am wondering about:
see attached file
I have my networkdrive conncted where i have models etc already but it looks like stuff is installed to /comfyui/models/ anyways even though stuff is already on the networkdrive? (isnt that unesscacry? and is causing start ups to take much longer?
i have the models on like flux etc on the networkdrive already
shortned log file, see latest message for compelte log file
which pytorch version is used by this?
since it seems it can not be used with rtx 5090
NVIDIA GeForce RTX 5090 with CUDA capability sm_120 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_50 sm_60 sm_70 sm_75 sm_80 sm_86 sm_90.
how to fix this?
here is also compelte logs file
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
@Elder Papa Madiator @Madiator2011 (Work)
Lets continue the communication here so we don't spam your Better Comfy Ui Slim Thread.
Still getting this error in the logs
I see that the CUDA Version is 12.8.1 now but also seeing this in the logs
ye it's not yet working
looks like someone forces outdated pytorch
@Salexes so I think I fixed bug pushing update and will let you know
it was installing CUDA 12.6 PyTORCH
building new image now
Ah got it that's why it did not work. Will wait for the new image and test once available
@Salexes
Nice! It works!
How can I use your image now and use one custom node that I need?
RUN comfy-node-install https://github.com/olduvai-jp/ComfyUI-HfLoader
GitHub
GitHub - olduvai-jp/ComfyUI-HfLoader
Contribute to olduvai-jp/ComfyUI-HfLoader development by creating an account on GitHub.
it's same as orginal repo
GitHub
worker-comfyui/docs/customization.md at main · runpod-workers/work...
ComfyUI as a serverless API on RunPod. Contribute to runpod-workers/worker-comfyui development by creating an account on GitHub.
just need to push updated base image to dockerhub
So I can just do this, and replace with correct image ofcourse, correct?
also pushed flux dev version
FROM runpod/worker-comfyui:5.2.0-base
replace to:
FROM madiator2011/worker-comfyui:5090-base
though wait first till I push updated image
@Salexes done make sure to pull latest base first
regarding that, does the flux dev version use the full 23 GB flux version or is it the fp8 version (that is i think 12gb) ?
Nice, will do! Thank you for the help with this, couldn't have done it without you!
np
let me know if that works now for you
Just tested it it works now!
btw pushed some images with models
thats awesome
thank you very much!
So happy right now!
now I can deep dive into testing further
@Salexes btw pushing right now the flux dev version
https://hub.docker.com/r/madiator2011/worker-comfyui/tags
Thats perfect, thank you
will take a little as it's chunky model
@Elder Papa Madiator
Thank you for all the help again. I am actively testing a lot of stuff right now.
Based on your experience, what is better to do
use lightweight base docker image without flux and have the flux model on network drive
OR
use the flux dev docker image that you made and use no network drive ?
and how does flashboot come into play here?
Which of these solutions would be more efficient on serverless?
Would one be loading faster/be ready faster than the other?
I am trying to optimize the total time it needs to run.
_
I hope you don't mind me asking these questions, just trying to get a deeper understanding of what the best approach would be so I can apply stuff that way in the future as well
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View
I pushed docker image with model in
Hi @Madiator2011
I do have a question is it normal that with 20 workers available it does not make use of all of them automatically?
I noticed that happening sometimes now, that there is 20 workers available and there is like 24 jobs
yes it depends on your settings
Is there a setting somewhere else than here where I can set amount of workers?
Since I would have expected it to use all 20 workers when there is 24 requests coming in
Oh now suddenly after 3 minutes it started to use at least 13 workers at once.
Still wondering why it does not utiize all 20 immedeatly, could that be a setting on my end as well?
I have set the delay to 2 seconds
"Queue Delay
Queue Delay scaling strategy adjusts worker numbers based on request wait times. With zero workers initially, the first request adds one worker. Subsequent requests add workers only after waiting in the queue for 2 seconds."
Sorry for the message spam, just tried to give you the full picture
Unknown User•3mo ago
Message Not Public
Sign In & Join Server To View
Where can I do that?
Same question for this
Unknown User•3mo ago
Message Not Public
Sign In & Join Server To View
1. If you plan to use 20, I’d suggest set max worker to be 30~.
2. If you want worker to scale as quickly as possible, use request count, and set scaler to be 1.