RunPod + ComfyUI Setup Help (Torch 2.4)
Hi all! I need some help setting up my RunPod environment using this template:
ComfyUI with Manager inst. Permanent Disk torch2.4
I’m using Network Storage. With the old Torch 2.2 template, I just ran ./run_gpu.sh and everything worked fine.
Now with Torch 2.4, I get errors about missing modules like aiohttp, safetensors, etc. Even if I install them, they disappear when I shut down and restart the pod.
I’m looking for someone who can help me configure the environment properly, make sure everything installs and works as expected, and do a quick check to confirm all is running smoothly.
Please DM me if you'd be so kind to help me🙏
35 Replies
Could you try switching to this https://canary.discord.com/channels/912829806415085598/1364178138048893019/1364178138048893019 template if you don't have any data in your Network Storage drive?
hello, is there anyone here that can help me???
i dont think anyone will dm you so easily, that help is kind of hard to do in a short time
Dj here has gave you an easy alternative that i'd also recommend to try if you want an easy way to setup
Thanks Jason, this template is probably exactly what I need right now actually
I'm relatively new to this and I'd highly appreciate a little bit of help from the team or someone competent 🙂
Can I ask you a couple of questions regarding this template?
-First one is - I read Optimized for 5090 - since I often choose the 4090 for working, does that work as well? where does the optimization part comes in here?
-Also, I read Pytorch Nightly... excuse my noob question but which version is it? 2.7? I'm looking to use I2V optimizations such as Sageattention, TeaCache, bf16.. will these be supported?
-Last one, I can read "FileBrowser for easy file management" what is this option?
Thank you so much for your help🙏
@Madiator2011 perhaps sir you could answer these few questions since you posted about this template, thank you very much
In that Discord message you'll see two templates linked. The first one ("Regular Template") does not require you to use a 5090.
PyTorch Nightly is 2.8, it's not released yet.
FileBrowser is just a web ui to help users download files from inside their pods without the terminal :)
@Madiator2011 do you mind If I can ask you a couple of questions here or in DM?
do not ask if you can ask just ask 🙂
thank you
I'm experimenting on some workflows and I was thinking about using a less expensive GPU like the A4500, just to see if there are errors n stuff; after having the workflow ready, I would like to use a more powerful GPU to just run the workflow and do some batch renders of images and videos.. and I was thinking about using a more powerful GPU like the 5090.
(I'm a novice) So my questions are the following:
- Should I choose the 5090 optimized template for this use case? with that template will I be able to also run other gpus with less advanced CUDA versions?
- last thing, about the Pytorch version, having a Nightly version for the optimized template can mean having some issues on Checkpoints, Loras or other assets, because it's still an experimental version?
Maybe the best option could be to go with the Regular Template and for batch rendering use a GPU that's less powerful than the 5090..
What would you do in my position?
Thank you very much for your help
so the regular template would work on all gpu's expect 5090
for 5090 will wait till supports get merged to stable PyTorch as nightly version can cause unknow issues
perfect, I'll go with the regular then, thank you!!
one last thing, I already have a network storage that I've used with another template previously, is it safe to just use that storage with this Regular Template? or is it better to just create one new?
first option would be better because I have a lot of checkpoints, Loras and workflows already installed there
yup slim template was made to not colide with other templates
you should be able to transfer checkpoints dirs manually. For custom nodes you will need reinstall dependency most likely
gotcha, so the checkpoints,Lora etc.. folders will not be directly available on the new template's directory? even if they are currently present in the network storage
so if you deploy template it will create new folder on workspace that inside of it will be ComfyUI folder and venv
after new install is done you can use mv command to move everything to new folder
Hi, I'm mounting your slim template but the cloudflare tunnel won't work
It stucks on that
It looks like a torch error
Traceback (most recent call last):
File "/workspace/madapps/ComfyUI/main.py", line 135, in <module>
import comfy.utils
File "/workspace/madapps/ComfyUI/comfy/utils.py", line 24, in <module>
import safetensors.torch
File "/workspace/madapps/ComfyUI/.venv/lib/python3.12/site-packages/safetensors/torch.py", line 11, in <module>
def storage_ptr(tensor: torch.Tensor) -> int:
^^^^^^^^^^^^
AttributeError: module 'torch' has no attribute 'Tensor'did you let template install first?
hi @Madiator2011 I’m using your template, however I’m constantly running into Out of Memory issues. My template crashes completely when VRAM hits 100%. It seems like the VAE decoder hits OOM, but instead of handling it (like switching to tiled mode), it just crashes the whole instance. It looks like the template I’m using doesn’t have any mechanism to prevent or recover from OOM errors, could you help me?
Hmm?
Tried restarting pod
Anyone can give some advice on a working template for ComfyUI? All the ones I'm trying are giving me the Cloudflare Bad Gateway error
Hi @papaerai I'm having a lot of troubles in Installing Sageattention.. would it be possible to have this template with Sageattention properly installed?🙏
I'm currently using this template:
🖥️ Regular Template - For most GPUs (CUDA 12.4)
@Huski what error you getting did you activated venv first?
yeah, basically says that Torch is not installed, but it actually was.. and for some reason I wasn't able to correctly install SageAttention
would need see full error
If I use a Network Storage that I've used with the Better ComfyUI Standard template and a 4090, will it cause issues if I switch to the 5090 optimized workflow, and a 5090 card?
slim templates are separate ones
wdym?
basically instead of the Regular one, I'd like to just use the 5090 optimised one (with a 5090 card) just not to redownload all models n stuff
would it be possible to add Sageattention to the template as well? it's a very useful optimization
You can install it into the venv
just to be sure, all these GPU will work in the 5090 template?
just because sometimes to experiment I'll just choose a less expensive one
Yup if you use filter and pick cuda 12.8
it takes like 10/12 minutes to load ComfyUI every time I open the pod.. I don't think that's quite normal, isn't it?
when I open the Report seems like it's downloading all sorts of stuff every time..
I'd love to keep using Runpod, but it'd need to receive assistance on this...is there I way I can get some help from your team?
my templates are to use with network storage as it installs comfyui
yes, I use my network storage all the time 😦
it should install once and then start
does it have to download new stuff if I change GPU?
I'm sorry for the dumb questions but I'm very new to this🙂
I just want to understand if it can cause problems to sometimes change GPU while using the same network storage and 5090 template