R
RunPod5mo ago
dstn

ComfyUI not launching

I've tried running ComfyUI using the runpod community template (ai-dock/comfyui:latest) and now both buttons in the "Connect" modal point to the "Service" endpoint even though the 8188 port should open the web interface. Clicking that link (Connect to HTTP Service on port 8188) opens the service logs which are stuck with "Waiting for workspace mamba sync..." repeating. I would expect ComfyUI to open on this port.
25 Replies
dstn
dstn5mo ago
No description
No description
dstn
dstn5mo ago
Now it says bad gateway.
dstn
dstn5mo ago
No description
dstn
dstn5mo ago
Here's a screenshot of the container logs from the runpod UI
No description
RobBalla
RobBalla5mo ago
The port opens with the logs until the sync is complete. This can take a while - when it's done the logging server vacates the port and comfy starts. You may have refreshed at the moment the changeover happens. The logs suggest that the ComfyUI service has started correctly. Did it remain unavailable?
dstn
dstn5mo ago
I don't know I stopped waiting. Last week these templates were up and running much faster. I'm trying again. It appears that ComfyUI is in a restart loop. You can see that also from the logs I shared yesterday.
dstn
dstn5mo ago
dstn
dstn5mo ago
Just tried again with the ghcr.io/ai-dock/comfyui:latest template and the logs are frozen at:
2024-01-08T21:25:49Z a8afc35451e3 Pull complete
2024-01-08T21:25:49Z 4a7f28ae2d5f Extracting [==================================================>] 2.203kB/2.203kB
2024-01-08T21:25:49Z 4a7f28ae2d5f Extracting [==================================================>] 2.203kB/2.203kB
2024-01-08T21:25:49Z 4a7f28ae2d5f Pull complete
2024-01-08T21:25:49Z 4199bb7a754f Extracting [====> ] 32.77kB/372.7kB
2024-01-08T21:25:49Z 4199bb7a754f Extracting [==================================================>] 372.7kB/372.7kB
2024-01-08T21:25:49Z 4199bb7a754f Extracting [==================================================>] 372.7kB/372.7kB
2024-01-08T21:25:49Z 4199bb7a754f Pull complete
2024-01-08T21:25:49Z Digest: sha256:5ca9e313cf1ec1732800d66bf943d9aba5ca14d4426b5e5f0c97cc4c90a67550
2024-01-08T21:25:49Z Status: Downloaded newer image for ghcr.io/ai-dock/comfyui:latest
2024-01-08T21:25:49Z create container ghcr.io/ai-dock/comfyui:latest
2024-01-08T21:25:50Z latest Pulling from ai-dock/comfyui
2024-01-08T21:25:50Z Digest: sha256:5ca9e313cf1ec1732800d66bf943d9aba5ca14d4426b5e5f0c97cc4c90a67550
2024-01-08T21:25:50Z Status: Image is up to date for ghcr.io/ai-dock/comfyui:latest
2024-01-08T21:25:51Z start container
2024-01-08T21:25:49Z a8afc35451e3 Pull complete
2024-01-08T21:25:49Z 4a7f28ae2d5f Extracting [==================================================>] 2.203kB/2.203kB
2024-01-08T21:25:49Z 4a7f28ae2d5f Extracting [==================================================>] 2.203kB/2.203kB
2024-01-08T21:25:49Z 4a7f28ae2d5f Pull complete
2024-01-08T21:25:49Z 4199bb7a754f Extracting [====> ] 32.77kB/372.7kB
2024-01-08T21:25:49Z 4199bb7a754f Extracting [==================================================>] 372.7kB/372.7kB
2024-01-08T21:25:49Z 4199bb7a754f Extracting [==================================================>] 372.7kB/372.7kB
2024-01-08T21:25:49Z 4199bb7a754f Pull complete
2024-01-08T21:25:49Z Digest: sha256:5ca9e313cf1ec1732800d66bf943d9aba5ca14d4426b5e5f0c97cc4c90a67550
2024-01-08T21:25:49Z Status: Downloaded newer image for ghcr.io/ai-dock/comfyui:latest
2024-01-08T21:25:49Z create container ghcr.io/ai-dock/comfyui:latest
2024-01-08T21:25:50Z latest Pulling from ai-dock/comfyui
2024-01-08T21:25:50Z Digest: sha256:5ca9e313cf1ec1732800d66bf943d9aba5ca14d4426b5e5f0c97cc4c90a67550
2024-01-08T21:25:50Z Status: Image is up to date for ghcr.io/ai-dock/comfyui:latest
2024-01-08T21:25:51Z start container
for about 10 min now
dstn
dstn5mo ago
no ComfyUI folder either
No description
dstn
dstn5mo ago
restarting the pod to see if it fixes itself I see the volume percentage creeping up at about 1% each 5 min. So I'm guessing things are just really slow over there. Do I get charged for the entire startup process as well?
RobBalla
RobBalla5mo ago
yeah syncing to network volume can be painfully slow and barely worth it. I never sync I just store models in /workspace/storage (see docs for the paths) and have my provisioning script pull nodes at start. Any models you store in the correct locations will be linked into the ComfyUI directory. Makes more sense that way to me and is much closer to how we should expect to use docker
dstn
dstn5mo ago
I'm running from the template mentioned above with no network drive and am 30 min in without ComfyUI available. Still stuck on "start container" and the Volume drive is now at 42%. Was at 10% 20 min ago.
RobBalla
RobBalla5mo ago
It's definitely still syncing if it's creeping up. It can be incredibly slow depending on region and other variables we cannot control. Try starting with a clean network volume and WORKSPACE_SYNC=false it should start very quickly. Models will be stored on the volume, just nodes won't persist And if an update is breaking comfy you can also run with AUTO_UPDATE=false
dstn
dstn5mo ago
ok, i'll try that. i have a custom docker build based off ai-dock/comfyui with no models, just nodes. i'll add that flag to my custom template and try again. then i'll sync my network volume to workspace/storage.
RobBalla
RobBalla5mo ago
If you're building the image then you definitely want to disable sync because you have full control of what is persisted to the container disk. Your only use for the volume is saving outputs and storing models so sync is actually going to give you far more trouble than it's worth Just make sure you replace the output dir from /opt/ComfyUI with a symlink to an appropriate location on the volume at startup
dstn
dstn5mo ago
ok, i'm not sure how to do that. that's in addition to changing the volume mount path?
No description
RobBalla
RobBalla5mo ago
No you'll need to do it at startup, so tag it on the end of the provisioning script You should also keep the volume mount as /workspace - a storage directory will be created there for you. Mounting there may also be fine but could cause bugs I haven't thought about yet But it will prevent anything you save in /workspace outside of storage from being saved, so there's no advantage to be gained
dstn
dstn5mo ago
are you suggesting I don't use the cloud sync feature to pull down models stored in my bucket but instead use a provisioning script to download models?
RobBalla
RobBalla5mo ago
It's what the provisioning script was designed for, but it's entirely up to you. I designed my container structure to be as flexible as possible but ultimately it is your choice how to manage it. Given that you'll only save to your volume once it probably doesn't matter how you get the models there - do whatever is easiest.
dstn
dstn5mo ago
thanks for your help @RobBalla . closing this issue now.
gabzak2024
gabzak20242mo ago
Hi. I'm trying to run ComfyUI from a POD, but I don't know hot todo it. I'm new on this. This is. my current situation:
No description
No description
gabzak2024
gabzak20242mo ago
Basically after that I have been doing: 1. Start web terminal 2. Connect to WebTerminal 3. Idon't know what to write in the terminal window. I have been trying anything and it doesn't work
digigoblin
digigoblin2mo ago
There is a README, read it.
gabzak2024
gabzak20242mo ago
Already even copied the codes written there and nothing works. Any idea?
digigoblin
digigoblin2mo ago
Works fine if you connect to port 3020. Check the container logs, its probably still syncing to /workspace.