Pod stuck when starting container

J

Jason•10/9/24, 2:20 PM

maybe its on your program ( not sure )

J

Jason•10/9/24, 2:21 PM

how does your handler file looks like, or describe how it works maybe

Vvitalik Yesterday I updated my serverless endpoint with "New release" button. However wh...

E

Encyrption•10/9/24, 3:07 PM

Returning 0 is reporting success. If you want it to count as a failure it will need to return a number other than 0.

Suggest you open a ticket with RunPod (under help/contact on their site) they are generally pretty good about those kinds of things.

JJason how does your handler file looks like, or describe how it works maybe

V

vitalikOP•10/9/24, 6:59 PM

It is just starting ComfyUI and executing prompts from incoming requests

if __name__ == "__main__":
    wait_for_service(url="http://{}/prompt".format(COMFYUI_ADDRESS))
    print("ComfyUI  is ready. Starting RunPod...")
    runpod.serverless.start({"handler": handler})

if __name__ == "__main__":
    wait_for_service(url="http://{}/prompt".format(COMFYUI_ADDRESS))
    print("ComfyUI  is ready. Starting RunPod...")
    runpod.serverless.start({"handler": handler})

if __name__ == "__main__":
    wait_for_service(url="http://{}/prompt".format(COMFYUI_ADDRESS))
    print("ComfyUI  is ready. Starting RunPod...")
    runpod.serverless.start({"handler": handler})

if __name__ == "__main__":
    wait_for_service(url="http://{}/prompt".format(COMFYUI_ADDRESS))
    print("ComfyUI  is ready. Starting RunPod...")
    runpod.serverless.start({"handler": handler})

EEncyrption Returning 0 is reporting success. If you want it to count as a failure it will ...

V

vitalikOP•10/9/24, 7:01 PM

Ok, thanks. The weird part is that it kept trying to execute single request for many hours, logs look like this

2024-10-09 06:55:45.855[n5bwvtgs4p6x3t][error]worker exited with exit code 0
2024-10-09 06:55:29.577[n5bwvtgs4p6x3t][error]worker exited with exit code 0
2024-10-09 06:55:13.271[n5bwvtgs4p6x3t][error]worker exited with exit code 0
...

and so on

Vvitalik It is just starting ComfyUI and executing prompts from incoming requests `if _...

E

Encyrption•10/9/24, 7:07 PM

I think I see the problem, You should NOT be querying ComfyUI in main. That should happen in your handler function. In main this is before you have any request.

E

Encyrption•10/9/24, 7:10 PM

Does your handler function have similar code or does your handler function call another function with similar code as below?

# The top level element "prompt" is required by ComfyUI
data = json.dumps({"prompt": workflow}).encode("utf-8")

req = urllib.request.Request(f"http://{COMFY_HOST}/prompt", data=data)

# The top level element "prompt" is required by ComfyUI
data = json.dumps({"prompt": workflow}).encode("utf-8")

req = urllib.request.Request(f"http://{COMFY_HOST}/prompt", data=data)

# The top level element "prompt" is required by ComfyUI
data = json.dumps({"prompt": workflow}).encode("utf-8")

req = urllib.request.Request(f"http://{COMFY_HOST}/prompt", data=data)

# The top level element "prompt" is required by ComfyUI
data = json.dumps({"prompt": workflow}).encode("utf-8")

req = urllib.request.Request(f"http://{COMFY_HOST}/prompt", data=data)

EEncyrption Does your handler function have similar code or does your handler function call ...

V

vitalikOP•10/9/24, 7:15 PM

yes, it has exact same code and some other stuff like uploading generated images to s3

E

Encyrption•10/9/24, 7:15 PM

Are you using this one? https://github.com/blib-la/runpod-worker-comfy

GitHub

GitHub - blib-la/runpod-worker-comfy: ComfyUI as a serverless API o...

ComfyUI as a serverless API on RunPod. Contribute to blib-la/runpod-worker-comfy development by creating an account on GitHub.

E

Encyrption•10/9/24, 7:16 PM

I use that one and this is my complete main:

# Start the handler only if this script is run directly
if __name__ == "__main__":
    runpod.serverless.start({"handler": handler})

# Start the handler only if this script is run directly
if __name__ == "__main__":
    runpod.serverless.start({"handler": handler})

# Start the handler only if this script is run directly
if __name__ == "__main__":
    runpod.serverless.start({"handler": handler})

# Start the handler only if this script is run directly
if __name__ == "__main__":
    runpod.serverless.start({"handler": handler})

V

vitalikOP•10/9/24, 7:17 PM

nice, i'll have a look at it, thank you

E

Encyrption•10/9/24, 7:17 PM

wait... that is for Serverless Worker, not pod.

V

vitalikOP•10/9/24, 7:18 PM

yes, i'm using serverless

E

Encyrption•10/9/24, 7:19 PM

oh ok yeah then that shold work for you

Pod stuck when starting container

Similar Threads

Similar Threads

Similar Threads