RunPod•3w ago

failing to start job

One of 10 times we are getting error message when trying to pass message. The error is no inside serverless container. Job is not getting processed by runpod itself. running from fastapi background_task. Sot the trace is not full

import runpod

train_endpoint = runpod.Endpoint(train_endpoint_id)
job = self.inference_endpoint.run(payload)

import runpod

train_endpoint = runpod.Endpoint(train_endpoint_id)
job = self.inference_endpoint.run(payload)

11 Replies

Dj•3w ago

"Connection Reset", are you still having this issue?

kudesnikOP•3w ago

Sorry, I didn't get your point? We are doing retrials and it helps. But I hope the serverless functionality is not intended to be so unstable

Dj•3w ago

I was pointing out the error and asking if you were still seeing this issue. I'll take a look into it for you, this isn't normal activity and can happen for a lot of reasons.

Jason•3w ago

is there something blocking your connection to runpod? vpn's, blocking from country to external sites? and how big is your payload in mb's?

kudesnikOP•3w ago

The payload is small, something like couple kb's. We are not sending any media in the payload. As for the blocking the connection. Up to my knowledge, there is no blocks from the country side. Is there a possibility of blocking on the runpod side? The theory about blocking is kinda weak, as we have the same static ip and requests fail sometimes, but not always

Jason•3w ago

hmm yeah im just guessing, its best to open a ticket and let staff's handle these any endpoint id's?

Dj•3w ago

Do you normally have connection issues to Cloudflare? Your connection through the proxy is a Cloudflare Tunnel :thinkMan: Another user saw success switching to a TCP port and their own CF Tunnel iirc

kudesnikOP•2w ago

I am sorry for the late response. I just checked, Clodflare services are available. What do you mean by "Your connection through the proxy"? I am not using proxy Yo be precise, we are using Yandex Cloud as hosting provider I asked the our provider's assistance. They recommended to debug the connection using the following command. What port and ip should I set? mtr -rzbw --tcp --port <port> -c 20 <address>

Jason•2w ago

Wait this is for endpoint serverless tho, they shouldnt be using tunnels I think you can't do this to send a job

kudesnikOP•2w ago

Any suggestions on debugging?

Jason•2w ago

Oh it still throws that error? if you try it using a vps from a known vps provider like digitalocean/vultr/aws/azure/google cloud, des it work? did you run it from your local pc? maybe you should open a support ticket

Gaming

Programming

failing to start job

Did you find this page helpful?