Runpod•6mo ago

failing to start job

One of 10 times we are getting error message when trying to pass message. The error is no inside serverless container. Job is not getting processed by runpod itself. running from fastapi background_task. Sot the trace is not full

import runpod

train_endpoint = runpod.Endpoint(train_endpoint_id)
job = self.inference_endpoint.run(payload)

import runpod

train_endpoint = runpod.Endpoint(train_endpoint_id)
job = self.inference_endpoint.run(payload)

11 Replies

Dj•6mo ago

"Connection Reset", are you still having this issue?

kudesnikOP•6mo ago

Sorry, I didn't get your point? We are doing retrials and it helps. But I hope the serverless functionality is not intended to be so unstable

Dj•6mo ago

I was pointing out the error and asking if you were still seeing this issue. I'll take a look into it for you, this isn't normal activity and can happen for a lot of reasons.

Unknown User•6mo ago

Message Not Public

kudesnikOP•6mo ago

The payload is small, something like couple kb's. We are not sending any media in the payload. As for the blocking the connection. Up to my knowledge, there is no blocks from the country side. Is there a possibility of blocking on the runpod side? The theory about blocking is kinda weak, as we have the same static ip and requests fail sometimes, but not always

Unknown User•6mo ago

Message Not Public

Dj•6mo ago

Do you normally have connection issues to Cloudflare? Your connection through the proxy is a Cloudflare Tunnel :thinkMan: Another user saw success switching to a TCP port and their own CF Tunnel iirc

kudesnikOP•6mo ago

I am sorry for the late response. I just checked, Clodflare services are available. What do you mean by "Your connection through the proxy"? I am not using proxy Yo be precise, we are using Yandex Cloud as hosting provider I asked the our provider's assistance. They recommended to debug the connection using the following command. What port and ip should I set? mtr -rzbw --tcp --port <port> -c 20 <address>

Unknown User•6mo ago

Message Not Public

kudesnikOP•6mo ago

Any suggestions on debugging?

Unknown User•6mo ago

Message Not Public

Gaming

Programming

failing to start job

Did you find this page helpful?