Runpod•2y ago

Intermittent connection timeouts to api.runpod.ai

{
"endpointId":"oic105cyzlovnk"
"workerId":"3cwou4m0x6hxl0"
"level":"error"
"message":"Failed to return job results. | Connection timeout to host https://api.runpod.ai/v2/oic105cyzlovnk/job-done/3cwou4m0x6hxl0/31062127-b452-4afa-a4ba-1d6f07134a4e-u1?gpu=NVIDIA+RTX+A5000&isStream=false"
"dt":"2024-06-13 19:16:55.63793120"
}

{
"endpointId":"oic105cyzlovnk"
"workerId":"3cwou4m0x6hxl0"
"level":"error"
"message":"Failed to return job results. | Connection timeout to host https://api.runpod.ai/v2/oic105cyzlovnk/job-done/3cwou4m0x6hxl0/31062127-b452-4afa-a4ba-1d6f07134a4e-u1?gpu=NVIDIA+RTX+A5000&isStream=false"
"dt":"2024-06-13 19:16:55.63793120"
}

8 Replies

digigoblinOP•2y ago

@1AndOnlyPika is also experiencing this Endpoint ID can be copied and pasted from my JSON above. @1AndOnlyPika can you add your endpoint id please?

Unknown User•2y ago

Message Not Public

digigoblinOP•2y ago

Yeah I think it retries when this happens, because I see I had 3 retried requests

Unknown User•2y ago

Message Not Public

1AndOnlyPika•2y ago

rtqb8oacytm879 in my worker code, i added a ping to a random website right before finishing the job and hwen it fails to return job restuls, the connection still goes through so its an issue with runpod and not the worker itself going to look into switching to a websocket for communicating with my worker and getting restuls

Unknown User•2y ago

Message Not Public

haris•2y ago

cc: @Satish

1AndOnlyPika•2y ago

2024-06-15T06:25:02.615876953Z b'66.114.112.126\n'
2024-06-15T06:25:02.615921744Z {"requestId": "13258cfa-7493-4bfc-9400-02a9f9e81073-u1", "message": "Failed to return job results. | Connection timeout to host https://api.runpod.ai/v2/rtqb8oacytm879/job-done/2qf6cgmhd4onn7/13258cfa-7493-4bfc-9400-02a9f9e81073-u1?gpu=NVIDIA+GeForce+RTX+4090&isStream=false", "level": "ERROR"}
2024-06-15T06:25:02.615939904Z {"requestId": "13258cfa-7493-4bfc-9400-02a9f9e81073-u1", "message": "Finished.", "level": "INFO"}

2024-06-15T06:25:02.615876953Z b'66.114.112.126\n'
2024-06-15T06:25:02.615921744Z {"requestId": "13258cfa-7493-4bfc-9400-02a9f9e81073-u1", "message": "Failed to return job results. | Connection timeout to host https://api.runpod.ai/v2/rtqb8oacytm879/job-done/2qf6cgmhd4onn7/13258cfa-7493-4bfc-9400-02a9f9e81073-u1?gpu=NVIDIA+GeForce+RTX+4090&isStream=false", "level": "ERROR"}
2024-06-15T06:25:02.615939904Z {"requestId": "13258cfa-7493-4bfc-9400-02a9f9e81073-u1", "message": "Finished.", "level": "INFO"}

it calls icanhazip.com before and outputs it, so internet is working, just runpod isnt also sometimes get this

requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://api.runpod.ai/v2/rtqb8oacytm879/status/964637fc-0084-4872-a91f-0e82a88b592c-u1

Gaming

Programming

Intermittent connection timeouts to api.runpod.ai

Did you find this page helpful?