url not timing out for a reason.

Hello, my scraper goes in an infinite run on some url for a reason even tho I put a timeout time

 request = r.get(url=url, timeout=1.5)

 request = r.get(url=url, timeout=1.5)

. Any ideas why?

12 Replies

wee-brownOP•15mo ago

I figured that it doesn't go in an infinite run but rather takes 1-2 minute to process the url or 5 minute sometime

2024-03-14T07:05:07.582Z https://dunamisenergy.com is an invalid url
2024-03-14T07:05:07.603Z Processing https://broadsign.com...
2024-03-14T07:07:38.962Z Processing https://heartlandchargingservices.com...

2024-03-14T07:05:07.582Z https://dunamisenergy.com is an invalid url
2024-03-14T07:05:07.603Z Processing https://broadsign.com...
2024-03-14T07:07:38.962Z Processing https://heartlandchargingservices.com...

2024-03-14T06:59:42.263Z Processing https://shinetsua.com...
2024-03-14T07:04:00.486Z Processing https://noralta.com...

2024-03-14T06:59:42.263Z Processing https://shinetsua.com...
2024-03-14T07:04:00.486Z Processing https://noralta.com...

noble-gold•15mo ago

Hello, is this a platform related issue or is it caused by some of your own logic how you process the request?

wee-brownOP•15mo ago

I believe it's platform related

noble-gold•15mo ago

could you give me id of the run

wee-brownOP•15mo ago

https://console.apify.com/view/runs/OmAMon9hYXW2CpIyS

Apify

Apify Console

Manage the Apify platform and your account.

MEE6•15mo ago

@Kirada just advanced to level 6! Thanks for your contributions! 🎉

wee-brownOP•15mo ago

wee-brownOP•15mo ago

here yahoo.com takes 1M30 second and yama-group takes 1m in my script, there is at maximum in total 21 requests that times out after 1.5s which is at maximum 31,5 second

wee-brownOP•15mo ago

and if the first request times out

wee-brownOP•15mo ago

it should skip it

wee-brownOP•15mo ago

here there is a time out time aswell

Gaming

Programming

url not timing out for a reason.

Did you find this page helpful?