Crawlee + Proxy = Blocked, My laptop + Proxy = unblocked
I have a weird situation. whenever I try to access a website via crawlee and with proxy the request is blocked but with the same proxy I can access the website without any problem on my system and with many other browsers and also in incognito mode. Its really puzzling me.
Any help would be highly appreciated. Thankyou.
2 Replies
other-emerald•2y ago
- Site could be checking the hardware.
- You could try playwright with different browsers.
- You could try try to run headful browser
- You could try different proxies, try Residential proxies.
harsh-harlequin•2y ago
If by
on my system you mean regular browser then there is nothing weird about that. The site just detects that you are using automated browser and blocks the request.