Proxy fails on SSL secured(httpS) websites
Hey!
I'm trying different proxy providers and I've noticed the issue in the title.
I'm setting the proxy in in the following format:
as I usually do. But with the current providers I'm testing, the request will fail with either 407 (Proxy Authentication Required) or 422 responses.
Strangely enough, if tried with - it works.
Any idea what could be causing it?
10 Replies
vicious-gold•3y ago
Can you provide the target URL? The syntax looks correct
fascinating-indigoOP•3y ago
example.com or amazon
vicious-gold•3y ago
Can you share more of the code? No one ever reported this so it is unlikely it would be a bug in Crawlee
fascinating-indigoOP•3y ago
Same proxy string works perfectly fine with curl, as mentioned above. Not sure why it "asks for authentication". Let me grab a screenshot with the resi proxies as well
fascinating-indigoOP•3y ago
There you go
fascinating-indigoOP•3y ago
Ahh, I think I know why it happens. There's a semicolon in the username and/or password in both cases. Let me see how should I escape it
@ThePhantom just advanced to level 2! Thanks for your contributions! 🎉
fascinating-indigoOP•3y ago
Can't seem to do so, though. Should not need to escape semicolon as it's not really a special character, but Crawlee is not happy with it :\
And that's with Cheerio. Same formatting works just fine with Playwright
vicious-gold•3y ago
I think I remember this issue but don't remember how I solved it 😄
fascinating-indigoOP•3y ago
Haha, happens to me too. That's why I started to "write things down"(thanks note taking apps and Github gists).
Let me know if you remember the solution. It'll be helpful for others as well, I'm sure 🙂