Crawlee & Apify•3y ago

socks5 passwore protected proxies

Dear all, how can I use socks5 proxies with crawlee ? Also in general of the proxy is password protected how to put it in proxyUrl ? I didnt find any exmaple to use password protected proxies and socks5 proxies are not supported by defualt. Anyway to get around it ? Best Regards

3 Replies

optimistic-gold•3y ago

Hi, we had an interesting conversation here, it's still not working for me on firefox/chromium with playwright thought https://discord.com/channels/801163717915574323/999250964554981446/1080132532495986819 I tried this with playwright/firefox:

 launchContext: {
        launchOptions: {
            proxy: {
                server: `socks5=https://${PROXY_DOMAIN}:${PROXY_PORT}`,
                username: 'username',
                password: 'password'
            } 
        },
    },

 launchContext: {
        launchOptions: {
            proxy: {
                server: `socks5=https://${PROXY_DOMAIN}:${PROXY_PORT}`,
                username: 'username',
                password: 'password'
            } 
        },
    },

and this one with playwright/chromium and puppeteer:

 
launchContext: {
        launchOptions: {
            args: [ `--proxy-server=socks5=https://${PROXY_USERNAME}:${PROXY_PASSWORD}@${PROXY_DOMAIN}:${PROXY_PORT}` ]
        }
    },

 
launchContext: {
        launchOptions: {
            args: [ `--proxy-server=socks5=https://${PROXY_USERNAME}:${PROXY_PASSWORD}@${PROXY_DOMAIN}:${PROXY_PORT}` ]
        }
    },

The second option using args in puppeteer launch works perfectly with my own implementation of my chrome scraper

dependent-tanOP•3y ago

Thanks for the help Mojo. But wouldnt that affect the algorithm by which crawlee rotate the proxies ? Here the proxy is bound to the browser but what if I want to rotate it ?

optimistic-gold•3y ago

good question

Gaming

Programming

socks5 passwore protected proxies

Did you find this page helpful?