Why do all workers use for same IP for sending requests?
It seems that all the requests sent by workers show the same IP (
This behaviour is somewhat unfortunate in cases where we use several workers for webscraping. In my case, if two workers happen to make the same request to a website I want to scrape data from, one of them is refused.
Can this be changed by some operation?
2a06:98c0:3600::103), as we can see when we request, e.g., https://icanhazip.com/.This behaviour is somewhat unfortunate in cases where we use several workers for webscraping. In my case, if two workers happen to make the same request to a website I want to scrape data from, one of them is refused.
Can this be changed by some operation?
