url with commas
Hi, in Web Scraper I try to pass list of urls via Start urls -> text file unfortunately urls contains commas (for example https://www.xyz.com/en/page,id,2) and when I run the task it reads only url before first comma -> https://www.xyz.com/en/page How to pass/encode urls to have it with commas? Thanks!
1 Reply
graceful-blue•2y ago
When passing URLs that contain commas via the Start URLs option or any similar configuration, you need to ensure that the URLs are properly encoded to prevent confusion with delimiter characters like commas.
So, You should encode them before using them as start URLs.
The standard way to encode characters in URLs is by using percent encoding (also known as URL encoding).
In a URL, a comma is encoded as %2C. So, the URL you provided "https://www.xyz.com/en/page,id,2" would be encoded as "https://www.xyz.com/en/page%2Cid%2C2".
In this way, when you pass this text file as the source for Start URLs, the URLs will be correctly interpreted by the system without any confusion between actual commas and delimiter commas.
So, You need to modify your links in your file in this way.