Marches online
I want to set up an automation using make.com to scrape this website: https://www.marchesonline.com/appels-offres/en-cours. I would like to use the Apify module, specifically the "Web Scraper" actor, to perform the scraping. Specifically, I want to connect the actor to the site, which will click on each title (1,392 notices in total), scrape the data on the title's page, then go back to the main page and click on the second title, and then scrape the data on that page. It should do the same for all the titles on the first page of the site. After that, it should click on page 2 of the site and repeat the same process. The scraped data from each title's page should be sent to ChatGPT for analysis and data extraction. My question is: How do I configure the "Web Scraper" actor to perform this task for me? Are there other actors more suitable for this job? Thank you for your help, and if you have any useful resources, I’d be happy to take a look.
1 Reply
Which actor you're referring to? (Please share the link)
I took a look at the website. It's Cloudflare protected (complexity to bypass is medium) but the most important thing is that the search query is not in the URL but somewhere in browser cookies/local storage, unless you would add entering your filters to the automation process and use a browser based actor..
if you have a budget, I'd recommend posting this to #💻hire-freelancers , there's an amazing pool of talented devs who might be happy to help