Routing issue
I have a listing website as INPUT and
enqueueLinks of it. These links (case studies) at the time has also multiple pages. When the cralwer adds the links with the new label attached, it's not happening anything. When using only case study page, it's scrapping the data and working. Not sure, what to do next and how to test it more. Does the Queue System waits to complete to add all links to start scrapping?3 Replies
Someone will reply to you shortly. In the meantime, this might help:
-# This post was marked as solved by Scai. View answer.
Hi @Scai would it be possible to put together some minimal reproducible example, when it happens? And probably an example of such a URLs.
enqueueLink usually put new link on the end of the queue. you may use forefront: true to change it https://github.com/apify/crawlee/issues/389 .GitHub
Add Socks Proxy support · Issue #389 · apify/crawlee
At the moment Apify has a cool option - 'proxy'. But unfortunately it does not support a socks type proxy. I think iIt would be awesome and very useful
xenial-blackOP•3mo ago
Yeah, from what I see, it was indeed forefront setting.