Crawl stops at 20 of 2000+ pages
Trying to crawl www.acecqa.gov.au and im seeing a count of 20, but no more. It stops there. Ive tried via API and got 72??? Not sure whats going on. I have adjusted for depth, max pages, and time to load to 5000. Any ideas?
0 Replies