CA
wise-white
dedupe queue
Hey All, When running a Task from the console, is there any way to dedupe the queue? I imagine that would be kind of a default, and an easy toggle in the config screen. I am havnig the same URL crawled twice per run.
1 Reply
Hi! Its deduped by default, so if URL crawled twice it means actor enforcing it by logic in code.