Rate limit based on key
Is it possible to rate limit based on a key? So basically it would only process 1 URL at a time per key.
2 Replies
xenial-black•2y ago
you can try maxConcurrency option but it wouldn't be actual rate limiting though
optimistic-gold•2y ago
There is no native way at the moment.
An idea is to start with one URL for each key, and pass the rest of the URLs to the
request.userData object, then after each URL is processed, add another one from request.userData to the queue.