❔ How to do faster HttpClient requests?
I've done a classic HttpClient initialize once and for-loop it and it takes about 0.7s per request, which is fine-ish but due to the size of my array I would really love to get it faster.
I'm aware that I may be API-limited, but what would be the best way going about proving that?
I've tried having the responses be saved in a concurrent bag, and parallel.foreach-ing my array while creating an httpclient in each iteration.
This was far from the best, and I've understood that. My current implementation has a ConcurrentQueue of HttpClients, with a maximum of 100 parallelisations at once.
Each thread would dequeue an httpClient, do the request and then enqueue it back. But I'm getting worse results like that. It's approximately 50-60 requests in 40s, which is still roughly 0.7-0.8s per requests.
Would this indicate an API limitation, or is my way of optimizing this wrong?
Thanks a ton!
(It's currently being done in a console-app with .NET 6, just to get proof it works)
I'm aware that I may be API-limited, but what would be the best way going about proving that?
I've tried having the responses be saved in a concurrent bag, and parallel.foreach-ing my array while creating an httpclient in each iteration.
This was far from the best, and I've understood that. My current implementation has a ConcurrentQueue of HttpClients, with a maximum of 100 parallelisations at once.
Each thread would dequeue an httpClient, do the request and then enqueue it back. But I'm getting worse results like that. It's approximately 50-60 requests in 40s, which is still roughly 0.7-0.8s per requests.
Would this indicate an API limitation, or is my way of optimizing this wrong?
Thanks a ton!
(It's currently being done in a console-app with .NET 6, just to get proof it works)
