Rate Limiting for OpenAI: Handling Multiple Parameters and Estimating Limits
Another question around rate limiting, I'm currently looking at using the rate limiter for OpenAI: https://platform.openai.com/docs/guides/rate-limits/usage-tiers?context=tier-three
There are 2 more tricky situations here:
1. I need to rate limit by 2 numbers: 1) Req / min and 2) Tokens / min.
2. The actual rate limit to rate limit by I'll only have once I know the return tokens, but I have an estimate beforehand.
Any points on how that could look like highly welcome!
There are 2 more tricky situations here:
1. I need to rate limit by 2 numbers: 1) Req / min and 2) Tokens / min.
2. The actual rate limit to rate limit by I'll only have once I know the return tokens, but I have an estimate beforehand.
Any points on how that could look like highly welcome!
