Runpod•2y ago

Issue in pod

i'm facing issue from last 2 days, sometime RTX 4090 generates 60 token/second and sometime it 30-20 token/second to generate same response.
don't know what is behind this ????

Runpod Join

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

21,202Members

Resources

Similar Threads

Was this page helpful?

Twitter GitHub Discord

Communities Docs About Terms Privacy

Star

Setup for Free

Twitter GitHub Discord

Communities Docs About Terms Privacy

Issue in pod - Runpod

Runpod•2y ago

Data_Warrior

Issue in pod

i'm facing issue from last 2 days, sometime RTX 4090 generates 60 token/second and sometime it 30-20 token/second to generate same response.
don't know what is behind this ????