How to reduce cold start & execution time?
Hi , i have a serverless endpoint and it have like 70 sec cold start and 50 sec execution time.I was trying to change the gpu's and
someting happened , it started to work so fast like 500ms cold starts and 10 sec exection time and output was fine?
How did that happen do you guys have any idea ? how can i achieve that again?
(Now it's broken, 70sec + 50 sec again) , its not about gpu's i guess im on 80GB gpu and it still tooks 50 secs.I dont know how that happened.Flash boot is enabled but its not working rn i guess.
someting happened , it started to work so fast like 500ms cold starts and 10 sec exection time and output was fine?
How did that happen do you guys have any idea ? how can i achieve that again?
(Now it's broken, 70sec + 50 sec again) , its not about gpu's i guess im on 80GB gpu and it still tooks 50 secs.I dont know how that happened.Flash boot is enabled but its not working rn i guess.
