Performance Difference between machine u3q0zswsna6v88 and cizgr1kbbfrp04

Hey all, what are the difference between these two machines? For the exact same code, u3q0zswsna6v88 takes 60s and cizgr1kbbfrp04 takes 8s. I repeat the same request multiple times and none of these requests hit cold start. Happened around 5:15pm today.
Thanks!

ashley•1/11/24, 5:02 AM

You probably have flash boot enabled and the one that took 8s was already warmed up while the one that took 60s still had to load the model etc from a cold start.

sssstevenOP•1/11/24, 2:47 PM

I checked the logs. None of them hit cold start and all the models are loaded.

sssstevenOP•1/11/24, 2:47 PM

sssstevenOP•1/11/24, 2:48 PM

The same input for all these requests, but the execution time is very different...

sssstevenOP•1/11/24, 2:54 PM

Here are more tests:

sssstevenOP•1/11/24, 2:54 PM

sssstevenOP•1/11/24, 2:54 PM

request 09ab4d26 hit cold starts and it takes 30s

sssstevenOP•1/11/24, 2:54 PM

request daf19ba4 no need cold start, finish in 8.61s

sssstevenOP•1/11/24, 2:55 PM

request 2ad5f3b6 cold start again

sssstevenOP•1/11/24, 2:55 PM

request 5675044f cold start on d1nlfjprou0nsj, and it is so much slower

sssstevenOP•1/11/24, 2:55 PM

request 8a189d57 no need cold start, finish in 8.61s

sssstevenOP•1/11/24, 2:55 PM

request dec2ad75 no need cold start, finish in 44.73s

sssstevenOP•1/11/24, 2:56 PM

im very confused about these two workers: d1nlfjprou0nsj and gnlbiyyjbgaca7

sssstevenOP•1/11/24, 2:58 PM

they are all the same input and code...

sssstevenOP•1/11/24, 7:15 PM

@Justin can you point me to how I can debug this? Thank you so much!

Sssssteven @Justin can you point me to how I can debug this? Thank you so much!

Justin Merrell•1/11/24, 8:27 PM

What type of work is being performed? Is it CPU intensive? Did you also confirm that the works are both using the same GPU type?

sssstevenOP•1/11/24, 9:00 PM

it's SDXL inference with custom comfy workflow.

sssstevenOP•1/11/24, 9:00 PM

I only selected 80GB in the list and it does print out the same A100 type

sssstevenOP•1/11/24, 9:01 PM

are these workers only performing my handlers? I mean are the CPUs also shared with others?

Sssssteven are these workers only performing my handlers? I mean are the CPUs also shared w...

Justin Merrell•1/12/24, 3:12 PM

CPU is dedicated correct, however the type of CPU between each worker might be slightly diffrent.

Performance Difference between machine u3q0zswsna6v88 and cizgr1kbbfrp04

Similar Threads

Similar Threads

Similar Threads