Runpod•2y ago

Slow CPU

Hi, I'm facing a big problem! On A100, L40 graphics cards in Secure cloud I am experiencing very low CPU performance

On 3080, 3090ti in community cloud CPU speed is quite good

SBlackOP•3/26/24, 2:54 PM

this is speed from A100 pod.
I changed faces in ComfyUI, this technology uses CPU+GPU

SBlackOP•3/26/24, 2:54 PM

this is with 3090ti

SBlackOP•3/26/24, 2:55 PM

and this is my not powerfull laptop for comparison
(no external GPU, only processor)

SBlackOP•3/26/24, 2:58 PM

I realize that these computers have shared CPUs, but the speed is horribly slow.

flash-singh•3/26/24, 2:58 PM

if you can run on 4090s, I would use those, for some gpu servers, the cpu is better suited for multi-threaded workloads instead of single threaded ones, even your local laptop will likely have better single thread performance than some cpus with many more cores but low ghz per core

SBlackOP•3/26/24, 2:59 PM

I can't do anything that uses the processor even a little bit. With A100 and L40 (maybe other pods too?)

flash-singh•3/26/24, 3:00 PM

try the A40s

ashley•3/26/24, 3:00 PM

Why is ReActor using CPU and not GPU? GFPGAN is incredibly slow with CPU, its MUCH faster with GPU.

Fflash-singh if you can run on 4090s, I would use those, for some gpu servers, the cpu is bet...

SBlackOP•3/26/24, 3:01 PM

i checked on my home pc, all threads are used

SSBlack i checked on my home pc, all threads are used

ashley•3/26/24, 3:01 PM

Thought you are using ComfyUI? This is A1111.

Aashley Why is ReActor using CPU and not GPU? GFPGAN is incredibly slow with CPU, its MU...

SBlackOP•3/26/24, 3:02 PM

its using cpu AND gpu together, that is onnxruntime work

ashley•3/26/24, 3:02 PM

A1111 extension can be configured to use GPU.

flash-singh•3/26/24, 3:02 PM

you can go to filter and find pods with more cpu cores, give you more performance

SSBlack its using cpu AND gpu together, that is onnxruntime work

ashley•3/26/24, 3:02 PM

Face swap is fast, its the restoration using GFPGAN thats slow.

Aashley Thought you are using ComfyUI? This is A1111.

SBlackOP•3/26/24, 3:03 PM

yes i see that this is a1111! i checked maybe this is comfyui problem, but no

ashley•3/26/24, 3:04 PM

I do this to make ReActor extension for A1111 use CUDA instead of CPU.

Aashley Face swap is fast, its the restoration using GFPGAN thats slow.

SBlackOP•3/26/24, 3:04 PM

look at the first screenshots, swapping was not so fast. about 13 seconds. on normal CPUs this takes 1-3 seconds, ok 5 on slow

Aashley I do this to make ReActor extension for A1111 use CUDA instead of CPU.

SBlackOP•3/26/24, 3:05 PM

"reactor force" version working only with CUDA

Fflash-singh you can go to filter and find pods with more cpu cores, give you more performanc...

SBlackOP•3/26/24, 3:05 PM

i know, i test with pod with 16 cores

Fflash-singh you can go to filter and find pods with more cpu cores, give you more performanc...

SBlackOP•3/26/24, 3:06 PM

this is 16 cores

SBlackOP•3/26/24, 3:08 PM

i sure that 2 cores on community cloud will be faster, but i spend all money for this "tests"

flash-singh•3/26/24, 3:20 PM

do you see all cores being used when you run the test?

Fflash-singh do you see all cores being used when you run the test?

SBlackOP•3/26/24, 3:31 PM

no i didn`t try it

SBlackOP•3/27/24, 8:46 AM

SBlackOP•3/27/24, 8:49 AM

this is community 3090ti with good speed

flash-singh•3/27/24, 12:08 PM

did you try selecting many cpus in filters option

Fflash-singh did you try selecting many cpus in filters option

SBlackOP•3/27/24, 2:28 PM

yes, i try 16 vcpu

Fflash-singh did you try selecting many cpus in filters option

SBlackOP•3/27/24, 2:30 PM

now i`m testing a6000 on "NO" datacenter, speed is ok. About 3 seconds

SBlackOP•3/28/24, 12:24 PM

this is slow!

SBlackOP•3/28/24, 12:24 PM

this is fast!

SBlackOP•3/28/24, 7:32 PM

@flash-singh

flash-singh•3/28/24, 9:22 PM

will have to dig deeper and see if they have different cpu, noisey neighbhor will cause cpu to be more balanced, while in community cloud or some servers, you may get more cpu than your allocated as we allow spikes in cpu activity if no one else is using it

Fflash-singh will have to dig deeper and see if they have different cpu, noisey neighbhor wil...

SBlackOP•3/28/24, 9:45 PM

I realize that regulating CPU usage on virtual machines is probably not as easy as it seems. But you rent out powerful server graphics cards, in my humble opinion you should give more than the minimum power on the CPU. Or at least offer to pay extra for a higher power limit

In large pipelines part of the work is done by the processor and a powerful video card becomes useless if everything hangs on small processor calculations

SSBlack I realize that regulating CPU usage on virtual machines is probably not as easy ...

flash-singh•3/28/24, 10:39 PM

We have plans to make CPU more isolated, right now its shared. CPU pods are more isolated since cores are dedicated in those cases.

Fflash-singh We have plans to make CPU more isolated, right now its shared. CPU pods are more...

SBlackOP•4/5/24, 6:47 AM

i cant use anything now, very slow(

SBlackOP•4/5/24, 7:18 AM

@flash-singh ComfyUI starting more than 6 minutes!! i have a video

SBlackOP•4/5/24, 7:19 AM

it should be 10 seconds

flash-singh•4/5/24, 4:16 PM

its likely writing and reading back and forth from network volume, thats been the slowest part for anyone using network volume as main disk to read from and write to

Slow CPU

Similar Threads

Similar Threads

Similar Threads