RTX 5090 model loading extremely slow compared to other GPUs
We have been experimenting with RTX 5090 for some of our workflows and we are finding these machines are extremly slow at loading the model. In a ComfyUI workflow for example, the identical checkpoint loading take 2 seconds on other machines but takes 60 seconds on any of the RTX 5090 machines we have tested.
11 Replies
are you loading from a network volume?
or from local storage (container disk)
Local storage on both
hmm?
both secure cloud right?
Yes, the RTX 5090s we have tried so far have been in the EUR-NO-1 will see if see the same thing in other data centers
container disk or pod volume?
im pretty sure that doesn't matter but just in case
container disk, but its the same on both
comfyUI shouldn't download anything right?
No. nothing. Identical workflows on all a very simple SDXL T2I
lol
Interesting enough if we set the comfy to --gpu-only this problem is mostly gone, my current take is that this skips the step from loading to ram and then to VRAM and goes directly to VRAM, meaning loading to ram is very slow on these
👋
did you fix it