how much vram? i think min 8 necessary
how much vram? i think min 8 necessary
device_map had weights offloaded to the disk. Please provide an offload_folder for them. Alternatively, make sure you have safetensors installed if the model you are using offers the weights in this format.