How to use multiple GPUs for Kohya Training?
I am using 2x gpus for training using Kohya(Dreambooth). But it is not utilizing both the gpus and instead only 1 gpu is being utilized.
I have tried following solutions:
1. added "set CUDA_VISIBLE_DEVICES=1" to gui.bat file of kohya_ss
2. added the parameters(hardcoded): "aceelerate launch --num_processes=[NUM_YOUR_GPUS_PER_MACHINE] --num_machines=[NUM_YOUR_INDEPENDENT_MACHINES] --multi_gpus --gpu_ids=[GPU_IDS] args..." in the run_cmd of dreambooth_gui.py
3. Changed the config files where I set num_processes = 2
Still somehow when I restart the training, only 1 gpu is being utilized. How can I solve this to use multiple gpu for training?
p.s. docker image I am using: ashleykza/stable-diffusion-webui:3.12.0
I have tried following solutions:
1. added "set CUDA_VISIBLE_DEVICES=1" to gui.bat file of kohya_ss
2. added the parameters(hardcoded): "aceelerate launch --num_processes=[NUM_YOUR_GPUS_PER_MACHINE] --num_machines=[NUM_YOUR_INDEPENDENT_MACHINES] --multi_gpus --gpu_ids=[GPU_IDS] args..." in the run_cmd of dreambooth_gui.py
3. Changed the config files where I set num_processes = 2
Still somehow when I restart the training, only 1 gpu is being utilized. How can I solve this to use multiple gpu for training?
p.s. docker image I am using: ashleykza/stable-diffusion-webui:3.12.0
