is "Rank_3_18950MB_9_05_Second_IT" config for gpus with 24gb vram bf16 or are only the configs for 4
is "Rank_3_18950MB_9_05_Second_IT" config for gpus with 24gb vram bf16 or are only the configs for 48gb vram bf16?
Frank 3 is for 24 gb
Frank 1 and 2 for 48gb
W@Dr. Furkan Gözükara Using the Rank5 and the latest Torch 2.5, I'm trying to train, it gets here and then just seems to not be doing the training. Also even though it says it uses less memory, my GPU is still using 23.37g out of 24.
Wnevermind it just started training although extremely slowly. If the lora source images, and target direectory are on a slower drive would that cause a delay?
Fit can cause delay in initial
Fstart
Fbut not after it
LIm curious how much RAM are you running with, looks like FLUX is heavy on the PC RAM as well
Fye it uses a lot
Fi have 64 gb so never cared :d
Falso you can set virtual ram
Fram is nowadays easier
LYeah I have more coming in the mail, but I think paging is what is causing some peoples slow down
LHey, so I ran a couple of successful Flux lora's. with the rank three settings, but. the file size is like 2.3 gigs and I'm thinking I must have done something wrong to end up with that file size. I remember having that mistake before when I accidentally ran it on the Dreambooth tab instead of the Lora tab. But that's not what I did here. Is there anything apparent that I'm doing wrong?
LFrom the video it looked like Furkans where about that size
LOh, so maybe that's the expected size using this approach. It's big, but if it works, then I'm totally fine with that.
LYeah I tried the rank 12 which does i think 512x512 training and the model was 525MB
LI am sure it goes up in size with the resolution, since it is gathering more information
Mis rank 3 config bf16?
Fall are bf16 but rank 5 and lower cast into fp8
Fsome 16 gb configs doesnt cast but they are lower resolution
MI was using this extension https://github.com/cheald/sd-webui-loractl with a1111 to change the loras weight at different steps for example I can start one lora at first step and stop it at 15th step and start another lora at 15th step until the end. Now I'm looking if I can do it in SwarmUI and the only thing I found is Lora Block Weight node in ComfyUI-Inspire-Pack.
MSo could I download this https://github.com/ltdrdata/ComfyUI-Inspire-Pack and use it inside SwarmUI?
Fnot sure may not work
Sit has been running over 3hrs and only 844/2800 done. How do I pause it to continue later? Running 14 photos on RTX 3060 12GB.
J@Dr. Furkan Gözükara You testing T5 training yet?
Fdidnt have chance yet
Fi have been trying to fix broken scripts reply people :d
Fsadly you cant
Fthere is use last checkpoint for kohya but people say not working i didnt test
Flook at the tips
F#tips-info
Sok, so the average speed is OK for my setup? ~4hrs and only 875/2800 on 12GB RTX3060
F20 second / it is best
Fyou can get with 1024
Ftell me your step speed
S21 sec/it, avg loss 0.4
Fexcellent news we got for 16gb gpus
F
Fit is good
DI believe there were already 2 configs/ ranks (one slow & the other fast) for 16GB GPUs?
Swhere do I type my prompt to generate images? Or is Kohya_SS GUI doesn't support it?
Mthe learning rate in the flux config with single gpu is 0.00005. with sdxl it was 0.0004. is there a reason why it is much smaller? and does it mean we have to train it twice as much steps? so for example if i was training 4500 steps with sdxl do i have to train more than that because the learning rate is much smaller?
X"ERROR: Could not find a version that satisfies the requirement xformers==0.0.27.post2 (from versions: none)
ERROR: No matching distribution found for xformers==0.0.27.post2" Getting this error in very first step for "FLUX LoRA Training Simplified" I ran "pip install xformers" and it says that the xformers installed succsefully but when i try to install the first step again it says the same error
ERROR: No matching distribution found for xformers==0.0.27.post2" Getting this error in very first step for "FLUX LoRA Training Simplified" I ran "pip install xformers" and it says that the xformers installed succsefully but when i try to install the first step again it says the same error
TCan you do pip freeze in your venv?
TAnd post the result
XIm going to uninstall kohya_ss from my pc then reinstall now that i have necessary requirements hopefully it works
Xi uninstalled "kohya_ss from my pc" after getting this "ERROR: Could not find a version that satisfies the requirement xformers==0.0.27.post2 (from versions: none)
ERROR: No matching distribution found for xformers==0.0.27.post2
[notice] A new release of pip is available: 23.0.1 -> 24.2
[notice] To update, run: python.exe -m pip install --upgrade pip" installing xformers then reintalled it but still getting same error
ERROR: No matching distribution found for xformers==0.0.27.post2
[notice] A new release of pip is available: 23.0.1 -> 24.2
[notice] To update, run: python.exe -m pip install --upgrade pip" installing xformers then reintalled it but still getting same error
Sbro I had exact same problem and Gozukara replied it's fine
S