Yeah I would like to extract the lora.
Yeah I would like to extract the lora.
D
Wtraining checkpoint of flux is require much better hardware than lora, so training a checkpoint then extract a lora is not an efficient way to get a lora. I rather train a lora directly
V@wardensc2 I tried the 4090 config the Dr. had an it said about 4000 hours for 100 images.
V96gb ram
Wit's true
VI had to edit his lora config as well, was too slow.
WI train an rank 128 lora fp16, I need 4 a6000 with 48gb Vram and nearly 5 hours
Wbut I get the best quality lora I ever have
VI can only do 64 rank since I need to work as well.
V2.61 s/it
Wthat why we use runpod or mass compute
Wbecause we can get something like H100 or A100 which speed is double or quadruple of the 4090
VYeah I plan on switching but wanted to learn how to do everything before being charged $$$ for failures.
DOn my 4090 it took 2.5 hours to train a lora
Wwhen I use 4 a6000 basically I have a vga with 4x speed of a 3090 and 48gb Vram
V@daroth How many epochs?
Wcost me only 4 usd for a lora
VI train slow and go for around 20.
Wlow rank lora as well
Wyes I know but I need best settings from you = ))))
WH100 = 2xA100 in term of speed
Ddunno. limited it to 2500 steps. rank doesn't affect speed really
DI will test a full train with regularization images and see the result, other than that is the same config, now I'm using higher learning rate to speed up the training and get results faster, but use the Dr. Furkan setting for best quality
Dcaption for the subjects very simple "name man" "name woman" etc
WYou dont use keyword prompt in Kohya don't you ?
W
DI use the name of the folder for captions and the repetitions if needed for regularization or for dataset balancing
Dfolder names "1_Name man" "1_Name woman" in that case 1 repetition
UDremember Flux is case sensitive
Dso for the Name I use capital letters
Xthis looks amamzing
XV@Dr. Furkan Gözükara Have you tried flux dora via onetrainer? Supposedly its comparable to extracting a lora from a full finetune.
Hthanks 
Dif we're doing a full finetune (then extract) instead of regular lora training, is the ~2000-2500 steps still valid?
ATried searching but can't find it (I assume it's in the video tutorial but I can't watch video from work) Can any SwarmUI user tell me where I need to put the T5xxl model so SwarmUI will detect it? Trying to figure out how to use Swarm for Flux and this is the only bit I can't see anywhere
Dmodels\clip
Fit should load your last saved preset
Fif i remember correctly
Fi never compared
Fnot that i know sadly
Fit should scale very well with H100. i would say speed is even faster than lora best settings if you have 48 gb gpu. if you have 24 gb little bit slower
Fsadly looks like min 48 gb is needed for 12 gb gpus
Fi think not. not even fp8 works
Ffp8 training works for lora but not for fine tuning
Fsadly fp 8 fine tuning not supported yet. i asked kohya and he said he doesnt know how to
Fbatch size 7 fine tunng take 3 hours for 3000 steps
Fso 1$ 
Fhopefully will fully research this