Also, has anyone here rented and used an H100? Is there really a significant speed increase in train
Also, has anyone here rented and used an H100? Is there really a significant speed increase in training? Compared to 4x A100
VThere is also the method that ai toolkit uses to train. Some sort of addon model that is based on schnell. A few other distilled as well.
WTo be more specific I train 2 loras with about 30 images and about 5800 step, result is no improvement
VSame config?
Dflux-dev-de-distill is the best for training I'm very impressed, then you can use the lora on regular Flux-dev any quantization as normal
Wyes
Wjust change the checkpoint only
VMaybe it needs different config since the data is distilled.
Dthe best option will be to full finetune flux-dev-de-distill and then extract the lora
VI want to do that, but I doubt my 4090 can do it in a reasonable time.
Wdo you read the author said
Whe said he finetune with 3 million high quality data
WI bet he need to have a company like facebook or amazon
Wto do that
W= )))
WI dont buy it at all
Dthe config is the same for training, is different for inferring it uses real cfg rather than Distilled CFG Scale so is slower to generate pictures
Wbase on how long to train just 30 images for a checkpoint
Dfor that reason I use the trained lora on regular flux-dev
VYeah I would like to extract the lora.
D
Wtraining checkpoint of flux is require much better hardware than lora, so training a checkpoint then extract a lora is not an efficient way to get a lora. I rather train a lora directly
V@wardensc2 I tried the 4090 config the Dr. had an it said about 4000 hours for 100 images.
V96gb ram
Wit's true
VI had to edit his lora config as well, was too slow.
WI train an rank 128 lora fp16, I need 4 a6000 with 48gb Vram and nearly 5 hours
Wbut I get the best quality lora I ever have
VI can only do 64 rank since I need to work as well.
V2.61 s/it
Wthat why we use runpod or mass compute
Wbecause we can get something like H100 or A100 which speed is double or quadruple of the 4090
VYeah I plan on switching but wanted to learn how to do everything before being charged $$$ for failures.
DOn my 4090 it took 2.5 hours to train a lora
Wwhen I use 4 a6000 basically I have a vga with 4x speed of a 3090 and 48gb Vram
V@daroth How many epochs?
Wcost me only 4 usd for a lora
VI train slow and go for around 20.
Wlow rank lora as well
Wyes I know but I need best settings from you = ))))
WH100 = 2xA100 in term of speed
Ddunno. limited it to 2500 steps. rank doesn't affect speed really
DI will test a full train with regularization images and see the result, other than that is the same config, now I'm using higher learning rate to speed up the training and get results faster, but use the Dr. Furkan setting for best quality
Dcaption for the subjects very simple "name man" "name woman" etc
WYou dont use keyword prompt in Kohya don't you ?
W
DI use the name of the folder for captions and the repetitions if needed for regularization or for dataset balancing
Dfolder names "1_Name man" "1_Name woman" in that case 1 repetition
UDremember Flux is case sensitive
Dso for the Name I use capital letters