you could provide info like, learning rates, optimizers, noise offset etc
you could provide info like, learning rates, optimizers, noise offset etc
DI could, I just have no reason to assume that is what makes the difference - because the difference is not just a little, but night and day
D
Dnevermind the "train TE1 embedding" switch, that's just something I'm currently trynig. my samples were without
Dalso, LR is not comparable to yours because I am using alpha 1
DI actually don't think this will be useful in the end, because it limits training of the concept you want to train (or 'instance' as kohya would call it) too much. but that's the opposite of the problem of what you're seeing
Dthat enough?
Moh wow your learning rate is super high
Dno it's not. it's because I'm using alpha 1.
Dif you are using 16 you can practically divide the LR by 16
Dwhat LR and alpha are you using?
M128x128 and 3e-06
Dthen my LR is approximately equivalent to 8e-06 in your world
Dso 3x but Flux is quite tolerant on a wide range of LR. I don't expect that to be the difference
DI think it's gonna be more productive if one of you just sends me their dataset and config, and I'll see if I can make it work.
Dit's not like Furkan's face is a secret, it's all over the internet 
Dso send me the 16 pics + config and I'll try
B@Dr. Furkan Gözükara which of your Flux Lora training configs is a good balanece of speed with true photo realism?
Bim trying to get the trainer on fal.ai to get as good results in terms of detail and realism as your kohya configs. the fast trainer doesn't do it very well so im using the regular trainer.
Fboth fast and good on 24 gb gpus
BThank you
Dusing embeddings or TE training might solve that issue though. Not bad on the left with embedding, on the right just "woman" with no bleeding.
FTE training doesnt solve
Ehey @Dr. Furkan Gözükara quick question. Can Flux 1.1 be Lora trained or even downloaded? As far as I see it can only be used in platforms at the moment.
Factually i trained single layer
Fit didnt solve too :d
Fnope they didnt publish any model
Dno it doesn't. it might solve MY problem that the "instance" is harder to train using regularization. it doens't solve yours. I don't know what's going on with yours. wanna share config+pics?
Fyes my config same as on patreon nothing different 
same as kohya one
same as kohya oneFmy dataset here
EI didnt' think so either, but then I just came across this? Looks like pro 1.1 is available?
https://civitai.com/models/618692?modelVersionId=922358
https://civitai.com/models/618692?modelVersionId=922358
Dsince we are looking for a specific problem I'd prefer to have original data. config file + image files. DM
Fi think only inference via api
Fsure sending dm
Bwhats the minium steps needed you think for training 40 images, for true photorealism?
Flets say 100 epoch for good resemblance
F4000 steps batch size 1
Bgreat thanks
Fi will update batch size 1 fine tuning configs
Ffor latest kohya
Bwhen training lora did you notice much difference between adafactor and adamw8bit?
Fi plan to investigate
Fnewest optimizers hopefully soon
Fbut each one would require different parameters
Fhahaha 4765 MB can do flux fine tuning now :d
Cexcept it cant. you know what dpo is?
Fit is i am doing right now. for single concept it will work perfect
Cyou can't fine tune, or train, flux. you are just creating something that gives you an image but you're not updating the weights or training the model, Doctor
Fif i am not updating weights of the model it must be magic then
Ci'll ask you again, do you know what DPO is