when training lora did you notice much difference between adafactor and adamw8bit?
when training lora did you notice much difference between adafactor and adamw8bit?
Fi plan to investigate
Fnewest optimizers hopefully soon
Fbut each one would require different parameters
Fhahaha 4765 MB can do flux fine tuning now :d
Cexcept it cant. you know what dpo is?
Fit is i am doing right now. for single concept it will work perfect
Cyou can't fine tune, or train, flux. you are just creating something that gives you an image but you're not updating the weights or training the model, Doctor
Fif i am not updating weights of the model it must be magic then 
Ci'll ask you again, do you know what DPO is
Ftell me full words of it
Ddid you try man or woman
Cthe paper's on arXIv - i suggest you read it
Dthat's hilarious statement.
Dif you know so much about how Flux dev was created, please tell us how exactly the distillation from pro to dev was done, and how DPO was applied (which I'm not doubting - why wouldn't they)
EOK. Just asking cuz I have a 4090 & I've seen upwards of 60GB of system RAM usage when not rendering, when I start a render it goes down (I'm guessing that's loaded on to the gpu).
Cyou don't apply dpo to a base model - not if you're smart
Dtell us why. DPO is proposed as an alternative to RLHF which obviously was applied very successfully for LLMs
Ffine tuning configs for 6 gb, 8gb, 10 gb , 12 gb , 16gb and 24 gb
F
CI'll let you go read the paper and think
Dso you don't know the answer. alright.
Ci do, but i don't feel like hand-feeding you. and you trying to manipulate me into handing you the research you don't want to do isn't going to work
Dyou are making unsubstantiated claims that you know more about how Flux was created than the general public. anyone can read papers. you have to substantiate your claim.
Ci owe you nothing. i direct you, if you want a place to start your research other than arXiv, to all the posts on this discord ALONE where people talk about how many times they've tried to train flux for something and how it doesn't work. even Furkan himself has expressed issues with that. and that is all i'm giving you for free. go do the research
Cbe just as snarky as you like now, i'm done with you
Dblocked
Cblocking on discord is funny. you still see that the other person has posted something, but you can't see what. you can't tell if they're talking about you or not. blocking on discord is a good way to drive yourself crazy. and when you're the one that's tried to start an arguement, and then block the person you were trying to argue with because they wouldn't play your game, that says something about you to everyone watching
BWhats the best upscaler model that allows commercial use?
Ctopaz's models
Bcan they be intergrated into comfyUI?
Cno. they're propiratory and exclusive to topaz, and not free - but they are the best and you can use the output commercially
Byeah its not the output commercially im looking to do. Its for a the backend of a SaaS product.
Dcmon everyone, be nice to each other
D@Dr. Furkan Gözükara you can try flux finetuning with regularization and in the last epochs remove the regularization, may be the model get resemblance at the end but stays more flexible
Fit is an idea
Fmay work
Fi am testing each gpu config to get max speed
Fman this is taking time
Dflux-dev-de-distill is giving this error @kohya_tech is already informed and he said that he will fix it as soon a possible.
Fwow batch size 7 way faster than before
Fkohya did huge improvement
Ftraining takes under 2 hours haha
F3000 steps
F
Fah but my current speed test of batch size 7 not accurate
Fi need to give bigger
Fye made the images count 256 testing its speed again
Fye for 256 images not that fast
Fso i guess best would be testing 14 images
D@crystalwizard You keep claiming that flux-dev can't be trained I was doing many tests with flux-dev-de-distill, I think that it can work. Its behavior on inference is completely different than regular flux-dev it works perfect without distilled cfg, what they did to model probably works, its behavior is similar to sdxl, if you prompt random you get random noise but if you prompt properly you get a perfect image, on regular flux-dev with distilled cfg you always get a coherent image, did you test it?