He writes fine tune vs extracted lora vs lora on another image, so I'm guessing it's just lora
He writes fine tune vs extracted lora vs lora on another image, so I'm guessing it's just lora
KDoes this training mean I could have ran it longer?!
SWould be strange though since all the other lora showcases it looked pretty much like him but in this comparison his lora looked nothing like him. Makes me wonder if the other ones are cherry picked
KInteresting thought
K
KYou are correct... If that's a LORA it's not very good and his usually are
KI mean the eyecolour for once... Completely off
SYeah exactly
SStrange and unfair comparison if that’s the case
NI was noticing that with SDXL as well, but didn't want to overfit due to all my training bashing my 4090 to a pulp... maybe its time to transfer to the cloud? 
KWTF?!
Trained a LORA on the "The Next Day" concept car from Lynk&Co yesterweek. After 4000 steps I get this:
Trained a LORA on the "The Next Day" concept car from Lynk&Co yesterweek. After 4000 steps I get this:
KSame prompt afte 4000 steps Dreambooth finetune:
KDreambooth is not even close and it's heavily Mercedes inspired
KWell that was 14 totally wasted GPU-hours
Mooof
Myou should really sample every 100 or 200 steps when training
KLR UNET was set according to @Dr. Furkan Gözükara config file (4e-6 = 0,000004), but with AI-toolkit I have been training LORA with 1e-4 (0,0001)
Mwell yeah dreambooth you need a 1/10 off the learning rate
Mso if you're doing 1e-4 for lora you need 1e-5 for dreambooth
KYeah well... It really doesn't seem like it
M4e-6 is too low for concepts
Mheck i used 3e-5 and 5e-5
KIt has learned out of 4000 steps what the LORA learned the first 500
KI would say that @Dr. Furkan Gözükara config is way off
Mkeep in mind furkans configs are for people training
Mnot cars
KHave I missed something in his video maybe
KYeah, but seeing how little it has learned I have a hard time thinking this would be enough for a face
M
KI guess "I'll" spend 14 more GPU hours on 0,00001
KMaybe torch 2.5.0 could help shorten the time 
NGood luck sir
Aany one has past experience in uploading custom lora to hugging face ?
Aneeded a bit help
DI made a comparison between ai-toolkit vs kohya lora training. For all the same parameters except for rank(kohya:128/128, ai-toolkit:64/64), kohya lora gives much better likeness and overall image quality. Does anyone have any observation on this or have a comment why kohya lora outperforms?
KKohya-installation according to @Dr. Furkan Gözükara = ?
KYepp... Same with Torch 2.5.0
KSo many errors... Soooo many errors 
KI hate torch and "generating weels"
KStill... Seem to be faster than 2.4.0
DDr. Gozukara, have you got any experiments on greater learning rates than 5e5 with fewer iterations? What have been your observations?
�It will outperform you are using a higher rank in kohya so it will learn more
�On how many images are you training? are you using captions? what learning rate? what network dim and alpha?
�@Dr. Furkan Gözükara How are you mentioning what blocks to train in kohya gui? I tried this but got an error -
Fyes weight 1.1 most close
Fi will post an article
Fi am aobut to post an article grid
Fdifferent workflows
Fkohya is more professionals has more experience 
Flooks like xformers requires new cuda
Fi will just remove it