Software Engineering Courses (SECourses)•16mo ago

Would be strange though since all the other lora showcases it looked pretty much like him but in thi

Would be strange though since all the other lora showcases it looked pretty much like him but in this comparison his lora looked nothing like him. Makes me wonder if the other ones are cherry picked

Kallamamran•9/18/24, 6:30 AM

Interesting thought

Kallamamran•9/18/24, 6:30 AM

Kallamamran•9/18/24, 6:31 AM

You are correct... If that's a LORA it's not very good and his usually are

Kallamamran•9/18/24, 6:31 AM

I mean the eyecolour for once... Completely off

smokeOP•9/18/24, 6:40 AM

Yeah exactly

smokeOP•9/18/24, 6:40 AM

Strange and unfair comparison if that’s the case

RRa25 Clamp 1 definitely helped, I also had to use a checkpoint that I considered over...

Neo•9/18/24, 7:03 AM

I was noticing that with SDXL as well, but didn't want to overfit due to all my training bashing my 4090 to a pulp... maybe its time to transfer to the cloud?

Kallamamran•9/18/24, 7:33 AM

WTF?!
Trained a LORA on the "The Next Day" concept car from Lynk&Co yesterweek. After 4000 steps I get this:

Kallamamran•9/18/24, 7:33 AM

Same prompt afte 4000 steps Dreambooth finetune:

Kallamamran•9/18/24, 7:34 AM

Dreambooth is not even close and it's heavily Mercedes inspired

Kallamamran•9/18/24, 7:35 AM

Well that was 14 totally wasted GPU-hours

Maxivi•9/18/24, 7:39 AM

ooof

Maxivi•9/18/24, 7:39 AM

you should really sample every 100 or 200 steps when training

Kallamamran•9/18/24, 7:39 AM

LR UNET was set according to @Dr. Furkan Gözükara config file (4e-6 = 0,000004), but with AI-toolkit I have been training LORA with 1e-4 (0,0001)

Maxivi•9/18/24, 7:40 AM

well yeah dreambooth you need a 1/10 off the learning rate

Maxivi•9/18/24, 7:40 AM

so if you're doing 1e-4 for lora you need 1e-5 for dreambooth

Kallamamran•9/18/24, 7:40 AM

Yeah well... It really doesn't seem like it

Maxivi•9/18/24, 7:41 AM

4e-6 is too low for concepts

Maxivi•9/18/24, 7:41 AM

heck i used 3e-5 and 5e-5

Kallamamran•9/18/24, 7:41 AM

It has learned out of 4000 steps what the LORA learned the first 500

Kallamamran•9/18/24, 7:41 AM

I would say that @Dr. Furkan Gözükara config is way off

Maxivi•9/18/24, 7:41 AM

keep in mind furkans configs are for people training

Maxivi•9/18/24, 7:41 AM

not cars

Kallamamran•9/18/24, 7:41 AM

Have I missed something in his video maybe

Kallamamran•9/18/24, 7:42 AM

Yeah, but seeing how little it has learned I have a hard time thinking this would be enough for a face

Maxivi•9/18/24, 7:42 AM

Kallamamran•9/18/24, 7:48 AM

I guess "I'll" spend 14 more GPU hours on 0,00001

Kallamamran•9/18/24, 8:00 AM

Maybe torch 2.5.0 could help shorten the time

KKallamamran Maybe torch 2.5.0 could help shorten the time 😛

Neo•9/18/24, 8:19 AM

Good luck sir

anikey•9/18/24, 8:27 AM

any one has past experience in uploading custom lora to hugging face ?

anikey•9/18/24, 8:27 AM

needed a bit help

di_en_ey•9/18/24, 8:31 AM

I made a comparison between ai-toolkit vs kohya lora training. For all the same parameters except for rank(kohya:128/128, ai-toolkit:64/64), kohya lora gives much better likeness and overall image quality. Does anyone have any observation on this or have a comment why kohya lora outperforms?

Kallamamran•9/18/24, 8:32 AM

Kohya-installation according to @Dr. Furkan Gözükara = ?

Kallamamran•9/18/24, 8:42 AM

Yepp... Same with Torch 2.5.0

Kallamamran•9/18/24, 8:45 AM

So many errors... Soooo many errors

Kallamamran•9/18/24, 8:46 AM

I hate torch and "generating weels"

Kallamamran•9/18/24, 8:57 AM

Still... Seem to be faster than 2.4.0

di_en_ey•9/18/24, 10:24 AM

Dr. Gozukara, have you got any experiments on greater learning rates than 5e5 with fewer iterations? What have been your observations?

Ddi_en_ey I made a comparison between ai-toolkit vs kohya lora training. For all the same ...

�

𝗡𝗶𝘁𝗶𝗻 𝗣𝗮𝗿𝗲𝘃𝗮•9/18/24, 10:26 AM

It will outperform you are using a higher rank in kohya so it will learn more

KKallamamran WTF?! Trained a LORA on the "The Next Day" concept car from Lynk&Co yesterweek. ...

�

𝗡𝗶𝘁𝗶𝗻 𝗣𝗮𝗿𝗲𝘃𝗮•9/18/24, 10:30 AM

On how many images are you training? are you using captions? what learning rate? what network dim and alpha?

�

𝗡𝗶𝘁𝗶𝗻 𝗣𝗮𝗿𝗲𝘃𝗮•9/18/24, 10:32 AM

@Dr. Furkan Gözükara How are you mentioning what blocks to train in kohya gui? I tried this but got an error -

MMaxivi are you using with weight 1.1?

Furkan Gözükara SECourses•9/18/24, 10:34 AM

yes weight 1.1 most close

Furkan Gözükara SECourses•9/18/24, 10:34 AM

i will post an article

Ssmoke Would be strange though since all the other lora showcases it looked pretty much...

Furkan Gözükara SECourses•9/18/24, 10:35 AM

i am aobut to post an article grid

KKallamamran LR UNET was set according to @Dr. Furkan Gözükara config file (4e-6 = 0,000004),...

Furkan Gözükara SECourses•9/18/24, 10:35 AM

different workflows

Ddi_en_ey I made a comparison between ai-toolkit vs kohya lora training. For all the same ...

Furkan Gözükara SECourses•9/18/24, 10:37 AM

kohya is more professionals has more experience

KKallamamran Kohya-installation according to @Dr. Furkan Gözükara = ?

Furkan Gözükara SECourses•9/18/24, 10:37 AM

looks like xformers requires new cuda

Furkan Gözükara SECourses•9/18/24, 10:37 AM

i will just remove it

Furkan Gözükara SECourses•9/18/24, 10:40 AM

removed it

Furkan Gözükara SECourses•9/18/24, 10:40 AM

https://www.patreon.com/posts/110879657

Patreon

Kohya FLUX LoRA Training Full Tutorial For Local Windows and Cloud ...

Get more from SECourses: Tutorials, Guides, Resources, Training, FLUX, MidJourney, Voice Clone, TTS, ChatGPT, GPT, LLM, Scripts by Furkan Gözü on Patreon

Would be strange though since all the other lora showcases it looked pretty much like him but in thi

Similar Threads

Similar Threads

Similar Threads