Software Engineering Courses (SECourses)•13mo ago

I have a Flux fine-tuned on a person, and I'm trying to pair with a LoRA trained on clothing. The fa

I have a Flux fine-tuned on a person, and I'm trying to pair with a LoRA trained on clothing. The face is getting really messed up unless I turn off the last lora block, and then the clothing lora isn't nearly as good, but the face works.

yolkypalkyOP•12/13/24, 8:09 PM

I've now cropped out the faces in my clothing LoRA dataset, hoping that'll help

Yyolkypalky <@205854764540362752> I saw you mentioned you were doing research on the bleed ...

Furkan Gözükara SECourses•12/13/24, 8:10 PM

they didnt work

yolkypalkyOP•12/13/24, 8:13 PM

Do you think only training specific LoRA blocks would help me, just cropping the faces out?

Yyolkypalky Do you think only training specific LoRA blocks would help me, just cropping the...

Furkan Gözükara SECourses•12/13/24, 8:16 PM

i tested it

Furkan Gözükara SECourses•12/13/24, 8:17 PM

it doesnt help

Furkan Gözükara SECourses•12/13/24, 8:17 PM

certain blocks

Furkan Gözükara SECourses•12/13/24, 8:17 PM

even single layer training still turns entire model into your face :d

Zhigarev•12/13/24, 10:29 PM

Could someone tell me if stylistic Loras are applied correctly to your fine-tuned models? Because on my own model, Loras don't work with the same settings that work on the shared flux.dev model. ... If it turns out that Loras don't work with fine-tuned models, that would be a failure...))

Yyolkypalky I have a Flux fine-tuned on a person, and I'm trying to pair with a LoRA trained...

Zhigarev•12/13/24, 10:32 PM

I understand this is a real problem with fine-tuned models?? Why was this not mentioned anywhere? Creating custom Loras and applying them to my characters was the main plan... Why did no one say a word about this? This is a mega failure...

Zhigarev•12/13/24, 10:32 PM

yolkypalkyOP•12/13/24, 10:51 PM

@Zhigarev yeah this has been a real issue for me. The only thing that helps is disabling the last double block of the LoRA but it also really hurts the style

Yyolkypalky <@1237671795432226822> yeah this has been a real issue for me. The only thing th...

Zhigarev•12/13/24, 10:53 PM

Unfortunately, this destroys the entire pipeline I had planned for future potential clients, as I won’t even be able to apply something like a branded armored t-shirt from a specific company to them... let alone a vintage photography style, like in this example...

Zhigarev•12/13/24, 10:54 PM

What are the options for this mistake@Furkan Gözükara SECourses

ZZhigarev What are the options for this mistake<@205854764540362752>

Furkan Gözükara SECourses•12/13/24, 10:55 PM

loras overwriting your character

Furkan Gözükara SECourses•12/13/24, 10:55 PM

because they are overfit

Zhigarev•12/13/24, 10:57 PM

okey whats the solution ?

FFurkan Gözükara SECourses loras overwriting your character

Zhigarev•12/13/24, 10:57 PM

You do understand that people mostly use models with Loras, right?

ZZhigarev You do understand that people mostly use models with Loras, right?

Furkan Gözükara SECourses•12/13/24, 10:58 PM

they use to generate random stuff

Furkan Gözükara SECourses•12/13/24, 10:58 PM

not certain people

Furkan Gözükara SECourses•12/13/24, 10:58 PM

that is the whole world difference

FFurkan Gözükara SECourses that is the whole world difference

Zhigarev•12/13/24, 10:59 PM

Well, to create serious works, the consistency of the character and the uniformity of the style are crucial.

Zhigarev•12/13/24, 11:00 PM

Let’s focus on how this issue can be resolved?

Zhigarev•12/13/24, 11:02 PM

Is it possible, for example, to blend a style or object Lora into your custom model? (I’m not talking about another character or a man.)

ZZhigarev Is it possible, for example, to blend a style or object Lora into your custom mo...

Furkan Gözükara SECourses•12/13/24, 11:34 PM

normally a good trained style should work

Furkan Gözükara SECourses•12/13/24, 11:34 PM

but civitai loras or models are all overfit

Furkan Gözükara SECourses•12/13/24, 11:36 PM

how can you solve is a good research

Furkan Gözükara SECourses•12/13/24, 11:36 PM

most likely training style and person together

Zhigarev•12/14/24, 10:14 AM

How to reduce the size of an extracted LoRA from 11GB to normal size around 600MB + - more less ?

ZZhigarev How to reduce the size of an extracted LoRA from 11GB to normal size around 600M...

Furkan Gözükara SECourses•12/14/24, 10:46 AM

extracted lora loses quality when you reduce the lora network rank

Furkan Gözükara SECourses•12/14/24, 10:46 AM

only way is that

Furkan Gözükara SECourses•12/14/24, 10:47 AM

also save as fp8

FFurkan Gözükara SECourses also save as fp8

Zhigarev•12/14/24, 10:56 AM

I performed the extraction in FP8, and the size remained the same at 12GB.

Zhigarev•12/14/24, 10:58 AM

That is, from the FP8 model, which became 12GB, I performed LoRA extraction, and the size did not change at all.

Zhigarev•12/14/24, 10:58 AM

same 12 gb FP8 to 12 gb Lora...

ZZhigarev same 12 gb FP8 to 12 gb Lora...

Furkan Gözükara SECourses•12/14/24, 11:10 AM

well that is just impossib le you made mistake somewhere

Furkan Gözükara SECourses•12/14/24, 11:10 AM

best lora with fp8 is like 2-3 gb

Zhigarev•12/14/24, 11:16 AM

whats wrong here? in the step 2 i choose my finetuned FP8 model

ZZhigarev whats wrong here? in the step 2 i choose my finetuned FP8 model

Furkan Gözükara SECourses•12/14/24, 11:19 AM

save precision

Furkan Gözükara SECourses•12/14/24, 11:19 AM

make it fp8

Zhigarev•12/14/24, 11:21 AM

Alright will try, it's just that this isn't mentioned anywhere, or did I miss it??

ZZhigarev Alright will try, it's just that this isn't mentioned anywhere, or did I miss it...

Furkan Gözükara SECourses•12/14/24, 11:21 AM

it is for saving space not mandatory

Furkan Gözükara SECourses•12/14/24, 11:22 AM

float is best quality

FFurkan Gözükara SECourses it is for saving space not mandatory

Zhigarev•12/14/24, 11:26 AM

There is no option to select FP8... only FP16, BF16, Float, and None.

FFurkan Gözükara SECourses float is best quality

Zhigarev•12/14/24, 11:32 AM

The 11GB size of LoRA… that’s why it’s better in quality. There’s no magic here. If LoRA weighed 1GB while maintaining quality, that would be a real breakthrough. But models with a size of 23GB, where LoRA doesn’t work properly, and extracted LoRAs at 11GB, are a significant negative effect that needs to be known in advance.

ZZhigarev There is no option to select FP8... only FP16, BF16, Float, and None.

Furkan Gözükara SECourses•12/14/24, 11:32 AM

ok then you need 2 steps

FFurkan Gözükara SECourses ok then you need 2 steps

Zhigarev•12/14/24, 11:34 AM

Which steps?

ZZhigarev Which steps?

Furkan Gözükara SECourses•12/14/24, 11:34 AM

convert to fp8

Furkan Gözükara SECourses•12/14/24, 11:34 AM

https://www.patreon.com/posts/how-to-convert-3-115376830

Patreon