Software Engineering Courses (SECourses)•17mo ago

I've tried Florence2 for a training and it went good, was setting up another Lora today but this tim

I've tried Florence2 for a training and it went good, was setting up another Lora today but this time I actually read the captions first and noticed Florence got some key details very wrong, which could possibly mess up the Lora a bit so I dunno. I wonder if Llava is better or there's a free local Captioning tool that is a lot more consistent

JJ2_Nedo good info for Forge here: https://github.com/lllyasviel/stable-diffusion-webui-f...

ivaxsirc -4090•8/19/24, 6:36 PM

not works for me i see the creation image but after grey image, what's wrong?

Iivaxsirc -4090 not works for me i see the creation image but after grey image, what's wrong?

J2_Nedo•8/19/24, 6:38 PM

must be something with the sampler type, i got it too, but it works correct with the right one, try normal and others

�𝔸𝖨ΞӾ I've tried Florence2 for a training and it went good, was setting up another Lor...

J2_Nedo•8/19/24, 6:39 PM

i always read the captions and edit them manually if i see something is wrong

dsienra•8/19/24, 6:39 PM

I'm doing some testing training flux on kohya but I'm having a bleeding problem, I'm training pictures of myself as a kid and as and adult and a friend, so the captions are "myname man" "myname boy" "myfriendname man" and the concepts bleed, this problem doesn't happen training sdxl models, there is no bleeding. maybe the problem is because I'm not training the text encoder?

dsienra•8/19/24, 6:42 PM

I hade a similar problem with my sd3 testing

Ddsienra I'm doing some testing training flux on kohya but I'm having a bleeding problem,...

Furkan Gözükara SECourses•8/19/24, 6:48 PM

ye not training text encoder will cause

Furkan Gözükara SECourses•8/19/24, 6:48 PM

more bleeding

Furkan Gözükara SECourses•8/19/24, 6:48 PM

since you only train unet

Furkan Gözükara SECourses•8/19/24, 6:48 PM

clip l training not supported yet

dsienra•8/19/24, 6:49 PM

ohh thanks, so may be is that

JJ2_Nedo must be something with the sampler type, i got it too, but it works correct with...

ivaxsirc -4090•8/19/24, 6:50 PM

I've found it was the vae, are for shell not for dev

FFurkan Gözükara SECourses ye not training text encoder will cause

dsienra•8/19/24, 7:03 PM

Another question: the CLIP models, such as clip_l, are stored in separate files from the base model. If I use a concept that doesn’t exist in CLIP, such as ohwx or another non-existent word, would it still work during inference if it's not part of the model because the model was not trained? Additionally, if I train clip_l, would it be saved as a separate file? Currently, the text encoders are not in the same safetensors as the base model

Ddsienra Another question: the CLIP models, such as clip_l, are stored in separate files ...

Furkan Gözükara SECourses•8/19/24, 7:22 PM

it will work but lesser degree

Furkan Gözükara SECourses•8/19/24, 7:22 PM

since you will train unet of the model

Furkan Gözükara SECourses•8/19/24, 7:22 PM

or if you have another question explain in more details

Furkan Gözükara SECourses•8/19/24, 7:22 PM

started second 4 gpu machine

JJ2_Nedo must be something with the sampler type, i got it too, but it works correct with...

YellowjacketXVI•8/19/24, 7:23 PM

Yeah, it a lot faster to edit prompts than write them

FFurkan Gözükara SECourses or if you have another question explain in more details

dsienra•8/19/24, 7:28 PM

The doubt arises because the text encoder is in a different file, and if I train it, would Koha save it as a separate file? In that case, should I also use my custom clip_l for inference? Ideally, the text encoders should be included in the same SafeTensor file, as happens in SDXL

Ddsienra The doubt arises because the text encoder is in a different file, and if I train...

Furkan Gözükara SECourses•8/19/24, 7:32 PM

if you trained you have to use it during inference for best quality

dsienra•8/19/24, 7:35 PM

thanks, I hope in the future the text encoders can be embebed in the model file

Ddsienra thanks, I hope in the future the text encoders can be embebed in the model file

Furkan Gözükara SECourses•8/19/24, 7:39 PM

kohya will support very likely when supporting clip l training

dsienra•8/19/24, 8:15 PM

thanks!! I hope so!! and I hope that 24gb vram will be enough to train flux + clip_l

Ddsienra thanks!! I hope so!! and I hope that 24gb vram will be enough to train flux + c...

Furkan Gözükara SECourses•8/19/24, 8:18 PM

i hope so

Furkan Gözükara SECourses•8/19/24, 8:18 PM

currently only unet trained

Furkan Gözükara SECourses•8/19/24, 8:18 PM

with kohya lora

Furkan Gözükara SECourses•8/19/24, 8:18 PM

24 gb barely sufficient

Furkan Gözükara SECourses•8/19/24, 8:18 PM

updated post with experiments

dsienra•8/19/24, 8:19 PM

I will check it

Furkan Gözükara SECourses•8/19/24, 8:21 PM

FFurkan Gözükara SECourses Click to see attachment

ivaxsirc -4090•8/19/24, 8:55 PM

with 24GB only Loras?

Iivaxsirc -4090 with 24GB only Loras?

Furkan Gözükara SECourses•8/19/24, 9:09 PM

Ye this is only lora but I think fine tuning has better optimizations we will see. Also I asked kohya since there is half model training but still uses 18 gb gpu

Furkan Gözükara SECourses•8/19/24, 10:09 PM

more than 50 epoch completed time to start doing some testing

FFurkan Gözükara SECourses Click to see attachment

dxqb•8/19/24, 10:14 PM

so nothing for 16 GB?

Ddxqb so nothing for 16 GB?

Furkan Gözükara SECourses•8/19/24, 10:15 PM

there is one config supposed to work but not working i messaged him

Furkan Gözükara SECourses•8/19/24, 10:15 PM

i think fine tuning may work for 16

Furkan Gözükara SECourses•8/19/24, 10:15 PM

for lora i think there is bug

Furkan Gözükara SECourses•8/19/24, 10:15 PM

32 rank full precision lora is 613 mb

dxqb•8/19/24, 10:15 PM

wdym fine tuning on 16 GB but not lora?!

Ddxqb wdym fine tuning on 16 GB but not lora?!

Furkan Gözükara SECourses•8/19/24, 10:15 PM

Furkan Gözükara SECourses•8/19/24, 10:16 PM

because of optimiziations

dxqb•8/19/24, 10:16 PM

how can fine tuning work on 16 GB when the model is already 12B * 2 bytes?

Furkan Gözükara SECourses•8/19/24, 10:16 PM

there is half model training for lora but it is peaking like this

Ddxqb how can fine tuning work on 16 GB when the model is already 12B * 2 bytes?

Furkan Gözükara SECourses•8/19/24, 10:16 PM

a lot of optimization with speed loss

Furkan Gözükara SECourses•8/19/24, 10:16 PM

this above is lora training half model

Furkan Gözükara SECourses•8/19/24, 10:16 PM

it peaks 18 gb

johnasdf.•8/19/24, 10:17 PM

Do you have a config I can download for Flux training on Kohya, just to get setup quickly and I can make the changes for my computer specific information?

FFurkan Gözükara SECourses a lot of optimization with speed loss

dxqb•8/19/24, 10:18 PM

speed loss?

Ddxqb speed loss?

Furkan Gözükara SECourses•8/19/24, 10:18 PM

yes slower training

dxqb•8/19/24, 10:18 PM

ah you mean loss of performance. was thinking of loss functions

I've tried Florence2 for a training and it went good, was setting up another Lora today but this tim

Similar Threads