Software Engineering Courses (SECourses)•17mo ago

good info for Forge here: https://github.com/lllyasviel/stable-diffusion-webui-forge/discussions/105

good info for Forge here: https://github.com/lllyasviel/stable-diffusion-webui-forge/discussions/1050

GitHub

[GGUF and Flux full fp16 Model] loading T5, CLIP + new VAE UI · lll...

The old Automatic1111’s user interface of VAE selection is not powerful enough for modern models . Forge make minor modifications so that the UI is as close as possible to A1111 but also meet the d...

J2_NedoOP•8/19/24, 5:52 PM

this is how it looks for me:

YellowjacketXVI•8/19/24, 6:22 PM

been training on flux and wow so great, I've been typing my captions out by hand, what tool goes well with the Clip-I?

YYellowjacketXVI been training on flux and wow so great, I've been typing my captions out by hand...

J2_NedoOP•8/19/24, 6:29 PM

i use Florence 2 for this, my Flux Loras looking really good, but i am still in research mode

JJ2_Nedo i use Florence 2 for this, my Flux Loras looking really good, but i am still in ...

YellowjacketXVI•8/19/24, 6:29 PM

The man! .....not assuming anything tho..... XD

JJ2_Nedo i use Florence 2 for this, my Flux Loras looking really good, but i am still in ...

YellowjacketXVI•8/19/24, 6:31 PM

It can be used for a clip seg? How indepth of a description can you give, I have a hard time seperating one man from another with SAM

JJ2_Nedo i use Florence 2 for this, my Flux Loras looking really good, but i am still in ...

�

𝔸𝖨ΞӾ•8/19/24, 6:35 PM

I've tried Florence2 for a training and it went good, was setting up another Lora today but this time I actually read the captions first and noticed Florence got some key details very wrong, which could possibly mess up the Lora a bit so I dunno. I wonder if Llava is better or there's a free local Captioning tool that is a lot more consistent

JJ2_Nedo good info for Forge here: https://github.com/lllyasviel/stable-diffusion-webui-f...

ivaxsirc -4090•8/19/24, 6:36 PM

not works for me i see the creation image but after grey image, what's wrong?

Iivaxsirc -4090 not works for me i see the creation image but after grey image, what's wrong?

J2_NedoOP•8/19/24, 6:38 PM

must be something with the sampler type, i got it too, but it works correct with the right one, try normal and others

�𝔸𝖨ΞӾ I've tried Florence2 for a training and it went good, was setting up another Lor...

J2_NedoOP•8/19/24, 6:39 PM

i always read the captions and edit them manually if i see something is wrong

dsienra•8/19/24, 6:39 PM

I'm doing some testing training flux on kohya but I'm having a bleeding problem, I'm training pictures of myself as a kid and as and adult and a friend, so the captions are "myname man" "myname boy" "myfriendname man" and the concepts bleed, this problem doesn't happen training sdxl models, there is no bleeding. maybe the problem is because I'm not training the text encoder?

dsienra•8/19/24, 6:42 PM

I hade a similar problem with my sd3 testing

Ddsienra I'm doing some testing training flux on kohya but I'm having a bleeding problem,...

Furkan Gözükara SECourses•8/19/24, 6:48 PM

ye not training text encoder will cause

Furkan Gözükara SECourses•8/19/24, 6:48 PM

more bleeding

Furkan Gözükara SECourses•8/19/24, 6:48 PM

since you only train unet

Furkan Gözükara SECourses•8/19/24, 6:48 PM

clip l training not supported yet

dsienra•8/19/24, 6:49 PM

ohh thanks, so may be is that

JJ2_Nedo must be something with the sampler type, i got it too, but it works correct with...

ivaxsirc -4090•8/19/24, 6:50 PM

I've found it was the vae, are for shell not for dev

FFurkan Gözükara SECourses ye not training text encoder will cause

dsienra•8/19/24, 7:03 PM

Another question: the CLIP models, such as clip_l, are stored in separate files from the base model. If I use a concept that doesn’t exist in CLIP, such as ohwx or another non-existent word, would it still work during inference if it's not part of the model because the model was not trained? Additionally, if I train clip_l, would it be saved as a separate file? Currently, the text encoders are not in the same safetensors as the base model

Ddsienra Another question: the CLIP models, such as clip_l, are stored in separate files ...

Furkan Gözükara SECourses•8/19/24, 7:22 PM

it will work but lesser degree

Furkan Gözükara SECourses•8/19/24, 7:22 PM

since you will train unet of the model

Furkan Gözükara SECourses•8/19/24, 7:22 PM

or if you have another question explain in more details

Furkan Gözükara SECourses•8/19/24, 7:22 PM

started second 4 gpu machine

JJ2_Nedo must be something with the sampler type, i got it too, but it works correct with...

YellowjacketXVI•8/19/24, 7:23 PM

Yeah, it a lot faster to edit prompts than write them

FFurkan Gözükara SECourses or if you have another question explain in more details

dsienra•8/19/24, 7:28 PM

The doubt arises because the text encoder is in a different file, and if I train it, would Koha save it as a separate file? In that case, should I also use my custom clip_l for inference? Ideally, the text encoders should be included in the same SafeTensor file, as happens in SDXL

Ddsienra The doubt arises because the text encoder is in a different file, and if I train...

Furkan Gözükara SECourses•8/19/24, 7:32 PM

if you trained you have to use it during inference for best quality

dsienra•8/19/24, 7:35 PM

thanks, I hope in the future the text encoders can be embebed in the model file

Ddsienra thanks, I hope in the future the text encoders can be embebed in the model file

Furkan Gözükara SECourses•8/19/24, 7:39 PM

kohya will support very likely when supporting clip l training

dsienra•8/19/24, 8:15 PM

thanks!! I hope so!! and I hope that 24gb vram will be enough to train flux + clip_l

Ddsienra thanks!! I hope so!! and I hope that 24gb vram will be enough to train flux + c...

Furkan Gözükara SECourses•8/19/24, 8:18 PM

i hope so

Furkan Gözükara SECourses•8/19/24, 8:18 PM

currently only unet trained

Furkan Gözükara SECourses•8/19/24, 8:18 PM

with kohya lora

Furkan Gözükara SECourses•8/19/24, 8:18 PM

24 gb barely sufficient

Furkan Gözükara SECourses•8/19/24, 8:18 PM

updated post with experiments

dsienra•8/19/24, 8:19 PM

I will check it

Furkan Gözükara SECourses•8/19/24, 8:21 PM

FFurkan Gözükara SECourses Click to see attachment

ivaxsirc -4090•8/19/24, 8:55 PM

with 24GB only Loras?

Iivaxsirc -4090 with 24GB only Loras?

Furkan Gözükara SECourses•8/19/24, 9:09 PM

Ye this is only lora but I think fine tuning has better optimizations we will see. Also I asked kohya since there is half model training but still uses 18 gb gpu

Furkan Gözükara SECourses•8/19/24, 10:09 PM

more than 50 epoch completed time to start doing some testing

FFurkan Gözükara SECourses Click to see attachment

dxqb•8/19/24, 10:14 PM

so nothing for 16 GB?

Ddxqb so nothing for 16 GB?

Furkan Gözükara SECourses•8/19/24, 10:15 PM

there is one config supposed to work but not working i messaged him

Furkan Gözükara SECourses•8/19/24, 10:15 PM

i think fine tuning may work for 16

Furkan Gözükara SECourses•8/19/24, 10:15 PM

for lora i think there is bug