Software Engineering Courses (SECourses)•17mo ago

The vae path, T5-XXL and Clip-L Paths options are not showing in my latest installed Kohya GUI

Dreamer@TDFilmStudioOP•8/29/24, 7:08 PM

The options are visible in your tutorial of v19

FLUX_LoRA_Training_Simplified__From_Zero_to_Hero_with_Kohya_SS_GUI_8GB_GPU_Windows_Tutorial_Guide.mp4_snapshot_00.32.34.504.png

DDreamer@TDFilmStudio The vae path, T5-XXL and Clip-L Paths options are not showing in my latest insta...

Furkan Gözükara SECourses•8/29/24, 7:08 PM

select model first

Furkan Gözükara SECourses•8/29/24, 7:08 PM

it will become available

Furkan Gözükara SECourses•8/29/24, 7:08 PM

select dev fp16 flux model

FFurkan Gözükara SECourses but the issue is flux itself has inner captioning

dsienra•8/29/24, 7:11 PM

Really???? ohh that's cool!! but a huge problem at the same time, must to be be a way to override this, its ok if the model recognize you as a man but not any man only "ohwx man"

Ddsienra Really???? ohh that's cool!! but a huge problem at the same time, must to be be...

Furkan Gözükara SECourses•8/29/24, 7:12 PM

i dont know any overwrite way atm

Furkan Gözükara SECourses•8/29/24, 7:12 PM

it is how it is designed

Furkan Gözükara SECourses•8/29/24, 7:12 PM

but fine tuning may overcome this issue

dsienra•8/29/24, 7:15 PM

I don't understand how it works , to auto caption in have to analyze the images first caption them and then train, but I don't think is doing that, may be is the way that organize its weights, and it auto detect that the patterns relates to the weights that overlaps with the concept of man, but really idk

Ddsienra I don't understand how it works , to auto caption in have to analyze the images ...

Furkan Gözükara SECourses•8/29/24, 7:17 PM

it uses attention of both image itself (this is where auto captioning happens in neural network) and the prompts we provide

dsienra•8/29/24, 7:19 PM

I edited my message to make it more clear, hehe

dsienra•8/29/24, 7:20 PM

ok I see your point

dsienra•8/29/24, 7:21 PM

this is documented somewhere or everybody is making assumptions?

Ddsienra this is documented somewhere or everybody is making assumptions?

Furkan Gözükara SECourses•8/29/24, 7:23 PM

well i asked this to top developers and 2 said similar stuff

Furkan Gözükara SECourses•8/29/24, 7:23 PM

i think people knows it from sd3 architecture

Furkan Gözükara SECourses•8/29/24, 7:24 PM

my experiment also proves this

Furkan Gözükara SECourses•8/29/24, 7:24 PM

i didnt use man class

Furkan Gözükara SECourses•8/29/24, 7:25 PM

it perfectly learnt

Furkan Gözükara SECourses•8/29/24, 7:25 PM

and man prompt is still fully impacted

Furkan Gözükara SECourses•8/29/24, 7:25 PM

but car prompt is not

dsienra•8/29/24, 7:25 PM

I'm training same parameters no class and regularization images, in the first samples 5 epochs nobody looks like me, this is good I think, the resemblance must apear in the next epochs

FFurkan Gözükara SECourses and man prompt is still fully impacted

dsienra•8/29/24, 7:33 PM

yes but that happens with sd and sdxl too at a some extent, it generalizes if you train with your name only it bleeds to people even to other genders if you overtrain enough

Ddsienra yes but that happens with sd and sdxl too at a some extent, it generalizes if yo...

Furkan Gözükara SECourses•8/29/24, 7:35 PM

sd and sdxl doesnt have exact same mecahnism though

Furkan Gözükara SECourses•8/29/24, 7:35 PM

so if you dont caption it will make huge impact

Furkan Gözükara SECourses•8/29/24, 7:35 PM

for example even though i fully captioned, it reduced the likelineess very small. impact on sdxl way bigger

dsienra•8/29/24, 7:36 PM

that's true

dsienra•8/29/24, 7:38 PM

Lets see, it needs more research, It will be cool to know exactly how the base model was trained

Ddsienra Lets see, it needs more research, It will be cool to know exactly how the base m...