Software Engineering Courses (SECourses)•17mo ago

@Dr. Furkan Gözükara After seeing your tutorial I am confused. We dont need to caption our training

@Dr. Furkan Gözükara After seeing your tutorial I am confused. We dont need to caption our training images for flux lora, right? (To achieve better quality)

DDreamer@TDFilmStudio @Dr. Furkan Gözükara After seeing your tutorial I am confused. We dont need to c...

Sharinel•8/29/24, 6:23 PM

I've done a few character loras now and just had to put in the trigger name then ,woman. And I don't think you even needed 'woman'

SSharinel I've done a few character loras now and just had to put in the trigger name then...

Dreamer@TDFilmStudioOP•8/29/24, 6:24 PM

So no point in using something like joycaption to caption all images meant for training?

Sharinel•8/29/24, 6:25 PM

I did that with the first couple, then just did the trigger word with the next few. No appeciable difference imo

SSharinel I did that with the first couple, then just did the trigger word with the next f...

Dreamer@TDFilmStudioOP•8/29/24, 6:25 PM

Ok thanks for confirming

DDreamer@TDFilmStudio @Dr. Furkan Gözükara After seeing your tutorial I am confused. We dont need to c...

Furkan Gözükara SECourses•8/29/24, 7:04 PM

for character yes just ohwx man or woman

Furkan Gözükara SECourses•8/29/24, 7:04 PM

for style i will test

DDreamer@TDFilmStudio So no point in using something like joycaption to caption all images meant for t...

Furkan Gözükara SECourses•8/29/24, 7:04 PM

yes not at this moment

Furkan Gözükara SECourses•8/29/24, 7:04 PM

we will see in fine tuning

FFurkan Gözükara SECourses for character yes just ohwx man or woman

Dreamer@TDFilmStudioOP•8/29/24, 7:07 PM

The vae path, T5-XXL and Clip-L Paths options are not showing in my latest installed Kohya GUI

Dreamer@TDFilmStudioOP•8/29/24, 7:08 PM

The options are visible in your tutorial of v19

FLUX_LoRA_Training_Simplified__From_Zero_to_Hero_with_Kohya_SS_GUI_8GB_GPU_Windows_Tutorial_Guide.mp4_snapshot_00.32.34.504.png

DDreamer@TDFilmStudio The vae path, T5-XXL and Clip-L Paths options are not showing in my latest insta...

Furkan Gözükara SECourses•8/29/24, 7:08 PM

select model first

Furkan Gözükara SECourses•8/29/24, 7:08 PM

it will become available

Furkan Gözükara SECourses•8/29/24, 7:08 PM

select dev fp16 flux model

FFurkan Gözükara SECourses but the issue is flux itself has inner captioning

dsienra•8/29/24, 7:11 PM

Really???? ohh that's cool!! but a huge problem at the same time, must to be be a way to override this, its ok if the model recognize you as a man but not any man only "ohwx man"

Ddsienra Really???? ohh that's cool!! but a huge problem at the same time, must to be be...

Furkan Gözükara SECourses•8/29/24, 7:12 PM

i dont know any overwrite way atm

Furkan Gözükara SECourses•8/29/24, 7:12 PM

it is how it is designed

Furkan Gözükara SECourses•8/29/24, 7:12 PM

but fine tuning may overcome this issue

dsienra•8/29/24, 7:15 PM

I don't understand how it works , to auto caption in have to analyze the images first caption them and then train, but I don't think is doing that, may be is the way that organize its weights, and it auto detect that the patterns relates to the weights that overlaps with the concept of man, but really idk

Ddsienra I don't understand how it works , to auto caption in have to analyze the images ...

Furkan Gözükara SECourses•8/29/24, 7:17 PM

it uses attention of both image itself (this is where auto captioning happens in neural network) and the prompts we provide

dsienra•8/29/24, 7:19 PM

I edited my message to make it more clear, hehe

dsienra•8/29/24, 7:20 PM

ok I see your point

dsienra•8/29/24, 7:21 PM

this is documented somewhere or everybody is making assumptions?

Ddsienra this is documented somewhere or everybody is making assumptions?

Furkan Gözükara SECourses•8/29/24, 7:23 PM

well i asked this to top developers and 2 said similar stuff

Furkan Gözükara SECourses•8/29/24, 7:23 PM

i think people knows it from sd3 architecture

Furkan Gözükara SECourses•8/29/24, 7:24 PM

my experiment also proves this

Furkan Gözükara SECourses•8/29/24, 7:24 PM

i didnt use man class

Furkan Gözükara SECourses•8/29/24, 7:25 PM

it perfectly learnt

Furkan Gözükara SECourses•8/29/24, 7:25 PM

and man prompt is still fully impacted

Furkan Gözükara SECourses•8/29/24, 7:25 PM

but car prompt is not

dsienra•8/29/24, 7:25 PM

I'm training same parameters no class and regularization images, in the first samples 5 epochs nobody looks like me, this is good I think, the resemblance must apear in the next epochs

FFurkan Gözükara SECourses and man prompt is still fully impacted

dsienra•8/29/24, 7:33 PM

yes but that happens with sd and sdxl too at a some extent, it generalizes if you train with your name only it bleeds to people even to other genders if you overtrain enough

Ddsienra yes but that happens with sd and sdxl too at a some extent, it generalizes if yo...

Furkan Gözükara SECourses•8/29/24, 7:35 PM

sd and sdxl doesnt have exact same mecahnism though

Furkan Gözükara SECourses•8/29/24, 7:35 PM

so if you dont caption it will make huge impact

Furkan Gözükara SECourses•8/29/24, 7:35 PM

for example even though i fully captioned, it reduced the likelineess very small. impact on sdxl way bigger

dsienra•8/29/24, 7:36 PM

that's true

dsienra•8/29/24, 7:38 PM

Lets see, it needs more research, It will be cool to know exactly how the base model was trained

Ddsienra Lets see, it needs more research, It will be cool to know exactly how the base m...