Software Engineering Courses (SECourses)•15mo ago

After my failure I'm not so sure about "Apply T5 Attention Mask, and Train T5-XXL enabled" you are t

After my failure I'm not so sure about "Apply T5 Attention Mask, and Train T5-XXL enabled" you are training the T5, but in inference you load a T5 model that doesn't know your token. forge o SwarmUI load the regular T5 model that was not trained. so I'm not so sure

Ddsienra After my failure I'm not so sure about "Apply T5 Attention Mask, and Train T5-XX...

magical•10/16/24, 4:18 PM

oh the t5 model should be trained as well for it to work properly?

dsienraOP•10/16/24, 4:18 PM

@Dr. Furkan Gözükara may be Apply T5 Attention Mask can help, but I don't understand how can Train T5-XXL helps, because on inference you load a T5 model that was not trained . forge or SwarmUI load the regular T5 model that was not trained

Mmagical oh the t5 model should be trained as well for it to work properly?

dsienraOP•10/16/24, 4:23 PM

Is the thing I don't understand, so really IDK now I'm training again with both options disabled, My question can apply to everyone, but in my case is different because I'm training flux-dev-de-distill so in my case is broking something. I'm having artifacts on the images with this on and also is not learning the concepts correctly

dsienraOP•10/16/24, 7:04 PM

@Dr. Furkan Gözükara Update on https://huggingface.co/nyanko7/flux-dev-de-distill I started new training - "T5 Attention Mask and T5-XXL both disabled" same lr 30 epochs now. I tested the lora checkpoint and all problems are fixed, is going great using regular flux-dev-fp8 for inference, I'm training tree people same class, same prompt changing just the name renders the correct subject, is undertrained but goes very well so far

nyanko7/flux-dev-de-distill · Hugging Face

ar.1•10/16/24, 9:42 PM

ar.1•10/16/24, 9:42 PM

did i do something wrong? why is there 17.6 million steps ??

Mmagical oh and I didn't try training T5 XXL. If there is no text in my training images w...

Furkan Gözükara SECourses•10/16/24, 9:42 PM

T5 XXL made minor diff that i didnt notice :d

Aar.1 Click to see attachment

Furkan Gözükara SECourses•10/16/24, 9:43 PM

yep you did indeed

Furkan Gözükara SECourses•10/16/24, 9:43 PM

sub folder

Furkan Gözükara SECourses•10/16/24, 9:43 PM

more than 1 repeat

FFurkan Gözükara SECourses more than 1 repeat

ar.1•10/16/24, 9:45 PM

ah it was at 40

Aar.1 ah it was at 40

Furkan Gözükara SECourses•10/16/24, 9:45 PM

yep

FFurkan Gözükara SECourses yep

ar.1•10/16/24, 9:46 PM

ar.1•10/16/24, 9:46 PM

is that too many still

ar.1•10/16/24, 9:46 PM

2206 images

Aar.1 Click to see attachment

Furkan Gözükara SECourses•10/16/24, 9:46 PM

sub folders

Vigilence•10/16/24, 9:47 PM

The config could use an update as he made the same mistake as me.

VVigilence The config could use an update as he made the same mistake as me.

Furkan Gözükara SECourses•10/16/24, 9:48 PM

i tested config today works as expected

Vigilence•10/16/24, 9:48 PM

The repeat was at 40 when I used it, is that not saved in the config?

VVigilence The repeat was at 40 when I used it, is that not saved in the config?

Furkan Gözükara SECourses•10/16/24, 9:49 PM

nope

Furkan Gözükara SECourses•10/16/24, 9:49 PM

it is read from folder

Furkan Gözükara SECourses•10/16/24, 9:49 PM

you need to fix folder name

Furkan Gözükara SECourses•10/16/24, 9:49 PM

i explained this in details in tutorial

Furkan Gözükara SECourses•10/16/24, 9:49 PM

but you are skipping :d

Vigilence•10/16/24, 9:49 PM

I see. Lol.

Furkan Gözükara SECourses•10/16/24, 9:50 PM

Dazzastrous [4090]•10/16/24, 9:50 PM

need to be more vigilent?

Dazzastrous [4090]•10/16/24, 9:51 PM

had to

Dazzastrous [4090]•10/16/24, 9:51 PM

lol

Vigilence•10/16/24, 9:51 PM

So here is what I suggest, as I watched some parts of all your tutorials, but not all (and sometimes the bookmarks are wrong). Add an image in the config file showing the repeat section and saying it needs to be 1.

Vigilence•10/16/24, 9:51 PM

Should be labeled (Step 1: Read First!) or something similar.

Dazzastrous [4090]•10/16/24, 9:52 PM

yeh not the easiest to navigate through

Dazzastrous [4090]•10/16/24, 9:52 PM

so many videos

Dazzastrous [4090]•10/16/24, 9:52 PM

bit overwhelming for beginners

Furkan Gözükara SECourses•10/16/24, 9:52 PM

that 40 repeat is not saved in config

Furkan Gözükara SECourses•10/16/24, 9:52 PM

therefore i cant set it