Software Engineering Courses (SECourses)•2y ago

The workflow works on this clip but other clips just dont load at all and give that error

dxqb•5/20/24, 10:48 PM

so whenever you have the chance, it would be great to get your opinion.

Ddxqb no, I've talked to the OT developer and he is not interested. he thinks it is to...

Furkan Gözükara SECourses•5/20/24, 10:50 PM

sure i would like to test

Furkan Gözükara SECourses•5/20/24, 10:50 PM

currently i was using kohya

Furkan Gözükara SECourses•5/20/24, 10:50 PM

do you have same for kohya?

Furkan Gözükara SECourses•5/20/24, 10:50 PM

that would be a nice comparison but i can test on OT too

dxqb•5/20/24, 10:51 PM

I had it for kohya, as I was using kohya until we noticed the quality isn't good anymore. so I think testing this for kohya only makes sense after you have found the perfect hyperparameters again. otherwise we'd compare against something that is bad for other reasons

Ddxqb I *had* it for kohya, as I was using kohya until we noticed the quality isn't go...

Furkan Gözükara SECourses•5/20/24, 10:54 PM

ye i found them

Furkan Gözükara SECourses•5/20/24, 10:54 PM

going to update now

dxqb•5/20/24, 11:04 PM

I DMed you a patch. The class it controls against is hardcoded to 'man', so if you are training a woman or something else, you need to change that

dxqb•5/20/24, 11:05 PM

parameters: leave everything as-is, but turn on TE2, too, and set both TE1 and TE2 learning rates to the main learning rate 1e-05

dxqb•5/20/24, 11:05 PM

if you want it for kohya, I'd have to first test it again tomorrow, as it was a while ago and I don't know if I have left it in a working state

dxqb•5/20/24, 11:07 PM

off to bed now, let me know if you have any questions!

Furkan Gözükara SECourses•5/21/24, 12:39 AM

Kohya config updated https://www.patreon.com/posts/very-best-for-of-89213064

AIGambinoOP•5/21/24, 1:13 AM

Whats the difference between kohya and onetrainer?

AAIGambino Whats the difference between kohya and onetrainer?

Furkan Gözükara SECourses•5/21/24, 1:14 AM

different scripts

Furkan Gözükara SECourses•5/21/24, 1:14 AM

different features

Furkan Gözükara SECourses•5/21/24, 1:14 AM

and produces even different results in same settings :d

AIGambinoOP•5/21/24, 1:14 AM

Interesting

AIGambinoOP•5/21/24, 1:14 AM

How long should using 20 images take to train using the 24gb json file roughly?

AAIGambino How long should using 20 images take to train using the 24gb json file roughly?

Furkan Gözükara SECourses•5/21/24, 1:17 AM

depends on gpu

AIGambinoOP•5/21/24, 1:17 AM

3090 is what I have

Furkan Gözükara SECourses•5/21/24, 1:17 AM

but lets say you did 150 repeat

Furkan Gözükara SECourses•5/21/24, 1:17 AM

ye around 1.7 second / it lets say

Furkan Gözükara SECourses•5/21/24, 1:17 AM

Furkan Gözükara SECourses•5/21/24, 1:18 AM

6000 * 1.7 second

AIGambinoOP•5/21/24, 1:19 AM

in your latest post there what did you do change the 24gb json file configs?

AAIGambino in your latest post there what did you do change the 24gb json file configs?

Furkan Gözükara SECourses•5/21/24, 1:20 AM

learning rates

Furkan Gözükara SECourses•5/21/24, 1:20 AM

read the patreon post very detailed

Furkan Gözükara SECourses•5/21/24, 2:15 AM

i should test and see if SD 1.5 broken too or not :/

FFurkan Gözükara SECourses i should test and see if SD 1.5 broken too or not :/

dsienra•5/21/24, 2:23 AM

I'm training on an rtx 3090 with Gradient checkpointing off and CrossAttention sdpa at 1.52s/it

Ddsienra I'm training on an rtx 3090 with Gradient checkpointing off and CrossAttention s...

Furkan Gözükara SECourses•5/21/24, 2:31 AM

nice speed

Furkan Gözükara SECourses•5/21/24, 2:31 AM

spda speed up

Furkan Gözükara SECourses•5/21/24, 2:31 AM

but i dont know how much quality loss

dsienra•5/21/24, 2:33 AM

1.51s/it now, still slower than previous versions with the same settings but it is faster than using Gradient checkpointing on this capture is with an older version

FFurkan Gözükara SECourses nice speed

dsienra•5/21/24, 2:35 AM

I didn't know that this option can decrease quality, what about xformers, with it also can get quality loss?

dsienra•5/21/24, 2:37 AM

I get almost the same speeds with xformers and a little more vram usage 20gb or so

Ddsienra I didn't know that this option can decrease quality, what about xformers, with i...

Furkan Gözükara SECourses•5/21/24, 3:08 AM

yes in past both degraded

Furkan Gözükara SECourses•5/21/24, 3:08 AM

but it is getting better

dsienra•5/21/24, 3:17 AM

ok thanks!! I will check if I can see any difference

Furkan Gözükara SECourses•5/21/24, 3:18 AM

i will do a training with onetrainer

Furkan Gözükara SECourses•5/21/24, 3:18 AM

lets see difference between kohya

dsienra•5/21/24, 3:23 AM

that will be an interesting test!! a good test can be also to do a run test with xformers and spda, its a little speed up but can be very significant speed up for very large datasets and very long trainings.

Furkan Gözükara SECourses•5/21/24, 3:25 AM

doing a fresh install

Furkan Gözükara SECourses•5/21/24, 3:26 AM

of onetrainer

Furkan Gözükara SECourses•5/21/24, 3:26 AM

ye for massive dataset

Furkan Gözükara SECourses•5/21/24, 3:26 AM

you have to have xformers and batch size bigger

NNicB Yes. I'll work on getting that fixed tomorrow with the team

RMK•5/21/24, 3:27 AM

Hi just checking to see if this works now ? And if so, how do I do it?

dsienra•5/21/24, 3:31 AM

LR Scheduler sdpa never worked on kohya_ss an sdxl models, seems to be an interesting concept I read that it automatically adjusts the learning rate or something like that.

dsienra•5/21/24, 3:32 AM

I don't know if it was designed to work in conjunction with CrossAttention sdpa

Ddsienra LR Scheduler sdpa never worked on kohya_ss an sdxl models, seems to be an intere...

Furkan Gözükara SECourses•5/21/24, 3:36 AM

i use static LR