Software Engineering Courses (SECourses)•16mo ago

Can you describe more out which step to do it. Where I can load the latest checkpoint and how to cal

Can you describe more out which step to do it. Where I can load the latest checkpoint and how to calculate the remain step for the latest checkpoint to continue

dxqb•9/19/24, 5:41 AM

I've seen Flux not really converge on realism when the LR is too low, no matter how long you run it. It changes facial features but remains in its plastic AI style with a low LR

dxqb•9/19/24, 5:42 AM

are the e-06 values above with rank 128 alpha 16?

Wwardensc2 Can you describe more out which step to do it. Where I can load the latest check...

Furkan Gözükara SECourses•9/19/24, 10:54 AM

it wont have those

Furkan Gözükara SECourses•9/19/24, 10:54 AM

you need to memorize and do yourself

Ddxqb are the e-06 values above with rank 128 alpha 16?

Furkan Gözükara SECourses•9/19/24, 10:55 AM

dont change alpha we have it affects LR

dxqb•9/19/24, 11:41 AM

I have it at 1 but different LRs, that's why I'm asking. Alpha 16 multiplies the LR by 16 compared to 1

Ddxqb I have it at 1 but different LRs, that's why I'm asking. Alpha 16 multiplies the...

Furkan Gözükara SECourses•9/19/24, 12:06 PM

rank 16 + alpha 16 you trying?

Furkan Gözükara SECourses•9/19/24, 12:06 PM

you should make them equal

Furkan Gözükara SECourses•9/19/24, 12:06 PM

or your LR need to be researched again

dxqb•9/19/24, 12:06 PM

rank 128 alpha 1

dxqb•9/19/24, 12:06 PM

they don't have to be equal

dxqb•9/19/24, 12:07 PM

alpha is there so you can change the rank without changing the LR

Ddxqb they don't have to be equal

Furkan Gözükara SECourses•9/19/24, 12:07 PM

then you need new LR

dxqb•9/19/24, 12:07 PM

so whatever alpha you use, don't change it. but it doesn't matter what alpha you use.

Furkan Gözükara SECourses•9/19/24, 12:07 PM

actually we make alpha equal to rank

Furkan Gözükara SECourses•9/19/24, 12:07 PM

so that you can use same LR

dxqb•9/19/24, 12:08 PM

it's the other way around. if you use alpha 16, you can change the rank without changing the LR. OT (and I think kohya) automatically scale the LR by (alpha / rank)

Furkan Gözükara SECourses•9/19/24, 12:08 PM

i recently researched this i am 100% sure

dxqb•9/19/24, 12:08 PM

just keep alpha the same

Furkan Gözükara SECourses•9/19/24, 12:08 PM

i even talked this with kohya

Furkan Gözükara SECourses•9/19/24, 12:08 PM

alpha is just a multiplier that is used to balance LR for different lora ranks

dxqb•9/19/24, 12:08 PM

exactly. we mean the same thing