Software Engineering Courses (SECourses)•16mo ago

i think thats one of your biggests posts too

MaxiviOP•9/15/24, 1:30 AM

lot of comments seemed happy

FFurkan Gözükara SECourses the LoRA training making that impact. As you do train more epochs, that annoying...

Sebastianblood•9/15/24, 1:34 AM

Thank you

MMaxivi i think thats one of your biggests posts too

Furkan Gözükara SECourses•9/15/24, 1:36 AM

yep huge upvote

Furkan Gözükara SECourses•9/15/24, 1:36 AM

it was one of the most viral posts

FFurkan Gözükara SECourses Click to see attachment

Sebastianblood•9/15/24, 1:39 AM

Oh no, damn! This was the flux post with those many awesome images ?

SSebastianblood Oh no, damn! This was the flux post with those many awesome images ?

Furkan Gözükara SECourses•9/15/24, 1:40 AM

yep

Furkan Gözükara SECourses•9/15/24, 1:43 AM

time to do some fine tuning

FFurkan Gözükara SECourses time to do some fine tuning

MaxiviOP•9/15/24, 2:04 AM

just got word that sd3 8b will be out soon, no distilation and easier to train and finetune

MMaxivi just got word that sd3 8b will be out soon, no distilation and easier to train a...

Furkan Gözükara SECourses•9/15/24, 2:07 AM

licence?

MaxiviOP•9/15/24, 2:07 AM

i think same as the the updated sd3 medium one with more clarity

MaxiviOP•9/15/24, 2:07 AM

lykon mentioned

MaxiviOP•9/15/24, 2:07 AM

looks like it can be better than flux

Furkan Gözükara SECourses•9/15/24, 2:08 AM

nice

MaxiviOP•9/15/24, 2:08 AM

https://x.com/Lykon4072/status/1834917552209744318

Lykon (@Lykon4072) on X

@linaqruf_ @oron1208 Wait for 8b. It's basically Flux without distillation and heavy hands dpo. This should make it easy to finetune (and dpo).
We're also trying a new scaling down mechanism for mmdit, the new 2b is gonna work much better.

Twitter

•

9/14/24, 11:30 AM

Furkan Gözükara SECourses•9/15/24, 2:20 AM

full fine tuning

MMaxivi https://x.com/Lykon4072/status/1834917552209744318

Furkan Gözükara SECourses•9/15/24, 2:20 AM

nice

Furkan Gözükara SECourses•9/15/24, 2:20 AM

it is also fp16 atm

Furkan Gözükara SECourses•9/15/24, 2:20 AM

not even fp8 it will be lower

FFurkan Gözükara SECourses full fine tuning

MaxiviOP•9/15/24, 2:22 AM

you can try training on a a100 to see if results are good first

MMaxivi you can try training on a a100 to see if results are good first

Furkan Gözükara SECourses•9/15/24, 2:28 AM

not needed

Furkan Gözükara SECourses•9/15/24, 2:29 AM

it works perfect on A6000

Furkan Gözükara SECourses•9/15/24, 2:29 AM

fused back pass reduced VRAM usage by far

Furkan Gözükara SECourses•9/15/24, 2:29 AM

also there are block swapping

Furkan Gözükara SECourses•9/15/24, 2:29 AM

so it will work on 24gb too

Furkan Gözükara SECourses•9/15/24, 2:29 AM

but fp8 didnt make any diff

MaxiviOP•9/15/24, 2:29 AM

but very slow though right?

Furkan Gözükara SECourses•9/15/24, 2:29 AM

for some reason

MMaxivi but very slow though right?

Furkan Gözükara SECourses•9/15/24, 2:29 AM

same speed as lora

MaxiviOP•9/15/24, 2:29 AM

with block swapping?

MMaxivi with block swapping?

Furkan Gözükara SECourses•9/15/24, 2:29 AM

no without

Furkan Gözükara SECourses•9/15/24, 2:29 AM

with block it will be slower i expect

Furkan Gözükara SECourses•9/15/24, 2:30 AM

MaxiviOP•9/15/24, 2:30 AM

but it wont fit on 24gb without block swapping i think

MMaxivi but it wont fit on 24gb without block swapping i think

Furkan Gözükara SECourses•9/15/24, 2:33 AM

with fp8 it should

Furkan Gözükara SECourses•9/15/24, 2:33 AM

but it didnt work

MaxiviOP•9/15/24, 2:33 AM

yeah the fp8 models arent making any difference not sure why

MMaxivi yeah the fp8 models arent making any difference not sure why

Furkan Gözükara SECourses•9/15/24, 2:36 AM

just reported it as a bug to the kohya

Furkan Gözükara SECourses•9/15/24, 2:36 AM

it should reduce vram to fit into 24 gb

Furkan Gözükara SECourses•9/15/24, 2:37 AM

cpu offloading almost there

FFurkan Gözükara SECourses just reported it as a bug to the kohya

MaxiviOP•9/15/24, 4:02 AM

what kohya said

If you specify fp8_base for LoRA training, the flux will be cast to fp8 from bf16, so the VRAM usage will be the same even with full (bf16) base model.**

FFurkan Gözükara SECourses fused back pass reduced VRAM usage by far