Software Engineering Courses (SECourses)•17mo ago

I took this 2x RTX A6000 [ALT Config] 14 vCPU, 48 GB RAM, 512 GB storage and I'll use 48_GB_GPUs_v

I took this 2x RTX A6000 [ALT Config]

14 vCPU, 48 GB RAM, 512 GB storage and I'll use 48_GB_GPUs_v2.json

Ffadounav I took this 2x RTX A6000 [ALT Config] 14 vCPU, 48 GB RAM, 512 GB storage and I...

Furkan Gözükara SECourses•8/26/24, 8:22 PM

nice should work perfectg

Furkan Gözükara SECourses•8/26/24, 8:22 PM

so now you need to start multi gpu

Furkan Gözükara SECourses•8/26/24, 8:23 PM

you can do this train in fp8

Furkan Gözükara SECourses•8/26/24, 8:23 PM

increase batch size for each gpu

Furkan Gözükara SECourses•8/26/24, 8:23 PM

like 2 3 how many it goes

Furkan Gözükara SECourses•8/26/24, 8:23 PM

calculate total batch size

Furkan Gözükara SECourses•8/26/24, 8:23 PM

divide 2 and multiply LR

FFurkan Gözükara SECourses nice should work perfectg

fadounavOP•8/26/24, 8:25 PM

Excuse me but I feel like I'm reading hieroglyphics lol, I'm a newbie in this field, can you please clarify?

Ffadounav Excuse me but I feel like I'm reading hieroglyphics lol, I'm a newbie in this fi...

Furkan Gözükara SECourses•8/26/24, 8:26 PM

cant help further here it is complex

Furkan Gözükara SECourses•8/26/24, 8:26 PM

but i can give private lecture if you need

Furkan Gözükara SECourses•8/26/24, 8:26 PM

multi gpu is another level

AIGambino•8/26/24, 8:26 PM

renaissance style cat in a hat lol

FFurkan Gözükara SECourses cant help further here it is complex 😄

fadounavOP•8/26/24, 8:28 PM

oh ok, well in that case I'll lower the epochs and go back to a gpu it's already complex enough for me.

FFurkan Gözükara SECourses you can delete all and apps should redownload usually

Kallamamran•8/26/24, 8:32 PM

Phew

Furkan Gözükara SECourses•8/26/24, 9:00 PM

all the tests i made so far

message.txt3.21KB

Furkan Gözükara SECourses•8/26/24, 9:01 PM

total 56 tests

Furkan Gözükara SECourses•8/26/24, 9:01 PM

https://www.patreon.com/posts/110838414

Patreon

All The LoRA FLUX Training Experiments I Have Done So Far | SECours...

Get more from SECourses: Tutorials, Guides, Resources, Training, FLUX, MidJourney, Voice Clone, TTS, ChatGPT, GPT, LLM, Scripts by Furkan Gözü on Patreon

Dazzastrous [4090]•8/26/24, 9:03 PM

so in your opinion what settings give the very best quality in low , medium , and highest quality

DDazzastrous [4090]so in your opinion what settings give the very best quality in low , medium , ...

Furkan Gözükara SECourses•8/26/24, 9:04 PM

i ranked them already here :d

Furkan Gözükara SECourses•8/26/24, 9:04 PM

https://www.patreon.com/posts/kohya-flux-lora-110293257

Patreon

Kohya FLUX LoRA and Fine Tuning Training Full Tutorial For Local Wi...

Get more from SECourses: Tutorials, Guides, Resources, Training, FLUX, MidJourney, Voice Clone, TTS, ChatGPT, GPT, LLM, Scripts by Furkan Gözü on Patreon

Furkan Gözükara SECourses•8/26/24, 9:04 PM

Dazzastrous [4090]•8/26/24, 9:04 PM

ahh

Dazzastrous [4090]•8/26/24, 9:04 PM

better

Dazzastrous [4090]•8/26/24, 9:06 PM

so rank for 4090

Dazzastrous [4090]•8/26/24, 9:06 PM

is the magic number

DDazzastrous [4090]ahh

Furkan Gözükara SECourses•8/26/24, 9:22 PM

ye 3 best

Furkan Gözükara SECourses•8/26/24, 10:19 PM

testing different batch size speeds for flux

Furkan Gözükara SECourses•8/26/24, 10:19 PM

to add new configs

Furkan Gözükara SECourses•8/26/24, 10:34 PM

the batch size speed gain with flux is minimal. almost no gain. testing on RTX A6000

nicearoni•8/26/24, 10:40 PM

I've wondered about the avr_loss and tensorboard graphs available during training... anything to look for there that indicates how the training is progressing?

Nnicearoni I've wondered about the avr_loss and tensorboard graphs available during trainin...

Furkan Gözükara SECourses•8/26/24, 10:42 PM

useless

Furkan Gözükara SECourses•8/26/24, 10:50 PM

multi gpu config and batch size statistics added : https://www.patreon.com/posts/110293257

Patreon

Kohya FLUX LoRA and Fine Tuning Training Full Tutorial For Local Wi...

Get more from SECourses: Tutorials, Guides, Resources, Training, FLUX, MidJourney, Voice Clone, TTS, ChatGPT, GPT, LLM, Scripts by Furkan Gözü on Patreon

Furkan Gözükara SECourses•8/26/24, 10:50 PM

you can literally get 1.4375 second / it with 4x gpu with minimal loss of quality

Furkan Gözükara SECourses•8/26/24, 10:51 PM

so 3000 steps takes around 72 minutes

Dazzastrous [4090]•8/26/24, 10:51 PM